Big data SMACK

One characteristic of fast data applications is the ingestion of vast amounts of data streams. Note the big
difference between ingestion and storage. Businesses require real-time analysis and the need to combine
transactions on live data with real-time analytics

Fast data applications solve three challenges:
• Data streams analysis
• Data pipelines
• Real-time user interaction

In a nutshell, you can say that fast data is data on the move. It is the streaming of hundreds of millions
of endpoints to applications and servers. Imagine if you can mobile devices, sensors, financial transactions,
logs, retail systems, telecommunication routers, authorization systems, and so forth.

Data Pipeline Strategies and Principles
The following are data pipeline strategies and principles:
• Asynchronous message passing
• Consensus and gossip
• Data locality
• Failure detection
• Fault tolerance / no single point of failure
• Isolation
• Location transparency
• Parallelism
• Partition for scale
• Replay for any point of failure
• Replicate for resiliency
• Scalable infrastructure
• Share nothing
• Dynamo systems principles


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s