Bbcjg5iqw9tk8lscjndo
SkillsCast

Fast Data

24th October 2016 in London at CodeNode

This SkillsCast was filmed at Why SMACK for Fast Data

SMACK stands for Spark, Mesos, Akka, Cassandra and Kafka - a combination that's being adopted for 'fast data'. Join this talk by Big Data expert Dean Wampler and learn how Spark and the SMACK stack help solve big data and IoT-type challenges where speed of response is essential.

Why SMACK for Fast Data

The SMACK stack (Spark, Mesos, Akka, Cassandra, Kafka) is well positioned as the ideal platform for building “Fast Data” applications. The term Fast Data emphasizes how Big Data architectures and applications are evolving to be stream oriented, so that information is extracted as quickly as possible from incoming data, while still supporting traditional data scenarios, such as data warehousing, batch processing, and interactive exploration.

We’ll explore in depth how the SMACK components (and variations) support the requirements of Fast Data systems:

  • Spark (and similar streaming engines): Used to implement ETL, queries, aggregations, applications of machine learning, etc.

  • Mesos: The flexible cluster infrastructure that addresses the limitations of Hadoop YARN. It can host and manage the cluster resources for all your applications.

  • Akka: Microservice development with high scalability, durability, and low-latency processing. Perhaps it should really be Lightbend’s entire Reactive Platform, in which case we have SMRCK (“smirk”)?

  • Cassandra: Scalable, resilient, distributed database for persistent, durable storage. Most environments will also use a distributed file system like HDFS or S3.

  • Kafka: The backplane and integration tool for all stream flows. Provides highly scalable and durable short-term storage, organized into topics using message queue semantics.

YOU MAY ALSO LIKE:

Thanks to our sponsors

Fast Data

Dean Wampler

Dean Wampler, Ph.D., is the Architect for Big Data Products and Services in the Office of the CTO at Lightbend, where he focuses on the evolving “Fast Data” ecosystem for streaming applications based on the SMACK stack, Spark, Mesos, Akka (and the rest of the Lightbend Reactive Platform), Cassandra, Kafka, and other tools.

SkillsCast

SMACK stands for Spark, Mesos, Akka, Cassandra and Kafka - a combination that's being adopted for 'fast data'. Join this talk by Big Data expert Dean Wampler and learn how Spark and the SMACK stack help solve big data and IoT-type challenges where speed of response is essential.

Why SMACK for Fast Data

The SMACK stack (Spark, Mesos, Akka, Cassandra, Kafka) is well positioned as the ideal platform for building “Fast Data” applications. The term Fast Data emphasizes how Big Data architectures and applications are evolving to be stream oriented, so that information is extracted as quickly as possible from incoming data, while still supporting traditional data scenarios, such as data warehousing, batch processing, and interactive exploration.

We’ll explore in depth how the SMACK components (and variations) support the requirements of Fast Data systems:

  • Spark (and similar streaming engines): Used to implement ETL, queries, aggregations, applications of machine learning, etc.

  • Mesos: The flexible cluster infrastructure that addresses the limitations of Hadoop YARN. It can host and manage the cluster resources for all your applications.

  • Akka: Microservice development with high scalability, durability, and low-latency processing. Perhaps it should really be Lightbend’s entire Reactive Platform, in which case we have SMRCK (“smirk”)?

  • Cassandra: Scalable, resilient, distributed database for persistent, durable storage. Most environments will also use a distributed file system like HDFS or S3.

  • Kafka: The backplane and integration tool for all stream flows. Provides highly scalable and durable short-term storage, organized into topics using message queue semantics.

YOU MAY ALSO LIKE:

Thanks to our sponsors

About the Speaker

Fast Data

Dean Wampler

Dean Wampler, Ph.D., is the Architect for Big Data Products and Services in the Office of the CTO at Lightbend, where he focuses on the evolving “Fast Data” ecosystem for streaming applications based on the SMACK stack, Spark, Mesos, Akka (and the rest of the Lightbend Reactive Platform), Cassandra, Kafka, and other tools.