Spark 2.0 – What’s New

Spark 2.0 – What’s New

Earlier this month DataBricks provided an overview of Apache Spark’s next major release, Spark 2.0. The following post shows some of the changes in the abstraction, API and Libraries. Spark 2.0 is expected to be released in early June 2016. What is Apache Spark...
Apache Spark — Sparking Interest

Apache Spark — Sparking Interest

Over the past few years we have all been enthralled with the buzz generated by IoT.  Now it looks like its time for Apache Spark to take its place in the lexicon of Big Data buzzwords. While performing my research for trends on Google, I was surprised to find out that...
Apache Storm vs. Apache Spark

Apache Storm vs. Apache Spark

Storm and Spark This is the last post in the series on real-time systems. In the first post we discussed Apache Storm and Apache Kafka. In the second post we discussed Apache Spark (Streaming). In both posts we examined a small Twitter Sentiment Analysis program....
Real Time Streaming with Apache Spark

Real Time Streaming with Apache Spark

Twitter/Real Time Streaming with Apache Spark (Streaming) This is the second post in a series on real-time systems tangential to the Hadoop ecosystem. Last time, we talked about Apache Kafka and Apache Storm for use in a real-time processing engine. Today, we will be...
Real Time Streaming with Apache Storm and Apache Kafka

Real Time Streaming with Apache Storm and Apache Kafka

Kenny Ballou  |  zData Inc. Big Data Engineer  | @kennyballou The following post is one in the series of real-time systems tangential to the Hadoop ecosystem.  First, exploring both Apache Storm and Apache Kafka as a part of a real-time processing engine. These two...