One of the core use cases for Hadoop is ETL offload and there are a number of framework options to choose from. This presentation will focus on the transformation processing aspects of ETL (or is it ELT?) and compare & contrast three of the most popular frameworks; Spark, Hive and Pig. History and background of each of these frameworks will be presented, but the focus will be on strengths and weaknesses, as well as community adoption, for each.

The format of the talk will be to rate how each framework addresses various functional requirements of transformation processing and to present sample code to visualize how well each framework works in practice. The intention is to help teams facing technology choices make the most appropriate decisions, thus suggested team skills & experiences will also be discussed for Pig, Hive and Spark.

While not formally part of the comparison, other frameworks such as MapReduce, Cascading and Flink will also be discussed.