Difference between revisions of "Sahara/SparkPlugin"
< Sahara
| Line 9: | Line 9: | ||
== Status == | == Status == | ||
| − | We | + | We are running unit and integration tests on the plugin, that is almost finished. In January we plan to publish the code for feedback and review. |
Development is done by: Do Huy-Hoang and Vo Thanh Phuc | Development is done by: Do Huy-Hoang and Vo Thanh Phuc | ||
Revision as of 07:49, 20 December 2013
Introduction
Spark is an in-memory implementation of MapReduce written in Scala.
This blueprint proposes a Savanna provisioning plugin for Spark that can launch and resize Spark clusters and run EDP jobs.
Requirements
Support for version 0.8.0 of Spark and later is planned, since it has relaxed dependencies on Hadoop and HDFS library versions. Spark in standalone mode is targeted, there will be no no support for Mesos or YARN modes for now.
Status
We are running unit and integration tests on the plugin, that is almost finished. In January we plan to publish the code for feedback and review.
Development is done by: Do Huy-Hoang and Vo Thanh Phuc
