Difference between revisions of "Sahara/SparkPlugin"
(→Status) |
|||
| Line 11: | Line 11: | ||
We are running unit and integration tests on the plugin, that is almost finished. In January we plan to publish the code for feedback and review. | We are running unit and integration tests on the plugin, that is almost finished. In January we plan to publish the code for feedback and review. | ||
| − | Development is done by: Do Huy-Hoang and Vo Thanh Phuc | + | Development is done by: Do Huy-Hoang and Vo Thanh Phuc (Master students at Eurecom), Daniele Venzano (Research Engineer at Eurecom), under the supervision of Prof. Pietro Michiardi (at eurecom). This work is partially supported by the BigFoot project, a EC-funded research project. |
== Related Resources == | == Related Resources == | ||
* [[Savanna/PluggableProvisioning/PluginAPI]] | * [[Savanna/PluggableProvisioning/PluginAPI]] | ||
* [https://blueprints.launchpad.net/savanna/+spec/spark-plugin Blueprint] | * [https://blueprints.launchpad.net/savanna/+spec/spark-plugin Blueprint] | ||
Revision as of 10:16, 15 January 2014
Introduction
Spark is an in-memory implementation of MapReduce written in Scala.
This blueprint proposes a Savanna provisioning plugin for Spark that can launch and resize Spark clusters and run EDP jobs.
Requirements
Support for version 0.8.0 of Spark and later is planned, since it has relaxed dependencies on Hadoop and HDFS library versions. Spark in standalone mode is targeted, there will be no no support for Mesos or YARN modes for now.
Status
We are running unit and integration tests on the plugin, that is almost finished. In January we plan to publish the code for feedback and review.
Development is done by: Do Huy-Hoang and Vo Thanh Phuc (Master students at Eurecom), Daniele Venzano (Research Engineer at Eurecom), under the supervision of Prof. Pietro Michiardi (at eurecom). This work is partially supported by the BigFoot project, a EC-funded research project.
