Difference between revisions of "Meteos/DatasetsandModels"
m (HiroyukiEguchi moved page Meteos/Models to Meteos/DatasetsandModels) |
(→Meteos Dataset) |
||
| Line 5: | Line 5: | ||
Meteos currently supports following data format. | Meteos currently supports following data format. | ||
| − | + | ==== CSV data format ==== | |
<pre> | <pre> | ||
<label>,<value1>,<value2>, ... <valueN> | <label>,<value1>,<value2>, ... <valueN> | ||
</pre> | </pre> | ||
| − | + | ==== LibSVM data format ==== | |
<pre> | <pre> | ||
<label> <index1>:<value1> <index2>:<value2> ... <indexN>:<valueN> | <label> <index1>:<value1> <index2>:<value2> ... <indexN>:<valueN> | ||
| Line 19: | Line 19: | ||
A Source_dataset_url has two url types as follows: | A Source_dataset_url has two url types as follows: | ||
| − | + | ==== Swift URL ==== | |
| − | Swift URL is used when creating a model from swift. | + | Swift URL is used when creating a model from dataset in swift. |
If it is no neccesary to parse a dataset, user can create a model from dataset in swift directly by specify source_data_url as below. | If it is no neccesary to parse a dataset, user can create a model from dataset in swift directly by specify source_data_url as below. | ||
| Line 29: | Line 29: | ||
</pre> | </pre> | ||
| − | + | ==== Internal HDFS URL ==== | |
| − | Internal HDFS URL is | + | Internal HDFS URL is used when creating a model from internal hdfs of meteos experiment. |
| − | + | ||
| + | "Internal" is meaning that dataset has been already downloaded or parsed by Meteos. | ||
When creating a model from dataset in hdfs, user have to specify a url as below. | When creating a model from dataset in hdfs, user have to specify a url as below. | ||
| Line 39: | Line 40: | ||
internal://<dataset_id> | internal://<dataset_id> | ||
</pre> | </pre> | ||
| − | |||
== Meteos Prediction Model == | == Meteos Prediction Model == | ||
Revision as of 05:43, 5 December 2016
Contents
Meteos Dataset
Dataset is a data to create a prediction model.
Meteos currently supports following data format.
CSV data format
<label>,<value1>,<value2>, ... <valueN>
LibSVM data format
<label> <index1>:<value1> <index2>:<value2> ... <indexN>:<valueN>
When creating a prediction model, user specify a "source_dataset_url" parameter which show the place where dataset is located.
A Source_dataset_url has two url types as follows:
Swift URL
Swift URL is used when creating a model from dataset in swift.
If it is no neccesary to parse a dataset, user can create a model from dataset in swift directly by specify source_data_url as below.
swift://<container_name>/<object_name>
Internal HDFS URL
Internal HDFS URL is used when creating a model from internal hdfs of meteos experiment.
"Internal" is meaning that dataset has been already downloaded or parsed by Meteos.
When creating a model from dataset in hdfs, user have to specify a url as below.
internal://<dataset_id>
Meteos Prediction Model
Currently Meteos supports these following prediction models of Apache Spark.
Apache Spark has two machine learning libraries (MLlib and Ml).
MLlib and ML has multiple prediction models by data mining and machine learning algorithms.
MLlib
ML
Not Supported.
