Meteos/ExampleDecisionTree
< Meteos
Contents
Make a Decision to buy a stock using Meteos
In this example, you creates a prediction model which predict stock market by using Decision Tree Model.
1. Create a experiment template
Create template of experiment. Experiment is a workspace of Machine Learning.
You have to confirm a glance image id of meteos image, and a neutron network id before creating a template.
You can use a format located in python-meteosclient/sample/json/template.json
$ glance image-list | grep meteos | 45de4bbd-8419-40ff-8ed7-fc065c05e34f | meteos |
$ neutron net-list | grep public | 84c13e76-ced9-4142-a885-280784f1f7a3 | public | a14de1c5-b8d4-434b-a056-9b0049b93402 |
$ vim sample/json/template.json
$ cat sample/json/template.json
{
"display_name": "example-template",
"display_description": "This is a sample template of experiment",
"image_id" : "45de4bbd-8419-40ff-8ed7-fc065c05e34f",
"master_nodes_num": 1,
"master_flavor_id": "4",
"worker_nodes_num": 2,
"worker_flavor_id": "2",
"spark_version": "1.6.0",
"floating_ip_pool": "84c13e76-ced9-4142-a885-280784f1f7a3"
}
$ meteos template-create --json sample/json/template.json +---------------+-----------------------------------------+ | Property | Value | +---------------+-----------------------------------------+ | cluster_id | None | | created_at | 2016-12-04T07:16:29.000000 | | description | This is a sample template of experiment | | id | 8b7b9b89-f119-4b9b-b9b0-31598f819f1a | | master_flavor | 4 | | master_nodes | 1 | | name | example-template | | project_id | 67401cca74c2409b939e944bc6c8fcbe | | spark_version | 1.6.0 | | status | available | | user_id | 181b1caa9d5b470393ca66b9e511d5b0 | | worker_flavor | 2 | | worker_nodes | 2 | +---------------+-----------------------------------------+
2. Create a experiment from template
Create a experiment by using template created in the above step. You have to confirm a neutron private network id and create keypair before creating a template.
You can use a format located in python-meteosclient/sample/json/experiment.json
$ nova keypair-add key1 > ~/key1.pem && chmod 600 ~/key1.pem
$ neutron net-list | grep private | 8abc626e-2b06-4c67-9b2c-0231f0cef5b8 | private | cb58940f-859b-48c6-b92a-3861470f1fc1 20.0.0.0/26 |
$ vim sample/json/experiment.json
$ cat sample/json/experiment.json
{
"display_name": "example-experiment",
"display_description": "This is a sample experiment",
"key_name": "key1",
"neutron_management_network": "8abc626e-2b06-4c67-9b2c-0231f0cef5b8",
"template_id": "8b7b9b89-f119-4b9b-b9b0-31598f819f1a"
}
$ meteos experiment-create --json sample/json/experiment.json +--------------------+--------------------------------------+ | Property | Value | +--------------------+--------------------------------------+ | created_at | 2016-12-04T07:20:11.000000 | | description | This is a sample experiment | | id | 91504a65-01cf-428f-81aa-596be7ca8619 | | key_name | key1 | | management_network | 8abc626e-2b06-4c67-9b2c-0231f0cef5b8 | | name | example-experiment | | project_id | 67401cca74c2409b939e944bc6c8fcbe | | status | creating | | user_id | 181b1caa9d5b470393ca66b9e511d5b0 | +--------------------+--------------------------------------+
Meteos creates a experiment using OpenStack Sahara spark plugin.
You can see a sahara cluster and nova VMs created by Meteos as below.
$ openstack dataprocessing cluster list (or sahara cluster-list) +------------------+--------------------------------------+-------------+----------------+----------+ | Name | Id | Plugin name | Plugin version | Status | +------------------+--------------------------------------+-------------+----------------+----------+ | cluster-91504a65 | 13418fd9-5d2a-4ee6-b384-cb250b7e7714 | spark | 1.6.0 | Spawning | +------------------+--------------------------------------+-------------+----------------+----------+
$ openstack server list (or nova list) +--------------------------------------+----------------------------+--------+----------+------------+ | ID | Name | Status | Networks | Image Name | +--------------------------------------+----------------------------+--------+----------+------------+ | 58818eb5-ade7-407c-8c76-9fd9809632b4 | cluster-91504a65-workers-1 | BUILD | | meteos | | a151dbd9-de51-43ca-afb8-1fdeecce2891 | cluster-91504a65-workers-0 | BUILD | | meteos | | d02d85c5-0960-4b7e-880c-26b73c5dd8ad | cluster-91504a65-master-0 | BUILD | | meteos | +--------------------------------------+----------------------------+--------+----------+------------+
3. Upload a raw data
Upload a raw data (in this example past stock market data) to OpenStack Swift.
You can use a sample data located in python-meteosclient/sample/data/decision_tree_data.txt
Raw data shows "Flag which indicate buy or not buy", "Related Stock Code : Price Change",... from left.
$ cd sample/data/
$ head decision_tree_data.txt 1 1:-5 2:3 3:1 4:-5 5:-4 6:3 0 1:2 2:-5 3:0 4:0 5:1 6:-4 1 1:-3 2:1 3:4 4:-5 5:-3 6:4 0 1:4 2:-4 3:4 4:4 5:0 6:-1 1 1:-4 2:4 3:0 4:-5 5:-4 6:0 0 1:1 2:-4 3:3 4:3 5:0 6:-4 1 1:-3 2:4 3:0 4:-4 5:-3 6:2 0 1:2 2:-5 3:2 4:2 5:4 6:-3 1 1:-1 2:0 3:1 4:-2 5:-5 6:4 0 1:0 2:-4 3:1 4:1 5:4 6:-4
/sample/data$ swift upload meteos decision_tree_data.txt decision_tree_data.txt
4. Create a prediction model
In this example, you create a Decision Tree Model from dataset in swift directly.
$ vim sample/json/model_decision_tree.json
$ cat sample/json/model_decision_tree.json
{
"display_name": "sample-tree-model",
"display_description": "Decision Tree Model",
"source_dataset_url": "swift://meteos/decision_tree_data.txt",
"model_type": "DecisionTreeRegression",
"model_params": "{'numIterations': 100}",
"dataset_format": "libsvm",
"experiment_id": "fad1b912-8f7d-4600-b048-7db27aae1af3",
"swift_tenant": "demo",
"swift_username": "demo",
"swift_password": "nova"
}
$ meteos model-create --json sample/json/model_decision_tree.json +-------------+--------------------------------------+ | Property | Value | +-------------+--------------------------------------+ | created_at | 2016-12-08T03:18:47.000000 | | description | Decision Tree Model | | id | 5c16c3f7-0093-4f82-a4f9-2ae421fc901f | | name | sample-tree-model | | params | eydudW1JdGVyYXRpb25zJzogMTAwfQ== | | project_id | 0e3e0f01952948848d8ae438279122fe | | status | creating | | stderr | None | | stdout | None | | type | DecisionTreeRegression | | user_id | 2c3120e8228c4d0e9768f09346fe842d | +-------------+--------------------------------------+
$ meteos model-list +--------------------------------------+-------------------+------------------------+-----------+------------------------+-----------------------------------------+----------------------------+ | id | name | description | status | type | source_dataset_url | created_at | +--------------------------------------+-------------------+------------------------+-----------+------------------------+-----------------------------------------+----------------------------+ | 5c16c3f7-0093-4f82-a4f9-2ae421fc901f | sample-tree-model | Decision Tree Model | available | DecisionTreeRegression | swift://meteos/decision_tree_data.txt | 2016-12-08T03:18:47.000000 | +--------------------------------------+-------------------+------------------------+-----------+------------------------+-----------------------------------------+----------------------------+
