Jump to: navigation, search

Difference between revisions of "Meghdwar"

m (Creating and using Cloudlet (CMU Readme file ) added)
(Meeting Updates and Plans)
Line 8: Line 8:
 
irc Channel : #openstack-meghdwar
 
irc Channel : #openstack-meghdwar
  
July  Wedneday 12th & Wednesday 26th
+
July  Wedneday 13th & Wednesday 27th
  
 +
Follow up
 +
1 The last week agenda and actions taken or pending
 +
2 Megdwar Gateway API discussions
 +
3. What other modules are needed in OpenStack 'meghdwar'
 +
    a. Cloudlet (existing)
 +
    b  Cloudlet Clinet (existing)
 +
    c. Cloudlet Gateway Management  (Cluser Management)
 +
    d. python  Cloudlet  Cluster Management
 +
    e. Cloudlet Agent
 +
    f. Cloudlet horizon plugin for supporting d & e as GUI instead as CLI
 +
4. How do we go about priority
 +
5. Any other mssing items.
 +
 +
 +
Meghdwar Meeting dated 7/2/2016
 +
 +
Meghdwar Meeting dated 7/13/2016
 +
Summary of discussions:
 +
 +
Review of  API for Edge Cloud services
 +
what is edge mamangement?
 +
Means we need nodes at the edge which will host servcies
 +
so we start with cloudlets which are the ones used by most developers
 +
Now coming to cloudlet , single cloudlet we do have base code
 +
Satya has tried on rackspace server 'narmada" do you have feedback on that
 +
I did not reviewed the actual code.. First I tried to install cloudlet library and cloudlet openstack extension
 +
Noticed that, it will support only Ubuntu 12.04/14.04
 +
That should not be a problem as Cloudlet is a VM and can be disk image creation with instantiation by Glance should work on any Linux host for now.
 +
Does it have APIs like CRUD Cloudlet? yes
 +
Refer
 +
https://github.com/OpenEdgeComputing/elijah-provisioning
 +
https://github.com/OpenEdgeComputing/elijah-openstack
 +
https://github.com/OpenEdgeComputing/ejijah-provisioning
 +
Tried to use Cloudlet on Launchpad to see if we can import codes from elijah or oec, both have issues for Bazar and henace we will need to rethink and start with API definitions in docs tree. SO first step will be to use devstack temlate in Infra module before we add api's for meghdwar which can use any cloudlet like elijah,oec or any others like mit etc. The management part is what we will build ground up.
 +
https://launchpad.net/cloudlet (We will devlop cloudlet here)
 +
https://launchpad.net/meghdwar ( we will devlop edge cloud gateway services here)
 +
The Megdhdwar link is updated on launchapd recently to reflect this now.
 +
 +
APIs:
 +
a. Compute node and connecting 2 VMs in a POD (kubernetes like definition)
 +
b. VM in two different PODS  and connecting them through Cluster (or should it be a Binder) - May be we take cue or use Senlin with some changes for VM distribution
 +
c.  What are the hooks to start, stop, pause, update and abort a Cloudlet application  process within VM.(do we need cloudlet agent in VM?)
 +
d. What is difference between Package Management vs. Metadata management          e. Do we cache, attach/detach stores to VM for state context or for now we use Clinet manages application state context?
 +
We did try multiple Openstack to test Cloudlet in Liberty and had succeeded with some issues documented about vNIC / IP issues and are looking at prroblems in defining two node or two cloudlet cluster
 +
We tested cloudlet on Rackspace servers but failed to get fab module to support Ubuntu 16.04 for Mitaka. A bus is filed in OpenEdge Computing wiki.
 +
We need to continue debate on using POD like in Docker or use Cluster as in Senlin but we need then across differnt Gateway sites yet serve one application across edges or many application along many edges. Wil current Cloudlet we can possibly attampt two node cluster with one application served at two edges.
 +
 +
 +
Senlin demo:
 +
 +
Senlin supports clustering of nodes, and  question will be is it applicable to cloudlet, if so we can call Senlin to cluster Cloudlets so that's the idea
 +
Eeven if we test two or more VMs (ubuntu) ciro images we wil know if it works.
 +
Satya has verified that by adding an Yaml profile he can add to cisrosVM as nodes and create a cluster ammping these nodes to Cluster.
 +
Do we plan to use Selin to scale in/out the infrastructure and thus integrate new cloudlets ?
 +
Yes that is the catch if cloudlet will scale across horizontally using Senlin in WAN environemnt, plus handoff is our intrest which is bit different than just scaling, probaly we may call it ditributing the application access from different edge along with contxt specific to that session.
 +
 +
In liberty we ran into issue of mutiple openstack for Cloudlet and did by exchanging keystone IDs. So here we may need some Global and Local catalog APIs to make this happpen.
 +
That will be part of Clodlet Provisioning as how we serve service from Edge while continue to subscribe at the Core or Central cloud, if you look at OEC presentaion we have some Data Models we are looking at
 +
 +
Is hierarchy of Central and Edge for application partitioning correct approach, what are other options?
 +
Here the issue is UE or end device registers with central cloud where as the delivery of service occurs at edge so should we keep entire app at edge or how do we partition an app?
 +
So this is the real provisiong issue and as long as we have a Discovery process we may be able to solve through Catalos like Edge and Glocbal Catalogues
 +
 +
clarkb> I recently ran tests for all projects using the python-jobs zuul template nd notoced your Gerrit repo is still empty. Anything we can do to help on the Gerrit/CI side? No we are still debaing api and code to borrow from upstream vs. build groundup, plus we want Apche licensed code so we are sorting that out.
 +
 +
Meeting Attended by:
 +
Clark B (OPnestack Infra team)
 +
ad_rien: adrien.lebre@inria.fr
 +
Ronan Cherrueau (Inria fr)
 +
pramchan: pramchan@yahoo.com
 +
narinder: narinder.gupta@canonical.com
 +
satyak: satyavittal@gmail.com
 +
 +
Agenda: Meghdwar dated  7/13/2016
 +
 +
Review of  API for Edge Cloud services
 +
1. Cloudlet Objects for a simple two node cluster (assume one Compute Node can have one or more VMs)
 +
              a. Compute node and connecting 2 VMs in a POD (kubernetes definition)
 +
              b. VM in two different PODS  and connecting them through Cluster (or should it be a Binder)
 +
              c.  What are the hooks to start, stop, pause, update and abort a Cloudlet application  process within VM.
 +
              d. What is difference between Package Management vs. Metadata management
 +
              e. Do we cache, attach/detach stores to VM for state contexts or should we stick to soft states only
 +
              f. Limitations of current Cloudlets like CPU emulation dependencies etc.
 +
2. Report on Senlin and Scaling demo on Backspace by Satya/Prakash
 +
3. What about OEC specs - should we focus on Element Management like Cluster first?
 +
4. Is hierarchy of Central and Edge for application partitioning correct approach, what are other options?
 +
5. Review of any missing items and plan for next meeting and action plans
  
 
Specs TBD:
 
Specs TBD:

Revision as of 20:00, 26 July 2016

The need of new architecture and standard APIs to manage edge cloud applications to achieve low latency of few miliseconds orders and yet be able to support multi-vendor innovation for emerging tactile Internet, industrial automation, IOT all need a new edge cloud gateway technology at the edge. Megh (Cloud) and Dwar (Gate) and thus name "meghdwar" refers to a new way of organizing applications at edge offloading workload device to edge or cloud to edge or vice versa. Refer : http://www.cs.cmu.edu/~satya/docdir/ha-mobisys-vmsynthesis-2013.pdf.

"Details Under Construction" refer https://launchpad.net/meghdwar for gerrit link refer: https://review.openstack.org/#/c/319466/

Meetings: 2nd and 4th Wednesday's of month from 7-8am PDT (Wed 14:00-15:00 UTC) irc Channel : #openstack-meghdwar

July Wedneday 13th & Wednesday 27th

Follow up 1 The last week agenda and actions taken or pending 2 Megdwar Gateway API discussions 3. What other modules are needed in OpenStack 'meghdwar'

   a. Cloudlet (existing)
   b  Cloudlet Clinet (existing)
   c. Cloudlet Gateway Management  (Cluser Management)
   d. python  Cloudlet  Cluster Management 
   e. Cloudlet Agent 
   f. Cloudlet horizon plugin for supporting d & e as GUI instead as CLI

4. How do we go about priority 5. Any other mssing items.


Meghdwar Meeting dated 7/2/2016

Meghdwar Meeting dated 7/13/2016 Summary of discussions:

Review of API for Edge Cloud services what is edge mamangement? Means we need nodes at the edge which will host servcies so we start with cloudlets which are the ones used by most developers Now coming to cloudlet , single cloudlet we do have base code Satya has tried on rackspace server 'narmada" do you have feedback on that I did not reviewed the actual code.. First I tried to install cloudlet library and cloudlet openstack extension Noticed that, it will support only Ubuntu 12.04/14.04 That should not be a problem as Cloudlet is a VM and can be disk image creation with instantiation by Glance should work on any Linux host for now. Does it have APIs like CRUD Cloudlet? yes Refer

https://github.com/OpenEdgeComputing/elijah-provisioning
https://github.com/OpenEdgeComputing/elijah-openstack
https://github.com/OpenEdgeComputing/ejijah-provisioning

Tried to use Cloudlet on Launchpad to see if we can import codes from elijah or oec, both have issues for Bazar and henace we will need to rethink and start with API definitions in docs tree. SO first step will be to use devstack temlate in Infra module before we add api's for meghdwar which can use any cloudlet like elijah,oec or any others like mit etc. The management part is what we will build ground up. https://launchpad.net/cloudlet (We will devlop cloudlet here) https://launchpad.net/meghdwar ( we will devlop edge cloud gateway services here) The Megdhdwar link is updated on launchapd recently to reflect this now.

APIs: a. Compute node and connecting 2 VMs in a POD (kubernetes like definition) b. VM in two different PODS and connecting them through Cluster (or should it be a Binder) - May be we take cue or use Senlin with some changes for VM distribution c. What are the hooks to start, stop, pause, update and abort a Cloudlet application process within VM.(do we need cloudlet agent in VM?) d. What is difference between Package Management vs. Metadata management e. Do we cache, attach/detach stores to VM for state context or for now we use Clinet manages application state context? We did try multiple Openstack to test Cloudlet in Liberty and had succeeded with some issues documented about vNIC / IP issues and are looking at prroblems in defining two node or two cloudlet cluster We tested cloudlet on Rackspace servers but failed to get fab module to support Ubuntu 16.04 for Mitaka. A bus is filed in OpenEdge Computing wiki. We need to continue debate on using POD like in Docker or use Cluster as in Senlin but we need then across differnt Gateway sites yet serve one application across edges or many application along many edges. Wil current Cloudlet we can possibly attampt two node cluster with one application served at two edges.


Senlin demo:

Senlin supports clustering of nodes, and question will be is it applicable to cloudlet, if so we can call Senlin to cluster Cloudlets so that's the idea Eeven if we test two or more VMs (ubuntu) ciro images we wil know if it works. Satya has verified that by adding an Yaml profile he can add to cisrosVM as nodes and create a cluster ammping these nodes to Cluster. Do we plan to use Selin to scale in/out the infrastructure and thus integrate new cloudlets ? Yes that is the catch if cloudlet will scale across horizontally using Senlin in WAN environemnt, plus handoff is our intrest which is bit different than just scaling, probaly we may call it ditributing the application access from different edge along with contxt specific to that session.

In liberty we ran into issue of mutiple openstack for Cloudlet and did by exchanging keystone IDs. So here we may need some Global and Local catalog APIs to make this happpen. That will be part of Clodlet Provisioning as how we serve service from Edge while continue to subscribe at the Core or Central cloud, if you look at OEC presentaion we have some Data Models we are looking at

Is hierarchy of Central and Edge for application partitioning correct approach, what are other options? Here the issue is UE or end device registers with central cloud where as the delivery of service occurs at edge so should we keep entire app at edge or how do we partition an app? So this is the real provisiong issue and as long as we have a Discovery process we may be able to solve through Catalos like Edge and Glocbal Catalogues

clarkb> I recently ran tests for all projects using the python-jobs zuul template nd notoced your Gerrit repo is still empty. Anything we can do to help on the Gerrit/CI side? No we are still debaing api and code to borrow from upstream vs. build groundup, plus we want Apche licensed code so we are sorting that out.

Meeting Attended by: Clark B (OPnestack Infra team) ad_rien: adrien.lebre@inria.fr Ronan Cherrueau (Inria fr) pramchan: pramchan@yahoo.com narinder: narinder.gupta@canonical.com satyak: satyavittal@gmail.com

Agenda: Meghdwar dated 7/13/2016

Review of API for Edge Cloud services 1. Cloudlet Objects for a simple two node cluster (assume one Compute Node can have one or more VMs)

              a. Compute node and connecting 2 VMs in a POD (kubernetes definition)
              b. VM in two different PODS  and connecting them through Cluster (or should it be a Binder)
              c.  What are the hooks to start, stop, pause, update and abort a Cloudlet application  process within VM. 
              d. What is difference between Package Management vs. Metadata management
              e. Do we cache, attach/detach stores to VM for state contexts or should we stick to soft states only
              f. Limitations of current Cloudlets like CPU emulation dependencies etc.

2. Report on Senlin and Scaling demo on Backspace by Satya/Prakash 3. What about OEC specs - should we focus on Element Management like Cluster first? 4. Is hierarchy of Central and Edge for application partitioning correct approach, what are other options? 5. Review of any missing items and plan for next meeting and action plans

Specs TBD: https://github.com/pramchan/meghdwar/wiki

Use base VM from CMU for testing Cloudlet on Ubuntu [Base VM for Ubuntu-12.04.01-i386-Server] Base VM : https://storage.cmusatyalab.org/cloudlet-vm/precise-hotplug.zip (Ubuntu account for VM : cloudlet, password: cloudlet) you can import this ``base VM`` using command line tool

$ cloudlet import-base ./precise-hotplug.zip
> $ cloudlet import-base ./precise-hotplug.zip
   > INFO     create directory for base VM
   > INFO     Decompressing Base VM to temp directory at /tmp/cloudlet-base-k7ANqB
   > INFO     Place base VM to the right directory
   > INFO     Register New Base to DB
   > INFO     ID for the new Base VM: abda52a61692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8
   > INFO     Success
 You can check the imported ``base VM`` by
   > $ cloudlet list-base
   > hash value    path
   > \-------------------------------------------------------------------------

   > abda52a    /home/krha/.cloudlet/abda52a/precise.raw
   > \------------------------------------------------------------------------------------------

2. Perform __VM synthesis__ using a sample ``VM overlay``.

 First, launch the VM synthesis server at Cloudlet.
   > $ synthesis_server
   > INFO     --------------------------------------------------
   > INFO     * Base VM Configuration
   > INFO      0 : /home/krha/.cloudlet/abda52a61692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8/precise.raw (Disk 8192 MB, Memory 1040 MB)
   > INFO     --------------------------------------------------
   > INFO     * Server configuration
   > INFO      - Open TCP Server at ('0.0.0.0', 8021)
   > INFO      - Disable Nagle(No TCP delay)  : 1
   > INFO     --------------------------------------------------
 Then, you can test VM synthesis using a sample **VM overlay URL** of htop
 command. This process will resume a backend VM that is running a htop
 command. VM overlay URL http://128.2.213.110/overlay/overlay-htop.zip
 Demo for the Fluid simulation's back-end server
 ([YouTube](https://www.youtube.com/watch?v=f9MN-kvG_ko))  is temporarily
 unavailable. Instead, you can create your own VM overlay following [How To
 Create VM Overlay](#how-to-create-vm-overlay).
 You can perform VM synthesis using sample VM overlay
   > $ synthesis_client -s localhost -u http://128.2.213.110/overlay/overlay-ht

op.zip

 For more details, check out help command **synthesis_client -h**
 If VM synthesis is successful, you will see a screen-shot like
 <img src="https://raw.github.com/cmusatyalab/elijah-provisioning/master/doc/screenshot-synthesis-success-htop.png" align=middle width=480>
 The synthesized VM shows a running htop command that was saved in the VM
 overlay.  Note that synthesis_client is a TCP client, so you can execute   it at a different machine.

If the VM synthesis is failed showing a screen-shot like below, it is due to

 the memory snapshot compatibility of the ``base VM``.
 <img src="https://raw.github.com/cmusatyalab/elijah-provisioning/master/doc/screenshot-synthesis-fail.png" align=middle width=480>
 This kernel panic is causes by CPU flag incompatibility coming from QEMU
 side. We do our best to make a VM memory snapshot work across heterogeneous
 host machine, but it can failed in an old machine. In this case, you need to  create your own ``base VM`` 

(See more at [How to create your own Base VM](#How-to-create-your-own-Base-VM))


3. VM synthesis in different ways

 In addition to using a synthesis server and a client, you can perform VM
 synthesis 1) using command line tool and 2) using an Android client.
   1) Command line tool
       > $ cloudlet synthesis /path/to/base_disk.img /path/to/overlay.zip
   2) Android client
  We have sample Android client at $HOME/android/android and you can import
   it to ``Eclipse`` as an Android project. This client program will
   automatically find nearby Cloudlet using UPnP if both client and Cloudlet
   are located in same broadcasting domain (e.g. sharing WiFi access point).
   Otherwise, you can manually specify IP address of the Cloudlet by clicking
   cancel in discovery pop-up.
   Once installing application at your phone, you have to copy your VM overlay
   to Android phone. For example with Fluid overlay example, download VM
   overlay from <a
   href=https://storage.cmusatyalab.org/cloudlet-vm/overlay-fluid-portable.zip
   target="_blank">https://storage.cmusatyalab.org/cloudlet-vm/overlay-fluid-po

rtable.zip</a>

   and **unzip it**, then copy both overlay-meta and overlay-blob_1.xz file to
   /sdcard/Cloudlet/overlay/fluid/ directory. This directory name, _fluid_,
   will be appeared to your Android application when you're asked to select
   ``Overlay VM``. Right directory name is important since the directory name
   will be     saved as appName in internal data structure and being used to
   launch associated mobile application after finishing ``VM synthesis``.
Please recall that the VM synthesis client is designed for **provisioning a custom
 back-end server program at cloudlet** and you need to launch your
 mobile application after finishing VM synthesis. This client application will
 communicate with the back-end server you just provisioned. To launch mobile
 application after VM synthesis, we use Android Activity launcher and the
 directory name is used as an index for associated mobile application. See
 more details at handleSucessSynthesis() method at CloudletConnector.java
 file.


Sample application: Fluid Simulation


  • Fluid Simulation is an interactive fluid dynamics simulation, that renders a l

iquid sloshing in a container on the screen of a phone based on accelerometer in puts. The application back-end runs on Linux and performs a [smoothed particle hydrodynamics](http://dl.acm.org/citation.cfm?id=1531346) physics simulation usi ng 2218 particles, generating up to 50 frames per second. The structure of this

application is representative of real-time (i.e., not turn-based) games.

n.

  • Video demo
 - <a href=https://www.youtube.com/watch?v=f9MN-kvG_ko target="_blank">Using Cloudlet</a>
 - <a href=https://www.youtube.com/watch?v=hWc2fpejfiw target="_blank">Using Am

azon EC2 West</a>

 - <a href=https://www.youtube.com/watch?v=aSjQnfkUoU8 target="_blank">Using Am

azon EC2 Asia</a>

  • ~~Code (temporarily unavailable)~~
 - ~~Binary back-end server: $ HOME/test/app-server/fluid-bin32/~~
 - Android client source code: $HOME/android/android_fluid/
 - Python client source code: $HOME/test/app-client/scripts/graphics_client.py
   > $ ./graphics_client.py -s localhost -p 9093 -i acc_input_50sec
 - ~~VM overlay for the back-end server: <a href=https://storage.cmusatyalab.or

g/cloudlet-vm/overlay-fluid-portable.zip target="_blank">https://storage.cmusaty alab.org/cloudlet-vm/overlay-fluid-portable.zip</a>~~ Details Usage


      1. How to create your own Base VM

You can also create your own ``base VM`` from a regular VM disk image. Here the _regular VM disk image_ means a raw format virtual disk image you normally use for KVM/QEMU or Xen.

   > $ cloudlet base /path/to/base_disk.img
   > % Use raw file format virtual disk

This command will launch GUI (VNC) connecting to your guest OS and the code will start creating ``base VM`` right after you close VNC window. So, please close the VNC window when you think it's right point to snapshot the VM as a base VM (typically you close it after booting up). Then, it will generate snapshot of the current states for both memory and disk and save that information to DB. You can check list of ``base VM`` by

   > $ cloudlet list-base
      1. How to create VM overlay

Now you can create your customized VM based on top of ``base VM``

   > $ cloudlet overlay /path/to/base\_disk.img
   > % Path to base_disk is the path for virtual disk you used earlier
   > % You can check the path by "cloudlet list-base"

This will launch VNC again with resumed ``base VM``. Now you can start making any customizations on top of this ``base VM``. For example, if you're a developer of ``face recognition`` back-end server, we will install required libraries, binaries and finally start your face recognition server. After closing the GUI windows, cloudlet will capture only the change portion between your customization and ``base VM`` to generate ``VM overlay`` that is a minimal binary for reconstructing your customized VM.

``VM overlay`` is using zip container and inside of the zip file there are 2 types of files; 1) ``overlay-meta file`` ends with .overlay-meta, 2) compressed ``overlay blob files`` ends with .xz

    • If your application need specific TCP/UDP port to communicate with a client,

then you can make a port forwarding using -redir parameter as below.**

   > $ cloudlet overlay /path/to/base_disk.img -- -redir tcp:2222::22 -redir tc

p:8080::80

 This will forward client connection at host port 2222 to VM's 22 and 8080 to 8

0, respectively.

    • Note**: If you have experience kernel panic error like

[this](https://github.com/cmusatyalab/elijah-cloudlet/issues/1), You should upgrade your Linux kernel to at least 3.13.0 version. This bug is fixed since kernel version 3.13.0. If you cannot upgrade the kernel, then please avoid using EPT. You can disable EPT as follows:

   > $ sudo modprobe -r kvm_intel
   > $ sudo modprobe kvm_intel "ept=0"


Directories


<b>HOME</b>
  ├── bin: executable binaries such as command line tool, VM synthesis server an
d client
  │
  ├── elijah: Cloudlet provisioning code using VM synthesis
  │     └─ test: unittest
  │
  ├── cloudletfs: FUSE file system for creating VM overlay and performing early
start
  │               optimization at VM Synthesis
  │
  ├── android: Android client
  │     ├─ android: main android client for VM synthesis
  │     ├─ android_fluid: fluid simulation client used in demo videos
  │     │       ├─ YouTube: <a href=https://www.youtube.com/watch?v=hWc2fpejfiw
target="_blank">Using Amazon EC2 West</a>
  │     │       └─ YouTube: <a href=https://www.youtube.com/watch?v=f9MN-kvG_ko
target="_blank">Using Cloudlet</a>
  │     └─ android_ESVMRecogn, android_ESVMTrainer: under development
  │
  ├── test: Test applications' code
  │     ├─ app-client
  │     │       ├─ scripts: client codes for each test application
  │     │       └─ batch_files: batch scripts to test application using VM synth
esis on x86 (not Android)
  │     └─ app-server: server binary for each test application
  │
  └── fabric.py: installation script using <a href=http://docs.fabfile.org/en ta
rget="_blank">Fabric</a>

Troubleshooing


1. KVM permission error at the first run

   > failed to initialize KVM: Permission denied
   > No accelerator found!
   >
   > Error, make sure previous VM is closed and check QEMU_ARGUMENT
   >
   > Failed to create overlay
 Please logout your session and re-login.

2. Fuse permission error at the first run

 Make sure the current user to have fuse permission. The qemu-kvm library
 changes fuse access permission while it's being installed, and the permission
 is recovered if you reboot the host.  This is a known bug in qemu-kvm
 installation script
 (https://bugs.launchpad.net/ubuntu/+source/udev/+bug/1152718), so you can
 either reboot the machine to have valid fuse permission of just revert the
 permission manually as bellow.
         > $ sudo chmod 644 /etc/fuse.conf
         > $ sod sed -i 's/#user_allow_other/user_allow_other/g' /etc/fuse.conf



Last meeting: Meghdwar on irc June 22nd 7am-8am PDT Minutes: Action Items Review:

Status of Tests on Mitaka/Senlin Stya tested and confirmed that thre was a memory shortage and we need update to the Rackspace server for testing. Nova was able to start from command line but starting additional VM or adding Senlin cluster failed for lack of memory.

Any codes reviews - was postponed for next meeting

Report from OPNFV meeting at Berlin for Edge Cloud Services link https://wiki.opnfv.org/download/attachments/6819410/2016-06-21%20Open%20Edge%20Computing%20-%20published%20final.pdf?version=1&modificationDate=1467035211000&api=v2

Reviewing the requirements for Meghdwar to plan for Specs. Specs was pointed to in OEC link on https://github.com/pramchan/meghdwar/wiki Access is provided to ad_rien who wanted to review specs. Anyone needing access please let me know through email to pramchan@yahoo.com Attendees: ad_rien: adrien.lebre@inria.fr pramchan: pramchan@yahoo.com narinder: narinder.gupta@canonical.com satyak: satyavittal@gmail.com

Meetings: Wednesday's from 7-8am PDT (Wed 14:00-15:00 UTC) irc Channel : #openstack-meghdwar


Meghdwar-irc June 15th 7am-8am PDT Minutes /summary Topic: Distributed WAN IaaS and Grid 5000 link: https://hal.inria.fr/search/index/q/*/level1_domain_s/info.info-dc pramchan: For Meghdwar what we are interested in is how to form a cluster of Cloudlets does it address it? Ad_rien: bascially, you can book some servers on Grid'5000 to perform experiments on performance, energy (whatever criteria you want to investigate) of a middleware

form a cluster

[07:12] <ad_rien_> it address basically all scenarios you want to address [07:12] <pramchan> Is Grid 5000 a cluster of Cloudlets? [07:12] <ad_rien_> in the sense that you get raw servers and you deploy what you want [07:12] <pramchan> cluster of Cloudlets [07:12] <ad_rien_> no [07:12] <ad_rien_> Grid5000 is a testbed [07:13] <ad_rien_> i.e 1500 servers [07:13] <ad_rien_> that are deployed throughout 10 sites [07:13] <ad_rien_> there is nothing on the servers [07:13] <pramchan> OK how do you from Grid of 5000 nodes using Nova? [07:13] <ad_rien_> you deploy OpenStack [07:13] <pramchan> 1500 servers ok [07:13] <pramchan> What changes you make to OPenStack Nova? [07:13] <ad_rien_> that is 1./ you book your nodes (if you want the whole infrastructure, you book everything) [07:14] <pramchan> OK so its more a reservation system [07:14] <ad_rien_> 2./ you deploy the environment you want, let's say Ubuntu with OpenStack [07:14] <ad_rien_> 3./ you perform you experiments [07:14] <ad_rien_> 4./ you collect the results, make analysis and write research reports, publications .... [07:14] <pramchan> OK got it [07:14] <ad_rien_> or you can improve your prototype and then test it once again [07:15] <ad_rien_> this is a testbed for distributed computing research activities [07:15] <pramchan> I have https://bitbucket.org/ which contains cloudlet code [07:16] <ad_rien_> https://scholar.google.fr/scholar?q=Adding+virtualization+capabilities+to+the+Grid%E2%80%995000+testbed&btnG=&hl=fr&as_sdt=0%2C5 [07:16] <pramchan> Have asked Satya to review it and he is not there today so we skip that [07:16] <ad_rien_> if you want to have a complete overview of what you can do with Grid'5000 [07:16] <ad_rien_> ok [07:17] <pramchan> Coming to cluster formation I reviewed the Senlin project in OpenStack and we can possibly combine Cloudlet , concespt of WAN IaaS and Senlin together [07:17] <ad_rien_> link to senlin ? [07:17] <ad_rien_> please [07:20] <pramchan> https://wiki.openstack.org/wiki/Senlin [07:20] <ad_rien_> ok I found it. can you explain in few words what is about ? [07:20] <pramchan> This project creates cluster of nodes of domain objects like nova compute [07:21] <pramchan> This can be also use Heat and any other OpenStack module [07:21] <ad_rien_> this is a deployment tool ? sorry I'm not sure I correctly understood [07:21] <pramchan> Now if I have group of nova compute cpontroller I can place them across the WAN at different edges [07:21] <ad_rien_> ok so it is a deployment tool ? [07:22] <pramchan> Now define a Profile that will manage those federated Novas [07:23] <ad_rien_> ok what is the difference with projects like triCircle ? [07:23] <pramchan> Define Policies that will create compute nodes and place them at diffetrent locations using the Nova or heat [07:23] <ad_rien_> ok I have to give it a look [07:23] <ad_rien_> I didn't know this project [07:25] <pramchan> TriCircle is similar but Senlin is just focused on Cluster of Objects to scale-in Scale-out as comapred to TriCircle is workin on installation underlay/Overlay aspects [07:25] <pramchan> OK I will send all our team a presentation slidies fro Senlin by email [07:26] <ad_rien_> yes but scaling in/out over wide area network should be done carefully [07:26] <pramchan> You can review and discuss this next time, also I am trying to see if this can be used [07:26] <ad_rien_> ok thanks [07:26] <ad_rien_> ok [07:26] <ad_rien_> can you also send us a link toward a recent presentation of your cloudlet proposal [07:26] <pramchan> We will review any other options you want to bring to table as we need to have good discussions before we decide on any option [07:27] <ad_rien_> it is still unclear on my side whether we are targeting the same objective or not and how we can mutualize our effot [07:27] <ad_rien_> s/effot/effort [07:27] <pramchan> Absolutely [07:28] <pramchan> #link https://github.com/pramchan/meghdwar [07:30] <pramchan> I am working with openedgecomputing,org fols from Vodafone and CMU to define the APis and details are on google docs proposed by team there in the link, if you need access to it wil get added your name too [07:30] <ad_rien_> please do it I would appreciate to go through any documents/articles that present the concept [07:30] <pramchan> https://docs.google.com/document/d/10-ncLyIoSvN4y7ygAnsjvw6PRttAA3BIErw0IYLscIc/edit?pref=2&pli=1#heading=h.gjdgxs [07:31] <pramchan> See if you have gmail account try or I will ask narinder's and your name by email added to it [07:31] <ad_rien_> that's ok [07:31] <ad_rien_> I can see the document thank [07:32] <pramchan> What we have there is that three version of Cloudlet is required, one simple for developers, one for Workgroup or cluster based deployment and third for ETSI MEC like architecture Edge Gateways [07:33] <pramchan> Currently we have mostly development centric which is good only for Developers and not for Massive scale deployments as we need [07:35] <pramchan> So I would say I am reviewing this at Berlin Summit f OPNFV where I am meeting with Dr Rolf Schuster from CMU/Vodafone team to sort out more details. Plus Intels Babu is also inovolved along with Guenther so lets see what results we get from there [07:36] <pramchan> So let us see how this pans out but will share the documents through email [07:36] <ad_rien_> ok [07:37] <ad_rien_> so in the Discovery initiative we are targetting what you are calling a cloudlet cluster [07:38] <pramchan> arinder is also coming to OPNFB Berlin Summit on June 20-24 and we plan to meet there with service providers to get a clear picture how to go about this project [07:38] <ad_rien_> we want to be able to operate a significant number of cloudlet clusters through a common OpenStack [07:41] <pramchan> That is wh we have a lot of common things and like to see if we can sort them out, possibly will try a Gotomeeting or webex next week [07:41] <ad_rien_> sure [07:42] <ad_rien_> We are organizing a plenary session at Rennes (France) on July 11-12 [07:42] <pramchan> Send me details [07:42] <ad_rien_> I don't know whether you can come (actually, what is your location ? ) [07:42] <pramchan> I will see if I can send some one there [07:43] <ad_rien_> ok [07:43] <pramchan> Im in Santa Clara California and Narider in Austin Texas in USa [07:43] <ad_rien_> I will, The complete agenda should be available by June the 20th. [07:44] <ad_rien_> ok … not sure whether it makes sense for you to make such a trip. Maybe a webconference can be a better action for the moment [07:44] <ad_rien_> (s/better/more fruitful) [07:44] <pramchan> OK share with us and see if we can Dr Rolf Schuster from Drusseldorf to attend it [07:44] <ad_rien_> sure [07:45] <pramchan> Narinder do you have anything to share [07:46] <pramchan> If not would like to see our action iytems from last meeting [07:47] <pramchan> #info last week we had tried testing install Mitaka and had issues on apache2 not being able to get Hiorizon up [07:48] <pramchan> #info I worked with Satya and he tried using Ubuntu PPA for horizon wsgateway package and we got vendor mix error

[07:14] <ad_rien_> or you can improve your prototype and then test it once again [07:15] <ad_rien_> this is a testbed for distributed computing research activities [07:15] <pramchan> I have https://bitbucket.org/ which contains cloudlet code [07:16] <ad_rien_> https://scholar.google.fr/scholar?q=Adding+virtualization+capabilities+to+the+Grid%E2%80%995000+testbed&btnG=&hl=fr&as_sdt=0%2C5 [07:16] <pramchan> Have asked Satya to review it and he is not there today so we skip that [07:16] <ad_rien_> if you want to have a complete overview of what you can do with Grid'5000 [07:16] <ad_rien_> ok [07:17] <pramchan> Coming to cluster formation I reviewed the senlin project in OpenStack and we can possibly combine Cloudlet , concespt of WAN IaaS and Senlin together [07:17] <ad_rien_> link to senlin ? [07:17] <ad_rien_> please [07:20] <pramchan> https://wiki.openstack.org/wiki/Senlin [07:20] <ad_rien_> ok I found it. can you explain in few words what is about ? [07:20] <pramchan> This project creates cluster of nodes of domain objects like nova compute [07:21] <pramchan> This can be also Heta and any other openstack module [07:21] <ad_rien_> this is a deployment tool ? sorry I'm not sure I correctly understood [07:21] <pramchan> Now if I have group of nova compute cpontroller I can place them across the WAN at different edges [07:21] <ad_rien_> ok so it is a deployment tool ? [07:22] <pramchan> Now define a Profile that will manage those federated Novas [07:23] <ad_rien_> ok what is the difference with projects like triCircle ? [07:23] <pramchan> Define Policies that will create comute nodes and place them at diffetent locations using the Nova or heat [07:23] <ad_rien_> ok I have to give it a look [07:23] <ad_rien_> I didn't know this project [07:25] <pramchan> Tric Circle is similar but Senlin is just focused on Cluster of Objects to scale-in Scale-out as comapred tp Tric is worlkin on installation aspects [07:25] <pramchan> OK I will sedn all our team a presentaion sldies by email [07:26] <ad_rien_> yes but scaling in/out over wide area network should be done carefully [07:26] <pramchan> You can review and discuss this next time, also I am trying to see if this can be used [07:26] <ad_rien_> ok thanks [07:26] <ad_rien_> ok [07:26] <ad_rien_> can you also send us a link toward a recent presentation of your cloudlet proposal [07:26] <pramchan> We will review any other options you want to bring to table as we need to have good discussions before we decide on any option [07:27] <ad_rien_> it is still unclear on my side whether we are targeting the same objective or not and how we can mutualize our effot [07:27] <ad_rien_> s/effot/effort [07:27] <pramchan> Absolutely [07:28] <pramchan> #link https://github.com/pramchan/meghdwar [07:30] <pramchan> I am working with openedgecomputing,org fols from Vodafone and CMU to define the APis and details are on google docs proposed by team there in the link, if you need access to it wil get added your name too [07:30] <ad_rien_> please do it I would appreciate to go through any documents/articles that present the concept [07:30] <pramchan> https://docs.google.com/document/d/10-ncLyIoSvN4y7ygAnsjvw6PRttAA3BIErw0IYLscIc/edit?pref=2&pli=1#heading=h.gjdgxs [07:31] <pramchan> See if you have gmail account try or I will ask narinder's and your name by email added to it [07:31] <ad_rien_> that's ok [07:31] <ad_rien_> I can see the document thank [07:32] <pramchan> What we have there is that three version of Cloudlet is required, one simple for developers, one for Workgroup or cluster based deployment and third for ETSI MEC like architecture Edge Gateways [07:33] <pramchan> Currently we have mostly devlopment centric which is good only for Developers and not for Massive scale deployments as we need [07:35] <pramchan> So I would say I am reviewing this at Berlin Summit f OPNFV where I am meeting with Dr Rolf Schuster from CMU/Vodafone team to sort out more details. Plus Intels Babu is also inovolved along with Guenther so lets see what results we get from there [07:36] <pramchan> So let us see how this pans out but will share the documents through email [07:36] <ad_rien_> ok [07:37] <ad_rien_> so in the Discovery initiative we are targetting what you are calling a cloudlet cluster [07:38] <pramchan> arinder is also coming to OPNFB Berlin Summit on June 20-24 and we plan to meet there with service providers to get a clear picture how to go about this project [07:38] <ad_rien_> we want to be able to operate a significant number of cloudlet clusters through a common OpenStack [07:41] <pramchan> That is wh we have a lot of common things and like to see if we can sort them out, possibly will try a Gotomeeting or webex next week [07:41] <ad_rien_> sure [07:42] <ad_rien_> We are organizing a plenary session at Rennes (France) on July 11-12 [07:42] <pramchan> Send me details [07:42] <ad_rien_> I don't know whether you can come (actually, what is your location ? ) [07:42] <pramchan> I will see if I can send some one there [07:43] <ad_rien_> ok [07:43] <pramchan> Im in Santa Clara California and Narider in Austin Texas in USa [07:43] <ad_rien_> I will, The complete agenda should be available by June the 20th. [07:44] <ad_rien_> ok … not sure whether it makes sense for you to make such a trip. Maybe a webconference can be a better action for the moment [07:44] <ad_rien_> (s/better/more fruitful) [07:44] <pramchan> OK share with us and see if we can Dr Rolf Schuster from Drusseldorf to attend it [07:44] <ad_rien_> sure [07:45] <pramchan> Narinder do you have anything to share [07:46] <pramchan> If not would like to see our action items from last meeting [07:47] <pramchan> #info last week we had tried testing install Mitaka and had issues on apache2 not being able to get Hiorizon up [07:48] <pramchan> #info I worked with Satya and he tried using Ubuntu PPA for horizon wsgateway package and we got vendor mix error

Attendees: Prakash Ramchandran Adrien Lebre Narinder Gupta

Meghdwar-irc June 8th 7am-8am PDT #opnestack-meghdwar summary

Topic : What is meghdwar

link https://launchpad.net/meghdwar This is a follow up project to create a project for Edge Cloud Services im Openstack Earlier effort for a Micro Service API as Cloud let failed link https://launchpad.net/cloudlet Cloudlet as defined by CMU/OEC is a VM at the edge supporting AR/VR applications In a three tier the client is an UE/Mobile equipment/Smartphone with client and connecting to edge Cloudlet The edge cloudlet serves the AR/VR application from within the VM The Central Cloud is where you may register and downlaod applications or apps link http://beyondtheclouds.github.io/ Review the Video of Discovery initiative, mainly supported through the Inria Project Labs program and the I/O labs, a joint lab between Inria and Orange Labs. The aim of project is to try build sustainable (power reduction) network Pops distrubuted as IaaS.The 8 minute video shows how openstack is used to deliver the Discovery service with Nova and Redis in-meory data structure stores. link https://hal.inria.fr/hal-01320235 Revising OpenStack Internals to Operate Massively Distributed Clouds is on this link.

action on satyak - Support to resolve Devstack ticket for Mitaka action on prakash - Ask Kiryong Ha CMU to provide bitucket access to Cloudlet code for Proevisioning and Openstack projects in CMU ija or OEC action on prakash/ad_rien - Review beyondclouds.github.io and work with ad_rien to see if we can use it along CMU/OEC Cloudlet or independednt of it

Next meeting same time Wednesday 7AM-8 AM PDT /

Attendees: ad_rien: adrien.lebre@inria.fr pramchan: pramchan@yahoo.com narinder: narinder.gupta@canonical.com satyak: satyavittal@gmail.com