ETC Cloud API Prototype project

Project Lead Sean Roberts (sarob)

purpose

The Entertainment Technology Center @ the University of Southern California (ETC) formally launched “Production in the Cloud,” a new project that brings together a core group of key media and cloud-resource leaders to develop guidelines and accelerate innovation and adoption of next-gen cloud-based content creation, production, and distribution tools and processes. Senior executives from the six major studios in coordination with Rackspace, EMC, EVault, Front Porch Digital, DAX, Google and other cloud companies convened recently to serve as governing body to collectively guide this process. The project is looking at the life cycle of film and media production, from pre-production collaboration, production, post production and through to archiving.

This specific part of the effort is around

Bringing together various competitive organizations to work on a common goal
Goal of developing an interoperable cloud framework
First steps of socializing well underway
Next important step of creating a prototype was discussed in October
Now we need to execute on design and implementation

group structure

design

High Level Goals

Ingesting data from the many changes throughout the media pipeline
Metadata from the data stored and the pipeline activities
URL/URN marker identification for assets

What the Services Will Do

Client Ingest and Server Ingest that use REST API
Create metadata on data ingest (e.g. Exif)
Create additional metadata on post-ingest (e.g. hashed ID)

Sprint Work Ideas

document IO, String REST APIs
build IO client and daemon
build State daemon
build Transform daemon
build String client and daemons

use other ETC cloud projects design details so far to influence this prototype design (some project details on basecamp)

metadata: first pass at identification process and markers File:ETC process examples.pdf. We will use some parts of the EIGR format for the data object URI.

C4 as a code base for some of the services

C4 is a Desktop/Cloud image production platform currently in development at Studio Pyxis. It is built on top of a Domain Specific Language (DSL) for high preferment cloud integrated imaging applications that will be made open source in the near future.

The C4 DSL includes an API interface to communicate with remote resources that may be useful as a reference for the ETC cloud project. Following are some highlights of the API.

Resources that are accessible through a REST API are grouped into abstract categories.

Identity - Abstraction of "account" for individual users, and organizations
- User
- Company
- Group - AKA friends, club, team, etc.
- Family
- Agent - AKA parent, representative, lawyer, etc.
Assets - Open ended abstract asset class identified by the unique SHA256 digest of the asset in question.
- Image
- Video
- Sound
- Document
Groups - Groups are usually, but not always hierarchical
- File System - AKA folder or directory
- Project
- Portfolio
- Tag
- Store
- Collection
- Archive
- Time Line - AKA clip
- Version
- Stream
- Transformation - Geometric transformation, i.e. Translate, Rotate, Scale, etc.
Actions - Operations, usually version controlled, that can be applied to other resources.
- Process - Data transformation operations such as image processing, mixing, etc.
- Movement - Operations that physically modify the location of a resource on the network, copy, move, delete, etc.
- Human - A human only process on an asset such as approve, comment on, create or modify
- Cryptographic
- Interactive
Resources
- Storage
- CPU
- GPU
- Transport Channel
Links - Relationships between resources
- Permissions
- Relationship ("agent", "friend", "employee", "team member", etc)
- Attributes
- Foreign Account

Contributing

Create a Launchpad account
https://login.launchpad.net/+new_account
Join the OpenStack Foundation free
https://www.openstack.org/join

(Use the email address you plan to use with git for code contributions)
Agree to the the Individual Contributor License agreement:
https://review.openstack.org/#/settings/

Look under "Agreements" on the left menu.
Add an SSH public key to your launchpad account
https://help.launchpad.net/YourAccount/CreatingAnSSHKeyPair
Add the same SSH key to your OpenStack Foundation / Gerrit Identity as well
https://review.openstack.org/#/settings/

Look under "SSH Public Keys" on the left menu.
Add the same SSH keys to your GitHub account
Be sure to configure git on your development machine as well if you haven't already.
$ git config --global user.name "Firstname Lastname"

$ git config --global user.email "your_email@youremail.com"
Install git-review. (In addition to python3, and pip3 )
$ sudo pip install git-review
Clone the GitHub project
$ git clone https://github.com/stackforge/milk
Work on the project (fix bugs, implement requirement, contribute code, etc)
Submit your patch for review
$ git review
Track your contributions
https://review.openstack.org/#/
If necessary manually add a reviewer
Click on your patch listed from #10 above enter one of the following names or emails in the "Add Reviewer" field, and click add.
"Sean Robers"

"Joshua Kolden"