ETC Cloud API Prototype project

Project Lead Sean Roberts (sarob)

purpose

The Entertainment Technology Center @ the University of Southern California (ETC) formally launched “Production in the Cloud,” a new project that brings together a core group of key media and cloud-resource leaders to develop guidelines and accelerate innovation and adoption of next-gen cloud-based content creation, production, and distribution tools and processes. Senior executives from the six major studios in coordination with Rackspace, EMC, EVault, Front Porch Digital, Google and other cloud companies convened recently to serve as governing body to collectively guide this process. The project is looking at the life cycle of film and media production, from pre-production collaboration, production, post production and through to archiving.

This specific part of the effort is around

Bringing together various competitive organizations to work on a common goal
Goal of developing an interoperable cloud framework
First steps of socializing well underway
Next important step of creating a prototype was discussed in October
Now we need to execute on design and implementation

timeline

15 October 2013: Start design COMPLETED
November 2013: Setup Continuous Integration (CI) system INPROGRESS
December 2013: Prototype development starts PENDING
February 2014: Prototype early release
March 2014: Prototype release and debugging
07-08 April 2014: Present prototype at NAB
May 2014 start design on next stage ETC clould projects framework, archive, transport, security, metadata
Aug 2014 CineGrid Brazil next stage release?

group structure

Committed developers: sarob (Sean Roberts),
mailing lists milk-dev, milk-announce here
IRC on freednode.net channel #milk-dev
weekly meetings agenda link here
weekly meeting iCal schedule through openstack IRC meeting list
easedrop meeting history
gerrit review group ACLs here

design

Design High Level Goals

Ingesting data from the many changes throughout the media pipeline
Metadata from the data stored and the pipeline activities
URL/URN marker identification for assets

Prototype implementation plan

Create API with WSME/Pecan support metadata return function; PUT, POST, GET
first pass at identification process and markers File:ETC process examples.pdf

To build

IO client
- generate checksum
- client will post projectID, data type, original date&time, data binary, auth, extension for client defined metadata in XML format
IO API
- auth through keystone API
- passes to IO daemon
IO daemon
- support 15-20 data types
- generate reference URN
- listens to API
- put raw binary into swift API
state daemon
- zookeeper storage
- monitor workflow
- trigger alert on workflow timeouts, errors
transform daemon
- trigger on IO swift change
- check on data type, if match some data types, execute transform
- FFMBC for transform
- support collaborative standards like scripte XML transform