Milk

ETC Cloud API Prototype project

Project Lead Sean Roberts (sarob)

purpose

The Entertainment Technology Center @ the University of Southern California (ETC) formally launched “Production in the Cloud,” a new project that brings together a core group of key media and cloud-resource leaders to develop guidelines and accelerate innovation and adoption of next-gen cloud-based content creation, production, and distribution tools and processes. Senior executives from the six major studios in coordination with Rackspace, EMC, EVault, Front Porch Digital, DAX, Google and other cloud companies convened recently to serve as governing body to collectively guide this process. The project is looking at the life cycle of film and media production, from pre-production collaboration, production, post production and through to archiving.

This specific part of the effort is around

• Bringing together various competitive organizations to work on a common goal
• Goal of developing an interoperable cloud framework
• First steps of socializing well underway
• Next important step of creating a prototype was discussed in October
• Now we need to execute on design and implementation

timeline

• 15 October 2013: Start design COMPLETED
• November 2013: Setup Continuous Integration (CI) system COMPLETED
• December 2013: Prototype development starts INPROGRESS
• February 2014: Prototype early release
• March 2014: Prototype release and debugging
• March 4-6, 2014: SXSW possible bugfest, hackathon
• 07-08 April 2014: Present prototype at NAB
• May 2014 start design on next stage ETC clould projects framework, archive, transport, security, metadata
• Aug 2014 CineGrid Brazil next stage release?

group structure

• mailing list milk-dev@etcusc.org
• mailing list history
• developer IRC on freednode.net channel #milk-dev
• weekly meetings agenda
• weekly meeting iCal schedule through openstack IRC meeting list
• meeting history Past IRC Meetings
• gerrit review group ACLs here

design

High Level Goals

• Ingesting data from the many changes throughout the media pipeline
• Metadata from the data stored and the pipeline activities
• URL/URN marker identification for assets

details

• Create API with WSME/Pecan support metadata return function; PUT, POST, GET
• use other ETC cloud projects design details so far to influence this prototype design (some project details on basecamp)

1. metadata: first pass at identification process and markers File:ETC process examples.pdf. We will use some parts of the EIGR format for the data object URI.
2. archive: TBD
3. transport: TBD
4. security: TBD
5. framework: this project

To build

API / daemon names	function
IO API client	generate checksum, client will post projectID, data type, original date&time, data binary, auth, extension for client defined metadata in XML format
IO API	auth through keystone API, passes to IO daemon
IO daemon	support 15-20 data types, generate reference URN, listens to API, put raw binary into swift API
state daemon	zookeeper storage, monitor workflow, trigger alert on workflow timeouts, errors
transform daemon	trigger on IO swift change, check on data type, if match some data types, execute transform, FFMBC for transform, support collaborative standards like scripte XML transform
metadata API	...
metadata API client	...

concept	description
Example	Example

IO API

verb	URI	description
Example	Example	Example

metadata API

verb	URI	description
Example	Example	Example

Work to support APIs from

• DAX, PIX, Critique, 5thKind, CineGlass, OpenStack, FIMS, Scripte

Reference Images

• 

  12nov2013 ETC production in the cloud board meeting

• 

  etcc-design-meeting-17dec2013

• 

  ETC process examples

source

Link here to Milk stackforge repository

review status

Link here to Milk gerrit review page

project activity

overview of project contributors and activity to date

Last edit 40488 by Chris Chen on 20140124224112 (year+month+day+UTC hour+seconds)