Revision as of 23:56, 20 December 2013

ETC Cloud API Prototype project

Project Lead Sean Roberts (sarob)

purpose

The Entertainment Technology Center @ the University of Southern California (ETC) formally launched “Production in the Cloud,” a new project that brings together a core group of key media and cloud-resource leaders to develop guidelines and accelerate innovation and adoption of next-gen cloud-based content creation, production, and distribution tools and processes. Senior executives from the six major studios in coordination with Rackspace, EMC, EVault, Front Porch Digital, Google and other cloud companies convened recently to serve as governing body to collectively guide this process. The project is looking at the life cycle of film and media production, from pre-production collaboration, production, post production and through to archiving.

This specific part of the effort is around

• Bringing together various competitive organizations to work on a common goal
• Goal of developing an interoperable cloud framework
• First steps of socializing well underway
• Next important step of creating a prototype was discussed in October
• Now we need to execute on design and implementation

timeline

• 15 October 2013: Start design COMPLETED
• November 2013: Setup Continuous Integration (CI) system COMPLETED
• December 2013: Prototype development starts INPROGRESS
• February 2014: Prototype early release
• March 2014: Prototype release and debugging
• 07-08 April 2014: Present prototype at NAB
• May 2014 start design on next stage ETC clould projects framework, archive, transport, security, metadata
• Aug 2014 CineGrid Brazil next stage release?

group structure

• Committed developers: sarob (Sean Roberts), Guillaume Aubuchon, Brandon Bussinger
• mailing lists milk-dev, milk-announce here
• IRC on freednode.net channel #milk-dev
• weekly meetings agenda link here
• weekly meeting iCal schedule through openstack IRC meeting list
• easedrop meeting history
• gerrit review group ACLs here

design

High Level Goals

• Ingesting data from the many changes throughout the media pipeline
• Metadata from the data stored and the pipeline activities
• URL/URN marker identification for assets

details

• Create API with WSME/Pecan support metadata return function; PUT, POST, GET
• use other ETC cloud projects design details so far to influence this prototype design (some project details on basecamp)

1. metadata: first pass at identification process and markers File:ETC process examples.pdf. We will use some parts of the EIGR format for the data object URI.
2. archive: TBD
3. transport: TBD
4. security: TBD
5. framework: this project

To build

• IO client
  • generate checksum
  • client will post projectID, data type, original date&time, data binary, auth, extension for client defined metadata in XML format
• IO API
  • auth through keystone API
  • passes to IO daemon
• IO daemon
  • support 15-20 data types
  • generate reference URN
  • listens to API
  • put raw binary into swift API
• state daemon
  • zookeeper storage
  • monitor workflow
  • trigger alert on workflow timeouts, errors
• transform daemon
  • trigger on IO swift change
  • check on data type, if match some data types, execute transform
  • FFMBC for transform
  • support collaborative standards like scripte XML transform

Work to support APIs from

• PIX, Critique, 5thKind, CineGlass, OpenStack, FIMS, Scripte

Reference Images

• 

  12nov2013 ETC production in the cloud board meeting

• 

  etcc-design-meeting-17dec2013

• 

  ETC process examples

Next Stage projects likely will be built on top of the OpenStack projects Congress and Solum

source

Link here to Milk stackforge repository

review status

Link here to Milk gerrit review page

project activity

overview of project contributors and activity to date

Last edit 38837 by Sarob on 20131220235628 (year+month+day+UTC hour+seconds)