Netflix Escrow
Service
Table of Contents
I. Copyright Info ... 2
Ownership of copyright Permissions
Enforcement of copyright
II. Introduction ... 3
The Company Mission Statement
III. Objectives ... 3
Divorce Deal Factors From Operational Fulfillment Reduce Operational Compression
Eliminate Trickle Delivery Enable Flood Delivery
Remove Destruction Requirements Provide Additional Disaster Recovery
IV. Technical Details ... 7
Overview Transferring Files Storage Validation Access Proxy Files V. Metrics ... 9 Delivery Data Cost of Storage
VI. Appendix ... 12
Definitions
Copyright Info
Copyright © 2013 Netflix Inc. Ownership of copyrightThe copyright of this material (including without limitation the text, artwork, photographs, images is owned by Netflix Inc.
Permissions
You may request permission to use the copyright materials on by writing to cfetner@netflix.com. Enforcement of copyright
Netflix takes the protection of its copyright very seriously.
If Netflix discovers that you have used its copyright materials in contravention of the license above, Netflix may bring legal proceedings against you seeking monetary damages and an injunction to stop you using those materials. You could also be ordered to pay legal costs.
If you become aware of any use of Netflix’s copyright materials that contravenes or may contravene the license above, please report this by email to cfetner@netflix.com.
Introduction
The CompanyWith more than 30 million members in the United States, Canada, Latin America, the United Kingdom, Ireland, Norway, Denmark, Sweden and Finland, Netflix is the world’s leading Internet subscription service for enjoying movies and TV shows. Netflix members can instantly watch unlimited movies and TV episodes streaming over the Internet to PCs, Macs and TVs. Among the large and expanding base of devices streaming from Netflix are Microsoft’s Xbox 360, Nintendo’s Wii and Sony’s PS3 consoles; an array of Blu-‐ray disc players, Internet-‐connected TVs, home theater systems, digital video recorders and Internet video players; Apple’s iPhone, iPad and iPod touch, as well as Apple TV and Google TV. In all, more than 800 devices that stream from Netflix are available. For more information, visit www.netflix.com.
Mission Statement
"Connecting people with the movies they'll love."
Our passion for helping connect people to great movies is not all altruistic. When people love the movies they watch, they become more passionate about movies, and that helps our business. And as we continue to grow, we are, in turn, able to deliver more movie titles thus increasing customers’ delight.
Using the technology and community of the Internet, we help our customers select movies tailored to their individual tastes, so they love more of the movies they watch.
Objectives
We are proposing an asset escrow service addressing some of the inefficiencies and reliability issues we've discovered in the entertainment industry’s Digital Supply Chain.
At Netflix, we strive to give our subscribers the best experience possible. The quality of experience is not only a reflection upon Netflix but also represents the quality of all our partners. In order for Netflix to be truly successful we need all our partners to be successful. As we continue to increase our catalog of available content and expand internationally, we have to focus on efficiency, reliability, and scalability so the business can meet the needs of our growth. We believe by working with the content owners, and service partners, everyone can
benefit when operational and technical gains are achieved. Currently, there are several issues negatively impacting Digital Supply Chain both within the Content Owners operations and those of Service Partners. The Netflix Escrow Storage program is being established and funded by Netflix to mitigate these factors in advance of a large volume delivery to our platform.
Divorce Deal Factors From Operational Fulfillment
Content owners are understandably reluctant to deliver assets in advance of a ratified licensing agreement. This legacy operational function relates to Tape-‐Based workflows. Traditionally, significant costs were incurred during the creation of a distribution tape and until the deals were agreed, content owners avoided those costs. In a file-‐based workflow, little to no cost are incurred during the delivery process and when they are, Netflix typically indemnifies the
content owner against these costs. We do however understand content owners wish to
maintain control of their assets until a deal is completed. We believe by providing a temporary, secure, cloud-‐based storage solution over which our partners have absolute control, we can reduce the delays associated with this operational necessity and still provide a comfortable transaction to the Content Partner. Furthermore, by allowing Netflix to have insight into the contents of the storage, but not access…the content partner is providing us with a confidence in their ability to deliver on the deal within the specified deadlines in advance of the agreement’s ratification.
Reduce Operational Compression
Because under the current operational paradigm asset deliveries are held back until deals are ratified, the operational timeline gets compressed. This results in a mad rush to get the asset deliveries completed by the Content Partner Operational groups. This operational compression drives higher Quality Control failures, increases fulfillment costs by incurring “rush charges” from Service Partners, and in general fosters a suboptimal Digital Supply Chain. Within the current Post Production landscape, Content Partners are dis-‐incentivized to work more than a few weeks ahead of a delivery deadline. Often localized spinning disc storage reserves are insufficient for medium-‐term archive of manufactured assets. In the instances where storage is available through their service partners, content owners are usually charged a fee to keep the assets staged for delivery beyond a short period. Because of these storage infrastructure limitations, operational groups within the content owner organizations cannot easily begin work on file-‐based deliveries well in advance of deal ratification.
The Netflix Escrow Service provides a place where content owners, studios and post-‐houses can park assets and retain control over those assets. This provides the ability for the content
owners, studios and post-‐houses to get operational work completed ahead of deadlines regardless of deal disposition.
Eliminate Trickle Delivery
Post facilities and internal service departments are seldom staffed or resourced for a content partner’s highest potential delivery volume. Instead, facilities (internal or external) are designed and resourced based on averages. By doing this, the organization ensures optimal utilization metrics over the course of a fiscal year. If they see a general increase in demand for resources, they generate a CAPEX request and increase capacity as justified by the demand. This “thin” overhead model creates a highly efficient facility that can cost effectively fulfill the average needs of a content partner. However, when demands exceed the calculated averages, this trickle delivery model of fulfillment based on averages becomes an operational liability to successfully completing delivery. During Netflix launches into new territories, we routinely license thousands of hours of content requiring fulfillment to a single deadline. These surges in fulfillment volumes coupled with standard deliveries tax the limited infrastructure within Content Partner’s Digital Supply Chain. Limitations in scalability, connectivity and lack of mid-‐ term storage resources all result in slow delivery that is limited in a linear manner.
Enable Flood Delivery
By giving Content Providers the ability to park assets in the cloud ahead of delivery, they can extend the period over which fulfillment of a Netflix launch order is completed. The elongation of their Supply Chain allows the Content Partners to better fit the additional volume for a launch into their average monthly projections and avoids increasing capacity that would otherwise go under utilized during slower periods. Because the Netflix Escrow Storage is cloud-‐ based, the contents in the Escrow Storage can be activated to flow immediately into Netflix’s Digital Supply Chain once a deal is agreed. This cloud-‐to-‐cloud delivery allows a content owner to literally flood Netflix’s highly scalable Media Pipeline (MPL) with previously vetted mezzanine files, resulting in efficient delivery even when the license agreements are ratified very close to launch dates. This flood delivery from Escrow storage mitigates bandwidth connectivity issues, allows for better management of operational resources in the weeks leading to a launch
delivery and enables cloud-‐based automated QC/Validation to resolve basic files issue ahead of delivery.
Remove Destruction Requirements
Typical licensing agreements require destruction of assets once they fall out of contractual window dates. This requirement is also a legacy function of Tape-‐Based workflows where physical copies of the material posed a piracy threat should they be carelessly discarded. This same requirement to “destroy” an assets isn’t a necessary function to ensure that content is protected from piracy during periods it falls out of window. Under some circumstances, content that was delivered and onsite may fall out of window for a period before a new license is agreed. During this period, Netflix proposes using the same Escrow model of storage rather than destruction of the file. Effectively, Netflix would provide a locked cloud-‐based disc to hold this asset until a new deal is agreed or until a mutually agreed expiration date has been reached, at which time it would be automatically destroyed. By doing this, the original effort on both the Content Partners and Netflix operational teams is not lost and the velocity to restore the asset to the platform is maintained. Giving Content Partners the control of the file in Escrow Storage prevents duplication of effort on both sides of the partnership for subsequent licensing.
Provide Additional Disaster Recovery
In 2008, a major Hollywood studio suffered a severe facility fire that destroyed about 50,000 assets. The recovery of these assets was never fully achieved and many of the files were simply lost or remanufactured at additional costs. Although a major consideration for every content owner, disaster recovery is often a significantly under executed due to unfavorable risk versus cost modeling. Often the expense of providing a durable disaster recovery plan requires expenditures not supported by financial necessity. Generally, content partners don't hold conformed assets in the specifications required by Netflix beyond the agreed acceptance period outlined in the licensing agreement, sometimes even less. In the event of a major disaster during the fulfillment process, Netflix’s supply chain would be vulnerable to any weaknesses in the content partners ability to quickly recover or remanufacture those assets. We understand disaster recovery plans to protect against this relatively unlikely scenario are difficult to justify financially for the content partners, but extremely important to Netflix’s ability to ensure launch dates can be achieved. We understand fiscal constraints might make a ROI on this type of redundancy unfeasible for content partners. However, Netflix believes offering Escrow Storage in advance of deal ratification will provide a new level of Supply Chain durability without introducing additional costs to the content partner operational groups.
from the Escrow Storage. Any material held in Escrow Storage during the license period would effectively act as an additional copy to be accessed during a disaster recovery situation.
Technical Details
Overview Transferring FilesAssets will be transferred using Aspera (see Appendix for description of Aspera) with AES-‐128 encryption enabled.
Aspera utilizes TCP and UDP ports to enable high-‐speed transfers. If the Content Provider (client-‐side) sits behind a restrictive firewall or ACL, they will need to allow access to / or whitelist specific TCP ports, UDP ports and IP addresses.
The Content Provider (client-‐side) will also need to provide Netflix with any/all public-‐facing IP addresses used to transfer files so that Netflix can allow access to the Escrow Service. This is done by whitelisting these IP addresses using Amazon EC2's Security Group policies. Only IP addresses that have been whitelisted will be able to access the Escrow Service.
Content Owner OPS
Third Party Vendor (Post House) Escrow
Storage
Netflix Media Pipeline
Content Owner Sales Group Out of License Material
returned to Escrow
Bi-Directional Reporting to Vendor DAM possible
Netflix Content Acquisition Group
Netflix Content OPS
Storage
All assets will be stored on Amazon S3 (see Appendix for description of Amazon S3) with server side encryption at rest using AES-‐256. The data is written to disk encrypted while Amazon manages all encryption and decryption keys.
With server side encryption all objects are encrypted with a unique key. This object key is encrypted by a separate master key. A new master key is issued at least monthly. Encrypted data, encryption keys and master keys are stored and secured on separate hosts for multiple layers of protection.
Validation
The Escrow Service provides a set of automated checks to ensure the validity and correctness of video, audio and timed-‐text assets. This helps ensure the quality of assets and any assets found to have issues will need to be replaced.
Transport Stream conformance, MPEG-‐2 conformance, H.264 conformance and Telecine detection are some of the main automated checks currently in place for video. As for audio we have implemented silence detection and duration conformance checks. Timed-‐text files will also be checked for conformance relative to specifications.
Messaging of the automated check results will be implemented in the future and more detail will be provided once this portion has been fully designed.
Access
Access can be granted at the object level meaning you can assign unique access rights for every individual asset in Escrow Storage. Netflix will only have rights to access assets in Escrow Storage where the Content Provider has granted them.
Proxy Files
Once assets are transferred into Escrow Storage, proxy files will be generated for the primary audio and video.
Metrics
Delivery Data
*This graph illustrates the QC failure rate across all content suppliers based on the 9/1/2012 deadline for Nordic Launch Material. Clearly, the QC failure rate significantly climbed as the deadline approached.
Cost of Storage
Externally Owned Storage (Post Facility)
Table 1 - Using Service Vendor Storage (No CAPEX)
Vendor Days/GB 0-‐30 Days/GB 30+ Online /File
Nearline /File
Monthly
Cost/Online Cost/Offline Monthly
Averages $0.07 $0.04 $7.74 $4.46 $6,192.40 $3,570.80
Cost of Online
Storage 90 Days Cost of Offline Storage 90 Days $18,577.20 $10,712.40 Cost of Online
Storage 120 Days Cost of Offline Storage 120 Days $24,769.60 $14,283.20
This is based on average storage fees from Post Facilities in Los Angeles, assuming a launch volume of 800 hours of HD content in ProRes 422 (HQ). This illustrates that content partners would pay an additional $18k to manufacture and hold assets 90 days prior to delivery and $25K for 120 days of storage prior to delivery.
0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00%
QC Fail Percent Relative to Due
Date
Internally Owned Storage
Table 2 - Using Internally Owned Enterprise Storage (with CAPEX ~$1.5M, depreciated over 36 months)
This table illustrates the cost of holding 800 hours of HD content in ProRes 422 (HQ) on internally owned Enterprise Level storage for periods of 90 and 120 days ahead of launch. It also shows holding the same material in the same codec for the same periods on LTO5 tape.
Online Local Storage Days/GB 0-‐30 30-‐on /GB Online /File
Nearline /File
Monthly
Cost/Online Cost/Offline Monthly
Isilon 72NL(x10) $0.48 $0.48 $54.43 N/A $43,542
Cost of Local Online Storage 90 Days $130,625.00
Cost of Local Online Storage 120 Days $174,167.00
Offline Local Storage
Quantum LTO 5 Drive
(471 tapes needed for backup @$45 ea)
$0.244
$27.54 $22,028*
Cost of Local Offline Storage 90 Days $7,342
Cost of Local Offline Storage 120 Days $5,507
*After the initial purchase of the LTO5 stock during the first 30 days, subsequent costs only include the depreciation of the LTO drive.
Using Amazon S3
Table 3 - Amazon Storage Pricing (800 hours of HD ProRes 422 (HQ) @ 113 GB/hour)
This table illustrates the cost of holding 800 hours HD content in ProRes 422 (HQ) on Amazon S3 for periods of 90 and 120 days ahead of launch.
Storage
Standard Reduced Redundancy Glacier GB/
Month 90 Days 120 Days Month GB/ 90 Days 120 Days Month GB/ 90 Days 120 Days
1,024 GB $0.095 $291.84 $389.12 $0.076 $233.47 $311.30 $0.01 $30.72 $40.96 89,376 GB $0.080 $21,450.24 $28,600.32 $0.064 $17,160.20 $22,880.26 $0.01 $2,681.28 $3,575.04
Total
90,400 GB $21,742.08 $28,989.44 $17,393.67 $23,191.56 $2,712.00 $3,616.00
Based on Amazon’s public rates as of 2013-‐01-‐23 Table 4 - Amazon Storage Pricing Matrix (Tiered)
This table illustrates 90 Days and 120 Days costs which are calculated using minimum and maximum Storage Tier capacities.
Storage Tier/Month
Standard Reduced Redundancy Glacier GB/
Month (min/max) 90 Days (min/max) 120 Days Month GB/ (min/max) 90 Days (min/max) 120 Days Month GB/ (min/max) 90 Days (min/max) 120 Days
First 1 TB $0.095 $291.84 $0.285 $389.12 $0.38 $0.076 $233.47 $0.228 $311.30 $0.304 $0.01 $0.03 $30.72 $40.96 $0.04 Next 49 TB $0.080 $12,334.08 $292.08 $16,445.44 $389.44 $0.064 $9,867.27 $233.66 $13,156.36 $311.56 $1,536.00 $30.75 $2,048.00 $41.00 Next 450 TB $0.070 $109,102.08 $12,334.29 $145,469.44 $16445.72 $0.056 $87,281.67 $9,867.44 $116,375.56 $13,156.59 $15,360.00 $1,536.03 $20,480.00 $2,048.04 Next 500 TB $0.065 $109,102.28 $208,942.08 $145,469.70 $278,589.44 $0.052 $167,153.67 $87,281.83 $116,375.77 $222,871.56 $15,360.03 $30,720.00 $20,480.04 $40,960.00 Next 4000 TB $0.060 $208,942.26 $946,222.08 $1,261,629.44 $278,589.68 $0.048 $167,153.82 $756,977.67 $1,009,303.56 $222,871.76 $153,600.00 $30,720.03 $204,800.00 $40,960.04 Over 5000 TB $0.055 +$946,222.25 +$1,261,629.66 $0.037 +$756,977.79 +$1,009,303.71 +$153,600.03 +$204,800.04
Based on Amazon’s public rates as of 2013-‐01-‐23 Table 5 - Amazon Data Transfer Pricing
Pricing Data Transfer IN
All data transfer in $0.000 per GB
Data Transfer OUT / Month
First 1 GB $0.000 per GB Up to 10 TB $0.120 per GB Next 40 TB $0.090 per GB Next 100 TB $0.070 per GB Next 350 TB $0.050 per GB Next 524 TB Contact Amazon Next 4 PB Contact Amazon Greater than 5 PB Contact Amazon
Appendix
Definitions AsperaAspera offers a solution for high-‐speed file transfers utilizing both TCP and UDP protocols to accelerate file transfers. Aspera also supports multiple different encryption schemes for client to server transfers.
Amazon EC2
Amazon Elastic Compute Cloud is a web service that provides scalable compute capacity in the cloud.
Amazon S3
Amazon Simple Storage Service is a web service that provides scalable storage where data can be stored and retrieved in any amount, at any time, from anywhere on the web.