The commercial media industry is undergoing a major transition as content providers move away from proprietary web plug-in based delivery mechanisms (such as Flash or Silverlight), and replace them with unified plug-in free video players that are based on HTML5 specifications and commercial media encoding capabilities. Browsers are moving away from plug-ins as well, asChrome is with NPAPI andMicrosoft Edge is with ActiveX, and toward more secure extension models.
The transition to plug-in free media has been enabled through the recent development of new specifications:
These specs were designed and developed to enable interoperable streaming to a variety of media platforms and devices. By focusing on interoperable solutions, content providers are able to reduce costs and at the same time users are able to access the content they want on the device they prefer using the app or web browser of their choice. Microsoft believes that this is a huge benefit to both content producers and consumers, and is committed to supporting companies that make this transition.
This is a long blog, and we don’t want you to miss a topic that interests you. Here’s a glimpse of what we’ll cover:
Support for ActiveX has been discontinued in Microsoft Edge, and that includes removing support for Silverlight. The reasons for this have been discussed inprevious blogs and include the emergence of viable and secure media solutions based on HTML5 extensions. Microsoft continues to support Silverlight, and Silverlight out-of-browser apps can continue to use it. Silverlight will also continue to be supported in Internet Explorer 11, so sites continue to have Silverlight options in Windows 10. At the same time, we encourage companies that are using Silverlight for media to begin the transition to DASH/MSE/CENC/EME based designs and to follow a single, DRM-interoperable encoding work flow enabled by CENC. This represents the most broadly interoperable solution across browsers, platforms, content and devices going forward.
Plug-ins like Silverlight were intended to support interoperable media by providing versions for every browser they supported. This became more difficult as devices and platforms that support browsers multiplied. Now, as the old plug-in models are being removed, replacements for them are needed. For media, a great forward looking replacement can be based on DASH, MSE, EME and CENC.
Windows 10 and Microsoft Edge support DASH, MSE, EME and CENC natively, and other major browsers ship implementations of MSE and CENC compliant EME. This support allows developers to build plug-in free web video apps that runs across a huge range of platforms and devices, with each MSE/EME implementation built on top of a different media pipeline and DRM provider.
In the days when DRM systems used proprietary file formats and encryption methods, this variation in DRM providers by browser would have presented a significant issue. With the development and use of Common Encryption (CENC), the problem is substantially reduced because the files are compressed in standard formats and encrypted using global industry standards. The service provider issues the keys and licenses necessary to consume the content in a given browser,but the website code, content and encryption keys are common across all of them, regardless of which DRM is in use. An example of such an implementation isDASH.js, the open source industry reference player used to prove out these technologies and which serves as the basis for many players across the web today.
As shown above, Microsoft’s PlayReady DRM supports two modes of DRM protection: “Software DRM”, which uses our traditional software protected media path, and “Hardware DRM”, which moves the media path into secured hardware when available on the device. Hardware DRM was designed to meet the highest requirements for commercial media content protection, and will allow streaming of the highest quality content available. Not all devices will be Hardware DRM capable, but sites built on MSE/EME can accommodate the difference and stream the best possible content quality supported by the browser or device.
Like any new technology, the transition to DASH/MSE/EME/CENC can present some challenges. These include:
As part of helping the industry move to interoperable media delivery solutions, we are investing in technologies to address these challenges.
DASH content usually consists of media files encoded at various quality levels and a manifest that provides information on the files to the media application. An MSE based player is then responsible for parsing these files, downloading the most appropriate content and feeding it into the media element’s sourceBuffer(s). This is very flexible, but requires either investment in authoring site MSE implementations or use of a reference MSE implementation such as the aforementioned DASH.js.
There is another much easier option: native DASH streaming, where the site code simply sets the manifest as the media element source, and playback is automatically managed by the browser’s built-in streaming engine. This approach allows web developers to leverage the learnings and investments made by browser makers and easily provide a premium media experience on their site. We have added support for native DASH streaming to Windows 10 and Microsoft Edge, and more details on this are available in our previous blog:Simplified Adaptive Video Streaming: Announcing support for HLS and DASH in Windows 10.
A number of websites have large media libraries encoded in the Smooth Streaming format and are looking to move to an interoperable HTML5-based solution. One possible solution is to use a library that supports their current content through MSE/EME without re-encoding. Libraries are available now that are capable of playing back Smooth Streaming content using MSE and EME. For example, a version of the “hasplayer.js” library can do just that and is available for download athttps://github.com/eiximenis/hasplayer.js.
This hasplayer.js library is based ondash.js, and enables playback of both clear and protected Smooth content using PlayReady on Microsoft Edge. It is a client-side JavaScript library that provides content and manifest translations and is cross browser compatible. Thanks to the inclusion of EME poly-fill support, it may be easily extended to support other browser’s DRM solutions as well.
Here’s a sample of JavaScript that uses hasplayer.js to retrieve and play a DASH or Smooth media file:
| function setupVideo(url) { | |
| var context = new Custom.di.CustomContext(); | |
| var player = new MediaPlayer(context); | |
| player.startup(); | |
| player.attachView(document.querySelector('#videoplayer')); | |
| player.setAutoPlay(true); | |
| player.attachSource(url); | |
| } |
That makes it very simple to support streaming Smooth content on a website. We’ve provided a sample in theContoso Video Sample GitHub Repository that uses this library to play video. You can try the webpage yourself at theContoso Video Demo Website.
It is possible to provide a client-side translation library for Smooth Streaming content because the Protected Interoperable File Format (PIFF) underlying the Smooth Streaming protocol was a primary input into the ISO Base Media File Format (ISOBMFF) spec used with DASH and because PIFF introduced the multi-DRM protocol which was standardized as ISO MPEG Common Encryption (CENC). Two PIFF formats are in broad use today – PIFF 1.1 and PIFF 1.3 – and the hasplayer.js Smooth Streaming MSE/EME library supports both formats. These formats are supported by on-the-fly conversion from the PIFF format to the Common Media Format (CMF) used with DASH. This ensures that all browser content played back by the library is conformant with DASH CMF and capable of playback in all MSE-capable browsers.
Some content owners would prefer to focus on producing great content, not the technical details required to deliver their content to consumers. These companies may wish to leverage hosted media services that prepare content for web delivery, handle the streaming logic and player UI, and manage the DRM license servers. Azure Media Services offers this capability today, with support for both PlayReady and Widevine DRM systems. This service provides support for both Video on Demand (VoD) and live streaming. A single high quality file/stream is provided to Azure, where it undergoes dynamic compression and encryption into CENC protected content that can be streamed to end devices, and a complete player solution is available for developers to simply add to their website. Some details of this service are newly announced inAzure Media Services delivers Widevine encrypted stream by partnering with castLabs.
One powerful advantage of moving to DASH/MSE/EME/CENC streaming is that the same code running on your website can be packaged as a Universal Windows Platform (UWP) app. UWP applications run on all devices with Windows 10. A website developer can create both an interoperable website based player and a Windows app that uses the same code, but offers a richer experience using Windows platform capabilities. Their common code will thus be able to handle UI and media streaming details, AND take advantage of capabilities only available to apps through WinRT APIs.
These hosted web apps:
We talked about the power of hosted apps in our talk onHosted web apps and web platform innovations at theMicrosoft Edge Web Summit 2015. You can also find reference documentationon MSDN.
We’ve taken the demo Contoso Video website shown above and packaged it as a demo UWP app that takes advantage of Windows Platform APIs. This demo shows how simple it is to take the basic video player and integrate Cortana Voice Commands. The demo also customizes the colors used in the app’s title bar. All of the JavaScript code is part of the HTML website which is deployed as part of the standard web development workflow.
Three files are needed to integrate Cortana in your Hosted Web Applications (HWA): AVoice Command Definition (VCD) file, a JavaScript file, and an HTML file.
TheVoice Command Definition (VCD) File specifies the actions you want to support with voice commands. The code below informs Cortana of the app name (Contoso Video), that the app supports a “play” command, and how the “playing” state should be represented in the app UI.
| <?xml version="1.0" encoding="utf-8"?> | |
| <VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.2"> | |
| <CommandSet xml:lang="en-us" Name="ContosoVideo"> | |
| <CommandPrefix>Contoso Video</CommandPrefix> | |
| <!-- Example text displayed in Cortana UI --> | |
| <Example>Contoso video play elephants</Example> | |
| <!-- Command name --> | |
| <Command Name="play"> | |
| <Example>play {message} using contoso video</Example> | |
| <!-- Tell Cortana what to listen for --> | |
| <ListenFor RequireAppName="BeforeOrAfterPhrase">please play {streamSubject}</ListenFor> | |
| <!-- Tell Cortana what to display in the UI --> | |
| <Feedback>playing {streamSubject} with Contoso Video</Feedback> | |
| <Navigate/> | |
| </Command> | |
| <!-- Specifies a topic for large vocabulary recognition --> | |
| <PhraseTopic Label="streamSubject" Scenario="Dictation"></PhraseTopic> | |
| </CommandSet> | |
| </VoiceCommands> |
JavaScript must listen for the Windows Activation event and check for VoiceCommands.
| // Handle Cortana activation adding the event listener before DOM | |
| // Content Loaded parse out the command type and call the | |
| // respective game APIs | |
| if (typeof Windows !== 'undefined') { | |
| console.log("Windows exists!"); | |
| // Subscribe to the Windows Activation Event | |
| Windows.UI.WebUI.WebUIApplication.addEventListener("activated", function (args) { | |
| var activation = Windows.ApplicationModel.Activation; | |
| // Check to see if the app was activated by a voice command | |
| if (args.kind === activation.ActivationKind.voiceCommand) { | |
| var speechRecognitionResult = args.result; | |
| var textSpoken = speechRecognitionResult.text; | |
| // Determine the command type {play} defined in vcd | |
| if (speechRecognitionResult.rulePath[0] === "play") { | |
| // Determine the stream name specified | |
| if (textSpoken.includes('elephants') || textSpoken.includes('Elephants')) { | |
| // Movie selected is Elephants Dream | |
| setupVideo("http://amssamples.streaming.mediaservices.windows.net/b6822ec8-5c2b-4ae0-a851-fd46a78294e9/ElephantsDream.ism/manifest(filtername=FirstFilter)"); | |
| } | |
| else if (textSpoken.includes('Steel') || textSpoken.includes('steel')) { | |
| // Stream selected is Tears of Steel | |
| setupVideo("http://amssamples.streaming.mediaservices.windows.net/bc57e088-27ec-44e0-ac20-a85ccbcd50da/TearsOfSteel.ism/manifest"); | |
| } | |
| else { | |
| // No stream specified by user | |
| console.log("No valid stream specified"); | |
| } | |
| } | |
| else { | |
| // No valid command specified | |
| console.log("No valid command specified"); | |
| } | |
| } | |
| }); | |
| } |
The HTML file must add a meta element pointing to a VCD file on your server.
| <!DOCTYPE html> | |
| <html> | |
| <head> | |
| <title>Adaptive Streaming in HTML5</title> | |
| <meta name="msapplication-cortanavcd" content="http://localhost:3000/VCD/vcd.xml" /> | |
| <link href="./styles.css" rel="stylesheet" /> | |
| </head> | |
| <body> | |
| <!-- Create centered Video player with controls --> | |
| <div> | |
| <h1>Contoso Video</h1> | |
| <video controls></video> | |
| </div> | |
| <!-- Scripts: load last --> | |
| <script src="./js/hasplayer.min.js"></script> | |
| <script src="./js/windows.js"></script> | |
| <script src="./js/player.js"></script> | |
| <script src="./js/cortana.js"></script> | |
| </body> | |
| </html> |
With the addition of a VCD, and updates to the website HTML and JavaScript, our Contoso Video website demo can now be packaged as a Universal Windows Platform application that will run across every device that runs Windows 10. Further, users can launch the app to play a video by saying“Contoso, play Tears of Steel”. Cortana will understand the command, launch the Contoso Video app and start playing the video “Caves of Steel”. The app also has a customized view on the app bar.
The full source for the Contoso Video website and demo app is available in theContoso Video Sample GitHub Repository.
DASH/MSE/EME/CENC offer compelling advantages over plug-in based solutions, and we are quickly moving towards an era of broadly interoperable streaming media. Both content providers and consumers will benefit from this transition. While the adoption of these technologies may present short-term challenges, the features and options discussed in this blog are provided to assist companies as they make this change.
We’re eager for your feedback so we can further improve our streaming media offerings, and are looking forward to seeing what you do with the tools and approaches we’ve discussed here!
– David Mebane, Program Manager, Windows Media
– Jerry Smith, Senior Program Manager, Microsoft Edge
– Kevin Hill, Senior Program Manager, Microsoft Edge
