YouTube Engineering and Developers Blog: chrome

Engineering and Developers Blog

What's happening with engineering and developers at YouTube

Smoother <video> in Chrome

Tuesday, November 10, 2015

Video quality matters, and when an HD or HFR playback isn’t smooth, we notice. Chrome noticed. YouTube noticed. So we got together to make YouTube video playback smoother in Chrome, and we call it Project Butter.
For some context, our brains fill in the motion in between frames if each frame is onscreen the same amount of time - this is called motion interpolation. In other words, a 30 frames per second video won’t appear smooth unless each frame is spaced evenly each 1/30th of a second. Smoothness is more complicated than just this - you can read more about it in this article by Michael Abrash at Valve.
Frame rates, display refresh rates and cadenceYour device’s screen redraws itself at a certain frame rate. Videos present frames at a certain rate. These rates are often not the same. At YouTube we commonly see videos authored at 24, 25, 29.97, 30, 48, 50, 59.94, and 60 frames per second (fps) and these videos are viewed on displays with different refresh rates - the most common being 50Hz (Europe) and 60Hz (USA).
For a video to be smooth we need to figure out the best, most regular way to display the frames - the best cadence. The ideal cadence is calculated as the ratio of the display rate to frame rate. For example, if we have a 60Hz display (a 1/60 second display interval) and a 30 fps clip, 60 / 30 == 2 which means each video frame should be displayed for two display intervals of total duration 2 * 1/60 second.
We played videos a bunch of different ways and scored them on smoothness.
Smoothness scoreUsing off the shelf HDMI capture hardware and some special video clips we computed a percentage score based on the number of times each video frame was displayed relative to a calculated optimal display count. The higher the score, the more frames aligned with the optimal display frequency. Below is a figure showing how Chrome 43 performed when playing a 30fps clip back on a 60Hz display:

Smoothness: 68.49%, ~Dropped: 5 / 900 (0.555556%)
The y-axis is the number of times each frame was displayed, while the x-axis is the frame number. As mentioned previously the calculated ideal display count for a 30fps clip on a 60Hz display is 2. So, in an ideal situation, the graph should be a flat horizontal line at 2, yet Chrome dropped many frames and displayed certain frames for as many as 4 display cycles! The smoothness score reflects this - only 68.49 percent of frames were displayed correctly. How could we track down what was going on?
Using some of the performance tracing tools built into Chrome, we identified timing issues inherent to the existing design for video rendering as the culprit. These issues resulted in both missed and irregular video frames on a regular basis.

There were two main problems in the interactions between Chrome’s compositor (responsible for drawing frames) and its media pipeline (responsible for generating frames) --

The compositor didn’t have a timely way of knowing when a video frame needed display. Video frames were selected on the media pipeline thread while the compositor would occasionally come along looking for them on the compositor thread, but if the compositor thread was busy it wouldn’t get the notification on time.

Chrome’s media pipeline didn’t know when the compositor would be ready to draw its next new frame. This led to the media pipeline sometimes picking a frame that was too old by the time the compositor displayed it.

In Chrome 44, we re-architected the media and compositor pipelines to communicate carefully about the intent to generate and display. Additionally, we also improved which video frames to pick by using the optimal display count information. With these changes, Chrome 44 significantly improved on smoothness scores across all video frame rates and display refresh rates:

Smoothness: 99.33%, ~Dropped: 0 / 900 (0.000000%)
Smooth like butter. Read more in public design document, if you’re interested in further details.
Dale Curtis, Software Engineer, recently watched WARNING: SCARIEST GAME IN YEARS | Five Nights at Freddy's - Part 1Richard Leider, Engineering Manager, recently watched Late Art Tutorial.Renganathan Ramamoorthy, Product Manager, recently watched Video Game High School

Google

Labels: chrome , compositing , rendering , video

A Chrome Extension for YouTube Activity Feeds

Wednesday, December 22, 2010

Slave Jovanovski, an engineer at YouTube, has put together a Google Chrome extension that should be of interest to the YouTube API community. It’s called YouTube Feed, and after installing and authenticating with your YouTube account, it automatically will fetch your YouTube social activity stream (both subscriptions and friends’ actions) while you use Google Chrome. When a new event, like a YouTube friend uploading or commenting on a video, takes place, the extension will notify you and provide details on the activity, as well as links to view the actual video. You have control over which types of activities you’d like to be notified about, as well as how frequently you’d like the extension to check for updates.

While you’ll hopefully find the extension useful on its own merits, the fact that the source code has been released as part of an open source project means that the extension’s code can serve as inspiration (or a jumping off point) for writing your own JavaScript code that interacts with the YouTube API. Curious as to how to use OAuth to authenticate YouTube accounts from a Chrome Extension? Or request JSON data with a JavaScript callback? The answers await you in the source code!

Cheers,
–Jeff Posnick, YouTube API Team