Engineering and Developers Blog
What's happening with engineering and developers at YouTube
Smoother <video> in Chrome
Tuesday, November 10, 2015
Video quality matters, and when an HD or HFR playback isn’t smooth, we notice. Chrome noticed. YouTube noticed. So we got together to make YouTube video playback smoother in Chrome, and we call it Project Butter.
For some context, our brains fill in the motion in between frames if each frame is onscreen the same amount of time - this is called
motion interpolation
. In other words, a 30 frames per second video won’t appear smooth unless each frame is spaced evenly each 1/30th of a second. Smoothness is more complicated than just this - you can read more about it in this
article by Michael Abrash at Valve
.
Frame rates, display refresh rates and cadence
Your device’s screen redraws itself at a certain frame rate. Videos present frames at a certain rate. These rates are often not the same. At YouTube we commonly see videos authored at
24
,
25
, 29.97, 30, 48,
50
, 59.94, and
60
frames per second (fps) and these videos are viewed on displays with different
refresh rates
- the most common being 50Hz (Europe) and 60Hz (USA).
For a video to be smooth we need to figure out the best, most regular way to display the frames - the best cadence. The ideal cadence is calculated as the ratio of the display rate to frame rate. For example, if we have a 60Hz display (a 1/60 second display interval) and a 30 fps clip, 60 / 30 == 2 which means each video frame should be displayed for two display intervals of total duration 2 * 1/60 second.
We played videos a bunch of different ways and scored them on smoothness.
Smoothness score
Using off the shelf
HDMI capture hardware
and some
special video clips
we computed a percentage score based on the number of times each video frame was displayed relative to a calculated optimal display count. The higher the score, the more frames aligned with the optimal display frequency. Below is a figure showing how Chrome 43 performed when playing a 30fps clip back on a 60Hz display:
Smoothness: 68.49%, ~Dropped: 5 / 900 (0.555556%)
The y-axis is the number of times each frame was displayed, while the x-axis is the frame number. As mentioned previously the calculated ideal display count for a 30fps clip on a 60Hz display is 2. So, in an ideal situation, the graph should be a flat horizontal line at 2, yet Chrome dropped many frames and displayed certain frames for as many as 4 display cycles! The smoothness score reflects this - only 68.49 percent of frames were displayed correctly. How could we track down what was going on?
Using some of the
performance tracing tools
built into Chrome, we identified
timing issues
inherent to the existing design for video rendering as the culprit. These issues resulted in both missed and irregular video frames on a regular basis.
There were two main problems in the interactions between Chrome’s
compositor
(responsible for drawing frames) and its media pipeline (responsible for generating frames) --
The compositor didn’t have a timely way of knowing when a video frame needed display. Video frames were selected on the media pipeline thread while the compositor would occasionally come along looking for them on the compositor thread, but if the compositor thread was busy it wouldn’t get the notification on time.
Chrome’s media pipeline didn’t know
when
the compositor would be ready to draw its next new frame. This led to the media pipeline sometimes picking a frame that was too old by the time the compositor displayed it.
In Chrome 44, we re-architected the media and compositor pipelines to communicate carefully about the intent to generate and display. Additionally, we also improved which video frames to pick by using the optimal display count information. With these changes, Chrome 44 significantly improved on smoothness scores across all video frame rates and display refresh rates:
Smoothness: 99.33%, ~Dropped: 0 / 900 (0.000000%)
Smooth like butter. Read more in
public design document
, if you’re interested in further details.
Dale Curtis, Software Engineer, recently watched
WARNING: SCARIEST GAME IN YEARS | Five Nights at Freddy's - Part 1
Richard Leider, Engineering Manager, recently watched
Late Art Tutorial
.
Renganathan Ramamoorthy, Product Manager, recently watched
Video Game High School
A Chrome Extension for YouTube Activity Feeds
Wednesday, December 22, 2010
Slave Jovanovski, an engineer at YouTube, has put together a
Google Chrome extension
that should be of interest to the YouTube API community. It’s called
YouTube Feed
, and after installing and authenticating with your YouTube account, it automatically will fetch your YouTube social activity stream (both subscriptions and friends’ actions) while you use Google Chrome. When a new event, like a YouTube friend uploading or commenting on a video, takes place, the extension will notify you and provide details on the activity, as well as links to view the actual video. You have control over which types of activities you’d like to be notified about, as well as how frequently you’d like the extension to check for updates.
While you’ll hopefully find the extension useful on its own merits, the fact that the source code has been released as part of an
open source project
means that the extension’s code can serve as inspiration (or a jumping off point) for writing your own JavaScript code that interacts with the YouTube API. Curious as to how to use OAuth to authenticate YouTube accounts from a Chrome Extension? Or request JSON data with a JavaScript callback? The answers await you in the
source code
!
Cheers,
–Jeff Posnick, YouTube API Team
Labels
.net
360
acceleration
access control
accessibility
actionscript
activities
activity
android
announcements
apis
app engine
appengine
apps script
as2
as3
atom
authentication
authorization
authsub
best practices
blackops
bootcamp
captions
categories
channels
charts
chrome
chromeless
client library
clientlibraries
clientlogin
code
color
comments
compositing
create
curation
custom player
decommission
default
deprecation
devs
direct
discovery
docs
Documentation RSS
dotnet
education
embed
embedding
events
extension
feeds
flash
format
friendactivity
friends
fun
gears
google developers live
google group
googlegamedev
googleio
html5
https
iframe
insight
io12
io2011
ios
iphone
irc
issue tracker
java
javascript
json
json-c
jsonc
knight
legacy
Live Streaming API
LiveBroadcasts API
logo
machine learning
mashups
media:keywords keywords tags metadata
metadata
mobile
mozilla
NAB 2016
news
oauth
oauth2
office hours
open source
partial
partial response
partial update
partners
patch
php
player
playlists
policy
previews
pubsubhubbub
push
python
quota
rails
releases
rendering
reports
responses
resumable
ruby
samples
sandbox
shortform
ssl https certificate staging stage
stack overflow
stage video
staging
standard feeds
storify
storyful
subscription
sup
Super Chat API
survey
tdd
theme
tos
tutorials
updates
uploads
v2
v3
video
video files
video transcoding
virtual reality
voting
VR
watch history
watchlater
webvtt
youtube
youtube api
YouTube Data API
youtube developers live
youtube direct
YouTube live
YouTube Reporting API
ytd
Archive
2017
Mar
Jan
2016
Nov
Oct
Aug
May
Apr
2015
Dec
Nov
Oct
May
Apr
Mar
Jan
2014
Oct
Sep
Aug
May
Mar
2013
Dec
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
2012
Dec
Nov
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Nov
Oct
Sep
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
2007
Dec
Nov
Aug
Jun
May
Feed
YouTube
on
Follow @youtubedev