“Don’t let the fox guard the henhouse,” the old adage goes. But for our bug bounty program, we’ve flipped this conventional wisdom on its head to yield some strong results for the security of our online properties.
Since its inception three years ago, our bug bounty program has increasingly helped to harden the security of our products. Over this short period, we’ve received thousands of submissions, and, as of December 2016, the bounties awarded for reports that resulted in real bug fixes has now surpassed a total of $2 million. Just last month, a security researcher helped us identify and patch a vulnerability in Flickr.
In 2016 alone, we awarded nearly 200 researchers around the world. These bounties helped to fix vulnerabilities of varying severity across our web properties. Most bounties accounted for less impactful vulnerabilities, but some were more substantial.
Yes, this all comes with a degree of vulnerability. After all, we’re asking some of the world’s best hackers to seek out soft spots in our defenses. But it’s acceptable risk. The right incentives combined with some hackers who actually want to do some good has resulted in a diverse and growing global community of contributors to our security. Currently, our bug bounty program sees more than 2,000 contributors from more than 80 countries.
Visual representation of the locations of researchers who have contributed to Yahoo’s bug bounty program.
In 2017, we’ll look to continue to foster this healthy marriage in security. Attracting the highest skilled hackers to our program with meaningful bounties will continue to result in impactful bug reporting.
Following a recent investigation, we’ve identified data security issues concerning certain Yahoo user accounts. We’ve taken steps to secure those user accounts and we’re working closely with law enforcement.
What happened?
As we previously disclosed in November, law enforcement provided us with data files that a third party claimed was Yahoo user data. We analyzed this data with the assistance of outside forensic experts and found that it appears to be Yahoo user data. Based on further analysis of this data by the forensic experts, we believe an unauthorized third party, in August 2013, stole data associated with more than one billion user accounts. We have not been able to identify the intrusion associated with this theft. We believe this incident is likely distinct from the incident we disclosed on September 22, 2016.
For potentially affected accounts, the stolen user account information may have included names, email addresses, telephone numbers, dates of birth, hashed passwords (using MD5) and, in some cases, encrypted or unencrypted security questions and answers. The investigation indicates that the stolen information did not include passwords in clear text, payment card data, or bank account information. Payment card data and bank account information are not stored in the system the company believes was affected.
Separately, we previously disclosed that our outside forensic experts were investigating the creation of forged cookies that could allow an intruder to access users’ accounts without a password. Based on the ongoing investigation, we believe an unauthorized third party accessed our proprietary code to learn how to forge cookies. The outside forensic experts have identified user accounts for which they believe forged cookies were taken or used. We are notifying the affected account holders, and have invalidated the forged cookies. We have connected some of this activity to the same state-sponsored actor believed to be responsible for the data theft the company disclosed on September 22, 2016.
What are we doing to protect our users?
We are notifying potentially affected users and have taken steps to secure their accounts, including requiring users to change their passwords. We have also invalidated unencrypted security questions and answers so that they cannot be used to access an account. With respect to the cookie forging activity, we invalidated the forged cookies and hardened our systems to secure them against similar attacks. We continuously enhance our safeguards and systems that detect and prevent unauthorized access to user accounts.
What can users do to protect their account?
We encourage our users to visit our Safety Center page for recommendations on how to stay secure online. Some important recommendations we’re re-emphasizing today include the following:
Change your passwords and security questions and answers for any other accounts on which you used the same or similar information used for your Yahoo account;
Review all of your accounts for suspicious activity;
Be cautious of any unsolicited communications that ask for your personal information or refer you to a web page asking for personal information;
Avoid clicking on links or downloading attachments from suspicious emails; and
Consider using Yahoo Account Key, a simple authentication tool that eliminates the need to use a password on Yahoo altogether.
For more information about these security matters and our security resources, please visit the Yahoo Security Issue FAQs page, https://yahoo.com/security-update.
Statements in this press release regarding the findings of Yahoo’s ongoing investigations involve potential risks and uncertainties. The final conclusions of the investigations may differ from the findings to date due to various factors including, but not limited to, the discovery of new or additional information and other developments that may arise during the course of the investigation. More information about potential risks and uncertainties of security breaches that could affect the Company’s business and financial results is included under the caption “Risk Factors” in the Company’s Quarterly Report on Form 10-Q for the quarter ended September 30, 2016, which is on file with the SEC and available on the SEC’s website atwww.sec.gov.
By Kathleen Lefstad, Policy Manager, Trust & Safety
Yahoo’s “train the trainer” Digital Online Safety Course was shared with law enforcement in Quincy, Washington this past week, with school resource officers from Grant County, Warden, Ephrata, Yakima, Moses Lake and Quincy in attendance. With more than 1,000 officers trained to date, Yahoo was proud to bring this course to Quincy, providing the resources and tools to help officers facilitate discussions about online safety and good digital citizenship with their communities.
Police Chief Bob Heimbach was grateful for Yahoo’s commitment to bring the course to Washington saying, “With the world interconnected in this electronic age, this safety training, and providing us the ability to support our community members in digital safety, is invaluable. Yahoo has demonstrated their intent and commitment to being a good partner and community member here in Quincy.”
It was nearly eight years ago that the course was first created, when Officer Holly Lawrence approached Yahoo to create presentations for School Resource Officers to give about safety and citizenship in a digital world. The training has been successful due to it’s focus on education of the material, sharing of available resources and, specifically, how to present the material effectively for different audiences.
With an emphasis on communication, these presentations open the door to talk about online trends and safety issues, and identify workable solutions and preparedness together. “The old adage about ‘it takes a village’ is still true, but maybe we should start saying ‘it takes an ivillage,’” said Officer Holly Lawrence, Ret., a law enforcement partner of Yahoo, who helps run these courses nationwide. “As more communities develop and thrive in the digital space, kids and their trusted adults need the tools to be able to speak one-to-one (if not face-to-face) about the challenges and opportunities of life online.”
By Dylan Casey, Vice President of Product Management
We’re making it easier than ever to see and manage all of the devices connected to your Yahoo account. Today, you might notice some new improvements to help you keep track of the account activity and devices associated with your Yahoo account. This information is available to all users under “Account Info” here: https://login.yahoo.com/account/activity. Before we get too technical, let’s explain how this works in a real-world scenario.
Imagine that your phone falls out of your pocket in a taxi and later that day you realize that you’ve lost it. From a computer, tablet or alternate device, just sign in to your Yahoo account and head over to “Account Info.” There you’ll find a tab that says “Recent Activity.” Find the apps on your phone that are shown to have access to your account and remove them. This will invalidate the OAuth token so that no one else can use those apps to access your account on your lost phone. The same can be done for any other devices you might own that are authorized to use your Yahoo account, including a laptop, desktop computer, tablet or cell phone.
Users already had the ability to invalidate OAuth tokens through the Member Center, but this feature makes it easier to see and control which devices and apps are validated to access their Yahoo account, offering greater convenience and peace of mind.
We have confirmed that a copy of certain user account information was stolen from the company’s network in late 2014 in what we believe is a state-sponsored actor. The account information may have included names, email addresses, telephone numbers, dates of birth, hashed passwords (the vast majority with bcrypt) and, in some cases, encrypted or unencrypted security questions and answers. The ongoing investigation suggests that stolen information did not include unprotected passwords, payment card data, or bank account information; payment card data and bank account information are not stored in the system that the investigation has found to be affected. Based on the ongoing investigation, Yahoo believes that information associated with at least 500 million user accounts was stolen and the investigation has found no evidence that the state-sponsored actor is currently in Yahoo’s network. Yahoo is working closely with law enforcement on this matter.
We are taking action to protect our users:
We are notifying potentially affected users. The content of the email Yahoo is sending to those users will be available at https://yahoo.com/security-notice-content beginning at 11:30 am (PDT).
We are asking potentially affected users to promptly change their passwords and adopt alternate means of account verification.
We invalidated unencrypted security questions and answers so they cannot be used to access an account.
We are recommending that all users who haven’t changed their passwords since 2014 do so.
We continue to enhance our systems that detect and prevent unauthorized access to user accounts.
We are working closely with law enforcement on this matter.
Change your password and security questions and answers for any other accounts on which you used the same or similar information used for your Yahoo account.
Review your accounts for suspicious activity.
Be cautious of any unsolicited communications that ask for your personal information or refer you to a web page asking for personal information.
Avoid clicking on links or downloading attachments from suspicious emails.
Additionally, please consider using Yahoo Account Key, a simple authentication tool that eliminates the need to use a password altogether.
An increasingly connected world has come with increasingly sophisticated threats. Industry, government and users are constantly in the crosshairs of adversaries. Through strategic proactive detection initiatives and active response to unauthorized access of accounts, Yahoo will continue to strive to stay ahead of these ever-evolving online threats and to keep our users and our platforms secure.
For more information about this issue and our security resources, please visit the Yahoo Security Issue FAQs page, https://yahoo.com/security-update, which will be up beginning at 12pm (PDT).
Statements in this press release regarding the findings of Yahoo’s ongoing investigation involve potential risks and uncertainties. The final conclusions of the investigation may differ from the findings to date due to various factors including, but not limited to, the discovery of new or additional information and other developments that may arise during the course of the investigation. More information about potential risks and uncertainties of security breaches that could affect the Company’s business and financial results is included under the caption “Risk Factors” in the Company’s Quarterly Report on Form 10-Q for the quarter ended June 30, 2016, which is on file with the SEC and available on the SEC’s website at http://www.sec.gov./
By Katie Shay, Legal Counsel, Business & Human Rights
Twelve security trainers, tool developers and human rights activists from four continents came to our headquarters in Sunnyvale, California. Their mission? To share their unique perspectives with our Yahoo products, engineering, security, public policy and legal teams. Yahoo’s Business & Human Rights Program, the Paranoids and Yahoo for Good orchestrated this ‘hack of the minds’ in partnership with Internews and the USABLE Project.
USABLE Project’s aim is to inform the development of security tools that are easy to use and simple to understand for users from diverse backgrounds and skill levels. Their goal is to support vulnerable populations around the world who use the internet for more than just sharing pictures of cats or Venmoing a friend for lunch. In many cases, these users rely on the internet to exercise their right to free expression, expose corruption or fight against injustice in their communities. For these users, the ability to be secure online is critical.
In July, Yahoo was proud to sponsor the USABLE Project’s first ever public forum, UX in a High Risk World in San Francisco, bringing together frontline digital security practitioners, users, tool developers and UX experts from around the world. In addition, Yahoo participated in the final day of USABLE’s four-day closed-door workshop leading up to this event, working directly with this community to build concrete, actionable roadmaps to improve usability in security tools.
Following the forum, the delegation from USABLE that visited Yahoo shared their on-the-ground perspective on why remaining secure online is so important to their work. They explained how they use Yahoo products, including Flickr and Mail, why it’s important to have a principled approach to responding to government requests for user data and content moderation, as well as the importance of baking in security features to products from the outset by turning them on by default. These visionary leaders are working toward solutions for activists facing censorship, hacking, surveillance and suppression in some of the world’s most challenging environments.
During the delegation’s visit, our Yahoo teams asked pointed questions to understand the experience of some of our most vulnerable users and to explore how their experiences might inform Yahoo’s product development and online security work.
We are grateful to the USABLE team for sharing their stories with us, and for inspiring our teams to continue to find new and innovative ways to put our users first!
Recent headlines might lead you to believe that when a company runs a red team exercise that the red team should fail. After all, the company has invested in security teams, products and processes. So the outcome should be a win for the blue team and a failure for the red team. (For those of you who are lost already, a red team is an independent group within a company’s security organization that challenges the effectiveness of its security defenses. The red team performs analysis of systems and process gaps. Then it attacks you, hopefully before a real adversary does.) Let’s set the record straight on this critical aspect of modern security programs.
The red team always wins. Always.
It can be humiliating. And the timing is rarely convenient. Friday late night or on Christmas morning? Fair game.
The red team adopts the tools and techniques of actual adversaries. They use their understanding of attacks on other organizations that have been made public. They mimic the work of adversaries that the blue team has caught. They do not fight fair, nor will your adversaries.
Most companies prepare their defenses around best practices and compliance. Those alone will not get you very far. Even the organizations that use threat models and attack chains (i.e. the common events in an attack) need to practice. Practice. Measure. Learn. Repeat.
Most companies think they have a security plan. One of the great philosophers of our time, Mike Tyson, once remarked “Everybody has a plan until they get punched in the mouth.” Will your muscle memory kick in after getting hit? Or will you be stunned? Companies that engage in continuous red/blue battles are far more likely to detect and survive real attacks.
Having a security program without a red team is like practicing martial arts in the mirror rather than with a worthy sparring partner.
A red team exercise should not be an annual activity. It should represent a continuous clear and present danger. An employee, for example, may (incorrectly) doubt that they are the target of state-sponsored actors. They might think “Why should I close these minor gaps? It’s not like anyone would use these vulnerabilities against us!” They can, however, be sure that their red team is actively targeting them. Continuous red team exercises, over time, will give the blue team a fighting chance.
After the red team attack, what do you do? Do you “fix the glitch”? Or do you take time in the post-mortem to find the root cause and to fix it? More mature organizations will revisit the gaps over time. They provide input into the next planning cycle. Lessons learned from red team exercises contribute to a stronger defense and a better chance of stopping the real adversaries.
The real scandal is not that a red team won (the red team always wins!), but that many companies do not have red teams. Reporters: want a great story? Ask every CISO you talk to if they have a full-time, dedicated red team. Prepare yourself to hear some spin.
Unacceptable answers:
We are not the target of sophisticated adversaries.
We already know we have a lot of work to do so adding a red team report isn’t going to help.
We work in a highly regulated industry so it’s not necessary.
We have not had a breach in years.
Our attack surface is small.
Our IT team is great and we do a good job of user training.
Yahoo has its own internal red team known as Offensive Engineering (yes, that can be read two ways!). Their job is to take a contrarian view of Yahoo systems. They don’t care what the code was designed to do. They care about what it actually does. And yes, this red team always wins. Always. It’s what we pay them to do.
Let’s stop talking about red team wins as if they are a bad thing and let’s start talking about the red vs blue feedback loop: Practice. Measure. Learn. Repeat.
In our inaugural post to The Paranoid, we discussed the human element behind online attacks–the human adversary. We sought to give some perspectives as to who is behind online threats in order to better understand how to defend against them. Yahoo’s bug bounty program applies that insight in our ongoing efforts to provide a safe environment for our users. By thinking about the economics of security, we’ve found that we can tilt the advantage in our favor by partnering with industry-leading security researchers.
We often get questions from both security researchers, and people just interested in learning about how programs like these work. We thought we’d use this opportunity to take a quick look under the hood.
First, some background. Bug bounty programs essentially crowd-source security. They allow companies to improve coverage so they are able to add additional eyes where they need them. Bug bounty researchers also bring depth of expertise and different skill sets that can uncover hard to find bugs.
For the past two years, Yahoo has developed one of the largest and most successful bug bounty programs in the industry. We’ve paid out over $1.7 million dollars in bounties, resolved more than 2,000 security bugs and maintain a “hackership” of more than 2,000 researchers, some of whom make careers out of it.
Security researchers often ask us how we decide the payout associated with a given bug report. At first it might seem logical that we pay based on the type or classification of a security bug. Some bug types tend to be bad, so you might think that they would be paid the same. However, in the vast majority of cases, that’s not the complete story. So if the bug type alone is not what we use to determine the payout, what is? The missing input to the calculation is the impact of the vulnerability. We take into account what data might have been exposed, the sensitivity of that data, the role that data plays, network location and the permissions of the server involved. Those factors are of great importance.
Given the importance of the impact of a bug, the Yahoo bug bounty program does not reward researchers solely based on bug type. The type of bug a security researcher finds is mostly irrelevant. It’s what the bug allows them to do and where that are most important. What can an attacker actually do with this specific bug to potentially affect the security of Yahoo or our users? Furthermore, Yahoo’s application landscape is not necessarily uniform; certain properties or applications are more equal than others.
Here’s an example to show how these factors work in practice. SQL injection bugs are often a devastating bug class because they can provide full access to a database. Odds are, if a company has a presence on the web, they are storing sensitive information in databases. But just because an attacker can access the database does not mean it’s game over. The real reason that the SQL injection bug class can be so devastating is the data stored in the database may be accessed or changed by unauthorized parties. The typical impact of a SQL injection bug is high because the data exposed is typically sensitive, except when it’s not. What if the database doesn’t contain any sensitive data?
Part of the process in determining impact can seem opaque to the researcher, and we understand that. That obscurity is an unfortunate but necessary fact of life in a bug bounty program. As an external party, it is just not possible to have all the information. The sort of testing available to participants in a public bug bounty program is inherently “black box”–no documentation, no source code, what you see is what you get.
So we encourage bug reporters to include in their reports what they believe the impact of the vulnerability to be (example report here). Submitting a report that contains a thorough and detailed explanation of a legitimate security issue is much more highly valued and rewarded.
We also work closely with the developers to ensure the bug is fixed in a timely manner, and to obtain their expert opinion on impact if necessary. If the developers that created the application tell us that no sensitive data is stored in a particular database, we take that into consideration when awarding your bug. More detailed guidelines for our bug bounty program are available at hackerone.com/yahoo.
To paraphrase a little-known quote, “bug bounty programs don’t reward you for being clever.” Users and researchers should know that we place far more weight on how impactful bugs are to our platforms.
If the countless data breaches we read about in the news have confirmed anything, it’s that online security is somewhat of a moving target. We’ve witnessed compromised security at one point or another across every industry and government. From health records and email to financial information, intellectual property and critical infrastructure, it would seem nothing is secure these days.
Yet, despite being armed with this fundamental understanding of online security, it’s often treated as a static challenge–as if there is one solution for one vulnerability. In an inherently insecure world with ever changing threats, our conventional wisdom must evolve just as online threats do.
The obvious next question is how, and that’s a good question to ask with a plethora of answers. But in order to understand how we adapt to emerging threats, it’s first and foremost critical to understand the dynamics behind the threats themselves. Why are the threats changing and what allows them to continue to be successful?
In fact, the next best question to ask is who is behind today’s online threats. The most important aspect of online security that we can internalize is that we are up against dedicated, human adversaries who organize their activities into campaigns.
They are dedicated, which means they have a job to do, or a calling. They’re going to keep coming back until they achieve their goals. Maybe they work for a criminal syndicate, or for a foreign military. Or maybe they are on a mission from God.
They are also human, which means they can be creative and resourceful. They are like water in a cracked vase. It will find a way to seep out. They spend time learning your internal processes and reading your internal documentation before acting.
And finally, they work in campaigns. The data they seek from a system may not be valuable by itself. It may be that the data is valuable because it provides information about human rights activists in their own country. Or because they want to know what their political opponents are doing. They are likely targeting other services of peers and competitors. The data they collect is only valuable to the extent the campaign objectives are known.
Our activities as defenders, whether the casual user to the chief information security officer, need to line up against these characteristics of our adversaries. Are we considering how a phone call from an unfamiliar number but a familiar voice might be part of a social engineering scheme? Are we employing security tactics that eliminate an attack instead of letting it shift to a new vector?
Until we start thinking about online adversaries this way, we’ll continue to find ourselves playing whack-a-mole without ever turning the tide.
This is the first edition of our new Yahoo Tumblr series–The Paranoid–where we will delve into the security space and share how we’re working to protect our users, as well as useful tips for users to consider as they go about their everyday lives online. Like all good security researchers, we will look at security issues from the viewpoint of an adversary. Our goals with this series are to break conventional wisdom, ask tough questions about how we approach online security, and ultimately allow our users to hold us to a higher standard. Most importantly, we want to start a conversation to ultimately improve the safety and security of our users and our network.
We
put our users’ security first at Yahoo, and today we’re proud to
highlight one way in which we’re protecting our users against evolving
online threats through our bug bounty program. Partnering with HackerOne,
Yahoo’s bug bounty program has grown dramatically since our launch
about two years ago. Our bug bounty program boasts more than 2,000
security researchers and we’ve awarded $1.6 million in the last two
years. Our security team, known as the Paranoids, work night and day to
secure our users, but, with an online property as large as Yahoo, having
as many eyes as possible focused on the security of our users
crowd-sources what would otherwise be an impossible task for the
resources of a few.
Learn more about our growing bug bounty program here.
