Keeping fake traffic that originates from infected computers (aka “botnets”) out of our ads systems has been a priority since we launched, and over the years we've worked hard to put in place extensive technology checks and filters to safeguard against this type of traffic.

Today we're further reinforcing our existing botnet defenses across our ad systems through a new feature that automates the filtering of traffic from three of the top ad fraud botnets, amongst those we are monitoring and defending against. One of the key benefits of this new feature is that it is resilient to possible changes to the malware that generates this botnet traffic.

This move boosts our defenses against invalid traffic generated by some nasty ad fraud malware, including Bedep and two other malware families that we have code-named Beetal and Changthangi. Together these three botnets are comprised of over 500,000 infected user machines.

Today we’d like to take this opportunity to take a deeper look at ad fraud botnets.

Ad fraud botnets: a menace to the advertising ecosystem

Ad fraud botnets are armies of malware-infected computers that are controlled by malicious fraudsters intent on generating large amounts of non-human ad traffic volume, typically for unscrupulous publishers. As a result, ad fraud botnets are a major threat to the budgets of advertisers, the reputation of publishers, and the safety of consumers. And this threat is considerable, given that hundreds of thousands of computers around the globe are infected with malware used specifically for ad fraud.

The Bedep Botnet size over the course of 60 days. Dips in the graph represent weekends, when some infected machines are turned off.

Global distribution and concentration of Bedep Malware.

Botnet traffic is difficult to consistently filter in advertising platforms because malware authors try to make their fraudulent traffic look as human as possible so that it resembles legitimate traffic. For example, botnet traffic has many of the same characteristics as real traffic, including the use of common browsers, and typical user behavior on a web page (e.g., scrolling, clicking, and mouse movement).

Our move to consistently and confidently cut out the traffic from these botnets, despite any changes in the malware on which they’re based, represents a significant milestone in the defense of our advertising ecosystem.

The art and science of protecting against botnets at scale

Identifying ad fraud malware and protecting ad platforms against botnets is a sophisticated effort that requires deep technical knowledge, diligence, and the ability to think several steps ahead. It’s a game of chess against an opponent that is constantly changing the rules.

In addition, it takes robust and extensive infrastructure to properly analyze malware threats at scale. For example, there are millions of malware programs out in the wild, although not all of this malware is associated with ad fraud botnets. This scenario represents a considerable technical challenge, since the malware, along with a vast amount of botnet traffic, needs to be continuously analyzed. To compound the challenge, there are hundreds of thousands of new malware programs produced each day that our systems need to analyze as well.

Our team has expanded its expertise by working to gain a deep understanding of the Bedep, Beetal, and Changthangi malware families. Subsequently, we have expanded the capability to significantly protect our systems against traffic generated by this malware through an automated, scalable, and seamless filter. This filter is already available to all marketers on DoubleClick Bid Manager and Google Display Network (GDN).

A bold move, but there’s more to come

We believe in fighting the good fight in order to stop malicious actors in the advertising ecosystem. We also know that our success is not based solely on sophisticated algorithms or robust, highly-scalable infrastructure. Our success also relies on a team of warrior scientists that combines art and science to innovate and cultivate, relying on creativity and collective wisdom to effect change in unique ways.

This is a really exciting start to the year for us, yet we know that our work is not done yet. We will continue to be vigilant, working hard to protect our systems from fraudsters in 2016 and beyond. Stay tuned.

Posted by Andres Ferrate
Chief Advocate, Google Ad Traffic Quality