Cross-posted from The Keyword

A free and open web is a vital resource for people and businesses around the world. And ads play a key role in ensuring you have access to accurate, quality information online. But bad ads can ruin the online experience for everyone. They promote illegal products and unrealistic offers. They can trick people into sharing personal information and infect devices with harmful software. Ultimately, bad ads pose a threat to users, Google’s partners, and the sustainability of the open web itself.

We have a strict set of policies that govern the types of ads we do and don’t allow on Google in order to protect people from misleading, inappropriate, or harmful ads. And we have a team of engineers, policy experts, product managers and others who are waging a daily fight against bad actors. Over the years, this commitment has made the web a better place for you—and a worse place for those who seek to abuse advertising systems for their own gain.

In 2016, we took down 1.7 billion ads that violated our advertising policies, more than double the amount of bad ads we took down in 2015. If you spent one second taking down each of those bad ads, it’d take you more than 50 years to finish. But our technology is built to work much faster.

Last year, we did two key things to take down more bad ads. First, we expanded our policies to better protect users from misleading and predatory offers. For example, in July we introduced a policy to ban ads for payday loans, which often result in unaffordable payments and high default rates for users. In the six months since launching this policy, we disabled more than 5 million payday loan ads. Second, we beefed up our technology so we can spot and disable bad ads even faster. For example, “trick to click" ads often appear as system warnings to deceive users into clicking on them, not realizing they are often downloading harmful software or malware. In 2016, our systems detected and disabled a total of 112 million ads for “trick to click,” 6X more than in 2015.

Here are a few more examples of bad ads we took action against in 2016:

Ads for illegal products

Some of the most common bad ads we find online are ads promoting illegal activities or products. Although we've long had a policy against bad ads for pharmaceuticals, last year our systems detected an increase online. We disabled more than 68 million bad ads for healthcare violations, up from 12.5 million in 2015.

Similarly, we saw more attempts to advertise gambling-related promotions without proper authorization from regulators in the countries they operate. We took down more than 17 million bad ads for illegal gambling violations in 2016.

17M ads removed for illegal gambling violations

Misleading ads

We don't want you to feel misled by ads that we deliver, so we require our advertisers to provide upfront information for people to make informed decisions. Some ads try to drive clicks and views by intentionally misleading people with false information like asking, “Are you at risk for this rare, skin-eating disease?” or offering miracle cures like a pill that will help you lose 50 pounds in three days without lifting a finger. In 2016, we took down nearly 80 million bad ads for deceiving, misleading and shocking users.

1,300+ accounts suspended for tabloid cloaking

Bad ads on mobile

If you’ve ever been on your phone and suddenly, without warning, ended up in the app store downloading an app you’ve never heard of, a “self-clicking ad” could be to blame. In 2015, we disabled only a few thousand of these bad ads, but in 2016, our systems detected and disabled more than 23,000 self-clicking ads on our platforms, a huge increase year over year.

Ads trying to game the system

Bad actors know that ads for certain products—like weight-loss supplements or payday loans—aren’t allowed by Google's policies, so they try to trick our systems into letting them through. Last year, we took down almost 7 million bad ads for intentionally attempting to trick our detection systems.

In 2016, we saw the rise of tabloid cloakers, a new type of scammer that tries to game our system by pretending to be news. Cloakers often take advantage of timely topics—a government election, a trending news story or a popular celebrity—and their ads can look like headlines on a news website. But when people click on that story about Ellen DeGeneres and aliens, they go to a site selling weight-loss products, not a news story.

To fight cloakers, we take down the scammers themselves, and prevent them from advertising with us again. In 2016, we suspended more than 1,300 accounts for tabloid cloaking. Unfortunately, this type of bad ad is gaining in popularity because people are clicking on them. And a handful of scammers can pump out a lot of bad ads: During a single sweep for tabloid cloaking in December 2016, we took down 22 cloakers that were responsible for ads seen more than 20 million times by people online in a single week.

Promoting and profiting from bad sites

When we find ads that violate our policies, we block the ad or the advertiser, depending on the violation. But sometimes we also need to suspend the website promoted in the ad (the site people see after they click on it). So, for example, while we disabled more than 5 million payday loan ads last year, we also took action on 8,000 sites promoting payday loans.

Here are some examples of common policy violations we saw among bad sites in 2016:

• We took action on 47,000 sites for promoting content and products related to weight-loss scams.
• We took action on more than 15,000 sites for unwanted software and disabled 900,000 ads for containing malware.
• And we suspended around 6,000 sites and 6,000 accounts for attempting to advertise counterfeit goods, like imitation designer watches.

6,000 sites and 6,000 accounts removed for attempting to sell counterfeit goods

Publishers and website owners use our AdSense platform to make money by running ads on their sites and content, so we have strict policies in place to keep Google's content and search networks safe and clean for our advertisers, users and publishers. When a publisher violates our policies, we may stop showing ads on their site, or even terminate their account.

We've had long-standing policies prohibiting AdSense publishers from running ads on sites that help people deceive others, like a site where you buy fake diplomas or plagiarized term papers. In November, we expanded on these policies, introducing a new AdSense misrepresentative content policy, that helps us to take action against website owners misrepresenting who they are and that deceive people with their content. From November to December 2016, we reviewed 550 sites that were suspected of misrepresenting content to users, including impersonating news organizations. We took action against 340 of them for violating our policies, both misrepresentation and other offenses, and nearly 200 publishers were kicked out of our network permanently.

In addition to all the above, we support industry efforts like the Coalition for Better Ads to protect people from bad experiences across the web. While we took down more bad ads in 2016 than ever before, the battle doesn’t end here. As we invest in better detection, the scammers invest in more elaborate attempts to trick our systems. Continuing to find and fight them is essential to protecting people online and ensuring you get the very best from the open web.

Posted by Scott Spencer
Director of Product Management, Sustainable Ads

In the coming quarters, all major browsers, including Chrome, are phasing out the use of Flash technologies in favor of HTML5. HTML5 is not only available on more devices, but also offers improved security, reduced power consumption and faster page load times for users.

We began our transition to HTML5 with display ads across Google and DoubleClick back in 2015. We are now continuing that transition by shifting video ads in DoubleClick Digital Marketing, DoubleClick for Publishers, DoubleClick Ad Exchange and the Google Display Network to HTML5 over the next few quarters as follows:

• Starting April 3rd, 2017, new Flash video ads will no longer be able to be uploaded into DoubleClick Studio, DoubleClick Campaign Manager, DoubleClick Bid Manager, DoubleClick for Publishers or AdWords.
• Starting July 3rd, 2017, Flash video ads will no longer be able to run through DoubleClick Campaign Manager, DoubleClick Bid Manager, DoubleClick Ad Exchange, DoubleClick for Publishers or AdWords. Additionally, our Active View and Verification tools for video will no longer use Flash.

Transition timeline for HTML5 Video

It’s important to begin updating your ads and websites to HTML5 technologies in preparation for these dates. We fully support HTML5 Video across DoubleClick and AdWords and provide the tools to ensure advertisers and publishers can easily migrate all video ads to HTML5.

For guidance and best practices to help your team with this transition, see this Chrome one-sheeter, visit the DoubleClick help center or contact your DoubleClick sales representative.

Posted by Peentoo Patel and Sunil Gupta

Keeping fake traffic that originates from infected computers (aka “botnets”) out of our ads systems has been a priority since we launched, and over the years we've worked hard to put in place extensive technology checks and filters to safeguard against this type of traffic.

Today we're further reinforcing our existing botnet defenses across our ad systems through a new feature that automates the filtering of traffic from three of the top ad fraud botnets, amongst those we are monitoring and defending against. One of the key benefits of this new feature is that it is resilient to possible changes to the malware that generates this botnet traffic.

This move boosts our defenses against invalid traffic generated by some nasty ad fraud malware, including Bedep and two other malware families that we have code-named Beetal and Changthangi. Together these three botnets are comprised of over 500,000 infected user machines.

Today we’d like to take this opportunity to take a deeper look at ad fraud botnets.

Ad fraud botnets: a menace to the advertising ecosystem

Ad fraud botnets are armies of malware-infected computers that are controlled by malicious fraudsters intent on generating large amounts of non-human ad traffic volume, typically for unscrupulous publishers. As a result, ad fraud botnets are a major threat to the budgets of advertisers, the reputation of publishers, and the safety of consumers. And this threat is considerable, given that hundreds of thousands of computers around the globe are infected with malware used specifically for ad fraud.

The Bedep Botnet size over the course of 60 days. Dips in the graph represent weekends, when some infected machines are turned off.
Global distribution and concentration of Bedep Malware.

Botnet traffic is difficult to consistently filter in advertising platforms because malware authors try to make their fraudulent traffic look as human as possible so that it resembles legitimate traffic. For example, botnet traffic has many of the same characteristics as real traffic, including the use of common browsers, and typical user behavior on a web page (e.g., scrolling, clicking, and mouse movement).

Our move to consistently and confidently cut out the traffic from these botnets, despite any changes in the malware on which they’re based, represents a significant milestone in the defense of our advertising ecosystem.

The art and science of protecting against botnets at scale

Identifying ad fraud malware and protecting ad platforms against botnets is a sophisticated effort that requires deep technical knowledge, diligence, and the ability to think several steps ahead. It’s a game of chess against an opponent that is constantly changing the rules.

In addition, it takes robust and extensive infrastructure to properly analyze malware threats at scale. For example, there are millions of malware programs out in the wild, although not all of this malware is associated with ad fraud botnets. This scenario represents a considerable technical challenge, since the malware, along with a vast amount of botnet traffic, needs to be continuously analyzed. To compound the challenge, there are hundreds of thousands of new malware programs produced each day that our systems need to analyze as well.

Our team has expanded its expertise by working to gain a deep understanding of the Bedep, Beetal, and Changthangi malware families. Subsequently, we have expanded the capability to significantly protect our systems against traffic generated by this malware through an automated, scalable, and seamless filter. This filter is already available to all marketers on DoubleClick Bid Manager and Google Display Network (GDN).

A bold move, but there’s more to come

We believe in fighting the good fight in order to stop malicious actors in the advertising ecosystem. We also know that our success is not based solely on sophisticated algorithms or robust, highly-scalable infrastructure. Our success also relies on a team of warrior scientists that combines art and science to innovate and cultivate, relying on creativity and collective wisdom to effect change in unique ways.

This is a really exciting start to the year for us, yet we know that our work is not done yet. We will continue to be vigilant, working hard to protect our systems from fraudsters in 2016 and beyond. Stay tuned.

Posted by Andres Ferrate Chief Advocate, Google Ad Traffic Quality

Cross-posted from the Official Google Blog

When ads are good, they connect you to products or services you’re interested in and make it easier to get stuff you want. They also keep a lot of what you love about the web—like news sites or mobile apps—free.

But some ads are just plain bad—like ads that carry malware, cover up content you’re trying to see, or promote fake goods. Bad ads can ruin your entire online experience, a problem we take very seriously. That’s why we have a strict set of policies for the kinds of ads businesses can run with Google—and why we’ve invested in sophisticated technology and a global team of 1,000+ people dedicated to fighting bad ads. Last year alone we disabled more than 780 million ads for violating our policies—a number that's increased over the years thanks to new protections we've put in place. If you spent one second looking at each of these ads, it’d take you nearly 25 years to see them all!

Here are some of the top areas we focused on in our fight against bad ads in 2015:

Busting bad ads

Some bad ads, like those for products that falsely claim to help with weight loss, mislead people. Others help fraudsters carry out scams, like those that lead to “phishing” sites that trick people into handing over personal information. Through a combination of computer algorithms and people at Google reviewing ads, we’re able to block the vast majority of these bad ads before they ever get shown. Here are some types of bad ads we busted in 2015:

Counterfeiters

We suspended more than 10,000 sites and 18,000 accounts for attempting to sell counterfeit goods (like imitation designer watches).

Pharmaceuticals

We blocked more than 12.5 million ads that violated our healthcare and medicines policy, such as ads for pharmaceuticals that weren’t approved for use or that made misleading claims to be as effective as prescription drugs.

Weight loss scams

Weight loss scams, like ads for supplements promising impossible-to-achieve weight loss without diet or exercise, were one of the top user complaints in 2015. We responded by suspending more than 30,000 sites for misleading claims.

Phishing

In 2015, we stepped up our efforts to fight phishing sites, blocking nearly 7,000 sites as a result.

Unwanted software

Unwanted software can slow your devices down or unexpectedly change your homepage and keep you from changing it back. With powerful new protections, we disabled more than 10,000 sites offering unwanted software, and reduced unwanted downloads via Google ads by more than 99 percent.

Trick to click

We got even tougher on ads that mislead or trick people into interacting with them—like ads designed to look like system warnings from your computer. In 2015 alone we rejected more than 17 million.

Creating a better experience

Sometimes even ads that offer helpful and relevant information behave in ways that can be really annoying—covering up what you’re trying to see or sending you to an advertiser’s site when you didn’t intend to go there. In 2015, we disabled or banned the worst offenders.

Accidental mobile clicks

We’ve all been there. You’re swiping through a slideshow of the best moments from the Presidential debate when an ad redirects you even though you didn’t mean to click on it. We’re working to end that. We've developed technology to determine when clicks on mobile ads are accidental. Instead of sending you off to an advertiser page you didn't mean to visit, we let you continue enjoying your slideshow (and the advertiser doesn't get charged).

Bad sites and apps

In 2015, we stopped showing ads on more than 25,000 mobile apps because the developers didn’t follow our policies. More than two-thirds of these violations were for practices like mobile ads placed very close to buttons, causing someone to accidentally click the ad. There are also some sites and apps that we choose not to work with because they don’t follow our policies. We also reject applications from sites and mobile apps that want to show Google ads but don't follow our policies. In 2015 alone, we rejected more than 1.4 million applications.

Putting you in control

We also give you tools to control the type of ads you see. You can always let us know when you believe an ad might be violating our policies.

Mute This Ad

Maybe you’ve just seen way too many car ads recently. “Mute This Ad” lets you click an “X” at the top on many of the ads we show and Google will stop showing you that ad and others like it from that advertiser. You can also tell us why. The 4+ billion pieces of feedback we received in 2015 are helping us show better ads and shape our policies.

Ads Settings

In 2015, we rolled out a new design for our Ads Settings where you can manage your ads experience. You can update your interests to make the ads you see more relevant, or block specific advertisers all together.

Looking ahead to 2016

We’re always updating our technology and our policies based on your feedback—and working to stay one step ahead of the fraudsters. In 2016, we’re planning updates like further restricting what can be advertised as effective for weight loss, and adding new protections against malware and bots. We want to make sure all the ads you see are helpful and welcome and we’ll keep fighting to make that a reality.

Posted by Sridhar Ramaswamy SVP, Ads & Commerce

As we've stated before, we're committed to keeping fraudsters out of the broader advertising ecosystem. Today we're looking at a specific aspect of this fight: the false representation of domains in ad inventory.

Deceptive ad inventory

Imagine ordering a designer handbag, only to receive a cheap imitation. Unfortunately, this type of misrepresentation happens all too often with ad inventory.

The false representation of domains in ad inventory occurs when publishers intentionally make it look like their traffic is coming from another website (usually a well-known, premium website) in order to charge higher rates for ads. This practice deceives advertisers who end up paying to appear on sites with which they may not want to be associated, and harms legitimate publishers, who aren't actually receiving the funds from ads sold in their name.

The two examples below illustrate how a branded ad can end up on a sketchy website through false representation of a domain.

Protecting against falsely represented inventory in DoubleClick Bid Manager

As part of our commitment to strengthening the integrity of our digital marketing solutions, we've added a new feature to DoubleClick Bid Manager that blocks many cases of domain misrepresentation, one of the most severe types of falsely represented ad inventory.

When a source of fraudulently misrepresented domain information is identified in Bid Manager, a filter is used to exclude invalid inventory with a high degree of confidence before advertisers bid on it, regardless of exchange or reported domain. We’ve discovered that in some instances this type of activity has accounted for up to 40% of inventory for a particular exchange.

Percent of misrepresented ad domains by exchange¹

As a recent example, we noticed a publisher attempting to sell ad inventory on a pirated movie sharing website that was falsely represented with the domain name of a well-known newspaper. Fortunately, our new filter prevented ads from being purchased and displayed, thereby safeguarding advertisers from fraud and preventing this copyright infringing publisher from receiving advertising revenue.

Publishers benefit as well

It’s not just advertisers who will benefit from this new feature. The filter ensures that high quality inventory from top publishers is correctly valued and better defended against misrepresentation. As a case in point, we found that 10% of inventory offered for sale claiming to come from two popular US newspapers was in fact from other publishers falsely representing themselves as the two newspapers.

We also recognize that there are valid use cases for selling inventory via alternate domains, which is why we have taken great care to ensure that our filter targets only cases where the false representation masks the real value of the inventory.

Available to all marketers on Bid Manager

We continually look for new ways to improve and defend our advertising platforms against ad fraud, and we’re proud to offer this new feature directly on Bid Manager, without the need for advertisers to maintain blacklists or configuration settings. We’re happy to further protect advertisers and legitimate publishers by giving the boot to pretenders offering bad inventory.

Posted by Andres Ferrate Chief Advocate, Google Ad Traffic Quality

^{1 Includes only exchanges from which DoubleClick Bid Manager buys >1 million impressions per day}

Last week, the Trustworthy Accountability Group (TAG) announced the “Verified by TAG” initiative to help increase transparency of digital advertising transactions across the industry. We’re fully supportive of both programs outlined in TAG’s announcement and we’re currently in the process of applying for TAG Registration. To support the adoption of Payment IDs across the ecosystem, starting today our version of Payment IDs is available in DoubleClick Ad Exchange to all buyers globally.

Currently, if a programmatic buyer finds they’ve bought fraudulent inventory, there is no way to directly identify the supply source responsible for the fraud. The Payment ID system we proposed to the TAG Anti-Fraud working group fixes this problem by asking all supply sources (e.g. ad exchanges, ad networks, supply side platforms) of advertising inventory to create and provide unique and persistent anonymous identifiers that link every impression to who is paid in their accounting systems. If a buyer finds invalid activity from any source in their supply chain, these Payment IDs will help the buyer to identify who is responsible and blacklist those suppliers from their campaigns.

We’ve always invested heavily to keep DoubleClick Ad Exchange free of invalid activity and ensure that money spent on our platform only goes to support legitimate publishers, app developers, and content creators. To show our commitment to a better ads ecosystem, accelerate the adoption of Payment IDs, and help DSPs start integrating them, we’ve implemented the standard as it exists today, and we’ll continue to work closely with TAG and others in the industry to formalize an industry-wide Payment ID program. When the TAG Anti-Fraud Working Group has finalized the broader industry standard, we’ll happily make any changes to ensure we are compliant with TAG’s efforts.

"Google has been at the forefront of the fight against digital ad fraud, and this announcement advances our work together to develop an industry-wide Payment ID system. We look forward to continued collaboration with Google and other programmatic leaders through the TAG Anti-Fraud Working Group to create a fully transparent digital ad supply chain that will expose the bad actors and cut off their financial support."
Mike Zaneis, CEO, TAG

Leading programmatic buyers, DoubleClick Bid Manager, Dstillery, Magnetic, MediaMath, Rocket Fuel, The Trade Desk, and Turn have all committed to integrating Payment IDs into their systems in the coming months.

Posted by:

Vegard Johnsen
Product Manager, Google Ads Traffic Quality
Chetna Bindra
Product Manager, DoubleClick Ad Exchange

For the last few months, we’ve been raising awareness of the ad injection economy, showing how unwanted ad injectors can hurt user experience, jeopardize user security, and generate significant volumes of unwanted ads. We’ve used learnings from our research to prevent and remove unwanted ad injectors from Google services and improve our policies and technologies to make it more difficult to spread this unwanted software.

Today, we’re announcing a new measure to remove injected ads from the advertising ecosystem, including an automated filter in DoubleClick Bid Manager that removes impressions generated by ad injectors before any bid is made.

Unwanted ad injectors: disliked by users, advertisers, and publishers

Unwanted ad injectors are programs that insert new ads, or replace existing ones, in the pages users visit while browsing the web. Unwanted ad injectors aren’t part of a healthy ads ecosystem. They’re part of an environment where bad practices hurt users, advertisers, and publishers alike.

We’ve received almost 300,000 user complaints about them in Chrome since the beginning of 2015—more than any other issue, and it’s no wonder. Ad injectors affect all sites equally. You wouldn’t be happy if you tried to get the morning news and saw this:

Not only are they intrusive, but people are often tricked into installing them in the first place, via deceptive advertising, or software “bundles.” Ad injection can also be a security risk, as the recent “Superfish” incident showed.

Ad injectors are problematic for advertisers and publishers as well. Advertisers often don’t know their ads are being injected, which means they don’t have any idea where their ads are running. Publishers, meanwhile, aren’t being compensated for these ads, and more importantly, they unknowingly may be putting their visitors in harm’s way, via spam or malware in the injected ads.

Removing injected inventory from advertising

Earlier this quarter, we launched an automated filter on DoubleClick Bid Manager to prevent advertisers from buying injected ads across the web. This new system detects ad injection and proactively creates a blacklist that prevents our systems from bidding on injected inventory. Advertisers and agencies using our platforms are already protected. No adjustments are needed. No settings to change.

We currently blacklist 1.4% of the inventory accessed by DoubleClick Bid Manager across exchanges. However, we’ve found this percentage varies widely by provider. Below is a breakdown showing the filtered percentages across some of the largest exchanges:

We’ve always enforced policies against the sale of injected inventory on our ads platforms, including the DoubleClick Ad Exchange. Now advertisers using DoubleClick Bid Manager can avoid injected inventory across the web.

No more injected ads?

We don’t expect the steps we’ve outlined above to solve the problem overnight, but we hope others across the industry take action to cut ad injectors out of advertising. With the tangle of different businesses involved—knowingly, or unknowingly—in the ad injector ecosystem, progress will only be made if we all work together. We strongly encourage all members of the ads ecosystem to review their policies and practices and take actions to tackle this issue.

Posted by Vegard Johnsen Product Manager, Google Ads Traffic Quality

Today the Trustworthy Accountability Group (TAG) announced a new pilot blacklist to protect advertisers across the industry. This blacklist comprises data-center IP addresses associated with non-human ad requests. We're happy to support this effort along with other industry leaders—Dstillery, Facebook, MediaMath, Quantcast, Rubicon Project, The Trade Desk, TubeMogul and Yahoo—and contribute our own data-center blacklist. As mentioned to Ad Age and in our recent call to action, we believe that if we work together we can raise the fraud-fighting bar for the whole industry.

Data-center traffic is one of many types of non-human or illegitimate ad traffic. The newly shared blacklist identifies web robots or “bots” that are being run in data centers but that avoid detection by the IAB/ABC International Spiders & Bots List. Well-behaved bots announce that they're bots as they surf the web by including a bot identifier in their declared User-Agent strings. The bots filtered by this new blacklist are different. They masquerade as human visitors by using User-Agent strings that are indistinguishable from those of typical web browsers.

In this post, we take a closer look at a few examples of data-center traffic to show why it’s so important to filter this traffic across the industry.

Impact of the data-center blacklist

When observing the traffic generated by the IP addresses in the newly shared blacklist, we found significantly distorted click metrics. In May of 2015 on DoubleClick Campaign Manager alone, we found the blacklist filtered 8.9% of all clicks. Without filtering these clicks from campaign metrics, advertiser click-through rates would have been incorrect and for some advertisers this error would have been very large.

Below is a plot that shows how much click-through rates in May would have been inflated across the most impacted of DoubleClick Campaign Manager’s larger advertisers.

Two examples of bad data-center traffic

There are two distinct types of invalid data-center traffic: where the intent is malicious and where the impact on advertisers is accidental. In this section we consider two interesting examples where we’ve observed traffic that was likely generated with malicious intent.

Publishers use many different strategies to increase the traffic to their sites. Unfortunately, some are willing to use any means necessary to do so. In our investigations we’ve seen instances where publishers have been running software tools in data centers to intentionally mislead advertisers with fake impressions and fake clicks.

First example

UrlSpirit is just one example of software that some unscrupulous publishers have been using to collaboratively drive automated traffic to their websites. Participating publishers install the UrlSpirit application on Windows machines and they each submit up to three URLs through the application’s interface. Submitted URLs are then distributed to other installed instances of the application, where Internet Explorer is used to automatically visit the list of target URLs. Publishers who have not installed the application can also leverage the network of installations by paying a fee.

At the end of May more than 82% of the UrlSpirit installations were being run on machines in data centers. There were more than 6,500 data-center installations of UrlSpirit, with each data-center installation running in a separate virtual machine. In aggregate, the data-center installations of UrlSpirit were generating a monthly rate of at least half a billion ad requests— an average of 2,500 fraudulent ad requests per installation per day.

Second Example

HitLeap is another example of software that some publishers are using to collaboratively drive automated traffic to their websites. The software also runs on Windows machines, and each instance uses the Chromium Embedded Framework to automatically browse the websites of participating publishers—rather than using Internet Explorer.

Before publishers can use the network of installations to drive traffic to their websites, they need browsing minutes. Participating publishers earn browsing minutes by running the application on their computers. Alternatively, they can simply buy browsing minutes—with bundles starting at $9 for 10,000 minutes or up to 1,000,000 minutes for $625. 

Publishers can specify as many target URLs as they like. The number of visits they receive from the network of installations is a function of how long they want the network of bots to spend on their sites. For example, ten browsing minutes will get a publisher five visits if the publisher requests two-minute visit durations.

In mid-June, at least 4,800 HitLeap installations were being run in virtual machines in data centers, with a unique IP associated with each HitLeap installation. The data-center installations of HitLeap made up 16% of the total HitLeap network, which was substantially larger than the UrlSpirit network.

In aggregate the data-center installations of HitLeap were generating a monthly rate of at least a billion fraudulent ad requests—or an average of 1,600 ad requests per installation per day.

Not only were these publishers collectively responsible for billions of automated ad requests, but their websites were also often extremely deceptive. For example, of the top ten webpages visited by HitLeap bots in June, nine of these included hidden ad slots -- meaning that not only was the traffic fake, but the ads couldn’t have been seen even if they had been legitimate human visitors. 

http://vedgre.com/7/gg.html is illustrative of these nine webpages with hidden ad slots. The webpage has no visible content other than a single 300×250px ad. This visible ad is actually in a 300×250px iframe that includes two ads, the second of which is hidden. Additionally, there are also twenty-seven 0×0px hidden iframes on this page with each hidden iframe including two ad slots. In total there are fifty-five hidden ads on this page and one visible ad. Finally, the ads served on http://vedgre.com/7/gg.html appear to advertisers as though they have been served on legitimate websites like indiatimes.com, scotsman.com, autotrader.co.uk, allrecipes.com, dictionary.com and nypost.com, because the tags used on http://vedgre.com/7/gg.html to request the ad creatives have been deliberately spoofed.

An example of collateral damage

Unlike the traffic described above, there is also automated data-center traffic that impacts advertising campaigns but that hasn’t been generated for malicious purposes. An interesting example of this is an advertising competitive intelligence company that is generating a large volume of undeclared non-human traffic.

This company uses bots to scrape the web to find out which ad creatives are being served on which websites and at what scale. The company’s scrapers also click ad creatives to analyze the landing page destinations. To provide its clients with the most accurate possible intelligence, this company’s scrapers operate at extraordinary scale and they also do so without including bot identifiers in their User-Agent strings.

While the aim of this company is not to cause advertisers to pay for fake traffic, the company’s scrapers do waste advertiser spend. They not only generate non-human impressions; they also distort the metrics that advertisers use to evaluate campaign performance—in particular, click metrics. Looking at the data across DoubleClick Campaign Manager this company’s scrapers were responsible for 65% of the automated data-center clicks recorded in the month of May.

Going forward

Google has always invested to prevent this and other types of invalid traffic from entering our ad platforms. By contributing our data-center blacklist to TAG, we hope to help others in the industry protect themselves. 

We’re excited by the collaborative spirit we’ve seen working with other industry leaders on this initiative. This is an important, early step toward tackling fraudulent and illegitimate inventory across the industry and we look forward to sharing more in the future. By pooling our collective efforts and working with industry bodies, we can create strong defenses against those looking to take advantage of our ecosystem. We look forward to working with the TAG Anti-fraud working group to turn this pilot program into an industry-wide tool.

Posted by Vegard Johnsen, Product Manager Google Ad Traffic Quality

A lot of ink has recently poured onto the subject of digital advertising fraud—which is a great thing. Fraud is a real and serious problem, but some, we think, still hold a mental image of fraudsters as one-off bad actors sitting in a dark room racking up clicks on ads on their site to make a few extra bucks. The truth is far more troubling: the majority of ad fraud today is perpetrated by sophisticated organizations that devote vast resources to build and operate large scale botnets run on hijacked devices, to reap multi-million dollar payouts [1,2].

Stopping these bad actors requires an industry-wide, long term commitment to identifying and filtering fake traffic from the ecosystem. This is not a task any one company can take on alone. We need everyone across the industry to take steps toward making digital advertising more secure and transparent. Here are some actions we’re taking to help move the entire industry forward. (We hope others join us.)

Describing threats in common, precise language

Many of the statistics and headline-grabbing disclosures in the market today do a great job of creating panic, but share very little detail to help anyone actually solve the problem.

Imagine if police officers looking for a bank robber could only describe the criminal as “suspicious”. The robber would be free for life. And yet this is disappointingly how advertising fraud is policed today. “Fraud” and “suspicious” are seen as synonymous and applied to everything from completely legitimate ad impressions to fake traffic generated by zombie PCs infected with malware. Before we can stop advertising fraud, everyone needs to start using common, precise language to disclose fraudulent activity.

The IAB introduced its Anti-Fraud Principles and Proposed Taxonomy last September providing the industry with this common language and we strongly support these standards. But these are early steps – as an industry we can’t stop there. When fraud is identified it should be shared in a clear structured threat disclosure, mirroring how security researchers release security vulnerabilities. By increasing the amount of data we share in a transparent, helpful way, others in the industry will be able to corroborate any claims being made, remove the threat from their systems, removing it from the ecosystem. Further, if a public disclosure could lead to further damage, then vulnerable parties should be notified in advance.

Ensuring bad actors can't hide: Supplier Identifiers

If you bought a designer scarf in a store only to find out it’s a knock-off with a fake label, you’d expect a refund. You’d also know which store to avoid in the future. The same should hold true for fraudulent inventory. When fraud is identified, it should also be possible to identify the seller or reseller who should take responsibility for the inventory. 

Today this doesn’t hold true. As an illustration of the problem, we are currently finding significant volumes of inventory misrepresenting where the ads will actually appear and in many instances there is no reliable and verifiable mechanism to identify who in the supply chain is responsible for this misrepresented inventory.

To address this problem, we propose that the buyer of any branded (non-blind) impression should be passed a chain of unique supplier identifiers, one for each and every reseller (exchange, network, sell-side platform) and one for the publisher. With this full chain of identifiers for each impression, buyers can establish which supply paths for inventory can be trusted and which cannot. If a buyer finds a potential issue, and it’s clear where the problem lies in the supply path, then there should be an unambiguous process for refunds. It will also be easy to avoid this supply path in the future.

Ultimately the burden for ensuring the quality of online inventory starts with those who sell it. To this end, we submitted a proposal to create an industry managed supplier identifier to the IAB Anti-Fraud Working Group in February, and we’ve heard others in the industry support this call for more transparency. We've come to take this type of guarantee for granted when we shop in a store – let's work together and make it a standard for digital advertising as well.

Cleaning up campaign metrics

Before investing your hard-earned money in a local business, you’d definitely review their financial reports to understand if it’s a good investment or not. In digital, campaign metrics are the record of truth. They help advertisers evaluate which inventory sources provide the greatest value and outline a roadmap of where ad spend should be invested. But if these metrics are polluted with fake and fraudulent activity, it’s impossible to know which inventory sources provide the best return on spend.

Now, imagine if you invested in that small business only to find out it was actually a fictional front created by an organized crime ring, complete with receipts and a cashier, to cover up their back office money laundering operation. Fraudsters work hard to disguise their bot traffic as being human by having them do things like go window shopping or plan a vacation to create a whole world of made-up conversions and interactions before directing them to their final destination.

As long as fake traffic still appears to be delivering value, advertisers’ spend will continue flowing to the operators of fake traffic sources. Of course our industry should push for 100% fraud free ecosystem. The reality, though, is that some will likely always slip through. When it does, it's also our responsibility to keep it from skewing marketers' metrics. If we can keep reporting systems from giving credit to fake traffic, this removes the incentive for publishers to buy this bad traffic from bad actors.

As an industry, we owe it to our clients and ourselves to ensure that metrics are clean and accurate. Let’s work together to identify fraudulent traffic and invest in systems to filter it out of campaign metrics. 

A fraud-free ecosystem?

Advertising fraud is a real and serious problem, one that creates significant costs for advertisers, takes revenue from legitimate publishers, and enables the spread of malware to users, among other harms. To eliminate it, we must take action to remove the incentive for bad actors to create and sell fraudulent traffic. The steps I’ve outlined above seek to do this by cutting off their access to advertising spend and making it difficult for fraudsters to hide.

Over the coming months, we’ll be taking these steps and working with the industry to help others clean bad traffic from the ecosystem. 

Posted by Vegard Johnsen, Product Manager Google Ad Traffic Quality

Cross-posted from the Google Online Security Blog

It’s pretty tough to read the New York Times under these circumstances:

And it’s pretty unpleasant to shop for a Nexus 6 on a search results page that looks like this:

The browsers in the screenshots above have been infected with ‘ad injectors’. Ad injectors are programs that insert new ads, or replace existing ones, into the pages you visit while browsing the web. We’ve received more than 100,000 complaints from Chrome users about ad injection since the beginning of 2015—more than network errors, performance problems, or any other issue.

Injectors are yet another symptom of “unwanted software”—programs that are deceptive, difficult to remove, secretly bundled with other downloads, and have other bad qualities. We’ve made several recent announcements about our work to fight unwanted software via Safe Browsing, and now we’re sharing some updates on our efforts to protect you from injectors as well.

Unwanted ad injectors: disliked by users, advertisers, and publishers

Unwanted ad injectors aren’t part of a healthy ads ecosystem. They’re part of an environment where bad practices hurt users, advertisers, and publishers alike.

People don’t like ad injectors for several reasons: not only are they intrusive, but people are often tricked into installing ad injectors in the first place, via deceptive advertising, or software “bundles.” Ad injection can also be a security risk, as the recent “Superfish” incident showed.

But, ad injectors are problematic for advertisers and publishers as well. Advertisers often don’t know their ads are being injected, which means they don’t have any idea where their ads are running. Publishers, meanwhile, aren’t being compensated for these ads, and more importantly, they unknowingly may be putting their visitors in harm’s way, via spam or malware in the injected ads.

How Google fights unwanted ad injectors

We have a variety of policies that either limit or entirely prohibit, ad injectors.

In Chrome, any extension hosted in the Chrome Web Store must comply with the Developer Program Policies. These require that extensions have a narrow and easy-to-understand purpose. We don’t ban injectors altogether—if they want to, people can still choose to install injectors that clearly disclose what they do—but injectors that sneak ads into a user’s browser would certainly violate our policies. We show people familiar red warnings when they are about to download software that is deceptive, or doesn’t use the right APIs to interact with browsers.

On the ads side, AdWords advertisers with software downloads hosted on their site, or linked to from their site, must comply with our Unwanted Software Policy. Additionally, both Google Platforms program policies and the DoubleClick Ad Exchange (AdX) Seller Program Guidelines, don’t allow programs that overlay ad space on a given site without permission of the site owner.

To increase awareness about ad injectors and the scale of this issue, we’ll be releasing new research on May 1 that examines the ad injector ecosystem in depth. The study, conducted with researchers at University of California Berkeley, drew conclusions from more than 100 million pageviews of Google sites across Chrome, Firefox, and Internet Explorer on various operating systems, globally. It’s not a pretty picture. Here’s a sample of the findings:

• Ad injectors were detected on all operating systems (Mac and Windows), and web browsers (Chrome, Firefox, IE) that were included in our test.
• More than 5% of people visiting Google sites have at least one ad injector installed. Within that group, half have at least two injectors installed, nearly one-third have at least four installed.
• Thirty-four percent of Chrome extensions injecting ads were classified as outright malware.
• Researchers found 192 deceptive Chrome extensions that affected 14 million users; these have since been disabled. Google now uses the techniques we used to catch these extensions to scan all new and updated extensions.

We’re constantly working to improve our product policies to protect people online. We encourage others to do the same. We’re committed to continuing to improve this experience for Google and the web as a whole.

Posted by Nav Jagpal, Software Engineer, Safe Browsing

Ad fraud is a serious threat to the advertising ecosystem. It’s one we’ve been deeply committed to solving over the years, investing significantly in technology and expertise to keep bad ads and bad traffic out of our ads systems. Today we’re adding to those investments with a new feature in DoubleClick Bid Manager that automatically prevents advertisers from buying hidden ad slots, built from our spider.io technology.

What is a hidden ad?
Many in the industry are familiar with ad viewability, which is a measure of whether an ad was actually shown on a screen. The vast majority of non-viewable ad impressions are legitimate ads that are intended to be seen by a user, but were not viewed due to various ways people interact with content on the web. Products like Active View help advertisers and publishers address this by giving actionable reporting on ad viewability.

However, some bad actors deliberately hide ads to boost their ad impression numbers, resulting in advertisers paying for ads that have no chance of ever being seen. Below are some of many different methods employed by bad actors who create these “hidden” ads, one type of fraud identified in the IAB’s Anti-Fraud Principles and Proposed Taxonomy:

Bad actors often create sites and stack multiple ads in a single ad slot (like a pile of magazines), where only the top ad is visible. Or, they may adjust the styling of page content to make ads completely invisible. The typical approach, however, is to create a very small iframe to serve ads into that’s impossible for a user to see.

Even worse, some bad actors create adware that can inject hidden ads into a web page, without the publisher even realizing it:

What we’re doing
Practices like these have always been against our policies on the DoubleClick Ad Exchange. Thanks to the technology we’ve been investing in, we can detect this practice across the web. Our systems proactively blacklist suppliers of hidden ads, filtering them before they’re ever bid on, so advertisers won’t buy hidden ads.

Customers of DoubleClick Bid Manager don't have to make any changes to benefit from these new defenses against hidden ad slots. We currently blacklist 2.6% of the inventory accessed by DoubleClick Bid Manager across exchanges. However, we’ve found this percentage varies widely by provider. Below is a breakdown showing the filtered percentages across some of the largest exchanges:

It doesn’t stop here. Ad fraud is perpetrated by organized groups that constantly change tactics to defraud the industry for their own gains. That’s why we’re always researching and updating our defenses to ensure advertisers are getting the media they intend to purchase. Stay tuned for more updates as we continue refining our tools to promote a healthy, safe advertising ecosystem.


Posted by Payam Shodjai, Group Product Manager

Advertising helps fund the digital world we love today -- inspiring videos, informative websites, entertaining apps and services that connect us with friends around the world. But this vibrant ecosystem only flourishes if marketers can buy media online with the confidence that their ads are reaching real people, that results they see are based on actual interest. To grow the pie for everyone, we need to take head on the issue of online fraud.

This is a fight we’ve taken seriously from the beginning. Over the years, we’ve invested significantly in the technology and talent to prevent fraud and create greater accountability online. For example, we put extensive resources towards keeping bad actors out of our ad systems -- last year alone, we turned down millions of applications from sites looking to join our network because of suspected fraudulent activity. We also introduced new measurement tools, like MRC-accredited Active View, which lets advertisers buy only those ads that are viewable on a page. Active View offers greater peace of mind to all media buyers, but is especially important for brand marketers who want to know, first and foremost, that their ad has a chance to be seen.

Today we’re announcing our latest investment: we’ve completed an acquisition of spider.io, a company that has spent the past 3 years building a world-class ad fraud fighting operation.

Our immediate priority is to include their fraud detection technology in our video and display ads products, where they will complement our existing efforts. Over the long term, our goal is to improve the metrics that advertisers and publishers use to determine the value of digital media and give all parties a clearer, cleaner picture of what campaigns and media are truly delivering strong results. Also, by including spider.io’s fraud fighting expertise in our products, we can scale our efforts to weed out bad actors and improve the entire digital ecosystem.

Of course, this is not an issue we’re fighting alone. We applaud industry efforts like the IAB’s Traffic of Good Intent (TOGI) task force, which also play a critical role, as well as major commitments from others in the space. As an industry, we can address this issue and block those who seek to game the system. We can make digital the platform of choice for all marketers -- including brands -- to invest. And we can offer accountable media for all; we’re excited to take this big next step.


Posted by Neal Mohan, VP, Display Advertising

Cross post from the DoubleClick Publisher Blog

Recently there has been a great deal of discussion about applications that inject or overlay ads on sites without the express approval of users and those sites, and then monetize the inventory as their own. We believe that this kind of activity is bad for end users and damages the integrity of the advertising industry. In order for the programmatic marketplace to achieve its full potential and help as many marketers and publishers as we think it can, there needs to be trust between advertisers, publishers, and users.

We’ve invested, since the beginning, in strong policies and a system of checks and filters to ensure that the inventory on the DoubleClick Ad Exchange is the highest quality in the industry. Here’s a quick summary of what we do to stop invalid injected inventory from entering our exchange.

We don’t support spammy applications. Period.

Both the Google Platforms program policies and the DoubleClick Ad Exchange (AdX) Seller Program Guidelines strictly prohibit the use of systems, including toolbars, that overlay ad space on a given site without express permission of the site owner. In addition, we have numerous processes and technologies in place to review publishers’ inventory as well as advertisers’ ads to maintain a high standard of quality for how advertising is transacted on our platforms. 

In light of the increased concerns on this subject, many publishers have asked us for guidance on what to ask the exchanges or networks they work with. Here are three suggested questions any publisher partner should be able to answer in regards to protecting against injected inventory:

• Does your platform work with or supply advertising for clients who inject display ads in browsers?
• Do your program policies prohibit the use of systems to inject display ads in browsers, without first having obtained user consent or consent from the site affected?
• Please provide me a report of all the inventory partners on your platform serving my domain?

We do, and will always, support our publisher partners. 

Finally, I’d like to thank the millions of publishers who use the DoubleClick Ad Exchange, large and small, that day in and day out, provide amazing value both to their users and their advertisers. We welcome a broader discussion with our partners and with the industry about how to collectively solve this issue and others. Together, we can all ask the tough questions, hold each other accountable, and ultimately create the web we all want, where publishers, users and advertisers all thrive.

Posted by Scott Spencer, Director of Product Management

Brand advertisers want to know that the inventory they are buying is safe, clean, and high quality. We take this very seriously at Google and employ both cutting edge technology and human review to ensure that DoubleClick Ad Exchange (AdX) can offer the best environment to our buyers. We’re not alone in this fight. Media6Degrees has developed a proprietary system to filter out suspicious activity to protect their clients’ brands and we’re happy to say it validates AdX’s inventory quality. Brand advertisers who want to make sure their programmatic campaigns are running on safe and reliable traffic can look to DoubleClick Ad Exchange and Media6Degrees.

Finding the right customers -- at scale
Major M6D clients who buy media at scale -- brands like Verizon, Disney, Hyundai and Adobe -- rely on M6D and AdX to make sure they reach the right audience. For brands like these, placing their ads only on the best-quality inventory is paramount.

To find the right audience and the safest traffic, M6D relies on the DoubleClick Ad Exchange. "We help marketers drive scale in online media campaigns," says Alec Greenberg, VP of Media Operations for M6D. "Once our platform identifies the audience that matches the brand signal of a client, that's where AdX comes in. I want to buy the best targeting inventory and not worry about what is or isn't clean," says Greenberg. “AdX delivers. It's clean inventory and a ton of volume.”

Screening out invalid activity
"Earlier this year we turned off about 1000 real-time bidding publishers where we'd seen suspicious activity," says Alec. "When we compared notes with Google, we found that AdX had already turned off every single one. Every other partner on our list had suspicious sites still active, but Google had filtered out 100% of them."

To ensure their clients only buy the cleanest safest inventory, M6D's engineers developed a proprietary system to sniff out suspicious traffic and remove it from consideration. While M6D is diligent at protecting their clients, they’d prefer their ad partners remove suspicious activity before it even gets to them. With AdX, they get a partner that has been fighting invalid ad activity with great success for years.

"We are fanatical about what we do," says Alec Greenberg. "We believe we have the best targeting system out there and that we're delivering real results driven by real human behavior. That's why Google provides the largest piece of our inventory every day."

"Their scale and their quality are second to none. For anyone entering the fray of programmatic buying, AdX should be the first partner you engage."

Read the full case study here.

Posted by Alex Shellhammer, Product Marketing Manager