Photo by Dan Nguyen/ProPublica

Last week we published a big update to Dollars for Docs, our interactive news application of payments made to U.S. healthcare providers by 15 pharmaceutical companies. Compared to when we launched the project in 2010, the amount of data we’re collecting has grown enormously: The list of payments increased from around 750,000 to almost 2 million, and the grand total of the payments grew from around $750,000 to just under $2 billion.

Compiling the data for it has been an enormous project right from the beginning. After we published the first version, the original developer on the project, Dan Nguyen, compiled all of the things he had to learn into a guide to scraping data. This year’s update took more than eight months of full-time work by me, working with other news-app developers, and at times with our CAR team, a researcher, two editors and two health care reporters. It was a massive effort and presented huge technical and journalistic challenges.

Read More »