Internet Archive TV News Lab: Introducing Face-O-Matic, experimental Slack alert system tracking Trump & congressional leaders on TV news

Working with Matroid, a California-based start up specializing in identifying people and objects in images and video, the Internet Archive’s TV News Archive today releases Face-O-Matic, an experimental public service that alerts users via a Slack app whenever the faces of President Donald Trump and congressional leaders appear on major TV news cable channels: CNN, Fox News, MSNBC, and the BBC. The alerts include hyperlinks to the actual TV news footage on the TV News Archive website, where the viewer can see the appearances in context of the entire broadcast, what comes before and what after.

The new public Slack app, which can be installed on any Slack account by the team’s administrator, marks a milestone in our experiments using machine learning to create prototypes of ways to turn our public, free, searchable library of 1.3 million+ TV news broadcasts into data that will be useful for journalists, researchers, and the public in understanding the messages that bombard all of us day-to-day and even minute-to-minute on TV news broadcasts. This information could provide a way to quantify “face time”–literally–on TV news broadcasts. Researchers could use it to show how TV material is recycled online and on social media, and how editorial decisions by networks help set the terms of public debate.

If you want Face-O-Matic to post to a channel on your team’s Slack, ask an administrator or owner to set it up. The administrator can click on the button below to get started. Visit Slack to learn how to set up or join a Slack team. Questions? Contact Dan Schultz, dan.schultz@archive.org.

Add to Slack

To begin, Dan Schultz, senior creative technologist for the TV News Archive, trained Matroid’s facial detection system to recognize the president;  Senate Majority Leader Mitch McConnell, R., Ky., and Senate Minority Leader Charles Schumer, D, NY; and House Speaker Paul Ryan, R-Wis. and House Minority Leader Nancy Pelosi, D., Calif. All are high-ranking elected officials who make news and appear often on TV screens. The alerts appear in a constantly updating stream as soon as the TV shows appear in the TV News Archive

For example, on July 15, 2017 Face-O-Matic detected all five elected officials in an airing of MSNBC Live.

As can be seen, the detections in this case last as little as a second – for example, this flash of Schumer’s and McConnell’s faces alongside each other is a match for both politicians. The moment is from a promotion for “Morning Joe,” the MSNBC show that made headlines in late June when co-hosts Mika Brzezinski and Joe Scarborough were the targets of angry tweets from the president.  

The longest detected segment in this example is 24 seconds featuring Trump, saying “we are very very close to ending this health care nightmare. We are so close. It’s a common sense approach that restores the sacred doctor-patient relationship. And you’re going to have great health care at a lower price.”

Why detect faces of public officials?

First, our concentration on public officials is purposeful; in experimenting with this technology, we strive to respect individual privacy and harvest only information for which there is a compelling public interest, such as the role of elected officials in public life. The TV News Archive is committed to these principles developed by leading artificial intelligence researchers, ethicists, and others at a January 2017 conference organized by the Future of Life Institute.

Second, developing the technology to recognize faces of public officials contained within the TV News Archive and turning it into data opens a whole new dimension for journalists and researchers to explore for patterns and trends in how news is reported.  

For example, it will eventually be possible to trace the origin of specific video clips found online; to determine how often the president’s face appears on TV networks and programs compared to other public officials; to see how often certain video clips are repeated over time; to determine the gender ratio of people appearing on TV news; and more. It will become useful not just in explaining how media messages travel, but also as a way to counter misinformation, by providing a path to verify source material that appears on TV news.

This capability adds to the toolbox we’ve already begun with the Duplitron, the open source audio fingerprinting tool developed by Schultz that the TV News Archive used to track political ads and debate coverage in the 2016 elections for the Political TV Ad Archive. The Duplitron is also the basis for The Glorious ContextuBot, which was recently awarded a Knight Prototype Fund grant.

All of these lines of exploration should help journalists and researchers who currently can only conduct such analyses by watching thousands of hours of television and hand coding it or by using an expensive private service. Because we are a public library, we make such information available free of charge.

What’s next?

The TV News Archive will continue to work with partners such as Matroid to develop methods of extracting metadata from the TV News Archive and make it available to the public. We will develop ways to deliver such experimental data in structured formats (such as JSON, csv, etc.) to augment Face-O-Matic’s Slack alert stream. Such data could help researchers conduct analyses of the different amounts of “face-time” public officials enjoy on TV news.

Schultz also hopes to develop ways to augment the facial detection data with closed captioning, with for example OpenedCaptions, another open source tool he created that provides a constant stream of data from TV for any service set up to listen. This will make it simpler to search such data sets to find a particular moment that a researcher is looking for. (Accurate captioning presents its own technological challenges: see this post on Hyper.Audio’s work.)

Beyond this experimental facial detection, we have big plans for the future.  We are planning to make more than a million hours of TV news available to researchers from both private and public institutions via a digital public library branch of the Internet Archive’s TV News Archive. These branches would be housed in computing environments, where networked computers provide the processing power needed to analyze large amounts of data.

Researchers will be able to conduct their own experiments using machine learning to extract metadata from TV news. Such metadata could include, for example, speaker identification–a way to identify not just when a speaker appears on a screen, but when she or he is talking. Researchers could create ways to do complex topic analysis, making it possible to trace how certain themes and talking points travel across the TV news universe and perhaps beyond. Metadata generated through these experiments would then be used to enrich the TV News Archive, so that any member of the public could do increasingly sophisticated searches.

Feedback! We want it 

We are eager to hear from people using the Face-O-Matic Slack app and get your feedback.

  • Is the Face-O-Matic Slack app useful? What would make it more useful?
  • Would a structured data stream delivered via JSON, csv, and/or other means be helpful? What sort of information would you like to be included in such a data set?
  • Who is it important for us to track?
  • What else?

Please reach us by email at: tvnews@archive.org, or via twitter @tvnewsarchive. Also please consider signing up for our weekly TV News Archive newsletter. Or, comment or make contributions over here, where Schultz is documenting his progress; all the code developed is open source. (One observer already provided images for a training set to track Mario, the cartoon character.)

The weeds

The TV News Archive, our collection of 1.3 million+ TV news broadcasts dating back to 2009, is already searchable through closed captions.

But captions don’t always get you everything you want. If you search, for example, on the words “Donald Trump” you get back a hodge-podge of clips in which Trump is speaking and clips where reporters are talking about Trump. His image may not appear on the screen at all. The same is true for “Barack Obama,” “Mitch McConnell,” “Chuck Schumer,” or any name.

.

Search “Barack Obama” and the result is a hodge podge of clips.

Developing the ability to search the TV News Archive by recognizing the faces of public officials requires applying algorithms such as those developed by Matroid. In the future we hope to work with a variety of firms and researchers; for example, Schultz is also working on a separate facial detection experiment with the firm Datmo.

Facial detection requires a number of related steps: first, training the system to recognize where a face appears on a TV screen; second, extracting that image so it can be analyzed; and third, comparing that face to a set known to be a particular person to discover matches.

In general, facial recognition algorithms tend to rely on the work of FaceNet, described in this 2015 paper, in which researchers describe creating a way of “mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity.” In other words, it’s a way of turning a face into a pattern of data, and it’s sophisticated enough to describe faces from various vantage points – straight ahead, three-quarter view, side view, etc. To develop Face-o-Matic, TV News Archive staff collected public images of elected officials from different vantage points to use as training sets for the algorithm.

The Face-O-Matic Slack app is meant to be a demonstration project that allows the TV News Archive a way to experiment in two ways: first, by creating pipelines that run the TV News Archive video streams through Artificial Intelligence models to explore whether the resulting information is useful; second, by using a new way to distribute TV News Archive information through the popular Slack service, used widely in journalistic and academic settings.  

We know some ways it can be improved, but we also want to hear from you, the user, with your ideas. In the words of Thomas the Tank Engine, we aspire to be a “really useful engine.”

Face-O-Matic on GitHub

Follow TV News Archive progress in recognizing faces on TV on the following GitHub pages:

Tvarchive-faceomatic. The Face-o-Matic 2000 finds known faces on TV.

Tvarchive-ai_suite. A suite of tools for exploring AI research against video

This post is part of a blog series, TV News Lab, in which we demonstrate how the Internet Archive is partnering with technology, journalism, and academic organizations to experiment with and improve the TV News Archive, our free, public, online library of TV news shows. 

 

 

Posted in Announcements, News | Tagged , , , , , , , , , , , , , , , , | Leave a comment

IMLS Grant to Advance Web Archiving in Public Libraries

We are excited to announce that the Institute of Museum and Library Services (IMLS) has recently awarded our Archive-It service a Laura Bush 21st Century Librarian grant from its Continuing Education in Curating Collections program for the project Community Webs: Empowering Public Librarians to Create Community History Web Archives.

Working with partners from Queens Public Library, Cleveland Public Library, and San Francisco Public Library, and with OCLC’s WebJunction, which offers education and training to public libraries nationwide, the “Community Webs” project will provide training, cohort support, and services, for a group of librarians at 15 different public libraries to develop expertise in creating collections of historically valuable web materials documenting their local communities. Project outputs will include over 30 terabytes of community history web archives and a suite of open educational resources, from guides to videos, for use by any librarian, archivist, or heritage professional working to preserve collections of local history comprised of online materials.

We are now accepting applications from public libraries to participate in the program! Please help us spread the word about this opportunity to the entire public library community. You can also visit the program’s webpage for more information and the project’s grant materials are available through the IMLS award page.

Curating web archives documenting the lives of their patrons offers public librarians a unique opportunity to position themselves as the natural stewards of web-published local history and solidifies their role as information custodians and community anchors in the era of the web. We owe a debt of thanks to IMLS for supporting innovative tools and training for librarians and look forward to working with our public library friends and colleagues to advance web archiving within their profession and for the benefit of their local communities.

Posted in News | 1 Comment

Film Screening: Lost Landscapes of LA on August 7

By Rick Prelinger

Lost Landscapes of Los Angeles (2016, 83 minutes) is an experimental documentary tracing the changing city of Los Angeles (1920s-1960s), showing how its landscape expresses an almost infinite collection of mythologies. Made from home movies and studio-produced “process plates” — background images of the city shot by studio cinematographers for rear projection in feature films — Lost Landscapes depicts places, people, work and daily life during a period of rapid urban development. While audience  members are encouraged to comment, discuss and ask questions during the screening of this silent film, it is also a contemplative film that shows the life and growth of the U.S.’s preeminent Western metropolis as the sum of countless individual acts.

Lost Landscapes of Los Angeles is the latest of Rick Prelinger’s “urban history film events,” featuring rediscovered and largely-unseen archival film footage arranged into feature-length programs. Unlike most screenings, the audience makes the soundtrack — viewers are encouraged to identify places, people and events; ask questions; and engage with fellow audience members. While the films show Los Angeles as it was, the event encourages viewers to think about (and share) their ideas for the city’s future. What kind of a city do we want to live in?

Rick Prelinger is an archivist, filmmaker, and educator. He teaches at UC Santa Cruz and is a board member of Internet Archive. His films made from archival material have played at festivals, museums, theaters, and educational institutions around the world. Lost Landscapes of San Francisco (11 episodes, 2006-2016) plays every autumn in San Francisco. He has also made urban history films in Oakland and Detroit, and is currently producing a New York film for an autumn premiere. He thanks Internet Archive and its staff for making this film possible.

Get Tickets Here

Monday, August 7th, 2017
6:30 pm Reception
7:30 pm Interactive Film Program

Internet Archive
300 Funston Ave.
San Francisco, CA 94118

Posted in Announcements, Event, Movie Archive | Leave a comment

TV News Record: Donald Trump Jr makes “email” popular on TV again

This week the term “email” took on a new meaning in the annals of political controversy, President Donald Trump traveled to Poland, and the Senate continued to struggle with health care reform.

Email back on TV following Trump Jr.’s release of email exchange

Email as a technology may be on the way out (or just evolving), but its place in political history, already assured, got an even bigger boost this week when Donald Trump Jr. on Tuesday released a June 2016 email chain in which he exclaimed “I love it” to the prospect of receiving damaging information about Hillary Clinton through Russian intermediaries.

The term “email” is spiking again on TV news broadcasts, though it has not yet climbed to levels in the lead up to the November 2016 elections. In those months, particularly Fox news networks hammered on storylines of both hacked Democratic National Committee (DNC) emails and Hillary Clinton’s use of a private email server to do official business while serving as secretary of state.

However, with congressional and federal investigations of possible Russian tampering with the elections underway, we are early in the life cycle of this story. Stay tuned, and remember that searching terms on TV news is just a few clicks away on Television Explorer, which is fueled by TV News Archive data.

Search of term “emails” on Television Explorer, fueled by TV News Archive data. (Click on image to see larger.)



Following the TV 

The Watergate movie “All the President’s Men,” made the term “follow the money” an inspiration for journalists everywhere; thanks to the TV News Archive, enterprising reporters and researchers can “follow the TV” – find and link to past statements of public officials relevant to a current story.

With this week’s news putting Russia’s involvement in the election back in the headlines, past statements by members of the Trump camp become interesting watching. For example, here’s former Trump campaign chairman, Paul Manafort, in July 2016, saying “that’s absurd” to the allegation of a Putin-Trump connection.  Here’s Donald Trump Jr. in July 2016 saying it was “disgusting” to say the DNC email hack was perpetrated by the Russian government to support Trump. And here is advisor Kellyanne Conway in December 2016 saying “absolutely not” to a question about whether the Trump campaign was in contact with Russians trying to influence the election.



Factcheck: Obama knew about Russian interference in election and did nothing about it (mostly false)

At a joint press conference with Polish President Andrzej Duda last week, President Trump said “Barack Obama when he was president found out about this, in terms of if it were Russia, found out about it in August. Now the election was in November. That is a lot of time he did nothing about it.”

According to Lauren Carroll reporting for Politifact, the Obama administration took several steps after learning of the interference. Among them: “Obama personally confronted Russian President Vladimir Putin and told him to back off… On Oct. 7, the Obama administration publicly identified Russia for the first time as being behind election-related hacks, issuing a joint statement from Homeland Security and the Director of National Intelligence… Also, throughout August and up through the election, Homeland Security Secretary Jeh Johnson encouraged state-level election officials, through official statements and phone calls, to protect voting-related systems from cyber intrusions…However, the Obama administration took its most significant actions against Russia after Nov. 8. In late December, Obama ordered 35 Russian diplomats and suspected intelligence agents to leave the United States, and he also imposed narrow sanctions on some Russian individuals and organizations.”



Factcheck:  Billions are pouring into NATO because of the Trump administration (four Pinocchios)

During a speech in Poland last week, President Donald Trump said about about his calls for increased defense spending by other countries for NATO, “As a result of this insistence, billions of dollars more have begun to pour into NATO.”

“These budget decisions were made during the 2016 calendar year, before Trump became president,” reported Michelle Ye Hee Lee, for The Washington Post’s Fact Checker. She quoted Alexander Vershbow, former deputy secretary general of NATO, who said: “Who deserves the most credit? Vladimir Putin. It was the invasion of Crimea, the launching of insurgency backed by Russia in Eastern Ukraine, that was the wake-up call for the majority of the allies.”



Factcheck: hundreds of thousands will die if the Senate health care bill passes (can’t say)

With the Senate debating health care reform, FactCheck.org checked a recent statement by House Minority Leader Nancy Pelosi, D., Calif, where she said, “We do know that… hundreds of thousands of people will die if this bill (Senate health care bill) passes.”

Lori Robertson and Robert Farley wrote, “the research uses terms like ‘could’ and ‘suggests’ and ‘cannot definitively demonstrate a causal relationship,’ not the definitive ‘will’ favored by opponents of the bill. We can’t say whether any specific projection is a correct or valid number.”

To receive the TV News Archive’s email newsletter, subscribe here.

Posted in News, Television Archive | Tagged , , , , , , , , , , , , , , , , | Leave a comment

Net Neutrality Day of Action is Tomorrow, July 12!

Tomorrow, the Internet Archive will join with a huge list of Internet companies and organizations to protest the FCC Chair’s stated intentions to do away with net neutrality protections established by prior Commissioners.

Among other actions, the Archive will be displaying a pop-up message tomorrow on our sites to demonstrate the severity of this threat and simulate an Internet in which ISPs are given free reign to provide selective access to the web.

Organizations and site owners can learn more about joining in the Day of Action here.

Private individuals can help by 1. sending a letter to the FCC to voice your support for net neutrality (the deadline for comments is coming up fast!) and 2. spreading the message on social media.

The people fought back PIPA and SOPA before and we’re ready to stand up again!

Posted in News | 2 Comments

How to play and play with thousands of digitized 78rpm records


There are over
50k uploaded recordings from 78’s from users, and there are now 10’s of thousands of high-bitrate unrestored transfers of 78’s that are part of the Great 78 Project.

With this many, it gets hard to find things you want to explore.  Here are some techniques I do:

Again, I recommend the “play items” link as it plays along like youtube does.

To download for research and preservation purposes:

  • Download right hand side of a “details” page, and you can click to see the whole list of files.
  • There is the “best” stylus version (according to an audio engineer at George Blood Co) that is renamed to be itune-ish compatible, but all the stylii recordings are there in flat and equalized formats, and each of those in FLAC and MP3 formats.

To download many records for research and preservation purposes (requires linux or mac and command line skills):

  • Install the Internet Archive command line interface
  • To download metadata in json from our 78 transfers, in bash:
    • for item in `./ia search “collection:georgeblood” –itemlist`; do curl -Ls https://archive.org/metadata/$item/metadata ; echo “”; done
    • I installed gnu parallel to speed things up (I use “brew install parallel” on a mac)
    • ./ia search “collection:georgeblood” –itemlist | parallel -j10 ‘curl -Ls https://archive.org/metadata/{}/metadata’ > 78s.json
  • To download all of files of the high bitrate transfers (and is repeatable to update based on failures or new additions):
    • ./ia download –search=”collection:georgeblood”      (14TB at this point)
  • To download only the metadata and Flac’s:
    • ./ia download 78_–and-mimi_frankie-carle-and-his-orchestra-gregg-lawrence-kennedy-simon_gbia0006176a –format=”24bit Flac” –format=”Metadata”
  • To download only the metadata and mp3’’s of all ragtime recordings:
    • ./ia download –search=”collection:georgeblood AND ragtime” –format=”VBR MP3″ –format=”Metadata”

If you want to do more on downloading specific sets, I suggest the documentation or joining the slack channel.

How You can Help: Please help find dates for these 78’s

We are doing some by automatically matching against 78discography.com and discogs.com, but many are done by hand, finding entries in billboard magazine and on DAHR the like.  But many still need dates.

If you would like to help, then please do research and post your findings in the review of a 78rpm record, citing your sources.  Then someone with privileges will change the metadata in the item.

The complete collection has date facets on the left reflecting the dates we have found.  But we only have dates for about half, and there are thousands of 78rpm sides posted each month so we need help!

To find what others have done, you can list them in the order of the most recent reviews.

The most recent ones that do not have a date nor a review are here.  This is a good starting place.

But again, these need dates.  If you tried and could not find anything online, then please post a review to that effect so others do not spend time on the same one.   

If you find other information, or know other information about the performer, performance, or piece, please put it in.  Also links to youtube, wikipedia, and old magazines like cashbox and billboard.  

For those that get into it, we invite you to join the slack channel (a great tool if you have not used it already), then that is where there is some discussion.   Caitlin@archive.org can set you up.

Oh, and I have gotten a bit obsessed, and this is a twitter feed of a digital transfer every 10 minutes which I visit more times than I should probably.

Restoration techniques:

I have been using Dartpro MT – I like it because it has a “Filter builder” –
the program doesn’t like 24 bit but I transfer at 24/96,000 , resample to
16/96000 then decrackle starting with a setting of 50 repeating the process
with an increase each time by another 10 i.e. 50, 60, 70, 80 (maximum) if
more noise is still there, I run repeatedly at 80 until the reported
interventions get to a number of 4000 or so. I can then manually remove any
clicks that are left.

I don’t use the denoise or dehiss, preferring to use declick  at very low
settings (78’s don’t have hiss – what you hear as hiss is the combination of
many little clicks.

I start with a setting of 2 then 4 then 6 . I leave the settings the same
then just find whether 1 or 2 or 3 passes will polish the higher noise away.
Decrackle doesn’t affect the high frequencies but declick does. This process
takes a little time but I almost never find that any distortion is
introduced to the sound.  I hate getting to the end of the record where the
really growly trumpet sound is a distorted mess, and this workflow prevents
that. – Mickey Clark

My “go to” software for restoration work is Izotope. However, I also have Adobe Audition, Diamond Cut, Pro Tools, Samplitude, and Sound Forge available for specific situations. As Ted Kendall points out: Hearing and Judgment play the major roles in both transfer and restoration. To that I would add Experience. And, as always, start with the best possible source.

Get Involved

Please write to the Internet Archive’s music curator  bgeorge@archive.org or more generally to info@archive.org .

Please join this project to:

  • Share knowledge. Help improve the metadata, curate the collection, contact collectors, do research on the corpus, etc.
  • Include your digitized collection. If you have already digitized 78s or related books or media, we’d like to include your work in the collection.
  • Digitize your collection.  We’ve worked hard to make digitization safe, fast and affordable, so if you’d like to digitize your collection we can help.
  • Donate 78s.  We have 200,000 78s, but we are always looking for more.  We will digitize your collection and preserve the physical discs for the long term.

If you are in the bay area of California, we can also use help in packing 78’s for digitization and please come over for a lunch on a Friday.

Posted in Announcements, News | Leave a comment

TV News Record: Focus on North Korea

By Katie Dahl and Nancy Watzman

Following the U.S. government’s confirmation that North Korea had successfully fired an intercontinental ballistic missile, we focus on statements by public officials and pundits on the nuclear threat from the Korean Peninsula, including some past fact-checked segments.

What top-rated cable shows aired the day after

On Fox News, “Tucker Carlson Tonight” focused his report on the missile launch by interviewing Michael Malice, a New York-based ghost writer and author of Dear Reader: The Unauthorized Biography of Kim Yong Inalong with  George Friedman a founder of Geopolitical Futures. Malice said the launch amounted to a commercial for the country’s product, “It’s a great sales pitch to show they have weapons they could sell and make a lot of money off of.” Friedman emphasized that the “Chinese have no reason to solve this,” and also said he didn’t think North Korea has a “capable” nuclear missile at this point.

Over on MSNBC, Rachel Maddow interviewed NBC’s national security reporter, Courtney Kube, who said that North Korea hadn’t demonstrated its capability to deliver a nuclear warhead yet, although “I don’t know if you would find anyone in the U.S. military at the highest levels who would say with confidence or certainty that they don’t absolutely have that capability. I think that they’re hopeful they do not, since they haven’t demonstrated or tested it.”

In the first hour of Anderson Cooper 360 on CNN, John Berman, sitting in for Cooper, placed North Korea’s launch in a global context with President Donald Trump’s trip to Europe, interviewing a panel of former public officials, David Gergen, who advised Republican and Democratic presidents; John Kirby, who was a spokesperson for the State Department under the Obama administration, and Shamila Chaudry, who served on the National Security Council under the Obama administration.


What Congressional leaders have said about North Korea

Senate Majority Leader Mitch McConnell, R., Ky., on April 2o17, mentioned North Korea in context of the U.S. missile strike on Syria in response to chemical attacks on civilians as “a message to Iran and North Korea and the Russians that America intends to lead again.”

House Speaker Paul Ryan, R., Wis., when talking about a bill to strengthen sanctions back in 2016, said “[Obama’s] strategy of strategic patience with North Korea, it’s just not working.”

House Minority Leader Nancy Pelosi, D., Calif., in April 2017, said “The president is playing with fire when he’s talking about North Korea. We have to exhaust every diplomatic remedy.”

Senate Minority Leader Chuck Schumer, D., N.Y.,  in April 2017, said “The only way to really stop North Korea from doing what it’s doing short of war is to get China to fully cooperate, because they control all the trade. They control the entire economy, really, of North Korea. My view is to get the Chinese to do something real, you have to be tough with them on trade. Trade is their mother’s milk.”

And now for some past fact-checked segments on North Korea.

Trump never said that more countries should acquire nuclear weapons (False)

In November 2016, not long after he won the election, then-President-elect Donald Trump tweeted:

Lauren Carroll, reporting for PolitiFact, rated this claim “false,” citing several examples from the campaign trail where Trump had said just that. For example, in April 2016, Fox News’ Chris Wallace asked, “You want to have a nuclear arms race on the Korean peninsula?” Later in the broadcast, Trump said about Japan and South Korea, “”Maybe they would be better off — including with nukes, yes, including with nukes.”

China has “total control” over North Korea (Mostly False)

During a Republican primary debate in January 2016, Trump said that China has “total control just about” over North Korea. Reporting for PolitiFact, Louis Jacobson rated this claim as “mostly false.” “He has a point that China holds significant leverage over North Korea if it wishes to exercise it, since China provides the vast majority of North Korea’s international trade, including food and fuel imports. But Trump’s assertion, even slightly hedged as it is, overlooks some significant limits to that leverage, notably the North Korean government’s willingness to follow its own drummer even if that means its people suffer. The fact that North Korea recently conducted a nuclear test over the strenuous objections of China suggests that Beijing lacks anything approaching ‘total control’ over North Korea.”

China accounts for 90 percent of North Korea’s trade (True)

In April 2017, Secretary of State Rex Tillerson told the U.N. Security Council, “But China, accounting for 90 percent of North Korean trade, China alone has economic leverage over Pyongyang that is unique, and its role is therefore particularly important.”

PolitiFact’s John Kruzel rated this claim as “true.” “China’s role as an outsize trade partner of North Korea is a relatively new development. Since 2000, trade with the rest of the world has dropped off, as Chinese trade has risen. While the ratio is subject to change based on political factors, China now accounts for around 90 percent of North Korean trade.”

To receive the TV News Archive’s email newsletter, subscribe here.

Posted in Announcements, News, Television Archive | Tagged , , , , , , , , , , , , , | Leave a comment

78rpm Records from KUSF: Getting Ready for Digitization


by B. George, Music Curator and Director of the ARChive of Contemporary Music

It’s the 4th of July and we’re celebrating in Richmond, California where we just finished repacking and sorting the legendary KUSF archive – 22 pallets full, maybe 60,000 (whose counting?) discs. This collection was transferred in 2012, and it is now ready for its next stage.  We are finding the recordings are in good condition considering the heavy radio play. The normal surprise are the prominent call letters markings on most of the cover art.

The pop 78s are being prepped to be digitized as part of the Great 78 Project and the remaining 45s, classical 78s and LP’s physically preserved to be ready for the next stages of digitization and access.

KUSF was a non-commercial radio station owned by the University of San Francisco. From 1963 until 2011, the station broadcast at 90.3 FM MHz. KUSF currently broadcasts online and is still a student driven organization at the University of San Francisco. Hip and popular, the station was student run, freeform and eclectic. Many now-famous acts first gained exposure on KUSF, including The B-52’s and Metallica. We’re doing our best to keep these gone but important cultural icons alive.

Posted in Announcements, News | 3 Comments

K-12 Web Archivists Capture History in the Making

by Sylvie Rollason-Cass, Web Archivist, Archive-It

This year marked the 9th season of the K-12 web archiving program. Students from 11 schools around the country worked together to think critically about information on the web and to select websites to archive for the future. Their collections are centered around topics that reflect their interests, their day-to-day lives, current events, and topics they studied in class. Each school incorporates web archiving into its curriculum differently. This year 3 teachers generously shared their experiences participating in the K-12 Web Archiving program. Find out more about their year below, and be sure to check out all of the 2016-2017 Student Collections.

Web Archiving in the Civics Classroom at Williams Middle Magnet School 

Elizabeth Smith – Civics Teacher

My name is Elizabeth Smith and I teach Grade 7 Civics at Willaims Middle Magnet School in Tampa, Florida. I was so excited when I read about this project and could not wait to apply. I am technologically challenged but always look for ways to integrate technology into my classroom. Our civics curriculum is spiraled with analysis of primary and secondary sources and this project was a great way to enrich what we were already doing. We chose Florida as our focus as many of my students wanted to learn more about the state in which they live. Students chose to research websites of their own personal and academic interest. Several said this project would help them identify areas of research for their upcoming 8th grade community project. We are looking forward to being a part of the project again during the next archiving season!

Check out William’s Middle Magnet School’s 2016-2017 Collections >>> 

Using the Wayback Machine at the Rooftop School, Samuel discovers that YouTube was originally a dating website.

“Archive/Opera” – The Studio at Mayeda at Rooftop School

Andi Wong – Teaching Artist

At Rooftop Alternative PreK-8 School in San Francisco, 40 seventh and eighth graders worked with teaching artist Andi Wong to establish The Rooftop ARTchives. The “Archive/Opera” class at the school’s Mayeda Campus gave these students the opportunity to create both the “archives” (“public records” from the Greek ta arkheia,) and the opera (new “work”).

In this tumultuous political climate, the importance of community, civic responsibility and cultural memory became clear to our students. History will record how tribes of water-protectors gathered together at Standing Rock; millions of women marched around the world in pink knitted caps; scientists worked with archivists to save climate data and disappearing government websites; and the National Parks Service went rogue on Twitter. The act of archiving requires careful consideration of the past, present and future. As our students ventured beyond the walls of their classrooms to experience the stairways, slides and expansive vistas of Twin Peaks, their open conversations led to many questions. How can we know what is missing, if something has not yet been found? How many students have graduated from Rooftop? How far does the raven fly? Will San Francisco still experience fog in one hundred years? When today’s youth are sixty-four, will they still remember the lyrics to all of the songs from Hamilton?

The Archive/Opera class culminated with a community gathering — an Open House celebration of the Mayeda Campus’ 20th Anniversary, featuring artwork, musical performances, student speeches about the archiving experience. A tea serving ceremony honored principal Nancy Mayeda and the teachers who first opened the doors to the Mayeda Campus in 1997. The evening’s program closed with the presentation of a City proclamation and the dedication of the Rooftop ARTchives. When our students were asked to reflect on what they valued most about this year’s experience, they spoke of freedom, friendship and community pride in accomplishing something important together. Many thanks to the Internet Archive, the Library of Congress and Archive-It’s K12 Web Archiving Program for helping Rooftop’s students to capture history in the making. The act of archiving gave our students a very real sense of their collective power and responsibility as the keepers of their own stories and memories.

Check out Rooftop School’s 2016-2017 Collections >>>

Reflections on the 2016-17 K12 Web Archiving Project at Mount Dora High School

Patricia Carlton, PhD – Media Specialist

Challenging adult authority may be the bailiwick of teenagers, yet when questioning the authority of the Internet, teens are not as skilled or tenacious. Web archiving presents a fun and empowering way for my high school students to critically examine the authorship and credibility of the Internet, as well as identify what is historically and culturally significant. When this year’s web-archiving students began selecting and creating collections for the archive, I suggested they peer more closely under the hood of each site and object. What did they discover from their crawls that wasn’t immediately apparent from their first “reading” of the website? The following quotes excerpted from a sampling of the students’ final review and evaluation of the project reveal the type of discoveries made regarding their collections and the Internet in general.

The web is an extremely important factor in preserving things in order to view them in later years. The web, in my opinion is also much easier and more accessible to a wider range of people. While on the web, you have to be extremely careful on what you consider a reliable source. – Felicia

I have learned that the web is very contradictory and is filled with differing opinions, facts, and beliefs, and you normally can find an answer you like if you search long enough, despite general public beliefs. – Kacee

Most students assumed greater responsibility for controlling their crawls than my previous web-archivers, evidenced by their attention to their crawl scopes and carefully crafted descriptions at both collection and seed level metadata. The 2016-17 cohort “authored” their respective collections and even added corresponding MLA citations! They believed not only in the significance of their collections (conspiracy theories, political memes, and chick flicks to mention a few), but they also believed they were contributing new knowledge – real, meaningful content to the Internet that someone, someday might discover. And, their teen voices would be the authority behind their interpretation and curation!

Check out Mount Dora’s 2016-2017 Collections >>>

Posted in Announcements, News | Comments Off on K-12 Web Archivists Capture History in the Making

TV News Record: Fox on CNN, getting Gorsuch right + fact-checks

We changed our name! Our weekly updates from the TV News Archive are now titled TV News Record, to reflect our goal of providing many ways for viewers to put the news in context, whether through fact checks from our national fact-checking partners, visualizations of patterns of news coverage, or other ways of using TV news metadata to deepen analysis of the news.

This week we present fact-checks by fact-checking groups of claims about gang arrests, fraudulent votes, the Russia investigations, and Medicaid. But first, some observations about cable news coverage of news this week and realized hopes for the newest Supreme Court Justice Neil Gorsuch by the religious right.

Fox devotes most air time to CNN story retraction 

In a stark reminder of how which cable channel you watch affects what news you see, Dave Weigel of The Washington Post noted this week that while health care was big news on Capitol Hill Tuesday evening, Fox cable stations were busy reporting on the fallout over a retracted story by CNN related to Russia investigations: “The network’s prime-time shows, ratings kings of cable news, ignored the health-care story,” he wrote. Meanwhile, “The CNN story, sparked by a retracted report on Russia and Trump and inflamed by the sting video, sprawled across multiple segments.”

Weigel’s analysis is borne out by plugging key search terms into the Television Explorer, a tool built by data scientist Kalev Leetaru fueled by TV News Archive data. Search for the term “retraction” near “story” over the 72-hour period ending on June 30, and Fox news dominates.

A search for “retraction” near “story” over 72 hour period ending at 10:20 am ET, June 30, 2017.

The converse is also true: a similar search for “health care” near “Senate” yielded the following results, with CNN  devoting the most attention to the story, and more than four times as much as Fox News did.

“Health care” near “Senate,” 72 hour period ending 10:20 am ET, June 30 2017.

Such searches are easily done on Television Explorer. With the explosion of coverage over the past 24 hours about President Donald Trump’s tweets about MSNBC “Morning Joe” co-hosts Mika Brzezinski and Joe Scarborough, we’re keeping on eye on this search of “Trump” near “tweet.” Questions? Contact tvnews@archive.org.

Getting Gorsuch right

In the lead up to the Senate’s confirmation of Neil Gorsuch as the newest member of the U.S. Supreme Court earlier this year, many commentators noted that his appeal to conservatives was enhanced by his stance on religious issues. With the conclusion of the Court’s term on Monday, Gorsuch’s champions were vindicated with his decisions in such matters as Trinity Lutheran Church of Columbia, Inc. v. Comer, in which Gorsuch sided with the majority affirming that a church was eligible for public funding to build a playground. Here is CNN reporter Ariane de Vogue back in April 2017 explaining why the case matters:

And now on to this week’s highlighted TV news fact-checks.

Claim: MS-13 gang members are being deported by the thousands (hundreds)

At a rally speech in Iowa on June 21, President Trump said of MS-13, “They don’t like to shoot people. They like to cut people. They do things that nobody can believe. These are true animals. We are moving them out of the country by the thousands, by the thousands. We’re getting them out, MS-13.”

Michelle Ye Hee Lee reported for the Washington Post’s Fact Checker that while the Trump administration has increased enforcement against gang activity, deportations “are in the hundreds, not the thousands, under Trump.”  The reporters at FactCheck.org concurred, and quoted Danielle Bennett, a spokesperson for U.S. Immigration and Customs Enforcement, who said the agency “does not track gang removals by specific gang.”

Claim: People hired for independent Russia investigations are all Hillary supporters (mostly false)

In a recent TV interview, President Trump said “I can say that the people that have been hired (for the independent Russia investigation) are all Hillary Clinton supporters, some of them worked for Hillary Clinton.”

For PolitiFact, Manuela Tobias reported “Three of the eight available Mueller hires made campaign contributions to Clinton, which undermines Trump’s statement that all are Clinton supporters. Furthermore, none of them have worked for Clinton directly. Two represented either the Clinton Foundation or an aide, never her, and working for WilmerHale, which has also represented key members of Trump’s White House.”

You’re grandfathered in if you became a Medicaid recipient under Obamacare (mostly false)

White House adviser Kellyanne Conway said on Sunday about the proposed GOP health care legislation, “If you became a Medicaid recipient through the Obamacare expansion, you are grandfathered in.”

Joshua Gillin reported for PolitiFact, “The ‘grandfathered expansion enrollees’ would have to maintain near-continuous coverage, with no breaks of more than a month, in order to get the higher rate.” He continued, “Continuous coverage can be hard for Medicaid recipients to maintain. Medicaid patients have to continuously provide proof of income, and if a patient goes over the income limit, they lose coverage. This is a concept known as churning.”

Claim: 5.7 million undocumented immigrants might have voted in 2008 (wrong)

On a recent morning show, Fox co-host Ainsley Earhardt said of the 2008 election, “5.7 million – that’s how many illegal immigrants might have voted.”

According to Amy Sherman at PolitiFact, “Trump has made repeated claims about massive voter fraud and election rigging, which we’ve debunked again and again and again and again and again and again and again (and we debunked a claim by his spokesman Sean Spicer).” She goes on to explain that the figure used by Earhardt “is based on an extrapolation of a controversial study that relied on a very small number of responses. Researchers involved in the underlying survey of voters have cautioned against using their data to reach conclusions about noncitizen voters.”

To receive the TV News Archive’s email newsletter, subscribe here.

Posted in Announcements, Television Archive | Tagged , , , , , , , , , | Comments Off on TV News Record: Fox on CNN, getting Gorsuch right + fact-checks