{{Hatnote}}

Hashtags, One Year In

It’s been around one year since Hatnote announced hashtags on Wikipedia. The Wikimedia Blog has a full story on several of the impacts and inroads being made by supporting this powerful convention on Wikipedia.

Follow that link or if you’re in a hurry, check out the new Wikipedia Social Search interface directly!

Announcing the Hatnote Top 100

Moreso than any other major site, Wikipedia is centered around knowledge, always growing, and brimming with information. It’s important to remember that the insight of our favorite community-run encyclopedia often follows the focus of its massive readership. Here at Hatnote, we’ve often wondered, what great new topics is the community learning about now?

To shed more light on Wikipedia’s reading habits, we’re pleased to announce the newest addition to the Hatnote family: The Hatnote Top 100, available at top.hatnote.com. Because we can’t pass up a good headwear-based pun.

Updated daily, the Top 100 is a chart of the most-visited articles on Wikipedia. Unlike the edit-oriented Listen to Wikipedia and Weeklypedia, Top 100 focuses on the biggest group of Wikipedia users: the readers. Nearly 20 billion times per month, around 500 million people read articles in over 200 languages. Top 100’s daily statistics offer a window into where Wikipedia readers are focusing their attention. It also makes for a great way to discover great chapters of Wikipedia one wouldn’t normally read or edit.

Clear rankings, day-to-day differences, social media integration, permalinks, and other familiar simple-but-critical features were designed to make popular Wikipedia articles as relatable as albums on a pop music chart. In practice, popular news stories and celebrities definitely make the Top 100, but it is satisfying to see interesting corners of history and other educational topic sharing, if not dominating, the spotlight.

In addition to a clear and readable report, Top 100 is also a machine-readable archive, with reports dating back to November 2015, including JSON versions of the metrics, as well as RSS feeds for all supported languages and projects. It’s all available in over a dozen languages (and we take requests for more). The data comes from a variety of sources, most direct from Wikimedia, including a new pageview statistics API endpoint that we’ve been proud to pilot and continue to use. And yes, as with all our projects the code is open-source, too.

For those of you looking to dig deeper than Wikipedia chart toppers, there are several other activity-based projects worth mentioning:

• stats.grok.se - The original, venerable pageview grapher and API
• Wikimedia Report Card - Advanced metrics and data used by the Wikimedia Foundation
• The Open Wikipedia Ranking - Traffic stats and more
• @WikipediaTrends - A bot posting notable upward traffic spikes
• The Top 25 Report - A manually-compiled weekly report of views and likely reasons
• The Weeklypedia - Weekly edit statistics, emailed and archived by Hatnote

And there are other visualizations on seealso.org as well. But for those who like to keep it simple, hit up the Hatnote Top 100, subscribe to a feed, and/or follow us on Twitter. See you there!

P.S. The Wikimedia Blog featured us in the actual announcement for the PageViews API. Yay!

De Venezuela a Gran Hermano: los artículos más editados de Wikipedia en español

De Venezuela a Gran Hermano: los artículos más editados de Wikipedia en español. Noticias de Tecnología. 142 de los 500 artículos más revisados de Wikipedia en lengua española están relacionados con el fútbol. La geografía es la siguiente gran categoría en número de edición de artículos. ...

We recently helped El Confidencial get statistics about the most edited articles on Spanish Wikipedia. You can see a few clusters of popular topics, such as countries, sports teams, and TV shows. You can also compare the data with the most edited topics on English Wikipedia.

If you want weekly updates on the most edited Wikipedia articles, don’t forget to sign up for Weeklypedia!

Hatnote Recent Changes

Almost two weeks since launching IFTTT and it’s still keeping our hands full. There have been over 11 million Recipe requests handled. And Wikipedia’s public Recipes are popular enough to have already made it the #89 IFTTT Top Chef, too.

Even with all the success, Hatnote’s already back in the workshop cooking up new changes. We’ve talked to a few, but we’re definitely taking further user feedback from any thousands of Wikipedia+IFTTT who’d like to chime in on our Ask.

Other changes:

• Brand new Tumblr theme from scratch. Still minimalist, but much more responsive and technically compliant.
• New Social Search frontend and backend yielding 100x faster hashtag searches. Here’s one hashtag search from a recent #TechLadies editathon.
  
• Weeklypedia’s data is now automatically backed up daily on Github. There are probably a few neat visualization projects one could do with over a year of a week-long sliding window of edit data from everyone’s favorite encyclopedia.

And we’ve got plenty more updates in store, see you in another couple weeks!

Build Your Own Topic Bot

It’s been 10 days since the new Wikipedia IFTTT channel launch, and the response has been amazing. Our backend has seen almost 2.5 million requests for Recipe updates, from tens of thousands of unique users. In this post we wanted to do a quick hands-on guide showing one way IFTTT helps Wikipedians reach a wider web.

From drones to Florida news to aesthetic blogs to fake news to drones, topic-specific streams are drawing a big audience these days. Running one isn’t as easy as it looks, though. Content doesn’t create and curate itself. Unless you automate. Case in point, here at Hatnote, the fleet of special-interest Twitter bots built on Wikipedia and IFTTT just keeps getting longer. Here’s a selection from the last week:

• @WikiRedlist: Track the status of critically endangered species.
• @Wikidemics: Find out when doctors (PhDs, JDs, MDs, etc.) join Wikipedia.
• @NotableDeath: Reports recent passings of people with Wikipedia pages.
• @GreeceEdits: Updates on pages related to the Greece crisis.
• @SveWikipedia: Tweets about opportunities to collaboratively edit Wikipedia.
• @FlagsForDays: Updates on flag design and vexillology.

And you can have your very own with 5 simple steps. It only takes about 10 minutes, so let’s get started!

Step 1: Pick a topic

Before signing up for anything, first let’s sort out our goals. If you’re running an editathon or are an experienced editor, maybe you already have a topic or tracked hashtag in mind. For the rest of us, English Wikipedia alone has millions of articles and hundreds of thousands of categories. Think about your audience, but above all, pick something you’re interested in. If you’d like to pick something out of a hat, try a random article or try watching Listen to Wikipedia and seeing if anything catches your eye. Wikipedia excels as a source of random history and science, but also current events.

Here are some more fun entry points for browsing Wikipedia:

• List of lists of lists
• High-level categories
• Contents
• Recent changes
• Portals
  • Current Events Portal
  • All Portals

Step 2: Register your Twitter username

This is the hardest part. Finding and picking a name that is available. Twitter has a billion registered users. It might take you a minute or two to come up with a username that doesn’t have two tweets from 2011.

You’ll need to create a new Twitter account. Here’s a timesaver: Gmail (and other email providers) treat emails such as janedoe+anything@gmail.com the same as janedoe@gmail.com. So you can skip creating a new email and just use your existing one, such as youremail+yournewtwittername@gmail.com.

Step 3: Register for IFTTT

Even if you already have an IFTTT account, registering a new one makes the process go smoother and keeps your Recipes more organized in the long run. You wouldn’t want your bot’s recipes mixed up with your personal ones. You can use the same email trick from Step 2 on IFTTT.

Step 4: Connect the IFTTT Channels

Go to the Wikipedia IFTTT Channel and click Connect Channel. You should see a success message in a moment.

Go to the Twitter IFTTT Channel and click Connect Channel. Then, log in as the special-purpose account we just created to authorize IFTTT to post on its behalf. When you see the success message, you’re done!

There are many other channels worth exploring as outputs for your Wikipedia criteria. More on that later.

Step 5: Create the Recipes

Finally we get to the fun part. Choosing and creating the recipes. Here’s where you get creative, but there are three Triggers worth highlighting:

• Hashtag (docs) - Fires on edits with a #hashtag in the comment. Particularly useful if running an Editathon.
• Added to category (docs) - Triggers when an editor adds an article to a Category.
• Modified in category (docs) - Fires whenever an article in a Category is edited.

Depending on how articles have been categorized, to really cover a subject you may have to create more than one Recipe. Still, this is Wikipedia. If you see something that can be improved, don’t hesitate to try out creating or adding to a Category!

And that’s it! Your bot is operational. No backend work necessary, just some old-fashioned research on a new-fashioned encyclopedia. Here are some parting tips:

• Consider sprinkling in human commentary for a more engaging reader experience.
• Keep an eye on posting rate to make sure it’s not too fast-paced for your service or audience.
• If there’s not enough posts, add more Recipes!
• If you have an international audience, you may want to use interlanguage links, found at the bottom of the sidebar on the left of Wikipedia pages, to connect Categories from different languages’ Wikipedias.
• If you want to post to other channels, you can have a single recipe watch the Twitter feed and copy it over to Tumblr, Facebook, or other platform of choice.
• Twitter recommends you get and maintain real followers as soon as possible to ensure that your bot performs well in search rankings.

As always, as soon as you create your bot, tweet at us so we can add it to the list!

Wikipedia and IFTTT: A Technical Guide

Here at Hatnote we build on Wikipedia a lot. And while we love building projects like Listen to Wikipedia and The Weeklypedia, we have to admit programming, integrating, and maintaining reliable services can be a lot of work. Creating cool and functional Wikipedia projects remains out of reach of most busy Internet denizens. Until today.

With the aim of bringing Wikipedia to the wider web, Stephen and I are pleased to have worked with Wikimedia and IFTTT to build the brand-new Wikipedia IFTTT channel. This post is your 10-minute usage and technical guide to all things Wikipedia+IFTTT. For the official announcement, see the Wikimedia blog.

IFTTT connects web services, so when something happens on one service (the Trigger), it pushes an update to another service (the Action). With the new Wikipedia channel, so you can create customized combinations, called Recipes, to plug Wikipedia into your Internet ecosystem. With this new channel, Wikipedia offers eight new Triggers that can be connected to IFTTT’s 200+ other channels, including email, phone, Android, iOS, Twitter, Tumblr, Slack, and many more. There are already dozens of example Wikipedia Recipes ready for activation.

Even if you’re already familiar with IFTTT, a few things set Wikipedia apart from other channels. Foremost, whereas most other services provide an API to data and information, Wikipedia provides a direct interface to curated knowledge. So trust us when we say it pays to invest in learning the special nature of Wikipedia, as seen through the lens of the new IFTTT Triggers.

The Daily Triggers

• Article of the Day
• Picture of the Day
• Word of the Day

The simplest of the new Triggers, these three update once per day around midnight UTC. Their simplicity makes them the perfect candidates to introduce Wikipedia Fact #1:

> Fact #1: There are many Wikipedias! Most people are familiar with Wikipedia as one site. But unlike other IFTTT channel providers, the fact is there is one Wikipedia per language, each with its own conventions and quirks. We’ve made all the new Triggers customizable for all Wikipedia’s 290 languages. So keep in mind for future triggers, a recipe that works well on one language, may not apply at all to the others. As a multilingual Wikipedia editor, I’ve found clicking the interlanguage links on individual articles and referring to the Weeklypedia archives are good ways to get acquainted with the differences.

The Daily Triggers are are an exception to the interlanguage differences. Almost all Wikipedias, regardless of size and age, follow the convention of featured articles. So feel free to get a weekly dose of free knowledge in your language of choice!

Edits to an Article

When most people talk about Wikipedia, they’re talking about the articles, Wikipedia’s fundamental unit of knowledge organization. Articles are written by many users contributing what knowledge they can, edit by edit.

On the surface, the Edits to an Article Trigger is simple. If you have a favorite article, IFTTT will monitor it for changes and notify you when an edit occurs. But of course there are caveats.

> Fact #2: Wikipedia is a uniquely open platform. That openness is reflected in the data passing through IFTTT APIs. Don’t be surprised to see raw or non-constructive changes made (and usually reverted) by Wikipedia editors, often in quick succession.

> Fact #3: Wikipedia is a community with many subcommunities. Quality, approaches, and discipline will vary between these subcommunities. For instance, to match naming conventions, it’s common for articles to be renamed (“moved”) multiple times, especially when they’re brand new.

If you’re learning new things from the facts and think you might have some better solutions, Wikipedia wants to hear about them. In the meantime, you can have IFTTT DM you on Twitter when there are updates to the discussion or new terms that might help in the debate.

Edits from a User

Editors write articles. You don’t need to be logged in to edit Wikipedia, but it helps with site customizations and better edit history maintenance. Now, using the Edits from a User Trigger you can easily connect a given user’s edit activity with other social sites, making Wikipedia more personal than ever.

Other than Stephen and me, finding interesting users to follow is a topic for another post. But there is one type of user that has garnered a lot of attention in the past: the Bots. Wikipedia’s Bots are more administrative power tool than autonomous overlord, but people are still obsessed. Just as well, because this brings us to the next fact.

> Fact #4: Wikipedia grows at different speeds. Some pages change quickly, some slowly. Generally the larger the item, the faster it changes, unless it’s brand new. This is even visible at the site level: English Wikipedia sees about 100,000 changes per day, French has 20,000, and Farsi has 200. If your Trigger event is too high-velocity, occurring more than 50 times per hour, it is likely not a good fit for IFTTT, and you could get temporarily blocked by a Recipe’s Action’s service. If it’s too low-velocity, you might be disappointed with an otherwise sound recipe.

Some bots, like ClueBot, are very active, and would not play well with IFTTT. Other bots edit only a couple times a day, like SpaceFactsBot. It updates space-related world records in progress, and would be a fine IFTTT Trigger candidate. Tweet its edits (or your own), by filling in this recipe.

Edits with a Hashtag

Hatnote is a big fan of hashtagging on Wikipedia. Hashtags are the best way to unify edits with a common purpose across articles and editors. If you’re editing with friends, colleagues, or Editathon participants, easy-to-use hashtags build momentum and record your hard work.

> Fact #5: Set up earlier rather than later. This is hardly specific to Wikipedia, but whenever possible, setting up a recipe beforehand is way easier than retrieving the data later. Don’t forget to test them out to make sure they work! That’s what the Sandbox is for.

With a little bit of forethought and the Edits with a Hashtag Trigger, you can automatically build that record into a blog or other manifest. You can even push it to a Google Spreadsheet, and access that as a form of JSON API, giving you a whole API backend in no time.

Article added to a Category

We’ve saved the best for last, because now we’re getting into the most complex and exciting Triggers in this release. The Article added to Category Trigger is one of our favorites because it is so open-ended. You can get really creative with them. Just look at these Recipes:

• Call me if there is a new President of the United States
• Send me an email when a Wikipedia user earns a PhD
• If The Truth Is Out There, I Want To Believe

In case you hadn’t interacted with them before, Categories can be found by scrolling to the bottom of any Wikipedia article. An article’s associated Talk page often references administrative categories, which are interesting in their own right. In fact, due to technical limitations for this recipe and the next, Talk page changes are automatically resolve to their “main” namespace counterparts. Talk goes to the Article namespace, User talk to User, Project talk to Project, etc. It is not possible to monitor a Talk page exclusively.

Categories are full of promise, and we eagerly anticipate the community’s leveraging of this Trigger, but that promise comes with a price.

> Fact #6: Wikipedia Categories can be problematic. First, the caveats of the previous facts apply doubly to Categories. Category systems and conventions vary greatly between communities and languages. Some Categories are huge and change frequently, others are small or even empty, despite having broad-sounding names. Research is necessary.

>Second, Categories often assume an unpredictable tree structure, wherein parent categories do not automatically contain their subcategories’ members. Furthermore, cycles can exist. Finally, due to the way pages are added and removed to Categories, it is possible for a page to appear to be added to a Category multiple times, simply due to editing snafus. Do not actually rely on the Presidential phone call for your think tank’s next strategy summit.

Most of these complexities are manageable through other means. For instance, if one wanted to monitor a Category’s Subcategories, one would simply have to do a bit more mousework, and set up multiple Recipes to perform the same Action, as we’ve done in the examples for this next Trigger.

Edits to Articles in a Category

Some would argue the most complex Trigger, but for those who have been following along so far, you can probably guess exactly what this will and won’t do. Use the Edits to Articles in a Category Trigger to simultaneously monitor multiple articles related to a specific topic.

From Fact #4, we know that if the topic is too broad, the edit speed may outpace IFTTT’s effectiveness. From Fact #2 and #3 we know that Categories mean different things to different communities. From Fact #6 we know that a given Category may not always be what it appears. And finally, from Fact #1 we know that Categories don’t apply across languages, so we recommend disabling the language customization field for published Category-based recipes.

Add in the Talk-page caveat from the previous Trigger and it’s been a long road, but we’re already reaping the returns. For instance, using the Tweet about Wikipedia Updates in a Category Recipe, there are already three special-interest Twitter accounts running solely off Wikipedia with no specific programming overhead whatsoever:

• @LISedits: Tweets about Library Sciences. We’ve heard librarians love Wikipedia.
• @medstubs: Tweets about newly posted medical articles that could use contributions
• @wikibreakfast: Tweets about the most important meal of the day.

When it’s this easy, one might wonder where all the complexity went. Well, if you’re really curious about the technical details, read on.

Behind the Scenes

Hard at work at Hatnote HQ:

The Article Edit Trigger, User Edit Trigger, and Daily Triggers are based on basic queries to the Wikipedia API. The Wikipedia channel includes a daily update on the Wikipedia Article of the Day, Wikimedia Commons Photo of the Day, and the Wiktionary Word of the Day. All three of these feeds are also available as RSS feeds from the API.

The Hashtag Trigger and Category Triggers are all based on SQL queries to the Wikipedia database replicas that are openly available on Tool Labs. The service that does all the API calls, SQL queries, and serves all the IFTTT requests is written in Python.

The very astute might notice that all the Wikipedia IFTTT features are Triggers for reading as opposed to Actions for editing. While Wikipedia supports OAuth, there are existing community conventions and affordances for automated editing that require further consideration and discussion.

Thanks

Many thanks to Ori, Dario, Ed, Niki, Yuvi, Kirsten, and many others for their hard work and feedback during the Wikipedia+IFTTT development.

We hope you found the guide useful. If you build anything, tweet it out and CC us, our handles are linked below. Happy automating!

– Mahmoud and Stephen

New Weeklypedia Languages

Things are heating up in Hatnote land. First up we’ve got Weeklypedia updates, where we’ve added two new languages to our weekly update on editing activity.

So for you Spanish (hola!) and Farsi/Persian (!سلام)speakers, subscribe here:

• Spanish Weeklypedia
• Farsi Weeklypedia
  

We never spam and hate all that, this is just a summary, pure and simple. If it’s in the news, it’s on Wikipedia, and if it’s big news you won’t want to miss, it’s on Weeklypedia. Plus the interesting mix of all the culture and history you’ve come to expect from Wikipedia, too. Here’s an English example issue from last week. 

In case you’re feeling left out, feel free to submit a request to get your language added, and we’ll add it as soon as we can. Here’s a list of the previously supported languages:

• English
• German
• French
• Korean
• Estonian
• Swedish
• Danish
• Italian
• Catalan
• Esperanto (!)

Next up, Chinese and Urdu! BTW, how about that new site design Stephen built?

Not too shabby, eh?

{{Hatnote}}

Anonymous asked: Sounds like you guys have added bots to the mix. But now its a frantic strumming of activity because of so many bots crawling Wikipedia. Could you add in an option to filter out bots? I liked coming to hatnote for a nice, soothing background melody, but now its way too chaotic.

That’s an interesting observation! Bots have actually been supported from day 1, but you’re right, there is a much higher than usual bot activity right now.

For those that aren’t aware, many of Wikipedia’s more tech-savvy users write automated scripts to do a lot of the grunt work of editing and administrating Wikipedia. This could be anything from fixing typos to renaming categories to preventing vandalism. Every now and then there will be a large migration or bot-assisted editing effort, and Listen to Wikipedia will be rather cacophonous during that time.

So, I’ll see what I can do about adding that option this week, sorry for the inconvenience. For a more relaxing experience in the meantime I’d recommend listening to one or more of the other languages, as usually these traffic spikes only affect one language at a time, and only last for a few hours, maybe a day.

If for some reason you want to listen to a real botfest, check out the Wikidata “language”, which, last I checked, is over 95% automated edits!

Thanks for reaching out!

Mahmoud

{{Hatnote}}

Anonymous asked: Hi! Im contacting you on behalf of Wikimedia Finland. Would it be possible to include fiwiki in to the listen to wikipedia? We would like to use it with hashtags in our events! -Teemu Perhiö, WMFI.

Hey Teemu! Thanks for reaching out. We’ve just added Finland to Listen to Wikipedia, so feel free to hashtag away. Also, ping us on Twitter @mhashemi and @sklaporte so we can pass along the good word!

– Mahmoud

Photos for your tabs

Fill your boring empty Chrome tabs with beautiful photos with Commons Tab. It’s a Chrome extension that loads some of the best photographs from Wikimedia Commons (the free image repository for Wikipedia). You can filter the images to astronomy, birds, fortresses, and more. All of these photos are freely licensed or in the public domain, so you are welcome to remix or share what you discover. 

Source available on github, and let me know if you want to help port to FireFox.

If anything looks good, it’s thanks to Moiz Syed’s design help!

- Stephen

Cronfed: Minimum Viable Monitoring

Every project should be monitored. It’s common sense. We all know it. We’re all sensible developers. What is constructed must eventually fall, and without monitoring, we won’t know when or how to fix it.

And yet thousands of projects go without logging or monitoring, working fine until they don’t, preventably depressing countless others.

Cronfed is a Python package that embodies a **Minimum Viable Monitoring **mindset. Something is better than nothing, and there are far too many cron jobs with nothing. Whether lazy or busy, take 5 minutes and clear your conscience.

pip install cronfed

echo ”*/15 * * * * /usr/bin/python -m cronfed --output /var/www/mysite/assets/cronfed.rss /var/mail/myuser 2>&1 | tee -a /home/myuser/project/logs/cronfed.txt”

Just replace “myuser”, the “mysite” path, and the “logs” with the appropriate paths and you’ll be ready to point your feedreader or IFTTT at your site and commence breathing easier. Learn more on the Cronfed docs. (Or a bigger example crontab and output.)

If only all solutions were this easy and consequence-free.

We use Cronfed for Hatnote’s Recent Changes Map and Weeklypedia (which just turned 1! You should sign up, learn about Wikipedia, and experience the reliability firsthand!). Predictably, we are all very satisfied with a simple tool that does one job well.

{{Hatnote}}

Anonymous asked: I currently subscribe to both your English & Estonian version of Weeklypedia. I'm wondering if perhaps you could do something similar with "Requested Article" requests - for example, show the top x number of article requests made in the week and/or the top x number of requests overall. This would not only be interesting, but could also end up encouraging Weeklypedia subscribers to actually develop Wikipedia content if they see a request for a topic they'd be interesting in working on.

Hey! Thanks for subscribing! I think this is a really great idea, especially with regard to increasing engagement. Unfortunately, the conventions for requesting pages varies greatly across the different wikis and this would make it a major effort to add and support new languages. Some Wikipedias are small enough to just have one page (Estonian might be in this category), but English Wikipedia has a whole huge tree of very-loosely structured request: https://en.wikipedia.org/wiki/Wikipedia:Requested_articles

But again, that is a fantastic idea and would love to hear more. If we can build it, we will!

– Mahmoud

The Humble Hashtag, now on Wikipedia

Whether or not one approves of the term, hashtags are pretty much ubiquitous in the modern web. And why shouldn’t they be? They’re simple, interoperable, and most importantly semantic!

As of this week, we’ve added to L2W what may be the first instance of real hashtag support in the Wikipedia ecosystem. That’s a real screenshot above! Just scroll past the languages and start entering the comma-separated tags you want to follow. Then go edit and add hashtags. How, you say?

Whenever you make an edit, anonymous or otherwise, you can attach a comment or summary before you submit. Our livestream now searches for and includes any hashtags that an editor included in that field, allowing us to filter edits to just a select group.

We originally conjured up this functionality for Editathons, which are often theme-based periods of coordinated editing. Because these these gatherings of Wikipedians is often distributed, we felt there should be an interface that helps these groups highlight the impact they’re having among the normal Wikipedia edit activity. Honestly, we’re still working on the details of the feature, but we wanted it to be ready in time for the various #WomensHistoryMonth and #WomensDay editathons going on around the world this week and this month. (Art+Feminism being one.)

If all goes according to plan, in the near future we’ll be:

• Tweaking the hashtag interface to be a bit sleeker
• Adding @mention support to the UI (it’s already there in the Wikimon livestream)
• Releasing some tools for retrieving edit histories based on #hashtags and @mentions. We have a really basic one up here now. Example URL: http://tools.wmflabs.org/hashtags/search/blacklivesmatter. Hint though, there are virtually no hashtagged edits as of writing (Editors Note: by March 2016, there’s quite a few!).
  

Keep on editing!