Wikipedia:Editor engagement experiments

For more details on what editor engagement is, see Wikipedia:Editor engagement.

Editor engagement experiments are tests of ideas for attracting new editors and getting current editors to continue contributing. These experiments are being conducted by a team at the Wikimedia Foundation, with help from Wikimedians. The experiments team is a subset of the larger engineering and product development department. We're currently hiring too!

List of experiments

Our experimentation work officially started April 16, 2012. We also have a hub on meta explaining our work.

1. ✓ Done: A/B testing user talk templates to improve their effectiveness at communicating to editors
2. ✓ Done: Necromancy -- emailing lapsed editors and asking them to return
3. ✓ Done: Timestamp position modification to test whether a visible, linked, human readable timestamp improves edits on articles
4. ✓ Done: Editor milestones -- thanking editors here on English Wikipedia that have made their first 1,000 edits to articles
5. ✓ Done: Post-edit Feedback to test whether various feedback mechanisms improve editor activity
6. ✓ Done: Community Portal Redesign to test various redesigns to the Community portal that will make it a more useful space for editors
7. In progress: Account Creation UX to improve and streamline the account creation process; will be partnered with tasking and other experiments and will provide vital benchmark metrics'
8. In progress: Onboarding new Wikipedians -- onboarding, a term borrowed from human resources parlance, means to introduce brand new contributors to Wikipedia and show them how to edit. See also: Wikipedia:GettingStarted
9. In progress: Guided tours -- providing tooltip-like tours of the Wikipedia experience, see also Help:Guided tours

FAQ

So what exactly do you do?

We run short-term experiments on Wikipedia with the goal of making it easier for editors to contribute constructively to the encyclopedia. These experiments range from making small tweaks to existing processes to testing new methods of engaging readers and editors. When an experiment is completed, we use the results to inform how features currently planned or in development might work better. A few experiments might become new standalone features.

How is this different from other work at the Foundation?

Development projects like building a visual editor and fixing New Page Feeds are extremely important priorities for Wikipedia. But not all tasks or changes that make it easier for editors are so obvious. We believe we should try a myriad of different approaches to increasing participation; it's likely that only a few can and should be become permanent new parts of the Wikipedia experience. We will continue to experiment with ideas generated from the community, the Foundation, and elsewhere so that we can continuously improve Wikipedia.

Who at the Foundation is working on these experiments?

• Steven Walling, associate product manager
• Ryan Faulkner, data analyst
• Dario Taraborelli, senior research analyst
• Matt Flaschen, software developer
• Ori Livneh, software developer
• S Page, software developer
• Munaf Assaf, user experience designer

How can I opt-out of experiments?

Experimentation and testing are how we improve Wikipedia's experience as a service to editors. We highly recommend that even if you don't like a particular change you might see, you try it out and give us feedback about what you didn't like and what would serve your editing needs better. These are temporary trials, and will be removed by default when we have collected enough data to draw meaningful conclusions.

To permanently opt yourself out of feature experiments, you may do so at Preferences → Appearance → Exclude me from feature experiments. Some experiments may be directed at users who are unable (such as IPs) or unlikely (brand new registered accounts) to know about Wikipedia's many preferences, so this may not be an effective opt out for everyone.

How can I stay informed about these experiments?

Watchlisting this page is the simplest way of staying updated. We also make announcements on the technical Village Pump.

Where is the consensus to run test X?

One of our goals with this work is to give the movement more information about what we can actually do to increase participation in Wikipedia and the other Wikimedia projects. The consensus process will happen once we have enough data to make informed choices about whether we need to implement something long-term or permanently (via new features, policy changes, etc.). But there are so many things we just don’t know because we’ve only talked about trying them but never have -- so some of it we will figure out with the community along the way!

Here's some more detail on our thinking about why full consensus should only come after experiments run:

Lengthy discussion prevents rapid experimentation

In almost all cases, the consensus process would take longer than the duration of the actual tests we’d run (which will only be live for hours, days, or weeks at most, and almost always on a narrow slice of the site).

Consensus discussion would not reflect the views of readers, new contributors, or current contributors who don’t read meta spaces like the Village Pump

For the most part, editors and potential editors who do not currently participate in these meta spaces are the contributors we're trying to engage with. When it comes to topics like edit wars and AFD, having consensus among the people who bother to show up works. However, not all of the same rules apply when it comes to experimenting with new ways to participate in Wikipedia.

Many experiments will not be noticed by the community at all

It’s only by trying new things and measuring their effects that we can learn what works and what doesn’t: Many of these experiments will probably not show any measurable change in engagement on the projects, and that’s okay. Some experiments might succeed at impacting a few of our target variables, but not others. Some tests might yield complete surprises or overturn common assumptions. Regardless, every result will be a new and vital data point.

But what if I really hate an experiment and think it will harm the encyclopedia?

Please remember that our experiments are just that: tests to try and support a hypothesis about how we can get more people to help build the encyclopedia. Nothing we do will become permanent without experimental data to support it, so please be patient and let us know about any specific concerns you have.

If you do not want to see any new feature experiments, there is an opt-out checkbox for this in Preferences. If there are other ways of opting out of a particular test, we will publicize it.

Where do I go if I have comments, ideas, or further questions?

The best places to ask are (in rough order):

1. The associated talk page here
2. By asking Steven Walling on his user talk pages or via email. If you think a feature might be new, then be sure to check the technical Village Pump, where we post announcements about new deployments.
3. In IRC at #wikimedia-e3 ^connect. (We also do regular IRC office hours.)

See also

• WMF Executive Director Sue Gardner's announcement about the editor engagement experiments team
• Coverage in The Signpost
• Research and analysis documentation for experiments (Meta)
• Design and technical documentation for experiments (MediaWiki.org)