Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

Technology

News and information from the Wikimedia Foundation’s Technology department (RSS feed).

Preparing for the migration from the Wikimedia Toolserver to Tool Labs

Last week-end, the Wikimedia Hackathon took place in Amsterdam, and we notably worked on the migration from the Wikimedia Toolserver to Tool Labs. According to the Roadmap, Tool Labs will have all the necessary features by the end of June 2013. As of then, tool maintainers will have one year to migrate their software. By the end of 2014, the Toolserver will be decommissioned.

What is working?

First of all: Tool maintainers do not have to care about virtual instances or other background stuff. You will develop on servers similar to the Toolserver, e.g. with the infrastructure for web services. The servers are running Ubuntu Precise. At the moment, there are replicas of six (out of seven) database clusters. The last one (with CentralAuth) is due in June. You can already work with all the big and many small Wikipedias, with Commons and Wikidata. You have access to all the data that is visible to registered users without special privileges in a wiki. You can create your own user databases. According to the first experiences, Tool Labs is fast.

In addition to home directories, you also have shared project storage. Tool Labs wants to make it as easy as possible to develop software together, which is why you can add others to your projects via the web interface. There is also a time travel feature! You can reset your files to the state of the last three hours, the last three days and the last two Sundays. The job system that is used in Tool Labs is OpenGridEngine. You can find an intro on the Tool Labs help page. Bugs can be reported in Wikimedia’s Bugzilla: Please use the product “Wikimedia Labs” and one of the components “Tools” or “Bots”. If you miss software in Tool Labs that could be of interest for others, too, please file a bug!

What about “Tools” versus “Bots”?

These are the names of the two projects Tool Labs consists of. The larger environment (Wikimedia Labs) is organized in projects, two of which form Tool Labs. They are an environment inside Labs that is customized for Toolserver users. The naming might be a bit misleading: The difference between “Tools” and “Bots” is not what you run in which project, but that you can run your tools in two different environments. The “Tools” project is a stable environment maintained by four admins (one of them a volunteer). There are no experiments with software versions here. In contrast to this, the “Bots” project is a more flexible environment in which you can play with changes in the environment itself, too. Here, it will be easier to get root access. (If you are interested, ask on the mailing list.)

Open tasks?

Apart from the open tasks on the roadmap, the documentation needs improvement. The pioneers among you can help others a lot by documenting experiences. Magnus Manske and Russell Blau have started to lead by example by adding a lot of documentation, and you can help as well! We are thinking about how to redirect deprecated links to migrated tools in the easiest way possible. The Tool Labs user interface also needs some love; feel free to come to us if you want to help here!

If you run into problems or have questions when migrating tools, be bold and ask! The best places are the labs-l mailing list or the IRC channel #wikimedia-labs connect. The admins’ nicks are Coren and petan. There is also a list of Frequently Asked Questions that you can expand. And finally: If you find that your tool needs more adaptation than you think you can manage on your own, talk to Johannes Kroll or myself at Wikimedia Deutschland for support!

Silke Meyer
Projektmanagerin für den Toolserver, Wikimedia Deutschland

Test features in a right-to-left language environment

Wikimedia sites are facing many technical changes, like VisualEditor, Wikidata, Flow and Echo, just to name a few. As an ordinary Wikipedian, I like it very much and I’m pretty excited, but sometimes change is scary, especially for people who are working on “small” wikis. People constantly ask “Is this feature localized for my wiki?” or “Will it work properly?” and if you’re using a right-to-left (RTL) language (like Persian, Hebrew, Arabic, etc.), an otherwise great feature can quickly become a nightmare.

Sadly, it is difficult to test these features in your language before they are enabled on Wikipedia. Even if you’re an experienced MediaWiki developer, and you can install a wiki on your own server and add these features to test them, reporting bugs is hard because a locally-hosted wiki isn’t accessible to the rest of the world.

This is why we’ve set up a public test wiki dedicated to RTL languages. Wikimedians can come and test upcoming features, and see which interface messages are translated incorrectly. They can report bugs easily, communicate with other Wikimedians who are working in RTL languages, and work with them on features. Maybe you want to be sure a feature works properly in your language, or maybe you’re just curious and you want to know what Wikipedia will look like in the future. In both cases, the RTL test wiki can help you.

You can visit the RTL test wiki at http://tools.wmflabs.org/wikitest-rtl/w/ ; we have already installed some upcoming features, but if you think something is missing, feel free to contact me or Amir Aharoni and ask us to add it. This wiki uses the Universal language selector, which means that when you open it, the language of the interface is automatically set to the one your browser requests, and you can easily change it.

We hope to see you soon on the RTL test wiki.

Amir Sarabadani (User:Ladsgroup), editor on the Persian Wikipedia and pywikipedia developer

Go on a Wikipedia scavenger hunt with Wikipedia Nearby

This post is available in 2 languages:
Español 7% • English 100%

English

The Nearby feature in Vatican City

We are quick to use mobile applications to find places that fulfil our needs, whether it’s a place to grab a coffee that your friend recommends, where the nearest bus stop is, or where to go for a perfect first date.

But how hard have you thought about the history of your neighborhood, or the events that have shaped the place where you live? There is a wealth of information all around you as you read this blog post. There are historic sites, parks, museums, theatres, cafes and religious buildings. Thanks to the terrific work of our editor community, Wikipedia has accumulated a massive amount of location data associated with its millions of articles; until now we have not fully taken advantage of this information.

We are happy to announce that the Wikimedia Foundation mobile team has been working on a Nearby page to surface this information. Along with the goal of bringing awareness of the surrounding areas to our existing readers, we hope that this simple tool can attract new editors to these articles, whether it is to update the information on the exhibits in a local museum, or simply to add a photo of a nearby park that is in severe need of a properly licensed lead image.

Look out for the camera icon to show those articles that need your photos

As a first pass, the mobile team has focused on using the Nearby page to surfaces articles in close proximity that lack images, inviting users to add one. Upon visiting those pages, the user will be prompted to illustrate the article, which they can do quickly and easily if they’re on a mobile device that supports taking and uploading photos.

The Nearby feature, although designed with the Wikipedia mobile experience in mind, also works on the desktop version of Wikipedia. In the future, we envision this as a useful step in the editing onboarding process, helping new users learn about editing by encouraging them to improve an article on a topic nearby.

Help make Wikipedia more beautiful, vibrant and educational for all our readers! Explore your local area and find the pages near you that need a photo or updated information. Stay tuned to the Wikimedia blog for more opportunities to contribute via the mobile web, coming soon.

Jon Robson
Software Engineer, Mobile
Wikimedia Foundation

(more…)

Developing Distributedly, Part 1: Tools for Remote Collaboration

The mobile web engineering team at the Wikimedia Foundation confronts a significant question every day: How does a team stay synchronized and productive when teammates are scattered around the globe? Our team is highly distributed and effective communication is a challenge. We focus on some of the highest priority engineering areas for the WMF, and at any point our team may include members in California, Colorado, Arizona , Russia, India, Poland, and the United Kingdom. To help us clear our geographic and temporal hurdles, we embrace a culture of continuous improvement in the ways we engage with one another and our work.

For many, having a collocated team is critical for success. We, on the other hand, see big advantages in our geographic distribution. The diversity of opinions, world experiences, and cultural knowledge that we can bring to the table through global distribution helps us better cope with some of the challenges we face in developing easy-to-use and accessible tools for a global audience. Of course, we might achieve this by generally hiring people from around the world, but the pool of exceptional recruits is much bigger when you don’t have a relocation requirement.

Similarly, having team members in other locales gives us greater exposure to people using our products on devices you might not commonly find in the US. Our geographic distribution also gives us closer to around-the-clock coverage in case of emergencies. Finally, building support for working remotely into the team gives us all a huge amount of freedom; any of us can travel (for business, pleasure, emergencies, etc) and not have to worry about falling out of sync with the rest of the team.

We make it all work with a multitude of communication tools, organizational practices, and discipline. While sometimes it can feel like there is no real substitute for face-to-face communication, our approach to these challenges has given us great strength, freedom, and resilience. The list of tools we use is long, and many of them are useful even for collocated teams. I will cover a few of the most crucial in detail below.

Video conferencing

The mobile web team using Google Hangout for a regular meeting where nearly every participant is in a different location.

Due in part to its simplicity, we rely on Google Hangouts for all of our regular meetings and for the occasional ad-hoc meeting where asynchronous communication will not do. In order to be effective, it’s important for video chat tools to stay out of the way so we can focus on communicating rather than wrestling with technical complications. Of the many tools we use, only one closely approximates in-person, face-to-face communication: group video chat. It’s the highest-bandwidth (yeah, pun intended) tool at our disposal and when used right, it can make all of the distance between us melt away.

Hangouts does this very well; it highlights the video of the person speaking, mutes your microphone when you’re typing, and screen-sharing is very simple. Audio and video quality adjusts automatically based on your bandwidth, and toggling your camera or mic on/off is trivial. The ‘effects’ feature lets you dress up participants in things like pirate hats and mustaches, making meetings more fun and helping the miles between participants disappear. Oh, and it’s free if you use GMail or Google Apps.

We have been iterating to find the best hardware setup for supporting Google Hangouts in the office in as unobtrusive a way as possible. While the mobile web team and other distributed teams have been working hard for improvements, the IT department has been doing an outstanding job helping to find and implement what works. We’ve dedicated one conference room to be our test subject and we are very close to finding an excellent set up. It’s unclear whether this will work well in our other conference rooms (do not underestimate how challenging room acoustics can be!), but we have come a long way from wasting half of our meeting time trying to get the technology to work for remote teammates.

(more…)

Getting ready for ULS everywhere

The Wikimedia Language Engineering team recently completed their latest development sprint, with a special focus on preparing for the upcoming deployment of the Universal Language Selector (ULS) extension on multiple wikis. The team also hosted a ULS-specific office hour on May 8, 2013 (logs).

ULS deployment prep

The Language Engineering team is working on refining several important features of the Universal Language Selector. This extension will provide an umbrella of services including selection of UI language, input tools and fonts. ULS will superannuate Narayam and Webfonts to provide a unified solution for configuring language settings for MediaWiki. During this development sprint, critical bugs related to positioning of ULS’ activation area and its “cog icon” label were fixed. These affected multiple MediaWiki skins and interlanguage wiki pages. The improved version will be deployed over several phases. More information about the upcoming deployment can be found in the deployment schedule.

ULS testing

ULS features are to be verified based on the test scenarios identified. These scenarios, based on the Cucumber framework, can be adapted for automatic as well as manual testing. The scenarios cover core features of ULS: triggers, language settings panel, display settings, font selection and input tools selection. These have been written in a simple “Given-When-Then” format and provide the steps for easy walkthroughs. The testing instance hosts all the latest updates that are being made. The team is looking for volunteers who can help us with testing and reporting bugs. Let us know if you would like to join and help (write to runa at wikimedia dot org or ping us on #mediawiki-i18n) .

What’s next

The team will be completing all feature changes and testing them by end of the current sprint to be ready for kicking-off the roll-out of phase 1 of ULS. Roll-out will be coordinated by Niklas Laxström with administrators of all scheduled wikis. The team will also be hosting a bug triage session on May 29, 2013 on freenode.net IRC on the #mediawiki-i18n channel.

ULS is live on Commons!

Meanwhile, based on consensus reached by the Commons community, Universal Language Selector and the Translate extensions have been enabled on Commons.

For more details about the Language Engineering projects and ways to participate, please write to me [runa at wikimedia dot org] or ping us on #mediawiki-i18n.

Runa Bhattacharjee, Outreach and QA coordinator, Language Engineering

Request for proposals: MediaWiki release management

MediaWiki, the software that powers most of the Wikimedia movement, is an amazing piece of technology. It brings the power of a wiki-world to millions of people. Not only those who are amongst the 500 million who visit a Wikimedia movement site each month, but also those who participate on one of the countless other wikis it powers.

MediaWiki is being used in all kinds of environments, from internal and private corporate wikis to other Free Culture wikis. That is, of course, the great benefit behind Free and Open Source Software; the software can be modified and used in new situations the original authors didn’t necessarily expect.

Because the Wikimedia Foundation wants the MediaWiki project to be as healthy as possible, and also address the needs of as many different constituencies as possible, the Foundation invests a lot of time and effort into ensuring the entire MediaWiki community feels empowered, not just those that happen to have an @wikimedia.org email address. You can see this effort most notably from the Engineering Community Team and the efforts especially around volunteer coordination and outreach.

To encourage further outside investment in MediaWiki, we are opening a Request for Proposals (RFP) (PDF) for the release management of MediaWiki. The long-term goal of this effort is to jump-start these activities as community-supported functions, thus encouraging widespread leadership in the future of MediaWiki.

The process for this RFP is a community-involved one. There is a three-week period for organizations to prepare and submit their proposals, after which the community can comment on and ask questions of the proposers. The Wikimedia Foundation will take all of this feedback into account when making the final decision for who will lead the release management of MediaWiki for the next year.

With this, the future of MediaWiki looks bright, and we’re excited to see where this will lead us!

Greg Grossmeier, Release Manager
Rob Lanphier, Director of Platform Engineering
Wikimedia Foundation

Help us design our next-generation discussion system

This post is available in 2 languages:
Español 7% • English 100%

English

Flow logo.png

Flow is a planned improvement to discussion and collaboration in the MediaWiki software. The project is currently in the design phase and the Wikimedia Foundation is actively seeking feedback and suggestions about how to make the best possible product for you, our community of contributors.

There are many reasons for us to revamp our discussion and collaboration system, but for now we’re going to focus on three primary use cases:

  • Users expect and deserve a modern and intuitive discussion interface: Talk pages—as a discussion technology—are antiquated and user-hostile. Experienced editors lose a lot of valuable time dealing with people who can’t figure out how to reply to messages or who need assistance with things like signing their posts.
  • Users are surprised by the cultural norms of the community: Many things about the culture that has grown up around talk pages (such as “talkback” templates or being able to change other people’s comments) are confusing or inefficient.
  • We believe that a modern user-to-user discussion system will improve the projects: Better methods for collaboration will improve collaboration, which will help good editors be more productive.

We have set up three “portals” where people can go to get more information and to leave feedback. At these portals, you will find links to detailed user tests, use cases and other kinds of research, as well as visions about what Flow can become.

The portals are:

In order to help you get a feel for how Flow might work, we’ve built an interactive prototype. Please note that this is only a demo and is not reflective of the final product, which may look and behave entirely different. Nothing will be saved on the prototype and you can’t really break it so have fun!

Please read more about the Flow prototype, what it can do, what is planned and known issues with it at one of these locations:

How can I help?

I’m glad you asked.

We are actively seeking feedback of all kinds from our user community. This is not restricted to our editors — readers are welcome to comment, too! There are many, many use cases and scenarios that we want to account for and we need to know what those are. We’re interested in hearing about anything that is important to you. Your concerns, your enthusiasm, your ideas for features.

It is very important for us that you be involved in helping to design this bold step, so please go to the portal of your choice and get involved in the conversation.

Brandon Harris, Senior Designer, Wikimedia Foundation
(more…)

Updates from Language engineering: changes to the Language Selector, new Extension Bundle release

In the recently concluded development sprint, the Wikimedia Language Engineering team made a new release of the Mediawiki Language Extension Bundle (MLEB), fixed bugs related to the Page Translation feature in Translate UX (TUX) and began work on design changes for the Universal Language Selector (ULS). The team also hosted a bug triage session that was well attended.

Input Settings from the ULS Language Settings Panel

Universal Language Selector Design Changes

Development and design changes have been initiated for the Universal Language Selector. The option to position the extension’s main panel in the sidebar was added and this feature is now being polished. Changes to the layout of the Language Settings dialog have been initiated, and usability tests for the proposed design changes were also done.

Using Wikimedia’s default GeoIP locator, ULS can now infer the user’s location and suggest language preferences.

MLEB Release

The April release for the Mediawiki Language Extension Bundle (MLEB) was announced by Amir Aharoni. Starting with this release, MLEB is no longer compatible with MediaWiki 1.19. MLEB 2013.04 and its later versions can only be used with MediaWiki version 1.20.4 or above.

The notable changes include update to CLDR v.23, bug fixes to further stabilize TUX and design changes for the Universal Language Selector. An experimental feature to present a restricted translation environment for new translators was developed for TUX. This is not enabled by default. Basic support for the XLIFF file format has also been added to Translate.

Up Next

During the next development cycle, the team will complete the changes to the Universal Language Selector design and test the features. The team is also participating in Google Summer of Code (GSoC) and the Outreach Program for Women (OPW), and will be working on completing the tasks in the next stages of the programs. More information about the other open projects for internationalization can be found in the master list.

The next Language Engineering office hour will be held on 8 May 2013 at 17:00 UTC (10:00 PDT) in #wikimedia-office on Freenode IRC.

Runa Bhattacharjee, Outreach and QA coordinator, Language Engineering

Notifications launch on the English Wikipedia

Notifications inform you of new activity that affects you on Wikipedia — and let you take quick action.

We’re happy to announce this week’s release of Notifications on the English Wikipedia.

Notifications inform users about new activity that affects them on Wikipedia and other Wikimedia projects, such as talk page messages, page reviews or edit reverts. It also lets them take quick action to respond to these events.

This new notifications system (formerly called Echo) was developed by the Wikimedia Foundation’s editor engagement team, to encourage people to participate more actively on MediaWiki sites (see earlier post). It provides a modern, unified user experience that replaces or augments existing notification systems — and gives significantly more control to users.

Here’s a quick overview of this new engagement tool.

How do notifications work?

When someone takes an action that relates to you on a Wikipedia or MediaWiki site, a red badge shows up next to your user name, with the number of unread notifications. Clicking on that badge displays a flyout listing the most recent notifications (see screenshot). You can then click on the notification of your choice to learn more and take action.

This first release features a variety of notifications:

  • Talk page messages: when a message is left on your user talk page;
  • Mentions: when your user name is mentioned on a talk page;
  • Page reviews: when a page you created is reviewed;
  • Page links: when a page you created is linked;
  • Edit reverts: when your edits are undone or rolled back;
  • Thanks: when someone thanks you for your edit (coming soon);
  • User rights: when your user rights change;
  • Welcome: when you create a new account;
  • Getting started: easy ways for new users to start editing.

These notifications were created to support the needs of both new and experienced users. For example, new users who create an account receive special Welcome and Getting started notifications to guide them in their critical first steps on Wikipedia. A special Thanks notification lets experienced users give positive feedback to new users who made constructive edits, to encourage them to contribute more. And power users will benefit from the User rights notifications (which are sent when your user rights are changed) and Mentions (sent when someone mentions your name) — two features that were found useful by active editors we consulted for this project.

To learn more about notifications, visit this FAQ page. To customize your notifications, check your preferences on the English Wikipedia. Once you’ve received your first notifications, please take this quick survey and join the discussion on this talk page.

Next steps

During the next few weeks, we plan to fix bugs and tweak Notifications based on community feedback. We are working on a few more features for our next release, such as alternative displays of talk page messages, more visually appealing HTML emails and new ways to dismiss notifications you don’t want. Over time, we would also like to develop more notifications for both new and power users. If you have any suggestions for improving this tool, please let us know :).

Once Notifications have been improved and fully tested on the English Wikipedia, we plan to make this product available in more languages on other Wikipedias and sister projects. In parallel, we will start providing tools and guidelines to allow notifications to be extended by developers.

Thanks

We’d like to take this opportunity to thank some of the people who made this product possible. They include Ryan Kaldari, Benny Situ, Luke Welling, Vibha Bamba, Oliver Keyes, Brandon Harris, Steven Walling, Matthew Flaschen, Dario Taraborelli, Howie Fung, Terry Chay and Erik Moeller, to name a few of our colleagues. We’d also like to thank all the community members who have guided our development and everyone else who pitched in to help us bring this tool to life!

We look forward to continuing these collaborations in coming months and to helping engage millions of Wikimedia users to share free knowledge more productively.

Fabrice Florin, Product Manager
Wikimedia Foundation’s Editor Engagement Team

Wikimedia engineering April 2013 report