Beta Features/Hovercards

Wikimedia engineering activityWikimedia Engineering

Hovercards

Hovercards provides you with a short summary of an article whenever you hover a link to it.

Group:	Design
Start:	March 26, 2014
Team:	Prateek Saxena, Quiddity (WMF), Jdlrobson, Sherah (WMF)
Lead:	Jkatz (WMF)

The Hovercards beta feature solves the core problem of users needing to open multiple tabs to gain an understanding of a word in the context of the subject they are reading. With Hovercards, whenever a reader hovers over a link to another article, a short summary of the subject and an image (if available) is shown to them, so that they can decide whether they need to visit that subject more fully before continuing the current subject.

There have been many requests over the years for a similar feature; there are browser extensions (see list) and one heavily used gadget (the more editor-focused Navigation popups) that also exist to solve this problem.

The Hovercards settings panel enables logged-out users to disable it entirely. If Navigation Popups and Hovercards are enabled at the same time (typically, for logged in users who have opted in), Navigation popups takes precedence.

Intended users and objective

Our primary users are the casual readers who browse around the Wikipedia without any particular goal, simply enjoying reading the articles.

Hovercards will be of use to any user of Wikipedia who browses around the site frequently.

Our product will make it easier to get an overview of an article before you decide whether or not to browse to it. Consequently, the smoothness of the user experience for these users will be increased.

Example of a hovercard being viewed

FAQ

Why Hovercards?

This is a feature intended to improve the experience for any reader who normally would have clicked on a blue-link in wikipedia because they needed an overview (definition) of that entity.

How do we measure Hovercards performance?

The theory of impact for hovercards is that they lower the cost of exploring a link. This should mean that users are less inhibited about exploring links. We should see that the overall links clicked + (non-accidental) hovers exceed the number of links clicked without hovercards. This is what success looks like. There are also some indications of failure that we will look for:

• hovers can be accidental-we need to measure normal dwell time in a controlled condition to ensure that the likely rate of accidental hovers is not too high. To give an example, if a user must dwell on a link for 250 msecs before a hover shows, then we would want to make sure that there are not a large number of users who tend to dwell on a link for more than 250 msecs without clicking it.
• hovercards could lead to fewer pageviews, because the user gets the information that they need from the preview- this is not a problem, but we want to make sure that the decrease (if any) does not result in significantly less editing or fundraising.
• the % of hovers that result in a user continuing to the page is high - this would suggest that most people wanted to go to the page anyway. If this is the case, then the hover is likely just adding an unnecessary step. We expect some significant % of click throughs for hovers. It is ~60% on Android for a similar feature, but we expect it to be lower on desktop.

What if I have Navpops enabled by default?

If you have Navpops enabled, hovercards are automatically disabled. You will have to disable Navpops in order to experience hovercards. This is an intentional decision to ensure that Navpop users' do not have their preferences interrupted.

How many options do I have in Hovercards preferences?

Right now, hovercards will either be turned on or off by a user. The option to turn them off lives in each hover event. So at each hover event, a user can decide that they are no longer interested. In order for a user to turn hovercards back on, they will have to go to their user settings. This currently does not exist for logged-out users, so we are creating it!

At the project level, administrators can determine how long a user should dwell on a link before a hover is triggered. If a project wants to be conservative, the lag can be longer. WMF will have a recommendation as to optimum dwell time that provides the best user experience while minimizing accidental hovers.

Why can't users just turn on hovercards if they want them?

Unfortunately, there is no good way to tell users about a new feature. Central notice banners have been suggested, but running them for 2 weeks would not solve the needs of future users and we do not want to run them continuously. Notifying a user of a new feature is best done using a...hover-over. So that is why we are planning to show an initial hover on the users first hover event, followed by an explanation of what the user has seen. They can choose to turn them off in that dialogue.

If I have hovercards feedback or if I have a suggestion for making them better, where should I go?

Please go to the hovercards discussion page, here: Talk:Beta Features/Hovercards

Current design

• 
• 
• 
• 
• 
• 
• 
• 
• 

Key differences to Navigation Popups

Hovercards is styled to enable easy reading. Currently, Navpopups' type-size is small and the margins are tight.

• Create a consistent visual style with the article type treatment.
• Emphasize the lead images, so a user gets an idea of a term using both text and image.
• The action set in the gadget feels out of context, hence we need to validate which of those actions are useful for readers and editors.
• Include the last edited timestamp that we have tested on Wikipedia mobile web, but de-emphasize it.

Settings

Open/Close delay timing

You can change the timing of how fast Hovercards appear and disappear (open/close delay); see Extension:Popups#Show/hide timing for instructions.

Enable/Disable

If you turn Hovercards off via the cog-icon, you can turn it back on at the bottom of any wikipage with the "Enable previews" link.

Screenshot of page-footer

split this image into 3 images, and update with how it looks now

Logged-out users can turn Hovercards off via the cog-icon (settings saved in cookies or localstorage). Logged-in users who have the Navigation Popups enabled can also turn Hovercards off.

Hovercard settings dialogue. The "advanced mode" is not completed at the current time.

Future priorities

• Article titles
• Reference Tooltips (phab:T67114)
• Cross-wiki Hovercards (phab:T67117), and subsection links (phab:T65792), particularly for Wiktionary
• User names
• Integrate with Navigation Popups

Design mockups for:

• 

  Target object is a Wiktionary item

• 

  Reference Hovercard

Code

Main page: Extension:Popups

Analytics and instrumentation

• Hovercards are currently instrumented to measure many aspects of usage. Please see here for details and feel free to ask any clarifying questions

2016 a/b tests

Tracked in Phabricator
Task T139319

In 2016 we enabled a/b testing on hovercards so that we could look at hovercard behavior and see how that impacts learning, by comparing the sessions of users with hovercards auto-enabled and the sessions of users without hovercards auto-enabled.

Here is how we intend to interpret the data:

Are Hovercards good for our users:

1. Rate of hovercards being valuable v. disruptive
   1. Do the number of links hovered per page increase or decrease?
   2. Ideally we would measure if hovercards make a user to spend more time with us (where time is a proxy for learning) by measuring impact on time per session or sessions per user, but this might not be possible.
   3. What is the rate of erroneous triggers?
      • Estimated by taking the dwelledbutAbandoned time length in control group and seeing how many are above the trigger time.  Apply # of these per page to the number of hovers in experimental.  That’s at least my best guess.
2. What % of people dislike hovercards?
   1.  % of sessions where people disable hovercards?
   2. ^ over time
   3. Distribution of # of hovers before someone disables?
3. Impact on pageviews and fundraising
   1. What is the impact on links or hovers clicked of being in the experimental group?
   2. What is the impact on session depth of being in the experimental group?
   3. What is the impact on $ donated of being in the experimental group?
4. Getting in the way of people who just want the next article:

1. 1. If 100% of the time, people continue onto article after seeing a hover, this is bad.  We want to keep the ratio below 70%.
   2. If 0% of the time people continue onto the article, that is also a sign that we are making it too hard.  We want to keep this ratio above 10%
      • Of hovers where user continues onto article

Descriptive:

1. How long do people spend hovering before they click?
2. How long do people spend hovering before they dismiss the hover?

Diagnostic:

1. Is it ever the case that “perceived” wait (amount of time before hover shows) exceeds “popup delay” (amount of time to trigger) by more than 200 ms?  What % of the time?
2. Frequency of error states (per hover)

Results

Extension:Popups/Fundraising test 1 - Results from the first A/B test run on the Hungarian Wikipedia during a fundraising campaign.

2015 Greek and Catalan Wikipedia test

We ran a test where hovercards were turned on by default for Greek and Catalan Wikipedia for a period of 4 months. We have both quantitative and qualitative data for this experiment.

Here is a list of major bugs reported.

Known & documented issues:

• Cutting sentences randomly
• Info in parenthesis ignored (https://phabricator.wikimedia.org/T91344 )
• Period problem with TextExtracts  (send email to Gergo and Prateek)
• Hovercards not going away

Unknown and need more information:

• Text alignment issue in certain scripts
• Disambiguation text showing in some cases - this is when a page is just using plain indented/italic text instead of a template (e.g.). Or, if the hatnote template doesn't include class="noexcerpt".
• Half sentences are annoying, people want more text 
• People are unhappy with animation speed, some want it faster, some slower (replicating the existing Reference Tooltips options, would be best.

Known and hard to solve for

• Issues with PageImages (…)
• Map shown without dot (This should be fixed once Max and Yuri are done with the tile server)
• Reference hovercards (user and special pages too) (Prateek and Bene* are working on a restructure)
• Sub-section links (TextExtracts)
• Make it work with Wikidata (https://phabricator.wikimedia.org/T69434 )

Qualtrics Survey Results

In April of 2015, hovercards were rolled out to all users on Catalan and Greek Wikipedias. They also continued to be a beta feature on english and other wikipedias. During this time, a survey was shown to all hovercards users using a banner recruitment. There were over 1,700 responses, heavily dominated by non-logged in users on Catalan and Greek and logged in users (probably mostly editors) on English. Below are the aggregated results

For first question, users mostly gave positive feedback, but a good amount of issues identified

Questions 2-6 are the qualitative text answers to whatever the user chose for open feedback. We cannot share these answers on wiki, as they have not been vetted to remove potentially personally identifiable information.

Here are some random, contiguous sample rows from those answers:

• bug:

no se'm tanca una pestanya

Speaking people

elsexe

Hovercard doesn't go away Browser: Google Chrome Operating System: Win 7

ghgh

• other problem:

Encara que deixi desactivades les vistes prèvies i guardats els canvis cada cop que tanco l'explorador i el torno a engegar les vistes prèvies tornen a estar activades. Utilitzo Firefox amb cookies activades. Gràcies. (Catalan. Google translate says: Although longer disabled and previews the changes are saved each time you close the browser and start again previews are back on. I use Firefox with cookies enabled. Thank you." )

δεν μου αρεσουν και με μπερδευουν (Greek. Google translate says "Dislikes and confused"

- |The hover card, doesn't close.

Hover card for Edvard Munch not showing correct information when linked from page https://en.wikipedia.org/wiki/Norwegian_Air_Shuttle_/_Norwegian_Long_Haul_tail_art Only disambiguation info shown. The problem is with the hover card, as it appears on other articles where 'Edvard Munch' is linked. I am not able to fix this problem.

• positive feedback:

!que sexplica molt be el text. (Catalan, Google Translate says "Explains very well the text")

διότι δεν έχασα χρόνο να ανοίξω άλλη καρτέλα στον φυλλομετρητή μου (Greek. Google Translate says "because I did not lose time to open another tab in my browser"

És útil per saber si l'enllaç ens porta al contingut que ens interessa. (Catalan. Google Translate says "It is useful to know if the link leads to content that interests us."

These tooltips are awesome!

perfect :)

It is very convenient!

• suggestion for improvement:

The preview card should disappear INSTANTLY (less than 0.01 seconds) when the mouse cursor is moved away form the hyperlink text. My only guess for giving the preview card some time to disappear is so that the user can move the cursor to the preview card itself and click it to be linked to that article. However, this is unnecessary, (in fact the preview card itself doesn't need to be clickable/linked at all) the cursor is already on the link in the text, ready to click on spot, there is no need to give it time to move to the preview card (and again no need for the card to be clickable).

αυτή η σημαία δεν υπήρχε τότε , αναφερόμαστε στο 1200 (Greek. Google translate says: "this flag was not then referring to 1200" ??

Wikipedia should optionally allow displaying high-quality, low-compression (PNG or 95%+ quality JPEG) versions of images in hover cards on higher-speed connections.

Treu-re el link. No serveix per res (Catalan. Google Translate says "Remove the link. Is useless"

I'd like to know what happened in these epoch.

Users find hovercards easy to use!

Users find hovercards useful!

Users enjoy using hovercards

Users find hovercards simple and easy

Project-level rollout discussions

Here is where the potential rollout of hovercards is being discussed:

English Wikipedia: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)/Archive_131#Proposal:_Enable_Hovercards_by_default

See also

• Navigation popups - the editor-focused gadget, started in 2005 by Lupin, available at most wikis
• WMF blog post - Hovercards now available as a Beta Feature on all Wikimedia wikis (March 2014)
• The original specifications can be seen at File:NavigationPopups V1.pdf and hovercards-phase-3 trello notes.
• A list of similar browser extensions