Beta Features/Hovercards

Wikimedia engineering activityWikimedia Engineering

Hovercards

Hovercards provides you with a short summary of an article whenever you hover a link to it.

Group:	Design
Start:	March 26, 2014
Team:	Prateek Saxena, Quiddity (WMF), Jdlrobson, Sherah (WMF)
Lead:	Jkatz (WMF)

The Hovercards beta feature solves the core problem of users needing to open multiple tabs to gain an understanding of a word or concept within the context of the subject they are reading. With Hovercards, whenever a reader hovers over a link to another article, a short summary of the subject and an image (if available) is displayed. The user can then decide whether they wish to visit that subject more thoroughly before continuing with the current article.

There have been many requests over the years for a similar feature; there are browser extensions (see list) and one heavily used gadget (the more editor-focused Navigation popups) that also exist to solve this problem.

The Hovercards settings panel enables logged-out users to enable and disable the feature based on their preferences. Logged-in users can currently enable and disable the feature from their beta settings. If Navigation Popups and Hovercards are enabled at the same time (typically, for logged in users who have opted in via the beta feature), Navigation popups takes precedence. To enable hovercards, navigation popups must be disabled.

Intended users and objective

Hovercards will be of use to any user of Wikipedia who browses around the site frequently. For casual readers, hovercards will make it easier for users to get an overview of an article before deciding whether or not to browse to it. Users interested in reading an entire article in its totality will not be distracted or discouraged by viewing unfamiliar concepts - they can simply preview the concept without navigating to a new page. Consequently, the smoothness of the user experience for these users will be increased.

Example of a hovercard being viewed

FAQ

Why Hovercards?

This is a feature intended to improve the experience for any reader who normally would have clicked on a blue-link in wikipedia because they needed an overview (definition) of that entity.

How do we measure Hovercards performance?

The theory of impact for hovercards is that they lower the cost of exploring a link. This should mean that users are less inhibited and more focused when exploring links. We should see that the overall links clicked + (non-accidental) hovers exceed the number of links clicked without hovercards. This is what success looks like. There are also some indications of failure that we will look for:

• hovers can be accidental-we need to measure normal dwell time in a controlled condition to ensure that the likely rate of accidental hovers is not too high. To give an example, if a user must dwell on a link for 250 msecs before a hover shows, then we would want to make sure that there are not a large number of users who tend to dwell on a link for more than 250 msecs without clicking it.
• hovercards could lead to fewer pageviews, because the user gets the information that they need from the preview- this is not a problem, but we want to make sure that the decrease (if any) does not result in significantly less editing or fundraising.
• the % of hovers that result in a user continuing to the page is high - this would suggest that most people wanted to go to the page anyway. If this is the case, then the hover is likely just adding an unnecessary step. We expect some significant % of click throughs for hovers. It is ~60% on Android for a similar feature, but we expect it to be lower on desktop.
• the percentage of users who disable hovercards is low (given that users are aware on how to disable the feature) - this suggests that users enjoy the feature

What if I have Navpops enabled by default?

If you have Navpops enabled, hovercards are automatically disabled. You will have to disable Navpops in order to experience hovercards. This is an intentional decision to ensure that Navpop users' do not have their preferences interrupted. Note: for certain browsers, you might have to clear your browser cache first for the change to take place.

How many options do I have in Hovercards preferences?

Right now, hovercards will either be turned on or off by a user. The option to turn them off lives in each hover event. So at each hover event, a user can decide that they are no longer interested. For logged-in users, hovercards may be re-enabled from user settings. For logged-out users, hovercards may be enabled by selecting the "Enable Previews" link at the bottom of the page.

At the project level, administrators can determine how long a user should dwell on a link before a hover is triggered. If a project wants to be conservative, the lag can be longer. WMF will have a recommendation as to optimum dwell time that provides the best user experience while minimizing accidental hovers.

Why can't users just turn on hovercards if they want them?

Unfortunately, there is no good way to tell users about a new feature. Central notice banners have been suggested, but running them for 2 weeks would not solve the needs of future users and we do not want to run them continuously. Notifying a user of a new feature is best done using a...hover-over. So that is why we are planning to show an initial hover on the users first hover event, followed by an explanation of what the user has seen. They can choose to turn them off in that dialogue.

If I have hovercards feedback or if I have a suggestion for making them better, where should I go?

Please go to the hovercards discussion page, here: Talk:Beta Features/Hovercards

Current design

• 
• 
• 
• 
• 
• 
• 
• 
• 

Key differences to Navigation Popups

Hovercards is styled to enable easy reading. Currently, Navpopups' type-size is small and the margins are tight.

• Create a consistent visual style with the article type treatment.
• Emphasize the lead images, so a user gets an idea of a term using both text and image.
• The action set in the gadget feels out of context, hence we need to validate which of those actions are useful for readers and editors.
• Include the last edited timestamp that we have tested on Wikipedia mobile web.

Settings

Open/Close delay timing

You can change the timing of how fast Hovercards appear and disappear (open/close delay); see Extension:Popups#Show/hide timing for instructions.

Enable/Disable

If you turn Hovercards off via the cog-icon, you can turn it back on at the bottom of any wikipage with the "Enable Previews" link.

Screenshot of page-footer

split this image into 3 images, and update with how it looks now

Logged-out users can turn Hovercards off via the cog-icon (settings saved in cookies or localstorage). Logged-in users who have the Navigation Popups enabled can also turn Hovercards off.

Hovercard settings dialogue. The "advanced mode" is not completed at the current time.

Future priorities

• Article titles
• Reference Tooltips (phab:T67114)
• Cross-wiki Hovercards (phab:T67117), and subsection links (phab:T65792), particularly for Wiktionary
• User names
• Integrate with Navigation Popups

Design mockups for:

• 

  Target object is a Wiktionary item

• 

  Reference Hovercard

Code

Main page: Extension:Popups

Analytics and instrumentation

• Hovercards are currently instrumented to measure various aspects of usage. Please see the popups extension page for details and feel free to ask any clarifying questions

2016 a/b tests

Tracked in Phabricator
Task T139319

In 2016 we enabled a/b testing on hovercards on Hungarian, Italian, and Russian wikipedias to determine the impact of hovercards on learning. The test was structured to compare the sessions of users with hovercards auto-enabled and the sessions of users without hovercards auto-enabled.

Analysis

Are Hovercards good for our users:

Rate of hovercards being valuable v. disruptive

1. Do the number of links hovered per page increase or decrease?
2. What is the rate of erroneous triggers?
3. Is the usage of the back button reduced?

What % of people dislike hovercards?

1. Percent of sessions where people disable hovercards?#
2. Percent changes over time#
3. Distribution of hovers before someone disables?#

Impact on pageviews and overall engagement

1. What is the impact on session length? (control vs test)
   1. Define page actions as:
   2. Hovering a page
   3. Opening new (in same tab or new window)
   4. Compare page actions in control vs test group
2. What is the impact on links or hovers clicked? (one possibility to quantify this: compare the average number of clicks per page view session - i.e. the sum of all three click actions counted in the schema - divided by the number of pageLoaded events, or the number of different pageTokens recorded for all events, within a certain timespan)
3. What is the impact on session depth (with regard to page views)? (session depth defined as e.g. the number of different pageTokens recorded for that session ID within a certain timespan. Compare averages first
4. What is the impact on the number of link interactions (as defined in the schema) per session?
5. Same for the sum of page views (= session depth) and non-click link interactions (i.e. those that don't result in a new page view anyway), as a rough measure of overall engagement

Are they getting in the way of people who just want the next article?

1. Calculate ratio of hovers where the reader opens the linked page after seeing the hovercard.
   1. If 100% of the time, people continue onto article after seeing a hover, this is bad. We want to keep the ratio below 70%.
   2. If 0% of the time people continue onto the article, that is also a sign that we are making it too hard. We want to keep this ratio above 10% of hovers where user continues onto article.

Descriptive

1. How long do people spend hovering before they click?
2. How long do people spend hovering before they dismiss the hover?

Diagnostic

1. Is it ever the case that “perceived” wait (amount of time before hover shows) exceeds “popup delay” (amount of time to trigger) by more than 200 ms? What % of the time?
2. Frequency of error states (per hover)

Results

Extension:Popups/Fundraising test 1 - Results from the first A/B test run on the Hungarian Wikipedia during a fundraising campaign.

2015 Greek and Catalan Wikipedia test

We ran a test where hovercards were turned on by default for Greek and Catalan Wikipedia for a period of 4 months. We have both quantitative and qualitative data for this experiment.

Here is a list of major bugs reported.

Known & documented issues:

• Cutting sentences randomly
• Info in parenthesis ignored (https://phabricator.wikimedia.org/T91344 )
• Period problem with TextExtracts  (send email to Gergo and Prateek)
• Hovercards not going away

Unknown and need more information:

• Text alignment issue in certain scripts
• Disambiguation text showing in some cases - this is when a page is just using plain indented/italic text instead of a template (e.g.). Or, if the hatnote template doesn't include class="noexcerpt".
• Half sentences are annoying, people want more text 
• People are unhappy with animation speed, some want it faster, some slower (replicating the existing Reference Tooltips options, would be best.

Known and hard to solve for

• Issues with PageImages (…)
• Map shown without dot (This should be fixed once Max and Yuri are done with the tile server)
• Reference hovercards (user and special pages too) (Prateek and Bene* are working on a restructure)
• Sub-section links (TextExtracts)
• Make it work with Wikidata (https://phabricator.wikimedia.org/T69434 )

Qualtrics Survey Results

In April of 2015, hovercards were rolled out to all users on Catalan and Greek Wikipedias. They also continued to be a beta feature on english and other wikipedias. During this time, a survey was shown to all hovercards users using a banner recruitment. There were over 1,700 responses, heavily dominated by non-logged in users on Catalan and Greek and logged in users (probably mostly editors) on English. Below are the aggregated results

For first question, users mostly gave positive feedback, but a good amount of issues identified

Questions 2-6 are the qualitative text answers to whatever the user chose for open feedback. We cannot share these answers on wiki, as they have not been vetted to remove potentially personally identifiable information.

Here are some random, contiguous sample rows from those answers:

• bug:

no se'm tanca una pestanya

Speaking people

elsexe

Hovercard doesn't go away Browser: Google Chrome Operating System: Win 7

ghgh

• other problem:

Encara que deixi desactivades les vistes prèvies i guardats els canvis cada cop que tanco l'explorador i el torno a engegar les vistes prèvies tornen a estar activades. Utilitzo Firefox amb cookies activades. Gràcies. (Catalan. Google translate says: Although longer disabled and previews the changes are saved each time you close the browser and start again previews are back on. I use Firefox with cookies enabled. Thank you." )

δεν μου αρεσουν και με μπερδευουν (Greek: "I don't like them and they confuse me"

- |The hover card, doesn't close.

Hover card for Edvard Munch not showing correct information when linked from page https://en.wikipedia.org/wiki/Norwegian_Air_Shuttle_/_Norwegian_Long_Haul_tail_art Only disambiguation info shown. The problem is with the hover card, as it appears on other articles where 'Edvard Munch' is linked. I am not able to fix this problem.

• positive feedback:

!que sexplica molt be el text. (Catalan, Google Translate says "Explains very well the text")

διότι δεν έχασα χρόνο να ανοίξω άλλη καρτέλα στον φυλλομετρητή μου (Greek: "because I did not lose time to open another tab in my browser"

És útil per saber si l'enllaç ens porta al contingut que ens interessa. (Catalan. Google Translate says "It is useful to know if the link leads to content that interests us."

These tooltips are awesome!

perfect :)

It is very convenient!

• suggestion for improvement:

The preview card should disappear INSTANTLY (less than 0.01 seconds) when the mouse cursor is moved away form the hyperlink text. My only guess for giving the preview card some time to disappear is so that the user can move the cursor to the preview card itself and click it to be linked to that article. However, this is unnecessary, (in fact the preview card itself doesn't need to be clickable/linked at all) the cursor is already on the link in the text, ready to click on spot, there is no need to give it time to move to the preview card (and again no need for the card to be clickable).

αυτή η σημαία δεν υπήρχε τότε , αναφερόμαστε στο 1200 (Greek: "this flag did not exist at that time, we are talking about 1200 AD" - obviously referring to wrong choice of image)

Wikipedia should optionally allow displaying high-quality, low-compression (PNG or 95%+ quality JPEG) versions of images in hover cards on higher-speed connections.

Treu-re el link. No serveix per res (Catalan. Google Translate says "Remove the link. Is useless"

I'd like to know what happened in these epoch.

Users find hovercards easy to use!

Users find hovercards useful!

Users enjoy using hovercards

Users find hovercards simple and easy

Project-level rollout discussions

Here is where the potential rollout of hovercards is being discussed:

English Wikipedia: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)/Archive_131#Proposal:_Enable_Hovercards_by_default

See also

• Navigation popups - the editor-focused gadget, started in 2005 by Lupin, available at most wikis
• WMF blog post - Hovercards now available as a Beta Feature on all Wikimedia wikis (March 2014)
• The original specifications can be seen at File:NavigationPopups V1.pdf and hovercards-phase-3 trello notes.
• A list of similar browser extensions