Microsoft patent suggests HoloLens could keep track of your everyday objects

Will AR ensure you never lose your keys again? Microsoft patent suggests HoloLens could keep track of your everyday objects

Microsoft has filed a patent for object tracking in a head-mounted display

The technology monitors the environment to track and identify lost items

It will also communicate with other headsets in case a friend moves your item

The two will share data in order to find the last place the lost item was located

Published: 12:49 EST, 30 December 2016 | Updated: 13:36 EST, 30 December 2016

Many know the painful task of searching high and low for their keys, only to discover they were under your nose the whole time.

Instead of hunting for lost items, Microsoft has a solution that tracks inanimate movable objects with an augmented reality headset.

Illustrated in a new patent, the tech giant describes cameras and sensors in a headset that would monitor the physical environment to track and locate lost items.

Scroll down for video

Instead of hunting for lost items, Microsoft has a solution that tracks inanimate movable objects with an augmented reality headset. A new patent cameras and sensors in a headset that would monitor the physical environment to track and locate lost items

HOW DOES IT WORK?

Microsoft could find your lost items using augmented reality headsets.

The cameras and sensors in their design could be used to scan the real world environment to find whatever the users says is lost.

The patent also suggests that the headset, which could be their own HoloLens, could also learn what objects the user wants it to track and what each one looks like.

If any item goes missing, you tell the headset to locate them using voice commands.

And the cameras fitted on the device will search the environment for the missing item.

It will then 'tell' you where it last 'saw' your keys in order to help you locate them yourself.

The patent, called 'Object Tracking' filed in August 2012 and again in September 2016 was first reported by CNET.

'A mobile device, such as a see-through display worn by a user, may comprise image sensors that observe the user's environment,' reads the patent.

'Video data from the image sensors may be processed to detect objects in the user's environment, and to identify tracked objects in the video data.

'State information, such as location and other characteristics, of tracked objects may then be stored.'

'This may allow for the output of an alert regarding the state of a tracked object upon detection of an alert trigger, such as a user request or a contextual clue.'

If your keys, wallet, mobile device or any other inanimate object goes missing, you simply tell the headset to locate them using voice commands.

And the cameras fitted on the device will begin scouring the real-world environment for the missing item.

It then 'tells' you where it last 'saw' your keys in order to help you locate them yourself or shows you exactly where they are if the headset spots them itself.

The patent also suggests that the headset, which could be their own HoloLens, could also learn what objects the user wants it to track and what each one looks like.

Microsoft also believes that there is more power in numbers and could let the system interact with other users to help one another.

The cameras fitted on the device will begin scouring the real-world environment for the missing item. It then 'tells' you where it last 'saw' your keys in order to help you locate them yourself or shows you exactly where they are if the headset spots them itself

'Each person may become aware of changes to objects made by other users via the sharing of object tracking information,' explains the patent.

'In this manner, a user may be able to discover a most recent location of lost keys, may be provided with a reminder to buy more milk while browsing the dairy section at a grocery store, and/or may track and recall other object state information in any suitable manner.'

This means if one person puts down an object and their roommate moves it to another location, the headset will be able to tell the first person where the object is by communicating with their roommate's headset.

Microsoft has filed a patent for new eye-tracking technology for head-mounted displays (HMDs). The technology uses infrared beams to track the eyes, reducing the need for clunky cameras that obstruct your view as seen in current devices on the market

EYE-TRACKING

The glasses will use 'wave guides' to track infrared-lit eyes.

The patent states: 'A waveguide, which can be used in a head mounted display (HMD), but is not limited for use therewith, is transparent and includes an input-coupler and an output-coupler.

'The input-coupler comprises a grating area, formed by plurality of curved grating lines, that diffract light beams incident on the input-coupler into the waveguide and towards a common region at which is located the output-coupler.

'The curved grating lines of the input-coupler have a radially varying pitch.'

The patent goes on to describe how the infrared beams work with the curved grating lines of the input coupler to produce an eye-tracking effect.

The entire system would rely on a range of sensors, which could be image sensors, such as two-dimensional image sensors, and/or depth sensors that collect video data of the surrounding environment.

Microsoft filed another for technology that could be used with its HoloLens device, however if the firm does move forward with the process it would be for its next headset.

The patent was uncovered by MSPoweruser, and includes a complex diagram for new eye-tracking technology.

'Wave guides' will be used to track infrared-lit eyes, in an attempt to prevent your line of sight from being obstructed.

Currently, most eye-tracking devices rely on a clunky camera attached to the frame of the display, meaning that at least part of your view is hindered.

Alternatively, the only other eye-tracking technology available uses a reflector to bend the camera's line of sight towards your temple.

But this is a problems for people who wear glasses, which can get in the way.

If Microsoft's new patent is to be believed, it would remedy this problem, using a transparent wave guide to track the eyes using infrared.

The patent states: 'A waveguide, which can be used in a head mounted display (HMD), but is not limited for use therewith, is transparent and includes an input-coupler and an output-coupler.

'Wave guides' will be used to track infrared-lit eyes, in an attempt to prevent your line of sight from being obstructed

'The curved grating lines of the input-coupler have a radially varying pitch.'

The patent goes on to describe how the infrared beams work with the curved grating lines of the input coupler to produce an eye-tracking effect.

There are currently two versions of the HoloLens headsets (pictured) available, with the development edition selling for £2,719 ($3,320) and a commercial version for £4,529 ($5,530)

There are currently two versions of the HoloLens headsets available, with the development edition selling for £2,719 ($3,320) and a commercial version for £4,529 ($5,530).

It is unclear if or when Microsoft will begin using either technologies in its HoloLens devices or if the tech giant will even turn the documents into working products.

HOLOLENS FEATURES

The HoloLens development edition includes the headset, a clicker for interacting with applications, a carrying case, nose pads and overheads straps.

The headset is designed with multiple environment understanding sensors and powered by a custom-built Microsoft Holographic Processing Unit (HPU) and an Intel 32-bit chipset.

The device weighs just a little over one pound and consists of 64GB of Flash memory and 2GB of RAM.

The battery lasts for about two to three hours, with up to two weeks of standby time, says Microsoft.

HoloLens has a 'see-through holographic lens' that uses an optical projection system to create multi-dimensional full-colour 'holograms'.

According to Microsoft, 'the key to a great holographic experience is holograms that are light point rich, they have a high holographic density and are pinned, or anchored, to the world around you.

In order to achieve this, the device has an optimal holographic density of 2.5k radiants.