Kinect

Kinect for Xbox 360 logo
Kinect sensor device
Product family	Xbox, Windows
Generation	Seventh generation era
Units Sold	18 million (as of January 9, 2012)[1][2]
Release date	For Xbox 360: NA November 4, 2010[3] EU November 10, 2010[4] CO November 14, 2010[5] AUS November 18, 2010[6] JP November 20, 2010[7] For Windows: NA February 1, 2012[8] AUS February 1, 2012[8] JP February 1, 2012[8]
Platform	Xbox 360 Microsoft Windows (Windows 7 onwards)
Connectivity	USB 2.0 (type-A for original model; proprietary for Xbox 360 S)
Resolution	640×480 pixels @ 30 Hz (RGB camera) 640×480 pixels @ 30 Hz (IR depth-finding camera)[9]
Predecessor	Xbox Live Vision

Kinect is a motion sensing input device by Microsoft for the Xbox 360 video game console and Windows PCs. Based around a webcam-style add-on peripheral for the Xbox 360 console, it enables users to control and interact with the Xbox 360 without the need to touch a game controller, through a natural user interface using gestures and spoken commands.^[10] The project is aimed at broadening the Xbox 360's audience beyond its typical gamer base.^[11] Kinect competes with the Wii Remote Plus and PlayStation Move with PlayStation Eye motion controllers for the Wii and PlayStation 3 home consoles, respectively. A version for Windows was released on February 1, 2012.^[8]

Kinect was launched in North America on November 4, 2010,^[3] in Europe on November 10, 2010,^[4] in Australia, New Zealand and Singapore on November 18, 2010,^[6][12][13] and in Japan on November 20, 2010.^[14] Purchase options for the sensor peripheral include a bundle with the game Kinect Adventures and console bundles with either a 4 GB^[15][16][17] or 250 GB^[18] Xbox 360 console and Kinect Adventures.^{[15][16][17][18]}

After selling a total of 8 million units in its first 60 days, the Kinect holds the Guinness World Record of being the "fastest selling consumer electronics device".^[19][20][21] 18 million units of the Kinect sensor had been shipped as of January 2012.^[1]

Microsoft released Kinect software development kit for Windows 7 on June 16, 2011.^[22][23][24] This SDK will allow developers to write Kinecting apps in C++/CLI, C#, or Visual Basic .NET.^[25][26]

Technology[link]

Kinect builds on software technology developed internally by Rare, a subsidiary of Microsoft Game Studios owned by Microsoft, and on range camera technology by Israeli developer PrimeSense, which developed a system that can interpret specific gestures, making completely hands-free control of electronic devices possible by using an infrared projector and camera and a special microchip to track the movement of objects and individuals in three dimension.^[27][28][29] This 3D scanner system called Light Coding^[30] employs a variant of image-based 3D reconstruction.^[31][32]

The Kinect sensor^[10] is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an "RGB camera, depth sensor and multi-array microphone running proprietary software",^[33] which provide full-body 3D motion capture, facial recognition and voice recognition capabilities. At launch, voice recognition was only made available in Japan, the United Kingdom, Canada and the United States. Mainland Europe received the feature later in spring 2011.^[34] Currently voice recognition is supported in Australia, Canada, France, Germany, Ireland, Italy, Japan, Mexico, New Zealand, United Kingdom and United States. The Kinect sensor's microphone array enables the Xbox 360 to conduct acoustic source localization and ambient noise suppression, allowing for things such as headset-free party chat over Xbox Live.^[10]

The depth sensor consists of an infrared laser projector combined with a monochrome CMOS sensor, which captures video data in 3D under any ambient light conditions.^[10][35] The sensing range of the depth sensor is adjustable, and the Kinect software is capable of automatically calibrating the sensor based on gameplay and the player's physical environment, accommodating for the presence of furniture or other obstacles.^[36]

Described by Microsoft personnel as the primary innovation of Kinect,^[37][38][39] the software technology enables advanced gesture recognition, facial recognition and voice recognition.^[40] According to information supplied to retailers, Kinect is capable of simultaneously tracking up to six people, including two active players for motion analysis with a feature extraction of 20 joints per player.^[41] However, PrimeSense has stated that the number of people the device can "see" (but not process as players) is only limited by how many will fit in the field-of-view of the camera.^[42]

This infrared image shows the laser grid Kinect uses to calculate depth

The depth map is visualized here using color gradients from white (near) to blue (far)

Reverse engineering^[43] has determined that the Kinect sensor outputs video at a frame rate of 30 Hz. The RGB video stream uses 8-bit VGA resolution (640 × 480 pixels) with a Bayer color filter, while the monochrome depth sensing video stream is in VGA resolution (640 × 480 pixels) with 11-bit depth, which provides 2,048 levels of sensitivity. The Kinect sensor has a practical ranging limit of 1.2–3.5 m (3.9–11 ft) distance when used with the Xbox software. The area required to play Kinect is roughly 6 m², although the sensor can maintain tracking through an extended range of approximately 0.7–6 m (2.3–20 ft). The sensor has an angular field of view of 57° horizontally and 43° vertically, while the motorized pivot is capable of tilting the sensor up to 27° either up or down. The horizontal field of the Kinect sensor at the minimum viewing distance of ~0.8 m (2.6 ft) is therefore ~87 cm (34 in), and the vertical field is ~63 cm (25 in), resulting in a resolution of just over 1.3 mm (0.051 in) per pixel. The microphone array features four microphone capsules^[44] and operates with each channel processing 16-bit audio at a sampling rate of 16 kHz.^[41]

Because the Kinect sensor's motorized tilt mechanism requires more power than the Xbox 360's USB ports can supply,^[45] the device makes use of a proprietary connector combining USB communication with additional power. Redesigned Xbox 360 S models include a special AUX port for accommodating the connector,^[46] while older models require a special power supply cable (included with the sensor^[44]) that splits the connection into separate USB and power connections; power is supplied from the mains by way of an AC adapter.^[45]

History[link]

The depth sensing technology behind Kinect was invented in 2005 by Zeev Zalevsky, Alexander Shpunt, Aviad Maizels and Javier Garcia.^[47]

Kinect itself was first announced on June 1, 2009 at E3 2009 under the code name "Project Natal".^[48] Following in Microsoft's tradition of using cities as code names,^[10] "Project Natal" was named after the Brazilian city of Natal as a tribute to the country by Brazilian-born Microsoft director Alex Kipman, who incubated the project.^[10][49] The name Natal was also chosen because the word natal means "of or relating to birth", reflecting Microsoft's view of the project as "the birth of the next generation of home entertainment".^[37]

Three demos were shown to showcase Kinect when it was revealed at Microsoft's E3 2009 Media Briefing: Ricochet, Paint Party and Milo & Kate.^[50] A demo based on Burnout Paradise was also shown outside of Microsoft's media briefing.^[51] The skeletal mapping technology shown at E3 2009 was capable of simultaneously tracking four people,^{[35][37][40][49]} with a feature extraction of 48 skeletal points on a human body at 30 Hz.^[36][49]

It was rumored that the launch of Project Natal would be accompanied with the release of a new Xbox 360 console (as either a new retail configuration,^[52][53] a significant design revision^[54] and/or a modest hardware upgrade^[55]). Microsoft dismissed the reports in public and repeatedly emphasized that Project Natal would be fully compatible with all Xbox 360 consoles. Microsoft indicated that the company considers it to be a significant initiative, as fundamental to the Xbox brand as Xbox Live,^[56] and with a launch akin to that of a new Xbox console platform.^[57] Kinect was even referred to as a "new Xbox" by Microsoft CEO Steve Ballmer at a speech for the Executives' Club of Chicago.^[58][59] When asked if the introduction will extend the time before the next-generation console platform is launched (historically about 5 years between platforms),^[60] Microsoft corporate vice president Shane Kim reaffirmed that the company believes that the life cycle of the Xbox 360 will last through 2015 (10 years).^[37]

During Kinect's development, project team members experimentally adapted numerous games to Kinect-based control schemes to help evaluate usability. Among these games were Beautiful Katamari and Space Invaders Extreme, which were demonstrated at the Tokyo Game Show in September 2009.^[61] According to creative director Kudo Tsunoda, adding Kinect-based control to pre-existing games would involve significant code alterations, making it unlikely for Kinect features to be added through software updates.^[62]

Although the sensor unit was originally planned to contain a microprocessor that would perform operations such as the system's skeletal mapping, it was revealed in January 2010 that the sensor would no longer feature a dedicated processor. Instead, processing would be handled by one of the processor cores of the Xbox 360's Xenon CPU.^[63] According to Alex Kipman, the Kinect system consumes about 10-15% of the Xbox 360's computing resources.^[64] However, in November, Alex Kipman made a statement that "the new motion control tech now only uses a single-digit percentage of the Xbox 360's processing power, down from the previously stated 10 to 15 percent."^[65] A number of observers commented that the computational load required for Kinect makes the addition of Kinect functionality to pre-existing games through software updates even less likely, with concepts specific to Kinect more likely to be the focus for developers using the platform.^[63]

On March 25, Microsoft sent out a save the date flier for an event called the "World Premiere 'Project Natal' for the Xbox 360 Experience" at E3 2010. The event took place on the evening of Sunday, June 13, 2010 at the Galen Center^[66] and featured a performance by Cirque du Soleil. It was announced that the system would officially be called Kinect, a portmanteau of the words "kinetic" and "connect", which describe key aspects of the initiative.^[67] Microsoft also announced that the North American launch date for Kinect will be November 4, 2010.^[68] Despite previous statements dismissing speculation of a new Xbox 360 to accompany the launch of the new control system, Microsoft announced at E3 2010 that it was introducing a redesigned Xbox 360, complete with a connector port ready for Kinect. In addition, on July 20, 2010, Microsoft announced a Kinect bundle with a redesigned Xbox 360, to be available with the Kinect launch.^[17]

On June 16, 2011, Microsoft announced its official release of its SDK for non-commercial use.^[69][70]

On July 21, 2011, Microsoft announced that the first ever white Kinect sensor would be available as part of the "Xbox 360 Limited Edition Kinect Star Wars Bundle", which also includes custom a Star Wars-themed console and controller, and copies of Kinect Adventures and Star Wars Kinect.^[71] Previously, all Kinect sensors had been glossy black.

On October 31, 2011, Microsoft announced launching of the commercial version of Kinect for Windows program with release of the SDK to companies. David Dennis, Product Manager at Microsoft, said, "There are hundreds of organizations we are working with to help them determine what's possible with the tech".^[72]

On February 1, 2012, Microsoft released the commercial version of Kinect for Windows SDK and told that more than 300 companies from over 25 countries are working on Kinect-ready apps.^[73]

Launch[link]

A January 2010 promotional banner indicating the expected release of Kinect (then "Project Natal") by holiday 2010

Microsoft had an advertising budget of US$500 million for the launch of Kinect, a larger sum than the investment at launch of the Xbox console.^[74] The marketing campaign You Are the Controller, aiming to reach new audiences, included advertisements on Kellogg's cereal boxes and Pepsi bottles, commercials during shows such as Dancing with the Stars and Glee as well as print ads in various magazines such as People and InStyle.^[75]

On October 19, Microsoft advertised Kinect on The Oprah Winfrey Show by giving free Xbox 360 consoles and Kinect sensors to the people in the audience.^[76] Two weeks later, Kinect bundles with Xbox 360 consoles were also given away to the audience of Late Night With Jimmy Fallon.^[77] On October 23, Microsoft held a pre-launch party for Kinect in Beverly Hills. The party was hosted by Ashley Tisdale and was attended by soccer star David Beckham and his three sons, Cruz, Brooklyn, and Romeo. Guests were treated to sessions with Dance Central and Kinect Adventures, followed by Tisdale having a Kinect voice chat with Nick Cannon.^[78] Between November 1 and November 28, Burger King gave away a free Kinect bundle "every 15 minutes".^[79]

A major event was organized on November 3 in Times Square, where singer Ne-Yo performed with hundreds of dancers in anticipation of Kinect's midnight launch.^[80] During the festivities, Microsoft gave away T-shirts and Kinect games.^[81]

Kinect for Windows[link]

On February 21, 2011 Microsoft announced that it would release a non-commercial Kinect software development kit (SDK) for Windows in spring 2011, which was released for Windows 7 on June 16, 2011 in 12 countries.^[22][23][24]

The SDK includes Windows 7 compatible PC drivers for Kinect device. It provides Kinect capabilities to developers to build applications with C++, C#, or Visual Basic by using Microsoft Visual Studio 2010 and includes following features:

1. Raw sensor streams: Access to low-level streams from the depth sensor, color camera sensor, and four-element microphone array.
2. Skeletal tracking: The capability to track the skeleton image of one or two people moving within the Kinect field of view for gesture-driven applications.
3. Advanced audio capabilities: Audio processing capabilities include sophisticated acoustic noise suppression and echo cancellation, beam formation to identify the current sound source, and integration with the Windows speech recognition API.
4. Sample code and Documentation.^[82]

In March 2012, Craig Eisler, the general manager of Kinect for Windows said that almost 350 companies are working with Microsoft on custom Kinect applications for Windows.^[83]

Kinect for Windows 1.5[link]

In March 2012, Microsoft announced that next version of the Kinect for Windows will be available in May 2012. Kinect for Windows 1.5 was released on May 21, 2012. It adds new features, support for many new languages and debut in 19 more countries.^[84][85]

1. The Kinect for Windows 1.5 SDK woud include 'Kinect Studio' a new app that allows developers to record, playback, and debug clips of users interacting with applications.^[86]
2. Support for new "seated" or "10-joint" skeletal system that will let apps track the head, neck, and arms of a Kinect user - whether they're sitting down or standing; which would work in default and near mode.
3. Support for four new languages for speech recognition – French, Spanish, Italian, and Japanese. Additionally it would add support for regional dialects of these languages along with English.^[87]
4. It would be available in Hong Kong, South Korea, and Taiwan in May and Austria, Belgium, Brazil, Denmark, Finland, India, the Netherlands, Norway, Portugal, Russia, Saudi Arabia, Singapore, South Africa, Sweden, Switzerland and the United Arab Emirates in June.^[88]

Software[link]

Requiring at least 190 MB of available storage space,^[89] the Kinect system software allows users to operate the Xbox 360 Dashboard console user interface through voice commands and hand gestures. Techniques such as voice recognition and facial recognition are employed to automatically identify users. Among the applications for Kinect is Video Kinect, which enables voice chat or video chat with other Xbox 360 users or users of Windows Live Messenger. The application can use Kinect's tracking functionality and the Kinect sensor's motorized pivot to keep users in frame even as they move around. Other applications with Kinect support include ESPN, Zune Marketplace,^[89] Netflix, Hulu Plus^[90] and Last.fm^[91] Microsoft later confirmed that all forthcoming applications would require them to have Kinect functionality for certification.^[92]

Games that require Kinect have a purple sticker on them with a white silhouette of the Kinect sensor and "Requires Kinect Sensor" underneath in white text.^[93] Games that have optional Kinect support (meaning that Kinect is not necessary to play the game or that there are optional Kinect minigames included) feature a standard green Xbox 360 case with a purple bar underneath the header, a silhouette of the Kinect sensor and "Better with Kinect Sensor" next to it in white text.^[93]

Kinect launched on November 4, 2010 with 17 titles.^[94] Third-party publishers of available and announced Kinect games include, among others, Ubisoft, Electronic Arts, LucasArts, THQ, Activision, Konami, Sega, Capcom, Namco Bandai and MTV Games. Along with retail games, there are also select Xbox Live Arcade that exclusively use the Kinect peripheral.

Kinect Fun Labs[link]

At E3 2011, Microsoft announced Kinect Fun Labs: a collection of various gadgets and minigames that are accessible from the Xbox 360 Dashboard. These gadgets includes Build A Buddy, Air Band, Kinect Googly Eyes, Kinect Me, Bobblehead, Kinect Sparkler, Junk Fu^[95] and Avatar Kinect.^[96][97][98]

Open source drivers[link]

In November 2010, Adafruit Industries offered a bounty for an open-source driver for Kinect. Microsoft initially voiced its disapproval of the bounty, stating that it "does not condone the modification of its products" and that it had "built in numerous hardware and software safeguards designed to reduce the chances of product tampering".^[99] This reaction, however, was caused by a misunderstanding within Microsoft,^[100] and the company later clarified its position, claiming that while it does not condone hacking of either the physical device or the console, the USB connection was left open by design.

On November 10, Adafruit announced Héctor Martín as the winner,^[102][103] who had produced a Linux driver that allows the use of both the RGB camera and depth sensitivity functions of the device.^[104][105] It was later revealed that Johnny Lee, a core member of Microsoft's Kinect development team, had secretly approached Adafruit with the idea of a driver development contest and had personally financed it.^[106]

In December 2010, PrimeSense, whose depth sensing reference design Kinect is based on, released their own open source drivers along with motion tracking middleware called NITE.^[107] PrimeSense later announced that it had teamed up with Asus to develop a PC-compatible device similar to Kinect, which will be called WAVI Xtion and is scheduled for release in the second quarter of 2011.^[108]

Third party development[link]

A demonstration of a third party use of Kinect at Maker Faire. The visualization on the left, provided through Kinect, is of a user of a jacket with wearable electronic controls for VJing.

Numerous developers are researching possible applications of Kinect that go beyond the system's intended purpose of playing games. For example, Philipp Robbel of MIT combined Kinect with the iRobot Create to map a room in 3D and have the robot respond to human gestures,^[109] while an MIT Media Lab team is working on a JavaScript extension for Google Chrome called depthJS that allows users to control the browser with hand gestures.^[110] Other programmers, including the Robot Locomotion Group at MIT, are using the drivers to develop a motion-controller user interface similar to the one envisioned in the film Minority Report.^[111] The developers of MRPT have integrated open source drivers into their libraries and provided examples of live 3D rendering and basic 3D visual SLAM.^[112] Another team has shown an application that allows Kinect users to play a virtual piano by tapping their fingers on an empty desk.^[113] Oliver Kreylos, a researcher at University of California, Davis, adopted the technology to improve live 3-dimensional videoconferencing, which NASA has shown interest in.^[114]

Alexandre Alahi from EPFL presented a video surveillance system that combines multiple Kinect devices to track groups of people even in complete darkness.^[115] Companies So touch and Evoluce have developed presentation software for Kinect that can be controlled by hand gestures; among its features is a multi-touch zoom mode.^[116] In December 2010, the free public beta of HTPC software KinEmote was launched; it allows navigation of Boxee and XBMC menus using a Kinect sensor.^[117] Soroush Falahati wrote an application that can be used to create stereoscopic 3D images with a Kinect sensor.^[118]

For a limited time in May 2011, a Topshop store in Moscow set up a Kinect kiosk that could overlay a collection of dresses onto the live video feed of customers. Through automatic tracking, position and rotation of the virtual dress were updated even as customers turned around to see the back of the outfit.^[119]

Kinect also shows compelling potential for use in medicine. Researchers at the University of Minnesota have used Kinect to measure a range of disorder symptoms in children, creating new ways of objective evaluation to detect such conditions as autism, attention-deficit disorder and obsessive-compulsive disorder.^[120] Several groups have reported using the Kinect for intraoperative, review of medical imaging, allowing the surgeon to access the information without contamination.^[121][122] This technique is already in use at Sunnybrook Health Sciences Centre in Toronto, where doctors use it to guide imaging during cancer surgery.^[123] At least one company, GestSure Technologies, is pursuing the commercialization of such a system.^[124]

Reception[link]

Reception
Review scores
Publication	Score
Computer and Video Games	8.8/10[125]
Game Informer	8/10[126]
IGN	7.5/10[127]
CNET	3.5/5[128]
Engadget	6/10[129]
USA Today	[130]
TechRadar	[131]
The Washington Post	[132]
PCMag	4/5[133]
Game Guru	9/10 [134]

IGN gave the device 7.5 out of 10, saying that "Kinect can be a tremendous amount of fun for casual players, and the creative, controller-free concept is undeniably appealing", though adding that for "$149.99, a motion-tracking camera add-on for the Xbox 360 is a tough sell, especially considering that the entry level variation of the Xbox 360 itself is only $199.99".^[127] Game Informer rated Kinect 8 out of 10, praising the technology but noting that the experience takes a while to get used to and that the spatial requirement^{[clarification needed]} may pose a barrier.^[126] Computer and Video Games called the device a technological gem and applauded the gesture and voice controls, while criticizing the launch lineup and Kinect Hub.^[125]

CNET's review pointed out how Kinect keeps players active with its full-body motion sensing but criticized the learning curve, the additional power supply needed for older Xbox 360 consoles and the space requirements.^[128] Engadget, too, listed the large space requirements as a negative, along with Kinect's launch lineup and the slowness of the hand gesture UI. The review praised the system's powerful technology and the potential of its yoga and dance games.^[129] Kotaku considered the device revolutionary upon first use but noted that games were sometimes unable to recognize gestures or had slow responses, concluding that Kinect is "not must-own yet, more like must-eventually own."^[135] TechRadar praised the voice control and saw a great deal of potential in the device whose lag and space requirements were identified as issues.^[131] Gizmodo also noted Kinect's potential and expressed curiosity in how more mainstream titles would utilize the technology.^[136] Ars Technica's review expressed concern that the core feature of Kinect, its lack of a controller, would hamper development of games beyond those that have either stationary players or control the player's movement automatically.^[137]

The mainstream press also reviewed Kinect. USA Today compared it to the futuristic control scheme seen in Minority Report, stating that "playing games feels great" and giving the device 3.5 out of 4 stars.^[130] David Pogue from The New York Times predicted you will feel a "crazy, magical, omigosh rush the first time you try the Kinect." Despite calling the motion tracking less precise than Wii's implementation, Pogue concluded that "Kinect’s astonishing technology creates a completely new activity that’s social, age-spanning and even athletic."^[138] The Globe and Mail titled Kinect as setting a "new standard for motion control." The slight input lag between making a physical movement and Kinect registering it was not considered a major issue with most games, and the review called Kinect "a good and innovative product," rating it 3.5 out of 4 stars.^[139]

Sales[link]

18 million units of the Kinect sensor had been shipped by January 2012.^[1] Having sold 8 million units in its first 60 days on the market, Kinect has claimed the Guinness World Record of being the "fastest selling consumer electronics device".^{[2][19][20][21]} According to Wedbush analyst Michael Pachter, Kinect bundles accounted for about half of all Xbox 360 console sales in December 2010 and for more than two-thirds in February 2011.^[140][141] More than 750,000 Kinect units were sold during the week of Black Friday 2011.^[142][143]

Awards[link]

• The machine learning work on human motion capture within Kinect won the 2011 MacRobert Award for engineering innovation.^[144]
• Kinect Won T3's "Gadget of the Year" award for 2011.^[145] It also won the "Gaming Gadget of the Year" prize.^[146]
• 'Microsoft Kinect for Windows Software Development Kit' was ranked second in "The 10 Most Innovative Tech Products of 2011" at Popular Mechanics Breakthrough Awards ceremony in New York City.^[147][148]

See also[link]

• Dreameye
• EyeToy
• List of Kinect games
• Motion capture
• PlayStation Eye
• PlayStation Move
• Sega Activator
• Structured-light 3D scanner
• Wii Balance Board
• Wii Remote
• Xbox Live Vision

References[link]