The Most Manipulative Use of Kinect Imaginable

The good people of GeekWire spotted a patent application from Microsoft that envisions using Kinect to figure out your mood, and target ads at you accordingly. The application, filed back when Kinect was rather new (in December of 2010) was made public this week. (It’s not the first Microsoft patent application expressing an interest in tracking users’ moods.)

How exactly would it work? The idea is that Kinect’s motion and facial recognition technology could figure out whether you’re sad or happy, and serve up ads that jive with your mood. The patent application contains unusually colorful language about how exactly the Kinect (or other computing device) might infer mood.

“If the user on the videos or images from the webcams is dancing, the advertisement engine may assign a positive emotional state, such as, glad or happy, to the user…If the user on the videos or images from the computing device, e.g., Microsoft Kinect, is screaming, the advertisement engine may assign a negative emotional state, such as, upset, to the user. If the user on the videos or images from the computing device, e.g., Microsoft Kinect., is pacing back and forth, the advertisement engine may assign a negative emotional state, such as, worried, to the user.” 

Good thinking, Kinect!

It’s undoubtedly true that we respond differently to ads, depending on our mood. The patent application gives several examples; here’s one of the more unintentionally amusing ones: “Weight-loss product advertisers may not want their advertisement to appear to users that are very happy. Because, a person that is really happy, is less likely to purchase a self-investment product that leverages on his or her shortcomings.” The application also speculates that happy people are more likely to be interested in learning about consumer electronics (which makes me pleased to know that most people who click on this blog must be happy).

This is just a patent application; there’s no indication there’s a real product necessarily in the works. But it’s instructive as a thought experiment, to imagine to what degree we would let our electronics sense our mood, and serve us up relative content. My initial reaction to the proposal was one of disgust–who wants their TV to spy on them, and take advantage of moments of low self-esteem to flash a Bally Total Fitness ad?

Then again, advertisements are an inescapable feature of modern life. If this technology were to become truly refined, might it not help us through our day? I begin to envision the Kinect and its sensors as the first step towards an in-house personal robot that, between making us our meals and cleaning up after us, pitches us products at just the right moment: “Good evening, Dave. It’s nice to see you smiling. Would you like to buy an Xbox 720?”