Extrasensory phone: Apple’s iPhone comes with sensors that can detect changes in the phone’s position and environment. Researchers at other companies have been developing mobile-phone applications that can employ data collected by these sorts of sensors to infer a user’s behavior.
Apple

Computing

The iPhone's Untapped Potential

Apple could do a lot more with all the sensors in the iPhone.

  • Friday, June 29, 2007
  • By Kate Greene

Apple is known for its innovative gadget design, and with the release of the iPhone, it continues to live up to its hype. But while people are fawning over features like the smart, multitouch screen and the advanced Web browser, there is important technology under the hood that will likely go underappreciated. The iPhone has tiny, powerful sensors--an accelerometer, an ambient light sensor, and an infrared sensor--that are able to pick up cues from the environment and adjust the phone's functions accordingly. Apple has decided to use these sensors for detecting when to convert the screen view from portrait to landscape, for adjusting the brightness of the screen based on the brightness of the environment, and for disabling the touch screen when a person holds the phone to her ear.

Of course, Apple isn't the first to put sensors such as accelerometers in phones. Nokia, for example, has a sports phone (called the 5500) that uses an accelerometer as a pedometer. When a person takes the phone jogging, the accelerometer logs the rate of vibrations and sends that data to software that determines speed and distance. The 5500 also offers an accelerometer-based game in which a user tilts the device to navigate a ball through a maze. In addition, Nokia offers a developers' kit so that people can make their own accelerometer-based games, potentially mimicking the style of those played with Nintendo's popular Wii controller. (See "Hack: The Nintendo Wii.")

These functions, while useful and entertaining, are still pretty mundane, says Nathan Eagle, a research scientist at MIT. "These are trivial uses for what has the potential to provide a whole slew of new features and functionality," he says. Separate research taking place at MIT, Intel, and other companies suggests that, with the right software, built-in hardware such as accelerometers, light sensors, a GPS, and the phone's own microphone could provide contextual clues about people's activities and behaviors. A sensor-enabled phone could feasibly help monitor your exercise habits, keep track of an elderly relative's activities, and let your friends and family know if you're available for a call or instant-messaging conversation. It could even provide insight into social networks.

"If you get access to [a phone's] accelerometer data, you can get a variety of contextual clues about how the user is living their life," Eagle says--for instance, whether or not a user is riding a bike, taking the subway, walking up stairs, or sitting for a long period of time. The data can be used to let workers know if they need to take a break or if a person is meeting exercise goals, he says. Eagle and Sandy Pentland, professor of media arts and sciences at MIT, have used Nokia phones equipped with sensors to study the behavior of people in groups and even predict their actions to a certain extent. (See "Gadgets That Know Your Next Move.")

Advertisement

To explore other possibilities, researchers at Intel use a small gadget, about the size of a pager, that amasses data from seven sensors: an accelerometer, a barometer, a humidity sensor, a thermometer, a light sensor, a digital compass, and a microphone, says Tanzeem Choudhury, a researcher at Intel Labs Seattle. Most of the sensors are used to determine location and activity, but the microphone can provide interesting insight into social networks, she says, such as whether a person is having a business conversation or a social chat. Aware of privacy concerns, the researchers designed the microphone data to be immediately processed so that all words are removed, and only information about tone, pitch, and volume is recorded. Recently, Intel researchers equipped a first-year class of University of Washington graduate students with these sorts of sensors and, based on their interactions, were able to watch social networks develop over time.

Print

Related Articles

Shopping via Text Message

E-commerce startups are targeting emerging economies where cell phones, not browsers, rule.

Mapping a City's Rhythm

A phone application highlights hot spots and will soon show where different urban "tribes" gather.

Awaiting the Google Phone

Whether Google has succeeded in revolutionizing the cell phone may depend on who you ask.

Close Comments

To comment, please sign in or register

Forgot my password

hsfrey

13 Comments

  • 1681 Days Ago
  • 06/29/2007

WHY?

   Who could possibly benefit from such intrusive monitoring, other than Big Brother? Who would purposely buy and use a phone that would collect and send this intimate data?
   Just because something CAN be done doesn't mean that it SHOULD be done! The author gave not the slightest hint that such questions had ever crossed his mind. Nor did the editor.

Reply

stradric

33 Comments

  • 1681 Days Ago
  • 06/29/2007

Re: WHY?

Seriously... This type of monitoring seems good for people under house arrest or on parole, not for people doing everyday tasks.

Reply

Parijata

4 Comments

  • 1681 Days Ago
  • 06/29/2007

Re: WHY?

Yeah, seems that Google goes a little more Big Brother every day... Oh sorry, did I say Google? I meant Apple. (http://gigaom.com/2006/08/29/apple-google/)

"A team from Google Research has developed a prototype system that uses a home computer's internal microphone to listen to the ambient audio in a room, determine what is being watched on TV and offer web-based supplemental information, services and shopping contextual to each program being watched," reports TechCrunch.

Reply

VCRAGAIN

37 Comments

  • 1681 Days Ago
  • 06/29/2007

Re: WHY?

well we are all still in charge as long as we are told about the sensors and can choose to enable/disable or not have them in our gadgets at all - trouble will be when "big brother" decides to use this stuff without us knowing - how do we stop that ?
- now we need a sensor "sensor" to report who is watching or sniffing or listening - what a wondrous thing is man !!!

Reply

nicko82

3 Comments

  • 1677 Days Ago
  • 07/03/2007

Re: WHY?

You people have obviously never heard of ubiquitous, or pervasive, or context aware computing.
Search for these terms and you might come across some interesting, yet very useful, futuristic technological advances, which are still in the research stage. The iPhone is the only one that stands out, so far. But you'll hear of a lot more in the future.
I'll give you a simple example concerning ringtones, to give an insight of its uses:
The phone may sense that its in your pocket - switch to vibration.
The phone may sense that you are in a meeting - switch to silent.
The phone may sense that you re driving your car - switch to handsfree, or speaker.
The phone may sense you re sleeping - switch to silent.
...and a lot more. However some of the means of sensings these kind of things have not yet been invented, although the iPhone covers a lot of them.
Through sensing things the phone can help with a lot of things that you probably cannot be bothered, or usually forget to do.

Yes you re right though, there are lots of concerns for privacy. Its one of the main challenges for the designers. Thats an issue of its own.

Reply

anton

7 Comments

  • 1669 Days Ago
  • 07/11/2007

Re: WHY?

> The phone may sense that its in your pocket - switch to vibration.
> The phone may sense that you are in a meeting - switch to silent.
> The phone may sense that you re driving your car - switch to handsfree, or speaker.
> The phone may sense you re sleeping - switch to silent.

That's just confusing. If I set my cell phone on vibrate, I want it to vibrate. If I set it on silent mode, I want it to be silent - regardless wether it's sensors think I'm sleeping or not, driving a car or whatever. Also the sensors can never detect what *I* am doing, it can only detect what's happening to the unit. And who sleeps with his cellphone.

How to really improve cellphones:
* Increase in speech quality. Japan has one of the most advanced cell phone systems on the planet, still the sound quality is often incomprehensible. * Also, the user iterface for each cell phone is different, it's difficult using another cellphone than the one you are used to.
* The user interface needs a lot of improvement. Let's say I want to switch off one of the three (!) answering machines that come with my cell phone (one on the server, two are build-in): is the function to switch on/off the answering machine under "Settings 1", "Settings 2" or "Set up"?
* As someone suggested in another comment below, remote control replacement! That's a really great idea and would help me get rid of 6 separate remote controls.
* It would be really cool to have a cell phone with GPS and a map. Such cellphones exist in Japan, but they rely on a working GPS connection *and* Internet connection at the samer time = unreliable & expensive. It would be much better if the maps are saved in the cell phone, not retrieved from a server.

Reply

nicko82

3 Comments

  • 1665 Days Ago
  • 07/15/2007

Re: WHY?

> The phone may sense that its in your pocket - switch to vibration.
> The phone may sense that you are in a meeting - switch to silent.
> The phone may sense that you re driving your car - switch to handsfree, or speaker.
> The phone may sense you re sleeping - switch to silent.

>>That's just confusing. If I set my cell phone on >>vibrate, I want it to vibrate. If I set it on >>silent mode, I want it to be silent - regardless >>wether it's sensors think I'm sleeping or not, >>driving a car or whatever. Also the sensors can >>never detect what *I* am doing, it can only >>detect what's happening to the unit. And who >>sleeps with his cellphone.

Whats confusing is that some people are not so used to the technology yet to have ubiquitous computing in their lives. What I said was just an example.
What if you re in a meeting and you forget to switch to silent for example, and it starts ringing and embaresses you? Thats the point of it sensing things and setting itself. The main challenges still lay into how the phone should sense what you are doing.

As for quality of sound and maps and stuff, it could all be done with 3G. Ask the phone companies that paid billions to have the systems installed why they are not taking full advantage of the scheme.

Reply

Advertisement

PaulTiffany

2 Comments

  • 1680 Days Ago
  • 06/30/2007

Brainstorm for potential

I've been thinking about the untapped capabilities of cellphones for a while. 
One idea I have: Using something like Alltel's unlimited call circle to constantly record to Skype or something... essentially creating my own human speechome. ;)
Another idea: Why aren't all cellphones universal remotes already?  "Honey, can you call the remote?"
This site has already discussed using GPS/camera technology for amateur cartography.  That has the potential to be incredibly powerful. 
Using cell-phones in an ad-hoc wireless network for internet access outside the EVIL telecomm providers. 
What about putting the iPhone on its belly on a desk for silent mode? 

Regarding google and apple eroding privacy.  Oh well.  Privacy is over-rated.  Besides, I'm sure you'll always have a choice.  It's just a question of whether or not the forms have automatic yes checks. 

Reply

banditbulz

2 Comments

  • 1672 Days Ago
  • 07/08/2007

good idea, but...

This requires permanent attachment of iphone to human body... and although many people carry phones around, they don't carry it while sleeping, taking a shower, or exercising.

Reply

sniper555

1 Comment

  • 1669 Days Ago
  • 07/11/2007

Re: good idea, but...

It still can determine your exact location, I think privacy issue is over these days it is not related to cell phones only, many devices we use everyday can be tracked.

Reply

hollifieldcs

1 Comment

  • 1543 Days Ago
  • 11/14/2007

Re: good idea, but...

I would like to offer myself as a counter to this idea that no one takes their phone with them everywhere (bed, shower, etc.).  I have a Blackberry 8830 which is provided by work to function as my messaging device (email, sms, pin, pager) and as my cell phone.  Since the staff I manage have 24/7/365 requirements, it is assumed that I am available 24/7/365 as well.  Yes, it goes into the bathroom with me as well as on my night stand at night.  I would absolutely love it if I did not have to change the settings every single night, morning, meeting, time in the car, or time outside.

Regarding privacy, this is just another paradigm shift for most and will soon be considered acceptable by the masses.  Think about taking someone from 1920, 1950, 1980 or even 1990 and plopping them into 2007 with the amount of tracking and intrusive technology that exists today, but none of us give it a second thought. 

Reply

Advertisement

MAGAZINE

Can We Build Tomorrow's Breakthroughs?

Manufacturing in the United States is in trouble. That's bad news not just for the country's economy but for the future of innovation.

Sponsored Content

Technologies from National Instruments

Taking a Measurement
Learn how to use your PC to take measurements

> Click here for more National Instruments Videos <
Whitepaper

BUILD VERSUS BUY
Understanding the Total Cost of Embedded Design

National Instruments has gathered customer information and data regarding some of the cost differences between building a custom solution versus using NI off-the-shelf tools. Using this data, we built the Graphical System Design ‘Build vs. Buy’ Calculator. The calculator can help show the financial differences between building a custom solution versus buying an off-the-shelf system. This paper discusses the benefits and drawbacks of both a traditional custom design approach and off-the-shelf embedded tools.

View full PDF > Listen to story >
Find us on Youtube

Videos

Windows on an iPad

More

Advertisement

Technology Review Lists

TR50

Our list of the 50 most innovative companies, including the following:

Crowdcast

Facebook

Toyota

1366 Technologies

More

Advertisement

Facebook

Advertisement