In-car displays you hear, rather than see

We still have a lot in common with our caveman ancestors. (Yes, I know, they didn't all live in caves. Some lived in forests, others in savannahs, and still others in jungles. But I'm trying to make a point, so bear with me!)

Take, for example, our sense of hearing. At one time, we used auditory cues to locate prey or, conversely, avoid becoming prey. If a cave bear growled, getting a fix on the location of the growl could mean the difference between life and death. At the very least, it helped you avoid running directly into the bear's mouth.

Kidding aside, the human auditory system has a serious ability to fix the location, direction, and trajectory of objects, be they cave bears or Buicks. And it's an ability that's been honed from time immemorial. So why not take advantage of it when creating user interfaces for cars?

Which brings us to spatial auditory displays. In a nutshell, these displays allow you to perceive sound as coming from various locations in a three-dimensional space. Deployed in a car, they can help you intuitively identify voices and sources of instructions, and help pinpoint the location and relative trajectory of danger. They can also improve reaction times to application prompts and potentially hazardous events.

Interested in this topic? Learn more in Scott Pennock's ECD article, "Spatial auditory displays: Reducing cognitive load and improving driver reaction times."

I know, that's a lot to take in. So let's look at an example.

Locating the emergency vehicle, without really trying
Have you ever been cruising along when, suddenly, you hear an ambulance siren? I don't know about you, but I often spend time figuring out where, exactly, the ambulance is coming from. And I don't always get it right. That's called a location error.

Such errors can occur for a variety of reasons. For example, if the ambulance is approaching from the right, but your left window is open and a building on the left is reflecting sound from the siren, you might make the mistake of thinking that the ambulance is approaching from the left. Your mind realizes, quite correctly, that the sound is coming from the left, but the environment is conspiring to mask where the sound is actually coming from.

A spatial auditory display can help address this problem by controlling the acoustic cues you hear. The degree to which the display can do this depends, in part, on the hardware employed. For example, a display based on a large array of loudspeakers can provide more location information than one based on two loudspeakers.

In any case (and this is important), the display can help you determine the location more quickly and with less cognitive load — which means you may have more brain cycles to respond to the situation appropriately.


Helping the driver locate and track an emergency vehicle

A slight right, not a sharp right
I'm only scratching the surface here. Spatial auditory displays can, in fact, help improve all kinds of driving activities, from engaging in a handsfree call to using your navigation system.

For example, rather than simply say "turn right", the display could emit the instruction from the right side of the vehicle. It could even use apparent motion of the auditory prompt to convey a slight right as opposed to a sharp right.

But enough from me. To learn more about spatial auditory displays, check out a new article from my colleague Scott Pennock, whose knowledge of spatial auditory displays far surpasses mine. The article is called Spatial auditory displays: Reducing cognitive load and improving driver reaction times, and it has just been published by Embedded Computing Design magazine.
 

Can auto wag the ITS dog?

Intelligent Transport Systems (ITS) promise to deliver many benefits, from increased road safety to better traffic flow. It’s no surprise, then, that when people talk about ITS, it is often within the context of the auto industry. But is the auto industry alone big enough to attract the businesses needed to innovate and deploy ITS?

This question came up repeatedly at the Fully Networked Car Workshop held at the Geneva Auto Show, for the simple reason that auto is dwarfed by the telecom industry. Car volumes are measured in millions, whereas mobile phones are measured in billions. (In fact, evidence suggests the world may contain more mobile phones than toothbrushes.) It is likely, then, that businesses will focus on ITS-related applications, services, and infrastructure for phones — not cars.

Workshop panelists also pointed out that ITS extends beyond the auto or other transport industries; it is about enabling a connected society on the move. Consumers want mobility solutions that enable them to stay connected to all aspects of their lives in meaningful ways — not just optimize the efficiency and safety of getting from point A to point B.

The need for global standards
Regardless of whether auto or telecom drives ITS, we will need global standards for how these systems work. The good news is, major international standards development organizations (SDOs) are already trying to collaborate on the global standards needed to make ITS a reality. In fact, the Fully Networked Car Workshop was jointly sponsored by the International Telecommunication Union (ITU), the International Organization for Standardization (ISO), and the International Electrotechnical Commission (IEC).

The ITU has also started a collaboration on ITS Communications Standards. The next meeting of this collaboration will take place at SAE Headquarters in Troy, Michigan, USA on April 3, 2012.

A non-starter without industry participation
Mind you, success of ITS standards cannot solely depend on cooperation of the SDOs. Industry involvement is just as important.

Some workshop participants argued, however, that standardization inhibits product differentiation and thereby acts as a barrier to industry participation. But others felt this could be managed by standardizing on the lower layers (i.e. communications layers) and differentiating at the higher layers (i.e. application layers). The one caveat raised was that some aspects of the higher layers need to be standardized for safety reasons. But even then, it is possible to standardize on safety and still differentiate on user experience.

Panelists also discussed the threat of government regulation as a motivation to develop and deploy ITS. Someone pointed out, however, that competition often acts as a bigger motivator for companies than the threat of regulation.

Wagging the dog?
So what will drive innovation and deployment of ITS? It seems that global standards created with industry participation are crucial. Perhaps even more important, the auto industry must start working more closely with other industries such as telecom and consumer to develop ITS solutions that not only help people get from point A to point B efficiently and safely, but also deliver value in other ways to consumers on the move — unless of course you believe that the tail can wag the dog!
 

ITU-T: Developing standards to fight driver distraction

The telecommunication standardization sector of the International Telecommunications Union is poised to play a crucial role in tomorrow’s connected cars. And yet, many people in the auto industry have never heard of the ITU, or of its standardization sector, the ITU-T. So let’s start with a quick introduction.

The ITU is the United Nations agency that deals with telecommunications. More specifically, it is where governments come together to agree on international standards for Information and Communications Technologies (ICTs).

Currently, the ITU-T is developing recommendations for ICTs that interact with drivers — examples include networked-based navigation systems, web browsers, and mobile phones. These recommendations will apply to networks; to applications accessed by drivers; and to connected, nomadic, and factory-installed devices. Strictly speaking, these recommendations are voluntary, which is precisely why they are referred to as “recommendations.” However, they can become binding if mandated by regional government agencies or customers.

ITU-T recommendations for the car are needed for several reasons:

• ICTs are moving into the automotive cockpit — Increasingly, ICT systems under the scope of the ITU-T are finding their way into the cockpit and interacting with drivers. Technology-related driver distraction has been recognized as a global problem that needs to be addressed. Internationally agreed guidance on the design and performance of these systems can help increase safety.
   
• Easy-to-find guidance for ICT community — ICT designers, developers, and application authors need easy access to guidance on design and performance requirements for ICT systems that interact with drivers. The ICT community consults ITU-T recommendations for guidance and requirements on ICT systems. Therefore, there is real value in having ITU-T recommendations that can serve as a reference to existing driver distraction-related standards and to fill any standardization gaps.
   
• Internationally agreed standards — The ITU-T is where governments come together to agree on international telecommunications standards. This is what makes these standards unique. It also gives them more weight with regional Standards Development Organizations (SDOs) and regulatory authorities. There is even value in an ITU-T recommendation that simply references existing standards since it will have gone through the ITU-T approval process.

Focus group on driver distraction
Last year, I became the founding chair for the ITU’s Focus Group on Driver Distraction (FG Distraction), which was created to pull expertise from the automotive industry and human factors experts into the ITU-T standardization process. The group currently has 3 planned ITU-T recommendations related to driver distraction:

G.SAM — recommendation on mechanisms for managing the situational awareness of drivers:

 

G.V2A — recommendation on an automotive interface (for instance, APIs) for applications external to the vehicle gateway:



P.UIA — recommendation on automotive user interface requirements:



FG Distraction will finalize pulling together input from industry and human factors experts by December 2012. Approved ITU-T recommendations are not expected until late 2013.

Want to contribute?
If you’d like to participate in FG Distraction, the group is open to any individual from a country that is a member of ITU and who is willing to contribute to the work (which is just about everyone). This includes anyone who is also a member or representative of an interested SDO. There is no cost to participate.

For more information about FG Distraction and upcoming meetings, click here. The next meeting will take place on April 4-5 at the SAE Headquarters in Troy, Michigan, USA.
 

Speech interfaces: UI revolution or intelligent evolution?

Speech interfaces have received a lot of attention recently, especially with the marketing blitz for Siri, the new speech interface for the iPhone.

After watching some of the TV commercials you might conclude that you can simply talk to your phone as if it were your friend, and it will figure out what you want. For example, in one scenario the actor asks the phone, “Do I need a raincoat?”, and the phone responds with weather information.

A colleague commented that if he wanted weather information he would just ask for it. As in “What is the weather going to be like in Seattle?” or “Is it going to rain in Seattle?”.

Without more conversational context, if a friend were to ask me, “Do I need a raincoat?”, I would probably respond, “I don’t know, do you?” — jokingly, of course.

Evo or revo?

Are we ready to converse
with our phones and cars?

Kidding aside, systems like Siri raise an important question: Are we about to see a paradigm shift in user interfaces?

Possibly. But I think it will be more of a UI evolution than a UI revolution. In other words, speech interfaces will play a bigger role in UI designs, but that doesn't mean you're about to start talking to your phone — or any other device — as if it’s your best friend.

Currently, speech interfaces are underutilized. The reasons for this aren't yet clear, though they seem to encompass both technical and user issues. Traditionally, speech recognition accuracy rates have been less than perfect. Poor user interface design (for instance, reprompting strategies) has contributed to the overall problem and to increased user frustration.

Also, people simply aren't used to speech interfaces. For example, many phones support voice-dialing, yet most people don't use this feature. And user interface designers seem reluctant to leverage speech interfaces, possibly because of the additional cost and complexity, lack of awareness, or some other reason.


"Relying heavily on speech can lead
to a suboptimal user experience..."
As a further complication, relying heavily on speech as an interface can lead to a suboptimal user experience. Speech interfaces pose some real challenges, including recognition accuracy rates, natural language understanding, error recovery dialogs, UI design, and testing. They aren't the flawless wonders that some marketers would lead you to believe.

Still, I believe there is a happy medium for leveraging speech interfaces as part of a multi-modal interface — one that uses speech as an interface where it makes sense. Some tasks are better suited for a speech interface, while others are not. For example, speech provides an ideal way to provide input to an application when you can capitalize on information stored in the user’s head. But it’s much less successful when dealing with large lists of unfamiliar items.

Talkin' to your ride
Other factors, besides Apple, are driving the growing role of speech interfaces — particularly in automotive. Speech interfaces can, for example, help address the issue of driver distraction. They allow drivers to keep their “eyes on the road and hands on the wheel,” to quote an oft-used phrase.

So, will we see a paradigm shift towards speech interfaces? It's unlikely. I'm hoping, though, that we'll see a UI evolution that makes better use of them.

Think of it more as a paradigm nudge than a paradigm shift.

---

Recommended reading

Situation Awareness: a Holistic Approach to the Driver Distraction Problem
Wideband Speech Communications for Automotive: the Good, the Bad, and the Ugly

 

Wanted: Haunted Vehicles

Halloween is just around the corner, and that reminds me of the haunted room at Lucent Bell Labs. Mind you, it wasn’t really haunted. But for a moment, I was convinced.

Let me explain. As I entered the room, I could hear two of my colleagues talking to each other, and by the sound of their voices, they were both sitting right in front of me. But when I looked, I could see only one person. Creepy, to say the least.

It took a few seconds, but I finally realized what was happening: The other colleague was in a different room, talking over a perfectly tuned prototype of a conference phone. The sense of presence was so real that I couldn’t help but feel we were all in the same room — even after I became aware of the “trick” being played!

It was then that I realized it: We don’t know what we’re missing until we experience it.

Making it real
Current telephone calls don’t sound like face-to-face conversations because the telephone network and terminals band-limit speech from about 50-10000 Hz down to 300-3400 Hz. To make matters worse, the phone’s single channel of audio eliminates spatial information about the sound source. As a result, we perceive most sounds as coming from the same point in space.

But here's the thing: The historical reasons for transmitting these single-channel narrowband speech signals no longer apply. Current technologies — such as wideband speech coders, spatial audio, and VoIP — are enabling speech communications with wider bandwidth speech and greater spatial information.

Many in the industry refer to these next-generation telecommunications systems as telepresence systems. “Telepresence” refers to the degree of realism created by a telecommunications system. Traditional systems have low telepresence while newer systems that use wider bandwidth speech and spatial audio have high telepresence.

Some people believe that a visual display is a must-have for a telepresence system. In reality, a display can decrease telepresence if its quality is poor. Experience shows that an audio-only system can have such high telepresence that people can't distinguish it from face-to-face communications — witness my haunting experience at Lucent Bell Labs.

Until recently, widespread deployment of telepresence systems has hit a roadblock: lack of standardization. Fortunately, the IETF CLUE Working Group and ITU-T Study Groups 16 and 12 are actively developing standards to remedy this situation.

Pimp my ride with telepresence
Telepresence systems have a lot to offer in an automotive environment. For instance, they could:

• reduce driver distraction
• make it easier to understand speech in the presence of vehicle noise
• reduce the fatigue that comes from trying to understand a degraded voice signal

Moreover, a telepresence system makes the talker on the far end of the phone connection sound more like they are in the vehicle; it also makes the talker easier to identify.

Successful deployment of telepresence in an automotive environment depends on several factors:

• attention to the design of vehicle platforms
• use of high-performance acoustic processing algorithms (AEC, NR, etc.), such as those provided by the QNX acoustic processing suite
• the ability to transport telepresence signals between telephony terminals — this is being enabled by increased VoIP availability (via LTE, for instance)

I don't know about you, but I'm looking forward to the day when my vehicle is haunted like that lab in New Jersey!

For additional reading on this topic, download the whitepaper, "Wideband Speech Communications for Automotive: the Good, the Bad, and the Ugly".