Autonomous cars? Suddenly, I’m not so skeptical

Guest post from Emil Dautovic, European automotive business development manager for QNX Software Systems

As a driving enthusiast, I have always felt a bit skeptical about the notion of autonomous cars. The reason is simple: I actually enjoy driving and don’t want someone else to do it for me, in this case the car itself.

Recently, however, my skepticism has begun to soften. I am fascinated, for example, by the SARTRE road train project, where a lead vehicle takes responsibility for a platoon of semi-autonomous cars. Also, recent research from the U.S. Highway Loss Data Institute suggests that, when it comes to some driving tasks, ADAS systems can already put many human drivers to shame.

Autonomous drive will become especially important when today’s “always on” generation starts to buy cars in earnest. They will, no doubt, want to consume multimedia and interact through social media even while on the road, and automakers will need to accommodate them.

HMIs with more (and less) distraction
What would this mean for car makers? Among other things, the infotainment system in a self-driving car could offer an HMI mode that gives the driver more freedom to pay attention to non-driving activities. When the car subsequently needs a human driver (for instance, it becomes disconnected from a road train), the infotainment system could disable these features and immediately go back to a less distracting user interface.

Also, driver assist systems — such as those for detecting animals and pedestrians — would need to be integrated with the road train system to decide how to react when, say, a rabbit runs in front of the car. For instance, should the car brake and warn other cars of the fact, or would it be safer to simply keep driving? It will be interesting to follow this initiative and see how the technical and business aspects evolve, and how, for example, the owner of the lead vehicle will be paid.

For another interesting example of research into autonomous drive, check out the BRAiVE project led by the VisLab team at the University of Parma. The BRAiVE project uses a variety of sensors, with a focus on low-cost alternatives that could realistically integrated into in production cars.

Bells and whistles
So what kind of impact could all this have on a company providing automotive software platforms?

There will, I believe, be an increased demand for a platform that could run all of these applications, enabling the advanced use cases while ensuring that critical functions always have enough processor power. And, of course, the platform will have to be reliable. If this same platform could offer all the bells and whistles available in consumer electronics and demanded by younger drivers, the self-driving future might prove to be a bit closer than we think.

By the way, if you’re unfamiliar with the SARTRE road train project, check out this video:

---

More about Emil
Emil Dautovic is an automotive business development manager at QNX Software Systems, where he is responsible for the European automotive market. Prior to joining QNX, he worked as a business area manager for The Astonishing Tribe (TAT), where he built TAT's automotive business from scratch and helped transform the company into an important player in the automotive HMI field with leading automotive OEMs and tier ones. He has also worked at AU-System (later Teleca and Obigo), where he served as a consultant on GSM base station development and as a sales representative serving mobile phone OEMs and ODMs worldwide. Emil holds an M.Sc. in Electronic Engineering from Lunds Tekniska Högskola.

Trend Spotting at SAE Convergence 2012

Guest post from automotive journalist Doug Newcomb

One of the Technical Sessions at the semi-annual SAE Convergence in Detroit on October 16 and 17 is titled Mega Trends and Their Effect on Automotive Electronics. While you’ll have to wait to find out what the participating executives, engineers, and analysts will reveal in the session concerning the rapidly evolving car technology space, here are three areas that are bound to be hot topics at the show.

Driver Distraction
This issue is at the forefront of everyone’s minds — automakers, suppliers, safety advocates, government officials, and consumers — as cars become increasingly connected. In order to help drivers keep their eyes on the road and hands on the wheel while still accessing the features they want, car companies and suppliers like QNX are developing cutting-edge technologies ranging from intuitive and configurable touchscreen displays to more accurate voice-activation systems that make control easier and less distracting.

Automakers are also being proactive in anticipating distractions: Ford is developing technology that assesses a driver’s workload so that some features can be deactivated in certain situations, and BMW’s pioneering work in “pupilometry” helps determine how drivers visually react when receiving information behind the wheel.



Ford's driver workload estimator (source Ford)

Standards
As more automakers integrate portable devices into the dash, drivers are increasingly frustrated by the fragmentation that’s occurring with first-generation systems. Features that are available for one smartphone platform may not be available for another, for example, and incompatibility issues are common. A push for an industry-wide standard has resulted in the Car Connectivity Consortium (CCC), of which QNX is a member. With MirrorLink, CCC’s industry-wide standard, portable device integration would be more straightforward and seamless for consumers. Getting all parties onboard will take significant effort though, since automakers have traditionally developed proprietary systems. But MirrorLink has substantial support, and the HomeLink system that’s allowed integration of garage-door openers into vehicles for years shows that such standards can be achieved.

Autonomous Cars
Two years ago, self-driving cars would have seemed like a distant sci-fi dream. But since the last SAE Convergence in 2010, Google has logged more than a quarter of a million miles with its fleet of self-driving Toyota Prius and Lexus RX450h vehicles. And this year the company has been instrumental in pushing through legislation that’s made self-driving cars legal in Nevada and California.

Audi is another pioneer in the space, developing an autonomous TT that drove solo up Colorado’s Pikes Peak. BWM has also debuted self-driving technology, and Cadillac recently revealed that its semi-autonomous Super Cruise lane-keeping technology will be available by the middle of the decade. Plus, Google’s announcement of its intention at the SAE World Congress in April to work directly with automakers and suppliers on self-driving technology will undoubtedly help accelerate this game-changing trend.

These are three topics are sure to be heavily discussed — and debated — at SAE Convergence 2012. Stop by the QNX booth during the show to see what the company is doing in these and other areas — or to share what trends you’ve spotted.

---

More about Doug
A widely respected reporter and editor with nearly three decades of experience in automotive journalism, Doug Newcomb currently writes for WIRED Autopia and for his own car technology portal, dougnewcomb.com. In 2008, he joined Edmunds.com as a senior editor, where he created the site’s Car Technology section. Prior to Edmunds, he worked as an editor for a variety of automotive publications, including Car Audio and Electronics, Car Stereo Review, and Road&Track Road Gear; he also contributed to many others, including Popular Mechanics, MSN Autos, Corvette Quarterly, and SEMA News. In 2008, he published his first book, Car Audio for Dummies (Wiley).

MirrorLink misunderstood: 8 myths that need busting

If you're new to MirrorLink, it's a technology that bridges the mobile phone and the car. It allows specially written apps running on the phone to be displayed on the car's head unit, where the user can interact with them.

MirrorLink is intended to extend the life of in-vehicle systems by allowing them to interact with mobile content and to support new features that didn’t exist when the car rolled off the assembly line.

Here's an illustration of how it works:


MirrorLink in-car communication. The protocol between the head unit and the phone can run over several transports, including USB, Bluetooth, or Wi-Fi. This example assumes Bluetooth for the audio back-channel.

When I talk to people in the automotive and mobile industries, I find they share a number of common misconceptions about MirrorLink, which I’d like to clear up. So let's get started, shall we?

1. MirrorLink is an Android technology. In fact, MirrorLink works with multiple mobile platforms. Phones using Android can support it, but so can phones from any other phone maker that supports the standard. Even Apple phones could support it, though Apple has currently chosen to go their own route with Apple-specific solutions.
   
   
2. MirrorLink allows any mobile app to run in the car. This is incorrect. A MirrorLink app can run in the car only if the car maker grants “trust” to that app. Each car maker has a different concept of what brands to promote, what features are safe, or what works well with each car. So, in reality, each app will be enabled depending on the individual make — or even model — of car.
   
   
3. MirrorLink promotes “driver distracting” apps. Also incorrect. MirrorLink is an enabling technology that doesn’t promote any type of app in particular. In fact, because the car maker must grant trust to an app, the app developer can't control what apps run in the car. That responsibility remains the domain of car makers, who tend to avoid anything that will cause distraction when displayed on a front-seat screen.
   
   
4. MirrorLink is the only way to connect an app to the car. There are in fact two others: iPod Out and HTML5. Apple supports iPod Out for Apple devices, which allows selected applications to output analog video to the head unit. (Note that the new iPhone 5 doesn’t support iPod Out.) HTML5 also allows mobile apps to run in the head unit, though its use in car-to-phone bridging is still in the early stages. QNX Software Systems has demonstrated concept vehicles that use BlackBerry Bridge (an HTML5-based technology) to connect an HTML5 app on a BlackBerry phone to the car’s head unit.
   
   
5. Mobile app makers will benefit most from MirrorLink. In fact, car makers may end up taking best advantage of the technology. That’s because they can use MirrorLink to customize and create apps, and to refresh those apps as a way of delivering fresh, new functions to their customers. MirrorLink gives them the ability to do this using a standardized protocol supported by most mobile platforms. Car makers could use MirrorLink very effectively, even if they never allowed any third party apps into their cars.
   
   
6. HTML5 and MirrorLink are incompatible. Not necessarily true. Current versions of MirrorLink use the VNC protocol to exchange graphical data. None of the advantages of HTML5 would be incompatible with a future version of MirrorLink; in fact, some members of the Connected Car Consortium (CCC), including QNX Software Systems, would likely be interested in merging these two standards. That would result in a new version of MirrorLink that uses HTML5 as the underlying communication protocol. (The MirrorLink specification is controlled by the Car Connectivity Consortium, of which QNX is a member.)
   
   Even if MirrorLink does go to HTML5, the industry would still need a VNC-based form of MirrorLink. VNC has much lighter requirements on the head-unit side, so it makes more sense than HTML5 if the car doesn’t have a high-powered CPU or lots of memory. The broadest possible option would be to have phone apps support multiple versions of MirrorLink (today's version with VNC plus a future version with HTML5) and to use whichever one makes sense, depending on what the car supports.
   
   
7. MirrorLink obviates the need for car-downloadable apps. Yes, MirrorLink capability is somewhat similar in purpose to downloading apps into the car; they both extend the functionality of the car after it leaves the factory. Because the customer’s phone will almost certainly be newer than the car’s electronics, it will have a faster CPU, giving the raw speed advantage to a MirrorLink app on the mobile. The MirrorLink app will also have guaranteed data access since the hosting phone will always have a data pipe — something that isn't certain on the car side of the equation.
   
   On the other hand, MirrorLink doesn’t give an app access to car features that would available to a car-downloaded app — features such as vehicle bus access, telematics features, or the navigation system. Also, a car-downloaded app would likely have a faster HMI than any off-board app, even if the mobile had a faster CPU, because of latencies inherent to screen replication. The car-downloaded app would also have better visual integration, as it could take full advantage of the car features, instead of appearing as a bolt-on product. Other factors, based on automaker control, compatibility, or product roadmaps could also favor an in-car solution. Even if you could address some of these issues, there would still be enough reasons for MirrorLink and an auto app store to live side-by-side.
   
   
8. MirrorLink apps can be built today. This is technically true. But, in their enthusiasm, new converts can sometimes forget that cars need to support MirrorLink for anything to actually work. Currently, only aftermarket car stereos support MirrorLink; no production vehicles support it. So if you’re a mobile app developer, the market for MirrorLink apps today is negligible. But expect this situation to improve dramatically over the next two to three years as production vehicles start to ship with this capability built-in.

Will adaptive cruise control spell the end of traffic jams?

Did you know that rear-end collisions account for about 30% of car crashes? For that reason alone, widespread adoption of adaptive cruise control (ACC) can’t come too soon. ACC helps prevent such collisions in two ways: 1) by maintaining a safe, preset distance from the car ahead; and 2) by applying the brakes quickly if that car comes to a sudden stop — more quickly, in fact, than most humans.

Good news is, ACC may soon become pervasive. The folks at Global Industry Analysts crunched some numbers and determined that annual installations of ACC systems will reach 6.9 million units by 2017.

Mind you, ACC isn’t just about safety; it’s also about traffic flow. For instance, a study by Suzuki and Nakatsuji (2003) suggests that travel times shrink significantly when at least 20% of vehicles on the road use ACC. And a study by Kesting et al. (2008) suggests that, in some scenarios, traffic congestion simply disappears when 25% of vehicles use ACC.

Example of adaptive cruise control
Source: Volvo

The picture isn’t all rosy, however. ACC may improve traffic flow, but not in every situation, such as merging from an on-ramp onto a freeway. That said, a study by L. C. Davis (2010) suggests that a technique called cooperative merging can significantly the improve the performance of ACC in this scenario. Meanwhile, a study by Jerath and Brennan (2010) suggests that the benefits associated with ACC may come at a possible cost — “self-organized” traffic jams. This effect, caused mostly by human behavior, may occur in a traffic system where most, but not all, cars use ACC.

Caveats aside, ACC systems continue to evolve. Some drivers tend to slam their brakes and use heavy throttle in traffic, creating congestive shockwaves that ripple down the highway. According to J.C. Power, newer versions of ACC help alleviate this problem by smoothly modulating brakes and throttle in stop-and-go traffic.

And now, a look at ACC from 1939…
If you think the concept of ACC is relatively new, think again. Over 70 years ago, GM created a “Futurama” exhibit for the 1939 World's Fair that showcased a scale-model highway in which cars automatically maintain a safe, efficient distance from one another.

GM predicted this technology would be in place by 1960. They got the timing wrong, but the idea right. Click the video to see a surprisingly prescient look at the car of the future — I’ve already bookmarked the spot for you:



What about you? Have you had much experience with ACC? And if so, has it helped or hindered your driving experience?

Am I crazy for talking to my car?

Earlier this afternoon, I participated in a connected car panel at SpeechTEK 2012, hosted by our friend Mazin Gilbert from AT&T. The other panelists included Greg Bielby of VoltDelta, Thomas Schalk of Agero, and Hakan Kostepen of Panasonic.

Even though Mazin did a fantastic job, not every panelist had a chance to answer every question. I was itching to answer some, so here are my responses to the questions that I didn't get to answer, or where I feel I could have provided a more complete response.

Have speech technologies matured to the point where they can be used robustly in the car? The general answer to this question from the panel was yes, but I think the real answer is a qualified yes. The technologies exist, but often aren't applied or may need auto-specific adaptations to handle in-cabin noise or other issues. Natural language recognition was an oft-stated driving technology, but a missing piece to the puzzle is hybrid recognition. I don't mean pushing recognition wholesale to the cloud, like Siri does. I mean a true split of the recognition effort, where each part does what it’s best at. Put the front half of acoustic processing in the vehicle to clean up the audio and convert the waveform to frequency-domain data, then send the data to the cloud-based server. The cloud server can then parse and interpret the data, and send back the result.

Hybrid speech rec solves three problems at once: better audio signals (the car can improve audio specific to the in-cabin environment), better cost (frequency data is far more compressed than raw audio, so you pay less for data transfer), and better responsiveness (hybrid rec gives the server time to start working on the response while it's coming in instead of waiting for the whole utterance to finish before starting).

Is driver distraction a major business driver, or is it the "Siri effect"? Currently, the car industry seems to use driver distraction as a reason to push a lot of features into speech. Many of those uses are gimmicky. Personally, I don't care if I can set my climate control system with voice — why would I when I can simply turn a dial? I once had someone ask me about the feasibility of adding voice recognition commands for rolling down the windows. I asked him, "Yes, but wouldn't people just push the window button?"

We shouldn’t implement speech commands just because we can. They may have contributed to excitement in the early adopter crowd, but we're beyond that now. Mind you, there are some seriously useful ways to use voice. For instance, any time you need to pick from a huge number of choices, voice recognition is the natural way to go. Calling contacts ("Call Sarah Potter"), entering destinations ("Go to 3121 South Park Street"), or picking music ("Play Audioslave") are all much easier than using an HMI to enter the same information, and safer to boot. It just has to work consistently and accurately.

Will car makers see more speech moving to the cloud, or will it be a hybrid of cloud and embedded? I disagree with the majority of the panel on this one, and, I think, the majority of people in the industry. Most auto people believe a hybrid between embedded and cloud allows the best of both worlds — good recognition and updatability when connected, and consistent reliability when not. My colleague Andrew Poliak also champions this view with a memorable catch phrase: Zombie Apocalypse. That is, you still want the system to work, albeit partially, when the infrastructure isn't available.

But if you ask me, everyone is missing the point — theirs is a technology-centric point of view. Everyday customer acceptance of a particular technology is notoriously harsh: if it doesn't work well, it gets rejected out of hand. Good cloud solutions beat an embedded solution hands-down; they just need some improvements (see my hybrid bullet above). Once a customer experiences a good solution, they will become frustrated with one that performs poorly. In my opinion, it's better not to offer the service at all, than to try a graceful degradation of capability, because most customers won't understand or care. Spend the effort instead on making sure you always have an acceptable cloud connection — either through multiple redundant mechanisms or a car-based powerful antenna — and you'll be better off. Even when the car knows some data that the cloud doesn't (like a mobile's contact list or music selection), there's no need to handle that on the embedded side. The cloud recognition server is powerful enough to not require the data set a priori. And I think we can predict an eventual migration of phone data to cloud-based data (or cloud-synchronized data) that makes the car's knowledge either easily transferrable or less relevant.

Who makes money, and how, from voice-enabled agents or voice services? This was one of the best questions of the panel, because nobody really knows the exact model, but everybody agreed that customer tolerance is very low. The most likely candidate is ad-based revenue. This doesn't mean reading ads aloud to the driver, but rather, positively influencing search results for either active or temporary situation-based points of interest (POIs). Depending on how valuable the service is to the driver, there will still be an option for service-based payments and high-value apps.

Standards and building mobile apps — will it come? You need standards if you want to build an app platform that will promote application creation and adoption. That's what we're doing with the QNX CAR 2 application platform — creating a way for someone other than the car companies to join the ecosystem and to deploy their apps to the car in a controlled way. But don't forget, you need a standard way to deploy apps for the cloud half of the recognition, too.

To close, let me share two photos. One was taken outside the Marriott Marquis, the hotel hosting the conference just off of Times Square in NYC. The other is from our PR agency, Breakaway Communications. What do they have in common? Wooden water towers. Sorry, I couldn't help myself; I just love those things. They just look so quaint in a city full of glass and brick.

For safety’s sake, why don’t cars just disable phones?

With all the focus on driver distraction, this is a question that I get asked occasionally. It’s a simple question, with a less than simple answer.

Using technology to control inappropriate phone use has been a topic at some of the driver distraction meetings I've attended. One proposed solution involves a technique called micro location — using ultrasonic waves to identify where in the cabin the phone is located. There are other ways to triangulate the phone's position, but they all require coordination between the phone and car. Knowing where the phone resides in the car is a requirement, as most passengers wouldn’t be happy to have their phone automatically disabled, just because they’re in the car. And the solution can’t be based only on the GPS speed of the phone, or you’d have lots of irate bus, taxi, train, or subway riders.

The fact is, unless all phone makers and car makers agree on the same standard, there's no incentive for either side to build half of a feature. You’d need to deploy potentially expensive technology that wouldn’t work unless you pair exactly the right phone with the right car. This likely won't happen unless companies are legislated to do so.

Given the speed of automotive development, it’s impossible for the car guys to build a technology that the phone guys won't leave in the dust, unless some guarantees are put in place. The adoption of Bluetooth is a good example. It took years before Bluetooth became widespread in phones, but its adoption had more to do with Bluetooth earpieces, not connections to cars. Car makers took a long time to roll out Bluetooth support as a standard feature because too many phones either didn't have it or had an implementation that wasn't fully compatible. Eventually, the two markets synchronized, but it took several years.

One argument against a technology-mandated disable is that not all jurisdictions agree on what is, or isn’t, allowable. In the US, 45 out of 50 states have some form of prohibition against using phones in cars. But what is disallowed varies widely by state — some don't allow any use of the phone (even hands-free), some prohibit teenagers but no other age groups, some disallow texting but not hands-free, some disallow use for commercial vehicles but not private vehicles, and some allow everything.

Another argument against a technological solution is that people can be educated to assume responsibility for their behavior. For example, why don't all cars have a blood alcohol level blow-tester hooked up to the ignition? Technically it's possible, but it's very expensive to do it from the car maker's standpoint. One could argue that it is worth it to have cars protect us from ourselves. But as a society, we've decided that, in the case of drunk driving, we are willing to give people back the responsibility. Rather than control the problem with technology, we socialize and educate people that driving intoxicated is an undesirable behavior.

We could, of course, decide to do the same with mobile technology, by educating personally instead of solving technically. This approach may make more sense than a technology-based prohibition: technology always moves at light speed compared to legislative mechanisms of control.
 

Report from CTIA Wireless: Apps in the Car

You wouldn’t think that CTIA Wireless, a mobile show, would be a good venue for a car guy. But automotive journalist Doug Newcomb put together a set of panels that managed to attract everyone from the automotive industry who attended the show.

I met a good number of friends from a variety of automakers, tier one suppliers, and hardware and software vendors. I also had the distinct pleasure of participating in one of Doug's panels, which was moderated by Damon Lavrinc of WIRED.

The topic was the future of apps in the car, and it generated a spirited discussion. Panel participants included Geoff Snyder from Pandora, Michelle Avary from Toyota, Henry Bzeih from Kia, and Scott Burnell from Ford — all experts on the topic.

Andy speaking on the

apps panel. Videos of all

the panels are now online.

In general, we agreed: apps are coming to the car. They have already arrived in several cases, and it’s only a matter of time before they come to mass-market vehicles. And apps are not for North American alone: it's a worldwide phenomenon.

Mind you, we engaged in lively debate on a number of questions: What role does the mobile app developer play? How to deal with the fragmentation caused by different OEM app platforms? How to deal with driver distraction? And when will the "one man app" ever make it into the car? We all had good and varied opinions on these topics, and the session was very well received by the audience.

Derek Kuhn, QNX vice president of sales and marketing, also participated in a panel session, titled "Can we all just get along… for the consumer's sake?". That panel focused on how the industry as a whole can create a more seamless experience for the consumer. Derek's co-panelists included Mark Harland from GM, Leo McCloskey from Airbiquity, Brian Radloff from Nuance, and Niall Berkery from Telenav.

Did I mention? Videos of all the panels are now on Doug Newcomb's website — check them out!
 

Find me a Starbucks! QNX concept car showcases power of WATSON speech engine

Yes, you can talk to the QNX concept car and tell it what to do.

Recently, our friends at AT&T invited us to bring the concept car to their "Living the Networked Life" event in New York. We said yes, of course! After all, what could be cooler than riding the streets of Gotham City in a digitally pimped-out Porsche 911?

Kidding aside, the event provided an excellent opportunity to demonstrate how the car takes advantage of WATSON, AT&T's natural-language speech engine. To get an idea of what WATSON can do, check out this video from Terrence O'Brien of Engadget:



For the full Engadget article, click here. And stay tuned for more updates from the Living the Networked Life event.
 

Crossing the boundaries: Cooperation across industries will fuel the connected car

A guest post by Brian Salisbury of Telecommunication Systems (TCS)

Connected car – these two words appear together more and more these days. Consider, for example, two events that took place in February: The Connected Car Executive Lunch organized by Fierce Wireless and held during the Mobile World Congress in Barcelona, and the Telecom Council’s Mobile Forum: Connected Car meeting hosted by Marvell Semiconductor in Silicon Valley.

Speakers at these events came from mobile operators (AT&T Mobility, Orange, Sprint, Verizon), auto manufacturers (Ford, Hyundai, Nissan, Toyota), and platform and solution providers (Nokia, Pioneer, QNX Software Systems, TCS). No doubt about it, the car is now connecting industries.

Although these two events were held on different continents, the topics on the minds of attendees were very similar:

• Who “owns” the customer?
• Will the connection be part of the car, or brought to the car by its driver?
• How can the “wild west” of the Internet be safely incorporated into the car?
• What is the business model for such a multi-part solution?
• What will be the “killer app” for connected car, or is there no such thing?

The presentations and discussions were diverse, as each group sought to define their role in terms that extend logically from their own past experience, and that could provide them with some control over the outcome. Thankfully, every group shared the common goal of making sure that connected cars are safe cars, and that the introduction of new connected services doesn’t create driver distraction problems.

We are clearly on the verge of a new generation of services being extended into the car that can enhance many aspects of owning, operating, and riding in tomorrow’s vehicles. Those of us fortunate enough to be part of one of these groups will have some amazing opportunities to bring the best of our respective industries into this new space, and to build new relationships across industry boundaries.

For an example of how TCS is helping to enable the connected car, check out this post on the VW Polo that was showcased at Mobile World Congress — Ed.

---

Here’s a little more about Brian and TCS:

Brian Salisbury is director of business development at TeleCommunication Systems, Inc. (TCS), where he is responsible for developing new business with OEMs, platform providers, and developers in the LBS ecosystem. Brian has worked in the mobile industry for more than 25 years, with most of that experience being in mobile data and location-based services, and within semiconductor, device manufacturer, and network operator companies.

TCS (NASDAQ: TSYS) is a world leader in highly reliable and secure mobile communication technology. TCS infrastructure forms the foundation for market leading solutions in E9-1-1, text messaging, commercial location and deployable wireless communications. TCS is at the forefront of new mobile cloud computing services providing wireless applications for navigation, hyper-local search, asset tracking, social applications and telematics.
 

ITU-T: Developing standards to fight driver distraction

The telecommunication standardization sector of the International Telecommunications Union is poised to play a crucial role in tomorrow’s connected cars. And yet, many people in the auto industry have never heard of the ITU, or of its standardization sector, the ITU-T. So let’s start with a quick introduction.

The ITU is the United Nations agency that deals with telecommunications. More specifically, it is where governments come together to agree on international standards for Information and Communications Technologies (ICTs).

Currently, the ITU-T is developing recommendations for ICTs that interact with drivers — examples include networked-based navigation systems, web browsers, and mobile phones. These recommendations will apply to networks; to applications accessed by drivers; and to connected, nomadic, and factory-installed devices. Strictly speaking, these recommendations are voluntary, which is precisely why they are referred to as “recommendations.” However, they can become binding if mandated by regional government agencies or customers.

ITU-T recommendations for the car are needed for several reasons:

• ICTs are moving into the automotive cockpit — Increasingly, ICT systems under the scope of the ITU-T are finding their way into the cockpit and interacting with drivers. Technology-related driver distraction has been recognized as a global problem that needs to be addressed. Internationally agreed guidance on the design and performance of these systems can help increase safety.
   
• Easy-to-find guidance for ICT community — ICT designers, developers, and application authors need easy access to guidance on design and performance requirements for ICT systems that interact with drivers. The ICT community consults ITU-T recommendations for guidance and requirements on ICT systems. Therefore, there is real value in having ITU-T recommendations that can serve as a reference to existing driver distraction-related standards and to fill any standardization gaps.
   
• Internationally agreed standards — The ITU-T is where governments come together to agree on international telecommunications standards. This is what makes these standards unique. It also gives them more weight with regional Standards Development Organizations (SDOs) and regulatory authorities. There is even value in an ITU-T recommendation that simply references existing standards since it will have gone through the ITU-T approval process.

Focus group on driver distraction
Last year, I became the founding chair for the ITU’s Focus Group on Driver Distraction (FG Distraction), which was created to pull expertise from the automotive industry and human factors experts into the ITU-T standardization process. The group currently has 3 planned ITU-T recommendations related to driver distraction:

G.SAM — recommendation on mechanisms for managing the situational awareness of drivers:

 

G.V2A — recommendation on an automotive interface (for instance, APIs) for applications external to the vehicle gateway:



P.UIA — recommendation on automotive user interface requirements:



FG Distraction will finalize pulling together input from industry and human factors experts by December 2012. Approved ITU-T recommendations are not expected until late 2013.

Want to contribute?
If you’d like to participate in FG Distraction, the group is open to any individual from a country that is a member of ITU and who is willing to contribute to the work (which is just about everyone). This includes anyone who is also a member or representative of an interested SDO. There is no cost to participate.

For more information about FG Distraction and upcoming meetings, click here. The next meeting will take place on April 4-5 at the SAE Headquarters in Troy, Michigan, USA.
 

Seamless connectivity is for more than online junkies

As much as I’m not always enamored with sitting behind a computer all day, I find being off the grid annoying. Remember this email joke?

You know you’re an online junkie when you:
• wake up at 3:00 am to go to the bathroom and stop to check your email on the way back to bed
• rarely communicate with your mother because she doesn’t have email
• check your inbox. It says ‘no new messages,’ so you check it again 

Even though this joke circulated several years ago, it still strikes a chord. The big difference now is that there’s no longer a subculture of ‘online junkies.’ From the time we wake up in the morning to the time we go to bed, we all want to be connected — and that includes when we get behind the wheel. So to this joke I would add:

• resent driving because it means going off the grid

At QNX, we’re working toward a seamless experience where people can enjoy the same connectivity whether they’re texting their spouse from the mall or checking traffic reports while driving down the highway. See what I mean:



For more information about the technology described in this video, visit the QNX website.
 

Video: The secret to making hands-free noise-free

 

Explaining a highly technical product to a broad audience is tough. To succeed, you must reach out to people on their own terms, without being condescending. Most people love a good explanation, but everyone hates being talked down to.

Case in point: The QNX Acoustic Processing Suite. This software runs in millions of cars and offers a benefit that everyone can relate to: clear, rich, easy-to-understand hands-free calls. But once you start explaining how the suite does this, it's easy to get mired in technical jargon and to forget the bigger picture — something that even a technical audience wants to see.

So we dropped the jargon and opted for a creative approach. It involves a marching band, a rock guitarist, and, for good measure, an electric fan with a really long extension cord. Seriously.

Intrigued yet? Well, then, grab some popcorn and dim the lights:




Interested in learning more about this technology? Check out the acoustic processing page on the QNX website.

BTW, companies that use the QNX Acoustic Processing Suite in their products include OnStar, whose FMV aftermarket mirror recently won a CES Innovations Design and Engineering Award.

Posted by Paul Leroux
 

Pimp your ride with augmented reality — Part II

Last week, I introduced you to some cool examples of augmented reality, or AR, and stated that AR can help drivers deal with the burgeoning amount of information in the car.

Now that we’ve covered the basics, let’s look at some use-cases for both drivers and passengers. Remember, though, that these examples are just a taste — the possibilities for integrating AR into the car are virtually endless.


AR for the driver
When it comes to drivers, AR will focus on providing information while reducing distraction. Already, some vehicles use AR to overlay the vehicle trajectory onto a backup camera display, allowing the driver to gauge where the car is headed. Some luxury cars go one step further and overlay lane markings or hazards in the vehicle display.

Expect even more functionality in the future. In the case of a backup camera, the display might take advantage of 3D technology, allowing you to see, for example, that a skateboard is closer than the post you are backing towards. And then there is GM's prototype heads-up system, which, in dark or foggy conditions, can project lane edges onto the windshield or highlight people crossing the road up ahead:



AR can be extremely powerful while keeping distraction to a minimum. Take destination search, for example. You could issue the verbal command, “Take me to a Starbucks on my route. I want to see their cool AR cups”. The nav system could then overlay a subtle route guidance over the road with a small Starbucks logo that gets bigger as you approach your destination. The logo could then hover over the building when you arrive.

You'll no longer have to wonder if your destination is on the right or left, or if your nav system is correct when it says, “You have arrived at your destination.” The answer will be right in front of you.

AR for the passenger
So what about the passenger? Well, you could easily apply AR to side windows and allow passengers to learn more about the world around them, a la Wikitude. Take, for example, this recent video from Toyota, which represents one of the best examples of how AR could make long road trips less tedious and more enjoyable:


 

Pimp your ride with augmented reality — Part I

The use of electronics is exploding in automotive. Just last week, Intel proclaimed that the connected car “is the third-fastest growing technological device, following smartphones and tablets.”

Ten years ago, you’d be hard-pressed to find a 32-bit processor in your car. Now, some cars have 4 or more 32 bitters: one in the radio, another in the telematics module, yet another in the center display, and still another in the rear-seat system.

Heck, in newer cars, you’ll even find one in the digital instrument cluster — the QNX-powered cluster in the Range Rover, for example. Expect to see a similar demand for more compute power in engine control units, drive-by-wire systems, and heads-up displays.


The Range Rover cluster displays virtual speedometers and gauges, as well as warnings, suspension settings, and other info, all on a dynamically configurable display.

What do most of these systems have in common? The need to process tons of information, from both inside and outside of the vehicle, and to present key elements of that data in a safe, contextually relevant, and easy-to-digest fashion.

The next generation of these systems will be built on the following principles:

• Fully integrated cockpits — Vehicle manufacturers see system consolidation as a way to cut costs and reduce complexity, as well as to share information between vehicle systems. For instance, your heads-up display could discreetly let you know who is calling you, without forcing you to take your eyes off of the road. And it could do this even if the smarts integrating your phone and your car reside in another cockpit component — the telematics module, say.
   
• Augmented reality — With all of the data being generated from phones, cloud content services and, perhaps more importantly, the vehicle itself, presenting the right information at the right time in a safe way will become a major challenge. This is where augmented reality comes in.

Augmented reality is a cool use of cameras, GPS, and data to create smart applications that overlay a virtual world on top of the real world. Here are some of my favorite examples:

AR Starbucks cups — Use your phone to make your coffee cup come alive:



AR Starwars — Blast the rebel alliance squirrels!



AR postage stamp — Add a new dimension (literally) to an everyday object:



And here are a couple more for good measure:

AR ray gun — Blast aliens around the house!

Wikitude AR web browser — Explore the world around you while overlaying social networks, images, video, reviews, statistics, etc.

Stay tuned for my next post, where I will explore how AR could enhance the driving experience for both drivers and passengers — Andrew.
 

A cool and innovative speedometer... for 1939

Paul Leroux

Earlier this week, I referred you to a whitepaper written by my colleagues Scott Pennock and Andy Gryc. In the paper, Scott and Andy argue that driver distraction is not, in fact, a problem of distraction, but of situational awareness, or SA. Boost a person's SA, and you improve their ability to drive safely.

But how, exactly, do you improve SA? The paper discusses various techniques, and I couldn't possibly do justice to all of them here. But one approach is to supplement the driver's eyes and ears with indicators and warnings, based on information from sensors, roadside systems, and other vehicles.

Here's an example: A system in your car learns, through cloud-based traffic reports, that the road ahead is slick with ice. It also determines that you are driving much too fast for such conditions. The system immediately kicks into action, perhaps by warning you of the icy conditions or by telling you to ease off the accelerator.

Too bad the engineers who designed the 1939 Plymouth P8 didn't have access to such technology. I'm sure they would have embraced it totally.

You see, they too wanted to warn drivers about excess speed. Unfortunately, the technology of the time limited them to creating a primitive, one-size-fits-all solution — the safety speedometer.

Color coded for safety
From what I've read, these speedometers switch from green to amber to red, depending on the car's speed. I've only seen still photos of these speedometers, but allow me to invoke the magic of PhotoShop and reconstruct how I think they work.

The safety speedometer has a rotating bezel, and embedded in this bezel is a small glass bulb. At speeds from 0 to 30 mph, the bulb glows green:



At speeds from 30 to 50 mph, the bulb turns amber:



And at over 60 mph, the bulb turns red:



Given the limitations of 1939 technology, the Plymouth safety speedometer couldn't take driving conditions or the current speed limit into account. It glowed amber at 30 mph, regardless of whether you were cruising through your neighborhood or poking down the highway. As a result, it was more of a novelty than anything else. In fact, I wonder if people driving the car for the first time would have focused more on watching the colors change than on the road ahead. If so, the speedometer may have actually reduce situational awareness. Oops!

Compare this to a software-based digital speedometer, which could take input from multiple sources, both within and outside the car, to provide feedback that dynamically changes with driving conditions. For instance, a digital speedometer could acquire the current speed limit from a navigation database and, if the car is exceeding that limit, remind the driver that they risk a speeding ticket.

That said, I have a soft spot for anyone who is (or was) ahead of their time. Some enterprising Plymouth engineers in the 30s realized that, with faster speeds, comes the need for even greater situational awareness. Their solution was primitive but it offered a hint of what, more than 75 years later, can finally become reality.
 

Some people drive me to distraction

Paul Leroux

Hey, have you ever panned your camera? It's really easy: You just track a moving subject with your camera and then squeeze the shutter while both you and the subject are in motion. It's a great technique for creating images that evoke a sense of speed, which makes it popular among photographers for Motor Trend, Car and Driver, and other automotive magazines.

When you pan, you never really know what kind of image you're going to get. Often, the results are interesting. And sometimes, they're downright interesting. Take this shot, for example:


Lattés and overdrive don't mix. Just sayin'.

Now, holding a cellphone while rocketing down the highway is just plain wrong. To anyone who does it, I have one thing to say: "You're endangering other people's lives for the sake of a f***ing phone call. Where the hell do you get off doing that?"

But look at this guy. He's isn't holding a phone, but a coffee — even worse. Just imagine if he gets into a situation that demands quick, evasive action. He will, in all likelihood, hold on to the cup for fear of burning himself. Whereas if he had a phone, he would simply drop it and put his hand back on the wheel.

Mind you, I have no data to prove that coffee cups poses a greater evil than cellphones. But the core issue remains: Cellphone use is just one of many factors that contribute to driver distraction. In fact, research suggests that cellphones account for only 5% of distraction-related accidents that end in injury.

So, even if every cellphone on the planet disappeared tomorrow, we would still have a massive problem on our hands. To that end, my colleagues Scott Pennock and Andy Gryc suggest a new approach to designing vehicle cockpit systems in their paper, "Situational Awareness: a Holistic Approach to the Driver Distraction Problem."

The paper explores how system designers can use the concept of situational awareness to develop a vehicle cockpit that helps the driver become more aware of objects and events on the road, and that adapts in-vehicle user interfaces to manage the driver’s cognitive load.

It's worth a read. And who knows, perhaps someone, someday, will develop a cockpit system that detects if you are sipping something and tells you what you need to hear: "Dammit Jack, put that cup down. It's not worth endangering other people's lives for the sake of a f***ing latté."

Building a hands-free future

The end of my street is governed by a three-way stop. The other morning I was backing out of my driveway when someone rolled past the stop sign and came within inches of hitting me. I stopped, glared at him, and resumed driving. Two stop signs later, the same guy squeezed past my car (in the same lane), completely oblivious to what he was doing.

Why was he driving like this? Probably because he was deeply engrossed in a conversation on his cell phone.

Where I live, using a handset while driving has been illegal for over a year. You cannot talk, you cannot text, you cannot “Facebook”, you cannot Tweet — even if you're stopped at a red light. This makes perfect sense to me. As a driver, your primary responsibility is to control the vehicle. And yet I see people texting on the freeway, talking on their cell phones, and doing who knows what else on an alarmingly regular basis.

The QNX-powered BMW
ConnectedDrive system

Society has become obsessed with mobile devices, and it will take more than legislation to change its behavior. The answer, I think, is to embrace the behavior in a way that makes it possible to interact socially while maintaining control of the car. We’ve seen great progress in hands-free/phone integration, and BMW ConnectedDrive offers an example of how drivers can access email and other smartphone services more safely.

This is the tip of the iceberg. Integrating the handset with the infotainment unit is going to change the way you interact with your car. Intelligently designed apps, combined with multi-modal human machine interfaces, will let you Tweet or update Facebook using speech recognition, keeping your eyes on the road.

Without taking your hands of the wheel, you’ll be able to call a friend and decide that you want to go to dinner, do a local search to find out what’s available, check a restaurant review on Yelp, make a reservation, text your friend back with the time and place, and aim your navigation system at the restaurant. And you’ll be able to do it using natural language. None of this “please say a name” stuff.

Seems futuristic? It’s not. People are working on it today. In fact, QNX-based systems, such as Toyota Entune, already offer a taste of this hands-free and highly personalized future.
 

Speech interfaces: UI revolution or intelligent evolution?

Speech interfaces have received a lot of attention recently, especially with the marketing blitz for Siri, the new speech interface for the iPhone.

After watching some of the TV commercials you might conclude that you can simply talk to your phone as if it were your friend, and it will figure out what you want. For example, in one scenario the actor asks the phone, “Do I need a raincoat?”, and the phone responds with weather information.

A colleague commented that if he wanted weather information he would just ask for it. As in “What is the weather going to be like in Seattle?” or “Is it going to rain in Seattle?”.

Without more conversational context, if a friend were to ask me, “Do I need a raincoat?”, I would probably respond, “I don’t know, do you?” — jokingly, of course.

Evo or revo?

Are we ready to converse
with our phones and cars?

Kidding aside, systems like Siri raise an important question: Are we about to see a paradigm shift in user interfaces?

Possibly. But I think it will be more of a UI evolution than a UI revolution. In other words, speech interfaces will play a bigger role in UI designs, but that doesn't mean you're about to start talking to your phone — or any other device — as if it’s your best friend.

Currently, speech interfaces are underutilized. The reasons for this aren't yet clear, though they seem to encompass both technical and user issues. Traditionally, speech recognition accuracy rates have been less than perfect. Poor user interface design (for instance, reprompting strategies) has contributed to the overall problem and to increased user frustration.

Also, people simply aren't used to speech interfaces. For example, many phones support voice-dialing, yet most people don't use this feature. And user interface designers seem reluctant to leverage speech interfaces, possibly because of the additional cost and complexity, lack of awareness, or some other reason.


"Relying heavily on speech can lead
to a suboptimal user experience..."
As a further complication, relying heavily on speech as an interface can lead to a suboptimal user experience. Speech interfaces pose some real challenges, including recognition accuracy rates, natural language understanding, error recovery dialogs, UI design, and testing. They aren't the flawless wonders that some marketers would lead you to believe.

Still, I believe there is a happy medium for leveraging speech interfaces as part of a multi-modal interface — one that uses speech as an interface where it makes sense. Some tasks are better suited for a speech interface, while others are not. For example, speech provides an ideal way to provide input to an application when you can capitalize on information stored in the user’s head. But it’s much less successful when dealing with large lists of unfamiliar items.

Talkin' to your ride
Other factors, besides Apple, are driving the growing role of speech interfaces — particularly in automotive. Speech interfaces can, for example, help address the issue of driver distraction. They allow drivers to keep their “eyes on the road and hands on the wheel,” to quote an oft-used phrase.

So, will we see a paradigm shift towards speech interfaces? It's unlikely. I'm hoping, though, that we'll see a UI evolution that makes better use of them.

Think of it more as a paradigm nudge than a paradigm shift.

---

Recommended reading

Situation Awareness: a Holistic Approach to the Driver Distraction Problem
Wideband Speech Communications for Automotive: the Good, the Bad, and the Ugly

 

When will I get apps in my car?

I read the other day that Samsung’s TV application store has surpassed 10 million app downloads. That got me thinking: When will the 10 millionth app download occur in the auto industry as a whole? (Let’s not even consider 10 million apps for a single automaker.)

There’s been much talk about the car as the fourth screen in a person’s connected life, behind the TV, computer, and smartphone. The car rates so high because of the large amount of time people spend in it. While driving to work, you may want to listen to your personal flavor of news, listen to critical email through a safe, text-to-speech email reader, or get up to speed on your daily schedule. When returning home, you likely want to unwind by tapping into your favorite online music service. Given the current norm of using apps to access online content (even if the apps are a thin disguise for a web browser), this begs the question — when can I get apps in my car?

Entune takes a hands-free
approach to accessing apps.

A few automotive examples exist today, such as GM MyLink, Ford Sync, and Toyota Entune. But app deployment to vehicles is still in its infancy. What conditions, then, must exist for apps to flourish in cars? A few stand out:

Cars need to be upgradeable to accept new applications — This is a no-brainer. However, recognizing that the lifespan of a car is 10+ years, it would seem that a thin client application strategy is appropriate.

Established rules and best practices to reduce driver distraction — These must be made available to, and understood by, the development community. Remember that people drive cars at high speeds and cannot fiddle with unintuitive, hard-to-manipulate controls. Apps that consumers can use while driving will become the most popular. Apps that can be used only when the car is stopped will hold little appeal.

A large, unfragmented platform to attract a development community — Developers are more willing to create apps for a platform when they don't have to create multiple variants. That's why Apple maintains a consistent development environment and Google/Android tries to prevent fragmentation. Problem is, fragmentation could occur almost overnight in the automotive industry — imagine 10 different automakers with 10 different brands, each wanting a branded experience. To combat this, a common set of technologies for connected automotive application development (think web technologies) is essential. Current efforts to bring applications into cars all rely on proprietary SDKs, ensuring fragmentation.

Other barriers undoubtedly exist, but these are the most obvious.

By the way, don’t ask me for my prediction of when the 10 millionth app will ship in auto. There’s lots of work to be done first.