It's a Bentley! A guided tour of the new QNX technology concept car

"Bend it, shape it, any way you want it"
— Headline from a QNX advertisement, circa 1987

I’m about to show you some pictures of a car. Not just any car, but a powerful, luxurious, and stunningly beautiful car. A car that has undergone a technological transformation.

If you’re like me, you'll be fascinated by the car’s features, some of which have never been seen in a vehicle — until now. But if you can, remember that it isn’t just about the cool features. It’s also about the platform that enabled them.

I’m speaking, of course, of the QNX CAR application platform.

We created the new QNX technology concept car — a modified Bentley Continental GT — to demonstrate that flexibility and customization form the very DNA of the QNX CAR platform. If you’ve seen the QNX reference vehicle, you already know that the platform provides an extremely rich environment for in-car infotainment, complete with HMI frameworks, smartphone integration, an HTML5 engine, a mobile device gateway, and a host of pre-integrated partner technologies — everything to kickstart our customers' projects. But in the automotive world, differentiation is everything. So it’s just as important that the platform enables customers to add their own branding, features, and sizzle. And to do it quickly.

Ease of branding and
personalization is just one

of the capabilities of the
QNX CAR platform.

Which is where the new concept car comes in. To create it, we used the same base QNX CAR platform that we offer our customers. But when you compare the Bentley to the Jeep, which uses a stock version of QNX CAR, the differences are dramatic: different features, different branding, and a different look-and-feel. In effect, the Jeep shows what QNX CAR can do out of the box, while the Bentley shows what QNX CAR lets you do once you start bending it to your imagination. One platform, many possibilities.

Which brings me to the slogan at the top of this post. It's amazing to think that a core value of QNX technology in the 1980s — giving customers the flexibility to achieve what they want to do — remains just as true today. Some values, it seems, are worth keeping.

And now, the car…
I know that you’re anxious to peek inside the car and see what we’ve done. But before we go any further, take a moment to savor the car’s beautifully sculpted exterior. This is one classy set of wheels. In fact, if you ask me, the wheels alone are worth the price of admission:



The awesome (and full HD) center stack
Okay, time to hop in — but get ready to prop up your jaw. Because the first thing you’ll notice is the jaw-droppingly beautiful center stack. This immense touchscreen features a gracefully curved surface, full HD graphics, and TI’s optical touch input technology, which allows a physical control knob to be mounted directly on the screen — a feature that’s cool and useful. (In the photo below, the clock display appears within the knob.)

The center stack supports a host of applications, including a 3D navigation system from Elektrobit that makes full use of the display. Just check out this bird’s-eye view of the Las Vegas Strip:



So how big is the display? Big enough to provide access to other functions, such as the car’s media player or virtual mechanic, and still have plenty room for navigation. Check it out:



The awesome (and very polite) voice rec system
Time to talk to the car. Just say “Hello Bentley,” and the car’s voice recognition system immediately comes to life and begins to interact with you — in a British accent, no less. You can now tell the media player what you’d like to hear and the navigation system where you’d like to go.

To provide natural language speech recognition, the system uses the cloud-based AT&T Watson speech engine, as well as an “intent framework” from QNX. It also uses keyword spotting technology from Sensory so you can start the system simply by talking to it.

The awesome (and nicely integrated) smartphone support
The Bentley also showcases how the QNX CAR platform enables automakers to offer advanced integration with popular smartphones. For instance, the car can communicate with a smartphone to stream music, or to provide notifications of incoming email, news feeds, and other real-time information — all displayed in a manner appropriate to the automotive context. Here's an example:



The awesome (and just plain fun) web app
I know, I know: the car looks cool, but you’re not at CES this week to see it first-hand. But how about the next best thing? Just connect to the web app and keep tabs on the Bentley in real time. (Note: The car will go online later this morning.) The app lets you view a variety of data that the car publishes to the cloud, such as what song the infotainment system is playing and whether someone has just opened a door. It also displays information that would be extremely helpful if this were your personal car, such as fluid levels and tire pressure. (This is a preliminary screen for the app, so I'm not sure if the tire pressures are realistic.)



UPDATE: The web app is now live, and the desktop version features a live camera feed of the Bentley and Jeep. Check it out!



The awesome (and very configurable) digital instrument cluster
The instrument cluster is implemented entirely in software, though you would hardly know it — the virtual gauges are impressively realistic. But more impressive still is the cluster’s ability to morph itself on the fly. Put the car in Drive, and the cluster will display a tach, gas gauge, temperature gauge, and turn-by-turn directions — the cluster pulls these directions from the center stack’s navigation system (cool, that). Put the car in Reverse, and the cluster will display a video feed from the car’s backup camera.



There are other options as well. For instance, the cluster can display information from the media player or display the current weather:



The awesome everything else
I’ve only scratched the surface of what the car can do. For instance, it also provides:

• Advanced multimedia system — Offers direct support for Pandora radio and the first embedded in-car implementation of the Shazam music discovery service.
   
• Video conferencing with realistic telepresence — Separate cameras for the driver and passenger provide independent video streams, while high-definition voice technology from QNX offers expanded bandwidth for greater realism, as well as stereo telepresence for making the remote caller sound as if they’re sitting right next to you.
   
• LTE connectivity — The car features an LTE radio modem, as well as a Wi-Fi hotspot for devices you bring into the car.

Super size those images
Want to see the center stack and instrument cluster in all their high-resolution glory? Just check out our QNX Flickr account.

That's all for now, but stay tuned: We’ll have plenty more news for you today and all through this week.

Am I crazy for talking to my car?

Earlier this afternoon, I participated in a connected car panel at SpeechTEK 2012, hosted by our friend Mazin Gilbert from AT&T. The other panelists included Greg Bielby of VoltDelta, Thomas Schalk of Agero, and Hakan Kostepen of Panasonic.

Even though Mazin did a fantastic job, not every panelist had a chance to answer every question. I was itching to answer some, so here are my responses to the questions that I didn't get to answer, or where I feel I could have provided a more complete response.

Have speech technologies matured to the point where they can be used robustly in the car? The general answer to this question from the panel was yes, but I think the real answer is a qualified yes. The technologies exist, but often aren't applied or may need auto-specific adaptations to handle in-cabin noise or other issues. Natural language recognition was an oft-stated driving technology, but a missing piece to the puzzle is hybrid recognition. I don't mean pushing recognition wholesale to the cloud, like Siri does. I mean a true split of the recognition effort, where each part does what it’s best at. Put the front half of acoustic processing in the vehicle to clean up the audio and convert the waveform to frequency-domain data, then send the data to the cloud-based server. The cloud server can then parse and interpret the data, and send back the result.

Hybrid speech rec solves three problems at once: better audio signals (the car can improve audio specific to the in-cabin environment), better cost (frequency data is far more compressed than raw audio, so you pay less for data transfer), and better responsiveness (hybrid rec gives the server time to start working on the response while it's coming in instead of waiting for the whole utterance to finish before starting).

Is driver distraction a major business driver, or is it the "Siri effect"? Currently, the car industry seems to use driver distraction as a reason to push a lot of features into speech. Many of those uses are gimmicky. Personally, I don't care if I can set my climate control system with voice — why would I when I can simply turn a dial? I once had someone ask me about the feasibility of adding voice recognition commands for rolling down the windows. I asked him, "Yes, but wouldn't people just push the window button?"

We shouldn’t implement speech commands just because we can. They may have contributed to excitement in the early adopter crowd, but we're beyond that now. Mind you, there are some seriously useful ways to use voice. For instance, any time you need to pick from a huge number of choices, voice recognition is the natural way to go. Calling contacts ("Call Sarah Potter"), entering destinations ("Go to 3121 South Park Street"), or picking music ("Play Audioslave") are all much easier than using an HMI to enter the same information, and safer to boot. It just has to work consistently and accurately.

Will car makers see more speech moving to the cloud, or will it be a hybrid of cloud and embedded? I disagree with the majority of the panel on this one, and, I think, the majority of people in the industry. Most auto people believe a hybrid between embedded and cloud allows the best of both worlds — good recognition and updatability when connected, and consistent reliability when not. My colleague Andrew Poliak also champions this view with a memorable catch phrase: Zombie Apocalypse. That is, you still want the system to work, albeit partially, when the infrastructure isn't available.

But if you ask me, everyone is missing the point — theirs is a technology-centric point of view. Everyday customer acceptance of a particular technology is notoriously harsh: if it doesn't work well, it gets rejected out of hand. Good cloud solutions beat an embedded solution hands-down; they just need some improvements (see my hybrid bullet above). Once a customer experiences a good solution, they will become frustrated with one that performs poorly. In my opinion, it's better not to offer the service at all, than to try a graceful degradation of capability, because most customers won't understand or care. Spend the effort instead on making sure you always have an acceptable cloud connection — either through multiple redundant mechanisms or a car-based powerful antenna — and you'll be better off. Even when the car knows some data that the cloud doesn't (like a mobile's contact list or music selection), there's no need to handle that on the embedded side. The cloud recognition server is powerful enough to not require the data set a priori. And I think we can predict an eventual migration of phone data to cloud-based data (or cloud-synchronized data) that makes the car's knowledge either easily transferrable or less relevant.

Who makes money, and how, from voice-enabled agents or voice services? This was one of the best questions of the panel, because nobody really knows the exact model, but everybody agreed that customer tolerance is very low. The most likely candidate is ad-based revenue. This doesn't mean reading ads aloud to the driver, but rather, positively influencing search results for either active or temporary situation-based points of interest (POIs). Depending on how valuable the service is to the driver, there will still be an option for service-based payments and high-value apps.

Standards and building mobile apps — will it come? You need standards if you want to build an app platform that will promote application creation and adoption. That's what we're doing with the QNX CAR 2 application platform — creating a way for someone other than the car companies to join the ecosystem and to deploy their apps to the car in a controlled way. But don't forget, you need a standard way to deploy apps for the cloud half of the recognition, too.

To close, let me share two photos. One was taken outside the Marriott Marquis, the hotel hosting the conference just off of Times Square in NYC. The other is from our PR agency, Breakaway Communications. What do they have in common? Wooden water towers. Sorry, I couldn't help myself; I just love those things. They just look so quaint in a city full of glass and brick.

Techcrunch interview: Talking with (and about) AT&T's WATSON personal assistant

Yesterday, I shared an Engadget video highlighting how the QNX concept car takes advantage of WATSON, AT&T's cloud-based natural-language speech engine. Here's another video, from Jordan Crook of TechCrunch, in which AT&T's Mazin Gilbert describes what WATSON does and QNX's Derek Kuhn demonstrates how WATSON works.

The entire video is interesting, but jump to the 3:30 mark if you want to see the car. The video provides a very nice view of the QNX-equipped dash, btw.



The Engadget video and the TechCrunch video were both shot yesterday at AT&T's "Living the Networked Life" event in New York.

Find me a Starbucks! QNX concept car showcases power of WATSON speech engine

Yes, you can talk to the QNX concept car and tell it what to do.

Recently, our friends at AT&T invited us to bring the concept car to their "Living the Networked Life" event in New York. We said yes, of course! After all, what could be cooler than riding the streets of Gotham City in a digitally pimped-out Porsche 911?

Kidding aside, the event provided an excellent opportunity to demonstrate how the car takes advantage of WATSON, AT&T's natural-language speech engine. To get an idea of what WATSON can do, check out this video from Terrence O'Brien of Engadget:



For the full Engadget article, click here. And stay tuned for more updates from the Living the Networked Life event.
 

Making of the QNX concept car... honest

We created this video as a backdrop for CES 2012 where we unveiled the latest QNX concept car (a Porsche 911). Good thing, too, as people clearly stated that they would not have believed we did this cool retrofit ourselves without proof.


 

Mark your calendars: FOTA webinar this week

Paul Leroux

A heads-up that, in two days, QNX Software Systems and Red Bend will deliver a webinar on keeping connected cars up to date with Firmware Over-the-Air (FOTA) technology.

Here's a summary of the webinar:

The connected car is revolutionizing the automotive industry. It brings together wireless connectivity, IP communications infrastructure, middleware and applications to transform the driving experience. Yet as the amount and the complexity of software inside cars increases, so too does the need to update the software in order to provide new functionality and perform software maintenance. To help streamline these activities, OEMs can use proven firmware over-the-air (FOTA) updating technology to deliver software improvements efficiently and reliably.

Please join QNX Software Systems and Red Bend Software as they present how QNX CAR 2 application platform supports the latest advancements in performing over-the-air software management and application updates in connected cars.

And here are the coordinates:

Wednesday, February 8, 2012 2:00 p.m. EST Details and registration

The folks delivering the webinar are Yoram Berholtz from Red Bend Software, an industry leader in FOTA, and my inimitable colleague Andy Gryc. If you can't catch the webinar — or even if you can catch the webinar — I invite you to check out Andy's article on the Red Bend website, Fill the Tank, Check the Oil, and Update the Software.
 

HTML5 is a great option for the risk-adverse

The conservative nature of automakers makes them ideal candidates for the brand-spanking new HTML5. Say what?

While this seems paradoxical, it is an interesting take-away from the discussion between TCS' director of engineering Michael Camp and our very own Andy Gryc. If you've got three minutes to spare, watch the latest video in our HTML5 series for the whole story. (As usual, don't miss the ending. ;-)

This interview of Michael Camp is the fourth in a series from QNX on HTML5.

Is HTML5 a good gamble?

As the consumer and automotive worlds continue to collide, HTML5 looks like a good bet. And not a long shot either. In fact, the odds are all automakers will eventually use it. But since the standard won’t be mature for some time yet, should you take a chance on it now? 

To answer this, Andy Gryc talks to Matthew Staikos of RIM. Matthew is the manager of the browser and web platform group at RIM, and has over 10 years of software development experience with a strong focus on WebKit for mobile and embedded systems. Matthew co-founded Torch Mobile, which was acquired by RIM for their browser technology.

Andy’s conversation with Matthew is the subject of the following video, the second in an educational series designed to get an industry-wide perspective on HTML5. 

This interview of Matthew Staikos is the second in a series from QNX on HTML5.

What’s HTML5 got to do with automotive?

There’s been a lot of noise lately about HTML5. A September 2011 report by binvisions shows that search engines and social media web sites are leading the way toward adoption: Google, Facebook, YouTube, Wikipedia, Twitter, and plenty more have already transitioned to HTML5. Some are taking it even further: Facebook has an HTML5 Resource Center for developers and the Financial Times has a mobile HTML5 version of their website.

It won’t be long before HTML5 is ubiquitous. We think automakers should (and will) use it. 

To elucidate the technology and its relevance, we’ve created a series of educational videos on the topic. Here is the first in that series. Interviews with partners, customers, and industry gurus will soon follow. 

This simple overview is the first in a series from QNX on HTML5. (Personally I like the ending the best.)

Seamless connectivity is for more than online junkies

As much as I’m not always enamored with sitting behind a computer all day, I find being off the grid annoying. Remember this email joke?

You know you’re an online junkie when you:
• wake up at 3:00 am to go to the bathroom and stop to check your email on the way back to bed
• rarely communicate with your mother because she doesn’t have email
• check your inbox. It says ‘no new messages,’ so you check it again 

Even though this joke circulated several years ago, it still strikes a chord. The big difference now is that there’s no longer a subculture of ‘online junkies.’ From the time we wake up in the morning to the time we go to bed, we all want to be connected — and that includes when we get behind the wheel. So to this joke I would add:

• resent driving because it means going off the grid

At QNX, we’re working toward a seamless experience where people can enjoy the same connectivity whether they’re texting their spouse from the mall or checking traffic reports while driving down the highway. See what I mean:



For more information about the technology described in this video, visit the QNX website.
 

What’s next for the connected car?

It’s been almost three years since QNX Software Systems launched its connected car concept, and I thought it would be an interesting exercise to look at what has been accomplished in the automotive industry around the connected car and how some of the concepts are evolving. When the QNX CAR Application Platform was introduced, we provided a simple way to look at a connected car, using four “dimensions” of connectivity:

• Connected to portable consumer devices for brought-in media and handsfree communications
• Connected to the cloud for obvious reasons
• Connected within the car for sharing information and media between front and rear seats, between the center stack and the cluster, and for other similar functions
• Connected around the car for providing feedback to the driver about the environment surrounding the car, be it pedestrians, other cars, or vehicle to infrastructure communications

We’ve seen significant advances and evolutionary thinking on all fronts. Although QNX is not (and cannot be) at the forefront of all of these, our primary emphasis has been on cloud and consumer device connectivity. Nonetheless, it is interesting to look at each area.

Connected to consumer devices, connected to the cloud
Why lump these two together? There is not exactly a clear line between the two since consumer devices are often just extensions of the cloud. If my car connects to a smartphone which, in turn, draws information from the cloud, is there much point in creating a distinction between consumer device and cloud connections? Although it made sense to differentiate between cloud and consumer device connections when phones provided only handsfree calling and simple music playback, today the situation is quite different.

Device integration into the car has been a beehive of activity over the last few years. Smartphones, superphones, and tablets are providing entertainment, social networking, news, and access to a myriad of other content and applications to consumers anywhere, anytime. Automakers want to take advantage of many of these capabilities in a responsible, non-distracting way.

The primary issue here is how to marry the fast-paced consumer electronics world to the lifecycle of the car. At present, there are solutions at the opposite end of the spectrum: standardized Bluetooth interfaces that allow the car to control the smartphone; and screen replication technologies (iPod Out, VNC/Terminal Mode/Mirror Link) where the smartphone takes control and uses the car as a dumb display.

Neither of these scenarios takes full advantage of the combined processing power and resources of the car and the brought-in device. This, to me, is the next phase of car and cloud connectivity. How can the power of the cloud, brought-in devices, and the in-car systems be combined into a cooperative, distributed system that provides a better driver and passenger experience? (If the notion of a distributed system seems a bit of a stretch, consider the comments made by Audi Chairman Rupert Stadler at CES 2011.)

When looking for technologies that can bring the cloud, devices, and car together, you do not need to look any further than the web itself. The software technologies (HTML5, Javascript, AJAX, peer-to-peer protocols, tooling, etc.) that drive the web provide a common ground for building the future in car experience. These technologies are open, low cost, widely known and widely accessible. What are we waiting for?

Connected within the car
The integrated cockpit has emerged as a prevalent automotive design concept. It is now commonplace in higher-end vehicles to see seamless integration between center stack functions and the instrument cluster and driver information displays. For example, turn-by-turn directions, radio tuning, and song now playing are all available to the driver on the cluster, reducing the need to constantly glance over to the main display. One such example is the Audi cluster:

Connected around the car

Three years ago, systems in this category were already emerging, so there really wasn’t much of a crystal ball required here. Adaptive cruise control has become one of the most common features that illustrate how a car can connect to its surroundings. Adaptive cruise control detects the car’s surroundings (cars in front of you) and adjusts your speed accordingly. Other examples include pedestrian detection (offered in Volvo S60 and other models)automatic parking, lane departure warning, and blind spot detection/warning systems.

These Advanced Driver Assist Systems (ADAS) will become more common as cost reductions take place and the technology is provided in lower-end vehicles.

In contrast, vehicle-to-infrastructure communication that requires industry-wide collaboration is proceeding at a pace that you’d expect from an internationally standardized solution.