It's a Bentley! A guided tour of the new QNX technology concept car

"Bend it, shape it, any way you want it"
— Headline from a QNX advertisement, circa 1987

I’m about to show you some pictures of a car. Not just any car, but a powerful, luxurious, and stunningly beautiful car. A car that has undergone a technological transformation.

If you’re like me, you'll be fascinated by the car’s features, some of which have never been seen in a vehicle — until now. But if you can, remember that it isn’t just about the cool features. It’s also about the platform that enabled them.

I’m speaking, of course, of the QNX CAR application platform.

We created the new QNX technology concept car — a modified Bentley Continental GT — to demonstrate that flexibility and customization form the very DNA of the QNX CAR platform. If you’ve seen the QNX reference vehicle, you already know that the platform provides an extremely rich environment for in-car infotainment, complete with HMI frameworks, smartphone integration, an HTML5 engine, a mobile device gateway, and a host of pre-integrated partner technologies — everything to kickstart our customers' projects. But in the automotive world, differentiation is everything. So it’s just as important that the platform enables customers to add their own branding, features, and sizzle. And to do it quickly.

Ease of branding and
personalization is just one

of the capabilities of the
QNX CAR platform.

Which is where the new concept car comes in. To create it, we used the same base QNX CAR platform that we offer our customers. But when you compare the Bentley to the Jeep, which uses a stock version of QNX CAR, the differences are dramatic: different features, different branding, and a different look-and-feel. In effect, the Jeep shows what QNX CAR can do out of the box, while the Bentley shows what QNX CAR lets you do once you start bending it to your imagination. One platform, many possibilities.

Which brings me to the slogan at the top of this post. It's amazing to think that a core value of QNX technology in the 1980s — giving customers the flexibility to achieve what they want to do — remains just as true today. Some values, it seems, are worth keeping.

And now, the car…
I know that you’re anxious to peek inside the car and see what we’ve done. But before we go any further, take a moment to savor the car’s beautifully sculpted exterior. This is one classy set of wheels. In fact, if you ask me, the wheels alone are worth the price of admission:



The awesome (and full HD) center stack
Okay, time to hop in — but get ready to prop up your jaw. Because the first thing you’ll notice is the jaw-droppingly beautiful center stack. This immense touchscreen features a gracefully curved surface, full HD graphics, and TI’s optical touch input technology, which allows a physical control knob to be mounted directly on the screen — a feature that’s cool and useful. (In the photo below, the clock display appears within the knob.)

The center stack supports a host of applications, including a 3D navigation system from Elektrobit that makes full use of the display. Just check out this bird’s-eye view of the Las Vegas Strip:



So how big is the display? Big enough to provide access to other functions, such as the car’s media player or virtual mechanic, and still have plenty room for navigation. Check it out:



The awesome (and very polite) voice rec system
Time to talk to the car. Just say “Hello Bentley,” and the car’s voice recognition system immediately comes to life and begins to interact with you — in a British accent, no less. You can now tell the media player what you’d like to hear and the navigation system where you’d like to go.

To provide natural language speech recognition, the system uses the cloud-based AT&T Watson speech engine, as well as an “intent framework” from QNX. It also uses keyword spotting technology from Sensory so you can start the system simply by talking to it.

The awesome (and nicely integrated) smartphone support
The Bentley also showcases how the QNX CAR platform enables automakers to offer advanced integration with popular smartphones. For instance, the car can communicate with a smartphone to stream music, or to provide notifications of incoming email, news feeds, and other real-time information — all displayed in a manner appropriate to the automotive context. Here's an example:



The awesome (and just plain fun) web app
I know, I know: the car looks cool, but you’re not at CES this week to see it first-hand. But how about the next best thing? Just connect to the web app and keep tabs on the Bentley in real time. (Note: The car will go online later this morning.) The app lets you view a variety of data that the car publishes to the cloud, such as what song the infotainment system is playing and whether someone has just opened a door. It also displays information that would be extremely helpful if this were your personal car, such as fluid levels and tire pressure. (This is a preliminary screen for the app, so I'm not sure if the tire pressures are realistic.)



UPDATE: The web app is now live, and the desktop version features a live camera feed of the Bentley and Jeep. Check it out!



The awesome (and very configurable) digital instrument cluster
The instrument cluster is implemented entirely in software, though you would hardly know it — the virtual gauges are impressively realistic. But more impressive still is the cluster’s ability to morph itself on the fly. Put the car in Drive, and the cluster will display a tach, gas gauge, temperature gauge, and turn-by-turn directions — the cluster pulls these directions from the center stack’s navigation system (cool, that). Put the car in Reverse, and the cluster will display a video feed from the car’s backup camera.



There are other options as well. For instance, the cluster can display information from the media player or display the current weather:



The awesome everything else
I’ve only scratched the surface of what the car can do. For instance, it also provides:

• Advanced multimedia system — Offers direct support for Pandora radio and the first embedded in-car implementation of the Shazam music discovery service.
   
• Video conferencing with realistic telepresence — Separate cameras for the driver and passenger provide independent video streams, while high-definition voice technology from QNX offers expanded bandwidth for greater realism, as well as stereo telepresence for making the remote caller sound as if they’re sitting right next to you.
   
• LTE connectivity — The car features an LTE radio modem, as well as a Wi-Fi hotspot for devices you bring into the car.

Super size those images
Want to see the center stack and instrument cluster in all their high-resolution glory? Just check out our QNX Flickr account.

That's all for now, but stay tuned: We’ll have plenty more news for you today and all through this week.

Open standards, open source, and why the difference matters

As Andy Gryc reported in a previous post, Paul Hansen of the Hansen Report asked six automakers whether they plan to ship products based on the GENIVI open source platform. Not one of them said yes.

This underwhelming response to open source may seem surprising, especially to people outside of the auto industry. It seems even more surprising when you consider the many companies that belong to the GENIVI Alliance — a veritable who’s who of high-tech and automotive companies, from ARM to IBM to Volvo. Why the disconnect?

A couple of reasons come to mind. First, the automotive market is exceedingly competitive. Asking automakers to collaborate on a common OS platform — the GENIVI approach — is arguably a non-starter. Also, many automakers seem to grasp that open source OSs don't necessarily address the issues that matter to them most.

Allow me to explain. Automotive companies entertain the option of using open source for several reasons. They want to avoid vendor lock-in. They want to leverage a large developer community. They want to access a rich toolset. And, in many cases, they hope to avoid the costs of runtime licensing.

Yes, open source can help address these requirements. But more often than not, open standards offer a better route to achieving what automakers really need.

Vendor neutral, OS neutral, hardware neutral
Take the goal of avoiding vendor lock-in. An open standard is, by definition, vendor neutral. It is typically the product of a collaborative and transparent process free of domination by a single company or interest group. Likewise, it isn’t controlled or maintained by a single, self-interested entity. HTML 5, for instance, isn’t owned by any one company, but is a standard embraced by Apple, Google, Microsoft, Mozilla, QNX, and others.

HTML5 isn't just vendor neutral; it's also OS and hardware neutral. By using it as an HMI and application environment, automakers gain the freedom to choose the best OS platform for the job at hand, and the option to migrate across platforms, if required. In other words, HTML5 enables automakers to use the platform that can offer fastest boot speed, the highest reliability, the best mobile device integration, or the best performance on automotive silicon — things that can reduce costs and improve the user experience. (To put this another way, the underlying OS platform is anything but a commodity — a fact demonstrated every day in the mobile device world.)

An open source platform may or may not share these characteristics. Even though developers can access the source code, a single entity may still control the technology’s roadmap and licensing terms. In effect, the platform can constitute a single point of failure for the automaker — exactly what automakers try to avoid. Compare this to an open standard, which is defined collaboratively and then supported over a long period of time. POSIX, with its 20+ year history, comes to mind.

Also, open standards like HTML5 are unencumbered by the protective licensing terms often associated with open source OSs — terms that can lead to greater system costs and complexity. For instance, the GNU Public License (GPL) that governs use of the Linux OS ensures that any modifications to the original program are released as open source. That's a problem for any OEM that doesn’t want to “open source” its technology; for instance, vehicle bus information. It is also incompatible with the certifications and licenses of consumer device manufacturers whose licensing terms are designed to prevent integration of their code with GPL code bases; iPod support and integration is a good example. Such technologies must, as a result, be separated into another virtualized OS or onto external hardware modules. The result is a more expensive and more complex system — another thing that automakers try to avoid.

Delivering the goods
Of course, all this hinges on whether a standard like HTML5 can deliver the goods. And from my perspective, it does. For instance, it can provide all the capabilities of a traditional HMI toolkit, including a rendering engine, content authoring and packaging tools, and sophisticated graphic transitions. But unlike proprietary solutions, it can also help automakers:

• tap into a vast pool of apps and developers
• integrate with mobile devices
• build user interfaces that incorporate virtually any delivery model
• customize the UX and simplify access to mobile apps
• customize apps and the UX for context: park, creep, drive, etc.

In addition, HTML5 can, with the right platform, work in concert with other HMI technologies (Adobe AIR, OpenGL ES, Qt, etc.) and blend seamlessly with those technologies on same display. As a result, system designers can choose the most appropriate technology for each application.

Incorporating open source
So is open source a total non-starter in automotive? Absolutely not.

In fact, many standards incorporate open source. Let us once again consider HTML 5. While it is built on an open standard, many HTML5 implementations are developed using open source solutions. For instance, many of the current, industry-leading HTML5 solutions are built on Webkit, an open source solution governed by the Lesser GNU Public License or LGPL.

The point is, the most successful solutions will combine the best that open standards, open source, and proprietary platforms have to offer. But if you were mandate an “open” solution, an open standard would be the best to rally behind.

---

If you're interested in this topic, we recommend you listen to the webinar that Andrew gave last week, "In-vehicle product differentiation: open standards vs open source." — Ed.

Top 8 questions for squeezing high-end tech into low-end infotainment

A couple of weeks ago, I hosted a webinar that addressed the question, “Is it possible to build an infotainment system that meets today's customer demands with yesterday's price tag?” The webinar explored several ways to reduce RAM and ROM requirements, eliminate hardware, and share hardware, all with the goal of cutting BOM costs.

As always, the audience asked lots of great questions, several of which I have answered here. Of course, these provide only a hint of the ground covered in the webinar, so I invite you to download the archived version to get the full details.

Built-in phone module versus brought-in smart phone: what is your take on this, and is a hybrid approach feasible?
The approach will vary from automaker to automaker. I think that embedded phones will be required for certain cars, especially if they use systems that rely on cloud-based services. This approach adds to the BOM cost, of course, but it may reduce overall cost, depending on what features can be off-loaded to the cloud.

Some brands will encourage brought-in devices as the lowest-cost alternative. The consumer will then have to deal with the setup and maintenance issues required to pair or charge the phone. I don’t see a clear-cut analysis that says one method will be better than the other — it really depends on what you want to accomplish.

Any thoughts on using MirrorLink to clone a virtual display to a remote physical LCD?
If you’re talking about a remote (as in cloud-based) device, I would say that HTML5 is a much more natural choice for a server-based application. If it’s a brought-in device, then MirrorLink or HTML5 could be appropriate.

If GPS is moved to a brought-in phone, how will a stolen vehicle be located?
It won’t be, unless the phone was left in the vehicle. This is one of the trade-offs you make when trying to reduce cost.

Of the cost-saving techniques you discussed, which are most likely to be used?
Already, some system designers are removing wake-up micros and DSPs. I’m not aware of any systems where the LCD has been removed, but this approach would probably offer the largest cost savings, making it a likely choice for entry-level systems and cost-sensitive markets.

Security and reliability are the main concerns of a head unit. Squeezing high-end technologies into low-end systems won’t relax those expectations. For instance, smart phone integration will be an add-on instead of replacing functionality of the head unit. Thoughts?
The trade-offs will need to be communicated to the customer. You can never build a head-unit augmented with a smartphone that works as reliably as the head unit operating by itself. As an OEM or Tier 1, you just don’t have enough control over the brought-in devices.

As an industry, we need to educate consumers. If they start relying on the phone in the car to provide certain features, then they will have to expect an inevitable degradation in overall system quality. It comes back to that famous adage: “cost, quality, or time — pick two”.

MirrorLink has a defined communication interface to the head unit. You also mentioned HTML5 as an option. Is there a defined standard yet for transmitting the HTML5 up to the head unit?
The interface between web server (i.e. phone) and web client (i.e. head unit) is already well established and tested. For some specialized features (for instance, allowing HTML5 code to access vehicle services) some standardization may be required. This will hopefully be a topic of discussion in November, at the automotive HTML5 workshop hosted by the W3C in Rome. Some Connected Car Consortium members have also discussed the possibility that, in the future, MirrorLink could add a transport mechanism based on HTML5.

You discussed peripheral sharing, using QNX transparent distributed processing. Does QNX TDP require secure authentication between distributed boxes?
No, it does not. It relies on standard POSIX user group permissions to provide access rights to devices.

Can you discuss any trends you see regarding Ethernet or TCP/IP in the vehicle?
Ethernet is definitely becoming more interesting in the vehicle, due to the introduction of Ethernet AVB. It makes a very natural replacement for audio-video transmission over MOST, and the additions to AVB that fulfil strict timing requirements can replace CAN or MOST for non-media vehicle messages. Ethernet also has obvious advantages when you need to access Wi-Fi networks, cloud services, and mobile devices.

HTML5 SDK for the QNX CAR 2 platform — the back story

Kerry Johnson

Today, at SAE Convergence, QNX Software Systems announced the new HTML5 SDK for the QNX CAR 2 application platform. I’d like to provide some insight into this announcement, describe what you can expect to find in the SDK, and explain how it builds on the HTML5 capabilities already available in the QNX CAR 2 application platform.

Enabling apps for the car
Almost every consumer who owns a smart phone or tablet is familiar with the app experience: you go to an online marketplace, find apps of interest, and download them onto your device. With the HTML5 SDK, the automotive team at QNX is creating an analogous experience for the car.

Just as Apple, Android, and RIM provide SDKs to help vendors develop apps for their mobile platforms, QNX has created an SDK to help vendors to build apps for the QNX CAR 2 application platform. The closest analogies you will find to our HTML5 SDK are Apache Cordova and PhoneGap, both of which provide tools for creating mobile apps based on HTML5, CSS, JavaScript, and other web technologies.

App developers want to see the largest possible market for their apps. To that end, QNX also announced today that it will participate in the W3C’s Web and Automotive Workshop. The workshop aims to achieve industry alignment on how HTML5 is used in the car and to find common interfaces to reduce platform fragmentation from one automaker to the next. Obviously, app developers would like to see a common auto platform, while automakers want to maintain their differentiation. Thus, we believe the common ground achieved through W3C standardization will be important.

It bears mentioning that, unlike phone and tablet apps, car apps must offer a user experience that takes driver safety into consideration. This is a key issue, but beyond the scope of this post, so I won’t dwell on it here.

So what’s in the SDK, anyway?
As in any SDK, app developers will find tools to build and debug applications, and APIs that provide access the underlying platform. Specifically, the SDK will include:

• APIs to access vehicle resources, such as climate control, radio, navigation, and media playback
• APIs to manage the application life cycle: start, stop, show, hide, etc.
• APIs to discover and launch other applications
• A packaging tool to combine application code (HTML, CSS, JavaScript) and UI resources (icons, images, etc.) with QNX CAR APIs to create an installable application – a .bar file
• A emulator for the QNX CAR 2 platform to test HTML5 applications
• Oh yeah, and documentation and examples

The development and deployment flow looks something like this:




Emulator and debugging environment
The QNX automotive team has extended the Ripple emulator environment to work with the QNX CAR 2 application platform. Ripple is an emulation environment originally designed for BlackBerry smart phones that RIM has open sourced on github.

Using this extended emulator, application developers can test their applications with the correct screen resolution and layout, and watch how their application interacts with the QNX CAR 2 platform APIs. For example, consider an application that controls audio in a car: balance, fade, bass, treble, volume, and so on. The screenshot below shows the QNX CAR 2 screen for controlling these settings in the Ripple emulator.


Using the Ripple emulator to test an audio application. Click to magnify.

In this example, you can use the onscreen controls to adjust volume, bass, treble, fade, and balance; you can also observe the changes to the underlying data values in the right-hand panel. And you can work the other way: by changing the controls on the right, you can observe changes to the on-screen display. The Ripple interface supports many other QNX CAR 2 features; for examples, see the QNX Flickr page.

You can also use the emulator in conjunction with the Web Inspector debugger to do full source-code debugging of your Javascript code.

Creating native services
Anyone who has developed software for the QNX Neutrino OS knows that we offer the QNX Momentics Tool Suite for creating and testing C and C++ applications. With the QNX CAR 2 application platform, this is still the case. Native-level services are built with the QNX Momentics suite, and HTML5 applications are built with our new HTML5 SDK. We've decided to offer the suite and the SDK as separate packages so that app developers who need to work only in the HTML5 domain needn't worry about the QNX Momentics Tool Suite and vice versa. Together, these toolkits allow you to create HTML5 user interface components with underlying native services, where required.

Jivin’ up the Jeep — then and now

Do Jeeps have a unique power to bring out the inner hacker in their owners? Based on the sheer number of Jeep kits on the market, I'd say yes.

Maybe it has something to do with the rough-and-ready, take-on-all-comers personality of the Jeep brand. Or maybe it has to do with the inherent flexibility of the Jeep design. Or maybe it's simply because the brand attracts self-reliant do-it-yourselfers. Whatever the explanation, the history of Jeep modding is almost as old as the Jeep itself.

Jivin' then...
For instance, here are some examples of "jivin' up the Jeep" from a 1947 issue of Mechanix Illustrated magazine. (I found these on blog.modernmechanix.com — you have got to check this site out.)






And jivin' now...
With a history like this, is it any wonder the QNX concept team also chose to mod a Jeep, albeit with 21st-century tech? For instance, they added their own digital instrument cluster:



and some apps:



not to mention a virtual mechanic:



And is it any wonder they had so much fun doing this?



Hey, do you plan on attending SAE Convergence in October? If so, come by the QNX booth (815) for an even closer look at how the QNX concept team jived up this Wrangler with the connectivity and personalization features of the QNX CAR application platform.

Highlights (er, mods) of the Wrangler include:

• Customizable HMI for reskinning and personalization
  
• Ability to download apps
  
• Multimedia: streaming radio, mobile connectivity, album art, etc
  
• One-touch Bluetooth pairing with NFC
  
• HD hands-free communication with conversational voice recognition
  
• Reconfigurable digital instrument cluster
  
• Tablet-based rear-seat entertainment
  
• HTML5 framework for leveraging mobile ecosystem
  

 

So where is QNX going in automotive?

Want a short and sweet intro on what QNX is doing in the automotive industry? Then be sure to check out "A Look At The Near Future Of In-Car Technology," published this week in The Washington Post and in Motor Authority. (Same article in both cases, though Motor Authority has more pictures :-)

The article is based on an interview with my friend and colleague Andy Gryc. It covers the bases, from how QNX technology helps automakers project their brand identities to how it will enable a new generation of apps in the car.

Enough of my blather. Check out the article and let me know what you think.

QNX CAR 2 — the extended version

The world of video is a ruthless one; just as we posted the QNX CAR demo it was out of date.

But, hold on a minute. As I write this I realize it’s not the video world at all; it’s the software world that creates new technology at breakneck speed. And QNX certainly does its part.

The QNX CAR 2 application platform has come a long way in a matter of months. We needed to update the original video to keep pace with the technology but also to address customer demand for more detailed information.

So this video is a step-by-step demo – definitely not for the tire kicker. But if you really want details on what automakers and tier ones can achieve with QNX CAR technology, hit play.

8 steps to building a lean and mean HTML5 application

Guest post from Marc Lapierre, HMI developer for the QNX CAR 2 application platform

Have you seen photos of the QNX reference vehicle? If so, you've already caught a glimpse of the rich user experience that HTML5 can bring to car infotainment systems. The vehicle's head unit, in particular, makes extensive use of HTML5.

The members of the QNX CAR 2 team have considerable experience with HTML5, and we follow a number of “best practices” to achieve optimal performance. If you use HTML5, here are 8 techniques proven to help applications perform as smoothly and responsively as possible:

1. Use 3D, rather than 2D, transformations — For example, instead of translateX(x), use translate3d(x,y,z). This will hardware-accelerate the translation. Similar methods exist for most other transformations. Also, avoid animating with JavaScript libraries!

2. Use opacity, rounded corners, and gradients sparingly — If you use these elements sparingly and on mostly static objects, you should achieve decent performance. But when you mix them with animations, buttons, or anything else that gets redrawn often, performance will suffer. Consider using images for framing rather than building components with many specific CSS attributes.

3. When modifying elements, remove them from the DOM — This technique is especially helpful when updating several DOM fields at once. For example, if you are scrolling through a list of 100 contacts and want to refresh them, updating them one by one will cause the list to redraw 100 times. But if you remove the entire list, update it in memory, and then re-add it, you will incur only 2 redraws.

4. Avoid canvas and SVG — Hardware acceleration for canvas isn’t always available in WebKit or other browsers, and might incur performance hits in some cases. Likewise, SVG isn’t always accelerated on mobile and embedded platforms.

5. Hide elements you don’t need — Adding display:none to elements that don’t need to be displayed will prevent them from being rendered.

6. Don’t link across pages — When developing websites, it is common to link across pages. But in mobile applications, this approach detracts from the user experience — when using an app, it can be jarring to see the white screen that often appears when moving from one page to another. For a better UX, use AJAX requests to pull in data dynamically, and update your interface accordingly when the result is received.

7. Avoid libraries intended for the desktop — Some JavaScript libraries are designed for use on a desktop browser with a powerful CPU. Try to limit the number of third-party JavaScript libraries included in your application or seek out versions optimized for mobile use.

8. Use image sprites for pre-loading active element states — For example, using sprites for buttons with a “pressed” state allows you to have the alternative state pre-cached and ready to display, rather than having to load or draw assets on demand.

What about you? Do you have any resource-saving or performance-optimizing techniques that you’d like to share?

---

Marc Lapierre is an HMI developer on the QNX CAR 2 application platform team, where he focuses on development of user applications using HTML5, JavaScript and CSS3, and on improving coding efficiency and standards in this environment. Before joining QNX Software Systems, Marc worked at RIM, developing social networking and multimedia applications for smartphones and the BlackBerry PlayBook tablet.

How will HTML5 play out in the car? A video series roundup

Last Fall, my colleagues Andy Gryc and Nancy Young launched a video series on HTML5 in the car. To their credit, they took the surround-sound approach and asked a variety of people from the automotive ecosystem to weigh in on the topic. So far, they've interviewed executives from Audi, OnStar, and TCS, as well as automotive and web-technology experts from QNX Software Systems and RIM.

Naturally, everyone they spoke to has a different take on the topic. So I thought I'd bring all the videos together in one place to give you an overall view of how the industry sees HTML5 playing out in the car. Grab some popcorn, dim the lights, and check them out:

Kickoff video

Andy Gryc kicks off the series with his take on why he believes HTML5 is poised to become the foundation for next-gen automotive apps and HMIs:

Interview with Steve Schwinke of OnStar

Andy catches up with Steve Schwinke, director of advanced technology for OnStar, who believes that HTML5 can change the auto industry for the better. (Did you know? The OnStar RemoteLink App for BlackBerry was coded in HTML5):

Interview with Michael Camp of TCS

Andy Gryc sits down with Michael Camp, director of engineering for in-car telematics at TeleCommunication Systems (TCS), to get a software supplier's perspective on HTML5. Michael is a very articulate guy, and worth a listen:

The myth buster interview

Andy meets up with Kerry Johnson of QNX to poke holes into the most common myths about HTML5. They discuss how HTML5 apps can deliver snappy performance, run without a Web browser, and even work without an Internet connection:

Interview with Matthew Staikos of RIM

Andy talks with Matthew Staikos, web-technology manager at RIM, about the impact of HTML5 on hardware options, memory usage, and app stores:

Interview with Sheridan Ethier

Andy meets up with Sheridan Ethier of QNX to get a developer's perspective on HTML5:

Interview with Mathias Haliger of Audi

And last but not least, here's the most recent installment in the HTML5 video series, which we featured a couple of week ago. Mathias Haliger, head of MMI system architecture at Audi AG, speaks with QNX marketing VP Derek Kuhn about the importance of HTML5 to his company and why he considers it a game changer:


 

So why all the fuss over HTML5?

HTML5 is not your father’s HTML – or, for that matter, your younger self's HTML. Nor is it simply a standard for delivering web content. Unlike its predecessors, HTML5 allows you to create an immense variety of applications and human machine interfaces (HMIs). It even lets you build applications that are neither connected to the Internet nor based on a traditional browser.

But don’t take my word for it. Check out the recent whitepaper,“Why HTML5 Is Becoming the HMI Technology of Choice,” from my colleagues Andy Gryc and Marc Lapierre.

To explain why HMI developers are turning to HTML5, Andy and Marc explore several themes. For instance:

• HTML5 allows developers to construct applications either inside or outside of a browser, with capabilities such as databases, threading, and input from device hardware.
 
• Using CSS3 animations, the <canvas> element, WebGL, and SVG graphics, HTML5 provides control over HMI rendering that is precise enough for games and flexible enough for applications.
 
• The influx of software developers adopting HTML5 to build cross-platform applications is re-orienting the HMI development community, which is increasingly following traditional design patterns in its applications, separating the model (HTML/DOM), view (CSS), and controller (JavaScript) in a more maintainable architecture.

But what about the hardware?
Before you go, please note that the paper doesn't answer an important question; namely, how can an HMI developed with HTML5 communicate with a system's hardware devices? For instance, in the car, an HTML5-based HMI may need to communicate with the CAN bus, GPIO pins, and I2C and SPI devices, as well as with external devices like tablets and smartphones.

Fortunately, there's a paper for this, too. :-)

In "HTML5-Hardware Communication with PPS Messaging," also written by Andy Gryc, you'll find out how PPS, an HMI-agnostic, asynchronous messaging model, can provide a very flexible approach to communicating with in-vehicle hardware.

With PPS, devices don’t communicate with the HMI directly. Rather, they become publishers of data objects to which the HMI can subscribe. As a result, it becomes much easier to swap out or modify devices, as well as the HMI itself. This, of course, is the Reader's Digest version. Download the paper to get the fully skinny.

---

Further reading (and viewing)

If you're interested in HTML5 and the car, here are some other papers and posts I recommend:

• Moving beyond the browser: HTML5 as an automotive app environment
  
  
• Talking HTML5 with Audi’s Mathias Halliger
  
  
• HTML5: Bustin' the myths
  
  
• Why Automakers (Should) Care about HTML5
  

 

Video: Talking HTML5 with Audi’s Mathias Halliger

Derek Kuhn

From the elegant look outside to the technology inside, Audi has some of the most advanced cars on the road today. At CES this year, I sat down with Mathias Halliger, head of architecture, MMI system, for Audi AG, to talk about some of this technology and how HTML5 will transform the infotainment systems found in their cars.

Mathias firmly believes that HTML5 is an automotive game changer because of the doors it can open for OEMs, technology providers, app developers, and consumers. So pass the popcorn and without further ado, here is the latest video in our HTML5 video series.


 

QNX reference vehicle makes stopover at FTF Americas 2012

Fresh off Telematics Detroit, the QNX reference vehicle is on the road again. And this time, it’s headed to the Freescale Technology Forum (FTF) in San Antonio.

Have you seen photos of the vehicle? If so, you'll know it's a specially modified Jeep Wrangler. From the outside, the Jeep stills looks the same, but beneath the hood, something has changed. For the first time, the Jeep’s head unit and instrument cluster, both based on the QNX CAR 2 application platform, are using Freescale i.MX 6 processors. And what better place than FTF to show off this new processor support?

Closeup of Jeep's instrument cluster. See previous post for more photos of vehicle.

As before, the reference vehicle will showcase several capabilities of the QNX CAR 2 platform, including:

• auto-centric HTML5 framework
• integration with a variety of popular smartphones
• one-touch Bluetooth pairing with smartphones using NFC
• ultra HD hands-free communication
• DLNA support for phone- and home- based media
• tablet-based rear-seat entertainment
• reconfigurable digital instrument cluster
• Wi-Fi hotspot

The vehicle will also demonstrate several popular third-party technologies, including Pandora, Slacker, and TuneIn Internet radio; TCS navigation; Weather Network; Best Parking; and Vlingo/AT&T Watson voice recognition.

What, more demos?
The reference vehicle isn't the only place to catch QNX technology at FTF. QNX will also showcase:

• a 3D digital instrument cluster based on a Freescale i.MX 6 quad processor and the QNX Neutrino RTOS, and built with Elektrobit's EB GUIDE Human Machine Interface environment
 
• a complete dashboard, including head unit and digital cluster, based on the QNX CAR 2 platform
 
• demos for industrial controllers, medical devices, multi-core systems, and advanced graphics, all of which run on the QNX Neutrino RTOS and Freescale silicon

QNX at the podium
Did I mention? QNX experts will also in participate in several presentations and panels. Here's the quick schedule:

• The HTML5 Effect: How HTML5 will Change the Networked Car — June 19, 2:00 pm, Grand Oaks Ballroom A
 
• Using an IEC 61508-Certified RTOS Kernel for Safety-Critical Systems — June 20, 2:00 pm, Grand Oaks Ballroom P
 
• Embedded Meets Mobility: M2M Considerations and Concepts — June 20, 5:15 pm, Grand Oaks Ballroom E
 
• New System Design for Multicore Processors — June 21, 10:30 am, Grand Oaks Ballroom F

Visit the FTF website for details on these and other FTF presentations.

And if you're at FTF, remember to catch the QNX demos at pod numbers 1400 to 1405.
 

WIRED Autopia slips into driver's seat of QNX reference vehicle

Chances are, you've seen pictures of the new QNX reference vehicle. You may have even seen the "making of" video that QNX released a few days ago. But have you seen any video of the vehicle in action?

If not, check out this vid by Doug Newcomb of WIRED Autopia. Last week, at Telematics Detroit, Doug met up with Andrew Poliak of QNX for a tour of the vehicle and its various features, including a re-skinnable UI and voice-controlled Facebook integration. The camera was rolling, and here's what it caught:

HTML5 brings new buzz to infotainment system development

QNX to unveil QNX CAR 2 platform on Freescale i.MX 6 at FTF Americas — a guest post from Paul Sykes of Freescale

If you’ve visited the QNX website recently or attended the Telematics Detroit conference last week, then you’ve surely noticed that HTML5 is getting a lot of attention in automotive these days. The buzz around HTML5 focuses on two areas: as an application development and delivery framework, and as an HMI framework. In discussions with many industry participants, my impression is that the application framework part is generally accepted, while the HMI framework part still isn’t well understood.

I don’t intend to discuss these HTML5 aspects in detail. There are experts within the ecosystem that can do a much better job than me. But I will say that Freescale applications processors will offer the processing and graphics performance to run the desired applications and bring the HMI to life with stunning graphics.

Next week, Freescale will host the annual FTF Americas event in San Antonio, TX. We are very excited about the first public unveiling of the QNX CAR 2 application platform on i.MX 6. Since QNX CAR 2 is based on HTML5, it is particularly fitting to mention in this blog. For those with an interest in understanding more about HTML5 for infotainment systems, QNX and many other ecosystem partners will be on hand at FTF to discuss their thoughts and plans.

Paul Sykes is a member of Freescale’s driver information systems team.
 

The making of the QNX reference vehicle: Jeep Wrangler

Guest post from Nicole Forget of QNX Software Systems

Nicole Forget

Just one week ago, our new reference vehicle was revealed at Telematics Detroit 2012. The Jeep Wrangler features QNX’s digital instrument cluster, which is totally re-skinnable. In fact, the entire user interface of the head unit, which was created using HTML5, can also be re-skinned. The head unit supports loads of functions, too, including the virtual mechanic, which are outlined in an earlier post.

The following video gives you some insight into the hard work that was put into the making of the reference vehicle. Check it out!


 

Moving beyond the browser: HTML5 as an automotive app environment

If you’ve already visited this blog, you’ll know that we are bullish on HTML5 as a way to implement infotainment system HMIs. Not surprisingly, I’ve spent a fair amount of time searching the Web for facts and opinions on using HTML5 in the car, to see how this idea is catching on.

Overall, people see numerous benefits, such as the ability to leverage mobile app development to keep pace with the consumer demands, the availability of a large pool of knowledgeable developers, and the attractiveness of a truly open specification supported by many possible vendors.

But when it comes to the challenges of making HTML5 a reality in the car, I found a common thread of questions, mostly rooted in the erroneous belief that an HTML5 application environment is “just a browser.” Everyone is familiar with the concept of a browser, so it’s easy to see why people take this point of view.

So what are the key differences between a browser and an HTML5 application environment? Here’s my quick view.

The experience
Everyone is familiar with the browser experience. You navigate to a web site through bookmarks, a search engine, or direct entry of a URL. The browser implements a user interface (aka the chrome) around a rendering engine and provides bookmarks, URL entry, back and forward, scrolling and panning, and other familiar features.

An automotive HMI based on HTML5 provides a different experience — just look at the accompanying screen shots and decide for yourself if they look like a browser. In fact, the user experience of an HTML5-based HMI is similar to that of any other purpose-built HMI. It can consist of a main screen, window management, navigation controls, and other typical user interface widgets.


A radio tuner and a media player from the QNX CAR 2 application platform. Both apps are based on HTML5, but beyond that, they neither act nor look like a web browser.

A system that uses an HTML5-based HMI can include:

• core applications that look and act like native applications
   
• add-on (downloaded and installed) applications that have controlled interfaces to the underlying hardware
   
• “web link” applications that simply link to a cloud-hosted application that can be downloaded on demand and cached

The web link approach makes it easy to update applications: just update the server and the remote client systems will automatically pull the application when needed.

Local resources
Web browsers pull text, images, and other content from the web and render it on the user’s machine. The process of loading this remote content accounts for much of the user’s wait time. This paradigm changes with a local HTML5 application environment — because resources can exist locally, images and other components can load much more quickly.

What’s more, screens and user interfaces can be designed to fit the platform’s display characteristics. There is no need for panning and scrolling, and only limited need for zooming. Resources such as RAM can be optimized for this experience.

Security and sandboxing
Browsers load content and executable JavaScript code dynamically. This really is the power of the web technologies. The problem is, dynamically loaded code represents a threat to an embedded platform.

Browsers are designed to be sandboxed. By default, JavaScript code can execute only in the context of a browser engine, and cannot access the underlying operating system primitives and hardware. This approach changes in an HTML5 application environment. To give JavaScript code the ability to behave like a native application, the environment needs interfaces to the underlying OS through to the hardware. Plugins are used to implement these HTML5-to-OS interfaces.

Nonetheless, access to the underlying platform must be carefully controlled. Hence, a security scheme forms a critical component of the HTML5 application environment.

Application packaging
The app experience has become familiar to anyone who owns a smartphone or tablet. An HTML5 application environment in the car can also support this kind of experience: developers create and sign application packages, and users can download those packages from an application store. In an automotive context, authenticity of the applications and control over what they can or cannot do is critical. Again, a security model that enforces this forms a key part of the HTML5 application environment.

So, how should you think of an HTML5 application environment?
From my perspective, an HTML5 environment is like any other traditional HMI toolkit, but with much more flexibility and with inherent support for connected applications. In an HTML5 application environment, you can find technologies similar to those of any proprietary toolkit, including:

• a rendering engine (HTML5 rendering engine)
• a set of content authoring and packaging tools
• layout specifications (HTML5 and CSS3)
• a programming language (JavaScript)
• an underlying data model (DOM)

The difference is, these components are developed with a web experience in mind. This, to me, is the most significant benefit: the web platform is open, scalable, and well understood by countless developers.

Biff! Bap! Ker-Pow! It’s the BatBerry interview!

Paul Leroux interviews Tim Neil, a director of product management at RIM, who is building his very own Batmobile™. This project might sound like fun (and Tim assures us it is), but it also demands a wealth of skills, from welding to HTML5 programming.

Tim Neil

Tim, could you give us a quick overview of the BatBerry project?
The BatBerry combines my love of cars, Batman, and technology. I’ve always wanted to build this car and I’ve had a couple of unsuccessful attempts at creating a carputer. When RIM started creating a 7" tablet, I knew the time was right to bring all of these interests together.

How did you get started on this project?
I started my research about 15 years ago, trying to determine how and where to get started. For instance, I needed to track down the shifter, which is a throttle quadrant from a WWII US Navy bomber.

By 2010, I had finished modifying my custom Subaru WRX, and I needed to get started on something new — working on cars is my way of escaping and relaxing. The time was right, and I got the green light from my wife. Luckily for me, she knew of my desire to build this car when we met and it didn’t scare her away. :-)

The BatBerry, about a year after Tim launched his project

Reading your blog, I’m totally impressed by the scope of the BatBerry project — be it creating dashboard panels, writing control software, or building a retractable license plate. Do you do most of the work yourself?

Yes, I try to do as much of the work myself as possible. I leave important things that I don’t have experience in, like doing the frame stretch, to the professionals. I did the same thing building up my Subaru over the past 7 years: learning how to do body work, interior, stereo, engine modifications, etc. I like to learn things as I go and I’ve always had a knack for figuring out how things work. I always figure, what’s the worst thing that can happen? If screw up, I just have to try again.

To pull this off, you need to be a jack of all trades. I’m sure you had skills to begin with — but did you also have to pick up any along the way?
Welding is one of the biggest skills that I’ve picked up so far. I bought myself a welder, watched a couple of YouTube videos, and got to work. I can tell you, my welds look MUCH better now than my first ones. From all the welders I’ve talked to, it’s a skill that simply takes patience and practice.

Since I was a kid I have always been able to figure things out. When I was 8 years old I was wiring my bedroom up to have a switch on my headboard automatically open the door. The best way that I can describe to people how I see the world is by watching the movie Iron Man. When you see Iron Man’s computer JARVIS take an object and expand it out into a million pieces to show how it works, that’s what I see when I look at something.

Tim's other project — a highly modified Subaru WRX

What kind of power plant does the BatBerry use? Have you modded it?
The car currently has a 305 4.3L L99 V8. I haven’t really modified it yet. I will likely go with a re-built version of the same engine so that I can re-use the ECU. I’m not looking to make this car into a high-performance hot rod — that’s where my Subaru comes in. Plus, it’s nice to drive distances not always looking for a gas station that serves 94 octane. :-)

The V8 puts out 200hp, which should be pretty good for the BatBerry, considering it is basically a frame with a 400-pound fiberglass body mounted to it. As long as it sounds nasty I’ll be happy. I have a couple of Flowmaster 40 series mufflers for it.

Anyone who reads this blog knows we are bullish on HTML5. So I was fascinated to hear that the BatBerry project has an HTML5 connection. Could you tell us about it?
As the former development manager for BlackBerry WebWorks at RIM, I wanted to show what could be done with HTML5 technology. I wanted to build an interface on my PlayBook and BlackBerry Smartphone that could control some of the systems of the car.

I also wanted to share as much code as possible between the Smartphone and PlayBook, and using WebWorks and HTML5 allows me to do this. These devices pair with a Bluetooth connection on an Arduino board to control a series of relays that raise and lower the 30-cal machine guns, open and close the canopy, raise and lower the suspension, and perform other functions.

All the source code for the project, including Arduino microcontroller code, is being shared in my BatBerry repo on github.


Sample screen captures of the BatBerry user interface

What has been your greatest challenge? And what are you most proud of, so far?
My biggest challenge has been finding time! I’ve been travelling for work more on weekends and while this winter was pretty mild, it was still a bit hard to head out into a freezing cold garage to put in a couple hours of work during the evenings.

I would say the two things I’m most proud of so far are my welding skills and my dash panels. I really wanted to give back something to others who have been building their own versions of this car. Screen-accurate dash panels were something missing from the community. In general, I really like to share what I’m doing so that others who want to do something similar can see what worked, and what didn’t work, for me.

The Discovery Channel has been tracking the BatBerry project. Do they plan to broadcast anything soon?
Nothing to air at the moment. The next step will be to get updated footage of some of the technology integration points. I’m getting close to being able to show the combination of HTM5, Arduino, and the machine guns to get some new footage. Once we reveal the car, filming will wrap up and go into post-production for airing sometime in the future on Daily Planet.

When you aren’t working on the BatBerry, what do you do?
I spend my spare time hanging out with my family, doing something with cars, or playing with technology. My daughter is a big Star Wars fan so she and I have been having some epic lightsaber battles lately. I’ve done a lot of car shows in the past with my Subaru and I really like meeting up and trading experiences with the car community around Toronto. At RIM, I direct the product management group responsible for developer tools, APIs, and SDKs — our focus is on removing barriers and adding features to make developers successful.

One more question: Which Batman character do you most identify with?
I would say Batman himself. While I’m not on the tipping point of insanity and looking to be a vigilante, I identify with the desire to make a difference. I also relate to the do-it-yourself attitude and the love of cool tech and cars. Plus, I’m just a geek at heart. :-)

---

To track the progress of the BatBerry project, check out Tim’s blog. You can also follow him on Twitter.

And while you’re at it, visit Tim’s YouTube channel. Here, for example, is a video showing the BatBerry’s replica machine guns:




Neither Tim Neil, his vehicle, nor Research In Motion (BlackBerry) are licensed by, endorsed by, sponsored by or affiliated with DC Comics or the owners of the “Batman” properties.
 

Everything you wanted to know about HTML5 in the car, Part III

Welcome to the third installment in my Q&A series on HTML5 in the car. In Part II, we looked at web servers, native plug-ins, instrument clusters, and display updates. This week, we turn our attention to tools, touch gestures, UI performance, and vehicle resources.

Are there any HTML5 HMI builder tools available?
Most of the well-known IDEs, including Eclipse, Dreamweaver, and Netbeans, support some flavor of HTML5 in their latest release. Adobe Edge, a new tool now available in preview, also lets you create animated HTML5 content. I suggest you check out the HTML5 Tools site, which publishes up-to-date tool reviews.

Often, automotive customers will ask system designers to make an infotainment system work "like an iPhone,” with the popular gesture controls. Does HTML5 support "inertial" menus and two-finger zoom?
Multi-touch is handled at the app level; here’s an example. Pinch zooming at the browser level is browser-dependent — the QNX browser handles it, but not every browser does. As for physics-based scrolling, HTML5 doesn’t support it “out of the box”; it needs to be added. Frameworks like Sencha Touch provide these types of controls.

Will the performance of HTML and JavaScript be adequate for critical user interface components or computations, such as safety-related notifications?
This has to be tested on a case-by-case basis. For the UI elements, yes, the performance should be adequate. Our testing indicates you can build HMIs that are surprisingly responsive. Also, our WebKit port lets you do things things like run JavaScript code in other tabs, threads, or processes to ensure those ocmponents aren’t being thread-blocked by something less critical.

I do get a little gun-shy recommending HTML5 for safety-critical components, because JavaScript isn't inherently real-time. If you wouldn't feel comfortable using Java for a critical coding task, you shouldn't use HTML5 either. If you want predictable, real-time performance for a lower-level computation that cannot tolerate any delay, the code should execute in a non virtual-machine environment. Most code doesn’t really fit that description, so most of the time JavaScript should work just fine.

How do you call vehicle resources — vehicle HMI, vehicle diagnostics information, etc. — on a HTML web app in the car? What's the process in plain words?
In plain words, it’s kinda hard. :-) But here’s my best take on this question: we solve this by creating a vehicle-bus driver that exports data through a publish/subscribe mechanism. The HTML5 layer talks to that piece through a JavaScript interface.
 

A quick tour of the QNX CAR 2 application platform

If you're looking for a quick, two-minute intro to the QNX CAR 2 application platform, you've come to the right place.

In this video, Kerry Johnson, automotive product manager at QNX, takes us on a tour of the platform, including its home screen, media player, application area, HTML5 support, phone app, and acoustic processing.

Ready? Then hit the Play button and let's get started:



In case you didn't know, the QNX CAR 2 platform forms the basis of the QNX concept car, a specially modified Porsche 911 that demonstrates what to expect in next-generation car infotainment systems. Earlier this year, the platform drove home with a 2012 Best of CES award, in the Car Tech category.
 

HTML5: Bustin' the myths

Did you know you can build HTML5 apps that don't use an Internet connection? Did you know you can run HTML5 apps without a web browser? And did you know HTML5 apps can show snappy performance even on automotive silicon? (As you can well imagine, in-car infotainment systems don't ship with quad-core server-class CPUs.)

If you answered no to any of these questions, you need to stop for a minute and check out this interview with QNX Software Systems' Kerry Johnson. Heck, even if you answered yes to all three questions, you'll probably still appreciate what Kerry has to say — and besides, you'll catch a glimpse of a complete in-car UI coded in HTML5. What could be bad?



While I have you, check out Andy Gryc's Q&A series on HTML5, if you haven't already. You'll find the first two installments here and here.