Here's Why Gesture Controls in Your Car Are a Terrible Idea

We live in an age of rapid technological innovation. Everything around us is changing at an incredible pace. The phone in your pocket went from a handy, if a bit dumb piece of technology to being the gateway to any interaction or piece of information with anyone or anything on earth, within about a decade... and it's cheap! For as much as we like to complain about it sometimes (like OMG all the time), we can all pretty much agree that this tech revolution is awesome and is just getting better.

Well, most of the time.

There are instances where we are stuffing new technology into every little aspect of our lives, whether it's an improvement or not. In this case, it's a gimmick made to solve a problem that didn't even exist in the first place. BMW, Hyundai, Audi, Ford, and Mercedes-Benz are all working on their own version of gesture controls for navigating around the increasingly complex world that is your automobile.

I work in the tech industry and have had tons of hands on time playing with, demoing, and working with motion controls, and I am here to tell you exactly why it doesn't belong in our cars.

Here's Why Gesture Controls in Your Car Are a Terrible Idea

So far it looks like these systems will be mostly used for simple menu navigation or quick settings that people change often like changing songs or climate controls (although they also showed off some less than productive things, like playing Poker). OK, that sounds simple enough, right? Well, first let's look at the current model of doing things:

Here's Why Gesture Controls in Your Car Are a Terrible Idea

This is Ford's standard cram-into-everything-with-wheels steering wheel for the moment, more specifically from the Focus ST (which I own). From here I can:

  • Navigate menus (up, down, left, right, and select)
  • Adjust the stereo's volume
  • Change audio tracks
  • Pause/play my music
  • Answer and end calls
  • Enable voice controls
  • Set and adjust cruise control

All this without ever even lifting a hand from the steering wheel. The controls are conveniently placed right next to where your thumbs naturally sit, are clearly marked with easy to recognize symbols, and are well within your peripheral vision so that you can easily see as well as feel where each button is under your thumb. They even have convenient little ridges and shapes designed specifically to let you feel them out (like the home row of your keyboard), and give a satisfying click to let you know they worked. Couldn't be more simple, right?

The problem with performing simple tasks like this, but with gestures, is how unnecessary the movements are. These systems work on basic principals to prevent false positives and false negatives. This means that the sensors will be looking at a specific window of space, most likely right in front of the screen. So if you want to swipe to the side on a menu, you take your hand off the wheel and motion to the side in front of the system's cameras, and then look over to see if it worked. Did it? How cool is technology? That was like magic!

But wait, what did this accomplish that was more efficient than the existing methods? The same task could have been done from the little arrow under your thumb, or in the case of touch screens, you get to feel the screen under your finger and swipe to the side, a gesture that we're all familiar with from our phones and tablets. Now you might be thinking to yourself that I'm just a stodgy hold out for the days of the past and just need to embrace new technology, but lets try to remember what problems this is trying to solve:

1. Distracted driving — I'm not sure if you've ever used a Microsoft Kinect or a Nintento Wii, but navigating using gestures is nothing but a pain in the ass. You hover your hand over your approximation of where you think it should be, watch to make sure what you're doing is correct, and then go from there. I'm not saying that things will be this poor when operating the system in your car, but it won't be far off. The problem is that you rely completely on visual feedback to let you know that you accomplished anything. At best this means big, easily identifiable changes, and a worst it means something like an audio cue or alert to let you know you did anything.

This also means that the system would be limited to more basic interactions for the sake of simple, quick gestures. I'll go more in depth on this below.

2. Button memorization — I don't know about you, but I don't need any more than second glance after that to remember where a button is — and that's if it is outside of my peripheral vision, which is not often for common controls like this. Again, these systems are all pretty similar across platforms. Stereo controls have been in the center of your dash for decades, and it's a pretty solid guess that the climate controls will be somewhere below those, and again the signage for these controls have long since been standardized. Just have a look below and tell me you wouldn't immediately recognize these controls anywhere:

Here's Why Gesture Controls in Your Car Are a Terrible Idea

While we're talking about button memorization, how about gesture memorization? This is something that you cannot rely on visual cues to remember (unless they put big reminders everywhere, which would be annoying), unlike the symbols and words on the current controls that are immediately recognizable, allowing you to go by "I'll know it when I see it." This admittedly isn't an issue for simple navigation since that is fairly intuitive, but what happens when you start getting into more specific gestures or specific tasks?

Here's Why Gesture Controls in Your Car Are a Terrible Idea

Once I swipe through the menu, how do I select what I want? Do I hover my hand over the screen like shown above? Does that mean that the menu is simplified down to only one item on the screen at a time for me to select (like the image where he's sorting through albums), or will I have to hover specifically over the one I want? What if I hit a bump and my hand moves? Do I have to carefully swipe around the menu again because I accidentally swiped out of it?

If I want to bring up navigation, do I trace a square in the air for the shape of a map?

If I want to set the HVAC to recirculate, do I make a circular motion with my hands?

Is any of this an improvement over just pushing the god damned button I want and being done with it?

Here's Why Gesture Controls in Your Car Are a Terrible Idea

Look. I'm all for technological innovation, but we have to draw the line somewhere for certain tasks. I know that gesture controls are still in the relatively early stages of development, but the biggest problem is the incredibly imprecise method of input. Humans are great at reaching out and touching or grabbing things, but hovering our arms out in the air in specific locations with nothing to brace them against is something we're just not good at — especially not when going over bumpy roads and trying to stay alert to changing traffic conditions. We're much better at controlling physical objects, or even with voice controls, which are getting better and better all the time (but still aren't great).

While it looks cool being able to manipulate your environment by just waving your hands in air like some kind of cool, non-Scientologist Tom Cruise, this just isn't the answer for our distracted driving problem.