Fully Autonomous Vehicles Must Learn How To Read Pedestrians’ Intentions

There are so many variables to take into account when it comes to pedestrians. Human drivers know this all too well.

We are quite adept at recognizing whether a person intends to cross the street or is just standing by the crosswalk for no reason, we know what it means when a bicyclist gets into his or her pedaling motion, and we can gauge the pace of a jogger as they approach an intersection, anticipating whether they plan on continuing their run or if they will actually slow down.

Now, unless your computer’s name is Ultron or Skynet, odds are it’s not going to take these types of variables into account, at least not with such intuition as the one displayed by humans and certainly not with 2018 tech.

“You can’t stop for every human being standing by the side of the road,” stated Volvo R&D boss Henrik Green. “But you also need to stop at the right point when the pedestrian is about to step into the street.”

During this year’s LA Auto Show, Volvo and Luminar demonstrated how advanced their LiDAR technology is, as it can even detect human poses, including individual limbs, at distances of up to 250 meters (820 feet).

Even if sensors become exceedingly good at detecting motion, that alone may not be enough to give an accurate prediction of what could happen. For example, if a jogger has been detected for three seconds running towards an intersection, the best prediction for future intent may not be that exact trajectory, but rather their face and whether or not they were looking at the vehicle. Another good example would be if a pedestrian is detected looking down at their smartphone – such a thing would constitute a higher probability of risky behavior, reports Automotive News.

“The important thing is to understand these sorts of features rather than just looking at movement,” stated Leslie Nooteboom, co-founder and chief design officer at Humanising Autonomy in the UK.

Aside from software engineers, Humanising Autonomy also employs a team of behavioral psychologists who shift through camera footage and help train systems on how people behave when it comes to interacting with traffic.

“You have to have general behavior models that are very detailed, and then the next step is to make them more localized,” added Nooteboom. “We have a foundation of general behaviors for a particular city, and then you can link to specific locations and identify how people will behave at an intersection with an obscured stoplight or a crossing that stops in the middle of the road.”

Bottom line, self-driving systems pretty much cannot afford to predict human behavior incorrectly. As Pete Rander, president of Argo AI puts it: “If you don’t predict well, you have two options and neither of them are good enough. You’re either left playing it safe and creating a much more cautious bubble around you. Or you’re slamming on the brakes.”

more photos...
  • Mr. EP9

    This could take decades to sort out as people are unpredictable. What if a meth addict runs out in the street completely naked and jumps on your car? What if some guy in a polar bear outfit is dancing in the middle of the street?

    • Craig

      I think it’s impossible. And the examples you gave are just two out of countless possible scenarios.

    • Jay

      Agreed, this is one of the hardest things to do and additionally people behind the wheel can be just as unpredictable. The engineers have their work cut out for them. I don’t think it will ever get to be 100%, what really is that good but it’ll get pretty close.

    • Navy – Lincoln ✓ᵛᵉʳᶦᶠᶦᵉᵈ

      Floor the vehicle.

    • D3X

      All your scenarios would be impossible for even a human driver to decipher as a dangerous situation and react to it fast enough. We all know how many dumb and stupid drivers are out there.

      On that sense, AI autonomous software just needs to be statistically better than human driving, that would be enough. It doesn’t need to be perfect.

      • Mike anonymous

        The problem is while I think it may be able to process faster and do things at a faster rate, it wouldn’t be able to do it better, it would only do things with less likelihood. Humans are the one who are programing the AI therefore the AI will just inherit the problems or advancements of whoever programed it.

        Maybe it can be improved, maybe I’m wrong, but from what I know I do think AI can get to the point of being better than a human, but it certainly get to the point where it may value it’s own existence over that of the driver/user of the vehicle piloting it, or those outside of it.

        • D3X

          I think most people are not capable of understanding AI, not right now, as the future is really unknown. Programming an AI isn’t what most people would percieve it to be, machine learning is a completely new territory for humans.

          As much as I agree with your statement that although the AI from a driving standpoint may never surpass, one that I disagree with is the ability to communicate and have a ecosystem built around the AI system to allow the AI to be inherently better. Humans are poor reacting by perception and poor communicators especially with multitasking, an AI would never have this issue.

          For example, to solve this pedestrian problem at the most difficult areas; intersection s, crosswalks, high traffic areas, problem prone areas etc, installing Smart traffic system with emergency measures(pedestrian detection) that actively notify the interconnected AI systems real time would vastly make the AI a better driver.

          • Mike anonymous

            I absolutely agree. The capabilities of AI on it’s own are up for debate, and are not widely clear (to most people) as you mentioned. But you are right in that it does help what kind of system is built around that AI,. but then I believe it would come down to how much are companies willing to invest to keep people safe, and what lengths will they go to for money and/or power in any situation.

            But not to get off topic, I completely agree with you.

  • Navy – Lincoln ✓ᵛᵉʳᶦᶠᶦᵉᵈ

    Humans who have average intelligence knows the outcome at crosswalks: looks both ways to ensure there aren’t any motorists of all sorts or kind is coming in a specific direction the pedestrian is walking. This isn’t rocket science; this is common sense.

    • Finkployd

      yea in general you’re right. But some days you’re in a hurry / it rains / it’s night time / you ‘re listening to music, or twitting or drunk or just distracted … and then you don’t pay enough attention

  • Marty

    It doesn’t have to be 100 % safe.
    If it could make a US car as safe as a Scandinavian _human_ it would save like 10,000 lives every year in just the USA.

  • Mark Seven-Ultranine

    Let’s see…. How’s an autonomous vehicle going to handle a tractor trailer driver falling asleep and jack-knifing on the Pennsylvania Turnpike? This happened to me and I am alive only because I floored my Corvette between the encroaching side of his truck and shot through the narrow space next to the concrete berm. How’s the AV going to handle fog so thick you can’t see the front of your car? Or multiple cars going into spins on snow-covered roads? Or a hydroplaning car that slides across the lane while trying to pass you. I could go on and on. We are so very far away from these AVs being safe.

Cammer Brakes To Avoid Red-Light Runner, Promptly Gets Rear-Ended

Even so, the one who’ll foot the bill for the repair will probably be the driver of the car behind the cammer.

Even Valentino Balboni Was Impressed By The Lamborghini Miura Jota SVR

The Jota SVR started out life as a regular 1968 Miura S and its owners went to great lengths to get his wish fulfilled .

Wald Wants To Turn Your Mercedes C-Class Into An AMG C63 Lookalike

The bundle of upgrades is suitable to the W205 generation of the C-Class, from 2014 to 2018.

New Toyota Supra Races Porsche Cayman GTS, Audi TT RS And Alpine A110

Toyota has high aspirations about the all-new Supra – so how does it fare against some of its rivals?.

So You Think You Can Afford America’s Cheapest Mercedes E63 AMG?

A piece of advice: you shouldn’t do donuts in your W211 E63 if you don’t want your power steering to fail.

Two New Electric Crossovers Could Doom The Lincoln Continental In America

US production of the Continental will reportedly end in 2021.

The Average Dodge Challenger Buyer Is 51 Years Old, Still Younger Than Mustang And Camaro Customers

The Dodge Challenger has also comfortably outsold the Chevrolet Camaro in the first half of 2019.

Talk About Getting Dumped: Florida Man Pours Dirt On Cadillac CTS Driven By Girlfriend

The girlfriend didn’t own the Cadillac, so its owner was likely in for a big surprise.

Return Of The Sport Truck? Ram Considering Lowering Kit For 1500 Pickup

Previewed by the Low Down concept, the kit could reduce the ride height by approximately 2 inches.

Failed Aston Martin Vanquish Tooling Buyer Identified As Detroit Electric

Detroit Electric agreed to pay $24 million for the tooling of the Vanquish, but failed to come up with the money.