Fully Autonomous Vehicles Must Learn How To Read Pedestrians’ Intentions

There are so many variables to take into account when it comes to pedestrians. Human drivers know this all too well.

We are quite adept at recognizing whether a person intends to cross the street or is just standing by the crosswalk for no reason, we know what it means when a bicyclist gets into his or her pedaling motion, and we can gauge the pace of a jogger as they approach an intersection, anticipating whether they plan on continuing their run or if they will actually slow down.

Now, unless your computer’s name is Ultron or Skynet, odds are it’s not going to take these types of variables into account, at least not with such intuition as the one displayed by humans and certainly not with 2018 tech.

“You can’t stop for every human being standing by the side of the road,” stated Volvo R&D boss Henrik Green. “But you also need to stop at the right point when the pedestrian is about to step into the street.”

During this year’s LA Auto Show, Volvo and Luminar demonstrated how advanced their LiDAR technology is, as it can even detect human poses, including individual limbs, at distances of up to 250 meters (820 feet).

Even if sensors become exceedingly good at detecting motion, that alone may not be enough to give an accurate prediction of what could happen. For example, if a jogger has been detected for three seconds running towards an intersection, the best prediction for future intent may not be that exact trajectory, but rather their face and whether or not they were looking at the vehicle. Another good example would be if a pedestrian is detected looking down at their smartphone – such a thing would constitute a higher probability of risky behavior, reports Automotive News.

“The important thing is to understand these sorts of features rather than just looking at movement,” stated Leslie Nooteboom, co-founder and chief design officer at Humanising Autonomy in the UK.

Aside from software engineers, Humanising Autonomy also employs a team of behavioral psychologists who shift through camera footage and help train systems on how people behave when it comes to interacting with traffic.

“You have to have general behavior models that are very detailed, and then the next step is to make them more localized,” added Nooteboom. “We have a foundation of general behaviors for a particular city, and then you can link to specific locations and identify how people will behave at an intersection with an obscured stoplight or a crossing that stops in the middle of the road.”

Bottom line, self-driving systems pretty much cannot afford to predict human behavior incorrectly. As Pete Rander, president of Argo AI puts it: “If you don’t predict well, you have two options and neither of them are good enough. You’re either left playing it safe and creating a much more cautious bubble around you. Or you’re slamming on the brakes.”

more photos...
  • Mr. EP9

    This could take decades to sort out as people are unpredictable. What if a meth addict runs out in the street completely naked and jumps on your car? What if some guy in a polar bear outfit is dancing in the middle of the street?

    • Craig

      I think it’s impossible. And the examples you gave are just two out of countless possible scenarios.

    • Jay

      Agreed, this is one of the hardest things to do and additionally people behind the wheel can be just as unpredictable. The engineers have their work cut out for them. I don’t think it will ever get to be 100%, what really is that good but it’ll get pretty close.

    • Navy – Lincoln ✓ᵛᵉʳᶦᶠᶦᵉᵈ

      Floor the vehicle.

    • D3X

      All your scenarios would be impossible for even a human driver to decipher as a dangerous situation and react to it fast enough. We all know how many dumb and stupid drivers are out there.

      On that sense, AI autonomous software just needs to be statistically better than human driving, that would be enough. It doesn’t need to be perfect.

      • Mike anonymous

        The problem is while I think it may be able to process faster and do things at a faster rate, it wouldn’t be able to do it better, it would only do things with less likelihood. Humans are the one who are programing the AI therefore the AI will just inherit the problems or advancements of whoever programed it.

        Maybe it can be improved, maybe I’m wrong, but from what I know I do think AI can get to the point of being better than a human, but it certainly get to the point where it may value it’s own existence over that of the driver/user of the vehicle piloting it, or those outside of it.

        • D3X

          I think most people are not capable of understanding AI, not right now, as the future is really unknown. Programming an AI isn’t what most people would percieve it to be, machine learning is a completely new territory for humans.

          As much as I agree with your statement that although the AI from a driving standpoint may never surpass, one that I disagree with is the ability to communicate and have a ecosystem built around the AI system to allow the AI to be inherently better. Humans are poor reacting by perception and poor communicators especially with multitasking, an AI would never have this issue.

          For example, to solve this pedestrian problem at the most difficult areas; intersection s, crosswalks, high traffic areas, problem prone areas etc, installing Smart traffic system with emergency measures(pedestrian detection) that actively notify the interconnected AI systems real time would vastly make the AI a better driver.

          • Mike anonymous

            I absolutely agree. The capabilities of AI on it’s own are up for debate, and are not widely clear (to most people) as you mentioned. But you are right in that it does help what kind of system is built around that AI,. but then I believe it would come down to how much are companies willing to invest to keep people safe, and what lengths will they go to for money and/or power in any situation.

            But not to get off topic, I completely agree with you.

  • Navy – Lincoln ✓ᵛᵉʳᶦᶠᶦᵉᵈ

    Humans who have average intelligence knows the outcome at crosswalks: looks both ways to ensure there aren’t any motorists of all sorts or kind is coming in a specific direction the pedestrian is walking. This isn’t rocket science; this is common sense.

    • Finkployd

      yea in general you’re right. But some days you’re in a hurry / it rains / it’s night time / you ‘re listening to music, or twitting or drunk or just distracted … and then you don’t pay enough attention

  • Marty

    It doesn’t have to be 100 % safe.
    If it could make a US car as safe as a Scandinavian _human_ it would save like 10,000 lives every year in just the USA.

  • Mark Seven-Ultranine

    Let’s see…. How’s an autonomous vehicle going to handle a tractor trailer driver falling asleep and jack-knifing on the Pennsylvania Turnpike? This happened to me and I am alive only because I floored my Corvette between the encroaching side of his truck and shot through the narrow space next to the concrete berm. How’s the AV going to handle fog so thick you can’t see the front of your car? Or multiple cars going into spins on snow-covered roads? Or a hydroplaning car that slides across the lane while trying to pass you. I could go on and on. We are so very far away from these AVs being safe.

18-Year-Old Charged With DUI Smashes Lamborghini Gallardo Into Two Vehicles

The crash was so loud it made some mistake it for an explosion or something.

Maserati Quattroporte L’Ultimo Concept Is Italian Luxury At Its Finest

This futuristic Maserati Quattroporte render stays (mosty) true to the current model’s design language.

Lexus Safety System+ To Be Standard Equipment On All 2020 Models

The safety suite includes automatic high beam headlights, adaptive cruise control and a a pre-collision system.

Porsche North America Boss Says EVs Will Soon Become Commonplace

Porsche is confident it can easily out-sell its first year of production of the upcoming Taycan .

2020 MINI Electric Teased, Debuts July 9th

MINI is going electric, but the upcoming Cooper SE is only expected to have a range of approximately 114 miles.

Korean Brands Dominate Initial Quality Study, But Porsche’s 911 Comes Out On Top

Genesis had the highest initial quality and they were closely followed by Kia and Hyundai.

Ford Says Explorer ST Customers Aren’t Interested In A V8

The hottest Explorer on sale is powered by an EcoBoost 3.0-liter V6 with 400 HP.

SUV And Tow Truck Decide To Go Their Separate Ways

After the SUV fell off, the tow truck began rolling forward towards a parked van.

2020 Cadillac XT5 Facelift Bows In China With Minor Updates

The 2020 Cadillac XT5 has a new grille and an updated infotainment system with a rotary controller.

Spiderman Goes Electric, Takes A Spin In The Audi E-tron GT

The latest Spiderman flick will feature cameos by the Audi A7, Q8 and e-tron.