Autonomous vehicles enter highways

[Tamhas]

@Goofyhoofy

feeding data to the mother computer, which says “that is a tree, on the side of the road, and can safely be ignored.”

Which mother is this?

[Tamhas]

Most trunks are not in the road and most solid objects in the road are problems, but it doesn’t take much imagination to make the decision tree much more complex … starting with the tree, which we think of as fixed in space is actually rapidly moving in the field of view of camera and LIDAR, which fields of view are not identical.

[mostlylong]

@Tamhas

picture as numbers has nothing to which to react … just red 127 in this dot

Did you take some time to do any of below?

@mostlylong

Did you try

• a search “multi-label image classifier”?
• look at this page sensor fusion article
  or any similar pages?

Here’s some formulas and a neural network:

@mostlylong

Check it out, they are everywhere (functions, formulas, parameters): (https://www.researchgate.net/figure/Basic-schematic-of-neural-network-f-TH-parameter-estimation-scheme-Simulation-data_fig1_369333214)

[1poorguy]

This reminds me of a NOVA I saw a few years ago. “Look Who’s Driving”, I think was the title. It isn’t on PBS website anymore. But it may still be out there.

Driving is an enormously complex activity. We (humans) make it look easy. “Just have the computer do this or that, and you’re golden”. Bzzzzt. Just a simple example I remember from the program, it could recognize pedestrians (on sidewalks, in the street, etc). But a pizza delivery guy carrying the pizza on his shoulder didn’t flag as a pedestrian.

I was very optimistic about a driverless future until I saw that program. We can create a lot of assist features, but relying on full autonomy (for now, and for several years hence) is foolhardy. We humans process better and faster when driving (believe it or not), at least in part because our brains are able to quickly determine what is ignorable, and focus on what is important. A computer with optical and LIDAR inputs can’t really do that. We also recognize a pedestrian carrying a pizza as a pedestrian carrying a pizza.

[mostlylong]

@mostlylong

Did you take some time to do any of below?

On sensor fusion:

The perception module ingests raw sensor data and outputs an environment model – a structured representation of the surrounding objects, actors, and scene context. Waymo’s perception system relies heavily on deep learning. It uses convolutional neural networks (CNNs) and other learned models to detect vehicles, pedestrians, bicyclists, road signs, traffic signals, and other relevant objects in the sensor [11] . For lidar point clouds, Waymo has developed models for 3D object detection and tracking (often voxelizing or using range images as intermediate representations) [6]. For camera images, CNN-based vision models identify lane markers, traffic lights, and perform 2D/3D detection of objects. Sensor fusion combines these modalities to improve robustness – e.g. associating a 3D lidar detection with camera imagery to classify it with higher confidence. Detected objects are then fed into a tracking system (often a form of Bayesian filter or recurrent network) that maintains state estimates (position, velocity, orientation) over time for each object [6]. The result is a set of tracked agents around the vehicle, each with kinematic state and classification (car, pedestrian, cyclist, etc.).