Then you didn’t understand what I was saying. A color digital photo is 4 arrays of numbers, each number indicating the intensity of that color at that location in the array. Have fun plugging those numbers into your equations. Thee whole image is fed into the software that recognizes what is in the image. I don’t see any handy numbers there either.
The pattern recognition is not a layer of code.
No, I don’t know what is actually there in any of those implementations. I expect it is a long way from the simplistic merge the two images that some imagine. Likewise far from connect the two with some equations.
No, I don’t know what it is either. What I do know is that eight to 10 companies are pursuing the same end result, and only one of them is saying “Lidar makes it worse because you can’t decide which one to choose”. Every single other one that I can find is saying “We use Lidar because it gives more accurate data, and we also have no trouble having multiple inputs and decision making.”
So far every one of those companies I mentioned in every city I mentioned is actually operating, some have been for over two years and we’ve heard no big issues. Meanwhile the one that has barely started operating, and with human safety drivers, says “that’s not the way to do it and it will cause problems.”
For some reason you keep parroting that guy instead of looking at all the others. Weird.
I am neither parroting that guy nor advocating his position. I am recognizing the point and arguing against a number of positions taken in this forum that I believe have nothing to do with whatever success these companies are having.
It seems unlikely that every location you cited has a unique implementation. How many unique implementations are there? What are they actually doing?
For example, suppose an implementation in which the photo data was the primary driver, but for some objects in the image it referenced the LIDAR (and RADAR!?) to confirm the distance estimate. That would be a combined implementation, but a long way from imagined merging of images.
People have plenty of fun and success plugging in those numbers to lots of formulas.
Search multilabel image classifier.
A search on intro to neural networks might be helpful also.
Yes it is, in that a layer in a neural network is code, which it is.
1 Like
I didn’t see anyone write “simplistic.”
There’s a ton of info online.
Excerpts from a 2020 article found with a quick search.
Sensor fusion algorithms combine sensory data that, when properly synthesized, help reduce uncertainty in machine perception.
Let’s take a look at the equations that make these algorithms mathematically sound
Convolutional neural-network-based methods can simultaneously process many channels of sensor data. From this fusion of such data, they produce classification results based on image recognition.
It is simple just slow the car down to avoid an accident when any red flag is thrown.
If I can solve that you’d think someone to be paid 1 trillion in controlling equity would get a clue. Does Musk ever drive a car?
I play cat and mouse w the cops. If I can’t see over a hill or around a bend I slow down. I am not the first person or machine to use the brakes.
1 Like
We have had this discussion previously … there is code underlying the pattern recognition capability, but it is not as if one can be sticking one’s own equation in there.
My office window looks out to a two-lane road that terminates at Google Bay View campus. I see a lot more Waymo Jaguar vehicles on that road now. One event (or series-of-events in particular) made me aware of this. A Waymo Jaguar drives by in one direction (moving away from Bay View campus), and about 25 - 30 seconds later, a Waymo Jaguar headed in the opposite direction (headed to the Bay View campus). While my line of sight didn’t allow me to see this, imagine two (Waymo) Jaguars crossing paths in the middle of an urban jungle?
3 Likes
There can be abstract goals, ie, braking. And in fact there are.
This means there are overarching behaviors. Again braking.
Waymo seems to know this.
Someone atop Tesla might not get this.
There is not a simple one-line equation (more like an army of equations), but there are still equations that raw image data (numbers) are inputted into to generate image classifications as the output.
The input data are highly multivariate and can include data from multiple sensors and sensor types.
Did you try
- a search “multi-label image classifier”?
- look at this page sensor fusion article
or any similar pages?
Here’s some formulas and a neural network:
Well, we know that Tesla and Waymo have “unique” implementations. I hope that much is obvious. Let’s drill down:
“Are the nuTonomy and Baidu driving technologies different from each other?”
“ Yes, nuTonomy and Baidu self-driving technologies are different from each other, most notably in their core decision-making software approaches and their current operational strategies. [ * Approach: nuTonomy’s software is based on formal logic and a hierarchy of rules, giving priority to rules like “don’t hit pedestrians” over “maintain speed”. This rule-based approach makes it easier to debug and verify safety compared to a “black box” machine learning system. ]
“Are the AutoX, Elmo, and Zoox technologies driving different from each other?”
Yes, the driving technologies of AutoX, Elmo, and Zoox are different from each other in their fundamental approaches, hardware, and operational models. [ * Approach: AutoX, which has rebranded as Tensor in the U.S., focuses on a “full-stack” autonomous driving solution, utilizing off-the-shelf vehicles (like the Chrysler Pacifica or Lincoln MKZ) equipped with its proprietary technology. Its technology is described as “camera-first” but heavily supported by lidar and radar systems for safety and redundancy, in contrast to systems relying solely on cameras. ]
Elmo * Approach: Elmo’s core technology is teledriving, a fundamentally different concept from the others. Instead of a fully autonomous system where AI drives the car, a human operator controls the vehicle remotely from a “teledriving station” using advanced network connectivity ]
\ In summary, [Zoox] and AutoX aim for a future with no human intervention in the car, relying on advanced AI and sensor suites, while Elmo provides a commercial, road-legal solution using remote human operators.
Is the self driving technology of the VW Buzz and Bayanat in Abu Dhabi different from each other?
Overview
Volkswagen Self-Driving Vehicle Revealed; Ready for Production
Yes, the self-driving technology is different, as the VW ID. Buzz uses its own autonomous driving platform, while the Bayanat project in Abu Dhabi partners with Vay for its teledriving technology. The VW Buzz uses the Mobileye platform with cameras, LiDAR, and radar, while Bayanat’s system relies on a teledriver system where a remote operator can take control.
There are self-driving tests going on in 20 Chinese cities by a variety of vendors, so in addition to those already mentioned above, I asked “Are the selfdriving technologies of Didi, Ford, and Nio different from each other?”
Yes, the self-driving technologies of Didi, Ford, and Nio are different, primarily in their target level of autonomy and their end applications.
Nio * Focus: Nio develops in-house full-stack ADAS capabilities, known as NIO Assisted and Intelligent Driving (NAD), primarily within the Level 2/Level 3 range, with recent approvals to test L3 vehicles on public roads in China.
Didi’s self-driving technology is a Level 4 autonomous system that uses a combination of sensors, high-performance computing, and artificial intelligence to enable fully autonomous vehicles. Key components include its proprietary DiDi Gemini hardware platform, which features extensive sensor suites (lidar, radar, cameras), powerful compute systems (up to 700 TOPS on NVIDIA’s platform), and multiple layers of redundancy to ensure safety.
I suppose I could go on, but so far I count 10 or more quite different applications and approaches to “solve” the problem. One or two using one by line” programs, some involving stacks, some (Ford) aiming just for limited applications on pre-defined routes.
Yeah, different. All but one decided to have Lidar involved, at least for now.
1 Like
Of course there are numbers there. It’s all numbers. The entire digital image is nothing but numbers, and object detection is nothing but math.
3 Likes
The picture as numbers has nothing to which to react … just red 127 in this dot. One can;t do anything meaningful until one has progressed to human at 15’ in such and such a direction. Meanwhile, what does the LIDAR see?
It sees a radar reflection (light reflection, actually, unlike radar which is “radio wave”) that tells it whether that thing in front is a real object, a shadow, or something that’s not a solid mass.
As it does so, a thousand or a million times a second, it knows whether it is getting closer to the moving vehicle, at what speed, and whether it should take cautionary action like braking or steering.
We have all seen things that turn out to be other things: reflections in water, apparitions in shadows, even blinding headlights. Lidar informs whether the object is real or not, which I would think would be valuable information.
3 Likes
I understand the potential attraction of LIDAR. The complexity comes if attempting to combine the two, i.e. the trunk of that tree is 15’ away where the recognition of the tree and context comes from the camera and the distance comes from the LIDAR … does the LIDAR recognize the tree?
No. Lidar recognizes density (that thing exists!) and it’s 100 ft away, at 94° off on the right. It’s 99 ft away, at 95°off on the right. It continues track the object (the entire landscape, actually) feeding data to the mother computer, which says “that is a tree, on the side of the road, and can safely be ignored.”
It would have a different response if the object was moving (car, child, animal), or was an illusion.
1 Like
Does it need to?
I am not even an armchair expert in this stuff but here is how my brain thinks about how this would work.
The camera sees a tree trunk and identifies it correctly as a trunk - but isn’t completely sure because it could be something discolored in the road. LIDAR identifies it as a solid object that is clearly not the road. The two systems resolve it to be a trunk and the system takes appropriate action.
Alternative 1: Camera DOES NOT see it as a trunk, but LIDAR sees it as a solid object - the most cautious and safest course of action is to try and avoid the solid object.
Alternative 2: Camera sees it as a trunk but LIDAR does not see a solid object - the most cautious and safest course of action is to try and avoid an object that appears to be a trunk in the road.
In the case where there is disagreement, take the safest course of action by assuming it is indeed a hazard.
2 Likes