NVDA: Training at the edge

Investment Analysis Clubs Saul’s Investing Discussions
1

I’m at an automotive conference right now. Just heard a talk about AI in vehicles, and the speaker mentioned something I hadn’t considered before:

The normal model is to have sensor/behavioral data uploaded to servers, where a neural net can perform training. For automotive, there are significant privacy concerns with uploading of private data to clouds, especially since this may not be suitable for anonymization (vehicle wants to learn your preferences so that it can choose climate or navigation settings automatically for you).

Therefore, there is a need for enough training horsepower at the edge (in this case the edge is the vehicle). To me, this opens up the market for NVidia chips, typically used for training. I had previously considered that edge processor would be inference only (lower processing requirements, typically).

Anyway, while training in the cloud is easiest and cheapest, there may be drivers, such as privacy concerns, for having training in edge devices like cars or even smartphones.

10 Likes
2

Therefore, there is a need for enough training horsepower at the edge (in this case the edge is the vehicle). To me, this opens up the market for NVidia chips, typically used for training. I had previously considered that edge processor would be inference only (lower processing requirements, typically).

I too had considered “that edge processors would be inference only.” This opens a new market for Nvidia. Very, very, very interesting, getting better and better.

Thanks!

Denny Schlesinger

1 Like
3

The Drive AGX line has enough juice that it could easily do Training, especially the high end cards.

4

I too had never considered it. But then I’m not sure “driving preferences” and similar can be considered as PI that even raises a privacy issue. It’s not like revealing birthdate, mother’s maiden name and social security number. From what you posted, I’m not sure there’s a real privacy issue here.

5

The volume market will absolutely be inference at the edge, not training at the edge. There will be some need for training at the edge in some cases. But I question how much training will be going on with automobiles. It will be mostly inference, and inference at the edge will be a requirement because you can’t tolerant latency issues, connectivity, etc. As per the privacy concerns about things like HVAC settings, a Nest thermostat can do that type of “training” already. If you need a high powered GPU to train on how I like my seat adjusted, radio settings and climate control then you’re doing something fundamentally wrong.

2 Likes
6

Yeah interesting theory but I don’t think there’s much fruit in it.
Privacy is a big issue and there will need to be better solutions, but this doesn’t seem like a big deal.

Plus I want my preferences in the cloud so when I jump into a new car, be it a rental or vehicle as a service, the car will have detected my presence and adapted itself to my preferences before I even enter. No need to faff around with the seat or mirrors. Now that’s seamless service!

Plus, what exactly did the speaker suggest would require high level training st the edge? Would it be something like how the person accelerated and brakes…?

1 Like
7

It’s not like revealing birthdate, mother’s maiden name and social security number. From what you posted, I’m not sure there’s a real privacy issue here.

That view of privacy is narrow, and thankfully outdated today. One of the revelations from the Snowden and other releases is that the US and British governments (at least), track people’s metadata. Just location and time alone are incredibly telling for big brother. They can know who you meet (same location at same time), or think they know who you meet, for instance. Bad actors can use that to rob your house when you’re not home. I suggest reading up on GDPR and why that legislation was even proposed.

As per the privacy concerns about things like HVAC settings, a Nest thermostat can do that type of “training” already. If you need a high powered GPU to train on how I like my seat adjusted, radio settings and climate control then you’re doing something fundamentally wrong.

We’re talking more than rudimentary preferences here. Location/time has legit use for navigation systems that might want to predict where you’re going based on where you are and the current time/day. With driving assistance systems, how much should they learn about how you drive? You may end up with higher insurance rates because you drive more quickly than others, even though you haven’t (yet in their view) had an accident. Radio settings tell your taste and that’s valuable information for advertisers - do you want that shared? Some people will, but some people won’t.

Plus I want my preferences in the cloud so when I jump into a new car, be it a rental or vehicle as a service, the car will have detected my presence and adapted itself to my preferences before I even enter.

This isn’t AI related, but pragmatically, that’s difficult even within a single model of car since the servo used aren’t indexed, but even if that is solved, different models of cars will have different settings. There are ways to capture that information locally, encrypt it on your phone with a private key, send the encrypted data to the cloud (without the key). Then your phone can decrypt it in other vehicles.

Things like fingerprint readers or facial recognition are best done locally only, as Apple insists they do. Once that information is shared to a cloud you lose control over who has access, especially if that cloud is hacked.

Back to my original point. I’m not saying that all training needs to be done at the edge, just that more training will be performed at the edge than might be technically logical due to privacy concerns. Whether that has a big impact

1 Like
8

Just location and time alone are incredibly telling for big brother.

For anyone (which is just about everyone) with a cell phone this info is already already available. So it’s moot as related to autonomous vehicles. And it’s not just “big brother” who has access. And yes, I’m familiar with GDPR, but I’ve not studied it closely. Even before GDPR the EU privacy regulations were a pain in the behind for my employer. If you read GDPR (I have not) I’m pretty sure you’ll find that collecting private information necessary to carry out the business at hand or deliver whatever service is requested is not the problem. The problem comes from collecting unnecessary information or sharing with unauthorised parties that which is supposed to be kept private.

There’s a limit to how much one’s “privacy” can be protected in a connected world. The more connected we become (think IoT) the more exposure we all have. Buy a shirt with an RF tag and match that to you cell phone time/location and CC transaction data and we have insight into your shopping and fashion preferences. You can extrapolate this endlessly. Or you can go live in an off the grid cave.

As for the cloud being hacked, what internet connected database can’t be hacked? And what significant database is not internet connected (with the exception of black projects and even some of them are internet connected). BTW, these concerns are exactly the reason many of us are stoked about OKTA and ZS.

I’m not trying to be facetious, I’m just saying you can raise a “privacy” flag over just about everything having to do with interconnectivity. It’s ubiquitous problem, so the relationship with driving preferences is just another instance of an insoluble problem.

9

I don’t know about Android, but for iOS I have control over with whom my location is shared, and when it’s shared. Yes, the phone company knows which cell tower I’m using and so can do some approximate triangulation, but that’s still just an approximation. As for GDPR, the essence is that you have to tell people what information you’re collecting and why, and how you will use it. You also have to promise to protect it.

This does and has consequences - just ask Uber, who was forced into changing their settings from “Always” and “None” to also add “While Using the App.” This only happened because people like me knew about it, and pushed back. This is only one example. Just look at what’s going on with Facebook and privacy. Google has had similar problems with GMail, which prompted me to abandon the service for a competitor (TutaMail, if you must know). To me, the trend is clear that people’s data has value and people will more and more want to be “paid” for sharing their data. Heck, you can now literally buy a cup of coffee with your data (https://www.npr.org/sections/thesalt/2018/09/29/643386327/no… ).

I’m just saying you can raise a “privacy” flag over just about everything having to do with interconnectivity. It’s ubiquitous problem, so the relationship with driving preferences is just another instance of an insoluble problem.

My point is that it’s not insoluble, and that doing so requires AI training at the edge. Companies are today finding that not collecting some data, or not collecting it all the time, can be a good thing for adoption of their service/product. If a company can supply AI-enabled features without needing to send private data to a server, that would be considered, at least by some companies, as a competitive market advantage (if not an eventual requirement). And, this could be solved by processing at the edge instead of on the server, which can benefit companies that can enable that.

A really good example of this is Apple’s Face Recognition, which is based on Convolutional Deep Learning AI: https://machinelearning.apple.com/2017/11/16/face-detection… "

Most of the industry got around this problem by providing deep-learning solutions through a cloud-based API. … However, due to Apple’s strong commitment to user privacy, we couldn’t use iCloud servers for computer vision computations. …

2 Likes