Nvidia pursuing Inference, not just training

There has been concern that Nvidia GPU’s were just being used for AI training, and in the future, when mass markets using the inferences have been developed, other companies whose chips were cheaper and simpler would take their place. Here is news about huge new contracts for inference use!

Nvidia Pursues Inference Business
Nvidia Corp. recently announced agreements with two large corporations for drone applications that represent vast revenue potential in the high growth AI and deep learning segment. These moves show Nvidia seeking to secure a substantial share of the AI inference business to build on revenue derived from the training of neural networks. They also illustrate the company targeting the huge Chinese market for AI applications.

The first of the two agreements is with JD.com, (NASDAQ:JD), the second largest online retailing company in China, for the manufacture of drones, with aerial drones representing an area of autonomous robotics beyond autos. Under this agreement, drones will be applied to uses in delivery to retail customers, in agriculture, and for rescue operations.

Nvidia’s Jetson platform will provide navigation intelligence to an aerial drone and to a ground-based drone, with the latter needing to negotiate uneven terrain. JD declared that Jetson was selected due to its low power draw and low cost.

One Million Drones In Next Five Years
As to the performance parameters of the aerial drones, JD has said that they will fly at velocities of up to 100 km an hour to deliver packages of up to 30 kg, with future models perhaps able to deliver packages of approximately 200 kg.

Most interestingly for Nvidia, and demonstrating the centrality of such end markets in the company’s forward planning for inference applications, JD claims it will produce one million drones in the next five years. This projection is largely based on JD’s calculation that drones will reduce logistics costs by 70%, especially when it is considered that parts of China suffer from inadequate infrastructure which impedes overland delivery.

The potential impact on Nvidia’s stock price of the JD agreement is appreciable. With overall revenue from GPUs standing at approximately $1.6 billion in Q2 FY18, this and similar contracts will establish a new income stream for the company which may be described as a sub-group of auto, namely autonomous vehicles excluding automobiles.

Lower Marginal Cost
Much of the groundwork has been laid by the work the company has already done within the autonomous automobile sector, providing Nvidia the opportunity for market expansion of the autonomous segment with lower marginal cost, reaping the benefits of sunk costs and economies of scale.

With the prospect of one million drones being produced by JD alone over the next five years, clearly this inference application for the retail market around the world will be highly revenue-generative for Nvidia when the company’s current 58.39% revenue margin is borne in mind. Expanding the paradigm and building on the knowledge base derived from the JD project to serve other large retail adopters around the globe promises significantly enhanced revenue.

Nvidia currently depends for 58.3% of its revenue on one market, gaming, and consequently is in dire need of broadening its revenue streams to insulate it against a possible downturn in that segment. This provides strategic logic to its pursuit of end-user inference applications: a quest for greater defensive flexibility.

The second of the noteworthy agreements Nvidia has struck to develop aerial drones is with Avitas Systems, a General Electric Company venture. The purpose of the agreement is to combine AI, analytics, and drones to more efficiently undertake inspections of heavy industrial facilities, like oil rigs, where the installation is located in an inhospitable environment. Drones will shoot video of the installation, which will then be analyzed by AI computers to locate issues requiring attention, such as leaks and corrosion.

Hundreds Of Millions Spent On Inspections
Avitas estimates that industrial companies outlay hundreds of millions of dollars each year for inspection and maintenance, much of it mandated by insurance companies. Market expansion for Nvidia within the market for industrial testing, inspection, and certification will lay in the direction of autonomous underwater vehicles and robot crawlers.

As with the JD aerial drone project, when Nvidia has built an inference ecosystem for the Avitas project that will be transferable to other customers with similar operating needs, the company will have achieved a paradigm that will attract new customers within the sector at a lower incremental production cost.

Avitas will utilize Nvidia’s DGX-1 and DGX training systems to achieve defect identification, while developing convolutional neural networks for image categorization, with ancillary generative adversarial neural networks to reduce the compute necessary to identify images.

Large Potential Revenue Stream
Nvidia and Avitas believe that the cost of plant inspections may be reduced through the use of drones by 25%. End-user markets will include oil and gas, petrochemicals, aerospace and defense, construction, food and beverage, brewers and distillers. Transparency Market Research holds that the global market for testing, inspection, and certification will grow at an annual rate of 5.7% to 2024 to reach $285.34 billion, clearly representing a large potential revenue stream for Nvidia.

Previously Nvidia’s growth in AI has been driven by the neural network training market rooted in the data centers of U.S.-based cloud services providers. Now, building on the recent introduction of TensorRT3 software in all of China’s major data centers, the company is developing business in the machine learning inference segment of the fast-evolving Chinese AI market.

China’s State Council anticipates annual IT spending in the country will top $901 billion by 2020, while research firm i-Research projects that China’s AI market will grow at a 50% CAGR. China has been estimated to offer half of the most interesting AI opportunities in the world, according to venture capitalist Jim Breyer.

Inference To Contribute More Than Training
The inference market should in the longer run provide more revenue for Nvidia than will the training market, its erstwhile prime revenue contributor within the AI sector. However as Nvidia moves further towards end-user applications by increasing participation in the inference market, the company may anticipate that the level of competition will escalate.

As a consequence, Nvidia’s unit profit might decline as price competition sets in, yet increased volume, sunk costs and economies of scale should more than offset this consideration. Importantly, full immersion in the inference market is an essential strategic move for the company to maximize its continued growth, as it seeks to remain central to the evolutionary changes of the next generation of AI applications.

With the costs of penetration of the inference market reduced by the amount of research and production experience the company has already acquired through its activities in the autonomous autos market, Nvidia will enjoy a reduced risk position when pursuing end-user inference application opportunities…




there are big problems with battery powered aircraft. In one word, Range. Not mentioned in the PR I read.
A delivery drone has to go to the customer and come back. Also unlike fossil fuel aircraft your “fuel” load does not decrease during the trip. Half the trip includes the load weight of the delivery parcel. Then there is the incumbent weather problem, and the fact that failure means a heavy object crashing into whatever is below.
It seems to me that air “inspection” of various kinds is a more near term reality.

In 1980 or so who would have predicted that we would see CPU in things like watches and phones? The scope of AI may be just as broad, we are in the early stages. The scope of the data based decisions it can make will expand. Except for those all too frequent “gut” or emotion based decisions humans make aren’t our decisions supposed to be data based too?
The human mind is not getting any better it’s the same as it was 100,000 years ago. But machine minds will improve. A flat trend line vs a rising trend line. Eventually they will cross. Just like the trend lines for gas power vs electric power in cars.


there are big problems with battery powered aircraft. In one word, Range. Not mentioned in the PR I read.
A delivery drone has to go to the customer and come back. Also unlike fossil fuel aircraft your “fuel” load does not decrease during the trip. Half the trip includes the load weight of the delivery parcel. Then there is the incumbent weather problem, and the fact that failure means a heavy object crashing into whatever is below.

Certainly there will be some hurdles to getting drone technology for parcel deliveries to work, but it doesn’t seem out of the realm of possibilities like passenger aircraft, for instance. The quote from Mauser’s linked article about 223,000 pounds of liquid fuel equating to 4.5 million pounds of batteries says it all for large planes.

But we are working with orders of magnitude smaller aircraft for delivering packages. JD.com is talking about delivering packages up to 66 pounds with their eyes on 440 pounds later on - a significant difference than passenger aircraft.

Surely safety will be a concern which is why this is more likely to happen first in China as opposed to the US. But given how small these drones will be, I have faith that a good solution can be developed. This doesn’t seem to be such a far stretch.


It’s so very reassuring to know that we only have to worry about the weight of the drone and a 66 pound payload falling on our heads from several hundred feet as opposed to the weight of a passenger aircraft . . .

OK, a little sarcasm never hurt anyone very much. But safety is a big concern. The article mentioned remote areas where ground infrastructure is not well developed. But then there’s that nasty range problem. It’s the “last mile” in metro areas that’s making delivery costs a major problem. In China, where the vast majority of the people live in huge condo developments I don’t understand how a drone will deliver a package to someone living in the 2nd apartment on the 5th floor of building 32.

If it makes a phone call to the customer to inform him that package is on ground in front of the entry, then the customer has to lug that 60 pound package up five flights of stairs instead of the delivery person. And what if the customer isn’t home? Leave it on ground where it will likely be stolen? Carry it back to the central distribution point (is there enough left in the battery to haul the payload back)?

Point-to-point delivery is a pretty messy thing. It’s not good enough to just get close, often it requires the customer’s signature as proof of delivery.


You are assuming that the next delivery system with drones will look like what we have now.

What if the drones help the delivery process, and bring the package to the delivery man at a central point and he finishes the delivery. Instead of all the Fed Ex drivers always returning to hub, the hub continuously feeds them product for their area. Drivers spend more time delivering, and less time driving back and forth. The drones not only know where the drivers are, but where they will be when the drone meets up with them.

I think drones will be used in ways that we aren’t even thinking of. That is the blessing of new technologies and the capitalistic way. Someone will think of a way to make it work, and everyone else will say, “Why didn’t I think of that.”


Drones being used as transferring packages. As far as a fleet of small drones delivering packages will be a big gamble as far as liability of injuring humans if one malfunction. I have seen prototype drones that a person could fit in with a weight payload restriction and sq. footage to carry a fair amount of small to medium package with a human pilot with several large drone like blades. The article said that this would be used in rural to sparsely populated areas with electric motors with backup petroleum fuel (like Hybrids) and multi parachutes strategically located in case of emergency if all back up motors become inoperable to work.

1 Like

So you’re asserting that having a drone deliver a 60 pound package to a driver is somehow more efficient and cost effective than having a truck deliver hundreds of pounds of packages to a central delivery hub . . . Seems like a stretch to me.


Yeah, I get it. Use the drones to deliver to remote locations. But then there’s that range problem . . .

Delivery drones. Sure they will be a big market.

But I think we should focus on what’s really important. NVDA is making money off of training/machine learning in the cloud. But inferencing! It’s a much bigger opportunity and if NVDA can really get a big slice of that market then growth of NVDA can have another huge leg up. Why is it important? Inferencing will be used in devices at the edge. Basically, inferencing allows devices to be smart. It is like putting a brain inside a device. Think about that statement for a moment. Putting brain inside of a device, marking it start, giving it the ability to make decisions. Now if that device also has senses (sensors) and mobility. Wow! How many applications are there for intelligent devices, devices with a brain. NVDA is the picks and shovels provide. It enables the development and training of the brain and then it produces the actual brain (GPUs used for inferencing at the edge). Right now Jetson is their product for this and I’m pretty sure it’s currently a very small part of NVDA’s revenue with most of the AI sales going into the data centers. Now think about the double exponential: a) GPUs getting more powerful at a rate faster than Moore’s Law and b) AI algorithms and SW getting better exponentially. This means that the brains of devices will get more and more powerful very fast. What applications will be targeted? How many device brains will be developed and needed? If NVDA makes the best brain then they will get a big share of a market that’s surely about to see explosive growth.



You are assuming that the next delivery system with drones will look like what we have now.
laws of physics and aerodynamics have not changed since the Wright Brothers. Battery powered aircraft are not new, the R/C model crowd has been using them for decades, I have flown some myself.

Certainly there could be marked improvement in the way they are controlled.

But the actual machines are going to show only minor incremental improvement .
Unless we get batteries that are both lighter and store more energy. So far battery improvement has been incremental t at a 6% or so rate per year. Lithium is already #3 in lightness, weight , so progress will have to be made in the metal cladding (plastics??) and the relatively heavy metals used in electrodes.

Maybe we will eventually be using entirely different batteries. But maybe not, cars are still using gasoline after 100 years. But the gasoline is lots better than it was in the days of the Model T.

Look at the multi prop design of most drones. They become unstable if only one motor is out. And unlike winged aircraft can not glide. I do like the design of one I saw with 8 motors. Each “corner” of the platform having 2 motors one stacked on the other, which might mean failure of one can be compensated for. 8 motors mean 8 times the risk of failure…

OTOH electric motors are inherently reliable. I have a cheap box fan in my house that’s been running non stop for over 25 years.

Drones have uses or Amazon would not be putting so much time and effort into them. Seems to be a last mile sort of solution in some cases, in other cases same day 2 hour or less delivery. In some cases, maybe even food.

Drones will have their uses, and logistically they will be made to work.

They will not be delivering refrigerators any time soon, but there are many high volume smaller products that they can better enable.


On the topic of energy density of storage media, delivery of packages (particularly last mile) is a completely different use case than transporting humans, so I wouldn’t get hung up too much on that aspect.

(This coming from someone who is not too optimistic about widespread use of batteries as part of trying to stabilize the electric grid.)

Similarly, I happened to finally read within the past few days a tab about a company named Lilium I had had open on my phone for several weeks or more. This company has an interesting concept, using batteries and flight for short/medium-range taxis.


1 Like

Ever hear the word bee line? GPS for the sky road set up hub in a 360 degree Idea center figured out by artificial Intelligence. Using semi EV trucks from several located main distribution center strategically located throughout the U.S. Even electric rail lines could be feeding these Hubs. Let’s think outside of the box. CEO Jeff Bezos is a master of of this.

This is out of the box about EV air planes. What if engineers were able to Commercially copy a solar cell framed with the limpet tooth composite called “bionspiration” a natural biological design that could be copied commercially and design a solar roll that could be rolled out after the plane reaches a minimum altitude of the safest level to roll out cells to absorb solar power. Extreme strength observed in Limpet teeth. Link about these teeth here- http://rsif.royalsocietypublishing.org/content/12/105/201413…