AWS has its own chips: Here's how its CEO sees their future

Amazon (AMZN) Web Services (AWS) has entered the semiconductor market, developing its own chips to train AI models in competition with industry leaders like Nvidia (NVDA). At the 2024 Goldman Sachs Communacopia and Technology Conference, Yahoo Finance reporter Madison Mills interviewed AWS CEO Matt Garman to break down AWS's chip strategy.

Garman acknowledges Nvidia's strong market position, calling it "a great platform" with a large customer base. However, he emphasizes that the chip market is vast, with "potential for multiple options," stressing the importance of customer choice.

AWS's semiconductors, Inferentia and Trainium, are "specifically built for AI inference,"Garman explains. These chips offer particular value for small-scale inference tasks and helping customers reduce costs. He also notes that AWS is working on improving these chips to train large language models.

"We think that there's this really large market segment and there's room enough for customers to be using the best product for the use case for a long time," Garman told Yahoo Finance. Although he expressed support for other chipmakers, stating that AWS does not expect to become "fully reliant" on its own chips.

Catch Yahoo Finance's full interview with Matt Garman here.

For more expert insight and the latest market action, click here to watch this full episode of Morning Brief.

This post was written by Angel Smith

Editor's note: This post has been updated to fix an incorrect name.

Video transcript

You mentioned your chips business.

So let's let's talk about chips.

I'm just curious in terms of your own LLM.

Um, what is your current usage ratio wise of NVIDIA trip chips versus your own in house chips for training?

Well, look, we we don't share that specifically, but, um but, you know, if you if you talk to customers out there still, you know, the the large majority of usage is in NVIDIA chips, and they have a great platform, and, um and they're very popular with customers.

That said, we think that it's a really large, uh, market segment, and there's a tonne of potential for multiple options.

And so customer choice is super important.

And we're, um, seeing, I would say, an accelerating pace of customers getting excited about both training and inferential, which is our our custom processors.

And so inferential is the chip that's specifically built for a I inference, and that's now been on the market for about four or five years.

And, uh, and we're increasingly see customers use that, um, particularly it's really valuable for a couple of different use cases where there's small inference that needs to be done where customers can really save costs.

And we see sometimes customers save 6070 percent off of their inference bills when they move to inferential, and we're increasingly seeing really large, larger and larger customers do that move on the training side.

Um, we're on our first generation of trainum.

And so, uh, a lot of the work that we've done over the last year or two is to get the software stack in a place where it, um, can be really powerful for building some of these really large models.

And so, um, that's where we've seen a lot of the the market, um, and customer base progress over the last year or two.

And we have customers like, um uh, Ninja Tech, who built their entire model all on trainum.

And they're super excited about that platform.

And we have trainum too.

Um, that'll be launching by the end of this year.

And that is a platform that we feel incredibly excited about for really large training clusters where customers can get outsized price performance gains, um, relative to the the traditional, um, platforms that they've been using around NVIDIA.

Now, that being said, we think that there is gonna be, uh, a mix of of use cases for a long period of time, and we support Intel.

We support a MD, and we have our own general purpose processor called Graviton.

But we also support NVIDIA GPU, and we're going to support training in Foria, and we think that there's a really large market segment, and there's room enough for customers to be using the best product for the use case for a long time.

But given the price point of NVIDIA chips, wouldn't it be beneficial for you to continue to rely on your own chips more and more over time versus we think so.

And we think we think it is for us and for customers both.

So how long until you fully reliant on your own AWS chips?

Like I said, I actually think there's gonna be use cases for both over time, so I don't I don't know that we'll ever be fully reliant on them because I think that the NVIDIA team builds some great processors and some great some great products, and they execute really well.

So I think there's gonna be use cases where NVIDIA GP US are are the best solution for a really long time.

And, um, you know, we're gonna push really hard to open up and have more and more and more of those served really well by training.