Amazon is ditching Nvidia GPUs in favor of their own silicon

What just happened? Amazon has announced that they’re migrating their artificial intelligence processing to custom AWS Inferentia chips. This means that Amazon’s biggest inferencing services, like virtual assistant Alexa, will be processed on faster, specialized silicon instead of somewhat multi-purpose GPUs.
Amazon has already shifted about 80% of Alexa processing onto Elastic Compute Cloud (EC2) Inf1 instances, which use the new AWS Inferentia chips. Compared to the G4 instances, which used traditional GPUs, the Inf1 instances push throughput up by 30% and costs down by 45%. Amazon reckons that they’re the best instances on the market for inferencing natural language and voice processing workloads.

Alexa works like this: the actual speaker box (or cylinder, as it may be) does basically nothing, while AWS processors in the cloud do everything. Or to put it more technically… the system kicks in once the wake word has been detected by the Echo’s on-device chip. It starts streaming the audio to the cloud in real-time. Off in a data center somewhere, the audio is turned into text (this is an example of inferencing). Then, meaning is withdrawn from the text (another example of inferencing). Any required actions are completed, like pulling up the day’s weather information.

Once Alexa has completed your request, she needs to communicate the answer to you. What she’s supposed to say is chosen from a modular script. Then the script is turned into an audio file (another example of inferencing) and sent to your Echo device. The Echo plays the file and you decide to bring an umbrella to work with you.

Evidently enough, inferencing is a big part of the job. It’s unsurprising that Amazon has invested millions of dollars into making the perfect inferencing chips.

Speaking of, the Inferentia chips are comprised of four NeuronCores. Each one implements a “high-performance systolic array matrix multiply engine.” More or less, each NeuronCore is made up of a very large number of small data processing units (DPUs) that process data in a linear, independent fashion. Each Inferentia chip also has a huge cache, which improves latencies.

reference

content editor at zino.

۳ comments On Amazon is ditching Nvidia GPUs in favor of their own silicon

  • I and also my friends were looking through the excellent points located on your site and quickly came up with an awful feeling I had not expressed respect to the web site owner for those secrets. Those ladies became for this reason very interested to see them and have now in fact been having fun with them. Appreciation for really being indeed helpful and for opting for varieties of fine topics millions of individuals are really eager to understand about. Our honest regret for not expressing appreciation to earlier.

  • Thank you so much for providing individuals with a very breathtaking possiblity to discover important secrets from this web site. It really is very terrific and also packed with a great time for me and my office co-workers to search your site at a minimum thrice in one week to see the latest stuff you will have. Of course, I am always fascinated with all the brilliant suggestions you give. Selected two ideas in this article are surely the most beneficial we have all had.

  • I must show some thanks to this writer for rescuing me from such a issue. Because of looking through the online world and finding strategies which are not beneficial, I believed my entire life was done. Being alive without the solutions to the difficulties you’ve solved by way of your main short article is a crucial case, as well as the kind that would have in a negative way affected my entire career if I had not encountered your web site. Your good capability and kindness in controlling all the pieces was precious. I don’t know what I would have done if I had not discovered such a solution like this. I can at this moment look ahead to my future. Thanks a lot so much for your professional and sensible help. I will not hesitate to endorse your web page to anyone who will need assistance on this matter.

Leave a reply:

Your email address will not be published.

Site Footer

Sliding Sidebar

Categories