On Monday, Amazon Web Services ( AWS ) released a tool named TorchNeuron to help AI startups more easily migrate their AI applications to Amazon's AI chips Trainium. This move is also a direct challenge to NVIDIA's software advantages – it is these advantages that have helped NVIDIA become the default hardware choice for AI applications and driven it to become the preferred AI hardware supplier.
TorchNeuron can be used in conjunction with PyTorch. PyTorch is a set of free open-source toolkits that developers use to build, train, and run AI models. Amazon states that with the help of TorchNeuron, AI applications written using PyTorch can run on Trainium with just one line of configuration change, without the need to rewrite any code.
Amazon claims that approximately 85% of AI applications are written in PyTorch. PyTorch was initially developed by Facebook, but since 2022, it has become part of Linux Foundation and is maintained as an open-source project. Over the years, NVIDIA has been ensuring that its CUDA software—i.e., the software infrastructure that enables applications to run on NVIDIA GPU and other hardware—can seamlessly support AI applications written in PyTorch. As a result, NVIDIA's hardware is now deeply integrated into this toolchain and has almost become the default option.
TorchNeuron is adapted based on PyTorch, so developers familiar with PyTorch will also know how to use it. Amazon is not the first company to try to make its own chips more easily compatible with this platform. PyTorch Users can also migrate their applications to AMD GPU through simple configuration changes, and even to Apple chips.
However, Amazon believes that migrating applications to Trainium is not only simple but also allows AI startups to access Amazon's full range of cloud services. AWS clearly thinks this is better for startups than using neoclouds; the latter is heavily tied to NVIDIA and its hardware, such as Coreweave, Nebius, and Firmus.
"You are building on AWS. You're not just betting on a provider that offers compute power for a single purpose, and then scrambling to build everything else around it," Colin Brace, Vice President of Technology at AWS Annapurna Labs, stated to TechCrunch via email. Annapurna Labs refers to Amazon's in-house chip development department. He added, "On AWS, the infrastructure required from prototype to production is already in place, allowing the founders to move forward quickly and scale up when needed."
AWS indicates that since the same chip can be used for both training (teaching the AI model) and inference (putting the trained model into use, such as answering user questions), and it is closely integrated with the cloud infrastructure required by startups, even though the unit token price of AWS is not the lowest, its cloud services can still help customers save costs.
Brace says: "The value of Trainium lies in its cost-effectiveness, as well as the fact that it is a platform that covers both training and inference, which improves the economy of the entire workload. Take a look at Deepgram – they use native PyTorch to support migration to Trainium, and while achieving production-level accuracy, they have increased the amount of audio that can be processed per dollar by 70%." Deepgram is a AI voice application provider and also a competitor to ElevenLabs.
But Amazon is not satisfied with just providing PyTorch application developers with an easy option to migrate to its cloud.
Amazon also stated that it is working to move PyTorch away from being centered around NVIDIA automatically, and will contribute the code that achieves this goal to the main project of PyTorch. In a blog post announcing this new tool, the company wrote:
Amazon's approach is not limited to its own hardware. The company is contributing upstream changes to PyTorch itself and collaborating with the PyTorch community to remove the code that binds this framework to specific chips. The goal is to make PyTorch more hardware-independent for the entire developer community, not just for Trainium users.
Although this strategy is undoubtedly beneficial for hardware and cloud service providers that compete with NVIDIA, one of the purposes of PyTorch is to hide those troublesome details when AI code runs on hardware. We still need to wait and see whether pushing NVIDIA further away will indeed bring more choices and lower costs for developers, as Amazon claims, or if it will only lead to more configuration difficulties.
In any case, if Amazon truly succeeds in deeply decoupling PyTorch from NVIDIA, the impact could be far more significant than just dealing with chip competitors. As Brace says, "The real story is about eliminating that kind of lock-in effect that leaves developers with no real choices at all."
