The ecommerce giant is investing millions in training an ambitious large language model (LLMs), hoping it could rival top models from OpenAI and Alphabet, two people familiar with the matter told Reuters. The model, codenamed Olympus, has 2 trillion parameters, the people said, making it one of the most significant models being trained. The team behind the secretive project is spearheaded by Rohit Prasad, a former head of Alexa who now reports to Amazon CEO Andy Jassy. He has brought in researchers working on Alexa AI and the company’s science team to train the model, uniting them with dedicated resources. Amazon believes that having homegrown models could make its AI offering more attractive on its cloud business AWS, where enterprise clients want access to top-performing models. There is yet to be a timeline for releasing the model.
More extensive models take longer to train and require more computing power than smaller ones, but they are also better at understanding complex text and producing human-like responses. It is a race that the world’s top tech companies are waging, with each trying to outdo each other in terms of how sophisticated their models are.
Amazon, which has the benefit of its vast cloud infrastructure, is pushing to become a leader in artificial intelligence. In September, it launched Amazon Bedrock, a fully managed service that allows businesses to pick the models they want and fine-tune them with relevant data. It aims to serve as a one-stop shop for AI developers.
Last month, the company invested over $1 billion in its AI research labs and boosted investment in AI startups by more than a third. This is a big push for the company, which has previously focused on building hardware, such as its Echo smart speakers and other hardware devices, rather than developing software or AI models.
A lot of attention has been paid to the rumor that the new model Amazon is training has more than 100 trillion parameters, which would make it among the most significant models ever built. While having more parameters can lead to better performance, it is not always the case, Yann LeCun, a widely considered “godfather” of AI and Meta chief AI scientist, tweeted earlier this year.
The latest development shows that the company is doubling down on its efforts to become a significant force in the field of AI, and it may be a good move given its massive computing and server infrastructure and dominance in AWS hosting. However, there is a risk of overestimating the value of such models, which can still be wildly inaccurate and make mistakes. As such, measuring the quality of such models and ensuring they can perform as expected before deploying them for use in real-world applications is essential. That way, the technology can be used safely and correctly to benefit humans. For instance, if an AI system makes a mistake while performing a medical procedure, it could have serious consequences.
