The widely used Large Language models have revolutionized the way we talk about Artificial Intelligence nowadays.
In a recent statement from Sam Altman, Open Ai’s CEO, he quoted “Open AI is losing money on Pro subscription. People use it too much.” But what’s wrong in that, a user is paying subscription cost too. The problem lies in the operation cost of running and maintaining LLM models. With increasing usage, the operating costs increases further, resulting in losses. But that’s not all, in this article, we’re going to see the problems associated with Large Language models. But first, let’s see what LLMs are briefly.
What are LLMs?
Large language models, as we know are large deep learning models trained on large datasets performing several functionalities of language processing. The underlying Transformers are neural network designed to extract text statements and understand the relationship lying between the sequences of text. This deep learning methodology has been advancing over the past decades, but its been only the last decade when it has picked up pace and some breathtaking outcomes have been produced, thanks to also support from advancements in hardware to support such neural network and datasets processing.
From content creation to Data analysis, LLMs have found usage in several key operations today to bring more efficiency and help users focus on the end objectives.
Challenges with LLMs:
Training Costs
LLM models are trained with large CPU Units, often consuming enormous resources. That said, the Nvidia GPUs used for GPT models costs minimum 10k$. It is estimated that a whooping approx. 25000 A100 GPUs were used for training the latest GPT model. In total, a nearly $63 million were spent on training the GPT4-o model. This demonstrates how vast amount of money is required just to train the models and put into production.
| Model | Parameters | Approx Training Cost | Year |
|---|---|---|---|
| GPT-2 | 1.5 Billion | ~ $50,000 | 2019 |
| GPT-3 | 175 Billion | $12-15 Million | 2020 |
| GPT-4o | 1.8 Trillion | $ 500 Million | 2024 |
| Llama-2 | 70 Billon | $20+ Million | 2023 |
| Gemini Ultra | 1.56 Trillon | ~ $1-2 Billion | 2024 |
| Llama- 3.1 | 405 Billion | $1+ Billion | 2024 |
GPT3, the predecessor to the latest model however costed around $4.6 million according to OpenAI. The difference between the two has been the elevation of the sheer amount of parameters used during computational process. This has been the case with other Large LLM models too.
Operating Costs:
Once an LLM model is trained, Inference costs come into the picture, which comprises of cloud servers, GPUs, power consumption and maintenance costs. A new report published by Analyst Dylan Patel in 2023 estimated approximately $700,000 cost to run ChatGPT per day. The computational resources required to generate responses for user prompts often demand high performance hardware and thus contributing to the massive operating costs.
OpenAI currently offers subscription costs for its Plus and Pro applications priced at $20 monthly and Pro for $200 respectively. These are still not able to bring profitability in the organization due to the high investment in the training and subsequently operating GPT models. However, each year with optimization in the use of parameters while producing a response to a prompt, the organizations have been able to reduce the inference costs. But it would still take some years to run it sustainably.
Leave a Reply