OpenAI is reducing the price of the GPT-3 API — here’s why it matters

OpenAI is slashing the worth of its GPT-3 API service by as much as two-thirds, based on an announcement on the corporate’s web site. The brand new pricing plan, which is efficient September 1, might have a big impression on corporations which are constructing merchandise on high of OpenAI’s flagship giant language mannequin (LLM).

The announcement comes as current months have seen rising curiosity in LLMs and their purposes in numerous fields. And repair suppliers must adapt their enterprise fashions to the shifts within the LLM market, which is quickly rising and maturing.

The brand new pricing of the OpenAI API highlights a few of these shifts which are happening.

An even bigger market with extra gamers

The transformer architecture, launched in 2017, paved the best way for present giant language fashions. Transformers are appropriate for processing sequential knowledge like textual content, and they’re much extra environment friendly than their predecessors (RNN and LSTM) at scale. Researchers have constantly proven that transformers turn into extra highly effective and correct as they’re made bigger and skilled on bigger datasets.

In 2020, researchers at OpenAI launched GPT-3, which proved to be a watershed second for LLMs. GPT-3 confirmed that LLMs are “few-shot learners,” which mainly implies that they’ll carry out new duties with out present process further coaching cycles and by being proven just a few examples on the fly. However as an alternative of constructing GPT-3 obtainable as an open-source mannequin, OpenAI determined to launch a industrial API as a part of its effort to seek out methods to fund its analysis.

GPT-3 elevated curiosity in LLM purposes. A bunch of corporations and startups began creating new purposes with GPT-3 or integrating the LLM of their present merchandise. 

The success of GPT-3 inspired different corporations to launch their very own LLM analysis initiatives. Google, Meta, Nvidia and different giant tech corporations accelerated work on LLMs. Right this moment, there are a number of LLMs that match or outpace GPT-3 in measurement or benchmark efficiency, together with Meta’s OPT-175B, DeepMind’s Chinchilla, Google’s PaLM and Nvidia’s Megatron MT-NLG.

GPT-3 additionally triggered the launch of a number of open-source initiatives that aimed to convey LLMs obtainable to a wider viewers. BigScience’s BLOOM and EleutherAI’s GPT-J are two examples of open-source LLMs which are obtainable freed from cost. 

And OpenAI is not the one firm that’s offering LLM API providers. Hugging Face, Cohere and Humanloop are a number of the different gamers within the subject. Hugging Face gives a big number of completely different transformers, all of which can be found as downloadable open-source fashions or by way of API calls. Hugging Face just lately launched a new LLM service powered by Microsoft Azure, which OpenAI additionally makes use of for its GPT-3 API.

The rising curiosity in LLMs and the variety of options are two components which are placing stress on API service suppliers to cut back their revenue margins to guard and develop their complete addressable market.

{Hardware} advances

One of many causes that OpenAI and different corporations determined to offer API entry to LLMs is the technical challenges of coaching and operating the fashions, which many organizations can’t deal with. Whereas smaller machine studying fashions can run on a single GPU, LLMs require dozens and even tons of of GPUs. 

Other than enormous {hardware} prices, managing LLMs requires expertise in difficult distributed and parallel computing. Engineers should break up the mannequin into a number of elements and distribute it throughout a number of GPUs, which can then run the computations in parallel and in sequences. It is a course of that’s susceptible to failure and requires ad-hoc options for several types of fashions.

However with LLMs changing into commercially enticing, there may be rising incentive to create specialised {hardware} for big neural networks.

OpenAI’s pricing web page states the corporate has made progress in making the fashions run extra effectively. Beforehand, OpenAI and Microsoft had collaborated to create a supercomputer for large neural networks. The brand new announcement from OpenAI means that the analysis lab and Microsoft have managed to make additional progress in growing higher AI {hardware} and decreasing the prices of operating LLMs at scale.

Once more, OpenAI faces competitors right here. An instance is Cerebras, which has created a huge AI processor that may practice and run LLMs with billions of parameters at a fraction of the prices and with out the technical difficulties of GPU clusters. 

Different massive tech corporations are additionally bettering their AI {hardware}. Google launched the fourth technology of its TPU chips final 12 months and its TPU v4 pods this 12 months. Amazon has additionally launched particular AI chips, and Fb is growing its personal AI {hardware}. It wouldn’t be stunning to see the opposite tech giants use their {hardware} powers to attempt to safe a share of the LLM market.

High-quality-tuned LLMs stay off limits — for now 

The attention-grabbing element in OpenAI’s new pricing mannequin is that it’ll not apply to fine-tuned GPT-3 fashions. High-quality-tuning is the method of retraining a pretrained mannequin on a set of application-specific knowledge. High-quality-tuned fashions enhance the efficiency and stability of neural networks on the goal utility. High-quality-tuning additionally reduces inference prices by permitting builders to make use of shorter prompts or smaller fine-tuned fashions to match the efficiency of a bigger base mannequin on their particular utility.

For instance, if a financial institution was beforehand utilizing Davinci (the biggest GPT-3 mannequin) for its customer support chatbot, it might fine-tune the smaller Curie or Babbage fashions on company-specific knowledge. This manner, it might obtain the identical stage of efficiency at a fraction of the associated fee.

At present charges, fine-tuned fashions value double their base mannequin counterparts. After the worth change, the worth distinction will rise to 4-6x. Some have speculated that fine-tuned fashions are the place OpenAI is basically making a living with the enterprise, which is why the costs received’t change. 

One more reason is likely to be that OpenAI nonetheless doesn’t have the infrastructure to cut back the prices of fine-tuned fashions (versus base GPT-3, the place all clients use the identical mannequin, fine-tuned fashions require one GPT-3 occasion per buyer). If that’s the case, we will count on the costs of fine-tuning to drop sooner or later.

It is going to be attention-grabbing to see what different instructions the LLM market will take sooner or later.