Deleted Sam Altman talk minutes: Open AI also lacks GPU, cost reduction is the primary goal

Author | Lingzi County

Editor | Wei Shijie

Source丨Geek Park

Image source: Generated by Unbounded AI tool

SamAltman's European tour is still underway. Not long ago, in London, he had a closed-door discussion with the CEO of AI company HumanLooop. HumanLoop is a company that helps developers build applications on large language models.

HumanLoop CEO Raza Habib recorded the highlights of the conversation and made it public on the company's website. But then at the request of OpenAI, the minutes were withdrawn. This in turn increased the outside world's curiosity about the conversation. Some have speculated that some of OpenAI's thinking was involved in this change.

Geek Park, after browsing through the deleted minutes of the conversation, found that it not only involved the short-term planning of OpenAI in Sam's eyes, but also concealed the pressure on OpenAI after receiving strong support from Microsoft's cloud computing resources. After all, model fine-tuning and reasoning still consume a lot of computing resources. According to The Information, Open AI's model has cost Microsoft Azure $1.2 billion, concentrating computing resources on supporting OpenAI and limiting the servers available to other Microsoft departments.

In this regard, Sam said that cost reduction is the primary goal at present.

In addition, Sam also revealed that at present, services such as opening longer context windows and providing fine-tuning APIs are limited by GPU resources;

In this conversation, Sam Altman responded to many outside concerns, such as competition and commercialization:

Despite just hiring a world-class product manager, Peter Deng, OpenAI will not consider releasing more products;

The future application trend is to embed the functions of the large model into more APPs instead of growing more plug-ins on ChatGPT, because in reality most plug-ins do not show PMF (Product / Market Fit, that is, product-market fit);

In the past few years, OpenAI has expanded the model size by millions of times, but this speed is not sustainable. Next, OpenAI will continue to increase the model size at a rate of 1 to 3 times to improve model performance.

The minutes of the conversation were made public on May 29, and were deleted around June 3 according to the records of netizens. Here's what you get with the backup:

01, OpenAI is currently severely limited by the GPU

As conversations scale, the required computing resources grow exponentially

OpenAI currently has very limited GPUs, which has delayed many of their short-term plans. The biggest complaints from customers are the reliability and speed of the API. Sam acknowledged their concerns, explaining that most of the problems were due to a shortage of GPUs.

The longer 32k context can』t yet be rolled out to more people. OpenAI haven』t overcome the O(n^2) scaling of attention and so whilst it seemed plausible they would have 100k - 1M token context windows soon (this year) anything bigger would require a research breakthrough.

The longer 32K contexts are not available to more people. OpenAI has not overcome the O(n^2) scaling problem of the attention mechanism, although it looks like they will soon (this year) have a context window of 100k-1M Token. Any larger window would require research breakthroughs.

*Note: O (n^2) means that as the length of the sequence increases, the computing resources required to perform Attention calculations increase exponentially. O is used to describe the upper limit or worst case of the growth rate of the algorithm's time or space complexity; (n^2) means that the complexity is proportional to the square of the input size. *

The fine-tuning API is also currently limited by GPU availability. They haven't used efficient fine-tuning methods like Adapters or LoRa, so running and managing (the model) through fine-tuning is very computationally intensive. Better support for fine-tuning will be provided in the future. They might even host a community-based market for model contributions.

Dedicated capacity provisioning is limited by GPU availability. OpenAI offers dedicated capacity, providing customers with a private copy of the model. To get the service, customers must be willing to commit to paying $100,000 up front.

02, OpenAI's recent roadmap

2023, smart cost reduction; 2024, limited demonstration of multimodality

Sam also shared what he sees as an interim near-term roadmap for the OpenAI API.

2023:

Cheaper and faster GPT-4 — that’s their top priority. Overall, OpenAI's goal is to reduce the "cost of intelligence" as much as possible, so they will work hard to continue to reduce the cost of the API over time.

Longer context window — in the near future, the context window may be as high as 1 million tokens.

Fine-tuning API — The fine-tuning API will be extended to the latest models, but the exact form will depend on what the developers say they really want.

A stateful API - When calling the chat API today, you have to go through the same session history over and over, paying the same tokens over and over again. There will be a future version of the API that remembers session history.

2024：

Multimodal - This is being demonstrated as part of the GPT-4 release, but will not scale to everyone until more GPUs come online.

03. Commercial prediction and thinking: plug-ins "without PMF" may not appear in the API soon

Many developers are interested in making ChatGPT plugins accessible through the API, but Sam said he doesn't think those will be released anytime soon. In addition to the Brosing plugin, the use of other plugins shows that there is no PMF (Product/Market Fit). He pointed out that a lot of people think they want their app to be inside ChatGPT, but what they really want is ChatGPT inside the app.

04. Except for ChatGPT, OpenAI will avoid competing with its customers

Great companies have a killer app

Many developers said they were nervous about developing with the OpenAI API, because OpenAI may eventually release products that are competitive with them. Sam said that OpenAI will not release more products outside of ChatGPT. Historically, great platform companies have had a killer app, he said. ChatGPT will allow developers to become customers of their own products to improve the API. The vision of ChatGPT is to become a super intelligent work assistant, but there are many other GPT use cases that OpenAI will not be involved in.

05. Regulation is needed, but not now

"I'm skeptical about how many people and companies are capable of holding big models"

While Sam called for future models to be regulated, he doesn't think existing models are dangerous and that regulating or banning them would be a big mistake. He reiterated the importance of open source and said that OpenAI is considering making GPT-3 open source. They haven't been open-sourced yet, in part because he's skeptical about how many individuals and companies are capable of holding and serving large language models (LLMs).

06. The laws of scale still apply

The expansion speed of millions of times in a few years cannot continue forever.

There have been a lot of articles lately claiming that "the age of giant AI models is over". This is not accurate. (Note: At an event at MIT in April, Sam Altman said: We are now nearing the end of the era of gigantic models.)

OpenAI's internal data shows that the scaling laws of model performance still apply, and increasing model size will continue to improve performance.

Since OpenAI has scaled up models by millions of times in just a few years, this rate of scaling cannot be sustained. That doesn't mean OpenAI won't keep trying to make models bigger, but it does mean that they're likely to double or triple in size every year, rather than by many orders of magnitude.

The fact that the laws of scale are in effect has important implications for the AGI development timeline. The assumption of the law of scale is that we probably already have most of the ingredients needed to build AGI, and that the rest of the work is mainly to scale existing methods to larger models and larger datasets. If the age of scale is over, we may be even further away from AGI. The fact that the laws of size continue to apply strongly implies a shorter timeline.