A new LLM named GPT-2 is making the rounds in the LMSYS arena. It feels similar, if not slightly better, than other models including GPT-4, Claude, and Gemini 1.5. Some call it the latest LLM from OpenAI.
Is GPT-2 actually an older LLM from before GPT-3, only fine-tuned now to achieve remarkable results? This is the current hot mystery in the world of AI and apparently everyone is trying to solve it. It’s mostly a Tweet-storm, really. Some are also saying that it could be OpenAI testing an older model like GPT-2 with more advanced techniques or logic, something of AGI level through their work on Q*, and that’s what we’re interacting with.
The LMSYS Chat is a cool website where you can test out different LLMs for free, including the most cutting-edge ones (it has resource limiters and a slower speed). The platform has multiple modes such as direct chat and side-by-side comparisons. The most interesting mode is the Arena or the Battle Mode. The website gives you two chatbots at random. You don’t know which side is which. You give it a prompt and see both chatbots offering their responses in real time. Then, you judge them (for example, “A is better”). Once voting is done, it tells you which model was which.
In this, two new models have started springing up more recently (being only one earlier). These are “im-a-good-gpt2-chatbot” and “im-also-a-good-gpt2-chatbot,” most likely two weights or variations of a model. OpenAI had not claimed any GPT-2 release (as it’s working on GPT-5 and Q* reportedly) since users started seeing GPT-2 on LMSYS a few days ago on Twitter.
Today, however, OpenAI is confirmed to be behind this mysterious model. Multiple users got rate limit errors in the OpenAI style, the template you see when you hit the rate limiter of any ChatGPT model. See the following tweet for a quick coverage of this:
A couple days ago, Sam Altman had also published a cryptic tweet saying just “im-a-good-gpt2-chatbot,” which is the name of the LLM. So, what is GPT-2? Well, let’s just wait. Or tweet about it.
