So, what is GPT-4o?

What Is GPT-4o?

GPT-4o is the ChatGPT developer OpenAI's newest AI model, revealed at its early May 2024 "Spring Update" event. It will coexist with its previous top-performing model, GPT-4 Turbo, at least for now, and brings a huge number of updates to the tool.

Unlike its predecessors, GPT-4o is completely multi-modal from launch (the "o" in the model name stands for "omnimodal"). OpenAI's Spring Update event showcased GPT-4o making fluent conversation with the event hosts, chopping and changing between interactions, showcasing "personality," and illustrating how it could become the virtual assistant users have dreamed about.

It can accept combinations of audio, text, image, and video as inputs and output in text, audio, and image (no video support yet, but expect that to change once OpenAI's Sora text-to-video tool launches—at least, this is what I'm guessing will happen).

In terms of the raw numbers provided by OpenAI, GPT-4o outperforms all of its previous models, along with its nearest competitors, such as Claude 3 Opus, Gemini Pro 1.5 and Ultra 1.0, and Llama 3 400B.

Now, numbers are all very well and good, but what does that actually translate to? Well, again, working from OpenAI's numbers, GPT-4o "matches GPT-4 Turbo performance" for English writing and coding, is significantly faster in "non-English languages," and, most importantly, is faster and cheaper in terms of API use.

GPT-4o Live Capabilities Are Astounding

I've worked in tech for a long time, and I've seen a lot of shiny new "game-changers" come and go. But GPT-4o's conversational speech is truly brilliant. GPT-4o can hold proper conversations with you, even allowing you to interrupt, change the conversation focus, change topics, and more, almost without skipping a beat.

Its ability to rapidly converse gives it a whole host of new applications. While ChatGPT already had a voice function, it was limited as it first had to write a response that could then be spoken to you. You could also interact with ChatGPT using your voice, but it would take time to process your request.

Now, GPT-4o's real-time voice is near-seamless. What's more, it can express emotion and specific styles, which again were impossible before this update.

This is also applicable to live translation, in which GPT-4o showed an enormous improvement. Now, I'm not well versed in any other language, but the live translation from English to Italian and back was well received; anything that makes communication easier when you're abroad will be an enormous boon, especially given the speed of translation.

I was in Morocco recently, and even with Google Translate helping get some meaning into Arabic, the full context of the translation is never completely accurate. GPT-4o's live translation would have been incredibly useful!

Coding and Tutoring

GPT-4o also brings significant upgrades to code interpretation and assistance using its multi-modal capabilities. Similar to the other tools, yes, ChatGPT could already work with some data, but its new model drastically steps this up.

The ability to debug code using just your voice is remarkable. However, its real use will only become clear when actual programmers and developers begin using the tool. While ChatGPT's coding abilities are useful, they're only as useful as the knowledge of the user, like most generative AI tools.

When Does GPT-4o Launch? Is GPT-4o Free?

GPT-4o launched immediately to ChatGPT Plus subscribers paying the $20 monthly fee. But, in another enormous moment for generative AI, OpenAI revealed that GPT-4o would launch for all users—including free users—in due course.

There is no specific date for GPT-4o to hit free ChatGPT free accounts, but given the speed of other rollouts, it shouldn't take too long.

Other aspects of the new model are still unavailable, too. For example, I wanted to make a short clip of the new live voice feature for this article, but the feature hasn't launched yet (I'm a long-term ChatGPT Plus subscriber), nor has it found its way to any colleague's accounts.

GPT-4o will also bring a long-awaited ChatGPT desktop version, starting with macOS, but again, it hasn't launched yet.