OpenAI's early May 2024 ChatGPT update was enormous, bringing heaps of new features to the world's leading generative AI chatbot. Most of the improvements center around enhancing user interaction, new functionalities, and improving overall performance, and there are heaps of reasons why you should give them a try.

Image Input and Output Capabilities

ChatGPT is now capable of taking images as input and generating images as output! The new image input capability allows users to upload images directly into their chat interactions with ChatGPT. This feature expands the ways users can seek help and share information, making it possible to discuss visual content seamlessly.

In this example, I asked ChatGPT for recipes for the items visible in the fridge.

It gave me some pretty accurate results in identifying items in the fridge and giving recipes I could make using the ingredients identified.

Aside from taking images as input, ChatGPT can also generate DALL-E images within the interaction.

I asked ChatGPT what MakeUseOf would look like if it were a robot.

As you can see, ChatGPT provides some promising results, providing an image plus a description of the image. However, it still seems like you'll need to use third-party solutions to save the image in JPG or PNG, since there isn't a way to save the image beside WebP.

ChatGPT Plus users were immediately given access to the new features using OpenAI's new ChatGPT-4o (Omnimodal) model. Free-tier users will also have access to these features but with limited usage.

Expanded Knowledge Base and Cross-Chat Memory

ChatGPT's upgraded web browsing capabilities allow it to retrieve up-to-date information from the web with better accuracy. It ensures responses are based on the latest available data and insights, making it particularly valuable for quickly evolving topics like news, technology, and market trends.

In addition to web browsing capabilities, ChatGPT's memory feature is now generally available. The memory feature allows ChatGPT to retain details about your preferences, past interactions, and ongoing projects, providing a more personalized and coherent user experience. This means that ChatGPT can remember your favorite topics, tailor its responses to suit your needs and keep track of long-term projects or ongoing conversations.

I asked ChatGPT to check the Saturday weather forecast and advise whether I should reschedule my activities based on the weather update.

ChatGPT provided sound advice by browsing the web for the weather and reviewing its memory of my schedule. It even gave me a list of activities I could do instead based on my past interactions with ChatGPT.

The browsing capability feature is now available for both Plus and Free tier users, while ChatGPT Memory is only available to Plus users.

Multi-Modal Voice and Video Interactions

Everyone wanted to see more modality in ChatGPT. The new ChatGPT-4o model gives access to multi-modal voice and video interaction, allowing users to engage in continuous voice and video conversations with ChatGPT, like making a video call with a human assistant. ChatGPT-4o's conversation support, video capability, and human-like voice and emotions make this new feature different from other voice assistants.

In the video, ChatGPT talked with the hosts, showing varying levels of emotions. Furthermore, the hosts were also able to interrupt ChatGPT mid-conversation, allowing for real-time conversations with little delay.

Pairing this feature with ChatGPT's browsing and memory functionality allows for a highly versatile AI companion. This enhanced version of ChatGPT can now better assist you with shopping, planning, and organizing. GPT-4o's conversation functionality makes it possible to use ChatGPT as a hands-free assistant using voice commands, making it easier to interact with the AI while you're on the go or multitasking.

GPT-4o's voice interaction is available to all users using the ChatGPT Android/iOS app. However, the video interaction feature is unavailable at the time of writing but is expected to roll out in the near future.

Upgraded Code Analysis

The previous ChatGPT Code Interpreter is now called Advanced Data Analysis (ADA). Since it is already out of beta and ChatGPT plugins are no longer available, ADA is now an integrated feature within both ChatGPT-4 and ChatGPT-4o, and both Free and Plus tier users now have this feature. No plugin or activation is needed; simply prompt ChatGPT to run, analyze, or write code, and it should do the rest for you.

Here, I’ve given ChatGPT-4o an SQL file containing a database from an e-commerce project and asked it to analyze and graph the top products commonly ordered.

I already knew the numbers before running it using ADA on ChatGPT-4o, so I know the provided graph is accurate. However, running it on ChatGPT-4 continually gave me errors, was slow, and never really provided me with the graph. So, although ADA is not functionally different from the old code interpreter, running ADA right on ChatGPT-4o was certainly easier, faster, and smarter.

OpenAI's demonstrations also showed live code debugging using voice commands, which is another enormous upgrade for ChatGPT. Instead of having to type commands and work bit by bit, live interactions with your code could speed up the process.

Temporary Chat

The new Temporary Chat feature in ChatGPT-4o is ideal for privacy-conscious users and those seeking quick interactions with ChatGPT. This feature enables users to engage in a temporary chat with ChatGPT without the conversation being saved in history or used by OpenAI to train their models.

Temporary Chat is available to both Plus and Free tier users. You can access it through the model drop-down menu by selecting the Temporary Chat option. Once activated, you can now enjoy a private and non-persistent chat experience.

The latest updates to ChatGPT introduce a host of features that cater to a wide range of user needs, from enhanced interactivity and data analysis to real-time information access and personalized experiences. Whether you're a professional, student, or casual user, these features offer powerful new ways to leverage ChatGPT for improved productivity, learning, and everyday convenience.