Why Custom GPTs Don't Quite Work yet
In January 2024, OpenAI unveiled the custom GPT store, a long-awaited milestone that promised to bring the landscape of AI-powered capabilities into its 'App Store' era.
In a few short days, thousands, if not millions, of custom GPTs have flooded the store, each claiming to be a specialized version of the formidable GPT4, engineered to excel at specific tasks.
It's clear that specialized AI assistants will be supremely helpful in the future.
Why is there a strange feeling that the Custom GPT store isn't going to take off?
This should be the biggest, most innovative new marketplace launch in tech history. But it's mostly crap.
Several key issues cast doubt on the practicality and effectiveness of custom GPTs.
The Essence of Custom GPTs
Custom GPT creation represents a departure from the traditional programming languages and even no-code website builders. Rather than crafting code in the conventional sense, these AI models are predominantly driven by interaction through prompts and conversational exchanges.
It's an approach that blurs the line between coding and conversation.
The Problem with Custom GPTs
The iOS App Store vs. GPT: Infinite Possibilities or Finite Excellence?
We're familiar with the iOS App Store, a curated collection of apps that serve countless purposes. It's like a well-stocked store with a vast but finite inventory, offering a range of choices to cater to your needs.
iOS itself is already doing a lot, and it has several dozen supremely useful applications ("it's a phone, it's an iPod, it's an web browser...).
It's very clear how the next ten-thousand specialized apps on the App Store improve the iPhone.
Let's turn our attention to GPT4. Unlike iOS, GPT4 is already more like an infinite universe of knowledge and capabilities.
In this context, the question that emerges is whether we truly need the concept of an "app store" for GPT. Does it make sense to create specialized GPTs when the foundational model itself is infinitely adaptable? The iOS App Store serves as an extension to a finite platform, providing a curated set of tools. In contrast, GPT offers a limitless playground of knowledge and possibilities.
So, are we introducing an App Store to a universe where it might be seen as less of an incremental value and more of an attempt to tame the boundless potential of GPT?
An "app store" for GPTs is possibly like trying to structure infinity — and also susceptible to whatever logarithmic improvements GPT5 brings to what the core model can do on its own.
2. The Untapped Potential of Actions and Proprietary Data
There are limits to what the supreme AI model can do. It can't literally know data from private, proprietary databases, no matter how much it will scrape the Internet and ingest everything it can.
The true potential of custom GPTs is unlocked when they harness unique actions and proprietary data.
However, a substantial proportion (almost all) of these custom GPTs falls short of realizing this potential. The value proposition of custom AI models lies in their capacity to perform tasks or provide insights that are distinct from generic GPTs. Therefore, the failure to leverage actions and proprietary data hinders the realization of their full capabilities.
Custom GPTs that succeed will need to have some kind of data advantage, not just a clever personality/prompt-based advantage.
Ideally, the proprietary API that custom GPTs call — the actual secret sauce — may combine and synthesize multiple sources of data to give back a
3. The Pitfall of Laziness in Configuration
A common misconception surrounding custom GPTs is the belief that they operate like magic, requiring minimal effort in terms of configuration and fine-tuning.
The interface itself encourages you to just type a few sentences on what you want your GPT to do and hit Publish. It's deceptively easy to get a chatty custom GPT up and running in three minutes.
In reality, these AI models demand meticulous setups to deliver reliable and valuable results.
Developers and users must invest time and resources into configuring custom GPTs to meet their specific needs, paying attention to suboptimal, probabilistic answers.
The inclination towards expecting instant gratification from these models without adequate configuration will lead to disappointment.
4. The Challenge of Unpredictable Outputs
Unlike traditional computer code, AI models, including custom GPTs, can occasionally produce probabilistic, unpredictable outputs.
When a conventional deterministic program behaves unexpectedly, developers can typically identify and rectify the underlying bug.
However, with AI models, the approach to problem-solving is different. Instead of pinpointing and fixing bugs, custom GPT builders may find themselves resorting to nudging and prompt engineering, akin to managing a well-trained but occasionally wayward pet. This unpredictability poses a unique challenge that developers and users must contend with when building and debugging custom GPTs.
Debugging undesired outputs now adds some mysterious art to the logical science of software engineering.
5. The Evolution Beyond Text
The internet has undergone a remarkable evolution, transcending the boundaries of mere text. It now encompasses a rich tapestry of multimedia elements, including apps, widgets, videos, images, carousels, and galleries.
If this article was optimized for SEO, instead of just something I decided to write on my blog on Sunday morning, it should have a series of graphs, screenshots, and pictures to optimize its search engine scores.
Engaging with the digital realm has become a hypervisual, hyper-video-based experience, replete with 4K/60FPS images and videos, and interactive content. Booking travel, for instance, has transitioned from a predominantly textual affair to a captivating journey through a visual landscape. In this context, the limitations of engaging in mostly textual conversations with AI models become apparent.
Human psychology is inherently drawn to visual stimuli, and this evolution beyond text challenges the relevance of custom GPTs in delivering engaging and satisfying user experiences.