OpenAI has for the first time made its Decisions API model widely available to users, entering a fast-growing, low-cost intelligent market. This market has recently enabled TypeSafe, a startup founded by a former researcher from a leading laboratory, to become a company valued at 7.6 billion US dollars.

The Jev model of TypeSafe has pioneered a new class of deep learning models: they function like "super classifiers." Given a set of options, these models can make quick decisions and provide a confidence score for their choices. Software developers quickly realized that such decision-making models, or System One models, could replace large language models (LLM) in various workflows, running faster and at a much lower cost.

The Nikunj Handa in charge of the API product at OpenAI told TechCrunch that after seeing Jev, his colleague said, "Wow, we have about a million use cases." The team at Handa then quickly launched their own version: instead of training a specialized, calibrated decision-making model, they reconfigured ChatGPT Luna to improve inference speed and reduce costs.

Handa said, "The biggest breakthrough of Decisions API lies in speed; it has reduced processing time from 1 to 3 seconds to 100, 120, and 180 milliseconds. In the future, we might conduct specialized training to make the model perform even better in these areas. But for now, it is this ordinary Luna that actually gets the work done."

OpenAI announced this product on September 29th during Developer Day and made it available to all customers on Thursday. The company has also added new voice and visual modalities to the public version, and is working to reduce latency around the world by deploying inference resources according to geographical regions.

In preliminary tests with clients such as Box, Datadog, Ramp, and Speak, engineers replaced LLM with API to achieve higher throughput. Speak, the Chief Technology Officer of the team developing AI language tutor product, stated that Decisions API enabled the company to reduce the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used this product to improve the response speed of their own Dots voice assistant.

However, it is not yet entirely clear what constitutes a good decision-making model, beyond the criterion of "being useful to you." Optimizations regarding intelligence, cost, calibration, and speed seem to vary depending on the workload. TypeSafe The CEO Diogo Almeida does not trust benchmark tests and encourages people to try these models in their own work, but TypeSafe also released its own rankings last week, alongside the latest Jev model.

Public benchmarks such as AIM System One Index and Artificial Analysis Decision Index tend to favor the new decision-making model D1 Ultra released last week by Liquid, as well as Elephant-1 of Base Lab. The latest Jev model is still competitive on these two indices, ranking seventh and third respectively; whereas Decision Model of OpenAI ranks eighth and fourteenth respectively. However, for now, the products of TypeSafe are the most frequently queried on OpenRouter.

However, the most interesting Jev benchmark test might be JevMan – in this benchmark, the decision-making model has to play Pac-Man. The latest Jev model ranks first in this ranking. The Opper AI Chief Growth Officer who built this benchmark said that this classic game “is an interesting test because it requires the model to react quickly in constantly changing, high-risk decisions. At each crossroads, the model must decide which direction to take within a few hundred milliseconds.”

OpenAI may have an advantage in its multimodal capabilities. Handa told TechCrunch that in terms of image classification, Luna is very powerful and performs even better during calibration, and added that OpenAI customers also hope that Decisions API can be more flexible when outputting images.

Handa believes that further improvements in speed will bring about new application scenarios: "Could we achieve around 60 milliseconds? With that, you can imagine that each frame in the video would call API once."

As these products become faster and cheaper, they may validate the view of the economist who gave name to Jev: a cheap commodity will only become more ubiquitous. As Handa said, decision-making models have revealed "that large number of use cases" that were obvious but have been overlooked all along.