5 months ago
47
- Last updated November 6, 2024
- In AI News
This predictive tool is deemed to be ideal for scenarios where developers know the general structure of the required output.
Illustration by Nalini Nirad
OpenAI has launched the Predicted Outputs feature for developers using GPT-4o and GPT-4o-mini which is designed to improve efficiency, while also reducing the latency of responses.
This feature allows users to input a ‘prediction string’, an anticipated segment of the output which significantly reduces response times during repetitive tasks or minor document edits.
— OpenAI Developers (@OpenAIDevs) November 4, 2024OpenAI has said that since most of the output of an LLM is known before generation, predicting them in advance means generating fewer tokens. It is almost always the highest latency step when using an LLM: as a general heuristic, cutting 50% of your output tokens may cut ~50% user latency.
According to users who tested it, the typical applications of this feature include updating existing text or making small changes to code, such as renaming variables or rephrasing specific content, where the AI response can closely match the provided input.
Testing of this feature has shown it performs best when there’s a close match between the prediction and the model’s response. If the prediction string diverges from the model’s output, the tool can become less efficient, leading to slower responses and higher costs.
Read more about the feature here.
This predictive tool is deemed to be ideal for scenarios where developers know the general structure of the required output.
For example, if a user wants the model to reword a document with minor changes or adjust specific lines of code, the predictive input helps streamline responses by letting the model reuse parts of the pre existing text.
By contrast, the feature isn’t as beneficial for creating unique, original content, where responses cannot easily be anticipated in advance.
OpenAI encourages developers to experiment with the predictive text feature in controlled, predictable tasks to maximize efficiency, particularly in contexts that require frequent minor adjustments.
Tanisha Bhattacharjee
Journalist with a passion for art, technological development and travel. Discovering the dynamic world of AI, one article at a time.
8th Nov 2024
Meet 30+ top CDOs, ITDMs & AI Leaders.
Upcoming Large format Conference
India's Biggest Developers Summit
February 5 – 7, 2025 | Nimhans Convention Center, Bangalore
Download the easiest way to
stay informed
Subscribe to The Belamy: Our Weekly Newsletter
Biggest AI stories, delivered to your inbox every week.
Rising 2025 | DE&I in Tech & AI Summit
Mar 20 and 21, 2025 | 📍 J N Tata Auditorium, Bengaluru
Data Engineering Summit 2024
May 30 and 31, 2024 | 📍 Bangalore, India
February 5 – 7, 2025 | Nimhans Convention Center, Bangalore
MachineCon GCC Summit 2024
June 28 2024 | 📍Bangalore, India
September 25-27, 2024 | 📍Bangalore, India
25 July 2025 | 583 Park Avenue, New York
Our Discord Community for AI Ecosystem, In collaboration with NVIDIA.
World's Biggest Media & Analyst firm specializing in AI
AIM publishes every day, and we believe in quality over quantity, honesty over spin. We offer a wide variety of branding and targeting options to make it easy for you to propagate your brand.
AIM Brand Solutions, a marketing division within AIM, specializes in creating diverse content such as documentaries, public artworks, podcasts, videos, articles, and more to effectively tell compelling stories.
ADaSci Corporate training program on Generative AI provides a unique opportunity to empower, retain and advance your talent
With MachineHack you can not only find qualified developers with hiring challenges but can also engage the developer community and your internal workforce by hosting hackathons.
Conduct Customized Online Assessments on our Powerful Cloud-based Platform, Secured with Best-in-class Proctoring
AIM Research produces a series of annual reports on AI & Data Science covering every aspect of the industry. Request Customised Reports & AIM Surveys for a study on topics of your interest.
Immerse yourself in AI and business conferences tailored to your role, designed to elevate your performance and empower you to accomplish your organization’s vital objectives.
© Analytics India Magazine Pvt Ltd & AIM Media House LLC 2024
