- Last updated October 12, 2024
- In AI News
OpenAI has also released MLE-bench, a benchmark designed to assess how effectively AI agents can perform machine learning engineering tasks.

As the AI world is shifting towards building agentic systems, OpenAI’s team has decided to give the world an open source gift. The team has released Swarm, an open source framework for building, orchestrating and deploying multi-agent systems.
Click here to check out the GitHub repository.
Managed by the OpenAI Solutions team, the framework is still in its experimental phase and is currently not intended for production.
Swarm is designed to make agent coordination and execution lightweight, highly controllable, and easy to test. It achieves this using two core abstractions: Agents and handoffs. An Agent holds instructions and tools and can transfer control to another Agent at any time.
These simple yet powerful abstractions enable rich interactions between tools and networks of agents, allowing for scalable, real-world solutions without a steep learning curve.
Swarm Agents are not the same as Assistants in the Assistants API. Though similarly named for convenience, they are entirely different. Swarm operates with the Chat Completions API and remains stateless between calls.
While the Assistants API offers a fully-hosted solution with built-in memory management, Swarm is perfect for developers seeking full transparency and granular control over context, steps, and tool usage. Running almost entirely on the client, Swarm—like the Chat Completions API—does not retain state between calls.
Swarm is lightweight, scalable, and highly customisable, ideal for scenarios with many independent capabilities and instructions that don’t fit into a single prompt.
Along with this, OpenAI also released MLE-bench, a benchmark designed to assess how effectively AI agents can perform machine learning engineering tasks. To create this, the team compiled 75 ML engineering-related competitions from Kaggle, offering a wide range of challenges that test key skills like model training, dataset preparation, and experiment execution.
OpenAI evaluated several language models on this benchmark using open-source agent scaffolds. Notably, the best-performing configuration — OpenAI’s o1-preview with AIDE scaffolding — achieved the level of a Kaggle bronze medal in 16.9% of the competitions.
Mohit Pandey
Mohit dives deep into the AI world to bring out information in simple, explainable, and sometimes funny words.
Subscribe to The Belamy: Our Weekly Newsletter
Biggest AI stories, delivered to your inbox every week.
Rising 2025 | DE&I in Tech & AI Summit
Mar 20 and 21, 2025 | 📍 J N Tata Auditorium, Bengaluru
Data Engineering Summit 2024
May 30 and 31, 2024 | 📍 Bangalore, India
February 5 – 7, 2025 | Nimhans Convention Center, Bangalore
MachineCon GCC Summit 2024
June 28 2024 | 📍Bangalore, India
September 25-27, 2024 | 📍Bangalore, India
Nov 21-22 2024 | 📍Santa Clara Convention Center, California, USA
Our Discord Community for AI Ecosystem, In collaboration with NVIDIA.
AIM publishes every day, and we believe in quality over quantity, honesty over spin. We offer a wide variety of branding and targeting options to make it easy for you to propagate your brand.
AIM Brand Solutions, a marketing division within AIM, specializes in creating diverse content such as documentaries, public artworks, podcasts, videos, articles, and more to effectively tell compelling stories.
ADaSci Corporate training program on Generative AI provides a unique opportunity to empower, retain and advance your talent
With MachineHack you can not only find qualified developers with hiring challenges but can also engage the developer community and your internal workforce by hosting hackathons.
Conduct Customized Online Assessments on our Powerful Cloud-based Platform, Secured with Best-in-class Proctoring
AIM Research produces a series of annual reports on AI & Data Science covering every aspect of the industry. Request Customised Reports & AIM Surveys for a study on topics of your interest.
© Analytics India Magazine Pvt Ltd & AIM Media House LLC 2024