Notes from MIT: What OpenAI's Head of ChatGPT Engineering Says Comes Next

I am writing this from MIT Media Lab, where today marks the opening of EmTech AI 2026 — MIT Technology Review's flagship applied AI conference. This year's theme is "The Great Integration": AI moving out of the experimental lane and into the core of how organizations actually work. Among the roughly 400 senior executives, technologists, and researchers in the room today, one conversation stood out. Sulman Choudhry — Head of Engineering for ChatGPT at OpenAI — sat down with Mat Honan, Editor in Chief of MIT Technology Review, for a session called "The Rise of the AI Platforms." These are my notes, and what I'm still thinking about.

"Everything constrained" — not just compute

One of the most honest things Choudhry said on stage, as I heard it, was that compute is still scarce at OpenAI — and that constraint shapes every decision. He described a company operating in a "compute restrained world," where resource allocation isn't a procurement problem you solve once but an ongoing tension woven into engineering culture.

That framing resonated with something he said in a May 2025 interview with Gergely Orosz at The Pragmatic Engineer, reflecting on the ChatGPT Images launch in March 2025. That launch added 100 million new users in the first week and generated 700 million images — numbers he described as "the craziest of his entire career," from someone who previously scaled Facebook Video to 5 billion daily views in 2014. In the aftermath, the constraint picture shifted dramatically: "A year ago ChatGPT was heavily GPU constrained — now it's everything constrained" — compute, storage, bandwidth, talent.

The word "everything" is doing a lot of work there. It's not a complaint. It's a description of what hypergrowth actually feels like from the inside: every system becoming a potential bottleneck simultaneously, and engineering culture having to be designed around that reality rather than around stability. On stage today, Choudhry made clear that even at OpenAI's current scale, this hasn't changed. Compute remains the scarce resource. The question is always: what are you choosing to do with it?

From chat to agents doing economically valuable work

The strategic pivot Choudhry described on stage was a narrowing of focus — not away from ChatGPT broadly, but toward a more specific definition of what ChatGPT is for. As I heard it, the organization is moving from treating ChatGPT as a general-purpose conversational tool toward targeting what he called "knowledge worker agents" doing economically valuable work.

That phrase is worth sitting with. Not "tasks." Not "productivity." Economically valuable work. It signals that OpenAI is measuring success less by engagement and more by whether the tool is doing something that matters in someone's professional life — something that would otherwise cost real time or real money.

He described both the ChatGPT consumer product and OpenAI's developer platforms as extending beyond their initial use cases into other knowledge worker contexts. The developer platform bet, in particular, seemed intentional from early on. As he framed it on stage, OpenAI bet early on developers — and the generation currently entering the workforce, already AI-familiar from their school years, is what makes that bet compound.

What he described at the engineering level echoes what he shared in a March 2026 interview with the Engineering Leadership Community: teams are getting smaller and more unified, with product, design, and data science functions increasingly collapsed into engineering. The Directly Responsible Individual (DRI) model means one person owns the whole project — not separate design, product, and engineering DRIs. That structure is not incidental. It's what allows a small team to move fast enough to build agents that actually do something.

Kepler: "a data scientist in your pocket"

One of the most concrete things Choudhry introduced on stage was Kepler — OpenAI's internal data agent, which he described, as I heard it, as "a data scientist in your pocket." You prompt it with what you're trying to evaluate, and it queries across data sources and synthesizes the response.

OpenAI has since published the details, and they are worth spelling out. According to OpenAI's official writeup on Kepler: it was built by just 2 engineers in 3 months, with 70% of the code written by AI (via Codex). It now serves roughly 4,000 of OpenAI's ~5,000 employees daily. It reasons over 600+ petabytes across ~70,000 datasets. It operates through Slack, web, and IDE integrations. And it saves users 2 to 4 hours per query.

Powered by GPT-5 (specifically GPT-5.2) and led by Emma Tang (OpenAI Head of Infrastructure) with authors Bonnie Xu and Aravind Suresh, Kepler represents a specific design philosophy: don't build for abstraction, build for the actual bottleneck. VentureBeat's deep dive on Kepler frames it as a template — not just an internal tool.

What I find most interesting is what OpenAI says about replication. They explicitly state that anyone can replicate Kepler — the models, APIs, and orchestration are all publicly available. They have no plans to release it as a product. The invitation, as I read it, is to build your own version for your own organization.

For the nonprofits and small organizations I work with through Our Community Tech, that's a meaningful signal. Two engineers. Three months. Publicly available building blocks. The gap between "OpenAI can do this" and "we could do something like this" is closing faster than most leaders realize. The barrier isn't the technology anymore. It's the clarity to know what bottleneck you're actually trying to solve.

The dream is tight closed-loop systems so agents can run 24 hours a day.

— Sulman Choudhry, as I heard it on stage at EmTech AI 2026

The students who are already AI-familiar

One observation Choudhry made on stage — that the current generation entering the workforce is "AI familiar in school" — landed differently for me than it might for most people in that room. I heard it as a strategic observation about OpenAI's developer bet: invest early, and the people who grow up with the tools become the builders and multipliers. But I also heard it from three other seats I occupy simultaneously.

As an instructor at SMU and UTD: "AI familiar" is not uniform among my students — some have built with these tools extensively, some have only encountered them as consumers. That gap produces meaningfully different outcomes in how they frame problems, scope projects, and evaluate their own work. The institutions that acknowledge it explicitly will produce different graduates than the ones that pretend it isn't there.

As a K-12 tech leader at Uplift Education: There's a version of AI familiarity that is purely consumer — you've used ChatGPT to help with homework. And a version that is more generative — you've had teachers who modeled how to evaluate AI outputs, question them, and build on them deliberately. Those are not the same thing, and we are not producing the second version consistently or at scale. Building that infrastructure is the actual work.

As the parent of an elementary schooler: My child will graduate high school in a world where agents that do economically valuable knowledge work are ordinary — not futuristic. The habits she's building right now around curiosity, questioning, and evaluating information are either preparing her for that world or they aren't. I can't outsource that judgment to a curriculum.

"Token boards and AI slop"

Choudhry made a point on stage — which I'm paraphrasing from my notes — that token boards aren't efficient and tend toward what he called AI slop. My interpretation: surface-level metrics that reward volume of AI output rather than quality of insight. Organizations that measure AI adoption by how many prompts were run, how many tokens were consumed, or how many documents were generated end up optimizing for activity rather than value. The output looks productive. The underlying work often isn't.

I've seen this pattern directly in my work with nonprofits. Organizations get pitched AI "solutions" — dashboards, chatbots, automated summaries — that generate a lot of visible output without solving the actual problem. The tool is producing tokens. Nobody's asking whether those tokens are moving the mission forward.

Choudhry's framing of OpenAI's own engineering culture is almost an antidote to that trap. In the Engineering Leadership Community interview, he described a culture where structure follows demonstrated potential — Codex started as three to five parallel teams and consolidated only after real signals emerged — and where the DRI model forces accountability to outcomes, not activity. The engineers on a project aren't just shipping code; they're owning the result. That's a very different relationship to metrics than counting tokens.

He also described, as I heard it on stage, two specific ways OpenAI accelerates without losing quality: cleverly selecting which CI/CD tests to run out of thousands (rather than running all of them every time), and Kepler itself — which concentrates the data synthesis work into a tight loop rather than distributing it across dozens of ad hoc analyses. Neither of those is a token-maximizing approach. Both are insight-maximizing approaches.

What this means for the rest of us

The through-line of Choudhry's conversation with Mat Honan was a company that has moved past whether AI agents can do real work and into the harder question of how you design an organization around that reality. The tension he named on stage was real: how do you maintain a bottom-up innovation culture while being disciplined about limited compute? That's not a startup problem. It applies to organizations of every size.

For educators, nonprofit leaders, and small-org technologists: the agents are already here. Kepler is a 2-engineer, 3-month project built on publicly available tools. The capability curve is not waiting for enterprise budgets. The question is no longer whether AI is coming to your workflow. It is whether you are the one who gets to design what that looks like.

That design work requires being honest about what the actual bottleneck is — not just what looks impressive in a demo. It requires building for insight, not for output. And it requires teaching people to use these tools in ways that strengthen their judgment rather than replace it. Not evangelism. Stewardship.