Agent Builder, ChatKit, and the App SDK I Actually Ship With

By Muhammad Kashif · Draft · AI automation & cybersecurity builder, Pakistan

Eighteen minutes into ChatKit Studio I had a Mood DJ agent routing three sub-agents into a Spotify playlist widget. The clock was cute. The useful shock was different: structured outputs made the widget honest, and the Apps SDK — sitting on MCP — meant the same connector pattern I already use could surface inside ChatGPT as something a non-engineer can tap. That demo rewired how I brief multi-agent work for clients who still think AI means a text box.

AgentKit is a graph, not a chat

AgentKit bundles Agent Builder (drag-and-drop), ChatKit, evals, and connectors. The pitch is blunt — rival Zapier and n8n for agent workflows, not for simple webhooks. Apps inside ChatGPT ride the Apps SDK. That matters because control flow finally has a canvas, and the canvas can talk to tools I already trust.

I sketch in the builder, then I re-implement the risky paths where I need git history and auth I own. Visual graphs are excellent for stakeholder demos. They are a liability when production logic lives only in a proprietary canvas with no pull request. My rule: if a mutate cannot be reviewed as a diff, it does not leave staging.

Mood becomes lead intent on client work

The demo classifies a mood, fans out to three specialists, and returns playlists inside a ChatKit widget. Structured outputs keep the payload typed; the widget simply refuses to render garbage JSON. That is a better schema lesson than another slide deck about JSON Schema.

On client work I rename the pieces: mood → lead intent, Spotify → CRM or calendar, widget → confirmation card the human must approve before any write-back. Sub-agents stay narrow. The graph owns routing. I own side effects. I also keep a failure widget — empty result, auth error, rate limit — so operators see a card instead of a silent blank that looks like success.

What the Codex SDK numbers actually buy me

Field notes said Cisco cut code-review time roughly in half with the Codex SDK; Instacart wired it into their loop. I do not paste those quotes into proposals. I ask whether review latency is my bottleneck. Two weeks of measuring my own PR wait times answered that better than any keynote.

Codex earns a seat where diffs are large and tests exist. Agent Builder earns a seat for orchestration that mixes tools a coding agent should not own — ads, playlists, CRM mutations. Mixing those worlds in one agent usually produces a clever demo that fails a security review before lunch.

OAuth is the real product risk

Any agent on the Apps SDK can exfiltrate if scopes are lazy. My brief always lists: read scopes first, write scopes behind a human gate, secrets never in graph nodes. If Mood DJ can hit Spotify, it can hit anything OAuth grants. I scope tokens to playlist read and a disposable test user until evals pass.

Production gets a separate OAuth app, rotated secrets, and redirect URIs I typed myself. Every tool call gets a correlation id from ChatKit event to backend. When something weird ships, I want a trace — not a vibes postmortem written from memory.

Evals before the stakeholder demo

Golden-set evals for the happy path and three failure paths. Staging connectors only. Confirmation card on every mutate. Activity log exported nightly. If I cannot explain the graph in a one-page threat model, it stays in the sandbox.

I also score the widget UX: does a tired operator understand what they are approving? If the card is vague, they will click through. Vague approval is worse than no approval.

The pricing conversation I force

Eighteen-minute demos are allowed. Eighteen-minute production is not. When a stakeholder asks to "just ship the Mood DJ pattern to our CRM," I show them the checklist first. If they will not fund the gates, they are buying a prototype, and I price it that way. Prototype fees do not include on-call for an agent that can write to production CRM.

What I tell the first sceptical engineer

Show them the confirmation card and the correlation ids before the Mood DJ playlist. Engineers relax when they see traces. Stakeholders relax when they see the widget. I need both relaxed before anyone talks about production CRM write-back.