Orchestrate Agentic AI

Context, Checklists, and No-Miss Reviews

Calvin Hendryx-Parker, CTO

Six Feet Up

All Things AI 2026

Talk Slides

https://github.com/sixfeetup/2026_AllThingsAI_OrchestrateAgenticAI Talk Slides QR Code

What is this about, who is it for

Where we are going (today)

The root of the problem

Creating determinism

with nondeterministic systems.

quality input
constraint
dialectic
visibility

Operational work, technical or more business orient is almost alway non-deterministic.
LLMs are also non-deterministic
How can we take problems that we know do not have deterministic solutions and create meaningful value with non-deterministic tools?

If you work in risk & compliance, your job is often wading through ream of uncertainty to try to determine the best possible decision. An LLM may not provide perfect answer, but it can help you pinpoint issues and opportunities faster.

If we can narrow the frame and focus of the LLM, it can uncover important information faster, giving us the opportunity to take more time to dig deeper or respond or maybe just move on.

Anything that can remove complexity and uncertainty safely provides value. There is something similar with using LLMs, which while they can manage large of amount of data quickly, they inherently are non-deterministic.

The problem

Claude, a new document and me makes 3.

Let’s see this in action…

Tragedy of The Commons Context

Context is a Workspace, not a Warehouse.

Agents

“Agents are models using tools in a loop”
–Hannah Moran, Anthropic

The Tool For Job

a context buffer w/ LLM connection
local tools: skills, subagents, mcp, hooks
local resources: flat files, dbs, scripts, sockets, etc

Orchestration

Agent’s Orchestrated is Magic

Memory & Skills Demo

Skills, Skills, Skills

From search to assessment

Document Criteria
Start Building Beginnings of Evals

Search is great, but I have reviewed many documents like this, I have an idea about what I care about and would like to get some automated recon before I have to dig into the doc myself. I also want to delegate as much of this digging to my agents.

I want the LLM to give me feedback, but I want to be sure that feedback is useful, focused, and actionable.

To do this, we are going to use an idea from AI testing, the eval. When testing LLMs, we assume that our responses are nondeterministic, so we must create a sort of bracketing of test questions and an assessment of answers.

We will use an eval skill we have created where we give our skill a criterion file, the skill uses our memory in the dbs, and then returns answers.

demo - show eval skill - run better eval

The questions constrain and focus the agent’s return; our databases constrain the data acted upon.

Let’s Orchestrate

Ingest Documents
Output Assessments

Bonus: Auditing

Follow and Document the Agent’s Journey

TAKHOMASAK

https://github.com/sixfeetup/2026_AllThingsAI_OrchestrateAgenticAI Talk Slides QR Code

Your Future (could be) Agentic

Talk To Me

📩 calvin@sixfeetup.com
🤝 https://linkedin.com/in/calvinhp
✖️ @calvinhp
🐘 @calvinhp@fosstodon.org
🦋 @calvinhp.com