Realtime voice
OpenAI’s gpt-realtime-2.1-mini handles spoken interaction, turn-taking, and tutoring dialogue in the current implementation.
Student asks a question
Conversation selects a teaching action
Conversation selects a teaching action
One conversation.
Three systems.
Shared understanding.
CHALK separates conversation, lesson generation, and drawing—then connects them around the state of the visible whiteboard.
Structured reasoning
OpenAI’s gpt-5.6-luna produces bounded lesson steps with explanation text and drawing operations in the current implementation.
A lesson plan becomes structured steps
Validation checks the drawing instructions
Validation checks the drawing instructions
Deterministic drawing
Renders equations, graphs, and diagrams, and tracks what is visible.
Validated steps become ink
Board state informs the next response
Board state informs the next response
The shared context: a compact description of the committed board flows back into the tutoring conversation.
THE HARD PARTS ARE IN THE HANDOFFSBuilt around
Built around
continuity.
Keep the existing ink stable.
Adding a later lesson step should preserve the position of elements already on the board. The renderer is designed around stable, accumulating layout.
Ground in what was drawn.
Board context describes accepted, visible elements rather than the intended lesson plan. The conversation can refer to the same picture the student sees.
Validate before rendering.
Drawing instructions pass through schema, expression, reference, and budget checks. The browser also checks the expressions it draws.
Preserve the lesson through interruptions.
Interruptions pause the lesson. Request identity and cancellation controls help prevent stale work from changing the active board.
A WORKING SYSTEM, IN THE OPEN
View the repositoryExplore the implementation.
The repository documents the architecture, milestone evidence, local setup, and known limitations. It is the technical source of truth for the current prototype.
The current implementation uses OpenAI models; Claude is not integrated. The full tutor currently runs locally. The public site runs static interactive examples, with no live AI session, student accounts, or hosted voice service. Broader testing and product development remain part of the next stage.
EARLY ACCESS
Join the waitlistBuilding realtime AI? Let’s compare notes.
I welcome technical collaborators working on voice, visual interfaces, and learning systems.