Why build it
The reason correction has to stay subtle is pedagogical, not just a UX preference.
The teaching insight
When someone teaches us something, we don't always need the answer shipped into us. We want to think it through ourselves, while someone who's watching guides us there. Hand over the answer too early and the guiding stops being teaching — it becomes dictation.
Teaching is Clicky's strongest use case. Teaching means guidance. Guidance means correction — the model has to be able to say "not quite" when the user's screen doesn't match what the step called for. And correction that ships the answer defeats the teaching it was supposed to be doing.
Finding the dots
Two things about the existing system shaped everything that follows. Neither was written down anywhere — they came from reading how reasoning and state already worked.
The two-layer reasoning
It splits into two layers, and they happen in order.
Clarification vs guidance as taskless vs task-full
Clarification and guidance aren't the same shape of reply.
Connecting the dots
Four decisions came out of the two discoveries above — each follows from one of them directly.
Automation as the branch unchanged
The whole system got read before anything got added, and automation was already well built — so nothing changed there. Adding correction meant scoping it to where guidance was actually thin.
Correction goes subtle
The teaching insight becomes concrete here: correction runs through two channels, so it never has to interrupt what's being said.
Recovery keeps agency
A failed step offers two paths: redo it yourself, or ask into the bar to see what you missed. Correction never takes over the task.
Try it in the demo at the top of the page — click the wrong thing on purpose, then ⌘Z.
Shared attention as its own mechanism
Clicky's eyes don't just express state — they redirect to its own annotation on screen, so user and
AI are looking at the same thing mid-conversation. Where to look is its own mechanism, separate
from what the gestures express — see the pointer in the demo above and the
[POINT:…] tag under Step Tracker's system design.
Step Tracker
When someone asks Clicky to walk them through a task, each step gets tracked as a dot in the notch — one per atomic action, updating on its own as Claude watches the screen for evidence the step actually happened.
The position channel from Connecting the dots — and where recovery keeps agency: a failed step never gets fixed for you, it waits for you to redo it or ask.
A checklist that fills itself in
The stepper is deliberately quiet by default: a line of small dots connected by capsules, sitting just under the notch's status text, easy to check with a glance and easy to ignore otherwise.
response 0.35 / damping 0.8) showing the current step's title in accent blue, and one
line of body copy: encouragement while pending, the specific mistake in red if the last check failed,
or a small "party popper" banner across all-emerald once every step is done. The card is opt-in —
most people leave it collapsed and just watch the dots turn green.
| State | Dot | Connector into it | Detail card |
|---|---|---|---|
| Pending, not yet active | dim ring | grey | — |
| Pending, active | accent ring + core | grey | "Follow the guidance on screen to complete this step." |
| Success | solid emerald + check | emerald | — |
| Failed | solid red + ✕ | grey (unpassed) | the specific mistake, in red, ≤15 words |
| All steps complete | all solid emerald | all emerald | party-popper banner, "ALL STEPS COMPLETED!" |
A text protocol
Claude's reply is plain speech with small bracket tags appended, which the client parses out before the text is ever spoken aloud.
[STEPS: Step 1 | Step 2 | Step 3]. A regex in
parseAndApplyStepperTags(from:) pulls the titles out, and each becomes a
TaskStep with status: .pending.
[STEP_CHECK:success], [STEP_CHECK:fail:reason], or nothing at all (silently
pending). Success advances activeStepIndex; if that was the last step, the whole array
gets checked for completion. Every tag — [POINT:…], [STEPS:…], and
[STEP_CHECK:…] — is stripped from the text before it reaches text-to-speech, including
partially-streamed tags cut off mid-token, so nothing is ever read aloud by accident.
"perfect! now right-click in the empty space. " + [STEP_CHECK:success]
"no, you need to click inside the Xcode window first. " + [STEP_CHECK:fail:You clicked the desktop instead of Xcode]
startParallelStepVerificationLoop() runs
quietly in the background so the user never has to confirm each click by hand: while
voiceState == .idle and steps remain, it waits 2 seconds,
captures every connected screen as JPEG, and sends a narrow prompt to the same Claude
model — output only one of the three tags, no reasoning — with a strict rule that "not done
yet" must resolve to pending, never fail; failure is reserved for the user
visibly doing the wrong thing.
[STEPS:…][STEP_CHECK:…] → same parser updates the dot in place, no user action
required
Try the walkthrough
A working copy of the real component and its state machine — using the actual Xcode asset-import example from Clicky's system prompt. Step through it by hand, or let it run the way the background verification loop would.
Gestures
Clicky's only face is two small eyes in a camera notch, carrying five kinetic states — idle, nod, shake, thinking, and celebrate — built from one continuously-blended motion system.
The state channel from Connecting the dots — and where shared attention lives: the eyes don't just react, they redirect to what Clicky is pointing at.
Five states, one vocabulary
Each gesture is short, legible at a glance, and never blocks — a gesture can be interrupted by the next one at any point, which matters because Clicky's reactions often need to overlap real speech.
| Gesture | Reads as | Motion | Duration | Fires on |
|---|---|---|---|---|
| Idle | attentive, ambient | Eyes follow the cursor when it's nearby; otherwise drift on small random saccades | continuous | default state; resumed after every other gesture |
| Yes / Nod | agreement, confirmation | Pupils oscillate vertically, squint toward a horizontal line | 2.1s | reply opens with "yes / yeah / correct…"; a step is marked done |
| No / Shake | correction, disagreement | Pupils oscillate horizontally, whole eye group drifts with them | 1.8s | reply opens with "no / nope / sorry…"; a step is marked failed |
| Thinking | processing, please wait | Eyes lift and scan slowly left–right, looping | until resolved | voice state enters .processing |
| Yay / Burst | celebration | Forced happy-squint, small vertical bounce, particle burst | 1.8s | reply opens with "awesome / perfect / congrats…"; all steps complete |
Targets, weights, and a lerp
Nothing about a gesture snaps — every gesture is a target weight the render loop chases, frame by frame.
triggerGesture(type) zeroes every target and sets
one to 1.0 instantly. But the values actually driving the drawing —
weights.idle / .yes / .no / .think / .yay — only move a fraction of the remaining
distance toward their targets each frame:
shake = sin(t * 14) * 8 // horizontal pupil oscillation
think = sin(t * 3.5) * 5.5, look-up fixed at -9 // slow scan, lifted gaze
yay = -abs(sin(t * 12) * 5) // eye-group bounce, forces happy-squint
pos = maxOffset * sin(angle) / sin(0.8) // spherical projection → pupil position
squash = pupilRadius * cos(angle) // foreshortening → pupil width/height
Trigger a gesture
The actual weight/target/lerp system from the prototype, running against a live canvas. Move your cursor near the eyes for the idle behavior, or fire a gesture below — the bars on the right show the blend happening in real time.
I want in!!!
👋 Hi, I'm Narayan — my work has always been about thinking and building for the interaction layer between humans and AI, and what Farza and the team @ Clicky are building is so close to what I want to be working towards. This is a 5-day prototype I built as my pitch. Really hoping to work with the you guys!
let's talk ↗