The Oracle That Wears Us

A visitor approaches a portrait screen in a darkened room. They press a button. Over four turns of directed conversation, the system listens, not for what they say but for how they say it: the hesitation before a word, the flatness or fracture in a vowel, the acoustic texture of carrying something. At the end, a CNC plotter draws a symbolic mark onto a shared cloth. A thermal printer produces a receipt. The encounter ends; the mark remains.

The Oracle That Wears Us is an interactive AI installation structured as a ritual encounter. It asks what it feels like to be interpreted by a system that cannot explain itself.

The work began as a technical prototype in February 2026: a TouchDesigner pipeline connecting speech recognition, language generation, and voice synthesis. I led the AI pipeline, the state machine architecture, the visual system, the UX research, and the prompt design. Andrey Dyakov built the physical installation: a reverse-engineered NEJE CNC plotter, a Firebase pipeline, and a Flutter web archive. Dmitrii Kazantsev designed the oracle figure and the ambient generative sound layer.

The conceptual problem that held all of it together: AI systems are interpretive, not neutral. They don't reflect a visitor back to themselves; they translate the visitor through their own constraints, training data, and architectural assumptions. Oracle makes that translation visible. Every output is a reading, not a recording. Every mark is the system's interpretation, not the visitor's truth.

The full local pipeline: Voice Activity Detection identifies when the visitor stops speaking → Vosk transcribes the audio → Gemma3:4b (running locally via Ollama) generates the oracle's response and extracts a named object from the visitor's words → KittenTTS synthesises the oracle's voice → a custom ONNX emotion regression model trained on the RAVDESS dataset analyses acoustic features and classifies the emotional texture of how the visitor spoke → that classification maps to one of eight symbolic categories drawn from the historical Hobo sign lexicon → the symbol is rendered as SVG with geometric parameters unique to this session → the CNC plotter draws it → a thermal receipt prints automatically via ESP32.

The single external API call is FLUX img2img on Replicate, which transforms the visitor's captured portrait into the oracle's aesthetic register. The prompt that works: face preservation as the primary constraint. The transformation is only meaningful if the visitor can still recognise themselves in it.

The interaction runs through nine designed states (IDLE through RESET) without menus, navigation, or choices. The visitor does not control the progression. The oracle does.

Early testing revealed something that shifted the whole frame: visitors were not asking who authored the output. They were asking whether the system understood them. That question, am I being seen?, displaced the design problem entirely. The work stopped being about authorship and started being about recognition, pacing, and the legibility of machine perception.

Four rounds of usability testing, twelve design iterations. Three findings that shaped everything:

The entry was unclear. A physical button resolved this: one deliberate action, one moment of consent. But even after pressing the button, visitors were trying to orient their body in space, understand the premise, and process the opening provocation simultaneously. The AWAKENING state was split into two timed beats: first the body is positioned, then, only once the visitor is settled, the ritual begins.

Latency was being read as failure. The local AI pipeline carries 2–2.5 seconds of irreducible delay per turn. Hiding that delay wasn't possible. Making it legible was. The PROCESSING state reframes waiting as deliberation. The arc spinner (eight Hobo-derived marks cycling slowly) signals that the system is thinking, not broken.

The listening needed to be visible without being instructional. A ring surrounding the oracle figure responds directly to live microphone RMS: it swells when the visitor speaks, contracts in silence. The system shows it is listening. No text label states it.

Each conversational turn increases the opacity of a FLUX-transformed portrait of the visitor, overlaid on the oracle figure. The progression functions simultaneously as emotional escalation, progress indicator, and narrative accumulation. Turn 0: the oracle is fully present. Turn 4: the visitor occupies the oracle's place. No loading bar. No numbered steps. Only the slow convergence of two figures.

The oracle does not possess the visitor. The visitor possesses the oracle.

turn 0

turn 1

turn 2

turn 4

The Hobo sign system: a pre-digital symbolic communication network used by itinerant workers across early twentieth-century America. Marks scratched onto fence posts and barn doors, information passed between people with no shared verbal language, moving constantly, reading the world through material traces.

The ONNX emotion model maps acoustic features into eight emotional categories, each assigned to one Hobo mark: The Thorns (anger), The Sky Eye (calm), The Shriek (surprise), The Pitiful Story (sadness), The Kind Soul (happy), The Watcher (fear), The Bitter Root (disgust), The Horizon (neutral). The model outputs geometric parameters, not a fixed template, so two visitors receiving the same emotional category still receive materially different marks. The machine never reproduces a symbol identically.

The plotter draws each visitor's mark onto a shared cloth. No visitor controls the final composition. The work accumulates every interpretation, every encounter, every session across the installation's run. By May 2026 it held 165 marks.

The cloth is simultaneously a collective memory object, an emotional archive, an accumulated record of machine interpretation, and a garment. It does not belong to any single participant once the first mark is inscribed.

The thermal receipt was not anticipated as significant. It became unexpectedly important during testing. Participants repeatedly held it, reread it, photographed it, asked questions about it afterward. People who had been uncertain during the encounter engaged with it deeply once it was in their hands.

The receipt tells the visitor exactly what the machine perceived: acoustic measurements, emotional classification, thematic keywords, the oracle's Turn 4 statement, a QR code linking to a session-specific page in the web archive. Whether the system's perception was accurate is a different question. One the machine cannot answer.

Oracle operates in two visual registers. The installation screen: void black, cream text, gold mark, luminous, symbolic. Everything outside it (receipt, web archive, documentation) uses cream, charcoal, and IBM Plex Mono. Gold crosses both. On void it reads as divine. On cream it reads as aged, material, printed. The same hue; entirely different weight.

The oracle's reading did not need to be correct. It needed to feel considered.