YZOS Vision

Full specification: 00-product-spec.md.

In One Sentence

YZOS is a local observation system. It turns the real state of your software, AI tool sessions, chats, clipboard, voice, and screen into searchable context, giving upstream AI a factual foundation to work from.

Core Stance

• Supply, not conversation: YZOS produces a dataset, not another chat surface.
• Passive, not disruptive: you never need to switch into YZOS to get work done.
• Local, not exfiltrated: raw observation data stays on your machine by default.
• Structure first: read app/window/UI structure before falling back to OCR or VLM.
• OKF as source of truth: long-lived knowledge is written to Markdown, PostgreSQL is only an index.
• Reviewable: you can see exactly what YZOS observed.

Why Not a Screenshot-OCR Product

Full-screen screenshots or video streams are only usable as low-level evidence. Inferring behavior directly from screenshot OCR mixes in menu bars, the Dock, background windows, notifications, and unrelated text, and it cannot reliably tell whether you are reading, editing, sending, copying, or running a command.

YZOS instead follows this path:

flowchart TD
  AW["Active app / window"] --> UI["UI structure"]
  UI --> REG["Layout regions"]
  REG --> OCR["ROI OCR"]
  OCR --> VLM["VLM semantic labels"]
  VLM --> DIFF["Temporal diff"]
  DIFF --> EVT["Behavior events"]

This turns behavior recognition from “guess what is on screen” into “compare structured state changes.”

Relationship With Upstream AI

Question from upstream AI	What YZOS provides
What did I do today?	activity summary, sessions, apps, events
How is this project going?	project entity, AI sessions, knowledge, tasks
What did I just copy?	recent/searched clipboard entries
Who is involved?	person entities, links, sightings
Where is this piece of knowledge?	OKF knowledge search/read

YZOS does not answer general-knowledge questions and does not reason about problems on behalf of the upstream AI. It only supplies facts observed locally and their crystallized results.

Key Objects

Object	Origin
Activity Session	timeline of app/window/clipboard/screen/voice events
AI Tool Session	logs from Claude Code, Codex, Cursor, and similar tools
UI Snapshot	structured UI state of the active window at a point in time
Behavior Event	a behavior inferred from a UI state diff
Person	discovered jointly from sender, account, name, voice, AI sessions
Project	discovered jointly from repo, path, window title, AI sessions, tasks
Task	updated jointly from behavior events, AI tool summaries, and OKF
Knowledge	reusable technical or business facts distilled from observations

Success Criteria

• Zero overhead: you never need to log your work inside YZOS.
• Traceable: every crystallized result can be traced back to its observation, session, and source.
• Correctable: entities can be merged, renamed, or archived.
• Accurate behavior: behavior events depend on UI state diffs, not single-frame screenshot guesses.
• Stable supply: MCP and CLI return real data, and return an explicit error on failure rather than fabricating a result.
• Controlled privacy: the boundaries between raw text, redacted text, index, and export are explicit.

For the current development roadmap, see 08-roadmap.md.