YZOS Roadmap

Full specification: 00-product-spec.md.

Shipped

• Single-bundle runtime: yzos-desktop links yzos-runtime, PostgreSQL, and the MCP/HTTP endpoint in one process.
• Screen capture: ScreenCaptureKit segments indexed against activity sessions.
• Entity engine: candidate discovery, cleanup, and maintenance jobs for people, projects, and tasks.
• OKF crystallization: knowledge and entity documents with real content, not placeholder templates.

In Progress

• OCR and VLM: Apple Vision OCR on keyframes, VLM-generated screen semantics for regions with no extractable text.
• AI tool session import: Claude Code, Codex, Cursor, Continue, and Aider session history.
• Permission coverage: full TCC grant flow for screen recording, accessibility, and microphone.

Planned: UI State Engine

The next major capability upgrade moves behavior recognition from screenshot guessing to structured state diffing:

• Active window and AX tree snapshots stored per observation
• Layout region detection (editor, terminal, sidebar, input, canvas)
• ROI OCR cropped to detected regions, with bounding boxes and confidence
• Behavior events (edit, run_command, send_message, and more) derived from state diffs, each carrying its evidence

Planned: App Adapters

Deeper, source-specific structure for the applications people use most:

• Browser: URL, DOM, accessibility tree, tab context
• Terminal: command buffer, prompt, working directory, output diff
• Cursor / VS Code: workspace, file path, editor selection
• Figma: canvas object, file, and page metadata
• Chat apps: sender, thread, input, message diff

Planned: Beyond Desktop

• Voice: activity detection, transcription, speaker separation
• Chat integration: broader chat app coverage
• Multi-device sync: user-owned sync targets (iCloud, WebDAV, SSH, S3, Git)
• Review UI: full entity and activity review surface