YZOS Feature Overview
Dieser Inhalt ist noch nicht in deiner Sprache verfügbar.
Full specification: 00-product-spec.md.
1. Feature Groups
Section titled “1. Feature Groups”- Observe: desktop activity, UI state, screen segments, AI tool sessions, clipboard, chat, voice
- Crystallize: triage, entity signal extraction, OKF writes, knowledge/memory
- Index: PostgreSQL, full-text search, pgvector, OKF reconciliation
- Supply: MCP HTTP, HTTP API, CLI, Skill
- Maintain: scheduled jobs, entity cleanup, reindexing, rollups
- Review UI: activity, entities, settings, permissions, models
2. Desktop Activity Observation
Section titled “2. Desktop Activity Observation”| Item | Detail |
|---|---|
| Input | app switches, window focus, URL, screen segments, UI state |
| Structure sources | NSWorkspace, CGWindowList, AXUIElement, browser/app adapters |
| Visual sources | ScreenCaptureKit mp4 segments, ROI OCR, VLM labels |
| Output | activity sessions, UI snapshots, behavior events |
Desktop observation is built primarily on UI structure and layout regions. Full-screen screenshots are only an evidence layer.
3. AI Tool Sessions
Section titled “3. AI Tool Sessions”YZOS imports coding-agent session history from local tool data, including Claude Code, Codex, Cursor, Continue, Aider, and xAI Grok Code. Each imported session is normalized into ai_tool_sessions and linked to the relevant project, task, and activity session.
4. UI State Engine
Section titled “4. UI State Engine”- Active window tracking: bundle id, PID, title, bounds
- AX tree snapshot: role, label, value, focus, selection, bounds
- Layout region detection: editor, terminal, sidebar, input, canvas, modal
- ROI OCR: OCR cropped to a detected region, with bounding boxes preserved
- Temporal diff: comparison between consecutive UI states
- Behavior events: edit, run_command, send_message, and related event types
5. OCR / VLM Visual Gap-Filling
Section titled “5. OCR / VLM Visual Gap-Filling”- Apple Vision OCR: system OCR producing text boxes
- OCR postprocessing: bounding-box ordering, low-confidence filtering, chrome/edge filtering
- Screen semantics: VLM-generated short descriptions for regions with no OCR text or heavy visual content
- Scene router: decides whether a region needs a VLM pass
- Keyframe extraction: pulls representative frames from mp4 segments
6. Clipboard
Section titled “6. Clipboard”| Item | Detail |
|---|---|
| Input | text clipboard changes |
| Processing | PII redaction, deduplication, timeline |
| Output | recent entries, search |
7. Entity Engine
Section titled “7. Entity Engine”| Entity | Automatic signals |
|---|---|
| Person | sender, name, account, OCR, voice |
| Project | repo path, AI session, window title |
| Task | triage, AI session, behavior event |
The entity engine supports candidate discovery, matching, merging, archiving, OKF writes, and PostgreSQL indexing.
8. Supply
Section titled “8. Supply”- MCP HTTP
/mcp - HTTP
/api/call - CLI
yzos - Skill
yzos-data - Frontend hooks for the review UI
9. Maintain
Section titled “9. Maintain”| Job | Purpose |
|---|---|
| entity_promote | promote candidates to confirmed entities |
| entity_archive | archive noisy or stale entities |
| entity_reconcile | keep OKF and PostgreSQL consistent |
| entity_rollup | refresh entity summaries |
| entity_merge | suggest merges for duplicate entities |
| reindex | rebuild search indexes |