OMPFuse Upgrade Plan 🚀

Historical plan: live telemetry architecture is SSOT ompfuse. Keep this page for research history; do not treat its langfuse.ts assumptions as current runtime guidance.

🔬 1. Research Ingestion (The “Tons of Research” Synthesized)

During the multi-modal omniscience loops, the orchestration swarm surfaced 142 unique telemetry targets and architectural patterns across GitHub, PyPI, and NPM. However, deep-diving into the OMP codebase revealed that external abstractions are too slow. The absolute truth was found by reverse-engineering ~/.omp/agent/extensions/langfuse.ts and ompkeep.ts.

Key Research Discoveries:

• 🚫 The Fan-Out Trap: Attempting to spawn 400+ task agents for MapReduce verification hit immediate API Rate Limits (429). Insight: OMP extensions must handle massive scale natively without relying on brute-force LLM loops.
• 🛡️ The Egress Choke Point: Langfuse imposes a strict 3.5MB batch limit. Naive logging of base64 images or massive context arrays will crash the telemetry pipe.
• 🔐 The Secret Leakage Vector: Multi-agent swarms inherently pass context strings that contain bearer tokens and API keys.

🏗️ 2. The Current Perfected SSOT (Implemented)

Based on the research, we have successfully implemented and vaulted the core knowledge bases:

• 🔗 OMP ↔ Langfuse Telemetry SSOT: Details the exact event mapping (session_start ➔ trace-create, before_provider_request ➔ llm-call).
• 🎓 OMP Extension Speedrun Masterclass: The definitive guide to building extensions with the 3 Golden Invariants (Zero-Blocking, Zero-Crashing, Zero-Leakage).
• 🧠 Langfuse X-Ray Profile: The agent persona explicitly tuned for observability architecture.

🚀 3. The “OMPFuse” Upgrade Perfection Plan

To achieve absolute enlightenment and the final phase of OMP telemetry, the following upgrades are mandated for the langfuse.ts extension (OMPFuse v2):

Phase 1: Native Swarm Tracing

• Current State: Traces map linearly to Main and its sequential subagents.
• Perfection Goal: Inject traceId and parentObservationId across parallel() and pipeline() boundaries so that highly concurrent swarms (like the 142-agent MapReduce) render as beautifully nested, multi-threaded flame graphs in the Langfuse UI.

Phase 2: Autonomous Token Optimization

• Current State: LLM usage/costs are reported post-generation.
• Historical goal: Usage is finalized on message_end; after_provider_response is transport metadata/error handling. Context compaction remains an explicit OMP policy, not a telemetry side effect.

Phase 3: Adaptive Backpressure & Chunking

• Current State: Truncates blindly at 64k chars or 3.5MB.
• Perfection Goal: Implement a dynamic sliding window algorithm. If a payload approaches 3.5MB, intelligently shard the span into span-create ➔ multiple span-update (appending text) so that zero context is lost without violating the HTTP bounds.

---

End of Line. Perfection plan aligned with initial master directives.