⚡ Omp Cache Optimizer

🚀 Quick Start

• Install: omp plugin install omp-cache-optimizer.
• Reload: Run /reload in OMP to refresh extension hooks.
• Verify: Run /cache-optimizer doctor to confirm model/provider/compat status.
• Check Stats: Run /cache-optimizer stats for session-scoped hit rates after a few turns.

🔧 Configuration

• Environment Variables: Opt-out via PI_CACHE_OPTIMIZER_NO_PROMPT_REWRITE=1, PI_CACHE_OPTIMIZER_NO_SKILL_COMPRESSION=1, or PI_CACHE_OPTIMIZER_NO_OPENAI_CACHE_KEY=1.
• Models YAML (~/.omp/agent/models.yml):

  providers:
    your-provider:
      compat:
        supportsLongPromptCacheRetention: true # Only if endpoint explicitly supports OpenAI long retention

• Commands: Use /cache-optimizer fix to generate copyable YAML compat snippets. Use enable/disable for process-level toggles.

📦 Dependencies

• Designed for Oh My Pi (OMP) runtime.
• Forked from pi-cache-optimizer (adapted from JSONC to YAML, shifted state directories).

💡 Best Practices

• Prompt Stabilization: The optimizer automatically moves stable system prompt content ahead of dynamic contexts. It also compresses verbose skill XMLs to maximize static token prefix matching.
• OpenAI Cache Keys: Rely on the automatic prompt_cache_key fallback injection (using session id) for OpenAI-compatible APIs missing explicit keys.
• Router Integration: Extension authors should pass real upstream identity via assistant message metadata (provider, responseModel, usage.cacheRead/cacheWrite). This avoids importing the optimizer while gaining accurate footer stats.
• Session Affinity: Rely on OMP’s multi-credential auth and agent.db for session stickiness instead of legacy headers.

🚨 Gotchas / Anti-patterns

• Proxy Splitting: Proxies like LiteLLM/OneAPI often route identical sessions to different backend nodes. This shatters provider-side prompt caching.
• 400 Errors: If you see 400 Unsupported parameter: prompt_cache_retention, instantly remove supportsLongPromptCacheRetention: true from your models config.
• Obsolete Fields: NEVER use sendSessionAffinityHeaders, sendSessionIdHeader, or forceAdaptiveThinking. OMP natively handles session affinity and adaptive thinking detection.
• DeepSeek Formatting: Do NOT manually configure thinkingFormat: "deepseek". OMP automatically detects the reasoning format.
• Manual Fixes Required: The /cache-optimizer fix command currently lacks automatic YAML writing. You must manually copy and paste the suggested overrides.

🔍 Research / References

• Source: npm package omp-cache-optimizer (v1.0.7).
• Stats Location: ~/.omp/agent/omp-cache-optimizer-stats.json.