Last updated: July 24, 2026
Signals that operators need
Product observability differs from model eval dashboards. Start with identity, spend, error rates, and inventory—then deepen into traces.
- Requests attributed to workspace, agent, and API key.
- Credit pool burn-down and exhaustion events.
- Success/error rates for /v1 agent runs and completions.
- Agent inventory drift (unused or orphaned agents).
- Optional GatewayLogs in Log Analytics when APIM diagnostics are on.
Kimss-native observability
Execution logs and ledgers support chargeback and incident response for SDK-integrated agents.
See /docs/product_integrations for execution log concepts and /ai-spend-caps-and-credits for finance-facing units. Pair Kimss views with Azure Cost Management for infrastructure truth—credits govern product behavior; Azure still bills Foundry consumption.
Gateway and compliance telemetry
When APIM diagnostic settings stream to Log Analytics, gateway logs can support immutable audit discussions—after E2E verification.
Do not enable production gateway-only routing until chat and agent list flows succeed in your test slot. Prefer KQL over Log Analytics for investigations when that path is active.
Related reading: /ai-compliance and /ai-gateway.
Operating rhythm
Monthly reviews of keys, agents, and ledger entries prevent silent drift.
Remove unused credentials, archive obsolete agents, and adjust group budgets after major launches. Track product changes at /changelog.
Frequently asked questions
Does Kimss store full prompt bodies for observability?
Operational focus is metadata suitable for billing and audit. See /docs/trust_safety for retention posture.
How do we debug a single failed run?
Use workspace attribution, API error responses, and Azure diagnostics for the Foundry project—plus gateway logs if enabled.
Is App Insights required?
Not specifically for Kimss Credits. Your Azure estate may still use App Insights for application traces.
Where is architecture documented?
/docs/architecture.