Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

The best open-source LLM gateways and observability tools in 2026

The best open-source LLM gateways and observability tools right now are Langfuse, Anthropic Sandbox Runtime (srt) and Grafana, out of 36 we have reviewed. Each is ranked by our rating, with its price, our verdict and who it suits.

By ReviewRamble editors · Updated Nov 18, 2026 · How we review

How we ranked: by our rating out of 5, from reviews built on each product's own pricing, documentation and policies plus independent benchmarks where they exist, each checked on the date shown in the review. We take no affiliate commission.

  1. 1

    Framework-neutral LLM tracing you can own outright: the free self-hosted build covers nearly everything, and the cloud is priced down to the unit.

    Best for: Teams that want framework-neutral tracing and evaluation they can self-host for free or buy as a reasonably priced cloud service.

    Price: Free and open source under MIT, except ee/ folders under a commercial licence. Cloud: Hobby free (50k units/month); Core $29/month; Pro $199/month (Teams add-on +$300); Enterprise $2,499/month; overage $8 per 100k units, falling to $6 above 50M.

    Read our Langfuse review
    4.6/5
  2. 2

    Your coding agent can be talked into anything; the operating system can't, and srt puts the operating system in charge.

    Best for: Every developer running a coding agent on their own machine, as the first layer of protection.

    Price: Free and open source under Apache 2.0.

    Read our Anthropic Sandbox Runtime (srt) review
    4.4/5
  3. 3

    Grafana is the safe default for seeing what your LLM app is doing in production, as long as you accept that its smartest AI features live in Grafana Cloud rather than the open-source build.

    Best for: Teams that already run Grafana, or want one place for infrastructure and LLM metrics, and can live with AGPL or pay for Grafana Cloud.

    Price: Free and open source under AGPL-3.0. Grafana Cloud: Free at $0; Pro $19/month plus usage; Enterprise from $25,000/year. Grafana Assistant: 3 active AI users free, then from $20 per active AI user on Pro.

    Read our Grafana review
    4.4/5
  4. 4

    Still the default place to record ML experiments, now with LLM tracing and evaluation thrown in for free; lock the server down before anyone else can reach it.

    Best for: Python teams that train models and build LLM features and want one free, self-hosted system of record for both.

    Price: Free and open source under Apache-2.0; you pay only for the server, database and object storage. Managed MLflow on Databricks has no separate list price and is billed through Databricks compute; Free Edition is personal use only.

    Read our MLflow review
    4.4/5
  5. 5

    The standard place to collect, clean and route agent traces once more than one service or backend is involved, and an extra moving part before then.

    Best for: Agent teams with several services or more than one telemetry backend who want LLM traces they can redact, sample and reroute without changing code.

    Price: Free and open source under Apache 2.0, with no paid edition from the project. Costs are the machines it runs on and the backends it sends to; commercial distributions come from observability vendors.

    Read our OpenTelemetry Collector review
    4.4/5
  6. 6

    The standard open scanner for model-level weaknesses: run it before launch and after every model swap, but do not mistake it for a test of your agent's tools and permissions.

    Best for: Teams that want a repeatable, model-level vulnerability scan of an LLM endpoint before launch and after every model change.

    Price: Free and open source under Apache 2.0. You pay only for the model calls a scan makes.

    Read our garak review
    4.3/5
  7. 7

    Keycloak is the sensible self-hosted answer to who an agent is acting for, provided you treat its newest MCP and agent features as previews and never ship dev mode.

    Best for: Platform and security teams that self-host identity and want MCP servers and AI agents to act with each user's own permissions.

    Price: Free and open source under Apache-2.0. The project sells no paid tier; you pay for hosting, a database and the people who run it.

    Read our Keycloak review
    4.3/5
  8. 8

    The default self-hosted gateway for teams that need one API, per-team budgets and fallbacks across many providers, provided they pin versions and treat it as security-critical infrastructure.

    Best for: Platform teams that must give many developers governed access to many model providers from their own infrastructure.

    Price: Free and open source under MIT, except the enterprise/ folder, which is under a separate commercial licence. Enterprise is custom-priced with a free 30-day trial key.

    Read our LiteLLM review
    4.3/5
  9. 9

    Worth it if you want LLM traces sitting next to the product analytics you already need, but treat it as a cloud service, because the self-hosted build is unsupported and leaves out the AI features.

    Best for: Product teams shipping an AI feature who want model traces, cost and user behaviour in one cloud tool, and are content not to self-host.

    Price: MIT outside ee/; self-hosted hobby deploy free and unsupported. Cloud free monthly: 1M events, 5K recordings, 100K AI observability events, 500 PostHog AI credits. AI observability then $0.00035 per event, falling with volume. Boost $250/month, Scale $750/month, Enterprise $2,000/month.

    Read our PostHog review
    4.3/5
  10. 10

    Trivy remains the free scanner most AI teams should put in front of their images and IaC, as long as they pin it by digest and remember that in March 2026 the scanner itself was the attack.

    Best for: Platform and ML engineers who want one free check on every image, dependency file and IaC change that ships an AI service.

    Price: Free and open source under Apache-2.0. Aqua Security sells a commercial platform built on Trivy, priced on request; the trivy-mcp plugin is free under MIT.

    Read our Trivy review
    4.3/5
  11. 11

    The gateway to run when agents, tools and models all need one policy layer, especially on Kubernetes, as long as you keep up with a monthly release train and its MCP authorisation fixes.

    Best for: Platform teams on Kubernetes that must put authentication, authorisation and audit in front of MCP tool servers and agents, not only model APIs.

    Price: Free and open source under Apache 2.0. Solo Enterprise for agentgateway (hardened build, extra MCP governance, 24x7 support) is quoted on request; no price is published.

    Read our agentgateway review
    4.2/5
  12. 12

    Phoenix has the strongest evaluation loop of the open tracing tools, wrapped in a licence and an owner that legal teams will want to discuss.

    Best for: Engineers and data scientists who want standards-based tracing with a strong evaluation loop, starting on a laptop.

    Price: Free to self-host under the Elastic License 2.0 (source-available; no hosting it as a service for others). Managed Arize AX: Free (25k spans/month, 15-day retention, 1 GB); Pro $50/month (50k spans, 30 days, 10 GB); Enterprise custom.

    Read our Arize Phoenix review
    4.2/5
  13. 13

    PyRIT automates the slow, escalating conversations that break chatbots, provided someone on the team will write the Python that drives it.

    Best for: Security engineers who need to red-team chatbots and agents in depth, including multi-turn escalation and web front ends.

    Price: Free and open source under MIT. You pay for the model calls used by attacks and LLM-based scorers.

    Read our PyRIT review
    4.2/5
  14. 14

    The watchdog to put next to a self-hosted model server: quick to run, free, and it will page you when Ollama dies, but it measures availability, not model quality.

    Best for: Self-hosters and small teams running Ollama, Open WebUI or other AI services who want an alert the moment an endpoint or model stops responding.

    Price: Free and open source under MIT. No official hosted or paid tier; you run it yourself.

    Read our Uptime Kuma review
    4.2/5
  15. 15

    A heavyweight agent sandbox built for agents you leave running unattended, not ones you sit and watch.

    Best for: Teams running agents unattended or on untrusted code who need per-host and per-method network rules.

    Price: Free and open source under Apache 2.0.

    Read our NVIDIA OpenShell review
    4.1/5
  16. 16

    Instrument LLM calls once in standard OpenTelemetry and keep your choice of backend, after first changing two defaults that send full prompts to Traceloop's cloud.

    Best for: Teams standardised on OpenTelemetry who want LLM traces in their existing observability stack without a vendor SDK.

    Price: Free and open source under Apache 2.0. Traceloop's hosted platform, now owned by ServiceNow: Free up to 50,000 spans a month with 24-hour retention; Enterprise (SOC 2, on-prem) by quote.

    Read our OpenLLMetry review
    4.1/5
  17. 17

    A self-hosted answer to hosted code sandboxes, worth the Kubernetes work when you need thousands of short-lived agent environments on your own machines.

    Best for: Platform teams running agent evaluations, RL rollouts or multi-tenant code execution who want sandboxes on their own infrastructure.

    Price: Free and open source under Apache 2.0; no paid edition from the maintainers.

    Read our OpenSandbox review
    4.1/5
  18. 18

    Opik gives away the whole server under Apache 2.0 and sells the cheapest hosted plan, but the self-hosted build has no user accounts and phones home by default.

    Best for: Teams that want a permissively licensed, self-hostable tracing and evaluation platform, or a low-cost hosted one.

    Price: Free and open source under Apache 2.0. Cloud: Free (25k spans/month, 60-day retention); Pro $19/month (100k spans, then $5 per 100k; 400-day retention add-on $29 per 100k spans); Enterprise custom.

    Read our Opik review
    4.1/5
  19. 19

    Shannon turns a code review into proven exploits for the price of a model bill, which makes it worth running on every staging release, and never anywhere you lack written permission to attack.

    Best for: Engineering teams with their own web app, its source and a staging copy who want exploit-backed security tests on every release without paying for a pentest each time.

    Price: Shannon is free under AGPL-3.0; you pay your own model costs. Keygraph Pro is $50 per active developer per month, free for qualifying US nonprofits and pre-Series-A start-ups with up to 20 developers; Enterprise is custom-priced.

    Read our Shannon review
    4.1/5
  20. 20

    The skill scanner to put in front of your installs, provided you remember that its LLM stage sends the skill's files to build.nvidia.com unless you change the provider or pass --no-llm.

    Best for: Teams that install or publish agent skills and want a static gate in CI, with an optional model review, without signing up to a security vendor.

    Price: Free and open source under Apache 2.0. LLM analysis runs on your own provider account (build.nvidia.com by default) or a local model through Ollama.

    Read our SkillSpector review
    4.1/5
  21. 21

    Scanners tell you whether an MCP server looks dangerous; ToolHive limits what it can do if it is.

    Best for: Platform and security teams that need to run, contain and audit MCP servers across a company.

    Price: Open source under Apache 2.0; Stacklok Enterprise adds features and support, priced on request.

    Read our ToolHive review
    4.1/5
  22. 22

    NeMo Guardrails is the broadest open policy layer for agents, and it charges for that breadth in configuration, latency and a changelog full of breaking changes.

    Best for: Teams building agents or customer-facing assistants that need a configurable policy layer around every model and tool call.

    Price: Free and open source under Apache 2.0. NVIDIA's NemoGuard models and NIM microservices are distributed separately.

    Read our NVIDIA NeMo Guardrails review
    4.0/5
  23. 23

    A fast, Apache-licensed gateway with budgets in the free build, let down by a security default that left its admin API open until September 2026.

    Best for: Teams running high request volumes, or Go shops, that want an Apache-licensed gateway with budgets and are prepared to lock down its admin API.

    Price: Free and open source under Apache 2.0. Enterprise is custom-priced, with a 14-day free trial; the price is not published.

    Read our Bifrost review
    3.9/5
  24. 24

    The broadest catalogue of agent and application vulnerabilities in an open red-teaming library, but every verdict rests on an LLM judge you will need to check.

    Best for: Python teams building agents who want OWASP-mapped red-team coverage they can run locally.

    Price: Free and open source under Apache 2.0. Confident AI's platform runs from a free tier to $200 and $2,000 a month; its red-teaming module is an Enterprise add-on at custom pricing.

    Read our DeepTeam review
    3.9/5
  25. 25

    AgentShield is the quickest way to find out how much your Claude Code setup trusts by accident, and its findings are a checklist to work through rather than a verdict to frame.

    Best for: Developers and platform teams who run Claude Code with hooks and MCP servers and want a repeatable audit of that configuration in CI.

    Price: CLI and GitHub Action free under MIT. The --opus analysis bills your Anthropic API key. ECC Tools Pro GitHub App: $19/seat/month.

    Read our AgentShield review
    3.8/5
  26. 26

    The paper behind it makes the best public case for layered agent guardrails, but the package itself has not had a release since May 2025 and its strongest check depends on a large hosted model.

    Best for: Teams building tool-using agents who want a layered, self-hostable guard and can maintain an unpolished framework.

    Price: Free under the MIT licence. PromptGuard 2 and CodeShield run locally; AlignmentCheck calls a hosted model through Together AI by default, billed by Together.

    Read our LlamaFirewall review
    3.8/5
  27. 27

    OpenConnector is a credible self-hosted answer to Composio, with credentials kept behind a gateway, but its licence has just tightened and OAuth providers still mean registering your own apps.

    Best for: Teams building agent products who want a self-hosted credential boundary and a large connector catalogue without adopting a closed integration platform.

    Price: Self-hosting is free under the OpenConnector License (modified Apache 2.0). OOMOL hosted: 20,000 free App tool calls a month, then 0.0003 credits per call; the price of a credit is not published.

    Read our OpenConnector review
    3.8/5
  28. 28

    A sensible first check on the MCP servers and skills your agents trust, provided you accept sending their descriptions to Snyk and scan untrusted configs in a sandbox.

    Best for: Teams that let developers or agents install MCP servers and skills and want a supply-chain check before trusting them.

    Price: Free and open source under Apache 2.0; needs a Snyk account and API token. Fleet-wide background scanning reports into Snyk Evo, priced on request.

    Read our Snyk Agent Scan review
    3.8/5
  29. 29

    An ambitious open router for choosing models per request, suited to teams already running vLLM who can absorb a fast-moving project.

    Best for: Teams serving several models, often on their own vLLM fleet, with enough varied traffic to justify choosing a model per request.

    Price: Free and open source under Apache 2.0. No hosted paid service.

    Read our vLLM Semantic Router review
    3.8/5
  30. 30

    Worth putting in front of your skills repo, as long as you believe its own numbers: it catches the obvious, not everything.

    Best for: Teams running an internal skills repository who want a scan gate before anything reaches developers.

    Price: Free under Apache 2.0; optional LLM and cloud analysers bill through their own providers.

    Read our Cisco Skill Scanner review
    3.7/5
  31. 31

    A neat, MIT-licensed router that has stood still since Palo Alto Networks bought it; fine to run today, hard to build a roadmap on.

    Best for: Teams on edge runtimes, or enterprises already buying Palo Alto security, who want a light routing layer.

    Price: Free and open source under MIT. Hosted plans: Developer free (10,000 recorded logs a month, 3-day log retention); Production $49/month (100,000 logs, then $9 per 100,000); Enterprise custom.

    Read our Portkey AI Gateway review
    3.7/5
  32. 32

    A tidy answer to copying the same MCP config and API keys into four clients, still young enough that you should expect a new build most days.

    Best for: Developers who use several MCP clients on one machine and are tired of keeping their configs and keys in sync.

    Price: Free and open source under GPL-3.0; no paid plans published.

    Read our McpMux review
    3.6/5
  33. 33

    Worth turning on if your models already live in Weights & Biases; at $0.10 a megabyte, a hard sell for anyone starting fresh.

    Best for: Teams already using Weights & Biases for model training who want LLM traces and evaluations in the same place.

    Price: SDK free under Apache 2.0. Hosted: Free (1 GB Weave ingestion/month); Pro from $60/month (1.5 GB, then $0.10/MB); Enterprise custom ($0.20/MB overage listed), with dedicated deployment.

    Read our W&B Weave review
    3.6/5
  34. 34

    A capable model router hiding behind a misleading name, and one that ships listening on your whole network with authentication switched off.

    Best for: Hobbyists and students who want to run Claude Code or Codex on free and local models.

    Price: Free and open source under AGPL-3.0. Model costs depend on the providers you connect; many have free tiers with rate limits.

    Read our Free Claude Code review
    3.4/5
  35. 35

    OmniRoute is the most capable way to squeeze free model quota into a coding agent, and also the easiest way to get a provider account suspended.

    Best for: Solo developers and hobbyists who want a coding agent to run on free and subscription quota with automatic fallback, and accept the account risk.

    Price: Free and open source (MIT), self-hosted. You pay, or not, each provider you connect; the optional Radar catalog is for supporters.

    Read our OmniRoute review
    3.4/5
  36. 36

    A sound idea left without an owner: the Apache 2.0 code still runs, but nobody is merging fixes and the hosted pieces are gone.

    Best for: Existing users with deterministic validators who can pin versions and patch it themselves.

    Price: Free and open source under Apache 2.0. The hosted Hub registry and free remote inference closed on 25 August 2026.

    Read our Guardrails AI review
    3.2/5

Sources

The main official source behind each ranked review; every review lists its full set.

More guides