garak

garak.ai · LLM Evals

Open source

#5 in LLM Evals

NVIDIA's open-source (Apache 2.0) LLM vulnerability scanner that probes models and endpoints for jailbreaks, prompt injection, data leakage and toxic output.

#red-teaming#security#cli#open-source

Visit garak ↗GitHub ↗

About the LLM Evals category

Frameworks and platforms for testing language models and LLM applications — from academic benchmark harnesses to CI-friendly unit tests, LLM-as-judge scoring and red-teaming kits.

garak is one of 19 llm evals tools indexed on TrueIQ. Facts on this page come from the tool's official site and public APIs; prices and features change, so confirm details on the official website before you commit.

// related

Related tools

All LLM Evals →

PyRITmicrosoft.github.ioMicrosoft's open-source (MIT) Python Risk Identification Tool for red teaming generative AI, with composable targets, converters, scorers and multi-turn attacks.LLM EvalsOpen source

DeepTeamtrydeepteam.comOpen-source (Apache 2.0) LLM red teaming framework from Confident AI, built on DeepEval, covering 50+ vulnerability types with OWASP, NIST and MITRE presets.LLM EvalsOpen source

Promptfoopromptfoo.devOpen-source CLI for testing prompts, models and RAG apps with assertions, side-by-side comparisons and automated red teaming. Part of OpenAI since 2026 and still MIT licensed.LLM EvalsOpen source★ 25.7k

Inspect AIinspect.aisi.org.ukMIT-licensed framework from the UK AI Security Institute for rigorous LLM and agent evaluations, with tool use, sandboxed tasks and 200+ prebuilt evals.LLM EvalsOpen source★ 2.9k

Giskardgiskard.aiOpen-source Python library for evaluating and red teaming AI agents and RAG apps. Version 3 is a modular rewrite for multi-turn testing; v2 is no longer maintained.LLM EvalsOpen source★ 5.9k

DeepEvaldeepeval.comPytest-style framework for unit-testing LLM apps with metrics for hallucination, relevance, RAG quality and more.LLM EvalsOpen source★ 18.6k