github.com · LLM Evals
Open source AGPL-3.0 framework from Sentient for implementing, attacking and verifying LLM fingerprinting methods in a shared evaluation environment.
mlprints is one of 20 LLM eval tools on TrueIQ. See all LLM eval tools or how we check facts.
// guides
AI model fingerprinting embeds ownership signals in LLMs. How the Fingerprinting Leaderboard works, SaTML 2026 findings, and how it differs from SynthID.
// related
garakgarak.aiNVIDIA's open source (Apache 2.0) LLM vulnerability scanner that probes models and endpoints for jailbreaks, prompt injection, data leakage and toxic output.LLM EvalsOpen source★ 9.7k
PyRITmicrosoft.github.ioMicrosoft's open source (MIT) Python Risk Identification Tool for red teaming generative AI, with composable targets, converters, scorers and multiturn attacks.LLM EvalsOpen source★ 4.7k
DeepTeamtrydeepteam.comOpen source (Apache 2.0) LLM red teaming framework from Confident AI, built on DeepEval, covering 50+ vulnerability types with OWASP, NIST and MITRE presets.LLM EvalsOpen source★ 3.1k
Inspect AIinspect.aisi.org.ukMIT licensed framework from the UK AI Security Institute for rigorous LLM and agent evaluations, with tool use, sandboxed tasks and 200+ prebuilt evals.LLM EvalsOpen source★ 3.1k
Promptfoopromptfoo.devOpen source CLI for testing prompts, models and RAG apps with assertions, side by side comparisons and automated red teaming. Part of OpenAI since 2026 and still MIT licensed.LLM EvalsOpen source★ 26.5k
Giskardgiskard.aiOpen source Python library for evaluating and red teaming AI agents and RAG apps. Version 3 is a modular rewrite for multiturn testing; v2 is no longer maintained.LLM EvalsOpen source★ 5.9k