The week in brief

Behind it, Promptfoo (+314) and Opik (+219) followed.

Four new tools joined the directory this week: VIALS-AA, Terminal-Bench, GDPx and BenchLeader.

On Hacker News, “Show HN: Interactive notes on RLHF, attention, LoRA and LLM evals” drew the most attention in this space (41 points, 4 comments). Among listed tools, Artificial Analysis's site was linked from 89 Hacker News stories over the past 90 days, more than any other tool here.

In research, “LongBanana: An Expert-Verified Benchmark for Long Context Multi Reference Image Synthesis” collected the most upvotes on Hugging Face Daily Papers this week (23). Among new GitHub repositories created in the past 7 days, sshah03/recalc leads with 9 stars.

Biggest GitHub star gainers

GitHub star counts for the 40 open source tools we track, compared between Oct 31, 2026 and Nov 9, 2026 (217 hours apart).

Star gains, Oct 31, 2026 to Nov 9, 2026
ToolStars beforeStars afterGainChange
Langfuse36,78537,185+4001.09%
Promptfoo26,66726,981+3141.18%
Opik23,05923,278+2190.95%
Aider Leaderboards49,84450,030+1860.37%
DeepEval19,21219,381+1690.88%
garak9,8109,962+1521.55%
Arize Phoenix12,07912,211+1321.09%
lm-evaluation-harness14,35114,431+800.56%

New listings this week

Tools added to the directory this week
ToolCategoryListed
VIALS-AABenchmarksNov 08, 2026
Terminal-BenchBenchmarksNov 05, 2026
GDPxLeaderboardsNov 02, 2026
BenchLeaderLeaderboardsNov 02, 2026

Most discussed on Hacker News

The highest scoring Hacker News stories in this space during the week.

Listed tools with the most Hacker News links

Hacker News stories linking to each tool's own website over the past 90 days.

Notable new papers

The most upvoted papers on Hugging Face Daily Papers and the newest relevant arXiv preprints from this week.

  1. LongBanana: An Expert-Verified Benchmark for Long Context Multi Reference Image SynthesisHF Daily Papers · Nov 04, 2026 · 23 upvotes
  2. VisSearch-Bench: Evaluating Long Horizon Visual Search in Multimodal AgentsHF Daily Papers · Nov 02, 2026 · 22 upvotes
  3. Terminal style Mathematical Modeling Bench: The First Agentic and Verifiable Benchmark for Mathematical ModelingHF Daily Papers · Nov 05, 2026 · 12 upvotes
  4. Frontier3DBench: Benchmarking Coding Agents for 3D Creation across Objects, Scenes, and Interactive Open WorldsHF Daily Papers · Nov 05, 2026 · 6 upvotes
  5. Reducing Representational Interference in Retrieval Embeddings with Trainable Prefix TokensHF Daily Papers · Nov 04, 2026 · 6 upvotes
  6. Compaction, decay and reinforcement in a two tier conversational memory: a paired evaluation against retrieval augmented generationarXiv · Nov 06, 2026 · Osamudiamen Edogun, Joshua Obigwe, Irfan Mehmood · cs.CL
  7. Knowing When to Intervene: Causal Evaluation of AI AgentsarXiv · Nov 06, 2026 · Ayush Sawarni, Wenqian Xing, Tijana Zrnic et al. · cs.AI
  8. Are These Programmers Ripe Enough to Eat? Evaluating How LLMs Handle Lexical Knowledge Conflicts in TranslationarXiv · Nov 06, 2026 · Dongxu Wu, Sara Court, Micha Elsner · cs.CL
  9. One Run Is Not Enough: A Monte Carlo Protocol for Evaluating LLM Reliability on Data Science TasksarXiv · Nov 06, 2026 · Wenqi Ni, Ozan Evkaya, Miguel de Carvalho · cs.LG
  10. MemoryDojo: Benchmarking Poisoning Through LLM Agent MemoryarXiv · Nov 06, 2026 · David H. Yang, Blake Bullwinkel, Amanda Minnich et al. · cs.CR

New GitHub repos in LLM evaluation

Repositories created in the 7 days before Nov 09, 2026, ranked by stars.

New repositories ranked by GitHub stars
RepositoryStarsLanguageDescription
sshah03/recalc9RustA spreadsheet you share with your AI agents: every cell can hold a value, a formula, a terminal command, or a Claude Code or Codex agent.
teo-nex/TeoBench8PythonTeoBench: reproducible reasoning benchmarks, Easy baseline and community research statistics
atxatlarge-code/localbench-smb5PythonThe practical benchmark for running private, $0/month local AI across 9 everyday business workflows (email, writing, docs, voice, chat, spreadsheets, and automation).
alexcarterhq/wspbench4PythonControlled weight space programming evaluation with CPU replay, fixed neural networks, and audited LLM pilots.
ohs-foundation/labs-ohs-agent3PythonMeasures what it costs AI coding agents to build an offline first FHIR app, with and without the Open Health Stack libraries and agent skills. Harness, prompts, skills, run transcripts and scores incl
lilt/tau-bench-lilt3Pythonτ-bench-LILT: multilingual extension of τ-bench for evaluating tool using customer service agents
jackulau/coding.sh3TypeScriptOpen source, local first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost.
Accenture/ERPBench2PythonBenchmark for computer use agents on enterprise ERP software

How we compile this report

Every figure comes from a public source: star counts from the GitHub API, story scores and links from Hacker News (via Algolia), papers from arXiv and Hugging Face Daily Papers, and new repositories from GitHub search. Hacker News link counts include stories whose link points to the tool's own domain. Listing dates are the day a tool joined TrueIQ. Numbers are shown as recorded; if a figure is unavailable, it is left out rather than estimated.