The week in brief
Behind it, Promptfoo (+314) and Opik (+219) followed.
Four new tools joined the directory this week: VIALS-AA, Terminal-Bench, GDPx and BenchLeader.
On Hacker News, “Show HN: Interactive notes on RLHF, attention, LoRA and LLM evals” drew the most attention in this space (41 points, 4 comments). Among listed tools, Artificial Analysis's site was linked from 89 Hacker News stories over the past 90 days, more than any other tool here.
In research, “LongBanana: An Expert-Verified Benchmark for Long Context Multi Reference Image Synthesis” collected the most upvotes on Hugging Face Daily Papers this week (23). Among new GitHub repositories created in the past 7 days, sshah03/recalc leads with 9 stars.
Biggest GitHub star gainers
GitHub star counts for the 40 open source tools we track, compared between Oct 31, 2026 and Nov 9, 2026 (217 hours apart).
| Tool | Stars before | Stars after | Gain | Change |
|---|---|---|---|---|
| Langfuse | 36,785 | 37,185 | +400 | 1.09% |
| Promptfoo | 26,667 | 26,981 | +314 | 1.18% |
| Opik | 23,059 | 23,278 | +219 | 0.95% |
| Aider Leaderboards | 49,844 | 50,030 | +186 | 0.37% |
| DeepEval | 19,212 | 19,381 | +169 | 0.88% |
| garak | 9,810 | 9,962 | +152 | 1.55% |
| Arize Phoenix | 12,079 | 12,211 | +132 | 1.09% |
| lm-evaluation-harness | 14,351 | 14,431 | +80 | 0.56% |
New listings this week
| Tool | Category | Listed |
|---|---|---|
| VIALS-AA | Benchmarks | Nov 08, 2026 |
| Terminal-Bench | Benchmarks | Nov 05, 2026 |
| GDPx | Leaderboards | Nov 02, 2026 |
| BenchLeader | Leaderboards | Nov 02, 2026 |
Most discussed on Hacker News
The highest scoring Hacker News stories in this space during the week.
| Story | Points | Comments | Discussion |
|---|---|---|---|
| Show HN: Interactive notes on RLHF, attention, LoRA and LLM evals | 41 | 4 | thread |
| Benchfmt.rs: Bringing Go's benchmarking and benchstat workflow to Rust | 31 | 8 | thread |
Listed tools with the most Hacker News links
Hacker News stories linking to each tool's own website over the past 90 days.
Notable new papers
The most upvoted papers on Hugging Face Daily Papers and the newest relevant arXiv preprints from this week.
- LongBanana: An Expert-Verified Benchmark for Long Context Multi Reference Image Synthesis
- VisSearch-Bench: Evaluating Long Horizon Visual Search in Multimodal Agents
- Terminal style Mathematical Modeling Bench: The First Agentic and Verifiable Benchmark for Mathematical Modeling
- Frontier3DBench: Benchmarking Coding Agents for 3D Creation across Objects, Scenes, and Interactive Open Worlds
- Reducing Representational Interference in Retrieval Embeddings with Trainable Prefix Tokens
- Compaction, decay and reinforcement in a two tier conversational memory: a paired evaluation against retrieval augmented generation
- Knowing When to Intervene: Causal Evaluation of AI Agents
- Are These Programmers Ripe Enough to Eat? Evaluating How LLMs Handle Lexical Knowledge Conflicts in Translation
- One Run Is Not Enough: A Monte Carlo Protocol for Evaluating LLM Reliability on Data Science Tasks
- MemoryDojo: Benchmarking Poisoning Through LLM Agent Memory
New GitHub repos in LLM evaluation
Repositories created in the 7 days before Nov 09, 2026, ranked by stars.
| Repository | Stars | Language | Description |
|---|---|---|---|
| sshah03/recalc | 9 | Rust | A spreadsheet you share with your AI agents: every cell can hold a value, a formula, a terminal command, or a Claude Code or Codex agent. |
| teo-nex/TeoBench | 8 | Python | TeoBench: reproducible reasoning benchmarks, Easy baseline and community research statistics |
| atxatlarge-code/localbench-smb | 5 | Python | The practical benchmark for running private, $0/month local AI across 9 everyday business workflows (email, writing, docs, voice, chat, spreadsheets, and automation). |
| alexcarterhq/wspbench | 4 | Python | Controlled weight space programming evaluation with CPU replay, fixed neural networks, and audited LLM pilots. |
| ohs-foundation/labs-ohs-agent | 3 | Python | Measures what it costs AI coding agents to build an offline first FHIR app, with and without the Open Health Stack libraries and agent skills. Harness, prompts, skills, run transcripts and scores incl |
| lilt/tau-bench-lilt | 3 | Python | τ-bench-LILT: multilingual extension of τ-bench for evaluating tool using customer service agents |
| jackulau/coding.sh | 3 | TypeScript | Open source, local first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost. |
| Accenture/ERPBench | 2 | Python | Benchmark for computer use agents on enterprise ERP software |
How we compile this report
Every figure comes from a public source: star counts from the GitHub API, story scores and links from Hacker News (via Algolia), papers from arXiv and Hugging Face Daily Papers, and new repositories from GitHub search. Hacker News link counts include stories whose link points to the tool's own domain. Listing dates are the day a tool joined TrueIQ. Numbers are shown as recorded; if a figure is unavailable, it is left out rather than estimated.