The week in brief
Behind it, Promptfoo (+335) and Opik (+210) followed.
Five new tools joined the directory this week: FrontierCode, NoveltyBench, Infinity-Chat, Datadog Agent Observability and RAID.
On Hacker News, “Ontology-Driven Observability: Building an End to End Knowledge Graph at Netflix” drew the most attention in this space (130 points, 32 comments). Among listed tools, Artificial Analysis's site was linked from 89 Hacker News stories over the past 90 days, more than any other tool here.
In research, “UserArena: Benchmarking Interactive User Simulation for Long Horizon Shopping Agents” collected the most upvotes on Hugging Face Daily Papers this week (41). Among new GitHub repositories created in the past 7 days, KhulekanisiyandaMchunu/llm-trading-arena leads with 60 stars.
Biggest GitHub star gainers
GitHub star counts for the 39 open source tools we track, compared between Oct 18, 2026 and Oct 26, 2026 (180 hours apart).
| Tool | Stars before | Stars after | Gain | Change |
|---|---|---|---|---|
| Langfuse | 36,002 | 36,510 | +508 | 1.41% |
| Promptfoo | 26,113 | 26,448 | +335 | 1.28% |
| Opik | 22,721 | 22,931 | +210 | 0.92% |
| DeepEval | 18,938 | 19,109 | +171 | 0.90% |
| Aider Leaderboards | 49,618 | 49,756 | +138 | 0.28% |
| Arize Phoenix | 11,901 | 12,006 | +105 | 0.88% |
| garak | 9,608 | 9,705 | +97 | 1.01% |
| Ragas | 16,061 | 16,151 | +90 | 0.56% |
New listings this week
| Tool | Category | Listed |
|---|---|---|
| FrontierCode | Benchmarks | Oct 24, 2026 |
| NoveltyBench | Benchmarks | Oct 23, 2026 |
| Infinity-Chat | Benchmarks | Oct 23, 2026 |
| Datadog Agent Observability | Observability | Oct 23, 2026 |
| RAID | Benchmarks | Oct 21, 2026 |
Most discussed on Hacker News
The highest scoring Hacker News stories in this space during the week.
| Story | Points | Comments | Discussion |
|---|---|---|---|
| Ontology-Driven Observability: Building an End to End Knowledge Graph at Netflix | 130 | 32 | thread |
| Benchmarking Bot Detection Systems Against Modern AI Agents in 2026 | 49 | 8 | thread |
| Crows can evaluate visual info twice as fast as mammals | 27 | 1 | thread |
| LabRobotBench v0.1: Benchmarking wet lab robotics | 15 | 3 | thread |
Listed tools with the most Hacker News links
Hacker News stories linking to each tool's own website over the past 90 days.
Notable new papers
The most upvoted papers on Hugging Face Daily Papers and the newest relevant arXiv preprints from this week.
- UserArena: Benchmarking Interactive User Simulation for Long Horizon Shopping Agents
- AnyGameRenderer: Entity-Consistent Generative Rendering Across Games and an Entity-Level Benchmark
- Dual Exam: Controlled Evaluation of Multimodal Agents and World Models
- TenetBench: A Clause-Anchored Russian Corpus for Evaluating Language Models against Professional Codes of Ethics
- RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation
- RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms
- Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM Based Code Optimizations
- PHYRD: A Living Benchmark for Reproducing and Extending Theoretical Physics Research
- IGG: A Benchmark for Interactive GUI Grounding under Visibility Constraints
- SCI-AGENT-EXAM: Benchmarking Agents on Scientific Research Tasks
New GitHub repos in LLM evaluation
Repositories created in the 7 days before Oct 26, 2026, ranked by stars.
| Repository | Stars | Language | Description |
|---|---|---|---|
| KhulekanisiyandaMchunu/llm-trading-arena | 60 | JavaScript | Live trading arena for LLM agents: public reasoning, real market data, virtual capital |
| jwuthri-cloud/SelfJev | 56 | Python | Self hosted decision model (selfjev-4b) with a Jev compatible API: Python SDK, server, fine tuning and RLCD. |
| kling01/RubricLoop | 37 | Python | Rubric driven evaluation for AI agents. Executable criteria, code and model judges, and inspectable trace reports. |
| alexpiar/be-ambitious-harness | 23 | Python | Make AI agents aim higher without losing control: a Claude Code skill with two bars per task, a cheap gate check before expensive work, and an independent verifier that never sees the worker's reasoni |
| liulx25xx/DoubleTakeBench | 12 | Python | DoubleTakeBench: 525 evidence based judgment questions from coding agent trajectories for evaluating System 1 models and LLMs. |
| triageagent-dev/ego | 8 | Go | ego: agents' evaluations on the go (pure go) |
| Faraz6180/rag-agent-failure-catalog | 8 | Python | Catalog of silent failures in RAG systems and AI agents, with reproduce, detect and mitigate code |
| alphaparkinc/genpark-entropy-hallucination-detector-skill | 7 | Python | Token entropy and n-gram perplexity analyzer auditing LLM outputs for degenerate repetition loops and vocabulary collapse |
How we compile this report
Every figure comes from a public source: star counts from the GitHub API, story scores and links from Hacker News (via Algolia), papers from arXiv and Hugging Face Daily Papers, and new repositories from GitHub search. Hacker News link counts include stories whose link points to the tool's own domain. Listing dates are the day a tool joined TrueIQ. Numbers are shown as recorded; if a figure is unavailable, it is left out rather than estimated.