The week in brief
Behind it, Promptfoo (+287) and Opik (+182) followed.
Three new tools joined the directory this week: GDPx, BenchLeader and Fireworks Specialized Intelligence Index.
On Hacker News, “Dayjob: Tax. Evaluating AI agents on 50 professional tax assignments” drew the most attention in this space (266 points, 174 comments). Among listed tools, Artificial Analysis's site was linked from 93 Hacker News stories over the past 90 days, more than any other tool here.
In research, “CrossSWE: Benchmarking Cross Repository Joint Development for Coding Agents” collected the most upvotes on Hugging Face Daily Papers this week (19). Among new GitHub repositories created in the past 7 days, wanghao-SunSky/cs329z-chinese leads with 476 stars.
Biggest GitHub star gainers
GitHub star counts for the 39 open source tools we track, compared between Oct 27, 2026 and Nov 3, 2026 (169 hours apart).
| Tool | Stars before | Stars after | Gain | Change |
|---|---|---|---|---|
| Langfuse | 36,582 | 36,942 | +360 | 0.98% |
| Promptfoo | 26,488 | 26,775 | +287 | 1.08% |
| Opik | 22,960 | 23,142 | +182 | 0.79% |
| Aider Leaderboards | 49,770 | 49,919 | +149 | 0.30% |
| DeepEval | 19,135 | 19,269 | +134 | 0.70% |
| garak | 9,721 | 9,842 | +121 | 1.24% |
| Arize Phoenix | 12,021 | 12,118 | +97 | 0.81% |
| lm-evaluation-harness | 14,323 | 14,384 | +61 | 0.43% |
New listings this week
| Tool | Category | Listed |
|---|---|---|
| GDPx | Leaderboards | Nov 02, 2026 |
| BenchLeader | Leaderboards | Nov 02, 2026 |
| Fireworks Specialized Intelligence Index | Leaderboards | Oct 30, 2026 |
Most discussed on Hacker News
The highest scoring Hacker News stories in this space during the week.
| Story | Points | Comments | Discussion |
|---|---|---|---|
| Dayjob: Tax (evaluating AI agents on 50 professional tax assignments) | 266 | 174 | thread |
| Artificial Analysis v5 and the benchmarker's dilemma | 73 | 48 | thread |
Listed tools with the most Hacker News links
Hacker News stories linking to each tool's own website over the past 90 days.
Notable new papers
The most upvoted papers on Hugging Face Daily Papers and the newest relevant arXiv preprints from this week.
- CrossSWE: Benchmarking Cross Repository Joint Development for Coding Agents
- PhotoGuider: Composition-Aware Cropping and 360^{circ} View Framing with Explainable Photographic Assessments
- HALO: Human Grounded Granular Evaluation of Proactive Agents
- MAServeBench: Benchmarking Multi Agent LLM Serving Systems with Hierarchical Collaboration Workloads
- Measuring AI Agents as Teammates with Collaborative Intelligence
- Benchmark Prediction with Sparse and Unpaired Cached Responses
- ufakzeka kulak: Building and Evaluating a 31.6M-Parameter Turkish Speech Recognizer from Scratch
- Evaluating Uncertainty Calibration in Probabilistic Time Series Foundation Models
- NitiBench+: A Temporal Thai Legal Question Answering Benchmark Across 17 Domains
- Harder Than the Task: From Technical to Organizational Difficulty in AI Benchmarking
New GitHub repos in LLM evaluation
Repositories created in the 7 days before Nov 03, 2026, ranked by stars.
| Repository | Stars | Language | Description |
|---|---|---|---|
| wanghao-SunSky/cs329z-chinese | 476 | HTML | 斯坦福 Stanford CS329Z Engineering AI Agents 中文教程：面向 Python 开发者的 AI 智能体课程，19 讲与 4 项实践，涵盖 LLM、RAG、工具调用、记忆、多智能体、评测与安全。完整 Markdown、在线与离线 HTML。Chinese course notes. |
| ccb-ai-solutions/yz-saha-testi-2026 | 6 | HTML | YZ Saha Testi 2026: Türkçe kurum işlerinde 14 açık dil modeli ve 36 pipeline bileşeni (RAG, tool calling, OCR, guardrail, ASR, latency). Kod, test verisi, bütün cevaplar ve sonuçlar. Ekim 2026. |
| alexcarterhq/wspbench | 3 | Python | Controlled weight space programming evaluation with CPU replay, fixed neural networks, and audited LLM pilots. |
| arthurcunha51eng/geotech-rl | 3 | Python | Procedural RL environment for geotechnical judgement, with physics grader and anti shortcut gates. |
| SkillGauntlet/SkillGauntlet | 2 | JavaScript | Independent tests of agent skills (SKILL.md): does it trigger, does it help, is it safe. Reproducible data and an npx CLI. |
| Wellux/awesome-agent-engineering | 2 | Frameworks, multiagent systems, workflows, voice agents, guardrails, evals and the courses to build them. Hand picked. | |
| closestfriend/benchmark-designer | 2 | Python | Agent skill for designing and red teaming AI benchmarks and evals, with a scorer attack battery, judge meta-evaluation, and power analysis. |
| OpenWave-India/modeldiff | 1 | Python | Regression testing for local AI |
How we compile this report
Every figure comes from a public source: star counts from the GitHub API, story scores and links from Hacker News (via Algolia), papers from arXiv and Hugging Face Daily Papers, and new repositories from GitHub search. Hacker News link counts include stories whose link points to the tool's own domain. Listing dates are the day a tool joined TrueIQ. Numbers are shown as recorded; if a figure is unavailable, it is left out rather than estimated.