Skip to content

Benchmarks

Benchmark pages describe methodology, datasets, and reproduction material where available. A published result is current reproducible evidence only when a version-matched run artifact is recorded. Older result tables and example scripts without a matching run artifact remain historical evidence and must not be treated as current performance or adoption claims.

Available Benchmarks

Reasoning

Benchmark Dataset Winner Margin Link
ReAct vs LATS HotpotQA LATS +8.3% accuracy View

Memory & Retrieval

Benchmark Dataset Winner Margin Link
RAG retrieval strategies Custom HyDE +12% recall View
Memory injection methods Custom Top-K semantic Best cost/quality View

Tool Use

Benchmark Dataset Winner Margin Link
Function calling ToolBench Claude 3.7 +6% accuracy View

Reproducing a Benchmark

The example below demonstrates the repository's reproduction convention; it does not establish that every historical result on this page can be reproduced today.

git clone https://github.com/SamoTech/skills-tree.git
cd skills-tree
pip install -r requirements.txt
python benchmarks/reasoning/react-vs-lats.py

Contributing Benchmarks

Benchmarks are among the highest-value contributions. To add one:

  1. Create benchmarks/{category}/{name}.md following the existing format
  2. Include: dataset, methodology, results table, reproduction script
  3. Open a PR with title format: benchmark: [skill-a] vs [skill-b]

See contributing guide for full details.