Architecture
How poly-bench is structured, how benchmarks flow, and how the runtime isolation works
How poly-bench is structured, how benchmarks flow, and how the runtime isolation works
poly-bench is built as a modular Rust workspace where each crate handles a specific responsibility. This architecture enables clean separation between parsing, validation, code generation, execution, and reporting.
.bench) are the source of truth for benchmark definitionsEvery benchmark flows through the same pipeline. Your .bench file is parsed into an AST, lowered to IR, used to generate language-specific code, executed in isolated runtimes, and finally reported.
| Crate | What it's for |
|---|---|
poly-bench-dsl | Lexer, parser, AST, formatter, and validator for .bench files |
poly-bench-ir | Intermediate representation — normalized, validated benchmark structures |
poly-bench-runtime | Language-specific code generation and runtime execution (Go, TS, Rust) |
poly-bench-executor | Orchestrates benchmark runs, manages calibration, collects measurements |
poly-bench-reporter | Generates output in console, markdown, JSON, and SVG chart formats |
poly-bench-project | Project initialization, dependency management, manifest handling |
poly-bench-stdlib | Standard library modules (std::anvil, std::charting, std::constants) |
poly-bench-lsp-v2 | Language Server Protocol implementation for editor support |
poly-bench-dsl to parse .bench files in a custom analysis tool.When you run poly-bench run benchmark.bench, the DSL parser tokenizes and parses your file into an AST. The validator checks for semantic errors like undefined fixtures or missing language blocks.
The AST is lowered to an Intermediate Representation that's fully resolved and normalized. This step:
For each target language, poly-bench generates native benchmark code. The generated code includes:
1// Auto-generated by poly-bench2package main3
4import (5 "golang.org/x/crypto/sha3"6 "time"7)8
9// Fixture: data10var data = []byte{0x68, 0x65, 0x6c, 0x6c, 0x6f, ...}11
12// Helper from setup block13func keccak256Go(data []byte) []byte {14 h := sha3.NewLegacyKeccak256()15 h.Write(data)16 return h.Sum(nil)17}18
19func benchmark_keccak256Bench(iterations int) time.Duration {20 start := time.Now()21 for i := 0; i < iterations; i++ {22 _ = keccak256Go(data)23 }24 return time.Since(start)25}Each language runs in its own isolated subprocess:
| Language | Runtime | Memory Tracking |
|---|---|---|
| Go | Plugin-based execution or subprocess | runtime.ReadMemStats |
| TypeScript | Node.js subprocess | process.memoryUsage() |
| Rust | Cargo build + subprocess | Custom allocator tracking |
.polybench/runtime-env/. Each language has its own dependency directory, ensuring reproducible builds and no global pollution.In mode: "auto", poly-bench calibrates the iteration count to hit a target execution time:
This ensures:
After collecting raw measurements, poly-bench computes statistics:
warmup)count times1Summary: keccak256Bench2─────────────────────────────────────────────3
4│ Lang │ Mean (ns/op) │ Std Dev │ Ops/s │5├──────┼──────────────┼─────────┼──────────┤6│ rust │ 993 │ ± 1.8% │ 1,006,752│7│ go │ 1,216 │ ± 2.1% │ 822,714 │8│ ts │ 15,667 │ ± 3.1% │ 63,839 │9
10Comparison (baseline: go):11 rust: 1.22x faster12 ts: 12.88x slowerThe standard library (poly-bench-stdlib) provides built-in modules:
| Module | Purpose | Key Features |
|---|---|---|
std::anvil | Ethereum development | Spawn local Anvil node, provide RPC URL |
std::charting | Visualization | Tables, pie charts, speedup charts as SVG |
std::constants | Math constants | Pi, E, and other common values |
Standard library modules are implemented in Rust and invoked via directives in before/after blocks:
1use std::anvil2use std::charting3
4suite example {5 before {6 anvil.spawnAnvil(fork: "https://eth.llamarpc.com")7 }8
9 // ... benchmarks ...10
11 after {12 charting.drawTable(title: "Results")13 anvil.stopAnvil()14 }15}