Why poly-bench

The Problem

Imagine you're building a library that needs implementations in Go, TypeScript, and Rust. You want to know which is fastest for different operations. Today, you'd write:

1func BenchmarkKeccak(b *testing.B) {
2  data := []byte("hello world")
3  for i := 0; i < b.N; i++ {
4      keccak256(data)
5  }
6}

Three different APIs. Three different output formats. Three different iteration strategies. No easy way to compare.

The Solution

With poly-bench, you write one .bench file:

1suite keccak {
2  warmup: 100
3  compare: true
4  baseline: "go"
5
6  setup go {
7      import "golang.org/x/crypto/sha3"
8      helpers {
9          func keccak256(data []byte) []byte {
10              h := sha3.NewLegacyKeccak256()
11              h.Write(data)
12              return h.Sum(nil)
13          }
14      }
15  }
16
17  setup ts {
18      import { keccak256 } from 'viem';
19  }
20
21  setup rust {
22      import { use tiny_keccak::{Hasher, Keccak}; }
23      helpers {
24          fn keccak256(data: &[u8]) -> [u8; 32] {
25              let mut hasher = Keccak::v256();
26              let mut output = [0u8; 32];
27              hasher.update(data);
28              hasher.finalize(&mut output);
29              output
30          }
31      }
32  }
33
34  fixture data {
35      hex: "68656c6c6f20776f726c64"
36  }
37
38  bench keccak256 {
39      go: keccak256(data)
40      ts: keccak256(data)
41      rust: keccak256(&data)
42  }
43}

Run poly-bench run and get unified results across all three languages with automatic comparison tables.

Design Principles

poly-bench was built with these principles in mind:

1. DSL-First

The .bench file format is the source of truth. It's designed to be readable, version-controllable, and language-agnostic while supporting language-specific setup when needed.

2. Language-Agnostic IR

The DSL compiles to an intermediate representation (IR) that can generate code for any supported language. This ensures consistent behavior and makes adding new languages straightforward.

3. Isolated Runtimes

Each language runs in its own isolated environment:

• Go: Plugin-based execution with runtime.ReadMemStats for memory tracking
• TypeScript: Node.js subprocess with process.memoryUsage()
• Rust: Cargo subprocess with memory profiling support

4. Statistics Done Right

poly-bench includes proper statistical methodology:

• Warmup iterations before timing
• Multiple runs for reliable statistics
• IQR-based outlier detection
• Coefficient of variation targeting

What's Implemented

Getting Started

Ready to benchmark across languages? Head to the Getting Started guide to set up your first poly-bench project.