Reporting

1Running suite: keccak
2Calibrating iterations...
3Target time: 3000ms
4Calibrated to: 2,430,000 iterations
5
6Running: keccak256Bench (run 1/3)
7  go:   1,234 ns/op (810,372 ops/s) ± 2.1%
8  ts:   15,678 ns/op (63,784 ops/s) ± 3.4%
9  rust: 987 ns/op (1,013,171 ops/s) ± 1.8%
10
11Running: keccak256Bench (run 2/3)
12  go:   1,198 ns/op (834,725 ops/s) ± 1.9%
13  ts:   15,432 ns/op (64,800 ops/s) ± 2.8%
14  rust: 1,002 ns/op (998,004 ops/s) ± 2.0%
15
16Running: keccak256Bench (run 3/3)
17  go:   1,215 ns/op (823,045 ops/s) ± 2.3%
18  ts:   15,890 ns/op (62,933 ops/s) ± 3.1%
19  rust: 991 ns/op (1,009,082 ops/s) ± 1.6%
20
21─────────────────────────────────────────────
22Summary: keccak256Bench
23─────────────────────────────────────────────
24
25│ Lang │ Mean (ns/op) │ Std Dev │ Ops/s    │
26├──────┼──────────────┼─────────┼──────────┤
27│ rust │ 993          │ ± 1.8%  │ 1,006,752│
28│ go   │ 1,216        │ ± 2.1%  │ 822,714  │
29│ ts   │ 15,667       │ ± 3.1%  │ 63,839   │
30
31Comparison (baseline: go):
32  rust: 1.22x faster
33  ts:   12.88x slower
34
35✓ Suite completed in 9.2s

$poly-bench run benchmarks/keccak.bench --report markdown --output results/

1# Benchmark Report
2
3## Suite: keccak
4
5**Description:** Keccak256 hash benchmark
6
7**Configuration:**
8- Mode: auto (target: 3000ms)
9- Warmup: 1000 iterations
10- Runs: 3
11- Baseline: go
12
13### Results
14
15| Benchmark | Go (ns/op) | TS (ns/op) | Rust (ns/op) |
16|-----------|------------|------------|--------------|
17| keccak256Bench | 1,216 ± 2.1% | 15,667 ± 3.1% | 993 ± 1.8% |
18
19### Comparison
20
21| Benchmark | Go | TypeScript | Rust |
22|-----------|-----|------------|------|
23| keccak256Bench | baseline | 12.88x slower | 1.22x faster |
24
25### Statistics
26
27| Metric | Go | TypeScript | Rust |
28|--------|-----|------------|------|
29| Mean (ns/op) | 1,216 | 15,667 | 993 |
30| Ops/s | 822,714 | 63,839 | 1,006,752 |
31| Std Dev | ± 2.1% | ± 3.1% | ± 1.8% |
32
33*Generated by poly-bench v0.0.22*

$poly-bench run benchmarks/keccak.bench --report json --output results/

1{
2"version": "0.0.22",
3"timestamp": "2024-01-15T10:30:00Z",
4"suites": [
5  {
6    "name": "keccak",
7    "description": "Keccak256 hash benchmark",
8    "config": {
9      "mode": "auto",
10      "targetTime": 3000,
11      "warmup": 1000,
12      "count": 3,
13      "baseline": "go"
14    },
15    "benchmarks": [
16      {
17        "name": "keccak256Bench",
18        "results": {
19          "go": {
20            "meanNsPerOp": 1216,
21            "stdDev": 0.021,
22            "opsPerSec": 822714,
23            "runs": [1234, 1198, 1215],
24            "memory": null
25          },
26          "ts": {
27            "meanNsPerOp": 15667,
28            "stdDev": 0.031,
29            "opsPerSec": 63839,
30            "runs": [15678, 15432, 15890],
31            "memory": null
32          },
33          "rust": {
34            "meanNsPerOp": 993,
35            "stdDev": 0.018,
36            "opsPerSec": 1006752,
37            "runs": [987, 1002, 991],
38            "memory": null
39          }
40        },
41        "comparisons": {
42          "rust": { "ratio": 0.817, "label": "1.22x faster" },
43          "ts": { "ratio": 12.88, "label": "12.88x slower" }
44        }
45      }
46    ]
47  }
48]
49}

1use std::charting
2
3suite keccak {
4  // ... benchmark config and setup ...
5
6  bench keccak256Bench {
7      go: keccak256Go(data)
8      ts: keccak256Ts(data)
9      rust: keccak256_rust(&data)
10  }
11
12  after {
13
14      charting.drawTable(
15          title: "Keccak256 Performance",
16          output: "keccak-table.svg"
17      )
18  }
19}

$poly-bench run benchmarks/keccak.bench --output results/

1suite myBenchmarks {
2  compare: true       // Enable comparison table
3  baseline: "go"      // Use Go as the baseline (1.0x)
4  
5  // ... rest of suite
6}

1Comparison (baseline: go):
2  rust: 1.22x faster
3  ts:   12.88x slower

1suite rigorous {
2  warmup: 1000              // 1000 warmup iterations
3  count: 5                  // 5 runs per benchmark
4  outlierDetection: true    // Remove statistical outliers
5  cvThreshold: 5            // Stability target (coefficient of variation %)
6  
7  // ... rest of suite
8}

1suite withMemory {
2  memory: true    // Enable memory profiling
3  
4  // ... rest of suite
5}

1Running: keccak256Bench
2  go:   1,234 ns/op (810,372 ops/s) ± 2.1%  |  96 B/op, 2 allocs/op
3  ts:   15,678 ns/op (63,784 ops/s) ± 3.4%  |  1,024 B/op
4  rust: 987 ns/op (1,013,171 ops/s) ± 1.8%  |  0 B/op, 0 allocs/op

1name: Benchmarks
2
3on:
4pull_request:
5  branches: [main]
6
7jobs:
8benchmark:
9  runs-on: ubuntu-latest
10  steps:
11    - uses: actions/checkout@v4
12    
13    - name: Install poly-bench
14      run: cargo install poly-bench
15    
16    - name: Setup languages
17      run: |
18        # Go is pre-installed
19        # Node.js is pre-installed
20        rustup default stable
21    
22    - name: Install dependencies
23      run: poly-bench install
24    
25    - name: Run benchmarks
26      run: poly-bench run --report json --output results/
27    
28    - name: Upload results
29      uses: actions/upload-artifact@v4
30      with:
31        name: benchmark-results
32        path: results/