LSP / Editor Support

Language Server Protocol support and editor configuration for poly-bench

poly-bench includes a full Language Server Protocol (LSP) implementation that provides rich editor support for .bench files. This includes real-time diagnostics, completions, hover documentation, formatting, and semantic highlighting.

Features

Feature	Description
Diagnostics	Real-time error and warning reporting as you type
Formatting	Format documents on save or on demand
Completions	Intelligent completions for keywords, stdlib, and user symbols
Hover	Documentation and type information on hover
Semantic highlighting	Syntax highlighting tokens for accurate coloring
Embedded languages	Delegates to gopls, tsserver, and rust-analyzer for code blocks

Starting the LSP Server

The LSP server runs over stdio. Start it with:

$poly-bench lsp

This is typically invoked by your editor's extension, not run manually.

VS Code / Cursor Setup

Install the Extension

The poly-bench extension is available from the VS Code marketplace or can be installed from a .vsix file.

$# From marketplace (when available)
$code --install-extension poly-bench.poly-bench

$# From local .vsix file
$code --install-extension poly-bench-0.0.22.vsix

Manual Configuration

If you're not using the extension, configure the LSP manually in settings.json:

1{
2"poly-bench.serverPath": "poly-bench",
3"poly-bench.serverArgs": ["lsp"],
4"[bench]": {
5  "editor.formatOnSave": true,
6  "editor.defaultFormatter": "poly-bench.poly-bench"
7}
8}

File Association

Associate .bench files with the poly-bench language:

1{
2"files.associations": {
3  "*.bench": "bench"
4}
5}

The extension automatically associates .bench files when installed. Manual configuration is only needed if you're setting up the LSP client yourself.

Neovim Setup

Using nvim-lspconfig

1-- Add poly-bench to lspconfig
2local lspconfig = require('lspconfig')
3local configs = require('lspconfig.configs')
4
5-- Define the poly-bench language server
6if not configs.polybench then
7configs.polybench = {
8  default_config = {
9    cmd = { 'poly-bench', 'lsp' },
10    filetypes = { 'bench' },
11    root_dir = lspconfig.util.root_pattern('polybench.toml', '.git'),
12    settings = {},
13  },
14}
15end
16
17-- Set up the server
18lspconfig.polybench.setup({
19on_attach = function(client, bufnr)
20  -- Your on_attach configuration
21end,
22capabilities = require('cmp_nvim_lsp').default_capabilities(),
23})

File Type Detection

Add file type detection for .bench files:

1vim.filetype.add({
2extension = {
3  bench = 'bench',
4},
5})

Syntax Highlighting

For basic syntax highlighting before Tree-sitter support:

1" Keywords
2syn keyword benchKeyword suite setup bench fixture before after each use
3syn keyword benchKeyword import init helpers
4syn keyword benchLang go ts rust
5syn keyword benchOption description warmup compare baseline mode
6syn keyword benchOption targetTime minIterations maxIterations count
7syn keyword benchOption memory concurrency outlierDetection cvThreshold
8syn keyword benchOption timeout sink iterations
9
10" Standard library
11syn keyword benchStdlib std anvil charting constants
12
13" Booleans
14syn keyword benchBoolean true false
15
16" Strings
17syn region benchString start=/"/ end=/"/ skip=/\"/
18
19" Numbers
20syn match benchNumber '<d+>'
21syn match benchDuration '<d+(ms|s|m)>'
22
23" Comments
24syn match benchComment '//.*$'
25
26" Hex values
27syn match benchHex '<0x[0-9a-fA-F]+>'
28syn match benchHex '"[0-9a-fA-F]+"'
29
30" Highlighting
31hi def link benchKeyword Keyword
32hi def link benchLang Type
33hi def link benchOption Identifier
34hi def link benchStdlib Constant
35hi def link benchBoolean Boolean
36hi def link benchString String
37hi def link benchNumber Number
38hi def link benchDuration Number
39hi def link benchComment Comment
40hi def link benchHex String

Helix Setup

Add to your languages.toml:

1[[language]]
2name = "bench"
3scope = "source.bench"
4injection-regex = "bench"
5file-types = ["bench"]
6roots = ["polybench.toml"]
7comment-token = "//"
8indent = { tab-width = 4, unit = "    " }
9language-servers = ["polybench"]
10
11[language-server.polybench]
12command = "poly-bench"
13args = ["lsp"]

Zed Setup

Add to your settings.json:

1{
2"languages": {
3  "Bench": {
4    "language_servers": ["polybench"]
5  }
6},
7"lsp": {
8  "polybench": {
9    "binary": {
10      "path": "poly-bench",
11      "arguments": ["lsp"]
12    }
13  }
14}
15}

LSP Features in Detail

Diagnostics

The LSP reports errors and warnings in real-time:

Severity	Examples
Error	Syntax errors, undefined fixtures, missing required fields
Warning	Unused fixtures, deprecated syntax
Hint	Style suggestions

1error: undefined fixture 'data' at line 15
2--> keccak.bench:15:9
3 |
415 |     go: keccak256(data)
5 |         ^^^^^^^^^^^^^^
6 |
7 = help: define a fixture named 'data' before using it
8
9error: missing 'go' implementation in benchmark 'hash'
10--> keccak.bench:20:5
11 |
1220 |     bench hash {
13 |     ^^^^^^^^^^
14 |
15 = help: add 'go: <expression>' to the benchmark block

Completions

The LSP provides context-aware completions:

Context	Completions
Top-level	`suite`, `use`
Suite body	`setup`, `bench`, `fixture`, `before`, `after`, `each`, config options
Setup block	`import`, `init`, `helpers`
After block	Standard library directives (`charting.drawTable`, etc.)
Bench block	Language names (`go:`, `ts:`, `rust:`), fixture names

Type charting. in an after block to see available chart directives with documentation.

Hover Documentation

Hover over keywords and symbols to see documentation:

1targetTime
2─────────────────────────────────
3Target execution time for auto-calibration mode.
4When mode is set to "auto", poly-bench adjusts
5the iteration count to reach this target time.
6
7Type: duration (e.g., 3000ms, 5s)
8Default: 3000ms

Formatting

The formatter enforces consistent style:

4-space indentation
Consistent brace placement
Aligned configuration options
Proper spacing around operators

1suite test{
2warmup:100
3compare:true
4
5setup go{
6helpers{
7func add(a,b int)int{return a+b}
8}}
9
10bench addition{
11go:add(1,2)
12}}

Semantic Highlighting

The LSP provides semantic tokens for accurate highlighting:

Token Type	Examples
`keyword`	`suite`, `setup`, `bench`, `fixture`
`type`	`go`, `ts`, `rust`
`variable`	Fixture names, benchmark names
`function`	Helper function names
`string`	String literals, hex values
`number`	Numeric literals, durations
`comment`	Comments
`namespace`	Standard library modules

Embedded Language Support

The LSP delegates to language-specific servers for code inside helpers and init blocks:

Language	Delegate Server
Go	gopls
TypeScript	tsserver
Rust	rust-analyzer

This provides:

Completions from language-specific libraries
Type checking within code blocks
Go-to-definition for imported symbols

Embedded language support requires the respective language servers to be installed and available in your PATH.

Troubleshooting

LSP Not Starting

Verify poly-bench is installed: poly-bench --version
Check the server path in your editor settings
Look for errors in your editor's LSP log

No Completions

Ensure the file has a .bench extension
Check file type association in your editor
Verify the LSP is connected (check language server status)

Formatting Not Working

Check if format-on-save is enabled for .bench files
Verify the formatter is set to poly-bench
Try manual format command (Format Document)

Debug Logging

Enable verbose logging for debugging:

$# Set Rust log level
$RUST_LOG=debug poly-bench lsp

$# Or trace level for maximum detail
$RUST_LOG=trace poly-bench lsp

1-- Add poly-bench to lspconfig 2local lspconfig = require('lspconfig') 3local configs = require('lspconfig.configs') 4 5-- Define the poly-bench language server 6if not configs.polybench then 7configs.polybench = { 8 default_config = { 9 cmd = { 'poly-bench', 'lsp' }, 10 filetypes = { 'bench' }, 11 root_dir = lspconfig.util.root_pattern('polybench.toml', '.git'), 12 settings = {}, 13 }, 14} 15end 16 17-- Set up the server 18lspconfig.polybench.setup({ 19on_attach = function(client, bufnr) 20 -- Your on_attach configuration 21end, 22capabilities = require('cmp_nvim_lsp').default_capabilities(), 23})

1" Keywords 2syn keyword benchKeyword suite setup bench fixture before after each use 3syn keyword benchKeyword import init helpers 4syn keyword benchLang go ts rust 5syn keyword benchOption description warmup compare baseline mode 6syn keyword benchOption targetTime minIterations maxIterations count 7syn keyword benchOption memory concurrency outlierDetection cvThreshold 8syn keyword benchOption timeout sink iterations 9 10" Standard library 11syn keyword benchStdlib std anvil charting constants 12 13" Booleans 14syn keyword benchBoolean true false 15 16" Strings 17syn region benchString start=/"/ end=/"/ skip=/\"/ 18 19" Numbers 20syn match benchNumber '<d+>' 21syn match benchDuration '<d+(ms|s|m)>' 22 23" Comments 24syn match benchComment '//.*$' 25 26" Hex values 27syn match benchHex '<0x[0-9a-fA-F]+>' 28syn match benchHex '"[0-9a-fA-F]+"' 29 30" Highlighting 31hi def link benchKeyword Keyword 32hi def link benchLang Type 33hi def link benchOption Identifier 34hi def link benchStdlib Constant 35hi def link benchBoolean Boolean 36hi def link benchString String 37hi def link benchNumber Number 38hi def link benchDuration Number 39hi def link benchComment Comment 40hi def link benchHex String

1[[language]] 2name = "bench" 3scope = "source.bench" 4injection-regex = "bench" 5file-types = ["bench"] 6roots = ["polybench.toml"] 7comment-token = "//" 8indent = { tab-width = 4, unit = " " } 9language-servers = ["polybench"] 10 11[language-server.polybench] 12command = "poly-bench" 13args = ["lsp"]

Severity

Examples

Error

Syntax errors, undefined fixtures, missing required fields

Warning

Unused fixtures, deprecated syntax

Hint

Style suggestions

1error: undefined fixture 'data' at line 15 2--> keccak.bench:15:9 3 | 415 | go: keccak256(data) 5 | ^^^^^^^^^^^^^^ 6 | 7 = help: define a fixture named 'data' before using it 8 9error: missing 'go' implementation in benchmark 'hash' 10--> keccak.bench:20:5 11 | 1220 | bench hash { 13 | ^^^^^^^^^^ 14 | 15 = help: add 'go: <expression>' to the benchmark block

Context

Completions

Top-level

suite, use

Suite body

setup, bench, fixture, before, after, each, config options

Setup block

import, init, helpers

After block

Standard library directives (charting.drawTable, etc.)

Bench block

Language names (go:, ts:, rust:), fixture names

1targetTime 2───────────────────────────────── 3Target execution time for auto-calibration mode. 4When mode is set to "auto", poly-bench adjusts 5the iteration count to reach this target time. 6 7Type: duration (e.g., 3000ms, 5s) 8Default: 3000ms

Token Type

Examples

keyword

suite, setup, bench, fixture

type

go, ts, rust

variable

Fixture names, benchmark names

function

Helper function names

string

String literals, hex values

number

Numeric literals, durations

comment

Comments

namespace

Standard library modules

Language

Delegate Server

gopls

TypeScript

tsserver

Rust

rust-analyzer