Skip to content

Contributing

Contributions are welcome: adding a grammar, fixing a bug, improving a binding, or writing documentation.

For CI/CD workflow details, see the CI/CD reference.

You’ll need the following tools installed:

  • mise — the project toolchain and task runner
  • Rust stable toolchain via rustup
  • Python 3.11+ and uv
  • Node.js 22+ and pnpm
Terminal window
# Install mise (macOS)
brew install mise
# Clone the repository
git clone https://github.com/xberg-io/tree-sitter-language-pack.git
cd tree-sitter-language-pack
# Trust the checkout's mise.toml, then install all language dependencies
mise trust
mise run setup
# Build the Rust core
mise run build
# Run all tests
mise run test

On Debian/Ubuntu, install mise with the official installer.

Terminal window
mise tasks # show all available tasks
mise run build # build Rust core + bindings
mise run test # run all test suites
mise run lint # run all linters (clippy, ruff, oxlint, rubocop, …)
mise run format # auto-format all code
mise run e2e:generate # regenerate e2e test suites from fixtures
mise run go:e2e:test # run one binding's e2e tests (<lang>:e2e:test)
alef all --clean --skip-snippet-validation --skip-compile # regenerate the alef-managed bindings and docs

Run mise tasks to see all available tasks.

The most common contribution is adding a new tree-sitter grammar.

The grammar must:

  • Be permissively licensed — MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC, or Unlicense only. We do not accept GPL, AGPL, LGPL, MPL, or any copyleft license. This ensures tree-sitter-language-pack can be used freely in any project without imposing license obligations on downstream users.
  • Have a public Git repository.
  • Produce valid parser.c output from tree-sitter generate.
  • Compile cleanly on Linux, macOS, and Windows.

Edit sources/language_definitions.json and add an entry:

{
"mylang": {
"repo": "https://github.com/example/tree-sitter-mylang",
"rev": "abc123def456",
"branch": "main"
}
}

Always pin to an exact commit (rev), not a branch tip. This ensures reproducible builds.

Available fields:

Field Required Description
repo Yes Grammar repository URL
rev Yes Exact commit SHA to pin
branch No Branch name (used by scripts/pin_vendors.py to find latest)
directory No Subdirectory within the repo containing the grammar
extensions No File extensions that map to this language (e.g. ["rs"])
ambiguous No Extensions shared with other languages (e.g. {"h": ["cpp", "objc"]})
c_symbol No Override for the C symbol name when it differs from the language name
generate No Regenerate before compiling, except oversized committed parsers
force_generate No With generate, require regeneration even for oversized parsers; a timeout fails instead of using committed output
abi_version No Generation ABI target (default 14); the runtime supports ABI 13–15
Terminal window
# Compile the new parser
mise run build
# Run the test suite
mise run test
# Verify the parser works end-to-end
ts-pack download mylang
ts-pack parse example.mylang --language mylang

Add at least one fixture under fixtures/. Fixtures are the single source for both the e2e suites and the documentation snippet corpus — mise run e2e:generate renders each fixture into docs-site/src/snippets/generated/<lang>/<category>/<id>.md for all 14 bindings. Do not hand-write snippets; add a fixture instead.

A fixture file holds either a single JSON object or an array of them, grouped into a per-category directory — for example fixtures/process/python_intel.json. Only id and description are required, additionalProperties is false, the payload goes under input, and assertions is a list of typed assertion objects, not a map of booleans. See fixtures/schema.json for the full assertion-type enum.

{
"id": "mylang_function_process",
"description": "Intel: extract structure from a mylang function definition",
"category": "process",
"tags": ["intel"],
"input": {
"source_code": "// example mylang source",
"config": {
"language": "mylang"
}
},
"assertions": [
{
"type": "equals",
"field": "language",
"value": "mylang"
},
{
"type": "count_min",
"field": "structure",
"value": 1
},
{
"type": "equals",
"field": "metrics.error_count",
"value": 0
}
]
}

Then regenerate and run e2e tests:

Terminal window
mise run e2e:generate
alef test --e2e --jobs 1
  • Title: feat: add <language> parser
  • Body: link to the upstream grammar repository, note any quirks or limitations
  1. Check the issue tracker — the bug may already be reported.
  2. Write a failing test that reproduces the issue.
  3. Fix the bug in the appropriate crate.
  4. Confirm all tests pass with mise run test.
  5. Open a PR with a clear description of the root cause and fix.

Binding improvements (better error messages, idiomatic APIs, new methods) are welcome. Compiled binding crates live under crates/ (ts-pack-core-py, ts-pack-core-node, ts-pack-core-php, ts-pack-core-wasm, ts-pack-core-ffi, tree-sitter-language-pack-jni); the host-language packages they feed live under packages/. See the Architecture page for the full crate layout.

Binding changes must:

  • Not add logic that belongs in the Rust core. Bindings are pure translation layers.
  • Have test coverage in the binding’s native test suite.
  • Follow the existing API surface — most binding surfaces are alef-generated; regenerate them with alef all --clean --skip-snippet-validation --skip-compile rather than hand-editing generated files.

Doc fixes and new guides follow the same workflow as code changes:

  1. Fork and create a branch.
  2. Edit files under docs-site/src/content/docs/. Runnable snippets are generated from fixtures/ into docs-site/src/snippets/generated/<lang>/ — change the fixture and rerun mise run e2e:generate, never edit a generated snippet.
  3. Preview locally with pnpm --dir docs-site dev (the site is Astro / Starlight).
  4. Run mise run lint if you touch any scripted checks.
  5. Open a pull request.

Use the Edit button in the page header to jump directly from any docs page to the matching file on GitHub.

The project uses pre-commit hooks managed by prek:

Terminal window
prek install
prek install --hook-type commit-msg

Before committing, verify these three commands pass:

Terminal window
mise run lint # zero warnings required
mise run test # all tests must pass
mise run format # code must be formatted

Follow Conventional Commits:

feat: add kotlin parser
fix: correct memory layout in Java FFI array freeing
chore: update tree-sitter to 0.25
docs: add chunking guide
test: add e2e fixtures for ruby

Keep commits small and focused. Each commit should represent one logical change.

  • mise run test passes
  • mise run lint passes (zero warnings)
  • New language has at least one fixture under fixtures/, with the regenerated snippets under docs-site/src/snippets/generated/ committed alongside it
  • mise run e2e:generate && alef test --e2e --jobs 1 passes
  • mise run version:sync run if any manifest was bumped
  • PR description explains the change and links related issues