Textmate Grammar Python

Ruff Checked with mypy pre-commit CI/CD readthedocs

A Python port of the vscode-textmate grammar tokenization engine. It parses TextMate grammars and tokenizes source line by line while preserving the state stack between lines.

The engine’s token streams are checked against vscode-textmate 9.3.2 using a Node oracle and a differential corpus spanning MATLAB, Markdown, Python, and TypeScript.

flowchart LR G[TextMate grammar] --> R[Registry] C[source lines] --> R R --> T[tokens + rule stack] T -. differential comparison .-> O[vscode-textmate 9.3.2 oracle] click O "https://github.com/microsoft/vscode-textmate"

Index

Sources