textmate_grammar.engine.registry

Grammar registration and dependency loading from vscode-textmate v9.3.2.

The upstream Registry and SyncRegistry create the concrete Grammar class. That class is not available until task 3b, so the one deliberate seam here is an injectable grammar_factory(location, raw_grammar). Its default raises NotImplementedError; the later tokenizer task supplies the real constructor.

Upstream exposes asynchronous methods because its loader returns promises. This port intentionally uses synchronous callables returning raw grammar mappings (or None); dependency traversal, load caching, registration, and construction otherwise retain the upstream behavior. onig_lib is carried in the options for the eventual Grammar wiring, but is not consumed by this seam yet.

Module Contents

Classes

TokenizerLike

Minimal interface of a constructed grammar owned by the registry.

RegistryOptions

Synchronous counterpart of upstream RegistryOptions.

TopLevelRuleReference

Reference to an entire grammar, as in grammarDependencies.ts:15-22.

TopLevelRepositoryRuleReference

Reference to one named rule in a grammar’s top-level repository.

ExternalReferenceCollector

Deduplicating collector; raw-rule visitation uses identity like JS Set.

ScopeDependencyProcessor

Queue-based port of ScopeDependencyProcessor (lines 59-103).

SyncRegistry

Synchronous equivalent of upstream SyncRegistry.

Registry

Registry options, dependency resolution, and lazy grammar construction.

Functions

collect_includes

Collect external references from a grammar’s own rules and injections.

get_grammar_dependencies

Return all reachable full grammar scopes, including injection scopes.

Data

API

textmate_grammar.engine.registry.RawGrammar = None
class textmate_grammar.engine.registry.TokenizerLike

Bases: typing.Protocol

Minimal interface of a constructed grammar owned by the registry.

dispose() → None

Release tokenizer resources, when present.

class textmate_grammar.engine.registry.RegistryOptions

Bases: typing.TypedDict

Synchronous counterpart of upstream RegistryOptions.

Initialization

Initialize self. See help(type(self)) for accurate signature.

onig_lib: textmate_grammar.engine.oniglib.IOnigLib = None
load_grammar: collections.abc.Callable[[str], textmate_grammar.engine.raw_grammar.IRawGrammar | None] = None
get_injections: collections.abc.Callable[[str], collections.abc.Sequence[str] | None] = None
onigLib: textmate_grammar.engine.oniglib.IOnigLib = None
loadGrammar: collections.abc.Callable[[str], textmate_grammar.engine.raw_grammar.IRawGrammar | None] = None
getInjections: collections.abc.Callable[[str], collections.abc.Sequence[str] | None] = None
textmate_grammar.engine.registry.GrammarFactory = None
class textmate_grammar.engine.registry.TopLevelRuleReference

Reference to an entire grammar, as in grammarDependencies.ts:15-22.

scope_name: str = None
to_key() → str
class textmate_grammar.engine.registry.TopLevelRepositoryRuleReference

Reference to one named rule in a grammar’s top-level repository.

scope_name: str = None
rule_name: str = None
to_key() → str
textmate_grammar.engine.registry.AbsoluteRuleReference = None
class textmate_grammar.engine.registry.ExternalReferenceCollector

Deduplicating collector; raw-rule visitation uses identity like JS Set.

Initialization

add(reference: textmate_grammar.engine.registry.AbsoluteRuleReference) → None
class textmate_grammar.engine.registry.ScopeDependencyProcessor(repo: textmate_grammar.engine.registry.SyncRegistry, initial_scope_name: str)

Queue-based port of ScopeDependencyProcessor (lines 59-103).

Initialization

process_queue() → None
textmate_grammar.engine.registry.collect_includes(grammar: textmate_grammar.engine.raw_grammar.IRawGrammar) → list[textmate_grammar.engine.registry.AbsoluteRuleReference]

Collect external references from a grammar’s own rules and injections.

The result is an empty-base traversal useful for inspection. Full dependency loading should use get_grammar_dependencies(), which resolves local $base and scope references against the registry like upstream does.

textmate_grammar.engine.registry.get_grammar_dependencies(registry: SyncRegistry, scope_name: str) → list[str]

Return all reachable full grammar scopes, including injection scopes.

class textmate_grammar.engine.registry.SyncRegistry(onig_lib: textmate_grammar.engine.oniglib.IOnigLib = DEFAULT_ONIG_LIB)

Synchronous equivalent of upstream SyncRegistry.

Initialization

dispose() → None
register_grammar(grammar: textmate_grammar.engine.raw_grammar.IRawGrammar, injection_scope_names: collections.abc.Sequence[str] | None = None) → None
addGrammar(grammar: textmate_grammar.engine.raw_grammar.IRawGrammar, injection_scope_names: collections.abc.Sequence[str] | None = None) → None
lookup(scope_name: str) → textmate_grammar.engine.raw_grammar.IRawGrammar | None
injections(target_scope: str) → list[str] | None
get_external_grammar(scope_name: str, repository: collections.abc.Mapping[str, Any] | None = None) → textmate_grammar.engine.raw_grammar.IRawGrammar | None

Look up and initialize an included grammar, cached by scope and base.

Upstream caches per concrete Grammar instance. The registry-level substitute keys by the caller’s $base object as well, preventing one root grammar’s base repository from leaking into another.

grammar_for_scope_name(scope_name: str, grammar_factory: textmate_grammar.engine.registry.GrammarFactory | None = None) → textmate_grammar.engine.registry.TokenizerLike | None
grammarForScopeName(scope_name: str) → textmate_grammar.engine.registry.TokenizerLike | None
class textmate_grammar.engine.registry.Registry(options: textmate_grammar.engine.registry.RegistryOptions, *, grammar_factory: textmate_grammar.engine.registry.GrammarFactory | None = None)

Registry options, dependency resolution, and lazy grammar construction.

Initialization

dispose() → None
register_grammar(raw_grammar: textmate_grammar.engine.raw_grammar.IRawGrammar, injections: collections.abc.Sequence[str] | None = None) → None

Register an already parsed raw grammar and optional injection scopes.

load_grammar(initial_scope_name: str) → textmate_grammar.engine.registry.TokenizerLike | None

Load a grammar and every referenced grammar before constructing it.

loadGrammar(initial_scope_name: str) → textmate_grammar.engine.registry.TokenizerLike | None

vscode-textmate spelling of load_grammar().

addGrammar(raw_grammar: textmate_grammar.engine.raw_grammar.IRawGrammar, injections: collections.abc.Sequence[str] = (), initial_language: int = 0, embedded_languages: collections.abc.Mapping[str, int] | None = None) → textmate_grammar.engine.registry.TokenizerLike

Register and construct a grammar, matching upstream’s convenience method.