textmate_grammar.engine.rules¶
Port of the Rule classes, RuleRegistry and the rule factory from
vscode-textmate’s src/rule.ts (version 9.3.2, the pinned reference clone).
Ported from rule.ts: endRuleId/whileRuleId (rule.ts:17-20), the Rule base
class (rule.ts:43-89), CaptureRule (rule.ts:96-120), MatchRule
(rule.ts:122-164), IncludeOnlyRule (rule.ts:166-207), BeginEndRule
(rule.ts:209-288), BeginWhileRule (rule.ts:290-379) and the RuleFactory
(rule.ts:381-573) — i.e. createCaptureRule, getCompiledRuleId,
_compileCaptures and _compilePatterns. The registry lives in Grammar
upstream: registerRule/getRule are grammar.ts:245-254; the class
RuleRegistry here extracts exactly that behaviour.
parse_include ports parseInclude and IncludeReferenceKind from
src/grammar/grammarDependencies.ts (grammarDependencies.ts:247-295) — the
minimal piece the rule factory needs for include resolution. init_grammar ports
initGrammar (grammar.ts:410-421), which installs the $self/$base
repository hooks.
Semantics are 1:1 with the reference; naming is pythonic where it can be (upstream member names are kept when the python port is simply “same words”), and every non-obvious piece carries a comment referencing the ported lines of rule.ts / grammar.ts / grammarDependencies.ts.
Module Contents¶
Classes¶
Port of |
|
Port of |
|
Port of |
|
Port of the rule-id <-> Rule bookkeeping that lives in |
|
Port of the abstract |
|
Port of |
|
Port of |
|
Port of |
|
Port of |
|
Port of |
|
Port of |
|
Port of |
|
Port of the |
|
Port of |
Functions¶
Port of |
|
Port of |
|
Mirror of |
Data¶
API¶
- textmate_grammar.engine.rules.T = 'TypeVar(...)'¶
- textmate_grammar.engine.rules.WHILE_RULE_ID = None¶
- textmate_grammar.engine.rules.IRawRuleDict = None¶
- textmate_grammar.engine.rules.IRawRepositoryDict = None¶
- textmate_grammar.engine.rules.IRawGrammarDict = None¶
- textmate_grammar.engine.rules.ILocation = None¶
- class textmate_grammar.engine.rules.IRuleRegistry¶
Port of
IRuleRegistry(rule.ts:31-34): numeric rule registration and lookup by id.- abstractmethod get_rule(rule_id: int) textmate_grammar.engine.rules.Rule¶
Port of
getRule(ruleId)(rule.ts:32).
- abstractmethod register_rule(factory: collections.abc.Callable[[int], textmate_grammar.engine.rules.T]) textmate_grammar.engine.rules.T¶
Port of
registerRule(factory)(rule.ts:33): the factory is invoked immediately with the freshly allocated id.
- class textmate_grammar.engine.rules.IGrammarRegistry¶
Port of
IGrammarRegistry(rule.ts:36-38): resolves scope-name includes against grammars looked up by scope name. The concrete implementation arrives with the registry layer (task 4b); here it is only the hook shape.
- class textmate_grammar.engine.rules.IRuleFactoryHelper¶
Bases:
textmate_grammar.engine.rules.IRuleRegistry,textmate_grammar.engine.rules.IGrammarRegistryPort of
IRuleFactoryHelper(rule.ts:40-41). UpstreamGrammaradditionally implementsIOnigLib(grammar.ts:98), so the helper doubles as the onig-lib argument of the compile calls; the twocreateOnig*members come fromGrammar.createOnigScanner/createOnigString(grammar.ts:177-181).- abstractmethod create_onig_scanner(sources: collections.abc.Sequence[str]) Any¶
Port of
createOnigScanner(grammar.ts:177-179).
- class textmate_grammar.engine.rules.RuleRegistry(onig_lib: Any = None)¶
Bases:
textmate_grammar.engine.rules.IRuleFactoryHelperPort of the rule-id <-> Rule bookkeeping that lives in
Grammar(grammar.ts:100, 127-128, 245-254): monotonically increasing numeric ids starting at 1, with the rule objects stored under their id. Serves as the concreteIRuleFactoryHelperforRuleFactoryusers and, later (task 3), as the registry component ofGrammar.Note: upstream v9.3.2 has no standalone
RuleRegistryclass and no scope-name interning; the class here is simply the extractedGrammarbehaviour.Initialization
- register_rule(factory: collections.abc.Callable[[int], textmate_grammar.engine.rules.T]) textmate_grammar.engine.rules.T¶
Port of
Grammar.registerRule(grammar.ts:245-250).
- get_rule(rule_id: int) textmate_grammar.engine.rules.Rule¶
Port of
Grammar.getRule(grammar.ts:252-254). May return the (python)Noneof a not-yet-registered rule while its factory is still running — upstream yieldsundefinedthere, and the only reader that can act on it treats non-Nonelikeundefined(seeRuleFactory.compile_patterns).
- get_external_grammar(scope_name: str, repository: textmate_grammar.engine.rules.IRawRepositoryDict | None) textmate_grammar.engine.rules.IRawGrammarDict | None¶
Port of
Grammar.getExternalGrammar(grammar.ts:256-274): scope-name includes resolve through the grammar repository; without a registry layer behind this helper there is nothing to resolve (task 4b provides it).
- create_onig_scanner(sources: collections.abc.Sequence[str]) Any¶
Port of
Grammar.createOnigScanner(grammar.ts:177-179).
- class textmate_grammar.engine.rules.Rule(location: textmate_grammar.engine.rules.ILocation, id: int, name: str | None, contentName: str | None)¶
Port of the abstract
Rule(rule.ts:43-89): identity,name/contentNamewith capture expansion, and the pattern collection / compile contract the tokenization loop drives.Initialization
- getName(lineText: str | None, captureIndices: collections.abc.Sequence[textmate_grammar.engine.oniglib.IOnigCaptureIndex] | None) str | None¶
Port of
Rule.getName(rule.ts:70-75): expands capture references in the name against the matched text; non-capturing names pass through.
- getContentName(lineText: str, captureIndices: collections.abc.Sequence[textmate_grammar.engine.oniglib.IOnigCaptureIndex]) str | None¶
Port of
Rule.getContentName(rule.ts:77-82).
- dispose() None¶
Port of
Rule.dispose(rule.ts:63); the base rule holds nothing.Subclasses override to drop their compiled-pattern caches (kept for parity; garbage collection makes it a cache-buster only).
- abstractmethod collectPatterns(grammar: textmate_grammar.engine.rules.IRuleRegistry, out: textmate_grammar.engine.regexsource.RegExpSourceList) None¶
Port of
Rule.collectPatterns(rule.ts:84) — abstract.
- abstractmethod compile(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None) textmate_grammar.engine.regexsource.CompiledRule¶
Port of
Rule.compile(rule.ts:86) — abstract.
- abstractmethod compileAG(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None, allowA: bool, allowG: bool) textmate_grammar.engine.regexsource.CompiledRule¶
Port of
Rule.compileAG(rule.ts:88) — abstract.
- class textmate_grammar.engine.rules.CompilePatternsResult¶
Port of
ICompilePatternsResult(rule.ts:91-94): the ordered rule ids of a pattern list plus upstream’s flag recording that entries were dropped (include targets that could not be resolved).
- class textmate_grammar.engine.rules.CaptureRule(location: textmate_grammar.engine.rules.ILocation, id: int, name: str | None, contentName: str | None, retokenizeCapturedWithRuleId: int)¶
Bases:
textmate_grammar.engine.rules.RulePort of
CaptureRule(rule.ts:96-120) — one entry of acapturesmap.retokenizeCapturedWithRuleIdis 0 (no nested rule) unless the capture declarespatterns, in which case it is the id of the rule compiled from them (rule.ts:476-480).Initialization
- collectPatterns(grammar: textmate_grammar.engine.rules.IRuleRegistry, out: textmate_grammar.engine.regexsource.RegExpSourceList) None¶
- compile(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None) textmate_grammar.engine.regexsource.CompiledRule¶
- compileAG(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None, allowA: bool, allowG: bool) textmate_grammar.engine.regexsource.CompiledRule¶
- class textmate_grammar.engine.rules.MatchRule(location: textmate_grammar.engine.rules.ILocation, id: int, name: str | None, match: str, captures: list[textmate_grammar.engine.rules.CaptureRule | None])¶
Bases:
textmate_grammar.engine.rules.RulePort of
MatchRule(rule.ts:122-164): a singlematchpattern with optional captures.Initialization
- collectPatterns(grammar: textmate_grammar.engine.rules.IRuleRegistry, out: textmate_grammar.engine.regexsource.RegExpSourceList) None¶
- compile(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None) textmate_grammar.engine.regexsource.CompiledRule¶
- compileAG(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None, allowA: bool, allowG: bool) textmate_grammar.engine.regexsource.CompiledRule¶
- class textmate_grammar.engine.rules.IncludeOnlyRule(location: textmate_grammar.engine.rules.ILocation, id: int, name: str | None, contentName: str | None, patterns: textmate_grammar.engine.rules.CompilePatternsResult)¶
Bases:
textmate_grammar.engine.rules.RulePort of
IncludeOnlyRule(rule.ts:166-207): a rule made purely ofincludereferences whose own (possibly missing) patterns are collected at compile time.Initialization
- collectPatterns(grammar: textmate_grammar.engine.rules.IRuleRegistry, out: textmate_grammar.engine.regexsource.RegExpSourceList) None¶
- compile(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None) textmate_grammar.engine.regexsource.CompiledRule¶
- compileAG(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None, allowA: bool, allowG: bool) textmate_grammar.engine.regexsource.CompiledRule¶
- class textmate_grammar.engine.rules.BeginEndRule(location: textmate_grammar.engine.rules.ILocation, id: int, name: str | None, contentName: str | None, begin: str, beginCaptures: list[textmate_grammar.engine.rules.CaptureRule | None], end: str | None, endCaptures: list[textmate_grammar.engine.rules.CaptureRule | None], applyEndPatternLast: bool | None, patterns: textmate_grammar.engine.rules.CompilePatternsResult)¶
Bases:
textmate_grammar.engine.rules.RulePort of
BeginEndRule(rule.ts:209-288):begin/endwith captures on both, containedpatternsandapplyEndPatternLastplacement.Initialization
- getEndWithResolvedBackReferences(lineText: str, captureIndices: collections.abc.Sequence[textmate_grammar.engine.oniglib.IOnigCaptureIndex]) str¶
Port of
getEndWithResolvedBackReferences(rule.ts:248-250): the end pattern with backreferences substituted from the begin match.
- collectPatterns(grammar: textmate_grammar.engine.rules.IRuleRegistry, out: textmate_grammar.engine.regexsource.RegExpSourceList) None¶
- compile(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None) textmate_grammar.engine.regexsource.CompiledRule¶
- compileAG(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None, allowA: bool, allowG: bool) textmate_grammar.engine.regexsource.CompiledRule¶
- class textmate_grammar.engine.rules.BeginWhileRule(location: textmate_grammar.engine.rules.ILocation, id: int, name: str | None, contentName: str | None, begin: str, beginCaptures: list[textmate_grammar.engine.rules.CaptureRule | None], while_: str, whileCaptures: list[textmate_grammar.engine.rules.CaptureRule | None], patterns: textmate_grammar.engine.rules.CompilePatternsResult)¶
Bases:
textmate_grammar.engine.rules.RulePort of
BeginWhileRule(rule.ts:290-379):beginplus awhilecondition checked at the start of every line (rule idwhileRuleId= -2), with the containedpatternscompiled as forBeginEndRule.Initialization
- getWhileWithResolvedBackReferences(lineText: str, captureIndices: collections.abc.Sequence[textmate_grammar.engine.oniglib.IOnigCaptureIndex]) str¶
Port of
getWhileWithResolvedBackReferences(rule.ts:333-335): the while pattern with backreferences substituted from the begin match.
- collectPatterns(grammar: textmate_grammar.engine.rules.IRuleRegistry, out: textmate_grammar.engine.regexsource.RegExpSourceList) None¶
- compile(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None) textmate_grammar.engine.regexsource.CompiledRule¶
- compileAG(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None, allowA: bool, allowG: bool) textmate_grammar.engine.regexsource.CompiledRule¶
- compileWhile(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None) textmate_grammar.engine.regexsource.CompiledRule¶
Port of
compileWhile(rule.ts:361-363).
- compileWhileAG(grammar: textmate_grammar.engine.rules.IRuleFactoryHelper, endRegexSource: str | None, allowA: bool, allowG: bool) textmate_grammar.engine.regexsource.CompiledRule¶
Port of
compileWhileAG(rule.ts:365-367).
- class textmate_grammar.engine.rules.IncludeReferenceKind¶
Bases:
enum.IntEnumPort of
IncludeReferenceKind(grammarDependencies.ts:247-253).Initialization
Initialize self. See help(type(self)) for accurate signature.
- Base = 0¶
- Self = 1¶
- RelativeReference = 2¶
- TopLevelReference = 3¶
- TopLevelRepositoryReference = 4¶
- class textmate_grammar.engine.rules.IncludeReference¶
Port of the
IncludeReferenceunion (grammarDependencies.ts:240-276); fields beyondkindare set per kind (scope_name/rule_name).- kind: textmate_grammar.engine.rules.IncludeReferenceKind = None¶
- textmate_grammar.engine.rules.parse_include(include: str) textmate_grammar.engine.rules.IncludeReference¶
Port of
parseInclude(grammarDependencies.ts:278-295):$base/$self,#rule(relative),scope(top level),scope#rule(top-level repository reference). Note: this is the shared include parser; the full grammarDependency machinery (task 4b) should import it from here.
- textmate_grammar.engine.rules.init_grammar(grammar: textmate_grammar.engine.rules.IRawGrammarDict, base: textmate_grammar.engine.rules.IRawRuleDict | None = None) textmate_grammar.engine.rules.IRawGrammarDict¶
Port of
initGrammar(grammar.ts:410-421): deep-copies the raw grammar and installs the$selfand$baserepository entries the include resolution relies on ($self= the top-level patterns,$base= the caller-provided base grammar’s self entry or the grammar’s own).
- class textmate_grammar.engine.rules.RuleFactory¶
Port of
RuleFactory(rule.ts:381-573).- static create_capture_rule(helper: textmate_grammar.engine.rules.IRuleFactoryHelper, location: textmate_grammar.engine.rules.ILocation, name: str | None, contentName: str | None, retokenizeCapturedWithRuleId: int) textmate_grammar.engine.rules.CaptureRule¶
Port of
createCaptureRule(rule.ts:383-387): registers the capture rule under a fresh id.
- static get_compiled_rule_id(desc: textmate_grammar.engine.rules.IRawRuleDict, helper: textmate_grammar.engine.rules.IRuleFactoryHelper, repository: textmate_grammar.engine.rules.IRawRepositoryDict) int¶
Port of
getCompiledRuleId(rule.ts:389-447): the dispatch on the raw rule’s shape, memoized on the rule dict via its internalidfield.Note on recursion: upstream writes
desc.id = idinside the factory callback, which is invoked synchronously byregisterRule— so the id is stored before any nestedgetCompiledRuleIdruns, exactly as here.
- static compile_captures(captures: textmate_grammar.engine.rules.IRawRuleDict | None, helper: textmate_grammar.engine.rules.IRuleFactoryHelper, repository: textmate_grammar.engine.rules.IRawRepositoryDict) list[textmate_grammar.engine.rules.CaptureRule | None]¶
Port of
_compileCaptures(rule.ts:449-485): builds the(CaptureRule | None)[]indexed by capture group number 1..n.
- static compile_patterns(patterns: list[textmate_grammar.engine.rules.IRawRuleDict] | None, helper: textmate_grammar.engine.rules.IRuleFactoryHelper, repository: textmate_grammar.engine.rules.IRawRepositoryDict) textmate_grammar.engine.rules.CompilePatternsResult¶
Port of
_compilePatterns(rule.ts:487-572): compiles the raw pattern list to rule ids, preserving order, resolvingincludeentries and dropping rules whose include chain resolves to nothing instantiated.
- textmate_grammar.engine.rules.create_rule(desc: textmate_grammar.engine.rules.IRawRuleDict, helper: textmate_grammar.engine.rules.IRuleFactoryHelper, repository: textmate_grammar.engine.rules.IRawRepositoryDict) int¶
Mirror of
RuleFactory.getCompiledRuleId(task-card spelling); preferRuleFactory.get_compiled_rule_id().