textmate_grammar.engine.regexsource

Port of the RegExpSource machinery from vscode-textmate’s src/rule.ts.

This module ports from rule.ts (version 9.3.2, the pinned reference clone): IRegExpSourceAnchorCache (rule.ts:575-580), RegExpSource (rule.ts:582-735), RegExpSourceList (rule.ts:744-856) and CompiledRule (rule.ts:858-894), plus the two regex-source helpers they depend on from src/utils.ts: escapeRegExpCharacters (utils.ts:159-161) and the RegexSource class (utils.ts:66-115, $n/${n:/command} capture expansion used for scope names).

Semantics are 1:1 with the reference; naming is pythonic and every non-obvious piece carries a comment referencing the ported lines of rule.ts / utils.ts.

Module Contents

Classes

RegexSource

Port of utils.ts’s static RegexSource class (utils.ts:66-115): expands $n and ${n:/downcase|upcase} captures in scope names against the text a pattern captured.

AnchorCache

Port of IRegExpSourceAnchorCache (rule.ts:575-580): the four resolved variants of a source with anchors, named after which anchors remain allowed. A1 = \A kept, G1 = \G kept; disallowed anchors were rewritten to the U+FFFF sentinel, which can virtually never occur in scanned text.

RegExpSource

Port of rule.ts’s RegExpSource (rule.ts:582-735): a pattern source from the grammar, preprocessed on construction (\z workaround), detecting whether it uses anchors (\A/\G) or \n backreferences and able to resolve the anchor variants via its anchor cache.

IFindNextMatchResult

Port of IFindNextMatchResult (rule.ts:896-899): the matched pattern’s rule id and its capture indices.

CompiledRule

Port of rule.ts’s CompiledRule (rule.ts:858-894): a set of pattern sources compiled once into an OnigScanner, resolving matches back to the rule ids the patterns belong to.

RegExpSourceList

Port of rule.ts’s RegExpSourceList (rule.ts:744-856): a growing list of RegExpSource items compiled lazily into CompiledRule objects.

Functions

escapeRegExpCharacters

Port of utils.ts escapeRegExpCharacters (utils.ts:159-161) — escapes the characters that are special in regular expressions in the given string.

API

textmate_grammar.engine.regexsource.escapeRegExpCharacters(value: str) → str

Port of utils.ts escapeRegExpCharacters (utils.ts:159-161) — escapes the characters that are special in regular expressions in the given string.

class textmate_grammar.engine.regexsource.RegexSource

Port of utils.ts’s static RegexSource class (utils.ts:66-115): expands $n and ${n:/downcase|upcase} captures in scope names against the text a pattern captured.

static hasCaptures(regex_source: str | None) → bool

Port of RegexSource.hasCaptures (utils.ts:68-75).

static replaceCaptures(regex_source: str, capture_source: str, capture_indices: collections.abc.Sequence[textmate_grammar.engine.oniglib.IOnigCaptureIndex]) → str

Port of RegexSource.replaceCaptures (utils.ts:77-115): replaces each $n/${n:/command} occurrence with the content capture n covers in capture_source, applying the optional case command. References to captures that don’t exist are left verbatim. Unmatched captures (oniguruma -1 offsets) contribute ‘’ via the empty substring slice.

class textmate_grammar.engine.regexsource.AnchorCache

Port of IRegExpSourceAnchorCache (rule.ts:575-580): the four resolved variants of a source with anchors, named after which anchors remain allowed. A1 = \A kept, G1 = \G kept; disallowed anchors were rewritten to the U+FFFF sentinel, which can virtually never occur in scanned text.

a0_g0: str = None
a0_g1: str = None
a1_g0: str = None
a1_g1: str = None
class textmate_grammar.engine.regexsource.RegExpSource(reg_exp_source: str, rule_id: int)

Port of rule.ts’s RegExpSource (rule.ts:582-735): a pattern source from the grammar, preprocessed on construction (\z workaround), detecting whether it uses anchors (\A/\G) or \n backreferences and able to resolve the anchor variants via its anchor cache.

Initialization

Port of the constructor (rule.ts:590-639).

Walks the source once: \z (oniguruma-only, kept in tmlanguage sources) is rewritten to a JS-compatible equivalent and occurrences of \A/\G mark the source as anchored. Note that escaped anchors inside character classes are flagged too — upstream does the same (rule.ts:600-613).

source: str = None
rule_id: int = None
has_anchor: bool = None
has_back_references: bool = None
clone() → textmate_grammar.engine.regexsource.RegExpSource

Port of RegExpSource.clone (rule.ts:641-643).

set_source(new_source: str) → None

Port of RegExpSource.setSource (rule.ts:645-654): rebuilds the anchor cache from the new source — but only picks up has_anchor changes on the next construction (upstream keeps the old flag here too).

resolve_back_references(line_text: str, capture_indices: collections.abc.Sequence[textmate_grammar.engine.oniglib.IOnigCaptureIndex]) → str

Port of RegExpSource.resolveBackReferences (rule.ts:656-664): substitutes \n-style backreferences in the source with the regexp-escaped content of capture n from the (begin) match, e.g. for end patterns echoing the begun content. Unmatched or out-of-range captures substitute ‘’ — upstream capturedValues[...] || '' (rule.ts:662).

resolve_anchors(allow_a: bool, allow_g: bool) → str

Port of RegExpSource.resolveAnchors (rule.ts:716-734): the source variant with the disallowed anchors neutralized; non-anchored sources are returned verbatim (rule.ts:717-719).

class textmate_grammar.engine.regexsource.IFindNextMatchResult

Port of IFindNextMatchResult (rule.ts:896-899): the matched pattern’s rule id and its capture indices.

rule_id: int = None
capture_indices: tuple[textmate_grammar.engine.oniglib.IOnigCaptureIndex, ...] = None
class textmate_grammar.engine.regexsource.CompiledRule(onig_lib: textmate_grammar.engine.oniglib.IOnigLib, reg_exps: collections.abc.Sequence[str], rules: collections.abc.Sequence[int])

Port of rule.ts’s CompiledRule (rule.ts:858-894): a set of pattern sources compiled once into an OnigScanner, resolving matches back to the rule ids the patterns belong to.

Initialization

dispose() → None

Port of CompiledRule.dispose (rule.ts:865-869).

find_next_match_sync(string: str | textmate_grammar.engine.oniglib.OnigString, start_position: int, options: textmate_grammar.engine.oniglib.FindOption = FindOption.NONE) → textmate_grammar.engine.regexsource.IFindNextMatchResult | None

Port of CompiledRule.findNextMatchSync (rule.ts:879-893): scans and maps the matched pattern index back to IFindNextMatchResult.

class textmate_grammar.engine.regexsource.RegExpSourceList

Port of rule.ts’s RegExpSourceList (rule.ts:744-856): a growing list of RegExpSource items compiled lazily into CompiledRule objects.

For lists with anchors, the compiled scanners are cached in _anchor_cache — upstream carries the four A0_G0..``A1_G1`` fields (rule.ts:737-742, 751-760); they are keyed here by (allow_a, allow_g).

Initialization

push(item: textmate_grammar.engine.regexsource.RegExpSource) → None

Port of RegExpSourceList.push (rule.ts:790-793).

unshift(item: textmate_grammar.engine.regexsource.RegExpSource) → None

Port of RegExpSourceList.unshift (rule.ts:795-798).

set_source(index: int, new_source: str) → None

Port of RegExpSourceList.setSource (rule.ts:804-810): busts the caches when the source of an item actually changes.

compile(onig_lib: textmate_grammar.engine.oniglib.IOnigLib) → textmate_grammar.engine.regexsource.CompiledRule

Port of RegExpSourceList.compile (rule.ts:812-818): compiles the plain sources once and caches them.

compile_ag(onig_lib: textmate_grammar.engine.oniglib.IOnigLib, allow_a: bool, allow_g: bool) → textmate_grammar.engine.regexsource.CompiledRule

Port of RegExpSourceList.compileAG (rule.ts:820-850): non-anchored lists share the plain compile (rule.ts:821-822); anchored lists get one cached compiled variant per allowed anchor combination.