GH-1480: Read Spanish head rules without regular expressions

[krickert]

AncoraSpanishHeadRules matches head-rule tags with a small matcher instead of java.util.regex. Each tag is parsed once when the rules load into literals, * for zero or more characters and the character choices of the AnCora tagset ([MAS], [CS], [IS], [12], [AC]), and is matched without backtracking. The class has no @SuppressForbidden members left.

Rule files start with the header "# opennlp-ancora-head-rules 2" and write tags without regex escapes; the bundled es-head-rules files use it. A file without the header is read as the regular expression format of earlier releases, with dots and dollar signs escaped once or twice, and converted. Other regex constructs, unknown character choices, a direction other than 0 or 1, a repeated constituent type and blank lines fail to load with the line number and the reason.

Two compatibility points, also in the parser chapter:

• An escaped dot such as GRUP\\.VERB in Spanish models trained before 3.0 matched no constituent type. It now matches GRUP.VERB, so those models choose different heads at parse time than in training and need retraining.
• OpenNLP 2.x cannot read the version 2 format, so Spanish parser models trained with this release do not load there.

Tests: the matcher, rule and constituent priority, the longest pattern and a long near miss, both legacy escape forms, the bundled rules in both earlier forms read through HeadRulesSerializer, each dotted tag of the bundled rules, and each rejected rule with its message.

An eval build is needed before merge, since the bundled Spanish head rules change.

Fixes #1480

[rzo1]

Hi @krickert - thanks for the work here. Playing devil's advocate: I'm not sure we need this PR.

• Head-rule tags are user-supplied regular expressions, which is exactly the case #1348 allows with @SuppressForbidden. The guard already passes with that suppression, and the escaping bug in the bundled rules is fixed by #1461.
• The new rule format (# opennlp-ancora-head-rules 2) is stored in trained models, so Spanish parser models trained with 3.0 would no longer load in 2.x.
• Converting legacy rule files changes how existing models pick heads (GRUP\\.VERB suddenly matches), so models trained before 3.0 would need retraining. #1461 deliberately avoided that.
• Custom rule files that use other regex constructs would stop loading.
• No measured performance or correctness problem is addressed, and we would maintain a small regex engine of our own instead of java.util.regex.

I'd suggest closing this, or deferring it beyond 3.0.0. If the clearer load errors are the main benefit, we could add line-numbered validation on top of the existing regex matching, without a new format version. WDYT?

[krickert]

Hi @krickert - thanks for the work here. Playing devil's advocate: I'm not sure we need this PR.

• Head-rule tags are user-supplied regular expressions, which is exactly the case OPENNLP-1935: Regex removal (10/10): Forbid java.util.regex in production code #1348 allows with @SuppressForbidden. The guard already passes with that suppression, and the escaping bug in the bundled rules is fixed by GH-1460: Match escaped tags in the bundled Spanish head rules #1461.
• The new rule format (# opennlp-ancora-head-rules 2) is stored in trained models, so Spanish parser models trained with 3.0 would no longer load in 2.x.
• Converting legacy rule files changes how existing models pick heads (GRUP\\.VERB suddenly matches), so models trained before 3.0 would need retraining. GH-1460: Match escaped tags in the bundled Spanish head rules #1461 deliberately avoided that.
• Custom rule files that use other regex constructs would stop loading.
• No measured performance or correctness problem is addressed, and we would maintain a small regex engine of our own instead of java.util.regex.

I'd suggest closing this, or deferring it beyond 3.0.0. If the clearer load errors are the main benefit, we could add line-numbered validation on top of the existing regex matching, without a new format version. WDYT?

I'll close it. I was looking under covers and looking for any regex that might come out of the woodwork and found 2 PRs, this is one of them. Let's see how the other one does...

[krickert]

Closing, not needed