OPENNLP-1932: Regex removal (4/10): Match model finder wildcards without regular expressions

[krickert]

DirectoryModelFinder and SimpleClassPathModelFinder no longer turn the search mask into a regular expression. I added the package-private WildcardMatcher compares the mask with the decoded file part of each classpath URL.

AI summary:

• * matches any run of characters, including none. ? matches exactly one character, where a character outside the Basic Multilingual Plane counts as one. Every other character, including ., [, \, % and +, is literal. There is no escape character.
• The match is case-sensitive and covers the whole file part. The finders put * in front of the mask, so custom-opennlp-model.jar finds that jar in any directory.
• URL paths are decoded before matching, so a mask uses the real file name: my models/*.bin rather than my%20models/*.bin. A class loader URL that is not a valid URI, for example one with an unescaped space, is matched on its file part as is.
• The matcher keeps one backtracking point, so the worst case is O(n*m) with no recursion. *a*a*a*b against 100,000 as is rejected in milliseconds.

Breaking change: the protected asRegex and matchesPattern methods of AbstractClassPathModelFinder are removed. Subclasses call matchesWildcard(url, wildcard) instead. The model loading chapter describes the mask rules and the migration.

Other changes:

• A null jarModelPrefix throws IllegalArgumentException, like the other argument checks.
• JAR entry URIs are escaped, so entries with spaces, # or ? in their names can be opened.
• The class path is split with a new StringUtil.splitNonEmpty(CharSequence, char...), which splits on literal characters and skips empty fields. OPENNLP-1933: Regex removal (5a/10): Simplify dictionary and filename parsing #1320 and OPENNLP-1989: Regex removal (5b/10): Preserve feature identities in event parsing #1321 can use it too.

Testing

• 324 model-resolver tests and the opennlp-api tests pass locally. Checkstyle and forbiddenapis are clean.
• The tests cover the matcher (accept and reject cases, supplementary characters, the pathological mask), the finder base class, real jars with spaces, non-ASCII names and URI metacharacters, URLs that are not valid URIs, and splitNonEmpty against a regex split on random input.

[krickert]

I didn't want to be too pedantic, but there is a similar fix we did a few days ago on #1308 , but left it out because it would've bloated the ticket. There might be some logic we can unify there. I will make a follow-up issue to investigate it.

PR content changed

[krickert]

I think this is ready for review, I have everything noted about the repeat logic - it's not much and shouldn't block this merge.

[rzo1]

Thanks for the update. The switch to decoded paths and the rewritten docs and description address points 2 and 3 of my earlier review. Most of the code-level points are still open, and we've decided on point 1.

Blocking

1. Remove java.util.regex.Pattern from AbstractClassPathModelFinder. With matchesWildcard in place, the built-in finders no longer use regex. The rewritten asRegex is not the old contract either ((?s) plus quoted runs), so keeping it as a deprecated method doesn't give subclasses compatibility. It only keeps Pattern in production code, which #1282 would then have to suppress. 3.0.0 is a major release, so remove the regex API instead of deprecating it:
   • Delete asRegex(String) (:186-209), appendQuoted (:213), matchesPattern(URL, Pattern) (:232-243), the DOT_ALL and PATTERN_MUST_NOT_BE_NULL constants (:54-55), and the java.util.regex.Pattern import (:34).
   • Delete the tests that only exercise them: asRegexGlobs and its test, the matchesPattern assertions, LegacyFinder and its test, and asRegex(null) (GlobMatcherTest.java:221, :284-337, :346-381, :421).
   • Add a line to the manual/release notes: "Since 3.0, asRegex and matchesPattern are removed; subclasses use matchesWildcard(URL, String)."
   • Drop the matching suppression and comment from #1282.

Minor

• AbstractClassPathModelFinder.java:72. The constructor still uses Objects.requireNonNull (NPE), while matchesWildcard throws IAE. Please align it to IAE.
• GlobMatcher.java:44-52.
  • The null checks repeat those in matchesWildcard. Validate at the protected boundary only, and use the constants there instead of inline literals.
  • codePoints().toArray() allocates two int[] per call, twice per jar entry. Walk the strings with codePointAt/Character.charCount instead.
• GlobMatcher.java:22. "matches exactly one character" should say one Unicode code point, as matchesWildcard does.
• GlobMatcher.java. "Glob" suggests PathMatcher syntax ({a,b}, [..], escapes), while the API and docs say "wildcard". Rename it WildcardMatcher, or fold it into a private instance method of AbstractClassPathModelFinder.
• SimpleClassPathModelFinder.java:177. No need for splitClassPath to be static. The same "split and skip empty fields" loop also appears in #1320 (FrequencyDictionaryLoader.splitColumns) and #1321 (EventFields.split). Please add one helper to StringUtil and use it in all three.
• GlobMatcherTest.java:196. Once the regex tests are gone, move the remaining newProbeFinder/matchesWildcard tests to an AbstractClassPathModelFinderTest.
• GlobMatcherTest.java. Pin the linear worst case, the real improvement here: for example assertTimeoutPreemptively with *a*a*a*b against 100k as. The old regex didn't finish on inputs like that.
• DirectoryModelFinderTest.java:107. The hard-coded 4 duplicates the number of opennlp-models-* test dependencies. Assert against the number of jars copied in @BeforeAll.
• Title. Please use release-notes language, e.g. "OPENNLP-1932: Model finders treat regex metacharacters in jar and resource masks literally".

[krickert]

Thanks for the review. I have the pattern ready in another unpushed PR. I'll add it to this ticket - it's manageable.

I'll do another review of the other dependent tickets to isolate the Pattern dependency remainders for future tickets.

[krickert]

Thanks for the review. I pushed changes for all of it. AI provided a better summary than myself:

• Removed asRegex and matchesPattern outright, along with their tests. model-loading.xml now says they are gone in 3.0 and that subclasses should call matchesWildcard(url, "*.bin").
• A null jarModelPrefix now throws IllegalArgumentException, the same as matchesWildcard.
• The matcher walks both strings by code point in place, so it no longer allocates two int[] per call. The Javadoc now says "code point" throughout.
• Renamed GlobMatcher to WildcardMatcher.
• splitClassPath now delegates to a new StringUtil.splitNonEmpty(CharSequence, char...). OPENNLP-1933: Regex removal (5a/10): Simplify dictionary and filename parsing #1320 and OPENNLP-1989: Regex removal (5b/10): Preserve feature identities in event parsing #1321 can use the same helper instead of their own loops.
• Moved the matchesWildcard tests into a new AbstractClassPathModelFinderTest. WildcardMatcherTest now covers only the matcher.
• Added testPathologicalWildcardRejectsQuickly, which runs *a*a*a*b against 100k as.
• DirectoryModelFinderTest now counts the jars it copies instead of assuming 4.
• A class loader URL that is not a valid URI, for example one with an unescaped space, is matched on its file part as is instead of failing the scan.
• Shortened the model loading chapter paragraph, and put the removal note in its own paragraph.
• Updated the title.

[rzo1]

Thanks for the update, most of my points are addressed. I tried to push the remaining small fixes directly, but GitHub doesn't allow maintainer edits on PR branches of organization-owned forks (ai-pipestream/opennlp), so here is the patch instead.

It does three things:

• WildcardMatcher.matches no longer repeats the null checks, since matchesWildcard (its only caller) already validates. I dropped testMatchesRejectsNull as well.
• SimpleClassPathModelFinder.splitClassPath is no longer static, and it leaves the null check to StringUtil.splitNonEmpty.
• README.md: the "Migrating from 2.x to 3.x" section claimed there are no breaking changes. It now lists the removal of asRegex / matchesPattern under a new "Breaking changes" section.

Apply it on top of your branch with git am: save the block below as 1278.patch, then run git am 1278.patch. Please don't force-push over it without applying it first.

1278.patch

From 175023a8ab52ab24e12d12fc92725334903db4b9 Mon Sep 17 00:00:00 2001
From: Richard Zowalla <rzo1@apache.org>
Date: Fri, 25 Sep 2026 21:05:52 +0200
Subject: [PATCH] OPENNLP-1932: Drop duplicated null checks and document the
 breaking change

- WildcardMatcher.matches no longer repeats the null checks done by
  AbstractClassPathModelFinder.matchesWildcard, its only caller.
- SimpleClassPathModelFinder.splitClassPath is no longer static and
  relies on StringUtil.splitNonEmpty for argument validation.
- README: list the removal of asRegex and matchesPattern under
  breaking changes in the 2.x to 3.x migration section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 README.md                                        |  6 +++++-
 .../opennlp/tools/models/WildcardMatcher.java    |  7 -------
 .../simple/SimpleClassPathModelFinder.java       |  5 +----
 .../tools/models/WildcardMatcherTest.java        | 16 ----------------
 .../simple/SimpleClassPathModelFinderTest.java   |  6 +++---
 5 files changed, 9 insertions(+), 31 deletions(-)

diff --git a/README.md b/README.md
index 5a40f38e4..c0f5e0570 100644
--- a/README.md
+++ b/README.md
@@ -120,7 +120,7 @@ For more details please check our [documentation](https://opennlp.apache.org/doc
 
 ## Migrating from 2.x to 3.x
 
-The 3.x release line of Apache OpenNLP introduces **no** known breaking changes but modularizes the project for better usage as a library and to support future extensibility.
+The 3.x release line of Apache OpenNLP introduces only a few breaking changes, listed below, but modularizes the project for better usage as a library and to support future extensibility.
 The core API remains stable and compatible with 2.x, but the project structure has been reorganized into multiple modules.
 
 That means, that you can continue to use the previous `opennlp-tools` artifact as a dependency. However, we strongly recommend to switch to the new modular structure 
@@ -129,6 +129,10 @@ and import only the components you need, which will result in a smaller dependen
 Only `opennlp-runtime` needs to be added as a dependency, and you can add additional modules (e.g. `opennlp-ml-maxent`, `opennlp-model-resolver`, etc.) as required by your project.
 For users of the traditional CLI toolkit, nothing changes with the 3.x release line. CLI usage remains stable as described in the [project's dev manual](https://opennlp.apache.org/docs/).
 
+### Breaking changes
+
+- `AbstractClassPathModelFinder`: the protected `asRegex` and `matchesPattern` methods were removed. Custom model finders call `matchesWildcard(url, "*.bin")` instead ([OPENNLP-1932](https://issues.apache.org/jira/browse/OPENNLP-1932)).
+
 ### Thread safety
 
 Starting with 3.0.0, the core `*ME` classes (`POSTaggerME`, `TokenizerME`, `SentenceDetectorME`, `ChunkerME`, `LemmatizerME`, `NameFinderME`) are thread safe and a single instance can be shared across threads. This eliminates the need to pool or recreate ME instances per thread. The legacy `ThreadSafe*ME` wrappers from 2.x still work but are now deprecated; existing code does not need to change to upgrade.
diff --git a/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/WildcardMatcher.java b/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/WildcardMatcher.java
index 0b22b1090..94db098d1 100644
--- a/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/WildcardMatcher.java
+++ b/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/WildcardMatcher.java
@@ -42,15 +42,8 @@ final class WildcardMatcher {
    * @param input The text to test. Must not be {@code null}.
    * @return {@code true} if {@code input} matches {@code wildcard} from start to end,
    *     {@code false} otherwise.
-   * @throws IllegalArgumentException Thrown if {@code wildcard} or {@code input} is {@code null}.
    */
   static boolean matches(String wildcard, String input) {
-    if (wildcard == null) {
-      throw new IllegalArgumentException("wildcard must not be null");
-    }
-    if (input == null) {
-      throw new IllegalArgumentException("input must not be null");
-    }
     final int wildcardLength = wildcard.length();
     final int inputLength = input.length();
     int wi = 0;
diff --git a/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/simple/SimpleClassPathModelFinder.java b/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/simple/SimpleClassPathModelFinder.java
index d5df05d35..9ab57839d 100644
--- a/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/simple/SimpleClassPathModelFinder.java
+++ b/opennlp-core/opennlp-model-resolver/src/main/java/opennlp/tools/models/simple/SimpleClassPathModelFinder.java
@@ -175,10 +175,7 @@ public class SimpleClassPathModelFinder extends AbstractClassPathModelFinder imp
    * @return The non-empty class path entries in order.
    * @throws IllegalArgumentException Thrown if {@code classPath} is {@code null}.
    */
-  static String[] splitClassPath(String classPath, boolean isWindows) {
-    if (classPath == null) {
-      throw new IllegalArgumentException("classPath must not be null");
-    }
+  String[] splitClassPath(String classPath, boolean isWindows) {
     return StringUtil.splitNonEmpty(classPath,
         isWindows ? CLASSPATH_SEPARATOR_WINDOWS : CLASSPATH_SEPARATOR_UNIX);
   }
diff --git a/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/WildcardMatcherTest.java b/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/WildcardMatcherTest.java
index ed7653417..92df58bb6 100644
--- a/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/WildcardMatcherTest.java
+++ b/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/WildcardMatcherTest.java
@@ -218,22 +218,6 @@ public class WildcardMatcherTest {
     Assertions.assertFalse(WildcardMatcher.matches(wildcard, rejected));
   }
 
-  private static Stream<Arguments> nullInputs() {
-    return Stream.of(
-        Arguments.of(null, "a"),
-        Arguments.of("a", null),
-        Arguments.of(null, null));
-  }
-
-  /**
-   * Checks that a null wildcard or input fails fast instead of matching.
-   */
-  @ParameterizedTest
-  @MethodSource("nullInputs")
-  void testMatchesRejectsNull(String wildcard, String input) {
-    Assertions.assertThrows(IllegalArgumentException.class, () -> WildcardMatcher.matches(wildcard, input));
-  }
-
   /**
    * Checks a wildcard that makes a backtracking regular expression take polynomial time,
    * {@code .*a.*a.*a.*b} against a long run of {@code a}. The matcher keeps one backtracking
diff --git a/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/simple/SimpleClassPathModelFinderTest.java b/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/simple/SimpleClassPathModelFinderTest.java
index 48750e70f..5f4dcab60 100644
--- a/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/simple/SimpleClassPathModelFinderTest.java
+++ b/opennlp-core/opennlp-model-resolver/src/test/java/opennlp/tools/models/simple/SimpleClassPathModelFinderTest.java
@@ -69,7 +69,7 @@ public class SimpleClassPathModelFinderTest extends AbstractClassPathFinderTest
   @ParameterizedTest
   @MethodSource("unixClassPaths")
   void testSplitClassPathUnix(String classPath, String[] expected) {
-    Assertions.assertArrayEquals(expected, SimpleClassPathModelFinder.splitClassPath(classPath, false));
+    Assertions.assertArrayEquals(expected, new SimpleClassPathModelFinder().splitClassPath(classPath, false));
   }
 
   private static Stream<Arguments> windowsClassPaths() {
@@ -101,12 +101,12 @@ public class SimpleClassPathModelFinderTest extends AbstractClassPathFinderTest
   @ParameterizedTest
   @MethodSource("windowsClassPaths")
   void testSplitClassPathWindows(String classPath, String[] expected) {
-    Assertions.assertArrayEquals(expected, SimpleClassPathModelFinder.splitClassPath(classPath, true));
+    Assertions.assertArrayEquals(expected, new SimpleClassPathModelFinder().splitClassPath(classPath, true));
   }
 
   @Test
   void testSplitClassPathRejectsNull() {
     Assertions.assertThrows(IllegalArgumentException.class,
-        () -> SimpleClassPathModelFinder.splitClassPath(null, false));
+        () -> new SimpleClassPathModelFinder().splitClassPath(null, false));
   }
 }
-- 
2.55.0

Side note for future PRs: it would make things easier if you pushed your branches to the ASF repo (apache/opennlp) instead of the ai-pipestream fork, since you have write access there. Committers could then push small fixes like this one directly instead of going back and forth with patches.

[jzonthemtn]

I'm +1 once @rzo1's patch is applied and the merge conflicts are resolved.

[krickert]

Applied your patch and rebased onto main. The one conflict was StringUtil, where main now has the byte order mark helpers next to splitNonEmpty; both are kept.