OPENNLP-1933: Regex removal (5a/10): Simplify dictionary and filename parsing

[krickert]

Removes regex operations from Leipzig filename checks, Morfologik dictionary suffix handling, and frequency-dictionary column parsing. We had a complicated ticket on the fifth of ten regex removal, so this is part of that effort per @rzo1's guidance

Details:

• Leipzig filenames retain their three-letter lowercase ASCII language prefix, and Morfologik replaces only a trailing .info suffix.
• Dictionary columns use runs of spaces and tabs while preserving Unicode terms and the integer representations supported by Long.parseLong, including Arabic-Indic and fullwidth digits.
• Negative counts and overflow remain invalid, and Unicode whitespace-only lines are skipped.
• Regression tests cover filename boundaries, separators, Unicode terms, counts, and malformed input; the manual documents the supported formats and links the SymSpell dictionary specification.

AI assisted text:

JIRA: OPENNLP-1933. Part 5a/10 of the regex-removal work; targets main independently.

Validation on Linux with JDK 25:

• Affected formats, Morfologik, and spellcheck suites and their reactor dependencies pass verify, including 74 tests across the three directly affected test classes.
• Javadoc and the HTML manual build pass; existing warnings are confined to unchanged source.
• Local Maven runs used -Dopennlp.forkCount=1 -Drat.skip=true. Existing unrelated test skips remain; hosted CI has not run for this split.

[rzo1]

Blocking

1. spellcheck.xml:317-321. The manual now says counts may use Arabic-Indic and fullwidth digits and an optional sign. That's only what Long.parseLong happens to accept, and once it's in the manual, users will rely on it and we'll have to keep supporting it. SymSpell dictionaries use ASCII digits. Either restrict the count to ASCII digits (StringUtil.isAsciiDigit exists) or drop the sentence and just say "a nonnegative integer".

Minor

• FrequencyDictionaryLoader.java:254 vs :240, :273, :287. isSkippable became non-static, while stripBom, parseCount and saturatedAdd stay static. Either drop that change or align all.
• FrequencyDictionaryLoader.java:274. raw.trim() isn't needed any more. A column can no longer contain a space or tab, so trim() only strips control characters, and the<TAB>\u00015 is accepted with count 5. Drop the trim().
• FrequencyDictionaryLoader.java:214-238. splitColumns is the same "split and skip empty fields" loop as splitClassPath (#1278) and EventFields.split (#1321). Please add one helper to StringUtil and use it in all three. COLUMN_TAB/COLUMN_SPACE (:73-74) then become arguments to that helper; right now they're the only constants without Javadoc.
• FrequencyDictionaryLoader.java:240. BOM/stripBom duplicates the byte-order-mark handling in #1277 (AbstractDL, JsonScan) and HunspellDictionary. A shared StringUtil.stripByteOrderMark would cover all of them.
• spellcheck.xml:306-313. Too much detail for the manual: String.strip(), vertical tab, form feed, em space, no-break space. Say "columns are separated by TABs or runs of spaces; blank lines and lines starting with # are skipped", keep the SymSpell link, and drop the rest.
• langdetect.xml:181-184. Same here: "The language is taken from the first three characters of the file name, the lowercase ISO 639-3 code, e.g. eng_news_2005_10K-sentences.txt." is enough. The list of rejected prefixes can go.
• Description. "Hosted CI has not run for this split" is outdated, since CI is green. Please also remove "We had a complicated ticket…" and the "AI assisted text" block, and keep only what changes for users.

[krickert]

I replaced regex in Leipzig file name checks, Morfologik suffix handling and frequency dictionary parsing with direct character checks.

It matches only a trailing .info suffix. Split frequency columns on tabs or spaces, preserve Unicode terms and digits, and report line numbers for invalid counts.

I added tests for file names, separators, Unicode input and invalid counts.

Documented the dictionary formats in the manual.

LMK if I need to do anything else.