Speed up tokenization ~25%, add perf bench, clean up lint and the stale gold file - #64

Speed up tokenization ~25%, add perf bench, clean up lint and the stale gold file#64

vthorsteinsson merged 3 commits into

mastermideind/Tokenizer:masterfrom

perf/regex-precompile-and-cleanupmideind/Tokenizer:perf/regex-precompile-and-cleanupCopy head branch name to clipboard