Implement component-based typing for the MathJax variable

[dpvc]

This PR implements a mechanism for providing Typescript types for the MathJax global object. This is complicated by several factors. First, the MathJax object is used for initially for configuration and, then is altered by the startup process to include the appropriate commands based on the input and output jax that are loaded, and the types are different for those two uses. Second, the final form of the MathJax object depends on the components that are loaded, and Typescript doesn't know this at compile time (the loader.load array gives the components, by their code isn't loaded until run time), and so the typing needed for the initial MathJax as a configuration object can't be determined (the loader.load is part of that object, but the types need to be known before that object's value is given).

To handle these difficulties, you need to tell MathJax what components you are going to load as parameters to a type that can be used to specify the MathJax object, as well as list them in the ladder.load array. It is possible to list them only once, however, as shown in the examples that I will post in separate comments below. It is probably best to look at these first, before diving into the code changes, in order to see how the typing are to be used in practice.

The typing mechanism can be used in all four of the main ways of using MathJax illustrated in the MathJax-demos-node repository: the simple node-only method of using node-main, the component-based approach, the direct linking to MathJax modules, and the mixed direct-component approach, though each uses its own technique. They are all based on the same typing code, however, just used in different ways.

The typing works by adding explicit typing for the options objects for all the components that allow option settings in the MathJax configuration object. That involves nearly all the components, and so this PR touches a lot of files, and adds some new .d.ts files for the component definition files. Fortunately, however, most of the changes follow a common pattern, so that should help with the review.

The changes in the a11y and ui directories are the most complicated (as they are the ones that have to extend the MathDocument class and its options), so it is probably best to start with the core directory, with the input and output jax, and the MathDocument first. Then the input jax subclasses, their support files, and the TeX and MathML extensions. Then the output jax, and finally the a11y and ui components.

The main idea is to export an OPTIONS type (e.g., TEX_OPTIONS, CONFIGMACROS_OPTIONS, CHTML_OPTIONS, etc.) that specify the types for the options of the class that is being defined, then a const options: CHTML_OPTIONS = {...} (for example) that lists the default options. That is done as a separate object so that full type checking is performed (if it was done as public OPTIONS = { ... } as CHTML_OPTIONS, that would flag any incorrect or missing options, but would not flag extra ones, and we can't do public OPTIONS: CHTML_OPTIONS = {...} as the options from the parent class are given through ...Base.OPTIONS, and so aren't listed explicitly, and get flagged. As a separate object, it must match the option types exactly, and we can either use it directly as the OPTIONS value, or use ...options to include it as part of a larger OPTIONS object. (You'll see how it works in the code below).

For the various Handler objects, like the a11y and ui files that extend the MathItem and MathDocument classes, the situation is a little more complicated, as they have to add their options into the existing ones for the document class that they extend, and it is not always clear what that base class is going to be. So, for example, the menu code needs to be aware of the explorer options as well as the complexity options, so it has to build on several other document classes. For these, ther his an OPTIONS type that lists the new option types for the updated MathDocument subclass, with an associated options that gives the default values for those options (fully checked), as usual, plus an additional interface that extends the base document class's options with the new OPTIONS, and that is used to set the subclass' options property's type. A bit complicated, but it does the trick.

The SRE and A11Y options pose a similar challenge, as they get extended in several cases as well. A similar process is used (newline the new options, then extends an interface with the new options being added to the old).

The building of the MathJax object typings is done using the files in the new ts/stypes directory. The Types.ts file has the main machinery for combing the various types for the individual components into a larger single type that is what MathJax needs based on the components being loaded. Components get a COMPONENT_DEF that describes the configuration options and any properties that get added to the MathJax object, plus whether the component defines an input or output jax (since those produce conversion methods in MathJax). These objects get combined in the type2mjx types near the bottom of the file. This has some fancy type definitions to handle things like creating the input2outout conversion functions (which rely on both the input and output components), and turing the config options from being required pro0erties into optional ones, and putting everything into the correct place in the final MathJax object type definition. I'm actually astonished that this can be made to work.

The Components.ts file creates a type that lists all the components and created the COMPONENT_DEF object for each. This file could potentially be generated automatically as part of the MathJax build process, but currently that isn't being done. These component definitions are combined later to create the type for the MathJax object. That is done using the definitions in the mjx.ts file. That allows you to use a list of component names to produce the combination of the component definitions for those types, and then use the utilities in Types.ts to merge those into the final MathJax type.

Because the types depend on the DOM node types (the N, T, and D template values in the code), there is a ts/types/dom directory that has dom-specific versions of the typing macros that include the proper DOM types. The html.ts file has the ones for in browser use (or with one of the adaptors that uses HTMLElements, like the jsdom and linkedom adaptors), and lite.ts for the LiteDOM adaptor. That means that the results of the output conversions, for example, will be properly typed, and things like MathJax.chtmlStylesheet() will return an HTMLElement or LiteElement appropriately (rather than any).

There is a lot of pretty complicated Typescript in these files that we may want to look at together. I will write up more details later, but want to get this PR posted first. Look to the example files below to see how this can be used in practice.

Using MathJax in a Typescript project can involve loading MathJax component file (either from the components directory for the source versions, or from bundle for the packed versions), typescript will complain about the fact that there are no .d.ts files for these .js files. So this PR includes new .d.ts files for all the .js files in components/mjs, and modifies the build tools to generate .d.ts files in the bundle directory. Very few of the components/mjs files actually export anything, so most of those .d.ts files are just export {} in order to appease Typescript, but a few of them do include the actual exports for those files. The only really interesting one is for components/mjs/node-main, which exports the init() function. This is defined to take the list of components that you are loading and return the MathJax object properly typed for those, so that makes that node-only "simple" approach very straight-forward.

[dpvc]

Example of "simple" use of MathJax in typescript (node-only)

import { init } from '@mathjax/src';

const components = ['input/tex'] as const;
type COMPONENTS = (typeof components)[number];

const MathJax = (await init<COMPONENTS>({
  loader: { load: [...components] },
}));

console.log(await MathJax.tex2mmlPromise(process.argv[2] || ''));

Here, the components object is used to create the list of components that are to be loaded (just one in this case) that gets passed to init<...>, and also sets the loader.load array. That way, we are consistent about the twp places this data is used.

Note that MathJax is properly typed, so MathJax.tex2mmlPromise() is known and has proper typings. If you misspell it, or pass the wrong number of type of arguments, you should get a Typescript error.

If you haven't included the node types in your project, you can add

declare const process: { argv: string[] };

to get that to be defined.

[dpvc]

Example of using MathJax components in Typescript

import type { MATHJAX_OBJECT } from '@mathjax/src/js/types/dom/lite.js';
import { MathJax as MJX } from '@mathjax/src/js/components/global.js';

const MathJax = MJX as MATHJAX_OBJECT<['adaptors/liteDOM', 'tex-svg']>;

MathJax.config = {
  loader: {
    load: ['adaptors/liteDOM'],
    paths: {mathjax: '@mathjax/src/bundle'},
    require: (file) => import(file)
  },
  options: {
    sre: {
      locale: process.argv[3] || 'en',
      braille: (process.argv[4] as any) || 'nemeth'
    }
  },
  output: {
    linebreaks: {
      inline: false,
    }
  },
};

await import('@mathjax/src/bundle/tex-svg.js');
await MathJax.startup.promise;

/**
 * Convert a TeX expression to an SVG image
 *
 * @param {string} math                The TeX expression
 * @param {boolean} display            True for display mode, false for in-line mode
 * @returns {Promise<string | void>}   The SVG image
 */
async function typeset(math: string, display: boolean = true): Promise<string | void> {
  const node = (await MathJax.tex2svgPromise(math, {
    display: display,
    em: 16,                  // size of an em in pixels
    ex: 8,                   // size of an ex in pixels
    containerWidth: 80 * 16  // width of container for linebreaking of displayed equations
  }));
  const adaptor = MathJax.startup.adaptor;
  return(adaptor.serializeXML(adaptor.tags(node, 'svg')[0]));
}

const math = process.argv[2] || '';
const svg = await typeset(math);
console.log(svg);

MathJax.done();

Here we load the MATHJAX_OBJECT type constructor for the LiteDOM, and then get the MathJax variable from the global.ts MathJax module. This will create the initial MathJax object and set up MathJax.config as the configuration object (if there were already a global MathJax object, its value would be moved to MathJax.config. The other values and methods haven't yet been added to MathJax, but will be when the tex-svg component is loaded later.

We import MathJax as MJX because this version of MathJax is not properly typed, so we use MJX and MATHJAX_OBJECT to set MathJax (locally) as a typed version with the types needed for the components that we are loading. In this case, it is the LiteDOM and the tex-svg components, which we pass to MATHJAX_OBJECT<...> as an array. We could also use the components variable approach from the "simple" example above if we needed to set the loader.load array. It is also possible to use 'adaptors/liteDOM' | 'tex-svg' rather than an array.

Once MathJax has the proper types, you can use MathJax.config to set the MathJax configuration, and then load tex-svg. Alternatively, one could load startup instead and list all the needed components explicitly in the MATHJAX_OBJECT<...> definition and the loader.load array.

The rest of the code is standard usage of MathJax, but now with full typing being available.

[dpvc]

Example of using mixed direct/component loading in Typescript

import {mathjax as mjx} from '@mathjax/src/js/mathjax.js';
import {TeX} from '@mathjax/src/js/input/tex.js';
import TexError from '@mathjax/src/js/input/tex/TexError.js';
import {CHTML} from '@mathjax/src/js/output/chtml.js';
import {liteAdaptor} from '@mathjax/src/js/adaptors/liteAdaptor.js';
import {RegisterHTMLHandler} from '@mathjax/src/js/handlers/html.js';

import {Loader} from '@mathjax/src/js/components/loader.js';
import {Package} from '@mathjax/src/js/components/package.js';
import '@mathjax/src/components/js/startup/init.js';
import '@mathjax/src/components/js/core/core-lib.js';
import '@mathjax/src/components/js/input/tex/tex.js';
import '@mathjax/src/components/js/output/chtml/chtml.js';

import type { MATHJAX_OBJECT, N, T, D } from '@mathjax/src/js/types/dom/lite.js';
import { MathJax as MJX } from '@mathjax/src/js/components/global.js';

type MATHJAX = MATHJAX_OBJECT<'adaptors/liteDOM' | 'input/tex' | 'output/chtml' | 'startup'>;
const MathJax = MJX as MATHJAX;
const mathjax = mjx as MATHJAX['startup']['mathjax'];

Loader.preLoaded(
  'loader', 'startup',
  'core',
  'input/tex',
  'output/chtml',
);

const EM = 16;          // size of an em in pixels
const EX = 8;           // size of an ex in pixels
const WIDTH = 80 * EM;  // width of container for linebreaking

MathJax.config.loader.require = (file) => import(file);
mathjax.asyncLoad = (file) => import(Package.resolvePath(file));

const adaptor = liteAdaptor({fontSize: EM});
RegisterHTMLHandler(adaptor);

const texOptions: MATHJAX['config']['tex'] = {
  formatError(_jax: TeX<N, T, D>, err: TexError) {throw err},
  ...(MathJax.config.tex || {})
};
const chtmlOptions: MATHJAX['config']['chtml'] = {
  ...(MathJax.config.output || {}),
  ...(MathJax.config.chtml || {}),
  fontURL: 'https://cdn.jsdelivr.net/npm/@mathjax/mathjax-newcm-font/chtml/woff2',
};

const html = mathjax.document('', {
  InputJax: new TeX<N, T, D>(texOptions),
  OutputJax: new CHTML<N, T, D>(chtmlOptions),
  ...(MathJax.config.options || {})
});

html.convertPromise(process.argv[2] || '', {
  display: true,
  em: EM,
  ex: EX,
  containerWidth: WIDTH
}).then((node) => {
  //
  // Generate a JSON object with the CHTML output and needed CSS
  //
  console.log(JSON.stringify({
    math: adaptor.outerHTML(node as N),
    css: adaptor.cssText(html.outputJax.styleSheet(html))
  }));
}).catch((err) => console.error('Error: ' + err.message));

Here, we are loading some MathJax modules directly (in the top set of import commands) but also loading MathJax components (the imports from @mathjax/src/components). By setting up the MathJax Component framework, we can take advantage of TeX's require and autoload features, while still using the direct access to MathJax's methods rather than relying on MathJax.startup.

Here we import mathjax as mix similarly to how we handled MathJax in the previous example, so that we can create a properly typed version below. Similarly for MathJax as we did above.

Here, however, we create MATHJAX type that corresponds to the specific components that we have loaded, and to which we can refer later. We use that to create a typed version of MathJax from MJX as before, and also a typed version of mathjax, which we obtain from MATHJAX['startup']['mathjax'], since MathJax.startup.mathjax is a pointer to the mathjax object.

We use MATHJAX['config']['tex'] and MATHJAX['config']['chtml'] to type-check the configuration objects for the TeX and CHTML jax, though we also include options from an existing MathJax configuration if one was present before this file was imported (e.g., if this were to be used in a web page with its own configuration; but you would want to change lite.ts to html.ts in that case). Those options would not be type-checked, however, and would have to rely on MathJax's run-time option checking.

The rest is standard MathJax, but note that we need to bass the N, T, and D template values to the TeX and CHTML object, as they require those arguments. We import these from the lite.ts file so we don't need to know what they are or where they are imported from. Swiching to the html.ts file would require no changes to the rest of the code.

[dpvc]

Example of direct linking to MathJax modules in Typescript.

import { TeX } from '@mathjax/src/js/input/tex.js';
import { CHTML } from '@mathjax/src/js/output/chtml.js';
import { mathjax as mjx} from '@mathjax/src/js/mathjax.js';
import { RegisterHTMLHandler } from '@mathjax/src/js/handlers/html.js'
import { SpeechHandler } from '@mathjax/src/js/a11y/speech.js';
import { MathML } from '@mathjax/src/js/input/mathml.js';
import { liteAdaptor } from '@mathjax/src/js/adaptors/liteAdaptor.js';

import '@mathjax/src/js/input/tex/ams/AmsConfiguration.js';
import '@mathjax/src/js/input/tex/newcommand/NewcommandConfiguration.js';
import '@mathjax/src/js/input/tex/configmacros/ConfigMacrosConfiguration.js';

import '@mathjax/src/js/util/asyncLoad/esm.js';

import {MathJaxTermesFont} from '@mathjax/mathjax-termes-font/js/chtml.js';

import type { MATHJAX_OBJECT, N, T, D } from '@mathjax/src/js/types/dom/lite.js';

type MATHJAX = MATHJAX_OBJECT<[
  'adaptors/liteDOM',
  'input/tex-base',
  '[tex]/ams',
  '[tex]/newcommand',
  '[tex]/configmacros',
  'output/chtml',
  'a11y/speech',
  'startup',
]>;
const mathjax = mjx as any as MATHJAX['startup']['mathjax'];

SpeechHandler(RegisterHTMLHandler(liteAdaptor()), new MathML());

const texConfig: MATHJAX['config']['tex'] = {
  packages: { '[+]': ['ams', 'newcommand', 'configmacros'] },
  macros: { RR: '\\mathbf{R}' },
  tags: 'none',
};

const chtmlConfig: MATHJAX['config']['chtml'] = {
  fontData: MathJaxTermesFont
};

const html = mathjax.document('', {
  InputJax: new TeX<N, T, D>(texConfig),
  OutputJax: new CHTML<N, T, D>(chtmlConfig),
  enableBraille: false,
});

const node = await html.convertPromise(process.argv[2] || '', {display: false}) as N;
console.log(html.adaptor.outerHTML(node));

console.log(html.adaptor.getAttribute(node, 'data-semantic-speech-none'));
html.done();

This example shows how to use MATHAX<...> to provide typings even when using direct linking to the MathJax modules. Without this, the mathjax.document() call will produce a MathDocument<any, any, any>, and the various options used in creating the inputs and output jax, and the document itself, will not be type checked.

In this case, we use MATHJAX_OBJECT<...> to create a MATHJAX type for the needed components, as we did in the mix example above, and use that to define mathjax from mjx with the proper types. We use the same MATHJAX['config']['tex'] and MATHJAX['config']['chtml'] trick to type-check the options for creating the input and output jax, and use the N, T, and D values that we imported to make that easier.

In this example, we show how to use an alternative font, and that we can include speech generation without Braille.

[codecov]

Codecov Report

❌ Patch coverage is 98.38814% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.12%. Comparing base (7429f25) to head (8286063).

Files with missing lines	Patch %	Lines
ts/core/MathDocument.ts	91.45%	10 Missing ⚠️
ts/output/common/FontData.ts	77.41%	7 Missing ⚠️
ts/output/common.ts	96.58%	4 Missing ⚠️
ts/output/chtml.ts	95.00%	2 Missing ⚠️
ts/output/common/Wrapper.ts	75.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1554      +/-   ##
===========================================
+ Coverage    87.01%   87.12%   +0.11%     
===========================================
  Files          392      392              
  Lines        88279    89098     +819     
  Branches      3356     3356              
===========================================
+ Hits         76814    77628     +814     
- Misses       11465    11470       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[zorkow]

Another long one...
I found one typo. Not too bad, as the module is no longer used anyway.

Should we consider creating tests for the types/options? I am not sure how one would go about that, though.

[dpvc]

Wow, I was in the middle of writing up more details about this one. You saved me some work!

Another long one...

Yes, indeed. But 105 of the 173 files were the new .d.ts files or changes to config.json, so somewhat better than it initially looked. Still, white a slog, I'm sure.

Should we consider creating tests for the types/options? I am not sure how one would go about that, though.

I was wondering about that myself, but don't see how to do it, as these are compile-time issues, and jest is about run-time errors. The only thing I can think of is to have a test spawn a tsc process to compile a test file and trap any errors. Probably could be done, but I haven't looked into it.

Thanks for going through this long PR!