Vau

Try it in the browser: https://mgubi.github.io/vau/ (a viewer of TeXmacs documents).

Vau is an experiment/exercise over the TeXmacs codebase, to learn more about it. The initial goal is to extract enough machinery to be able to read and typeset arbitrary TeXmacs files. So Vau will be initially a viewer. This will allow me to understand the code dependencies and extract a minimal typesetting core, abstracted from the UI and the wider organization of the editor. In the meanwhile I plan to experiment about various refactorings.

Glue code refactoring

I rewrote the Scheme glue code via C++ templating and metaprogramming and make it local to each submodule. Glue code should be defined locally where the relevant C++ function belongs, so that if a module is not linked in the corresponding glue code is not compiled in and will not be visible from scheme.

Logbook

Oct 2026

• No TeX fonts anymore. The Metafont plugin and the TFM and Type 1 files are gone: the family roman, the default of TeXmacs, is Latin Modern, the OpenType version of Computer Modern, with Latin Modern Math for the formulas (which are typeset from its MATH table), Latin Modern Sans and Mono for the sans serif and typewriter variants. src/Vau/vau_fonts.cpp defines what the rest of the code expected from the plugin: the fonts asked by their TeX names (ecrm, cmr, ecss...), which the typesetter uses for its own marks (unknown macros, missing references, flags) and the interface as its default font, are Latin Modern fonts, and the fonts whose letters are calligraphic, fraktur or blackboard bold (math-font cal, Euler, Bbb) draw the corresponding letters of Latin Modern Math. The page breaks of old documents change, since the metrics do.

• No PDF Hummus anymore: the PDF is written by the MuPDF plugin (mupdf_pdf_renderer.cpp) in all the builds. The MuPDF of the WebAssembly build has a patch of TeXmacs (devel/mupdf-subset-cff.patch) so that CFF fonts, as Latin Modern, are subset; with a MuPDF without it they are embedded whole.

• Synced with TeXmacs: the sources and resources shared with TeXmacs now follow maxs_texmacs (TeXmacs 2.1.5, it contains wip_fixes), with the S7 port of wip_s7 (S7 11.9, boot-s7.scm, compat-s7.scm) and the MuPDF plugin of wip_other_guis (which also writes the PDF). On top of these comes maxs_texmacs, with the support of OpenType math fonts (see src/OPENTYPEMATH.md) and the fonts themselves. The Scheme code is the one of TeXmacs, unmodified: init-vau-s7.scm loads the common init-kernel.scm, and the primitives of the TeXmacs glue which Vau lacks are defined as stubs (vau-glue-stubs.scm, generated by devel/make-glue-stubs.py). The WebAssembly port was reworked on this basis (see below): new build, a viewer with scrolling, zoom, files of the user and PDF export, and a test under node.

• For a test without window: VAU_TEST_OUTPUT=/tmp/out ./Vau writes /tmp/out.png (first page) and /tmp/out.pdf; VAU_TEST_DOCUMENT selects the document.

Feb 2022

• The library renders pixmaps in a worker thread and this is then put on a canvas which fills the viewport of the viewer. Implemented zoom and page change. We are still very slow in bootstrapping a document due to the loading of many fonts variations (mainly TeX fonts, which I should get rid of). After this slow typesetting phase, rendering speed is very good.

• Ideally we want to move the UI logic in Scheme and be able to both drive the typesetter and manage the UI at the browser level. I guess for the moment I will settle to drive the UI from the C++ code, maybe using embind and val to interface C++ with Javascript.

• Later on I will implement loading files from URL and possibily better caching to improve bootstrap. We already pre-cached some informations by loading it in the virtual file-system at startup, but this has not produced a substantial speedup.

Jan 2022

• I have now a basic setup with an Xcode project and a CMake recipe. I switched to the S7 interpreter, added PDF Hummus and MuPDF support. Vau can read and export to PDF fairly complex documents with PDF Hummus and generate PNG images for pages with MuPDF. Typesetting is not optimized and all the document is retypeset from the beginning at every invocation of the output routines.

• I removed almost all the C++ code which is not used at this stage. And Vau load the minimal amount of Scheme code.

• Ideally we want to isolate away I/O, because e.g. in Webasm we do not have direct access to the filesystem and maybe this requires anyway async calls.

• From here I would like to understand how to isolate a well-defined interface for the editor which allow to call it from a scripting language (be it Scheme or Javascript). The editor receives commands and send back responses (e.g. bitmaps of the current page with metadata, like location of the cursors, etc..)

Build instructons

Use CMake. We depend on MuPDF, which you can find on github, for example. The MuPDF plugin needs a recent version (it is developed against 1.26–1.28, e.g. the one of Homebrew: MUPDF_DIR=/opt/homebrew). The native viewer also needs SDL2.

For the native architecture of the machine

mkdir build; cd build;
cmake  ..
make

If CMake cannot find MuPDF, set the environment variable MUPDF_DIR to the root of the MuPDF installation.

CMake configures also the Xcode project which is found in ./platform/macos/Vau.xcodeproj.

Build and run Vau/Wasm

The WebAssembly version needs Emscripten and a build of MuPDF (1.26 or later) for WebAssembly, with the same model of exceptions as Vau (-fwasm-exceptions). Two scripts of devel/, adapted from those of TeXmacs, set this up:

. devel/emenv.sh build-wasm            # Emscripten, with a Python >= 3.10
sh devel/build-mupdf-wasm.sh build-wasm  # fetches and builds MuPDF 1.28.5 in build-wasm
cd build-wasm
emcmake cmake .. -DCMAKE_BUILD_TYPE=Release -DMUPDF_SOURCE_DIR=$PWD/mupdf-1.28.5-source
emmake make -j8

MUPDF_SOURCE_DIR may point to sources of MuPDF built elsewhere in the same way (build/wasm/slim, or another directory given with -DMUPDF_WASM_BUILD=). Freetype and zlib are those which MuPDF carries.

The build gives Vau-wasm.js and Vau-wasm.wasm, the resources as packages (vau-files.json and pkg/, see below), and copies the page next to them. To try it:

node test-node.mjs    # without a browser: typesets a document, writes vau-test.png and vau-test.pdf
node serve.mjs 8080   # then open http://localhost:8080/

Under node the test reads the resources of the source tree, mounted as /Vau.

The resources in the browser

devel/package.py (run by the build) writes the resources, without the icons of the interface and the tests, as packages, with a manifest vau-files.json which gives for each file its package, offset and size. platform/wasm/vau_packages.js, which is part of the program, makes the whole tree of /Vau before Vau starts, every file as a placeholder of its size, and fills it:

• the boot package (2.2 MB compressed) before Vau starts: the Scheme code, the styles, the encodings, and the files of platform/wasm/boot-files.txt, those which Vau reads to start and typeset a plain document (the main Latin Modern fonts, the English hyphenation);
• the other packages (the languages, the patterns..., 10 MB) one after the other in the background once Vau runs; a file read before its package has come brings the whole package at once, with a synchronous request (Vau reads its files synchronously, and such requests are allowed in a worker);
• the fonts and the example documents (65 MB) are in no package: each is a file of its own, fetched when Vau first reads it. A font which is never used is never fetched.

The names of the packages and of these files carry a digest of their contents, for the cache of the browser. vau.packageStats () in the console of the page tells what came and how, vau.filesRead () lists the files read so far, which is how the boot list is made, and Vau.html?no-background leaves all the packages to the demand.

The files of platform/wasm:

• Vau.html, the viewer: it opens the documents which come with Vau, or a .tm file of yours (button or drop), shows a page at a time (wheel, arrows, Page Up/Down, Home/End), zooms (+, -, W for the width, 0 for the whole page, Ctrl+wheel) and exports the PDF. Vau.html?doc=/Vau/... opens a document of the library, Vau.html?log sends its output to the console.
• vau_worker.js, the worker in which the library runs: the page sends [method, id, args] and gets the result back, the pixmaps as transferred buffers. Its methods (openDocument, openBuffer, pageCount, pageSize, getPagePixmap, getViewPixmap, exportPdf, evalScheme, filesRead, packageStats) are functions of the object vau of the page which return promises.
• mylib.js, what the library calls in JavaScript, and vau_packages.js, which brings the resources (above).
• serve.mjs and test-node.mjs, the server and the test above.

The library exports the functions wasm_* of src/Vau/vau_lib.cpp.

The live page

The workflow .github/workflows/pages.yml builds the WebAssembly version at each push to main (MuPDF is cached), runs test-node.mjs and publishes the viewer at https://mgubi.github.io/vau/ with GitHub Pages (which is set to be built by GitHub Actions in the settings of the repository). The packages are fetched as their gzip copies, which the program decompresses (Pages does not compress such files itself).

Screenshot: