Skip to content

Build third-party dependencies at configure time using their CMake configs - #70

Draft
remia wants to merge 10 commits into
mainfrom
cmake-deps-refactor
Draft

Build third-party dependencies at configure time using their CMake configs#70
remia wants to merge 10 commits into
mainfrom
cmake-deps-refactor

Conversation

@remia

@remia remia commented Sep 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

Replaces the Install<pkg>.cmake modules, which wrapped an ExternalProject build in hand-written IMPORTED targets, with dependencies built at configure time into <build>/ext/dist and then found with find_package() through their own exported CMake configuration files.

With the old approach, every module had to guess library file names (debug postfixes, version suffixes, prefix changes), re-create the link graph between targets (9 targets for OpenEXR alone), and report <pkg>_FOUND and <pkg>_VERSION before anything was built. The same toolchain forwarding block was also copied into every module.

  • New ocio_build_dependency() helper (share/cmake/macros/ocio_build_dependency.cmake):
    • Downloads a release archive, checks its hash, and retries on transient network failures.
    • Configures, builds and installs the dependency into ext/dist, forwarding the toolchain, compilers, Apple/Android settings, visibility and PIC in one place.
    • A stamp file skips the build on later configure runs unless its inputs changed.
    • Multi-config generators build the Release and Debug configurations listed in CMAKE_CONFIGURATION_TYPES side by side, and configure fails if two configurations would install the same library file.
    • OCIO_<pkg>_SOURCE_DIR builds from local sources for offline builds.
  • One short recipe per dependency in share/cmake/deps/: expat, yaml-cpp, pystring, Imath, ZLIB, minizip-ng, lcms2, openfx, pybind11, OpenEXR.
  • pystring, lcms2 and openfx don't provide usable CMake builds, so OCIO provides small projects for them in share/cmake/projects/<pkg>/ that install proper package configs. The CI install scripts use them too.
  • macOS: the expat and minizip-ng symbols are now hidden with -load_hidden <exact archive of the target> instead of -L<dir> -hidden-l<name>. The old flags could resolve to another library in the search path: on a Mac with Homebrew, libOpenColorIO.dylib from main ends up depending on Homebrew's libminizip-ng.1.dylib.
  • Python bindings: /bigobj is added on MSVC directly. The pybind11 configs only add it through pybind11_add_module(), which is not a good fit for OCIO; the reasons are documented in src/bindings/python/CMakeLists.txt.
  • setup.py:
    • Reads the version from CMakeLists.txt, instead of running a full CMake configure that would now build every dependency.
    • Only builds the Release dependencies with the Visual Studio generator.
  • sse2neon and DirectX-Headers already use FetchContent with upstream targets, so they are unchanged.

OCIO_INSTALL_EXT_PACKAGES keeps its NONE / MISSING / ALL behavior.

Commits

Each commit leaves the build working and can be reviewed on its own:

  1. Add ocio_build_dependency (plus a fallback to the legacy Install<pkg>.cmake modules during the migration)
  2. macOS symbol hiding with -load_hidden
  3. yaml-cpp
  4. expat, Imath, pybind11
  5. ZLIB, minizip-ng
  6. OpenEXR
  7. pystring, lcms2, openfx
  8. Documentation
  9. Only build the dependency configurations requested
  10. setup.py version from CMakeLists.txt

Behavior changes

  • Dependencies are built during cmake configure rather than during the build. The first configure with all dependencies takes about 80 s on a 16-core Mac; later configures skip the builds.
  • A shared zlib is no longer left in ext/dist.
  • A static OCIO install can be consumed with only -DCMAKE_PREFIX_PATH=<install>;<build>/ext/dist, without the per-package <pkg>_ROOT / <pkg>_STATIC_LIBRARY flags. The CI flags are left unchanged and still work.

Testing

On this fork, on the same code as this branch (the runs also included two temporary workflow commits, dropped since):

Workflow Result
CI ✅ 20/20: Linux GCC/Clang static and shared, macOS Intel and ARM including universal, Windows Debug and Release
Wheel ✅ 31/31, with the numpy fix from AcademySoftwareFoundation#2350 cherry-picked
Platform latest ✅ 8/8 (edited to run on a fork)
Dependencies latest ✅ 8/8 (edited to run on a fork)

Locally (macOS, Ninja and Xcode):

  • The exported symbols of libOpenColorIO are identical to main: no expat, minizip-ng or yaml-cpp symbols leak.
  • The static-install consumer test builds and runs with only CMAKE_PREFIX_PATH.
  • Each Xcode configuration links its own Debug or Release dependency libraries.

Notes for review

🤖 Generated with Claude Code

remia and others added 10 commits September 24, 2026 21:00
@remia @claude
Add ocio_build_dependency to build dependencies at configure time
e9a1b34 
Add a helper downloading, building and installing a dependency into
<build>/ext/dist at configure time, so that it can then be located with
find_package() using the CMake configuration files exported by the
dependency itself, instead of hand-written imported targets wrapping an
ExternalProject.

- Toolchain, compilers, Apple and Android settings, visibility and PIC
  are forwarded once through a generated initial cache file.
- Downloads are verified against the archive hash, and retried to cope
  with transient network failures.
- A stamp file records the build inputs, so that subsequent configure
  runs skip the build when nothing changed.
- Multi-config generators build both Release and Debug side by side,
  and fail if both configurations install the same library file.
- OCIO_<dep>_SOURCE_DIR builds from local sources for offline builds.
- PROJECT_DIR allows an OCIO provided CMakeLists.txt for dependencies
  that don't provide a usable one.

ocio_install_dependency uses a share/cmake/deps/<dep>.cmake recipe when
one exists, and falls back to the legacy Install<dep>.cmake module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Hide static dependency symbols on macOS using the targets
8ac5e9d 
On macOS, the expat and minizip-ng symbols were hidden by adding
-L<dir> -hidden-l<name> linker flags, derived from the expat_LIBRARY and
minizip-ng_LIBRARY variables set by the find and install modules. The
linker could then resolve the name to another library in its search
path, e.g. a Homebrew libminizip-ng.dylib, which libOpenColorIO would
then depend on at runtime.

Pass the exact static archive of the expat::expat and
MINIZIP::minizip-ng targets to -load_hidden instead, which also works
when the dependencies are located through their CMake configuration
files and with multi-config generators.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Build yaml-cpp with ocio_build_dependency
03d57cc 
Replace Installyaml-cpp.cmake with a recipe using the yaml-cpp CMake
configuration files, which also provide YAML_CPP_STATIC_DEFINE. Remove
the yaml-cpp alias from Findyaml-cpp.cmake, which was only needed
because of the targets created by Installyaml-cpp.cmake.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Build expat, Imath and pybind11 with ocio_build_dependency
fef53c7 
Replace the corresponding Install modules with recipes using the CMake
configuration files of each package.

expat ignores CMAKE_DEBUG_POSTFIX and only uses a Debug postfix on
Windows, so set it explicitly with multi-config generators to avoid the
Debug library overwriting the Release one.

pybind11 is header only, so it is installed without searching for
Python. The pybind11 CMake configuration files only add /bigobj through
pybind11_add_module(), which OCIO doesn't use, while Installpybind11
added it to the pybind11::module target. Add it to the PyOpenColorIO
target directly on MSVC instead, whatever the origin of pybind11, and
document why pybind11_add_module() is not used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Build ZLIB and minizip-ng with ocio_build_dependency
ee509df 
ZLIB 1.3.1 doesn't provide CMake configuration files and always installs
both a shared and a static library. Remove the shared library from
ext/dist, so that anything searching ZLIB there (minizip-ng, consumers
of a static OCIO) uses the static library, and point CMake's FindZLIB
module to it. The results of a previous search are discarded first, as
a system ZLIB rejected for being too old (e.g. 1.2.7 on manylinux2014,
1.2.12 in the macOS SDK) would otherwise stay in the cache.

minizip-ng records the ZLIB library path in its exported targets, so it
is built against the per-configuration ZLIB libraries used by OCIO.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Build OpenEXR with ocio_build_dependency
cce9237 
Replace the 9 hand-written OpenEXR imported targets with the OpenEXR
CMake configuration files. OpenEXR is built against the Imath used by
OCIO, whether it was found or built by OCIO.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Build pystring, lcms2 and openfx with ocio_build_dependency
12ba3f3 
These packages don't provide usable CMake builds for OCIO: pystring
doesn't install its headers nor configuration files, lcms2 only has
Autotools and Meson builds, and openfx requires Conan provided
dependencies. Move the OCIO provided CMakeLists.txt files to
share/cmake/projects/<dep>, taking the location of the sources in
OCIO_DEP_SOURCE_DIR instead of being copied over the sources, and
install CMake configuration files for them.

Update the CI install scripts accordingly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Document dependencies built at configure time
7e951ff 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Only build the dependency configurations requested
a2afd61 
With multi-config generators, only build the Release and Debug
configurations of the dependencies listed in CMAKE_CONFIGURATION_TYPES.

The Python wheels are built with the Visual Studio generator on Windows,
only for the Release configuration, so set CMAKE_CONFIGURATION_TYPES
accordingly in setup.py, instead of building the Debug dependencies too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia @claude
Read the version from CMakeLists.txt in setup.py
9c38926 
setup.py ran a full CMake configuration only to read the version from
the generated OpenColorABI.h header. With the dependencies now built at
configure time, this built all the missing dependencies a first time
before building the wheel. Read the version and release type from the
top level CMakeLists.txt instead, the same way CMake builds
OCIO_VERSION_FULL_STR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rémi Achard <remiachard@gmail.com>
@remia
remia force-pushed the cmake-deps-refactor branch from e0c0e46 to 9c38926 Compare September 24, 2026 20:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@remia