diff --git a/.agents/docs/2026-11-19-add-openssl-abseil-protobuf-mwrap-plan.md b/.agents/docs/2026-11-19-add-openssl-abseil-protobuf-mwrap-plan.md new file mode 100644 index 00000000..8b0172e3 --- /dev/null +++ b/.agents/docs/2026-11-19-add-openssl-abseil-protobuf-mwrap-plan.md @@ -0,0 +1,376 @@ +# 新增 mcpplibs.mwrap.openssl / abseil / protobuf —— 设计方案 + +状态:**已实施(v5)**。v4 经 review 确认(abseil-m 命名、GitCode 上传策略、codegen 规则架构 §6.5);实施结果与偏差见 §12。 +修订记录:v2 加入平台分析;v3 吸收三路自审(§11);v4 命名与上传策略;v5 实施结果。 + +规范:[mwrap](../../docs/zh/mwrap.md) · [README 布局](../../docs/zh/mwrap-readme.md) · [许可](../../docs/zh/mwrap-licensing.md)。 +参照:libpng-m(compat 之上的薄封装、NOTICE 写法)、jolt-m / acl-m(导出审计、arm64 与 wasm 行)、miniaudio-m(compat 之上的 wasm 行)、 +grpc-m(codegen feature、rule 子包、版本耦合检查)、2026-10-24-add-mwrap-and-ort-plan.md(候选索引 commit 固定)。 + +## 1. 目标与交付 + +| 仓库(新建) | 包身份 | 版本 | 实现 | tag / 资产 / CN 镜像 | +|---|---|---|---|---| +| mcpplibs/abseil-m | `mcpplibs.mwrap.abseil` | `20250512.1` | `compat.abseil` | `mwrap-20250512.1` / `abseil-m-20250512.1.tar.gz` / `mcpp-res/mwrap-abseil`(tag `20250512.1`) | +| mcpplibs/protobuf-m | `mcpplibs.mwrap.protobuf` | `35.1` | `compat.protobuf` | `mwrap-35.1` / `protobuf-m-35.1.tar.gz` / `mcpp-res/mwrap-protobuf`(tag `35.1`) | +| mcpplibs/openssl-m | `mcpplibs.mwrap.openssl` | `3.5.9` | `compat.openssl`(新增 3.5.9) | `mwrap-3.5.9` / `openssl-m-3.5.9.tar.gz` / `mcpp-res/mwrap-openssl`(tag `3.5.9`) | +| mcpplibs/grpc-m(修订) | `mcpplibs.mwrap.grpc` | `1.83.0.1` | 依赖改 `compat.openssl 3.5.9` | `mwrap-1.83.0.1` | + +首发一次做完整:openssl 含 `digest`(默认)与 `tls-loopback` 模板;protobuf 含 `codegen`、`upb`、`gzip` feature 与 `message`(默认)、`codegen` 模板。 + +PR 一览(**每个仓库恰好一个 PR**,保留 checkpoint commits): + +| PR | 仓库 | 内容 | +|---|---|---| +| IDX | mcpp-index | compat.openssl 3.5.9;依赖方切换(带 revision);compat.protobuf Windows protoc;grpc 1.83.0.1;三个 mwrap 描述符;消费者成员;CI 接入;CHANGELOG | +| G | grpc-m | 1.83.0.1 | +| A | abseil-m | 新包 | +| P | protobuf-m | 新包 | +| O | openssl-m | 新包 | + +IDX 先以 draft 打开并持续推进;各封装仓库 CI 固定到 IDX 的某个候选 commit(§8),IDX 最后合入即发布。 + +## 2. 现状(2026-11-19 核查) + +- `compat.abseil 20250512.1`、`compat.protobuf 35.1`:mcpp 图内源码构建,linux/macosx/windows。compat.protobuf 的 feature: + `protoc`(libprotoc + protoc 二进制,要求同时开 `upb`)、`upb`、`gzip`。**Windows 上去掉了 `protoc` target**(host tool 子构建在 + 三个 abseil TU 的 `.ddi` 缺失处失败,原因未查明)。`compiler/importer.cc`、`parser.cc` 已在运行时源码列表中。 +- `compat.openssl 3.5.1`:install() 钩子在安装时为**宿主**运行 Configure + make(linux/macOS 带 asm);Windows 为 VC-WIN64A + nmake, + 且配置为 `no-asm`。产物绑定宿主 C 库(`provides = ["platform-sdk"]`),不能交叉构建;install 钩子本身与版本无关,升级只改 url/sha/注释。 +- 依赖 `compat.openssl 3.5.1` 的:index 描述符 asio(ssl)、curl、httplib、websocket、jwt-cpp、mysql-connector-cpp(`deps`、xpm 安装期依赖 + `compat:openssl@3.5.1` 两处、`find_dep_install_dir`);**清单在上游归档里、index 改不了的**:`mcpplibs.mwrap.grpc 1.83.0`、旧身份 + `grpc.grpc 1.83.0`(pkgs/g/grpc.lua)、`yizhinailong.mcr`;测试 `tests/test_select_members.py:91`、tests/examples 中的 openssl/asio-ssl/curl/grpc-*。 +- grpc-m:tag `v1.83.0`、`v1.83.0-2/-3/-4`、`mwrap-1.83.0`;`plugin/`、`rules/` 已为 1.83.0.1,且 index 中 `grpcgen`/`grpc-plugin 1.83.0.1` + 指向 **mwrap-1.83.0 的归档**;其 CI 断言 `runtime == 1.83.0` 且 `plugin == rule == runtime + ".1"`;`MCPP_VERSION` 固定 2026.10.24.1; + 无 Windows 行(理由“compat.openssl 无 Windows”已过时)。 +- mcpp v2026.11.18.1 仅发布 linux x86_64/aarch64、macosx-arm64、windows-x86_64 二进制。index CI 覆盖 linux x86_64、macos-15、windows-2022, + **无 wasm 作业、无 Linux aarch64**。index `min_mcpp` 为 2026.9.18.3。 +- 上游最新:OpenSSL 3.5.9(LTS)/ 3.6.5 / 4.1.0;Abseil 20260817.0;Protobuf v36.2。 +- 未合并的自动更新 PR:#634(compat.httplib 0.64.0)、#563(yizhinailong.mcr 0.3.0),都涉及 openssl 依赖方。 + +## 3. 已确认的决策 + +- **D1**:三库均薄封装于 compat,不 vendor 源码;与 grpc 同图时各只有一份实现。 +- **D2**:abseil `20250512.1`、protobuf `35.1` 与 compat/grpc 对齐;compat.openssl 新增 3.5.9,依赖方切换,grpc-m 发修订版,openssl-m 首发 3.5.9。 +- **D3 命名(v4 建议,待确认)**:仓库 `abseil-m`、`protobuf-m`、`openssl-m`;namespace `mcpplibs.mwrap`,name `abseil`/`protobuf`/`openssl`。 + 仓库名 = 包短名 + `-m`,与现有 40 个封装一致,**不产生例外**。上游仓库名 `abseil-cpp` 中的 `-cpp` 只用于区分 abseil-py, + 在只有 C++ 的生态里是冗余的;`abseil` 也是 compat.abseil、vcpkg、Conan、Homebrew、xmake 采用的包名。 + (godot-cpp、llama.cpp 保留后缀,是因为后缀本身就是产品名的一部分。)上游来源记录仍用原名:`upstream/abseil-cpp/`、documents.json 的 repo 字段。 + abseil 的全部名字: + + | 项 | 值 | + |---|---| + | 仓库 / 资产 / CN 镜像 | `mcpplibs/abseil-m` / `abseil-m-.tar.gz` / `mcpp-res/mwrap-abseil` | + | 包 | namespace `mcpplibs.mwrap`,name `abseil`;描述符 `pkgs/mwrap/abseil.lua` | + | 模块 | `mcpplibs.mwrap.abseil`、`mcpplibs.mwrap.abseil.strings` …、`mcpplibs.mwrap.abseil.macros`;根文件 `src/abseil.cppm` | + | 宏头文件 | `#include `(仓库内 `include/mwrap/abseil/macros.hpp`,同 libpng-m/freetype-m) | + | 宏映射 C++ 命名空间 | `mcpplibs::mwrap::abseil::macros` | + | 上游 API / 头文件 | 不变:`absl::…`、`` | + + protobuf、openssl 同理:``、``、``。 +- **D5**:protobuf 有 `codegen`(protoc + 规则包)。 +- **范围**:首发做完整,覆盖所有可行平台;做不到的写明原因。 + +## 4. 平台 + +### 4.1 矩阵(封装仓库 CI 行) + +| 行 | abseil | protobuf | protobuf codegen | openssl | +|---|---|---|---|---| +| Linux x86_64 gcc@16.1.0 / llvm@23.1.3 | ✓ | ✓ | ✓ | ✓ | +| Linux aarch64 gcc / llvm(ubuntu-24.04-arm) | ✓ | ✓ | ✓ | 待测(compat.openssl 的 aarch64 路径从未进 CI) | +| macOS arm64 llvm(macos-15) | ✓ | ✓ | ✓ | ✓ | +| Windows x86_64 llvm(windows-2025,MSVC ABI) | ✓ | ✓ | 取决于 §5.3 | ✓(no-asm) | +| wasm32-emscripten(node) | ✓ | ✓ | protoc 在宿主运行 ✓ | ✗ | + +`platforms`:abseil、protobuf 为 `["linux", "macos", "windows", "emscripten"]`;openssl 为 `["linux", "macos", "windows"]`。 + +不覆盖且原因明确的: + +- **macOS x86_64**:mcpp 没有 macOS x86_64 二进制,macos-15-intel 行无法启动。mcpp 发布该二进制后再加。 +- **Windows arm64**:mcpp 无该宿主二进制;compat.openssl 写死 `vcvars64.bat` / `VC-WIN64A`,该格必然失败。 +- **openssl 的 wasm32 / openkal(musl)/ 交叉目标**:compat.openssl 是安装时的宿主构建。改为图内构建需要按数据模型 + (LP64/LLP64/ILP32)做 Configure 快照,并且为保留 AES-NI/ARMv8 加速与常数时间实现,要按目标提交 perlasm 汇编;还会打破 + mysql-connector-cpp 对安装目录布局的依赖。这是独立的 compat 重构,另起计划。(Windows 现已 no-asm,即 C 查表 AES,README 写明。) +- abseil/protobuf 在 openkal 目标上由 index 的 openkal-compat 测量覆盖,不进封装仓库 CI。 + +### 4.2 各平台要点(已实测部分标 ✔) + +- **wasm32**:✔ em++ 6.0.9 不加 `-pthread` 可编译抽样的 abseil(mutex、cctz、symbolize、flags、log)与 protobuf(arena、map、JSON、message)TU。 + compat 描述符大概率不需修改(miniaudio 的 `-lpthread` 在 wasm 行无碍);只在封装 CI 实测失败时再在 IDX 中补 `target_cfg`。 + 多线程测试在该行跳过;时区只用 UTC/固定偏移;导出审计按目标分别记录(`long`/指针宽度不同)。index 不新增 wasm 作业,wasm 覆盖在封装仓库 CI(与 jolt/miniaudio 一致)。 +- **Windows**:✔ abseil 公共头不含 ``,protobuf 只有 `compiler/subprocess.h`;只有 openssl 经 `e_ostime.h` 引入 `` → + openssl-m 在 `cfg(windows)` 下设 `NOMINMAX`/`WIN32_LEAN_AND_MEAN` 并在 README 说明。wincrypt 宏冲突的 `#undef` 在 `openssl/types.h`,头文件混用测试覆盖。 +- **Linux aarch64**:compat.openssl 的 gcc sysroot + xim perl 路径首次在 CI 运行;失败则在 IDX 中修 compat,不降级声明。 + +## 5. compat 与 grpc 的变更(都在 IDX / G 中) + +### 5.1 compat.openssl 3.5.9 + +- 新增 3.5.9 条目(GLOBAL + CN 镜像;sha256 以上游 `.sha256` 文件核对,自审下载得 `603f5602…859a`),**3.5.1 条目不动**。 +- 依赖方切到 3.5.9:asio、curl、httplib、websocket、jwt-cpp、mysql-connector-cpp(三处);改的是已发布条目的依赖 → + 用 index `revision`(必要时 `floor`)刷新已安装载荷,并实测“已有旧缓存的构建”能刷新且运行(mwrap-licensing.md 末段)。 +- 改不了的:`mwrap.grpc 1.83.0`、`grpc.grpc 1.83.0`、`yizhinailong.mcr`。它们与切换后的依赖方或新包同图会出现**两份 OpenSSL**。 + CHANGELOG 与 openssl-m README 写明:用 OpenSSL 的 gRPC 请用 `mcpplibs.mwrap.grpc 1.83.0.1`;mcr 等待其上游更新清单。 +- 处理 #634 / #563:先合入或在 IDX 中 rebase,避免冲突。 +- 测试与注释同步:`tests/test_select_members.py`、tests/examples 各成员、openkal `members.toml` 中过时注释。 + +### 5.2 grpc-m 1.83.0.1(G) + +- 运行时包 `mcpplibs.mwrap.grpc` → `1.83.0.1`,依赖 `compat.openssl = "3.5.9"`,其他不变;`MCPP_VERSION` → 当前固定版本。 +- **`plugin/`、`rules/` 保持 1.83.0.1 且内容不变**:不新增它们的 index 条目(现有条目继续指向 mwrap-1.83.0 归档)。改写 CI 的版本检查为 + “runtime = upstream 或 upstream + 修订;plugin == rule == 1.83.0.1;`git diff mwrap-1.83.0 -- plugin rules` 为空”。 + 实施前先确认 mcpp 对 path 子包按路径/内容而非 name@version 缓存。 +- ci.yml 的 `paths-ignore: **.md` 导致纯文档合入 commit 无法打 tag:G 的合入 commit 必须触发 CI(或 release.yml 改为接受最近一次被 CI 覆盖的祖先)。 +- Windows:作为**附加尝试**加 windows-2025 行(210 分钟上限内);通过则 `platforms` 加 windows,并同步放开 index 中 grpc 成员的 `cfg(unix)` 门控; + 不通过则保持 linux/macos,并把过时注释改为真实原因。不阻塞其他 PR。 + +### 5.3 compat.protobuf:Windows 上的 protoc + +- 在 windows-2025 + 当前 mcpp 复现(注释写于较早引擎);取子构建内层 ninja 完整输出。已修复 → 去掉 `windows.targets` 覆盖; + 引擎问题 → 向 mcpp 提 issue 附最小复现。本轮修不了:protobuf-m 的 `codegen` 在 Windows 明确报错,README 与平台表注明。 + +### 5.4 许可通知 + +compat 三包没有 `license_files`。按 libpng-m 做法,**由各封装保留并部署其静态闭包的全部通知**: + +| 封装 | `upstream/` 保留并 `runtime.deploy` | +|---|---| +| abseil-m | `abseil-cpp/LICENSE`、`README.md` | +| protobuf-m | `protobuf/LICENSE`、`README.md`;`utf8_range/LICENSE`(MIT);`abseil-cpp/LICENSE`(compat.protobuf 依赖 compat.abseil) | +| openssl-m | `openssl/LICENSE.txt`、`README.md` | + +部署路径为 `licenses//upstream/<组件>/`,同时使用两个封装时不会互相覆盖。`check_installed_template.py` 的 `STATIC_NOTICES` 扩展为每包多个许可文件。 + +## 6. 三个封装仓库 + +### 6.1 通用约定 + +- 布局:`mcpp.toml / .xlings.json / upstream/<组件>/ + upstream/documents.json / src/.cppm 与子模块 / src/macros.cppm / + include/mwrap//macros.hpp / templates/ / tests/ / tools/(surface.py、surface.json、check_exports.py、check_mwrap_release.py、test_template.py)/ + docs/usage.md / README.md + README.zh.md / LICENSE / NOTICE`;`.gitattributes` 为 `upstream/** -text -whitespace`。 +- NOTICE 按 libpng-m 格式:mcpp-community contributors 版权行、Apache-2.0 适用范围,并写明上游许可与版权人—— + OpenSSL:Apache-2.0,The OpenSSL Project Authors;protobuf:BSD-3-Clause,Google LLC,另列 utf8_range(MIT)与 Abseil(Apache-2.0,The Abseil Authors); + abseil:Apache-2.0,The Abseil Authors。 +- 清单:`[package.metadata.mwrap] upstream-version`;`authors = ["mcpp-community"]`;Topics `mcpp-community`、`cpp-modules`、库名。 +- 导出:`tools/surface.py` 用 **clang** JSON AST 读公共头(按 compat 的实际宏配置),生成 `tools/surface.json`;`export namespace { using ::X; }` + (libc++ `std.cppm` 写法;protobuf 35.1 无 inline namespace)。**命名空间作用域运算符必须导出**(Clang 不导出就拒绝 `cout << status`、 + `Duration + Duration`;GCC 接受,所以审计以 Clang 为准)。不导出的实体每项带原因列清单,不静默 prune。 + GCC 对 abseil/protobuf 的 `-Wexpose-global-module-tu-local` 仅为警告(`kMsbs8Bytes`、`KeyMapBase` 等),模块 TU 不加 `-Werror`,并在审计中记录。 +- **头文件与 import 混用(✔ 已实测)**:Clang 两种顺序都可;GCC 下 **import 后再 include 会失败**(`std::__is_constant_evaluated` 重定义), + include 在前但包含的头少于模块 GMF 时也会失败。规则:**所有 #include 在前,import 在后**;protobuf 提供 + ``(含 `descriptor.pb.h` 等与模块 GMF 一致的头),生成代码的 TU 先 include 它。 + GCC 上 import-在前 是已知限制写入 README,测试只断言支持的顺序,另保留一个 Clang 两序用例。 +- 宏: + 1. 常量宏 → `mcpplibs::mwrap::::macros` 下同名 `constexpr`,`static_assert` 等于原宏; + 2. 单一调用转发 / 名称别名宏 → 同名 inline 函数; + 3. 字符串化、可变参、类型泛型、逗号列表、控制流、日志断言、依赖 `__FILE__/__LINE__` → 只经 `/macros.hpp>`; + 原宏与映射分别测试,docs/usage.md 列完整清单。 +- 模板:`[dependencies]` 写点分 key `{{self.name}} = { version = "{{self.version}}", … }`(不加引号,--strict 拒绝带引号形式)。 +- README:模块栏只列根模块与少数核心子模块 + `> More Modules API -> docs/usage.md`。 +- CI:`ci.yml` 按 §4.1,dev/release 两 profile,身份与双语 README 契约,导出审计(Linux LLVM 行重推导);`templates.yml` 对**每个模板**从安装副本 + `mcpp new` 后 strict build/run;`release.yml` 由 `mwrap-*` tag 触发。构建缓存按 toolchain+profile 键。索引固定方式见 §8。 +- mcpp:实施开始时核 remote main 与最新稳定 mcpp(目前 2026.11.18.1),清单/CI/模板/例子中精确固定。 +- 本地:80% 资源预算(mwrap-tests.slice、`MCPP_JOBS`),三库不并行全量编译,保留 ≥6 GiB 磁盘。 + +### 6.2 abseil-m(`mcpplibs.mwrap.abseil`) + +- 模块:根聚合 `.base` `.algorithm` `.cleanup` `.container` `.crc` `.debugging` `.flags` `.functional` `.hash` `.log` `.memory` `.meta` `.numeric` + `.random` `.status` `.strings` `.synchronization` `.time` `.types` `.utility`;`.macros`。`*_internal` 不导出。 +- 宏:版本常量、`ABSL_HAVE_*` → constexpr;`LOG`/`CHECK`/`ABSL_*LOG*`/`ABSL_*CHECK*`/`ABSL_FLAG`/`ABSL_DECLARE_FLAG`/`ABSL_PREDICT_*`/线程注解/属性 → macros.hpp。 +- 模板 `basics`:StrSplit/StrJoin/StrFormat、flat_hash_map、StatusOr 错误路径、UTC CivilTime、自定义 LogSink 捕获 LOG。 +- 测试:上述 API、Hash 一致性、Mutex/Notification(wasm 外多线程)、flags、Cord、random 分布边界、`operator<<`/算术运算符可用、宏与常量、混用规则。 + +### 6.3 protobuf-m(`mcpplibs.mwrap.protobuf`) + +- 依赖 `compat.protobuf = "35.1"`;**dev-dependency** `mcpplibs.mwrap.abseil`(仅用于类型同一测试)。模块不导出任何 `absl` 名字; + 使用 `absl::Status` 等名字的消费者自己 import `mcpplibs.mwrap.abseil`(README 说明)。 +- Features: + + | Feature | 内容 | + |---|---| + | `codegen` | `compat.protobuf = { features = ["upb"], tools = ["protoc"], reexport = true }` + 规则子包 `rules/`(host module `protogen`) | + | `upb` | 转发 compat `upb`(与 grpc 同图时并集,一份实现) | + | `gzip` | 转发 compat `gzip`(`GzipInputStream/OutputStream`,引入 compat.zlib) | + +- 规则:`protogen`,架构与取舍见 §6.5。 +- 模块:根(Message/MessageLite/Arena/RepeatedField/Map/Descriptor/Reflection/DynamicMessage/DescriptorPool/TextFormat + well-known types)、 + `.io`、`.util`(JsonUtil、MessageDifferencer、TimeUtil、FieldMaskUtil、delimited)、`.compiler`(Importer、DiskSourceTree、Parser——已在运行时源码中, + **不需要 feature**)、`.macros`。 +- 宏:`GOOGLE_PROTOBUF_VERSION`/`PROTOBUF_VERSION` → constexpr;`GOOGLE_PROTOBUF_VERIFY_VERSION` → macros.hpp。 +- 模板:`message`(默认,无 protoc:运行时构造 FileDescriptorProto → DynamicMessage;Struct/Timestamp ↔ JSON/文本);`codegen`(`proto/*.proto` + prelude + `protogen::generate_all()`)。 +- 测试:wire 字节 pin、JSON/文本往返、MessageDifferencer、Any、TimeUtil、gzip 往返、Importer 解析 .proto、`JsonStringToMessage` 的 `absl::Status` 与 mwrap.abseil 类型同一、codegen 模板。 + +### 6.5 代码生成规则的架构(protogen 与 grpcgen) + +**行业做法**:代码生成规则都归 protobuf 所有,是一个以插件为参数的通用层;gRPC 只提供一个插件,外加一层很薄的便捷封装。 + +| 体系 | protobuf 层(通用) | gRPC 层 | +|---|---|---| +| Bazel | `proto_library` + `cc_proto_library`(protobuf / rules_proto 仓库) | `cc_grpc_library` 建在 `cc_proto_library` 之上(grpc 仓库) | +| CMake | `protobuf_generate()`,随 protobuf 的 package config 发布 | 调同一个 `protobuf_generate(... PLUGIN protoc-gen-grpc=...)` | +| Buf | `buf generate` + 插件列表 | gRPC 只是其中一个插件 | +| xmake / Meson | `protobuf.cpp` 规则 / generator | 给同一规则传 grpc 插件参数 | + +共同点: +- 规则归 protobuf(它拥有 protoc 和 well-known types); +- 插件是一等参数; +- gRPC 依赖 protobuf 的规则,反过来不行; +- 生成器与运行时同一个版本。 + +**现状**:grpcgen 的 491 行里,大部分其实是这个通用层:L0–L3 分层、`plugin` 结构、WKT 定位、嵌套目录、每个 .proto 一条边、 +边描述中记录参数。gRPC 专有的只有 `cpp()` 插件和 `.grpc = true` 这个快捷参数。也就是说,通用层目前被放在 grpc-m 里。 + +**可选方案**: + +| 方案 | 分层 | 问题 | +|---|---|---| +| ① protobuf-m 复用 grpcgen | protobuf → grpc,依赖反转 | 只用 protobuf 的项目要依赖 grpc 的规则包,protobuf-m 的版本也绑到 grpc 的版本轴上;名字语义不对 | +| ② protobuf-m 自带私有 protogen(path 子包、不发布) | 正确 | 通用层有两份实现;grpcgen 没法依赖它(不能 path 依赖到别的仓库),会长期分叉 | +| ③ **protogen 作为通用层,独立发布;grpcgen 以后改为建在它之上** | 正确,与行业做法一致 | 本轮多一个 index 描述符和一个消费者成员 | +| ④ 写进 mcpp 引擎 | — | 引擎要绑定某个库的知识;CMake 已经把 FindProtobuf 让位给 protobuf 自带的 config,行业方向相反 | + +**建议 ③**: +- **身份**:namespace `mcpplibs`,name `protogen`,host module `protogen`(与 grpcgen 同一命名规则:裸包名 = 模块名,是生态自写的规则,不属于上游)。 + **版本 = protobuf 版本 `35.1`**,生成器、规则、运行时共用一个版本轴。 + 源码在 protobuf-m 的 `rules/`,从 protobuf-m 的 `mwrap-35.1` 归档发布(同 grpcgen 从 grpc-m 归档发布),描述符 `pkgs/p/protogen.lua`(与 `pkgs/g/grpcgen.lua` 同约定)。 +- **API**:从 grpcgen 抽出通用层,保持 L0–L3 的形态与参数名,让以后的迁移是机械替换: + `generate_all()`、`generate({"name"})`、`options{ .imports, .protoc_args, .plugins, … }`、`plugin{ name, tool, outputs, params }`、`plan_all()`/`submit()`。 + 默认只跑 `--cpp_out`;插件列表为空时不需要任何插件依赖。 + 另外支持 `.descriptor_set`(生成 FileDescriptorSet,给 DynamicMessage/反射场景用)和 `.lite`(`lite:`,对应 MessageLite)。 +- **接入方式**:protobuf-m 的 `codegen` feature reexport protoc 与 protogen;用户的 build.mcpp 写 `import protogen;`。 + 用户也可以直接依赖 `mcpplibs.protogen`(同 grpcgen 的用法)。 +- **与 grpc 共存**:本轮 grpc-m 只发 1.83.0.1,plugin/rules 不动(已确认),grpcgen 暂时保留自己那份通用层。 + 文档写明:gRPC 项目调用 `grpcgen`,纯 protobuf 项目调用 `protogen`。两者都在 build.mcpp 中显式调用,不会自动重复生成; + 组合成员 `grpc-protobuf-stack` 验证两个 feature 同时开启时,只调用 grpcgen 也能正常构建。 +- **后续(不在本轮)**:grpc-m 下一个上游版本里,grpcgen 依赖 `mcpplibs.protogen` 并删掉自己的通用层,只保留 `cpp()` 插件和 `.grpc` 快捷参数; + 公共 API 不变,用户的 build.mcpp 不用改。 +- **测试**:protogen 单元测试在 protobuf-m 中跑(嵌套目录、imports、自定义插件,用一个测试插件二进制,以及 descriptor_set); + index 消费者 `tests/mwrap/protobuf-codegen` 同时覆盖 protogen 描述符(避免 “no candidate consumer” 扫描失败); + `check_installed_template.py --template codegen`。 + +### 6.4 openssl-m(`mcpplibs.mwrap.openssl`) + +- 依赖 `compat.openssl = "3.5.9"`;无 feature。 +- 模块:根聚合 `.crypto`(EVP、BIO、ERR、RAND、X509/PEM、BN、EC、KDF/MAC、OSSL_PARAM、provider…)、`.ssl`(ssl.h、tls1.h、dtls1.h…)、`.macros`。 + 头文件按 compat 的配置读取;3.0 deprecated API 按实际声明导出(使用时仍有上游警告),模块 TU 内抑制。 +- 宏与内部链接实体(docs/usage.md 列完整清单): + 1. 常量(`SSL_OP_*`、`SSL_ERROR_*`、`TLS1_3_VERSION`、`EVP_MAX_MD_SIZE`、`NID_*`、`OPENSSL_VERSION_NUMBER`…)→ constexpr; + 2. 单一调用转发与别名(✔ `SSL_CTX_set_mode`、`BIO_get_mem_data`、`SSL_set_tlsext_host_name`;`SSLv23_method` → `TLS_method`)→ 同名 inline 函数; + 3. **`static ossl_inline` 公共 API**(`ERR_GET_LIB`、`ERR_GET_REASON`、`ERR_FATAL_ERROR`、byteorder 等,内部链接,无法 `export using`)→ + 在 `mcpplibs::mwrap::openssl` 中以同名函数重新声明,逐项验证语义; + 4. 类型泛型/可变参/逗号列表(`sk_TYPE_*`、`lh_*`、`DEFINE_*_OF`、`ERR_raise_data`、`OBJ_*` 列表宏、`OPENSSL_malloc/free`)→ 只经 macros.hpp。 +- 模板:`digest`(默认:SHA-256、EVP_MAC HMAC、AES-256-GCM 往返、ERR 错误路径);`tls-loopback`(运行时生成自签证书,内存 BIO 对完成 TLS 1.3 握手并收发,无网络、无外部文件)。 +- 测试:NIST/RFC 向量、内存 TLS 1.2/1.3 握手、X509 生成与校验、错误队列、四类分别测试、混用(Windows 覆盖 wincrypt)。 + +## 7. IDX(mcpp-index 唯一 PR)的内容 + +按以往 mwrap 新增(jolt 21c2013、acl 507e7d8)实际触及的文件: + +- `pkgs/c/compat.openssl.lua`(+3.5.9)、依赖方描述符(切 3.5.9 + revision)、`pkgs/c/compat.protobuf.lua`(若 §5.3 修复); +- `mcpplibs.protogen 35.1` 描述符(§6.5); +- `pkgs/mwrap/{abseil,protobuf,openssl}.lua`(GLOBAL/CN、sha256、autoupdate `mwrap-${version}`——已核 `tag_version` 能正确解析 20250512.1、35.1、3.5.9、1.83.0.1); + `pkgs/mwrap/grpc.lua` 增 1.83.0.1; +- 成员(空 `[indices]`,注册根 `mcpp.toml`):`tests/mwrap/abseil-basics`、`protobuf-message`、`protobuf-codegen`、`openssl-digest`、`openssl-tls`; + 组合成员 `tests/mwrap/grpc-protobuf-stack`(unix;mwrap.grpc 1.83.0.1 + protobuf + abseil + openssl:无重复符号、`absl::Status` 同一、只有一份 OpenSSL、分发包通知齐全); + `grpc-cpu` 与 validate.yml 中 grpc installed-template 的 pin 改 1.83.0.1;`grpc-tools` 不变(plugin/rules 仍 1.83.0.1); +- `tests/mwrap/check_installed_template.py`:`STATIC_NOTICES` 三项且支持多许可文件;新增 `--template ` 以覆盖非默认模板(`codegen`、`tls-loopback`); + `validate.yml` 对应步骤(`--deploy`); +- `tests/member-timings.tsv` 为新成员加耗时行(避免按中位数估价撑爆 90 分钟分片); +- `CHANGELOG.md` `[Unreleased]`;本计划终稿(实测 hash、导出数量、CI 结果)。 +- 不需要:站点/搜索/选择器改动(自动发现);`index.toml` `min_mcpp` 不变(2026.9.18.3)。 + +## 8. 执行顺序与候选索引 + +1. 开 IDX draft,先提交 compat 变更(openssl 3.5.9 + 依赖方 + protobuf Windows)→ 候选 commit **C1**,等其 CI 与 `publish-artifact.yml`(若候选需要)完成。 +2. G、A、O 的 CI 固定 C1:checkout `mcpp-index@`,`python3 tests/prepare_builtin_index.py --home <测试 home>`,把完整候选布局作为唯一内置索引 + (ORT/llama Vulkan 先例)。commit 在 PR 合入后仍可经 `refs/pull//head` 取到。 +3. A 合入 → tag → release → CN 镜像核对字节 → IDX 加 abseil 描述符得 **C2** → P 的 CI 固定 C2(类型同一测试需要 mwrap.abseil)。 +4. G、O、P 依次合入并发布,IDX 加对应描述符 → 最终 commit → IDX CI 全绿。 +5. **上传 GitCode 前必须本地验证完毕**:GitHub release 资产下载两次核对 sha256;在本机(Linux x86_64,80% 资源预算内)以 IDX 最终 commit + 做完整冷安装:`check_installed_template.py` 跑每个模板(`--deploy` 校验通知)与全部 tests/mwrap 新成员;另一台 runner 的 CI 结果覆盖 macOS/Windows/arm64。 + 全部通过后才用 gtc 上传 CN 镜像(tag 用裸版本号),上传后再次下载核对字节与 GLOBAL 一致。 +6. IDX 合入 → 等 `publish-artifact.yml` → 对四个包跑 `check_installed_template.py --published-index`,含 `MCPP_INDEX_MIRROR=CN`。 +7. 自动更新:G 打 `mwrap-1.83.0.1` 后每日扫描可能提出 grpc bump,与 IDX 内容相同,关闭即可;#634/#563 先处理(§5.1)。 + +回滚(已获授权):IDX 合入前发现本轮四个版本有缺陷,可覆盖首发——替换 GitHub tag 与资产、更新 hash,重跑当前提交 CI 与真实安装验收。 +GitCode 同名资产无法由工具覆盖:如确需覆盖,我会给出要删除的 release/资产、新文件路径与 sha256,**由你在 GitCode 上操作**,之后我核对字节。 +IDX 合入后的缺陷按规范追加修订尾号。 + +## 9. 风险 + +| 风险 | 应对 | +|---|---| +| abseil/protobuf 头文件 TU-local 暴露 | GCC 仅警告;Clang 审计逐项记录;无法导出的带原因列出 | +| GCC 的 include/import 顺序限制 | §6.1 规则 + prelude 头;限制写入 README | +| Windows protoc 子构建 | §5.3;最坏 codegen 在 Windows 明确报错 | +| openssl 依赖方切换 / 改不了的旧清单造成两份 OpenSSL | 3.5.1 保留;revision + 缓存刷新实测;CHANGELOG/README 指向 grpc 1.83.0.1 | +| grpc 版本检查与 plugin/rules 版本耦合 | §5.2 改写检查 + 目录不变断言 | +| Linux aarch64 上的 compat.openssl 首次进 CI | 失败则在 IDX 修 compat | +| CI 耗时(abseil+protobuf × 多行 × 2 profile;grpc Windows 210 分钟) | 构建缓存;审计只在一行重推导;成员耗时行;grpc Windows 不阻塞 | +| 封装 CI 依赖候选 commit | 固定 commit,经 refs/pull 可达;下次版本升级时改回已发布索引 | +| compat 对齐被他人单独升级打破 | 描述符注释记录对齐关系,升级须联动三个封装与 grpc-m | + +## 10. 规范更新(随 IDX 提交) + +`docs/mwrap.md` 与 `docs/zh/mwrap.md`: +- 命名:`<上游库名>-m` 中的“库名”取库本身的名字,不含仅用于区分语言的仓库后缀(abseil-cpp → abseil-m); +- 记录 GCC 的 include/import 顺序限制,以及 Clang 下运算符必须导出。 + +## 11. 自审修订记录(v2 → v3) + +| 来源 | 结论 | 处理 | +|---|---|---| +| 规范 | 两个 index PR 违反“每仓库一个 PR” | 合并为 IDX,封装 CI 固定候选 commit | +| 规范 | compat 不带许可文件,通知会缺失 | §5.4 由封装保留并部署整个静态闭包的通知 | +| 规范 | abseil 包名偏离先例 | D3 标为新例外并更新规范 | +| 规范 | 规则模块名应为裸名 `protogen`;documents.json 位置;`.gitattributes`;模板 key 写法;README 模块栏 | §6.1 / §6.3 已改 | +| 技术 | macOS Intel、Windows arm64 没有 mcpp 二进制 | 移出矩阵并写明原因 | +| 技术 | `compiler` feature 转发 protoc 会拉 libprotoc 和 upb,多余且可能链接失败 | 删除该 feature,`.compiler` 进基础模块 | +| 技术 | GCC 混用顺序限制(已实测) | §6.1 规则 + prelude 头 | +| 技术 | Clang 要求导出运算符 | 审计以 Clang 为准 | +| 技术 | OpenSSL 缺 `static ossl_inline` 一类 | 宏方案改为四类 | +| 技术 | Windows 上只有 openssl 需要 NOMINMAX;Windows 本已 no-asm | §4.2 / §4.1 已改 | +| 技术 | 依赖方清单不全;Linux aarch64 未测 | §2 / §5.1 / §4.1 已改 | +| 流程 | 封装 CI 看不到未发布依赖 | §8 候选 commit 流程 | +| 流程 | grpc-m CI 拒绝 1.83.0.1;plugin/rules 已占用 1.83.0.1 | §5.2 | +| 流程 | 依赖方先切换会在同图中出现两份 OpenSSL | 与 grpc 1.83.0.1 同在 IDX;旧清单的风险写明 | +| 流程 | 非默认模板未被安装检查覆盖;成员缺耗时行;index 无 wasm 作业 | §7;wasm 留在封装 CI | +| 流程 | 没有回滚路径;CN 资产不可覆盖 | §8 先冷安装再镜像;请求预先授权 | + +## 12. 实施结果(v5,2026-11-20) + +| 仓库 | 合入 | 发布 | sha256(GitHub = GitCode) | +|---|---|---|---| +| abseil-m | #1 squash b206127 | mwrap-20250512.1 | 14cdfc1a08576e20f57b5d81a33333c4d6007d5efd141e5d3b5e9333c7ce198e | +| protobuf-m | #1 squash d5ffe2d | mwrap-35.1 | 26b0305e7d602e049391aa579e7e3cef17ebedff27c4ff416054c03df9c687ed | +| openssl-m | #1 squash 836b473 | mwrap-3.5.9 | df3d0458c18538b9dc680221e2e78e9b5eacc632d37c33efbc744ac64def5e8f | +| grpc-m | #9 squash 48b91a5 | mwrap-1.83.0.1 | 5f71291472460f980f64e0d8d76ac8bccd5f76fa4ed3fd35904b22f8db9d17f9 | + +每个仓库一个 PR,CI 全绿后 squash 合入,main 的 CI 通过后打 tag;资产下载两次与 `.sha256` 一致,本机以真实归档跑 +`check_installed_template.py`(每个模板;默认模板带 `--deploy`)后才上传 GitCode,上传后再次下载核对字节。IDX 为 mcpp-index#845。 + +与方案的偏差与实测结论: + +1. **openssl-m 的结构(review 后改为方案 C)。** 首版提交生成物(约 15.7 万行,98% 由 tools/surface.py 生成)。改为只提交紧凑记录 + `src/surface.tsv`(25.7k 行:一行一项,带平台条件与宏的分类原因),由包的 `build.mcpp` 在构建时生成 `.crypto`、`.ssl`、`.macros` + 三个模块单元与常量测试的比较(对包自身与每个消费者都运行);生成结果与原生成物逐字节相同。`check_exports.py` 校验记录, + 并在 Linux LLVM 行由 clang 重新推导、要求文本一致。规范已加入该做法(docs/mwrap.md)。 +2. **平台。** openssl-m:Linux aarch64 只用 LLVM(mcpp 在该平台的 gcc@16.1.0 是 musl 工具链,compat.openssl 在安装时按 glibc 构建); + Windows 需在 OpenSSL 头文件前引入 ``(async API),`OSSL_BAD_ASYNC_FD`、`CRYPTO_ONCE_STATIC_INIT`、`OPENSSL_fork_*` + 按平台条件导出。grpc-m 1.83.0.1 在 Windows 上可构建链接,但运行时启动即中止(exit 133),因此不声明 Windows。 + protobuf-m 的 `codegen` 仅 Linux/macOS:compat.protobuf 35.1 在 Windows 构建 protoc 失败("multiple rules generate bin/protobuf.lib")。 +3. **mcr 与 compat.openssl。** yizhinailong.mcr 0.1.5 固定 3.5.1,与改用 3.5.9 的依赖同图时版本不可调和;成员或 target 表中的 override + 在工作区内不生效,改为根 mcpp.toml 的 `[workspace.dependencies] compat.openssl = { version = "3.5.9", override = true }`。 +4. **protogen 的依赖名。** 规则中以限定名 `compat.protobuf` 取 dep_dir:grpcgen 与 protogen 同在时短名有歧义。 +5. **GCC 16 的模块合并。** 同一翻译单元 import protobuf 与 Abseil 时须 import Abseil 的分组模块(根模块带入 `abseil.log`, + GCC 随后报 "failed to read compiled module cluster … Bad file data"),且不在命名空间作用域对 protobuf 调用取 decltype + (GCC 报 descriptor.h 成员 "conflicting type for imported declaration")。按此写法,`grpc-protobuf-stack` 在一个翻译单元中 + import 四个封装,GCC 与 Clang 均通过;Clang 两种写法都接受。这两条已写在 protobuf-m 的 abseil_identity_test 与成员注释中。 +6. **成员。** 实际为 `abseil-basics`、`protobuf-message`、`protobuf-codegen`、`openssl-tls`、`grpc-protobuf-stack`(方案中的 + `openssl-digest` 并入 `openssl-tls`;摘要由 openssl 的默认模板在 installed 检查中覆盖);`grpc-cpu` 与 installed gRPC 模板改为 1.83.0.1。 +7. **候选索引固定。** 四个封装仓库的 CI 固定 IDX 候选 commit(compat.openssl 3.5.9 尚未发布);下次版本升级时改回已发布索引。 +8. **本机环境(不影响 CI)。** 新建 home 时 compat.openssl 的安装 hook 经 xlings shim 调用 `gcc`,本机全局 subos 未装 gcc 时失败; + IDX 的测试 home 与 check_installed_template 的新 home 不受影响,openssl-m 的模板脚本在本机受影响(CI 通过)。 + +待 IDX 合入后:`check_installed_template.py --published-index` 对四个包冷安装验收,含 `MCPP_INDEX_MIRROR=CN`。 diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 87711943..301849e6 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -1107,8 +1107,16 @@ jobs: # macos stays at 2. It has never been cancelled, so raising it would # be a prediction rather than a measurement; the formula's 4 is there # the day it is. + # + # ── 2026-11-20: macos 2 -> 3 ──────────────────────────────────── + # + # Observed: on mcpp-index#845 `workspace (macos default 1/2)` was + # cancelled at the 90-minute cap. It carried grpc-cpu (1783s, untimed + # for macos and so priced at the default) and grpc-protobuf-stack, + # whose graph builds gRPC again (cancelled at unit 1800 of 1989). + # Both now have macos rows; three shards put them apart. ln=$(shards_for linux 6) - mn=$(shards_for macos 2) + mn=$(shards_for macos 3) wn=$(shards_for windows 5) # THE TWO LINUX LEGS ARE NOT THE SAME COST, AND THE PLANNER CANNOT # SEE THAT. `plan_shards.lua` takes a PLATFORM and reads one timing @@ -1281,7 +1289,7 @@ jobs: tar -xzf "$archive" echo "MCPP=$PWD/mcpp-${MCPP_VERSION}-${{ matrix.suffix }}/bin/mcpp" >> "$GITHUB_ENV" - name: Immutable archive, default scaffold, strict build and loopback RPC - run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.grpc@1.83.0 --toolchain "${{ matrix.toolchain }}" + run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.grpc@1.83.0.1 --toolchain "${{ matrix.toolchain }}" workspace: # The shard suffix appears only when the platform is actually split, and @@ -1779,6 +1787,31 @@ jobs: shell: bash run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.recastnavigation@1.6.0 --deploy + - name: Installed abseil wrapper archive, actual default basics template and its relocated notices + if: contains(format(' {0} ', env.MEMBERS), ' abseil-basics ') && matrix.toolchain == 'default' + shell: bash + run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.abseil@20250512.1 --deploy + + - name: Installed protobuf wrapper archive, actual default message template and its relocated notices + if: contains(format(' {0} ', env.MEMBERS), ' protobuf-message ') && matrix.toolchain == 'default' + shell: bash + run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.protobuf@35.1 --deploy + + - name: Installed protobuf wrapper archive, codegen template (protoc through the protogen rule) + if: contains(format(' {0} ', env.MEMBERS), ' protobuf-codegen ') && matrix.toolchain == 'default' && runner.os != 'Windows' + shell: bash + run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.protobuf@35.1 --template codegen + + - name: Installed OpenSSL wrapper archive, actual default digest template and its relocated notices + if: contains(format(' {0} ', env.MEMBERS), ' openssl-tls ') && matrix.toolchain == 'default' + shell: bash + run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.openssl@3.5.9 --deploy + + - name: Installed OpenSSL wrapper archive, tls-loopback template (TLS 1.3 over memory BIOs) + if: contains(format(' {0} ', env.MEMBERS), ' openssl-tls ') && matrix.toolchain == 'default' + shell: bash + run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.openssl@3.5.9 --template tls-loopback + - name: Installed ACL wrapper archive, actual default clip template and its relocated notices if: contains(format(' {0} ', env.MEMBERS), ' acl-clip ') && matrix.toolchain == 'default' shell: bash diff --git a/CHANGELOG.md b/CHANGELOG.md index 8247a614..f9598683 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,27 @@ ### Added +- **mcpplibs.mwrap.openssl 3.5.9、mcpplibs.mwrap.abseil 20250512.1、mcpplibs.mwrap.protobuf 35.1、mcpplibs.protogen 35.1(新包), + mcpplibs.mwrap.grpc 1.83.0.1。** 三个新的 mwrap 封装,都是 compat 之上的薄封装,版本与上游一致: + openssl-m(OpenSSL 3.5 LTS:头文件的全部声明以 `using` 导出,334 个 `static ossl_inline` 函数重新声明,11632 个常量与 510 个 + 调用宏映射到 `mcpplibs::mwrap::openssl::macros`,其余宏由 `` 提供;导出面记录在 `src/surface.tsv`, + 由包的 build.mcpp 在构建时生成模块单元;Linux x86_64、Linux aarch64(LLVM)、macOS、Windows),abseil-m(Abseil LTS + 20250512.1,按上游功能分组的子模块;Linux、macOS、Windows、Web),protobuf-m(Protocol Buffers 35.1:消息、描述符、反射、 + 知名类型、JSON 与文本格式、运行时解析 .proto;特性 `gzip`、`upb`、`codegen`;Linux、macOS、Windows、Web,`codegen` 仅 + Linux 与 macOS)。protogen 是 protobuf-m 发布包中的 `rules/` 包:可 import 的 protoc 构建规则,每个 .proto 一条构建边, + `codegen` 特性转发它。grpc 1.83.0.1 是封装修订:改用 compat.openssl 3.5.9,与 openssl-m 同一 OpenSSL;gRPC 源码、模块与 + API 不变,grpcgen 与 grpc-plugin 不变。四者共用 compat.abseil 20250512.1、compat.protobuf 35.1 与 compat.openssl 3.5.9, + 同一程序只链接一份;成员 `grpc-protobuf-stack` 在一个翻译单元中 import 四者并核对 `absl::Status` 同一、OpenSSL 为 3.5.9。 + GCC 16 下同时 import protobuf 与 Abseil 时,import Abseil 的分组模块而非根模块(根模块带入 `abseil.log`,GCC 随后读取模块失败), + 且不在命名空间作用域对 protobuf 调用取 decltype(GCC 报告 descriptor.h 成员 "conflicting type for imported declaration"); + Clang 两种写法都接受。GitHub 与 GitCode(mcpp-res/mwrap-openssl、mwrap-abseil、mwrap-protobuf、mwrap-grpc)字节一致。 +- **compat.openssl 3.5.9(3.5 LTS)。** 三个平台新增 3.5.9,3.5.1 保留;compat.curl、compat.httplib、compat.websocket、 + compat.jwt-cpp、chriskohlhoff.asio 改用 3.5.9,compat.mysql-connector-cpp 26.7.0 以修订 1 改用 3.5.9。仍固定 3.5.1 的已发布 + 清单(如 yizhinailong.mcr 0.1.5)与它们同图时版本不可调和:在工作区根的 `[workspace.dependencies]` 写 + `compat.openssl = { version = "3.5.9", override = true }` 选定一份(本仓库根 mcpp.toml 即如此)。 +- **mwrap 规范。** 上游库名取库本身的名字,不含仅区分语言的仓库后缀(abseil-cpp → abseil-m);同一翻译单元中 `#include` + 在 `import` 之前;以 Clang 审计导出面;手工无法维护的导出面以记录提交、由 build.mcpp 生成(docs/mwrap.md)。 + `tests/mwrap/check_installed_template.py` 新增 `--template`,以覆盖非默认模板。 - **mcpplibs.os 0.4.6。** 新增 `mcpplibs.os.text`:把被启动程序写出的文本按其编码解码为 UTF-8。编码为 `utf8`,或 `console`(Windows 控制台程序的约定:写入管道或文件时用所附着控制台的输出代码页,没有控制台时用系统的 ANSI 代码页;以 cl.exe 与 link.exe 测得,mcpp-community/mcpp#970)。`valid_utf8` 把不是 UTF-8 的序列按最大子部分替换为 U+FFFD。`windows::console_text_code_page` 与 `windows::utf8_from_code_page` 是其下的两项事实。openkal 的范围不变。 - **openkal-platform 0.31.0:openkal 0.26 一族。** 平台的各层、载体、工具与各行使用 openkal 0.26.0 及其各实现(openkal-linux 0.27.0、openkal-macos 0.24.0、openkal-windows 0.23.0、openkal-freebsd 0.2.0、openkal-uefi 0.12.0、openkal-dos 与 openkal-dos-tools 0.2.0、openkal-svr4 0.3.0)、openkal-musl 0.30.0、openkal-llvm-runtime 0.21.0 与 openkal-image 0.7.0。System V 一行可以启动其他程序:openkal-svr4 0.3.0 声明 `openkal.process` 与 `openkal.space`,运行器为 ibcs-us 4.3-2。构建程序链接 mcpp.plugins 0.26.1(mcpplibs.os 0.4.5 接受 openkal 0.26);平台的引擎下界仍为 mcpp 2026.11.17.1。 - **mcpp.platforms 0.2.1。** `aarch64-netbsd` 与 `aarch64-openbsd` 两行获得运行器(此前只构建):dirun 0.8.0 在 diff --git a/docs/mwrap.md b/docs/mwrap.md index 6260f2f7..e994f4b5 100644 --- a/docs/mwrap.md +++ b/docs/mwrap.md @@ -19,6 +19,9 @@ public names, namespaces, type identities, overloads and behavior. | Public API | original upstream spelling | `cv::Mat`, `Ort::Session` | | Macro mapping module | `.macros` | `mcpplibs.mwrap.opencv.macros` | +The upstream name is the library's own name, without a repository suffix that +only marks the language: Abseil's repository is abseil-cpp, the wrapper is +`abseil-m` and the package `mcpplibs.mwrap.abseil`. Do not add `-m` to package/module names or relocate ordinary upstream entities into a new C++ namespace. Preserve legacy published package identities and bytes; publish a new mwrap identity rather than rewriting old releases. @@ -95,6 +98,15 @@ Preserve argument evaluation and call-site behavior; an assertion macro is not automatically equivalent to a function. Active macros expand even in qualified names, so test and document raw-header versus mapped-constant usage. +A translation unit that both includes upstream headers and imports a wrapper +puts every `#include` before the first `import` (GCC rejects a header after an +import of a module that already declared its entities); state this in the README. +Clang requires an operator found by argument-dependent lookup at namespace scope +to be exported too, so audit the surface with Clang. A surface too large to keep +by hand is derived by a tool into a compact record committed to the repository, +and `build.mcpp` renders the module units from it at build time, so nothing +generated is committed (openssl-m: `src/surface.tsv`, `tools/surface.py`). + ## Features Use lowercase kebab-case and recognizable upstream vocabulary: diff --git a/docs/zh/mwrap.md b/docs/zh/mwrap.md index edc497cd..e2351212 100644 --- a/docs/zh/mwrap.md +++ b/docs/zh/mwrap.md @@ -17,6 +17,8 @@ | API | 保留上游 | cv::Mat、Ort::Session | | 宏映射模块 | `<根模块>.macros` | mcpplibs.mwrap.opencv.macros | +上游库名取库本身的名字,不含仅用于区分语言的仓库后缀:Abseil 的仓库是 abseil-cpp, +封装仓库为 `abseil-m`,包为 `mcpplibs.mwrap.abseil`。 包和模块名不带 -m,不创建另一套普通 API namespace。已发布旧身份、版本和内容保持不可变, 通过新 mwrap 身份迁移。简单实现优先依赖 compat.xxx;复杂库可以 vendor 到 upstream/。 同一程序的同一实现只能有一个所有者,不能把 compat 与 vendored 实现重复链接。 @@ -66,6 +68,12 @@ metadata 与上游锁文件仍明确记录来源。破坏性模块/API 变化必 控制流宏通过 `/macros.hpp>` 保留,在 import 前 include。 限定名仍会被宏展开;原始宏与映射常量的用法必须分别测试,断言函数不能默认代替断言宏。 +同一翻译单元既 include 上游头文件又 import 封装模块时,所有 `#include` 放在第一个 `import` 之前 +(GCC 拒绝在已声明同一实体的模块被 import 之后再引入头文件),README 写明。Clang 要求命名空间作用域上 +经实参依赖查找找到的运算符也要导出,因此以 Clang 审计导出面。手工无法维护的大型导出面由工具推导为 +紧凑的记录提交到仓库,由 `build.mcpp` 在构建时生成模块单元,不提交生成物 +(openssl-m:`src/surface.tsv`、`tools/surface.py`)。 + ## feature 采用小写 kebab-case,复用上游术语:模块 dnn/highgui/stitching;后端 highgui-gtk3、 diff --git a/mcpp.toml b/mcpp.toml index a9b4a51f..dab2bf94 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -45,11 +45,16 @@ members = [ "tests/mwrap/libjpeg-turbo-decode", "tests/mwrap/libarchive-bundle", "tests/mwrap/libc-stdio", + "tests/mwrap/abseil-basics", + "tests/mwrap/protobuf-message", + "tests/mwrap/protobuf-codegen", + "tests/mwrap/openssl-tls", "tests/mwrap/miniaudio-engine", "tests/mwrap/freetype-render", "tests/mwrap/harfbuzz-shape", "tests/mwrap/sdl3-headless", "tests/mwrap/grpc-cpu", + "tests/mwrap/grpc-protobuf-stack", "tests/mwrap/grpc-tools", "tests/mwrap/godot-math", "tests/mwrap/godot-sdk", @@ -313,5 +318,14 @@ members = [ # asio itself comes from the published index while its `ssl` feature dep, # compat.openssl, resolves from this checkout — which is what lets an # unmerged compat descriptor be tested through a feature that activates it. + +# The index's OpenSSL dependents build on compat.openssl 3.5.9 (3.5 LTS). +# yizhinailong.mcr's own manifest still pins 3.5.1 beside compat.curl (3.5.9), +# and an override is read only from the root's tables, so the workspace +# chooses 3.5.9 for every member graph that holds compat.openssl — what a +# project depending on mcr writes in its own mcpp.toml. +[workspace.dependencies] +compat.openssl = { version = "3.5.9", override = true } + [indices] compat = { path = "." } diff --git a/pkgs/c/chriskohlhoff.asio.lua b/pkgs/c/chriskohlhoff.asio.lua index 3fa095cd..a70e264a 100644 --- a/pkgs/c/chriskohlhoff.asio.lua +++ b/pkgs/c/chriskohlhoff.asio.lua @@ -344,7 +344,7 @@ using ::asio::ssl::error::make_error_code; }, ["ssl"] = { defines = { "MCPP_FEATURE_SSL" }, - deps = { ["compat.openssl"] = "3.5.1" }, + deps = { ["compat.openssl"] = "3.5.9" }, sources = { "*/src/asio_ssl.cpp" }, }, }, diff --git a/pkgs/c/compat.curl.lua b/pkgs/c/compat.curl.lua index 3c2a63e0..8398edc3 100644 --- a/pkgs/c/compat.curl.lua +++ b/pkgs/c/compat.curl.lua @@ -331,13 +331,13 @@ package = { linux = { cflags = { "-DHAVE_CONFIG_H", "-D_GNU_SOURCE" }, - deps = { ["compat.openssl"] = "3.5.1" }, + deps = { ["compat.openssl"] = "3.5.9" }, ldflags = { "-lpthread" }, }, macosx = { cflags = { "-DHAVE_CONFIG_H", "-D_GNU_SOURCE" }, - deps = { ["compat.openssl"] = "3.5.1" }, + deps = { ["compat.openssl"] = "3.5.9" }, -- curl reaches into the system for proxy configuration on Apple: -- SystemConfiguration for SCDynamicStoreCopyProxies, and -- CoreFoundation for the CF objects that hands back. diff --git a/pkgs/c/compat.httplib.lua b/pkgs/c/compat.httplib.lua index 0c575a25..3537b881 100644 --- a/pkgs/c/compat.httplib.lua +++ b/pkgs/c/compat.httplib.lua @@ -72,7 +72,7 @@ package = { features = { ["tls"] = { defines = { "CPPHTTPLIB_OPENSSL_SUPPORT" }, - deps = { ["compat.openssl"] = "3.5.1" }, + deps = { ["compat.openssl"] = "3.5.9" }, }, ["zlib"] = { defines = { "CPPHTTPLIB_ZLIB_SUPPORT" }, diff --git a/pkgs/c/compat.jwt-cpp.lua b/pkgs/c/compat.jwt-cpp.lua index 48a99950..5802e43e 100644 --- a/pkgs/c/compat.jwt-cpp.lua +++ b/pkgs/c/compat.jwt-cpp.lua @@ -11,7 +11,7 @@ -- OpenSSL is an UNCONDITIONAL compile-time dependency: jwt.h includes -- outside any feature guard (including ssl.h, even though only -- libcrypto APIs are called — upstream's CMake target links OpenSSL::SSL + --- OpenSSL::Crypto both). compat.openssl 3.5.1 provides headers AND carries the +-- OpenSSL::Crypto both). compat.openssl 3.5.9 provides headers AND carries the -- per-platform link flags itself; they propagate to consumers, so this -- descriptor adds no ldflags of its own. Windows support in compat.openssl is -- evidenced by merged #211 + the asio-ssl member's real-TLS test. @@ -78,6 +78,6 @@ int mcpp_compat_jwt_cpp_anchor(void) { return 0; } }, sources = { "mcpp_generated/jwt_cpp_anchor.cpp" }, targets = { ["jwt_cpp"] = { kind = "lib" } }, - deps = { ["compat.openssl"] = "3.5.1" }, + deps = { ["compat.openssl"] = "3.5.9" }, }, } diff --git a/pkgs/c/compat.mysql-connector-cpp.lua b/pkgs/c/compat.mysql-connector-cpp.lua index c9d27843..240ea979 100644 --- a/pkgs/c/compat.mysql-connector-cpp.lua +++ b/pkgs/c/compat.mysql-connector-cpp.lua @@ -12,7 +12,7 @@ package = { linux = { deps = { "compat:libmysqlclient@8.4.6", - "compat:openssl@3.5.1", + "compat:openssl@3.5.9", "xim:cmake@latest", "xim:make@latest", }, @@ -22,12 +22,14 @@ package = { CN = "https://gitcode.com/mcpp-res/mysql-connector-cpp/releases/download/26.7.0/mysql-connector-cpp-26.7.0.tar.gz", }, sha256 = "b2299862eefc33fd71c0aac68328305671805fc955e6bd2578ef205c10f98550", + -- 1: built against compat.openssl 3.5.9 (was 3.5.1). + revision = 1, }, }, macosx = { deps = { "compat:libmysqlclient@8.4.6", - "compat:openssl@3.5.1", + "compat:openssl@3.5.9", "xim:cmake@latest", }, ["26.7.0"] = { @@ -36,6 +38,8 @@ package = { CN = "https://gitcode.com/mcpp-res/mysql-connector-cpp/releases/download/26.7.0/mysql-connector-cpp-26.7.0.tar.gz", }, sha256 = "b2299862eefc33fd71c0aac68328305671805fc955e6bd2578ef205c10f98550", + -- 1: built against compat.openssl 3.5.9 (was 3.5.1). + revision = 1, }, }, }, @@ -48,7 +52,7 @@ package = { targets = { ["mysql_connector_cpp"] = { kind = "lib" } }, deps = { ["compat.libmysqlclient"] = "8.4.6", - ["compat.openssl"] = "3.5.1", + ["compat.openssl"] = "3.5.9", }, -- THE STANDARD LIBRARY THE STATIC LIBRARIES WERE COMPILED AGAINST, per @@ -214,7 +218,7 @@ function install() local ok, result = pcall(function() local srcroot = find_source_root() local mysql = find_dep_install_dir("compat:libmysqlclient", "8.4.6") - local openssl = find_dep_install_dir("compat:openssl", "3.5.1") + local openssl = find_dep_install_dir("compat:openssl", "3.5.9") hook_log("srcroot=" .. tostring(srcroot) .. " mysql=" .. tostring(mysql) .. " openssl=" .. tostring(openssl)) if not srcroot or not mysql or not openssl then diff --git a/pkgs/c/compat.openssl.lua b/pkgs/c/compat.openssl.lua index e4b63f02..29874ae7 100644 --- a/pkgs/c/compat.openssl.lua +++ b/pkgs/c/compat.openssl.lua @@ -1,4 +1,4 @@ --- compat.openssl — OpenSSL 3.5.1 built from source as a portable, static +-- compat.openssl — OpenSSL 3.5 LTS (3.5.9; 3.5.1 kept) built from source as a portable, static -- library that provides TLS/crypto for consumers (e.g. chriskohlhoff.asio's -- `ssl` feature, which wraps asio::ssl::context / asio::ssl::stream). -- @@ -76,6 +76,13 @@ package = { "xim:make@latest", "xim:perl@latest", "xim:glibc@>=2.39", "xim:linux-headers@5.11.1", }, + ["3.5.9"] = { + url = { + GLOBAL = "https://github.com/openssl/openssl/releases/download/openssl-3.5.9/openssl-3.5.9.tar.gz", + CN = "https://gitcode.com/mcpp-res/openssl/releases/download/3.5.9/openssl-3.5.9.tar.gz", + }, + sha256 = "603f5602e2eef00d77fbd429d34dcd5822bb301757a1bc9cdb24c670f1eb859a", + }, ["3.5.1"] = { url = { GLOBAL = "https://github.com/openssl/openssl/releases/download/openssl-3.5.1/openssl-3.5.1.tar.gz", @@ -98,6 +105,13 @@ package = { -- it resolves here and the platform gets the same -- known-complete interpreter linux does. deps = { "xim:perl@latest" }, + ["3.5.9"] = { + url = { + GLOBAL = "https://github.com/openssl/openssl/releases/download/openssl-3.5.9/openssl-3.5.9.tar.gz", + CN = "https://gitcode.com/mcpp-res/openssl/releases/download/3.5.9/openssl-3.5.9.tar.gz", + }, + sha256 = "603f5602e2eef00d77fbd429d34dcd5822bb301757a1bc9cdb24c670f1eb859a", + }, ["3.5.1"] = { url = { GLOBAL = "https://github.com/openssl/openssl/releases/download/openssl-3.5.1/openssl-3.5.1.tar.gz", @@ -109,6 +123,13 @@ package = { windows = { -- Same source tarball as the other platforms; the build differs -- (VC-WIN64A + nmake), not the source. See _install_windows(). + ["3.5.9"] = { + url = { + GLOBAL = "https://github.com/openssl/openssl/releases/download/openssl-3.5.9/openssl-3.5.9.tar.gz", + CN = "https://gitcode.com/mcpp-res/openssl/releases/download/3.5.9/openssl-3.5.9.tar.gz", + }, + sha256 = "603f5602e2eef00d77fbd429d34dcd5822bb301757a1bc9cdb24c670f1eb859a", + }, ["3.5.1"] = { url = { GLOBAL = "https://github.com/openssl/openssl/releases/download/openssl-3.5.1/openssl-3.5.1.tar.gz", diff --git a/pkgs/c/compat.websocket.lua b/pkgs/c/compat.websocket.lua index 1cc9f4bb..443b6b0a 100644 --- a/pkgs/c/compat.websocket.lua +++ b/pkgs/c/compat.websocket.lua @@ -181,7 +181,7 @@ package = { -- SocketOpenSSL and this is its one optional source. defines = { "IXWEBSOCKET_USE_TLS=1", "IXWEBSOCKET_USE_OPEN_SSL=1" }, sources = { "*/ixwebsocket/IXSocketOpenSSL.cpp" }, - deps = { ["compat.openssl"] = "3.5.1" }, + deps = { ["compat.openssl"] = "3.5.9" }, }, ["server"] = { implies = { "zlib" }, diff --git a/pkgs/mwrap/abseil.lua b/pkgs/mwrap/abseil.lua new file mode 100644 index 00000000..59b34c42 --- /dev/null +++ b/pkgs/mwrap/abseil.lua @@ -0,0 +1,54 @@ +-- Immutable upstream-aligned mwrap source release (abseil-m mwrap-20250512.1); +-- GLOBAL and CN bytes are identical. +-- +-- The wrapper is Apache-2.0; the library it links, compat.abseil (Abseil LTS +-- 20250512.1, the same release compat.protobuf 35.1 and mcpplibs.mwrap.grpc +-- build on, so a program using them links one Abseil), is Apache-2.0 too, +-- linked statically: the package deploys Abseil's README and LICENSE beside +-- programs (runtime.deploy). The repository is abseil-m (upstream's +-- repository is abseil-cpp; the library is Abseil). The wrapper's own CI covers +-- Linux, macOS, Windows and the Web (wasm32-emscripten, tests under node). +package = { + spec = "1", + namespace = "mcpplibs.mwrap", + name = "abseil", + description = "abseil-m: Abseil LTS 20250512.1 (strings, containers, hashing, status, time, synchronization, logging, flags, random) as C++23 modules", + licenses = { "Apache-2.0" }, + repo = "https://github.com/mcpplibs/abseil-m", + type = "package", + extensions = { + autoupdate = { + tag = "mwrap-${version}", + url = "https://github.com/mcpplibs/abseil-m/releases/download/${tag}/abseil-m-${version}.tar.gz", + }, + }, + xpm = { + linux = { + ["20250512.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/abseil-m/releases/download/mwrap-20250512.1/abseil-m-20250512.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-abseil/releases/download/20250512.1/abseil-m-20250512.1.tar.gz", + }, + sha256 = "14cdfc1a08576e20f57b5d81a33333c4d6007d5efd141e5d3b5e9333c7ce198e", + }, + }, + macosx = { + ["20250512.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/abseil-m/releases/download/mwrap-20250512.1/abseil-m-20250512.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-abseil/releases/download/20250512.1/abseil-m-20250512.1.tar.gz", + }, + sha256 = "14cdfc1a08576e20f57b5d81a33333c4d6007d5efd141e5d3b5e9333c7ce198e", + }, + }, + windows = { + ["20250512.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/abseil-m/releases/download/mwrap-20250512.1/abseil-m-20250512.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-abseil/releases/download/20250512.1/abseil-m-20250512.1.tar.gz", + }, + sha256 = "14cdfc1a08576e20f57b5d81a33333c4d6007d5efd141e5d3b5e9333c7ce198e", + }, + }, + }, +} diff --git a/pkgs/mwrap/grpc.lua b/pkgs/mwrap/grpc.lua index d58861f2..7ec6bbed 100644 --- a/pkgs/mwrap/grpc.lua +++ b/pkgs/mwrap/grpc.lua @@ -1,4 +1,10 @@ -- Immutable upstream-aligned mwrap source release; GLOBAL and CN bytes are identical. +-- +-- 1.83.0.1 (wrapper revision): compat.openssl 3.5.9 (3.5 LTS) instead of 3.5.1, +-- the OpenSSL of mcpplibs.mwrap.openssl 3.5.9, so a program using both links +-- one OpenSSL; the gRPC source, modules and API are those of 1.83.0, and +-- grpcgen and grpc-plugin are unchanged. Linux and macOS: on Windows the +-- package builds and links, and its runtime aborts at startup (not claimed). package = { spec = "1", namespace = "mcpplibs.mwrap", @@ -15,6 +21,13 @@ package = { }, xpm = { linux = { + ["1.83.0.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/grpc-m/releases/download/mwrap-1.83.0.1/grpc-m-1.83.0.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-grpc/releases/download/1.83.0.1/grpc-m-1.83.0.1.tar.gz", + }, + sha256 = "5f71291472460f980f64e0d8d76ac8bccd5f76fa4ed3fd35904b22f8db9d17f9", + }, ["1.83.0"] = { url = { GLOBAL = "https://github.com/mcpplibs/grpc-m/releases/download/mwrap-1.83.0/grpc-m-1.83.0.tar.gz", @@ -24,6 +37,13 @@ package = { }, }, macosx = { + ["1.83.0.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/grpc-m/releases/download/mwrap-1.83.0.1/grpc-m-1.83.0.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-grpc/releases/download/1.83.0.1/grpc-m-1.83.0.1.tar.gz", + }, + sha256 = "5f71291472460f980f64e0d8d76ac8bccd5f76fa4ed3fd35904b22f8db9d17f9", + }, ["1.83.0"] = { url = { GLOBAL = "https://github.com/mcpplibs/grpc-m/releases/download/mwrap-1.83.0/grpc-m-1.83.0.tar.gz", diff --git a/pkgs/mwrap/openssl.lua b/pkgs/mwrap/openssl.lua new file mode 100644 index 00000000..ba208b52 --- /dev/null +++ b/pkgs/mwrap/openssl.lua @@ -0,0 +1,56 @@ +-- Immutable upstream-aligned mwrap source release (openssl-m mwrap-3.5.9); +-- GLOBAL and CN bytes are identical. +-- +-- The wrapper is Apache-2.0; the library it links, compat.openssl 3.5.9 (the +-- 3.5 LTS, built at install time by OpenSSL's own Configure for the host; the +-- OpenSSL mcpplibs.mwrap.grpc 1.83.0.1 links, so a program using both links +-- one OpenSSL), is Apache-2.0 too, linked statically: the package deploys +-- OpenSSL's README and LICENSE beside programs (runtime.deploy). The module +-- surface is recorded in the archive's src/surface.tsv and rendered by its +-- build.mcpp as the package is built. The wrapper's CI covers Linux x86_64 +-- (GCC, LLVM), Linux aarch64 (LLVM: mcpp's GCC there is a musl toolchain), +-- macOS and Windows; there is no cross target. +package = { + spec = "1", + namespace = "mcpplibs.mwrap", + name = "openssl", + description = "openssl-m: OpenSSL 3.5 LTS (libcrypto and libssl) as C++23 modules: every declaration of the headers, the inline functions declared again, the constants and call macros under their names", + licenses = { "Apache-2.0" }, + repo = "https://github.com/mcpplibs/openssl-m", + type = "package", + extensions = { + autoupdate = { + tag = "mwrap-${version}", + url = "https://github.com/mcpplibs/openssl-m/releases/download/${tag}/openssl-m-${version}.tar.gz", + }, + }, + xpm = { + linux = { + ["3.5.9"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/openssl-m/releases/download/mwrap-3.5.9/openssl-m-3.5.9.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-openssl/releases/download/3.5.9/openssl-m-3.5.9.tar.gz", + }, + sha256 = "df3d0458c18538b9dc680221e2e78e9b5eacc632d37c33efbc744ac64def5e8f", + }, + }, + macosx = { + ["3.5.9"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/openssl-m/releases/download/mwrap-3.5.9/openssl-m-3.5.9.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-openssl/releases/download/3.5.9/openssl-m-3.5.9.tar.gz", + }, + sha256 = "df3d0458c18538b9dc680221e2e78e9b5eacc632d37c33efbc744ac64def5e8f", + }, + }, + windows = { + ["3.5.9"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/openssl-m/releases/download/mwrap-3.5.9/openssl-m-3.5.9.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-openssl/releases/download/3.5.9/openssl-m-3.5.9.tar.gz", + }, + sha256 = "df3d0458c18538b9dc680221e2e78e9b5eacc632d37c33efbc744ac64def5e8f", + }, + }, + }, +} diff --git a/pkgs/mwrap/protobuf.lua b/pkgs/mwrap/protobuf.lua new file mode 100644 index 00000000..80240152 --- /dev/null +++ b/pkgs/mwrap/protobuf.lua @@ -0,0 +1,58 @@ +-- Immutable upstream-aligned mwrap source release (protobuf-m mwrap-35.1); +-- GLOBAL and CN bytes are identical. +-- +-- The wrapper is Apache-2.0; the library it links, compat.protobuf (Protocol +-- Buffers 35.1 over compat.abseil 20250512.1, the release mcpplibs.mwrap.grpc +-- builds on, so a program using both links one protobuf and one Abseil), is +-- BSD-3-Clause, linked statically: the package deploys the protobuf, +-- utf8_range and Abseil notices beside programs (runtime.deploy). Features: +-- `gzip` (the gzip streams, over compat.zlib), `upb` (protobuf's C runtime), +-- `codegen` (protoc and the mcpplibs.protogen build rule, re-exported, so a +-- consumer's build.mcpp says `protogen::generate_all()`). The wrapper's own CI +-- covers Linux, macOS, Windows and the Web (wasm32-emscripten, tests under +-- node); `codegen` on Linux and macOS (compat.protobuf 35.1 does not build +-- protoc on Windows). +package = { + spec = "1", + namespace = "mcpplibs.mwrap", + name = "protobuf", + description = "protobuf-m: Protocol Buffers 35.1 (messages, descriptors, reflection, well-known types, JSON and text formats, run-time .proto parsing) as C++23 modules; protoc code generation through the codegen feature", + licenses = { "Apache-2.0" }, + repo = "https://github.com/mcpplibs/protobuf-m", + type = "package", + extensions = { + autoupdate = { + tag = "mwrap-${version}", + url = "https://github.com/mcpplibs/protobuf-m/releases/download/${tag}/protobuf-m-${version}.tar.gz", + }, + }, + xpm = { + linux = { + ["35.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/protobuf-m/releases/download/mwrap-35.1/protobuf-m-35.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-protobuf/releases/download/35.1/protobuf-m-35.1.tar.gz", + }, + sha256 = "26b0305e7d602e049391aa579e7e3cef17ebedff27c4ff416054c03df9c687ed", + }, + }, + macosx = { + ["35.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/protobuf-m/releases/download/mwrap-35.1/protobuf-m-35.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-protobuf/releases/download/35.1/protobuf-m-35.1.tar.gz", + }, + sha256 = "26b0305e7d602e049391aa579e7e3cef17ebedff27c4ff416054c03df9c687ed", + }, + }, + windows = { + ["35.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/protobuf-m/releases/download/mwrap-35.1/protobuf-m-35.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-protobuf/releases/download/35.1/protobuf-m-35.1.tar.gz", + }, + sha256 = "26b0305e7d602e049391aa579e7e3cef17ebedff27c4ff416054c03df9c687ed", + }, + }, + }, +} diff --git a/pkgs/p/protogen.lua b/pkgs/p/protogen.lua new file mode 100644 index 00000000..07791796 --- /dev/null +++ b/pkgs/p/protogen.lua @@ -0,0 +1,63 @@ +-- protogen: protoc as an importable mcpp build rule (host-module), the generic, +-- protobuf-owned layer of code generation: one build-graph edge per .proto, +-- C++ messages by default, any protoc plugin as a parameter. +-- +-- [build-dependencies] +-- mcpplibs.protogen = { version = "35.1", host-module = true } +-- +-- // build.mcpp +-- import mcpp; +-- import protogen; +-- int main() { return protogen::generate_all() ? 0 : 1; } +-- +-- mcpplibs.mwrap.protobuf's `codegen` feature brings it (and protoc) with one +-- line. Its version is protobuf's: the rule, the protoc it drives and the +-- runtime the generated code links are one release. It is the rules/ package +-- of the protobuf-m release (mwrap-35.1), the same bytes as +-- pkgs/mwrap/protobuf.lua; grpcgen (pkgs/g/grpcgen.lua) is its gRPC +-- counterpart, with the same API and the gRPC plugin added. +package = { + spec = "1", + namespace = "mcpplibs", + name = "protogen", + description = "protoc as an importable mcpp build rule (host-module): one edge per .proto, C++ messages, any protoc plugin", + licenses = { "Apache-2.0" }, + repo = "https://github.com/mcpplibs/protobuf-m", + type = "package", + extensions = { + autoupdate = { + tag = "mwrap-${version}", + url = "https://github.com/mcpplibs/protobuf-m/releases/download/${tag}/protobuf-m-${version}.tar.gz", + }, + }, + xpm = { + linux = { + ["35.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/protobuf-m/releases/download/mwrap-35.1/protobuf-m-35.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-protobuf/releases/download/35.1/protobuf-m-35.1.tar.gz", + }, + sha256 = "26b0305e7d602e049391aa579e7e3cef17ebedff27c4ff416054c03df9c687ed", + }, + }, + macosx = { + ["35.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/protobuf-m/releases/download/mwrap-35.1/protobuf-m-35.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-protobuf/releases/download/35.1/protobuf-m-35.1.tar.gz", + }, + sha256 = "26b0305e7d602e049391aa579e7e3cef17ebedff27c4ff416054c03df9c687ed", + }, + }, + windows = { + ["35.1"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/protobuf-m/releases/download/mwrap-35.1/protobuf-m-35.1.tar.gz", + CN = "https://gitcode.com/mcpp-res/mwrap-protobuf/releases/download/35.1/protobuf-m-35.1.tar.gz", + }, + sha256 = "26b0305e7d602e049391aa579e7e3cef17ebedff27c4ff416054c03df9c687ed", + }, + }, + }, + mcpp = "*/rules/mcpp.toml", +} diff --git a/tests/examples/asio-ssl/mcpp.toml b/tests/examples/asio-ssl/mcpp.toml index ac79efc8..929c7aaa 100644 --- a/tests/examples/asio-ssl/mcpp.toml +++ b/tests/examples/asio-ssl/mcpp.toml @@ -1,6 +1,6 @@ # chriskohlhoff.asio `ssl` FEATURE test — the end-to-end assertion for the # opt-in TLS surface: activating the feature compiles asio_ssl.cpp into the -# package, pulls compat.openssl (3.5.1, built from source), and links +# package, pulls compat.openssl (3.5.9, built from source), and links # libssl.a + libcrypto.a. tests/ssl.cpp then runs a real TLS handshake over # loopback. # diff --git a/tests/examples/mcr/mcpp.toml b/tests/examples/mcr/mcpp.toml index 391e4ed0..54294a29 100644 --- a/tests/examples/mcr/mcpp.toml +++ b/tests/examples/mcr/mcpp.toml @@ -16,3 +16,7 @@ version = "0.1.0" [dependencies.yizhinailong] mcr = "0.1.5" + +# mcr's own manifest (0.1.5 and 0.3.0) pins compat.openssl 3.5.1 beside +# compat.curl, which links 3.5.9; the workspace root's [workspace.dependencies] +# chooses 3.5.9 for the graph (an override is read from the root only). diff --git a/tests/examples/openssl/mcpp.toml b/tests/examples/openssl/mcpp.toml index 6fd2862b..d5425401 100644 --- a/tests/examples/openssl/mcpp.toml +++ b/tests/examples/openssl/mcpp.toml @@ -17,19 +17,19 @@ name = "openssl-tests" version = "0.1.0" [target.'cfg(linux)'.dependencies.compat] -openssl = "3.5.1" +openssl = "3.5.9" [target.'cfg(linux)'.build] cxxflags = ["-DHAVE_OPENSSL=1"] [target.'cfg(macos)'.dependencies.compat] -openssl = "3.5.1" +openssl = "3.5.9" [target.'cfg(macos)'.build] cxxflags = ["-DHAVE_OPENSSL=1"] [target.'cfg(windows)'.dependencies.compat] -openssl = "3.5.1" +openssl = "3.5.9" [target.'cfg(windows)'.build] cxxflags = ["-DHAVE_OPENSSL=1"] diff --git a/tests/member-timings.tsv b/tests/member-timings.tsv index 86a48611..75d036a7 100644 --- a/tests/member-timings.tsv +++ b/tests/member-timings.tsv @@ -20,6 +20,8 @@ linux eui-neo-sdl2 591 linux godot-cpp-module-v10 568 macos grpc-codegen 1156 macos grpc-module 719 +macos grpc-cpu 1800 +macos grpc-protobuf-stack 1800 macos opencv-module-dnn 325 macos protobuf-protoc 314 macos godot-cpp-module 276 @@ -41,6 +43,7 @@ windows protobuf-upb 151 linux ffmpeg-cpu 208 linux godot-math 758 linux grpc-cpu 1416 +linux grpc-protobuf-stack 1800 linux grpc-tools 480 linux imgui-headless 168 linux llamacpp-cpu 264 @@ -55,3 +58,17 @@ windows opencv-webp 1104 linux godot-sdk 300 macos godot-sdk 300 windows godot-sdk 700 + +# Abseil wrapper: the member, then its installed archive, default template and notices. +linux abseil-basics 420 +linux protobuf-message 900 +linux openssl-tls 600 +linux protobuf-codegen 1200 +macos abseil-basics 300 +macos protobuf-message 600 +macos openssl-tls 400 +macos protobuf-codegen 800 +windows abseil-basics 420 +windows protobuf-message 900 +windows openssl-tls 900 +windows protobuf-codegen 300 diff --git a/tests/mwrap/abseil-basics/mcpp.toml b/tests/mwrap/abseil-basics/mcpp.toml new file mode 100644 index 00000000..a42f9288 --- /dev/null +++ b/tests/mwrap/abseil-basics/mcpp.toml @@ -0,0 +1,11 @@ +[package] +name = "abseil-basics" +version = "20250512.1" +description = "Consume the immutable Abseil wrapper: strings, a hash map, StatusOr with its error path, civil time in UTC, a mutex shared by threads, the mapped release constant" + +[indices] +# Use the complete candidate built-in index prepared by tests/run_members.sh. +# An explicit empty table overrides the legacy workspace redirect. + +[dependencies] +mcpplibs.mwrap.abseil = "20250512.1" diff --git a/tests/mwrap/abseil-basics/tests/behavior.cpp b/tests/mwrap/abseil-basics/tests/behavior.cpp new file mode 100644 index 00000000..5ea1c790 --- /dev/null +++ b/tests/mwrap/abseil-basics/tests/behavior.cpp @@ -0,0 +1,64 @@ +// The installed Abseil wrapper through import alone: strings split, joined and +// formatted; a flat_hash_map counting words; StatusOr with its error path; +// civil time in UTC; a Mutex shared by threads (not on the Web, which runs +// without threads); the release constant mapped from its macro. +import std; +import mcpplibs.mwrap.abseil; + +namespace { +int failures = 0; +void check(bool ok, std::string_view what) { + if (!ok) { + ++failures; + std::println("FAIL: {}", what); + } +} + +absl::StatusOr parse_port(std::string_view text) { + int port = 0; + if (!absl::SimpleAtoi(text, &port)) return absl::InvalidArgumentError(absl::StrCat("not a number: ", text)); + if (port <= 0 || port > 65535) return absl::OutOfRangeError(absl::StrFormat("port %d out of range", port)); + return port; +} +} // namespace + +int main() { + std::vector words = absl::StrSplit("the quick brown fox jumps over the lazy dog the end", ' '); + absl::flat_hash_map counts; + for (const auto& word : words) ++counts[word]; + check(counts["the"] == 3 && counts.size() == 9, "flat_hash_map word counts"); + check(absl::StrJoin(std::vector{1, 2, 3}, "-") == "1-2-3", "StrJoin"); + check(absl::StrFormat("%08.3f", 3.14159) == "0003.142", "StrFormat"); + + const auto good = parse_port("8080"); + const auto bad = parse_port("http"); + const auto far = parse_port("70000"); + check(good.ok() && *good == 8080, "StatusOr value"); + check(!bad.ok() && absl::IsInvalidArgument(bad.status()) && bad.status().message() == "not a number: http", "StatusOr error"); + check(absl::IsOutOfRange(far.status()), "StatusOr out of range"); + + const absl::CivilSecond moment(2026, 11, 19, 12, 30, 0); + const absl::Time time = absl::FromCivil(moment, absl::UTCTimeZone()); + check(absl::FormatTime("%Y-%m-%dT%H:%M:%S", time + absl::Hours(36), absl::UTCTimeZone()) == "2026-11-21T00:30:00", "civil time in UTC"); + check(absl::ToUnixSeconds(time) == 1795091400, "Unix seconds"); + +#if !defined(__EMSCRIPTEN__) + absl::Mutex mutex; + long total = 0; + { + std::vector threads; + for (int t = 0; t < 4; ++t) + threads.emplace_back([&] { + for (int i = 0; i < 1000; ++i) { + absl::MutexLock lock(&mutex); + ++total; + } + }); + } + check(total == 4000, "Mutex across threads"); +#endif + + check(mcpplibs::mwrap::abseil::macros::ABSL_LTS_RELEASE_VERSION == 20250512, "ABSL_LTS_RELEASE_VERSION"); + std::println("{}", failures == 0 ? "abseil-basics: all checks passed" : "abseil-basics: FAILED"); + return failures == 0 ? 0 : 1; +} diff --git a/tests/mwrap/check_installed_template.py b/tests/mwrap/check_installed_template.py index 5e108f1b..2ccab41f 100644 --- a/tests/mwrap/check_installed_template.py +++ b/tests/mwrap/check_installed_template.py @@ -29,6 +29,9 @@ "mcpplibs.mwrap.libjpeg_turbo": ("libjpeg_turbo.cppm", "upstream/libjpeg-turbo/README.ijg", b"Independent JPEG Group", b"libjpeg-turbo Project"), "mcpplibs.mwrap.basis_universal": ("basis_universal.cppm", "upstream/basis_universal/NOTICE", b"Binomial LLC", b"Basis Universal"), "mcpplibs.mwrap.libc": ("libc.cppm", None, b"", b"ISO/IEC 9899:2018"), + "mcpplibs.mwrap.abseil": ("abseil.cppm", "upstream/abseil-cpp/LICENSE", b"Apache License", b"The Abseil Authors"), + "mcpplibs.mwrap.openssl": ("openssl.cppm", "upstream/openssl/LICENSE.txt", b"Apache License", b"The OpenSSL Project Authors"), + "mcpplibs.mwrap.protobuf": ("protobuf.cppm", "upstream/protobuf/LICENSE", b"Redistribution and use in source and binary forms", b"Google"), } @@ -41,6 +44,7 @@ def main(): parser.add_argument("--published-index", action="store_true", help="refresh the real public index instead of staging this checkout") parser.add_argument("--home", type=Path, help="reuse a dedicated test home across independent installed consumers") parser.add_argument("--run-arg", action="append", default=[]) + parser.add_argument("--template", help="a named template of the package; omit for its default") parser.add_argument("--legacy-exact-key", action="store_true", help="allow only mcpp's nonnumeric exact-version diagnostic; fail other build warnings") args = parser.parse_args() @@ -64,7 +68,8 @@ def main(): if args.published_index: commands.append([args.mcpp, "index", "update"]) commands += [[args.mcpp, "new", "--list-templates", args.package], - [args.mcpp, "new", "installed-quickstart", "--template", args.package]] + [args.mcpp, "new", "installed-quickstart", "--template", + args.package + (":" + args.template if args.template else "")]] for command in commands: print("+", " ".join(command), flush=True) subprocess.run(command, cwd=temporary, env=env, check=True) diff --git a/tests/mwrap/grpc-cpu/mcpp.toml b/tests/mwrap/grpc-cpu/mcpp.toml index f2374c0f..053677c8 100644 --- a/tests/mwrap/grpc-cpu/mcpp.toml +++ b/tests/mwrap/grpc-cpu/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "grpc-cpu" -version = "1.83.0" +version = "1.83.0.1" description = "Consume the immutable gRPC wrapper revision and check module status semantics" [indices] @@ -8,7 +8,7 @@ description = "Consume the immutable gRPC wrapper revision and check module stat # An explicit empty table overrides the legacy workspace redirect. [target.'cfg(unix)'.dependencies] -"mcpplibs.mwrap.grpc" = "1.83.0" +mcpplibs.mwrap.grpc = "1.83.0.1" [target.'cfg(unix)'.build] defines = ["MCPP_TEST_GRPC"] diff --git a/tests/mwrap/grpc-protobuf-stack/mcpp.toml b/tests/mwrap/grpc-protobuf-stack/mcpp.toml new file mode 100644 index 00000000..339a5484 --- /dev/null +++ b/tests/mwrap/grpc-protobuf-stack/mcpp.toml @@ -0,0 +1,18 @@ +[package] +name = "grpc-protobuf-stack" +version = "1.83.0.1" +description = "Consume the gRPC, Protocol Buffers, Abseil and OpenSSL wrappers in one program: one Abseil, one protobuf, one OpenSSL" + +[indices] +# Use the complete candidate built-in index prepared by tests/run_members.sh. +# An explicit empty table overrides the legacy workspace redirect. + +# gRPC is linux/macOS (its wrapper does not claim Windows), so the stack is too. +[target.'cfg(unix)'.dependencies] +mcpplibs.mwrap.grpc = "1.83.0.1" +mcpplibs.mwrap.protobuf = "35.1" +mcpplibs.mwrap.abseil = "20250512.1" +mcpplibs.mwrap.openssl = "3.5.9" + +[target.'cfg(unix)'.build] +defines = ["MCPP_TEST_STACK"] diff --git a/tests/mwrap/grpc-protobuf-stack/tests/behavior.cpp b/tests/mwrap/grpc-protobuf-stack/tests/behavior.cpp new file mode 100644 index 00000000..5b14bdd6 --- /dev/null +++ b/tests/mwrap/grpc-protobuf-stack/tests/behavior.cpp @@ -0,0 +1,58 @@ +// The four wrappers gRPC is built from, in one program and one translation +// unit. They share their libraries: protobuf speaks mcpplibs.mwrap.abseil's +// Abseil (one absl::Status type), and gRPC's TLS and mcpplibs.mwrap.openssl +// link one OpenSSL, 3.5.9. A second copy of either would fail to link, or show +// here as a different type or version. +// +// For GCC 16, as protobuf-m's tests/abseil_identity_test.cpp: the Abseil groups +// used are imported rather than the root (the root brings +// mcpplibs.mwrap.abseil.log, after which GCC fails reading the modules), and +// the status type is checked on a local variable (a decltype of the call at +// namespace scope makes GCC report "conflicting type for imported declaration" +// for descriptor.h's members). Clang takes either form. +#if defined(MCPP_TEST_STACK) +import std; +import mcpplibs.mwrap.protobuf; +import mcpplibs.mwrap.abseil.status; +import mcpplibs.mwrap.abseil.strings; +import mcpplibs.mwrap.openssl; +import mcpplibs.mwrap.grpc; + +namespace gpb = google::protobuf; + +namespace { +int failures = 0; +void check(bool ok, const char* what) { + if (!ok) { + ++failures; + std::printf("FAIL: %s\n", what); + } +} +} // namespace + +int main() { + // protobuf, through Abseil. + gpb::Timestamp stamp; + check(gpb::util::TimeUtil::FromString("2026-11-19T12:30:00Z", &stamp) && stamp.seconds() == 1795091400, "Timestamp"); + std::string json; + auto status = gpb::json::MessageToJsonString(stamp, &json); + static_assert(std::is_same_v, "protobuf reports through mcpplibs.mwrap.abseil's absl::Status"); + check(status.ok() && json == "\"2026-11-19T12:30:00Z\"", "Timestamp as JSON"); + check(absl::StrCat("seconds=", stamp.seconds()) == "seconds=1795091400", "absl::StrCat"); + + // One OpenSSL: the wrapper's, which gRPC's TLS credentials use. + namespace m = mcpplibs::mwrap::openssl::macros; + check(OpenSSL_version_num() == m::OPENSSL_VERSION_NUMBER && m::OPENSSL_VERSION_PATCH == 9, "OpenSSL 3.5.9"); + check(static_cast(grpc::SslCredentials(grpc::SslCredentialsOptions{})), "gRPC TLS credentials"); + + // gRPC's own status and credentials. + const grpc::Status error(grpc::StatusCode::INVALID_ARGUMENT, "stack"); + check(!error.ok() && error.error_code() == grpc::StatusCode::INVALID_ARGUMENT, "grpc::Status"); + check(static_cast(grpc::InsecureChannelCredentials()), "insecure credentials"); + + if (failures == 0) std::printf("grpc-protobuf-stack: all checks passed\n"); + return failures == 0 ? 0 : 1; +} +#else +int main() { return 0; } // Windows: the gRPC wrapper does not claim it. +#endif diff --git a/tests/mwrap/openssl-tls/mcpp.toml b/tests/mwrap/openssl-tls/mcpp.toml new file mode 100644 index 00000000..d1fdd0f9 --- /dev/null +++ b/tests/mwrap/openssl-tls/mcpp.toml @@ -0,0 +1,11 @@ +[package] +name = "openssl-tls" +version = "3.5.9" +description = "Consume the immutable OpenSSL wrapper: SHA-256 and AES-256-GCM vectors, the error queue, a self-signed certificate and a TLS 1.3 handshake over memory BIOs" + +[indices] +# Use the complete candidate built-in index prepared by tests/run_members.sh. +# An explicit empty table overrides the legacy workspace redirect. + +[dependencies] +mcpplibs.mwrap.openssl = "3.5.9" diff --git a/tests/mwrap/openssl-tls/tests/behavior.cpp b/tests/mwrap/openssl-tls/tests/behavior.cpp new file mode 100644 index 00000000..4fb7e55d --- /dev/null +++ b/tests/mwrap/openssl-tls/tests/behavior.cpp @@ -0,0 +1,107 @@ +// The installed OpenSSL wrapper through import alone: SHA-256 and AES-256-GCM against published vectors, the +// error queue read with the re-declared ERR_GET_LIB, a self-signed certificate verified, and a TLS 1.3 client +// and server over a pair of memory BIOs (no network, no files), with the mapped call macros. +import std; +import mcpplibs.mwrap.openssl; + +namespace m = mcpplibs::mwrap::openssl::macros; +namespace o = mcpplibs::mwrap::openssl; + +namespace { +int failures = 0; +void check(bool ok, std::string_view what) { + if (!ok) { + ++failures; + std::println("FAIL: {}", what); + } +} +std::string hex(const unsigned char* data, std::size_t size) { + std::string out; + for (std::size_t i = 0; i < size; ++i) out += std::format("{:02x}", data[i]); + return out; +} + +bool handshake(SSL* client, SSL* server) { + bool client_done = false, server_done = false; + for (int round = 0; round < 100 && !(client_done && server_done); ++round) { + for (auto [ssl, done] : {std::pair{client, &client_done}, std::pair{server, &server_done}}) { + if (*done) continue; + const int result = SSL_do_handshake(ssl); + if (result == 1) { *done = true; continue; } + const int error = SSL_get_error(ssl, result); + if (error != m::SSL_ERROR_WANT_READ && error != m::SSL_ERROR_WANT_WRITE) return false; + } + } + return client_done && server_done; +} +} // namespace + +int main() { + check(OpenSSL_version_num() == m::OPENSSL_VERSION_NUMBER && m::OPENSSL_VERSION_PATCH == 9, "OpenSSL 3.5.9"); + + unsigned char digest[m::EVP_MAX_MD_SIZE]; + unsigned int size = 0; + EVP_Digest("abc", 3, digest, &size, EVP_sha256(), nullptr); + check(hex(digest, size) == "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", "SHA-256(abc)"); + + const unsigned char key[32] = {}, iv[12] = {}, plain[16] = {}; + unsigned char sealed[16], tag[16]; + int out = 0, tail = 0; + EVP_CIPHER_CTX* cipher = EVP_CIPHER_CTX_new(); + EVP_EncryptInit_ex(cipher, EVP_aes_256_gcm(), nullptr, key, iv); + EVP_EncryptUpdate(cipher, sealed, &out, plain, sizeof plain); + EVP_EncryptFinal_ex(cipher, sealed + out, &tail); + EVP_CIPHER_CTX_ctrl(cipher, m::EVP_CTRL_GCM_GET_TAG, sizeof tag, tag); + EVP_CIPHER_CTX_free(cipher); + check(hex(sealed, 16) == "cea7403d4d606b6e074ec5d3baf39d18" && hex(tag, 16) == "d0d1c8a799996bf0265b98b5d48ab919", "AES-256-GCM"); + + ERR_clear_error(); + check(BIO_new_file("/no/such/file", "rb") == nullptr, "missing file refused"); + const unsigned long code = ERR_peek_last_error(); + check(o::ERR_GET_LIB(code) == m::ERR_LIB_BIO && o::ERR_GET_REASON(code) == m::BIO_R_NO_SUCH_FILE, "error queue"); + ERR_clear_error(); + + EVP_PKEY* pkey = EVP_PKEY_Q_keygen(nullptr, nullptr, "EC", "P-256"); + X509* certificate = X509_new(); + X509_set_version(certificate, m::X509_VERSION_3); + ASN1_INTEGER_set(X509_get_serialNumber(certificate), 1); + X509_gmtime_adj(X509_getm_notBefore(certificate), 0); + X509_gmtime_adj(X509_getm_notAfter(certificate), 3600); + X509_set_pubkey(certificate, pkey); + X509_NAME* name = X509_get_subject_name(certificate); + X509_NAME_add_entry_by_txt(name, "CN", m::MBSTRING_ASC, reinterpret_cast("localhost"), -1, -1, 0); + X509_set_issuer_name(certificate, name); + check(X509_sign(certificate, pkey, EVP_sha256()) > 0, "certificate signed"); + + SSL_CTX* server_context = SSL_CTX_new(TLS_server_method()); + SSL_CTX* client_context = SSL_CTX_new(TLS_client_method()); + SSL_CTX_use_certificate(server_context, certificate); + SSL_CTX_use_PrivateKey(server_context, pkey); + X509_STORE_add_cert(SSL_CTX_get_cert_store(client_context), certificate); + SSL_CTX_set_verify(client_context, m::SSL_VERIFY_PEER, nullptr); + m::SSL_CTX_set_min_proto_version(client_context, m::TLS1_3_VERSION); + SSL* server = SSL_new(server_context); + SSL* client = SSL_new(client_context); + BIO *server_io = nullptr, *client_io = nullptr; + BIO_new_bio_pair(&server_io, 0, &client_io, 0); + SSL_set_bio(server, server_io, server_io); + SSL_set_bio(client, client_io, client_io); + SSL_set_accept_state(server); + SSL_set_connect_state(client); + m::SSL_set_tlsext_host_name(client, "localhost"); + SSL_set1_host(client, "localhost"); + check(handshake(client, server), "TLS handshake"); + check(SSL_version(client) == m::TLS1_3_VERSION && SSL_get_verify_result(client) == m::X509_V_OK, "TLS 1.3, verified"); + char buffer[16] = {}; + SSL_write(client, "ping", 4); + check(SSL_read(server, buffer, sizeof buffer) == 4 && std::string_view(buffer, 4) == "ping", "data over TLS"); + + SSL_free(client); + SSL_free(server); + SSL_CTX_free(client_context); + SSL_CTX_free(server_context); + X509_free(certificate); + EVP_PKEY_free(pkey); + std::println("{}", failures == 0 ? "openssl-tls: all checks passed" : "openssl-tls: FAILED"); + return failures == 0 ? 0 : 1; +} diff --git a/tests/mwrap/protobuf-codegen/build.mcpp b/tests/mwrap/protobuf-codegen/build.mcpp new file mode 100644 index 00000000..6b0d0771 --- /dev/null +++ b/tests/mwrap/protobuf-codegen/build.mcpp @@ -0,0 +1,12 @@ +// protoc over proto/ through the protogen rule, which the codegen feature +// brings. Not on Windows, where compat.protobuf has no protoc: the import is +// a compile-time dependency and cannot be conditional, so the guard is on the +// call (the shape of tests/examples/grpc-codegen). +import std; +import mcpp; +import protogen; + +int main() { + if (std::string_view(mcpp::target_os()) == "windows") return 0; + return protogen::generate_all() ? 0 : 1; +} diff --git a/tests/mwrap/protobuf-codegen/mcpp.toml b/tests/mwrap/protobuf-codegen/mcpp.toml new file mode 100644 index 00000000..2b94c71b --- /dev/null +++ b/tests/mwrap/protobuf-codegen/mcpp.toml @@ -0,0 +1,21 @@ +[package] +name = "protobuf-codegen" +version = "35.1" +description = "Consume the immutable Protocol Buffers wrapper's codegen: protoc during the build through the protogen rule, generated messages beside the module" + +[indices] +# Use the complete candidate built-in index prepared by tests/run_members.sh. +# An explicit empty table overrides the legacy workspace redirect. + +# compat.protobuf has no protoc on Windows yet (its host-tool build fails +# there), so codegen is linux/macOS. On Windows the member still builds the +# indexed protogen rule on its own (mcpplibs.protogen), and its build program +# skips generation. +[target.'cfg(unix)'.dependencies] +mcpplibs.mwrap.protobuf = { version = "35.1", features = ["codegen"] } + +[target.'cfg(unix)'.build] +cxxflags = ["-DHAVE_PROTOC=1"] + +[target.'cfg(windows)'.build-dependencies] +mcpplibs.protogen = { version = "35.1", host-module = true } diff --git a/tests/mwrap/protobuf-codegen/proto/addressbook.proto b/tests/mwrap/protobuf-codegen/proto/addressbook.proto new file mode 100644 index 00000000..430935ca --- /dev/null +++ b/tests/mwrap/protobuf-codegen/proto/addressbook.proto @@ -0,0 +1,30 @@ +syntax = "proto3"; + +package tutorial; + +import "google/protobuf/timestamp.proto"; + +message Person { + string name = 1; + int32 id = 2; + string email = 3; + + enum PhoneType { + PHONE_TYPE_UNSPECIFIED = 0; + PHONE_TYPE_MOBILE = 1; + PHONE_TYPE_HOME = 2; + PHONE_TYPE_WORK = 3; + } + + message PhoneNumber { + string number = 1; + PhoneType type = 2; + } + + repeated PhoneNumber phones = 4; + google.protobuf.Timestamp last_updated = 5; +} + +message AddressBook { + repeated Person people = 1; +} diff --git a/tests/mwrap/protobuf-codegen/tests/behavior.cpp b/tests/mwrap/protobuf-codegen/tests/behavior.cpp new file mode 100644 index 00000000..44eec970 --- /dev/null +++ b/tests/mwrap/protobuf-codegen/tests/behavior.cpp @@ -0,0 +1,29 @@ +// Generated code beside the module: addressbook.pb.h comes from protoc during +// the build. The prelude (every header the modules include) first, then the +// generated header, then the import — GCC needs every #include before it. +#if defined(HAVE_PROTOC) +#include +#include "addressbook.pb.h" + +import std; +import mcpplibs.mwrap.protobuf; + +namespace gpb = google::protobuf; + +int main() { + tutorial::AddressBook book; + tutorial::Person* ada = book.add_people(); + ada->set_name("Ada Lovelace"); + ada->set_id(1815); + ada->add_phones()->set_type(tutorial::Person::PHONE_TYPE_HOME); + const bool stamped = gpb::util::TimeUtil::FromString("1843-07-01T00:00:00Z", ada->mutable_last_updated()); + tutorial::AddressBook back; + const bool ok = stamped && back.ParseFromString(book.SerializeAsString()) && + gpb::util::MessageDifferencer::Equals(book, back) && back.people(0).id() == 1815 && + tutorial::Person::descriptor()->full_name() == "tutorial.Person"; + std::println("{}", ok ? "protobuf-codegen: generated messages round-trip" : "protobuf-codegen: FAILED"); + return ok ? 0 : 1; +} +#else +int main() { return 0; } // Windows: no protoc yet; the rule package was built above +#endif diff --git a/tests/mwrap/protobuf-message/mcpp.toml b/tests/mwrap/protobuf-message/mcpp.toml new file mode 100644 index 00000000..309b3a90 --- /dev/null +++ b/tests/mwrap/protobuf-message/mcpp.toml @@ -0,0 +1,11 @@ +[package] +name = "protobuf-message" +version = "35.1" +description = "Consume the immutable Protocol Buffers wrapper: a message type built at run time, its wire bytes, JSON and text formats, the well-known types" + +[indices] +# Use the complete candidate built-in index prepared by tests/run_members.sh. +# An explicit empty table overrides the legacy workspace redirect. + +[dependencies] +mcpplibs.mwrap.protobuf = "35.1" diff --git a/tests/mwrap/protobuf-message/tests/behavior.cpp b/tests/mwrap/protobuf-message/tests/behavior.cpp new file mode 100644 index 00000000..3390226a --- /dev/null +++ b/tests/mwrap/protobuf-message/tests/behavior.cpp @@ -0,0 +1,70 @@ +// The installed Protocol Buffers wrapper through import alone (no protoc): a +// message type described at run time and instantiated through DynamicMessage, +// its wire bytes pinned, JSON and text formats round-tripped, a Timestamp and +// a Struct, and a .proto text parsed by the compiler's Parser. +import std; +import mcpplibs.mwrap.protobuf; + +namespace gpb = google::protobuf; + +namespace { +int failures = 0; +void check(bool ok, std::string_view what) { + if (!ok) { + ++failures; + std::println("FAIL: {}", what); + } +} +std::string hex(std::string_view bytes) { + std::string out; + for (unsigned char c : bytes) out += std::format("{:02x}", c); + return out; +} +} // namespace + +int main() { + gpb::FileDescriptorProto file; + file.set_name("point.proto"); + file.set_package("demo"); + file.set_syntax("proto3"); + gpb::DescriptorProto* point = file.add_message_type(); + point->set_name("Point"); + for (auto [name, number, type] : {std::tuple{"x", 1, gpb::FieldDescriptorProto::TYPE_INT32}, + std::tuple{"label", 2, gpb::FieldDescriptorProto::TYPE_STRING}}) { + gpb::FieldDescriptorProto* field = point->add_field(); + field->set_name(name); + field->set_number(number); + field->set_type(type); + field->set_label(gpb::FieldDescriptorProto::LABEL_OPTIONAL); + } + gpb::DescriptorPool pool; + const gpb::FileDescriptor* built = pool.BuildFile(file); + check(built != nullptr, "descriptor built"); + const gpb::Descriptor* type = pool.FindMessageTypeByName("demo.Point"); + gpb::DynamicMessageFactory factory(&pool); + std::unique_ptr message(factory.GetPrototype(type)->New()); + const gpb::Reflection* reflection = message->GetReflection(); + reflection->SetInt32(message.get(), type->FindFieldByName("x"), 150); + reflection->SetString(message.get(), type->FindFieldByName("label"), "hi"); + const std::string wire = message->SerializeAsString(); + check(hex(wire) == "08960112026869", "wire bytes (x = 150, label = \"hi\")"); + + std::string json; + check(gpb::json::MessageToJsonString(*message, &json).ok() && json == R"({"x":150,"label":"hi"})", "to JSON"); + std::unique_ptr back(message->New()); + check(gpb::json::JsonStringToMessage(R"({"x": 7, "label": "yo"})", back.get()).ok() && + reflection->GetInt32(*back, type->FindFieldByName("x")) == 7, + "from JSON"); + std::string text; + check(gpb::TextFormat::PrintToString(*message, &text) && text.find("x: 150") != std::string::npos, "text format"); + + gpb::Timestamp stamp; + check(gpb::util::TimeUtil::FromString("2026-11-19T12:30:00Z", &stamp) && stamp.seconds() == 1795091400, "Timestamp"); + gpb::Struct object; + (*object.mutable_fields())["n"].set_number_value(2.5); + std::string struct_json; + check(gpb::json::MessageToJsonString(object, &struct_json).ok() && struct_json == R"({"n":2.5})", "Struct"); + + std::println("{}", failures == 0 ? "protobuf-message: all checks passed" : "protobuf-message: FAILED"); + return failures == 0 ? 0 : 1; +} diff --git a/tests/test_select_members.py b/tests/test_select_members.py index 29a7027e..a65c14f7 100644 --- a/tests/test_select_members.py +++ b/tests/test_select_members.py @@ -88,7 +88,7 @@ def test_a_spelling_names_identities_of_this_index_only(self): ids = {("mcpplibs", "os"), ("compat", "openssl"), ("mcpplibs.mwrap", "ffmpeg"), ("compat", "os")} self.assertEqual(s.resolve("os", ids), {("mcpplibs", "os")}) self.assertEqual(s.resolve("compat.openssl", ids), {("compat", "openssl")}) - self.assertEqual(s.resolve("compat:openssl@3.5.1", ids), {("compat", "openssl")}) + self.assertEqual(s.resolve("compat:openssl@3.5.9", ids), {("compat", "openssl")}) self.assertEqual(s.resolve("mcpplibs.mwrap.ffmpeg", ids), {("mcpplibs.mwrap", "ffmpeg")}) self.assertEqual(s.resolve("xim:gcc@16.1.0", ids), set()) self.assertEqual(s.resolve("macos", ids), set())