Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 174 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
name: CI

on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true

env:
# The tests' dev-dependency mcpplibs.mwrap.abseil 20250512.1 is in mcpp-index#845 and not yet in
# the published index: until it is, the run resolves against that PR's candidate commit, the
# complete unchanged index layout installed as the built-in index of the run's home. Drop this
# pin and the steps that read it once the index publishes it.
MCPP_INDEX_CANDIDATE: "269cd2dd952b729ce1732bc82e6caa0fa15bea53"

jobs:
mcpp:
name: mcpp (${{ matrix.os }}, ${{ matrix.toolchain }})
strategy:
fail-fast: false
matrix:
include:
- { os: ubuntu-24.04, toolchain: 'gcc@16.1.0' }
- { os: ubuntu-24.04, toolchain: 'llvm@23.1.3' }
- { os: ubuntu-24.04-arm, toolchain: 'gcc@16.1.0' }
- { os: ubuntu-24.04-arm, toolchain: 'llvm@23.1.3' }
- { os: macos-15, toolchain: 'llvm@23.1.3' }
- { os: windows-2025, toolchain: 'llvm@23.1.3' }
runs-on: ${{ matrix.os }}
timeout-minutes: 120
defaults:
run:
shell: bash
env:
XLINGS_NON_INTERACTIVE: 1
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Mwrap identity, bilingual README contract, the codegen example is the template
run: |
python3 tools/check_mwrap_release.py
python3 tools/check_examples.py

- name: Install Xlings (Linux/macOS)
if: runner.os != 'Windows'
run: |
curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/refs/heads/main/tools/other/quick_install.sh | bash
echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH"

- name: Install Xlings (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
irm https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.ps1 | iex
"$env:USERPROFILE\.xlings\subos\current\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append

- name: Install project tools (.xlings.json)
run: |
for attempt in 1 2 3; do
xlings update > /dev/null 2>&1 || true
if xlings install; then break; fi
[ "$attempt" = 3 ] && exit 1
sleep 30
done
mcpp --version

- name: Candidate index (mcpp-index#845)
run: |
mcpp index update
# No early exit in awk: mcpp would take SIGPIPE, and pipefail report it.
home=$(mcpp self env | awk -F'= *' '/^MCPP_HOME/ && !h {h = $2} END {print h}')
test -n "$home"
git init -q "$RUNNER_TEMP/index"
git -C "$RUNNER_TEMP/index" fetch -q --depth 1 https://github.com/mcpplibs/mcpp-index "$MCPP_INDEX_CANDIDATE"
git -C "$RUNNER_TEMP/index" checkout -q FETCH_HEAD
python3 "$RUNNER_TEMP/index/tests/prepare_builtin_index.py" --home "$home"

- name: Select the toolchain
run: |
name="${{ matrix.toolchain }}"
mcpp toolchain install "${name%@*}" "${name#*@}"
mcpp toolchain default "$name"

- name: Build the modules
run: mcpp build

- name: Tests (wire bytes, text and JSON, the well-known types, .proto parsing, streams, every macro, header mixing)
run: mcpp test

- name: Tests, release profile
run: mcpp test --profile release

- name: The gzip feature (GzipOutputStream / GzipInputStream over compat.zlib)
working-directory: examples/gzip
run: mcpp run

# protoc is built for this machine from the same release, protogen generates the code during the build.
# Not on Windows: compat.protobuf 35.1 does not build protoc there (its host-tool sub-build fails).
- name: Code generation (templates/codegen instantiated)
if: runner.os != 'Windows'
working-directory: examples/codegen
run: mcpp run

# Evidence, not a gate: whether compat.protobuf's protoc host-tool sub-build still fails on Windows.
- name: Probe protoc on Windows (known to fail with compat.protobuf 35.1)
if: runner.os == 'Windows'
continue-on-error: true
working-directory: examples/codegen
run: mcpp run

- name: Export audit (the generated sources are fresh; on the LLVM row the record is derived again by clang)
run: |
if [ "${{ matrix.os }}" = "ubuntu-24.04" ] && [ "${{ matrix.toolchain }}" = "llvm@23.1.3" ]; then
export MWRAP_REQUIRE_CLANG=1
export MWRAP_CLANG="$(ls -d "$HOME"/.mcpp/registry/data/xpkgs/xim-x-llvm/23.1.3/bin/clang++ "$HOME"/.xlings/data/xpkgs/xim-x-llvm/23.1.3/bin/clang++ 2>/dev/null | head -1)"
fi
python3 tools/check_exports.py

# The Web: wasm32-emscripten (emsdk from xlings, through mcpp), the tests under node.
web:
name: mcpp (web, wasm32-emscripten)
runs-on: ubuntu-24.04
timeout-minutes: 90
defaults:
run:
shell: bash
env:
XLINGS_NON_INTERACTIVE: 1
steps:
- uses: actions/checkout@v4

- name: Install Xlings
run: |
curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/refs/heads/main/tools/other/quick_install.sh | bash
echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH"

- name: Install project tools (.xlings.json)
run: |
for attempt in 1 2 3; do
xlings update > /dev/null 2>&1 || true
if xlings install; then break; fi
[ "$attempt" = 3 ] && exit 1
sleep 30
done
mcpp --version

- name: Candidate index (mcpp-index#845)
run: |
mcpp index update
# No early exit in awk: mcpp would take SIGPIPE, and pipefail report it.
home=$(mcpp self env | awk -F'= *' '/^MCPP_HOME/ && !h {h = $2} END {print h}')
test -n "$home"
git init -q "$RUNNER_TEMP/index"
git -C "$RUNNER_TEMP/index" fetch -q --depth 1 https://github.com/mcpplibs/mcpp-index "$MCPP_INDEX_CANDIDATE"
git -C "$RUNNER_TEMP/index" checkout -q FETCH_HEAD
python3 "$RUNNER_TEMP/index/tests/prepare_builtin_index.py" --home "$home"

- name: Tests under node
run: mcpp test --target wasm32-emscripten

- name: Tests under node, release profile
run: mcpp test --target wasm32-emscripten --profile release
49 changes: 49 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: Release
on:
push:
tags: ["mwrap-*"]
permissions:
contents: read
jobs:
publish:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
actions: read
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Check tag, upstream version and bilingual entry pages
run: python3 tools/check_mwrap_release.py --tag "$GITHUB_REF_NAME"
- name: Require successful source CI at this exact commit
env:
GH_TOKEN: ${{ github.token }}
shell: bash
run: |
set -euo pipefail
result=$(gh run list --repo "$GITHUB_REPOSITORY" --workflow ci.yml --commit "$GITHUB_SHA" --limit 1 --json status,conclusion --jq '.[0] | .status == "completed" and .conclusion == "success"')
test "$result" = true || { echo 'No successful source CI at the tag commit'; exit 1; }
- name: Build immutable source asset and publish
env:
GH_TOKEN: ${{ github.token }}
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#mwrap-}"
repo="${GITHUB_REPOSITORY#*/}"
asset="${repo}-${version}.tar.gz"
git archive --format=tar.gz --prefix="${repo}-${version}/" "$GITHUB_SHA" > "$RUNNER_TEMP/$asset"
(cd "$RUNNER_TEMP" && sha256sum "$asset" > "$asset.sha256")
if gh release view "$GITHUB_REF_NAME" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
echo 'Release already exists; refusing to change immutable assets'; exit 1
fi
extra=()
if [[ "$version" == *-rc* ]]; then extra+=(--prerelease); fi
gh release create "$GITHUB_REF_NAME" "$RUNNER_TEMP/$asset" "$RUNNER_TEMP/$asset.sha256" \
--repo "$GITHUB_REPOSITORY" --verify-tag --title "$repo $version" \
--notes "Module wrapper for upstream $version. New mwrap package identities become installable after mcpp-index publication; see the bilingual README for the tested support scope." "${extra[@]}"
30 changes: 30 additions & 0 deletions .github/workflows/templates.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Actual project templates
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
permissions:
contents: read
concurrency:
group: templates-${{ github.ref }}
cancel-in-progress: true
env:
MCPP_VERSION: "2026.11.18.1"
jobs:
templates:
runs-on: ubuntu-24.04
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- name: Install the pinned mcpp
shell: bash
run: |
archive="mcpp-${MCPP_VERSION}-linux-x86_64.tar.gz"
curl -L -fsS -o "$archive" "https://github.com/mcpp-community/mcpp/releases/download/v${MCPP_VERSION}/$archive"
tar -xzf "$archive"
echo "$PWD/mcpp-${MCPP_VERSION}-linux-x86_64/bin" >> "$GITHUB_PATH"
- name: Actual scaffold, strict build and run of the default template (message)
run: python3 tools/test_template.py --toolchain gcc@16.1.0
- name: Actual scaffold, strict build and run of the codegen template (protoc during the build)
run: python3 tools/test_template.py --toolchain gcc@16.1.0 --template codegen
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
target/
compile_commands.json
mcpp.lock
.mcpp/
.xlings/
doctor.log
__pycache__/
5 changes: 5 additions & 0 deletions .xlings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"workspace": {
"xim:mcpp": "2026.11.18.1"
}
}
20 changes: 20 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
protobuf-m: C++ module wrapper
Copyright (c) 2026 mcpp-community contributors

The wrapper contributions are licensed under the Apache License, Version 2.0.
Upstream code, copied interfaces and dependencies retain their original
licenses and copyright notices:

Protocol Buffers, under the BSD 3-Clause License:
Copyright 2008 Google Inc. (Google LLC). All rights reserved.
utf8_range (third_party/utf8_range of the protobuf release), under the MIT
License:
Copyright (c) 2019 Yibo Cai. Copyright 2022 Google LLC.
Abseil (compat.abseil, which protobuf links), under the Apache License 2.0:
Copyright The Abseil Authors.

Their original README and licenses are retained in upstream/protobuf/,
upstream/utf8_range/ and upstream/abseil-cpp/. The library itself is built by
compat.protobuf 35.1 over compat.abseil 20250512.1 from the same releases.
rules/ (protogen) is derived from grpc-m's grpcgen (Apache-2.0, mcpp-community).
Provenance and checksums are recorded in upstream/documents.json.
96 changes: 96 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# protobuf-m

> Protocol Buffers 35.1 (the C++ runtime), through C++23 modules.

**English** · [简体中文](README.zh.md) · [Upstream](https://github.com/protocolbuffers/protobuf) · [Releases](https://github.com/mcpplibs/protobuf-m/releases) · [mcpp](https://github.com/mcpp-community/mcpp) · [Index](https://github.com/mcpplibs/mcpp-index)

[![C++23](https://img.shields.io/badge/C%2B%2B-23-blue.svg)](https://en.cppreference.com/w/cpp/23)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
[![mcpp-community](https://img.shields.io/badge/mcpp--community-mwrap-blue)](https://github.com/mcpp-community)
[![CI](https://github.com/mcpplibs/protobuf-m/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mcpplibs/protobuf-m/actions/workflows/ci.yml)

The upstream C++ API is unchanged: `google::protobuf::Message`, `DescriptorPool`,
`util::MessageDifferencer`, `Timestamp` are the upstream entities. The library is
`compat.protobuf` 35.1 over `compat.abseil` 20250512.1, built from the releases, and the
`codegen` feature adds protoc, built for the build machine from the same release, with
`protogen`, a build rule that generates the code of every `.proto` during the build.

## Modules

- `import mcpplibs.mwrap.protobuf;`: messages, descriptors, reflection, arenas, repeated fields, maps, text format, the well-known types; re-exports the modules below
- `import mcpplibs.mwrap.protobuf.io;`: coded and zero-copy streams, the tokenizer
- `import mcpplibs.mwrap.protobuf.util;`: JSON (`json::`), `MessageDifferencer`, `TimeUtil`, `FieldMaskUtil`, delimited messages
- `import mcpplibs.mwrap.protobuf.compiler;`: parsing `.proto` files at run time (`Importer`, `Parser`)
- `import mcpplibs.mwrap.protobuf.io.gzip;`: `GzipInputStream`, `GzipOutputStream` (feature `gzip`)

> More Modules API -> [Usage](docs/usage.md) · [every exported name](docs/api.md)

## Getting started

```sh
mcpp new people --template mcpplibs.mwrap.protobuf@35.1
cd people
mcpp run
```

The default template describes a message type at run time and uses it through reflection:
wire bytes, text and JSON, then Timestamp arithmetic, Struct and Any — no protoc. With
`--template mcpplibs.mwrap.protobuf@35.1:codegen`, `proto/addressbook.proto` is compiled by
protoc during the build and the generated classes are used with the module.

```toml
[dependencies]
mcpplibs.mwrap.protobuf = { version = "35.1", features = ["codegen"] }
```

```cpp
// build.mcpp
import mcpp;
import protogen;
int main() { return protogen::generate_all() ? 0 : 1; }
```

```cpp
// src/main.cpp: includes first (the prelude, then generated headers), imports after
#include <mwrap/protobuf/prelude.hpp>
#include "addressbook.pb.h"
import std;
import mcpplibs.mwrap.protobuf;
```

## Features

| Feature | Purpose |
|---|---|
| `codegen` | protoc for the build machine, from the same release, and the `protogen` rule (not on Windows: protoc is not built there) |
| `gzip` | `GzipInputStream` / `GzipOutputStream` over compat.zlib, `import mcpplibs.mwrap.protobuf.io.gzip;` |
| `upb` | protobuf's C runtime from the same release, as gRPC links it (one protobuf in a program with both) |

## Abseil and headers

protobuf's API takes and returns Abseil types (`absl::Status`, `absl::string_view`, which
is `std::string_view`); this package exports no `absl` name. To name them, import
`mcpplibs.mwrap.abseil` or include the Abseil header. With GCC every `#include` of a
translation unit comes before its first `import`; one that includes generated headers
includes `<mwrap/protobuf/prelude.hpp>` first. The version macros are constants in
`mcpplibs::mwrap::protobuf::macros`; `GOOGLE_PROTOBUF_VERIFY_VERSION` and the other
macros come from `<mwrap/protobuf/macros.hpp>`.

## Build and testing

CI on Linux (x86_64, aarch64; GCC 16 and LLVM 23), macOS and Windows builds the modules
and runs the tests in the dev and release profiles: wire bytes, text and JSON round trips,
the well-known types, MessageDifferencer and FieldMaskUtil, coded and delimited streams,
a `.proto` parsed at run time, every macro mapping, header mixing; the gzip feature; and
code generation through protogen (not on Windows). The Web row runs the tests as
wasm32 under node. The export audit derives the record again from the headers on the
Linux LLVM row. A separate job scaffolds both templates as an installed package and runs
them.

## License

[Apache-2.0](LICENSE) for the wrapper, by mcpp-community ([NOTICE](NOTICE)). Protocol
Buffers remains under the BSD 3-Clause License ([upstream/protobuf/LICENSE](upstream/protobuf/LICENSE),
[README](upstream/protobuf/README.md)), utf8_range under MIT
([upstream/utf8_range/LICENSE](upstream/utf8_range/LICENSE)), and Abseil, which it links,
under Apache-2.0 ([upstream/abseil-cpp/LICENSE](upstream/abseil-cpp/LICENSE)).
Loading
Loading