Skip to content

[lake/paimon] Bump Paimon version to 2.0.0 - #4093

Open
zuston wants to merge 2 commits into
apache:mainfrom
zuston:paimon2.0
Open

[lake/paimon] Bump Paimon version to 2.0.0#4093
zuston wants to merge 2 commits into
apache:mainfrom
zuston:paimon2.0

Conversation

@zuston

@zuston zuston commented Aug 25, 2026

Copy link
Copy Markdown
Member

Purpose

Linked issue: close #xxx

Brief change log

Tests

API and Format

Documentation

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Fluss’s Paimon lake integration to target Apache Paimon 2.0.0, aligning the build, documentation, Docker quickstart artifacts, and serializer compatibility behavior with the new Paimon version.

Changes:

  • Bump the project-wide Maven property paimon.version to 2.0.0, and update website version metadata/docs accordingly.
  • Update lake-tiering serialization to support versioned compatibility across Paimon upgrades, including explicit mapping between Fluss serializer versions and Paimon CommitMessageSerializer versions.
  • Refresh Docker quickstart Paimon JAR download coordinates and documented jar names to 2.0.0.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
website/fluss-versions.json Updates the “next” website metadata to reflect Paimon 2.0.0.
website/docs/streaming-lakehouse/datalake-formats/paimon.md Updates the documented Paimon compatibility matrix to include/require 2.0.
website/community/how-to-release/creating-a-fluss-release.mdx Updates release-process docs to use Paimon 2.0.0 in example version entries.
pom.xml Bumps the Maven property paimon.version to 2.0.0.
fluss-lake/fluss-lake-paimon/src/test/java/org/apache/fluss/lake/paimon/utils/FlussToPaimonPredicateConverterTest.java Updates expected predicate construction to match Paimon 2.0 predicate behavior.
fluss-lake/fluss-lake-paimon/src/test/java/org/apache/fluss/lake/paimon/tiering/PaimonSerializerCompatibilityTest.java Adds a new compatibility test to validate deserialization of Paimon 1.4-era state and current round-trips.
fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/tiering/PaimonWriteResultSerializer.java Introduces version mapping between Fluss serializer versions and Paimon commit-message serializer versions, with explicit unsupported-version handling.
fluss-flink/fluss-flink-common/src/test/java/org/apache/fluss/flink/tiering/source/TableBucketWriteResultSerializerTest.java Extends tests to validate nested serializer version persistence (v2) and backward behavior (v1).
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/tiering/source/TableBucketWriteResultSerializer.java Adds serializer versioning (v2) and persists nested write-result serializer version for better evolution/compatibility.
docker/quickstart-flink/README.md Updates documented Paimon jar names/versions used by the quickstart image.
docker/quickstart-flink/prepare_build.sh Updates Paimon jar download URLs/filenames to 2.0.0 and adds checksums for the Paimon jars.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@luoyuxia

Copy link
Copy Markdown
Contributor

IIUC, we do not need to bump TableBucketWriteResultSerializer or introduce nested serializer versioning for this upgrade. The Tiering Service is stateless and does not checkpoint TableBucketWriteResult; this serializer is only used for data exchange between the source and committer within the same job, where both sides use the same Paimon runtime.

Also, Paimon 1.4 should still be able to perform tiering. I do not see any incompatible Paimon APIs used by the Fluss tiering path between Paimon 1.4 and 2.0. The new hard-coded CommitMessageSerializer v12 check is what explicitly makes a Paimon 1.4 runtime fail.

Could we keep the original serializer behavior and avoid marking Paimon 2.0 as strictly required? It would be good to verify this with one Tiering Service integration test using the Paimon 1.4 runtime.

@zuston

zuston commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

make sense, thanks for your insightful comments. @luoyuxia

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants