Fix client-v2: do not declare a content encoding on a multipart request - #3116
Open
polyglotAI-bot wants to merge 2 commits into
Open
Fix client-v2: do not declare a content encoding on a multipart request#3116polyglotAI-bot wants to merge 2 commits into
polyglotAI-bot wants to merge 2 commits into
Conversation
The multipart body used to send statement parameters is always attached uncompressed, but the request still carried Content-Encoding: lz4 when both client request compression and HTTP compression were enabled. ClickHouse 26.8+ honours that header for multipart requests and fails the query with LZ4_DECODER_FAILED. The header is now removed for multipart requests, next to the decompress query parameter that was already dropped for them. Fixes: #3075
Client V2 CoverageCoverage Report
Class Coverage
|
JDBC V2 CoverageCoverage Report
Class Coverage
|
JDBC V1 CoverageCoverage Report
Class Coverage
|
Client V1 CoverageCoverage Report
Class Coverage
|
Keeps addHeaders() untouched: threading a multipart flag through it changed its declaration line, which reported its pre-existing cognitive complexity (sonar java:S3776) as a new-code issue and failed the quality gate. The header is now removed in createRequest(), still after addHeaders() ran, so an encoding set by the application with http_header_* is dropped too. Also pins the header spelling and the non-multipart contrast case.
TriageCategory: Summary What this impacts
Concerns
Required reviewer action
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #3075.
When statement parameters are sent in the request body (
client.http.use_form_request_for_query=true),HttpAPIClientHelper.createRequestattaches the multipart entity uncompressed andcreateRequestURIalready drops thedecompressquery parameter for it.addHeadershowever still setContent-Encoding: lz4whenever client request compression and HTTP compression were both enabled, so the request declared an encoding its body did not have. Servers before 26.8 ignored the header for multipart requests; ClickHouse 26.8+ honours it and fails the query withCode: 618 ... LZ4 decompression failed ... (LZ4_DECODER_FAILED).The request now carries no content encoding when it is a multipart request. The header is removed after the custom
http_header_*headers are applied, so an application-suppliedContent-Encodingcannot re-introduce the same broken request either — a multipart body cannot be compressed by the client, whatever the configuration says. This mirrors the client-v1 fix in #3095.Response compression is a separate concern and is unchanged:
Accept-Encodingandenable_http_compression=1are still sent, so a multipart query keeps a compressed response.Changes
client-v2 .../internal/HttpAPIClientHelper.java: thread the existing multipart flag intocreatePostRequest/addHeaders(the insert/data request passesfalse, as it streams and compresses its own body), and removeContent-Encodingfor a multipart request.CHANGELOG.md: bug-fix entry.Test
HttpAPIClientHelperTest:testRequestCompressionSignalling(@DataProvider) asserts, per configuration, theContent-Encodingof the request and of the body entity, theAccept-Encoding, and thedecompress/enable_http_compressionquery parameters. It pins the fix (multipart + both compression options -> no content encoding) and, as contrast, that every non-multipart combination keeps its previous signalling — including a query without parameters whileuse_form_request_for_queryis enabled, which is not a multipart request.testCustomContentEncodingHeaderRemovedForMultipartRequest: an application-sethttp_header_Content-Encodingis removed for a multipart request.testDataRequestKeepsContentEncodingWhenParamsInBodyEnabled: the data/insert request keeps compressing its body whenuse_form_request_for_queryis enabled.Both new failing assertions fail on
mainwithexpected [null] but found [lz4]and pass with the fix. The wholeclient-v2unit suite is green (676 tests).Verified on the live path against ClickHouse
26.9.1.954: aqueryAllwith parameters in the body,compressClientRequest(true)anduseHttpCompression(true)fails withLZ4_DECODER_FAILEDbefore the change and succeeds after it. The same shape reproduces with plaincurl(-H "Content-Encoding: lz4" -F "query=SELECT 1" "?enable_http_compression=1").Note:
HttpTransportTests#testMultiPartRequeststays red on 26.8+ for an unrelated reason — its first case fails while reading a compressed response (Invalid LZ4 magic byte), which is #3105 / PR #3106. That failure is identical onmainwith and without this change.Pre-PR validation gate
curl)mvn -pl client-v2 test: 676 passed)AGENTS.md(targeted Maven runs, TestNG@DataProviderfor the parametrized cases, no issue numbers in test code,CHANGELOG.mdupdated)docs/features.mdchange (behavior of a documented feature is restored, not changed)