branch-4.1: [fix](jdbc) Harden JDBC driver URL validation and remove file-upload HTTP API - #67149
Open
CalvinKirs wants to merge 1 commit into
Open
branch-4.1: [fix](jdbc) Harden JDBC driver URL validation and remove file-upload HTTP API#67149CalvinKirs wants to merge 1 commit into
CalvinKirs wants to merge 1 commit into
Conversation
…HTTP API (apache#65987) Hardens JDBC driver loading, locks down security-sensitive configs, and removes an unnecessary file-upload attack surface on the FE. | Area | Change | |---|---| | `jdbc_driver_secure_path` matching | Component-based (structural) instead of a raw string prefix. `file://` URLs are decoded once via `URI.getPath()` and compared with `Path.startsWith`; `http(s)` URLs compare scheme/host/port, **user-info and query**, plus a component-based path prefix. Closes prefix-confusion, path-traversal (literal **and** percent-encoded `%2e%2e`), and query/user-info bypasses. `*` and empty still mean allow-all (unchanged). | | Fail-closed parsing | A `driver_url` that cannot be parsed as a URI is now rejected instead of being accepted. | | Mandatory CREATE/ALTER rule | New `JdbcDorisConnector.checkDriverUrlSecurityRule()`, invoked from the connector's `validateProperties()` (run by `checkProperties()` on both CREATE and ALTER, both `!isReplay`). It **cannot be turned off**: rejects any `..` segment (decoded), and requires a scheme-less `driver_url` to be a bare `[A-Za-z0-9._-]+.jar` name resolved under `jdbc_drivers_dir`. Never runs during replay or at query time. | | Config immutability | `jdbc_driver_url_white_list`, `s3_load_endpoint_white_list`, `force_sqlserver_jdbc_encrypt_false` are now settable only in `fe.conf` (non-mutable). | | Removed endpoint | `UploadAction` (`/api/{ns}/{db}/{tbl}/upload`) and its `LoadSubmitter` / `TmpFileMgr` helpers, plus the now-orphaned `http_load_submitter_max_worker_threads` config. | | # | Before | After | Impact / who is affected | |---|---|---|---| | 1 | `POST/PUT/GET/DELETE /api/{ns}/{db}/{tbl}/upload` uploaded a file to FE and stream-loaded it | Endpoint **removed** (404) | Web-UI "Data Import" page / scripts hitting this route. Stream Load, Broker Load, Routine Load, INSERT, S3 Load, CopyInto are **unaffected**. | | 2 | `driver_url` with `..`, encoded `%2e%2e`, or a scheme-less path with directories was accepted at CREATE (and never re-checked on ALTER) | Rejected at **CREATE and ALTER CATALOG** | Only new/altered catalogs. Existing catalogs are **not** re-validated (replay and queries unaffected). | | 3 | `jdbc_driver_secure_path` matched by raw string prefix; parse failures were let through | Structural matching; parse failures rejected (fail-closed) | Driver URLs that only passed by a coincidental prefix, via `..`, `%2e%2e`, or a differing query/user-info are now rejected. Legitimate URLs are unaffected. `*` and empty both still mean allow-all, so default deployments see no change. | | 4 | `jdbc_driver_url_white_list`, `s3_load_endpoint_white_list`, `force_sqlserver_jdbc_encrypt_false` were runtime-mutable | Non-mutable (fe.conf only, restart to take effect) | Scripts using `ADMIN SET FRONTEND CONFIG` for these now get a "not mutable" error. | | 5 | `http_load_submitter_max_worker_threads` config existed | Removed | Old `fe.conf` entries get the existing unknown-key warning. | | Config | Before | After | |---|---|---| | `jdbc_driver_url_white_list` | `mutable = true` | fe.conf only (non-mutable) | | `s3_load_endpoint_white_list` | `mutable = true` | fe.conf only (non-mutable) | | `force_sqlserver_jdbc_encrypt_false` | `mutable = true` | fe.conf only (non-mutable) | | `jdbc_driver_secure_path` | raw string-prefix match | structural (component-based) match; `*`/empty still allow-all | | `http_load_submitter_max_worker_threads` | present | removed | Removed the `/api/{ns}/{db}/{tbl}/upload` file-upload endpoint. Hardened JDBC driver URL validation (structural `jdbc_driver_secure_path` matching incl. encoded-traversal and query/user-info handling, fail-closed parsing, and a mandatory CREATE/ALTER rule that forbids path traversal and requires a plain jar file name); `*` and empty `jdbc_driver_secure_path` still mean allow-all. `jdbc_driver_url_white_list`, `s3_load_endpoint_white_list` and `force_sqlserver_jdbc_encrypt_false` are now settable only in `fe.conf`. - Test <!-- At least one of them must be included. --> - [ ] Regression test - [x] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - Behavior changed: - [ ] No. - [x] Yes. <!-- See the "Behavior changes" table above. --> - Does this need documentation? - [ ] No. - [x] Yes. <!-- The removed endpoint and the config/validation changes should be documented. -->
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#65987