feat(gateways): Iggy Kafka bridge core — SDK client and stream/topic mapping

[ryerraguntla]

Summary

• Initial Kafka–Iggy bridge core for Kafka gateway: Kafka gateway IggyBridge core — SDK client and stream/topic mapping #3533
• Closes Kafka gateway: Kafka gateway IggyBridge core — SDK client and stream/topic mapping #3533
• Part of Kafka gateway: protocol parity tracking #3560
• Depends on Kafka gateway: Build Gateway to Iggy for existing Kafka users - Create a TCP Listener for Kafka users #3421 (TCP listener, request decode, ApiVersions firewall).

Rationale

Phase 1 of the Kafka → Iggy bridge rollout needs a shared IggyBridge so later API handlers (#3535 Produce, #3536 Fetch, #3537 ListOffsets) talk to Iggy through one client, one topic map, and one error map.
#3421 landed wire framing + stub responses with no Iggy backend. This PR lands Layer 3 (gateways/kafka/src/bridge/) as a library: connect, map, provision, watermark, Kafka error codes. Produce/Fetch/ListOffsets dispatch still stubs. Wiring is the follow-on issues, not this one.

What changed?

New module under iggy-gateway-kafka:

#3533 scope item	Landed as
Async Iggy TCP client lifecycle	IggyBridge::connect / close in src/bridge/iggy_bridge.rs
ensure_stream_and_topic(kafka_topic, partition_count)	same file; create-if-missing, idempotent on same partition count
Topic name → Iggy stream/topic mapping (TOML/env)	src/bridge/topic_map.rs + IggyBridgeConfig::from_env
Connection config + SecretString credentials	src/bridge/config.rs (IGGY_KAFKA_IGGY_*)
High watermark for ListOffsets	IggyBridge::high_watermark (see signature note)
Iggy errors → Kafka error codes	BridgeError::to_kafka_error_code in src/bridge/error.rs
Tests; no panic if Iggy unreachable	crate unit tests + tests/bridge_iggy_integration_tests.rs

Limits and Future PRs (still #3533 out of scope)

• Produce/Fetch handler wiring → Kafka gateway: Kafka Produce API (key 0) — bridge to Iggy send_messages #3535 / Kafka gateway: Kafka Fetch API (key 1) — bridge to Iggy offset-based polling #3536
• ListOffsets handler calling high_watermark → Kafka gateway: Kafka ListOffsets API (key 2) — earliest/latest via Iggy high watermark #3537
• Consumer group offset storage → later
• IggyBridgeConfig::from_env is implemented and env-allowlisted in main.rs, but main does not call IggyBridge::connect yet. Live Kafka handlers still stubs.
• Unreachable-Iggy test asserts IggyBridge::connect returns Err, not a panic. No Kafka handler calls the bridge yet.

Mapping / config (README)

Default: Kafka topic orders → Iggy stream IGGY_KAFKA_IGGY_STREAM (default kafka), topic orders.
Overrides: TOML via IGGY_KAFKA_TOPIC_MAP_PATH. File default_stream wins over the env var when both set.

Credentials: fluent TCP builder + SecretString, not iggy://user:pass@host (that URI splits on @ / :).
ensure_stream_and_topic uses Identifier::named so all-digit Kafka names (2024) stay names, not numeric Iggy IDs.
Existing topic with a different partition_count → BridgeError::PartitionCountMismatch → Kafka INVALID_PARTITIONS (37).

Error map

Condition	Kafka code
Stream/topic not found, partition out of range	UNKNOWN_TOPIC_OR_PARTITION (3)
Auth / credential failures	TOPIC_AUTHORIZATION_FAILED (29)
Disconnect / cannot connect	NOT_LEADER_OR_FOLLOWER (6), retriable
Partition count mismatch	INVALID_PARTITIONS (37)
Invalid config / unmatched Iggy error	UNKNOWN_SERVER_ERROR (-1)

Acceptance criteria (#3533)

• Bridge module compiles; invoked from tests/bridge_iggy_integration_tests.rs (real iggy-server harness, not a mock)
• ensure_stream_and_topic idempotent on repeated calls (incl. numeric topic name 2024)
• README documents mapping rules and IGGY_KAFKA_* config (gateways/kafka/README.md)
  Also covered beyond the issue checkbox list: high-watermark empty vs 3 produces; special password p@ss:word; connect to an unbound port returns Err.

Out of scope (issue)

• Produce/Fetch handler wiring (Kafka gateway: Kafka Produce API (key 0) — bridge to Iggy send_messages #3535, Kafka gateway: Kafka Fetch API (key 1) — bridge to Iggy offset-based polling #3536)
• Consumer group offset storage
• docs/BRIDGE_MAPPING.md (partition / group mapping) — still open on SCOPE.md, not this issue

Test plan

cargo fmt --all --check
cargo sort --no-format --workspace --check
cargo clippy -p iggy-gateway-kafka --all-targets --all-features -- -D warnings
cargo test -p iggy-gateway-kafka

Integration tests spawn a real iggy-server binary. Docker not required for this crate’s suite.

[ryerraguntla]

/author

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.47%. Comparing base (71f2961) to head (ddaaf4c).

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #4043      +/-   ##
============================================
- Coverage     87.47%   87.47%   -0.01%     
- Complexity     1575     1576       +1     
============================================
  Files          1280     1280              
  Lines        222999   222999              
  Branches     186362   186363       +1     
============================================
- Hits         195068   195059       -9     
+ Misses        23238    23215      -23     
- Partials       4693     4725      +32     

Components	Coverage Δ
Rust Core	88.54% <ø> (ø)
Java SDK	68.70% <ø> (+0.01%)	⬆️
C# SDK	77.41% <ø> (-0.02%)	⬇️
Python SDK	90.97% <ø> (ø)
PHP SDK	85.67% <ø> (ø)
Node SDK	96.34% <ø> (-0.09%)	⬇️
Go SDK	70.11% <ø> (+0.02%)	⬆️
see 46 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hubcio]

please dont include #PR_NUMBER in PR title because it'd look weird after merge, e.g.
feat(gateways): Iggy Kafka bridge core (#3533) (#4043)

[ryerraguntla]

@krishvishal - I am back from my travel and ready with PR #4043 for the Kafka gateway.

[ryerraguntla]

Please review

[krishvishal]

@ryerraguntla I'll review it.

[krishvishal]

Reviewed the bridge module in depth against the SDK, server and CI config it depends on. The work is careful, and several comments caught real, non-obvious hazards: numeric identifiers, the connection-string @/: split, the WebSocket port collision. The recurring problem in the criticals is that comments claim more than the code delivers; four of the six are a correct observation generalized one step too far.

[ryerraguntla]

/ready

[krishvishal]

Requesting changes. Eight things I would fix before this merges, plus a pile of nits inline.

The three that actually bite today: the gateway's own Iggy login failure comes back to the Kafka client as TOPIC_AUTHORIZATION_FAILED (29), which is fatal and blames the client's own ACLs; every bridge method except connect dials with no deadline, so a black-holed Iggy address parks the call for the kernel SYN-retry window; and a padded topic-map key like [topics." orders "] loads clean and then silently routes that topic to the default stream. The other five are public API shapes that #3535, #3536 and #3537 will inherit, so they are much cheaper to fix here than across four PRs.

Nothing calls the bridge module yet, which is why most of the error-code stuff below is a nit rather than a blocker. Worth landing before the handlers do though.

Reviewed against the Apache Kafka tree for the protocol claims (Errors.java, the exception hierarchy, and the request/response schemas).

[ryerraguntla]

/ready

[krishvishal]

Second pass. Almost none of this overlaps the last round, because most of it is what the fixes to the last round brought in: with_request_timeout on every call, the batch high_watermarks, the mapping-key check, the port band, the delete-and-recreate test.
Nothing here is fatal today, since no handler reaches the bridge yet, but the timeout helper and the high_watermarks signature are the two that get expensive once #3535, #3536 and #3537 are built on top of them.

[ryerraguntla]

/ready