Repository navigation
fix: Jaeger keeps at most 50 000 traces, and retention and sampling are listed in one table - #450
Merged
Merged
Conversation
…re listed in one table
github-actions
Bot
added
documentation
HardMax71
enabled auto-merge (squash)
October 22, 2026 16:56
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
Codecov Report✅ All modified and coverable lines are covered by tests.
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #380. #430 already removed the four admin monitoring fields and the unused
Settingsfields, and #381 moved the e2e settings tests off them. What's left is Jaeger: it keeps traces in memory with its built-in cap of 100 000. Prod keeps about 12 600 traces a day at roughly 2.3 KB each, so the cap needs about 260 MiB against a 256m limit, and Jaeger would be OOM-killed after about a week and lose every trace.Changes
Jaeger gets
--set=extensions.jaeger_storage.backends.some_storage.memory.max_traces=50000on top of its built-in config: about 4 days in roughly 145 MiB. deployment.md gets one Retention and sampling table pointing at all four values: VictoriaMetrics, Loki, Jaeger and the collector's tail sampling. Small fix found while verifying #449: the OTel SDK'sTransient error ... retryingline, printed without a level when a release restarts the collector, now counts as a warning, so it no longer sends an error-lines mail.Type of Change
Test Plan
cd backend && uv run pytest tests/unit)cd frontend && npm run test)docker compose exec -T backend sh -c 'uv sync --group test --no-dev --frozen --no-install-project --no-build && pytest tests/e2e')cd frontend && npm run test:e2e)./deploy.sh openapi, then./deploy.sh types), never edited by handValidation: jaeger:2.20.0 starts with the flag and refuses a misspelled key, so the value applies. The retry line goes from error to warning through the collector, and the 164 earlier sample lines keep their levels.
Checklist
pre-commit run --all-filesand all checks passSummary by cubic
Caps Jaeger's in-memory trace storage at 50 000 traces, down from the built-in 100 000, so prod Jaeger no longer gets OOM-killed after about a week and loses every trace. Consolidates the retention and sampling settings for VictoriaMetrics, Loki, Jaeger, and the collector's tail sampling into one "Retention and sampling" table in
deployment.mdand links it fromtracing.md.Bug Fixes
Transient error ... retryingline, printed without a level, now counts as a warning, so a release restart no longer triggers an error-lines mail.Written for commit ab737b1. Summary will update on new commits.