Skip to content

feat(search): add a --timeout flag to the index command - #13053

Merged
mmattel merged 2 commits into
owncloud:masterfrom
paul43210:fix/search-index-timeout
Oct 9, 2026
Merged

mmattel merged 2 commits into
owncloud:masterfrom
paul43210:fix/search-index-timeout

Conversation

@paul43210

Copy link
Copy Markdown
Contributor

Description

ocis search index --space <id> / --all-spaces calls IndexSpace with a hard-coded go-micro RequestTimeout of 10 minutes. The service walks the whole space inside that request's context, so once the timeout expires the next gateway call inside the walk fails with rpc error: code = Canceled (or DeadlineExceeded), the walk logs error walking the tree and stops, and the CLI prints failed to index space: … context deadline exceeded. Everything after the cancellation point stays unindexed, and a space whose walk takes longer than ten minutes can never be fully indexed from the command line.

This PR makes the timeout a flag:

  • --timeout <duration>, default 10m, so the behaviour is unchanged for anyone not passing it;
  • 0 waits indefinitely. go-micro's gRPC client applies context.WithTimeout(ctx, RequestTimeout) unconditionally when the context has no deadline, so a zero value would expire at once; the command substitutes a far-off duration instead (commented in the code);
  • the README's indexing section documents the flag.

No service, config or interface changes.

Related Issue

Motivation and Context

On an oCIS 8.2.0 single-binary instance with a Tika extractor, a 30,000-file personal space has three scanned PDFs the extractor cannot finish; each costs the walk several minutes, so a full walk takes 15–20 minutes. Every ocis search index run was cancelled at exactly 10 minutes (observed 2026-09-30 and 2026-10-01: error walking the tree: rpc error: code = Canceled desc = context canceled), while an event-triggered walk of the same space, which runs with context.Background(), completed. A nightly --all-spaces safety net is impossible without a way to allow a longer run.

How Has This Been Tested?

  • test environment: Go 1.26.0 (toolchain per go.mod), Linux amd64, branch based on current master (784c5ec).
  • go build ./services/search/... and go vet ./services/search/pkg/command/ clean; make changelog-lint clean.
  • Manual: the command is a thin gRPC call with no unit tests today; the flag parsing is urfave/cli's DurationFlag. The 10-minute cancellation itself is reproducible on the instance described above with the unmodified 8.2.0 binary.

Screenshots (if appropriate):

n/a

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added — the command has none; the change is a flag passed through to the client call
  • Acceptance tests added — not applicable
  • Documentation: README section on indexing updated in this PR; changelog entry included

🤖 Generated with Claude Code

https://claude.ai/code/session_01WpoeSoKYEdgQcWjEzTVKpT

@paul43210
paul43210 requested a review from a team as a code owner October 2, 2026 14:57
@kw-security

kw-security commented Oct 2, 2026 •

Copy link
Copy Markdown

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues
✅ Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@mmattel

mmattel commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

@paul43210 all your PRs fail because of proper semantic commits, please fix.
See: https://www.conventionalcommits.org/en/v1.0.0/

enhancement(search): add a --timeout flag to the index command - #13053 -->
feat(search): add a --timeout flag to the index command - #13053

@paul43210
paul43210 force-pushed the fix/search-index-timeout branch from 6510d1a to 31a05c5 Compare October 5, 2026 21:35
@paul43210 paul43210 changed the title enhancement(search): add a --timeout flag to the index command feat(search): add a --timeout flag to the index command Oct 5, 2026
@paul43210

Copy link
Copy Markdown
Contributor Author

@mmattel thanks, fixed: #13053 and #13055 now use feat(...) (commits and titles), and #13034's extra changelog: commit is folded into its fix(search) commit. #13051 and #13072 already pass.

ocis search index called the search service with a hard-coded 10 minute
request timeout. The service walks the whole space inside that request, so
when the timeout expired the walk was cancelled (error walking the tree:
context canceled) and everything after that point stayed unindexed. A space
whose walk takes longer than ten minutes could never be fully indexed from
the command line.

The timeout is now a flag: --timeout takes a duration, default 10m so the
behaviour is unchanged out of the box; 0 waits indefinitely (go-micro treats
a zero RequestTimeout as an already-expired deadline, so a far-off one is
substituted).

Fixes owncloud#13052

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Paul Faure <paul@faure.ca>
@paul43210
paul43210 force-pushed the fix/search-index-timeout branch from 31a05c5 to 8cf9a72 Compare October 7, 2026 00:06
@jvillafanez

Copy link
Copy Markdown
Member

I'm not against this, but I'm missing a warning somewhere: the search engine uses an exclusive lock by default. This means that, by default, if oCIS (the search service in particular) is started, this command won't work. If this command runs before oCIS starts, the search service won't search until the command finishes. Check the "scaling" section of the README file.

Since the SEARCH_ENGINE_BLEVE_SCALE=true is practically a requirement, I'd prefer to have it mentioned somewhere. I'm pretty sure someone will run the command without the timeout on a default installation and waste a lot of time without doing anything.

The index command is a gRPC client of the search service and never opens
the Bleve index, so it works with the default exclusive lock
(SEARCH_ENGINE_BLEVE_SCALE=false) but needs the service running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WpoeSoKYEdgQcWjEzTVKpT
Signed-off-by: Paul Faure <paul@faure.ca>
@paul43210

Copy link
Copy Markdown
Contributor Author

@jvillafanez thanks. I don't think the lock applies to this command, though. ocis search index never opens the Bleve index: it's a gRPC client that asks the running search service to do the indexing (IndexSpace, services/search/pkg/command/index.go). The service holds the lock, and the command only waits for it. We run it nightly on an 8.2 install with the default SEARCH_ENGINE_BLEVE_SCALE=false while oCIS is up, and it works.

The flip side is worth saying, so I added a sentence to the README in cfd12b0: the search service must be running, and the default exclusive lock doesn't block the command.

@mmattel
mmattel merged commit b5f843e into owncloud:master Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search: ocis search index cancels any space walk longer than 10 minutes

4 participants