Skip to content

init: list a git corpus with git ls-files --exclude-standard so gitignored files never enter the manifest - #9

Closed
vbanas wants to merge 1 commit into
vsov:mainfrom
vbanas:manifest-respects-gitignore
Closed

init: list a git corpus with git ls-files --exclude-standard so gitignored files never enter the manifest#9
vbanas wants to merge 1 commit into
vsov:mainfrom
vbanas:manifest-respects-gitignore

Conversation

@vbanas

@vbanas vbanas commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@vsov

vsov commented Oct 19, 2026

Copy link
Copy Markdown
Owner

Integrated locally as 03258cd (your commit, authorship preserved) + 6f85cde (follow-up). Thanks, @vbanas!

The follow-up gates the git listing on corpus/.git, the same gate segment._walk uses. With rev-parse --is-inside-work-tree, a corpus that is a subdirectory of a repo lost gitignored files the plan was built from, and a corpus inside an ignored directory got an empty manifest. Git output is now read as bytes + os.fsdecode, so a non-UTF-8 file name no longer raises. Tests: core/tests/test_manifest_gitignore.py. The CHANGELOG notes the one-time okfy diff / manifest_digest change for existing bundles. The change reaches this repository with the next export.

@vsov vsov closed this Oct 19, 2026
vsov pushed a commit that referenced this pull request Oct 19, 2026
Viktor Sovietov @vbanas @claude
Unreleased: round 20, fragment retrieval, pre-edit lookup, contribute…
7b32481 
…d PR #9

Version stays 0.29.0; details under "## Unreleased" in CHANGELOG.md.

- Pre-edit lookup: okfy preedit / okfy_preedit (MCP tool 8) lists the
  concepts that govern the paths an agent is about to edit, by declared
  path_scope, and says when the answer is incomplete. Advisory, not
  enforcement. A conflicts crosswalk relation with a validating reader;
  the federation code digest moves, so federated eval runs read stale.
- Fragment retrieval (opt-in): okfy query --passages / passages=true adds
  exact source passages for the top three hits under a token budget.
  Ranking is unchanged; the token economy is not yet measured.
- Round 20: migrate ids refuses hard-linked pages and a symlinked Purpose
  by its own path; git-mode diff reads NUL-separated paths; ledger,
  dissent and journal readers frame rows by LF only; a missing manifest
  no longer lets proposals claim checked sources; auth docs corrected.
- #9 The corpus manifest respects .gitignore for a corpus whose root
  holds .git, the gate the plan already used; existing bundles see a
  one-time okfy diff / manifest_digest change.

Contributed by @vbanas in #9.

Co-authored-by: vbanas <val.banas@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vsov

vsov commented Oct 19, 2026

Copy link
Copy Markdown
Owner

A design question, if you have a minute, @vbanas: your version gated the git listing on rev-parse --is-inside-work-tree, i.e. it also covered a corpus that is a subdirectory of a repository. Was that from a real layout you use (say repo/docs as the corpus), or a generalisation?

It matters for a follow-up decision. Today a subdirectory corpus is walked directly, gitignored files included, for both the plan and the manifest, and okfy survey now says so in a listing field. Moving subdirectories to git would change existing bundles, so we would only do it, with an explicit owner choice, if such corpora exist in practice.

@vbanas

vbanas commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

A design question, if you have a minute, @vbanas: your version gated the git listing on rev-parse --is-inside-work-tree, i.e. it also covered a corpus that is a subdirectory of a repository. Was that from a real layout you use (say repo/docs as the corpus), or a generalisation?

It matters for a follow-up decision. Today a subdirectory corpus is walked directly, gitignored files included, for both the plan and the manifest, and okfy survey now says so in a listing field. Moving subdirectories to git would change existing bundles, so we would only do it, with an explicit owner choice, if such corpora exist in practice.

@vsov My original intent was simply to exclude gitignored files and folders from okfy indexing. Ignored files may contain sensitive data or simply User doesn't want any links to the non-committed/ignored files inside the knowledge base. Usage of rev-parse --is-inside-work-tree was a simple generalization for different possible use cases. I think sub-directories is a rare case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vbanas @vsov