Skip to content

feat(stactools): index account fields as pgstac queryables - #372

Open
somoza wants to merge 1 commit into
mainfrom
feat/owner-index-pgstac
Open

somoza wants to merge 1 commit into
mainfrom
feat/owner-index-pgstac

Conversation

@somoza

@somoza somoza commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Registers oam:owner_id and oam:uploader_id as indexed PgSTAC queryables on the openaerialmap Collection, so a CQL2 search for one account's imagery uses an index instead of scanning the Collection. This is the indexing proposed in ADR 0011 (#370), and what "My Imagery" will query.

Related: #371 adds oam:owner_id to Items. This PR does not depend on it: the queryable can exist before any Item carries the field. CI fails on test_legacy_result_with_a_bare_user_id until #369 merges, as on main.

What type of PR is this? (check all applicable)

  • 🍕 Feature
  • 🐛 Bug Fix
  • 📝 Documentation
  • 🧑‍💻 Refactor
  • ✅ Test
  • 🤖 Build or CI
  • ❓ Other (please specify)

Related Issue

Part of #332.

Describe this PR

sync-collection --catalog OAM now also syncs the account queryables. The new queryables.py updates each queryable if it is already registered for the Collection and inserts it otherwise, so rerunning the command never trips PgSTAC's unique name check. Other catalogs are unchanged.

BTREE index with to_text. CQL2 compares these properties as text, so the index uses the same wrapper and the planner can use it. PgSTAC builds the index on every partition of the Collection, including partitions created later.

Docs. docs/dev/ingest/schema.md has a "Searching by account" section with an example search; docs/dev/ingest/index.md points to it from sync-collection.

Screenshots

image

Alternative Approaches Considered

Creating the indexes in a migration instead of through PgSTAC queryables. Queryables are how PgSTAC exposes filterable fields and keeps indexes on new partitions, so a manual index would miss partitions created later.

Review Guide

This writes to PgSTAC's queryables table, so a review from someone who runs the catalogue database would help.

uvx pre-commit run --all-files
just test stactools        # fails only on the #369 test until it merges

After merge, the queryables exist only once someone runs, against production:

hotosm sync-collection --catalog OAM

The index is not built concurrently, so writes to the Collection wait while it is created.

AI Tool Usage

How much of this PR was AI-assisted? (check one)

  • 0 - No AI at any point
  • 1 - AI helped me think, but I wrote all the code myself
  • 2 - I planned the change and decided the approach; AI helped write the code
  • 3 - AI planned and wrote it; I checked and approved each step as it went
  • 4 - I set the AI going and left it to it; I reviewed the finished result
  • 5 - I set the AI going and left it to it; nobody has read the result - no review, or AI review only

If an AI agent is filling this in: declare the level honestly, and open as a draft if it is 5. Do not lower the declared level to get the PR reviewed.

If level 1 or above:

  • Tool(s) used:
  • What was generated:
  • What you reviewed and changed:

Checklist before requesting a review

[optional] What gif best describes this PR or how it makes you feel?

`sync-collection --catalog OAM` now registers oam:owner_id and
oam:uploader_id as BTREE queryables on the openaerialmap Collection.
PgSTAC builds the index on each partition, so a CQL2 search for one
account no longer scans the whole Collection. Running the command again
updates them in place.

This is the indexing proposed in ADR 0011.

Signed-off-by: Julian Somoza <julian.somoza@gmail.com>

This branch was successfully deployed

1 active deployment
test — 477d93e7 Deployed Oct 30, 2026 by somoza via pytest / just #34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant