Repository navigation

Features and code reorganization to support the KBase Lakehouse - #243

Open

jeff-cohere wants to merge 107 commits into

mainfrom

kbase-lakehouse

Open

Features and code reorganization to support the KBase Lakehouse#243
jeff-cohere wants to merge 107 commits into
mainfrom
kbase-lakehouse

Commits on Aug 31, 2026
- A good start on HTTPS uploads, to simplify manifest transfers.
  jeff-cohere
  committed
  9102d2e
Commits on Sep 1, 2026
- Troubleshooting https PUT method of Globus access.
  jeff-cohere
  committed
  21f813e
Commits on Sep 2, 2026
Commits on Sep 3, 2026
- Worked up an MMS proxy.
  jeff-cohere
  committed
  d42563b
Commits on Sep 4, 2026
- Closing the loop on credentials and the KBase MMS.
  jeff-cohere
  committed
  c726d3c
Commits on Sep 9, 2026
- Reorganizing Globus requests, and adding logic to register ancillary user credentials.
  jeff-cohere
  committed
  fb66930
- Rephrased interoperability interface.
  jeff-cohere
  committed
  33c13c2
Commits on Sep 10, 2026
- Interim commit. Factoring out Globus APIs for clarity.
  jeff-cohere
  committed
  de13941
- Factored out Globus logic to understand things better.
  jeff-cohere
  committed
  39f2bec
Commits on Sep 11, 2026
- Minor fixes and a version bump.
  jeff-cohere
  committed
  fa723ca
- HTTPS uploads now work properly and have improved error checking.
  jeff-cohere
  committed
  aa11b2f
- Fixed static analysis errors caught by CI.
  jeff-cohere
  committed
  e65b1c2
- Removing stubbed kbase_lakehouse tests.
  jeff-cohere
  committed
  5d1d52a
- Changed root: to base_path: in various configs.
  jeff-cohere
  committed
  c9c7a9c
- Patched up an endpoint test
  jeff-cohere
  committed
  2aa7dc3
- Addressed code review comments.
  jeff-cohere
  committed
  889dece
Commits on Sep 14, 2026
- DTS now checks for existing S3 credentials and updates as needed.
  jeff-cohere
  committed
  dd23b56
- Addressed remaining code review comments.
  jeff-cohere
  committed
  4a76621
- Addressing some additional feedback.
  jeff-cohere
  committed
  f6bff15
Commits on Sep 15, 2026
- Using storage gateway policies to determine Globus connector provider.
  jeff-cohere
  committed
  42b95ab
- Switching out MinIO GitHub action.
  jeff-cohere
  committed
  241610a
- Updating to tagged version of MinIO GitHub action.
  jeff-cohere
  committed
  e57b79d
- Addressing static analysis errors.
  jeff-cohere
  committed
  f8fbad6
- Switching bucket to basepath from datapath for S3 endpoints.
  jeff-cohere
  committed
  10e2730
- Addressing some testing issues.
  jeff-cohere
  committed
  b3dfb55
- Swapping S3 base/data path.
  jeff-cohere
  committed
  8cdd7ec
- Changing some roots/base_paths to data_paths.
  jeff-cohere
  committed
  90c198b
- A couple more test fixes.
  jeff-cohere
  committed
  0fedd83
- A fix for a fix.
  jeff-cohere
  committed
  3e7cdef
- A few more minor fixes.
  jeff-cohere
  committed
  d8e5098
Commits on Sep 16, 2026
- Fixed a glitch in setting source endpoints for new transfers.
  jeff-cohere
  committed
  955a551
- Fixing a mock test condition.
  jeff-cohere
  committed
  d4395f8
- Final fix for the mock test.
  jeff-cohere
  committed
  ebbd669
- One more final fix.
  jeff-cohere
  committed
  5c181ae
- Repurposed existing KBase user federation approach for Lakehouse.
  jeff-cohere
  committed
  8ae5bfd
Commits on Sep 17, 2026
- Fixed an oversight in the deployment config file.
  jeff-cohere
  committed
  65a1ea4
Commits on Sep 18, 2026
- Registered kbase_lakehouse database with service.
  jeff-cohere
  committed
  368ace0
Commits on Sep 21, 2026
- Threading user credentials through transfer requests.
  jeff-cohere
  committed
  d9d3f66
- Some debugging text and some cleanup.
  jeff-cohere
  committed
  db262a7
- Fixing an oversight in determination of endpoint provider.
  jeff-cohere
  committed
  15e40d4
- A bit more debugging info
  jeff-cohere
  committed
  b142c4d
- Typo fix
  jeff-cohere
  committed
  7bc0727
- More debugging
  jeff-cohere
  committed
  c59d8c3
- More breadcrumbs
  jeff-cohere
  committed
  34c864f
Commits on Sep 22, 2026
- One last try for today.
  jeff-cohere
  committed
  8ad8346
- Debugging.
  jeff-cohere
  committed
  4fb0ba8
- Improving error propagation
  jeff-cohere
  committed
  6359493
- Whoopsy
  jeff-cohere
  committed
  a1cb94d
- Scope experiment
  jeff-cohere
  committed
  cf9300a
- Updating GCS manager auth scope
  jeff-cohere
  committed
  c4bc14e
- Simplifying mechanism to determine S3 capability
  jeff-cohere
  committed
  9fea9b7
- More debugging.
  jeff-cohere
  committed
  a73d28d
- Minor tweak
  jeff-cohere
  committed
  9551a4e
- Another experiment.
  jeff-cohere
  committed
  9213e32
- Flipping a bit.
  jeff-cohere
  committed
  29f738a
- Again.
  jeff-cohere
  committed
  a2a1b1c
- Trying another thing.
  jeff-cohere
  committed
  0eda402
- Another.
  jeff-cohere
  committed
  5a5d4f4
- Another permutation
  jeff-cohere
  committed
  404cb98
- One more try.
  jeff-cohere
  committed
  7aa5324
- A fix.
  jeff-cohere
  committed
  31ec0d2
- Trying something else.
  jeff-cohere
  committed
  3937d46
- Moving forward.
  jeff-cohere
  committed
  2a66b56
- Debugging
  jeff-cohere
  committed
  f523e3b
- And again
  jeff-cohere
  committed
  8b351db
- Printing stuff.
  jeff-cohere
  committed
  ac28d7b
Commits on Sep 23, 2026
- Re-establishing GCS manager API auth flow
  jeff-cohere
  committed
  64c0915
Commits on Sep 24, 2026
- Modifying GCS client auth scopes.
  jeff-cohere
  committed
  df69b12
- Trying 'nonfunctional endpoint' for GCS manager API access.
  jeff-cohere
  committed
  759e32f
- Adding an error check.
  jeff-cohere
  committed
  045d863
- Fixing some marshalling issues.
  jeff-cohere
  committed
  b8ebe52
- Addressing more GCS issues.
  jeff-cohere
  committed
  52bd522
- Addressing a few more issues.
  jeff-cohere
  committed
  204e90b
Commits on Sep 25, 2026
- Instrumenting GCS errors with diagnostics.
  jeff-cohere
  committed
  964a8f4
- Attempting S3 cred registration on multiple gateways.
  jeff-cohere
  committed
  df70635
- Updating to community-maintained MinIO image.
  jeff-cohere
  committed
  e3977ec
Commits on Sep 29, 2026
- Adding a Globus ID column to the KBase user federation logic.
  jeff-cohere
  committed
  9242a4e
- Connecting the dots for KBase user Globus IDs.
  jeff-cohere
  committed
  6f36e16
- Fixing a merge conflict.
  jeff-cohere
  committed
  e8fb69d
- Eliminating an unused variable.
  jeff-cohere
  committed
  8e3ad8c
- Removing prints and fixing a bug.
  jeff-cohere
  committed
  901896c
Commits on Sep 30, 2026
- Another try.
  jeff-cohere
  committed
  0f4fbff
- And again.
  jeff-cohere
  committed
  aa84441
- And again.
  jeff-cohere
  committed
  9adc4cf
- And again.
  jeff-cohere
  committed
  33182c2
- Fixed.
  jeff-cohere
  committed
  d790073
- More debugging.
  jeff-cohere
  committed
  c9d8967
- More debugging.
  jeff-cohere
  committed
  70d7e8f
- Disabling a test for NMDC data that went bad.
  jeff-cohere
  committed
  c5dfd90
- Again.
  jeff-cohere
  committed
  fe64493
- Fixed a glitch in the new KBase ORCID spreadsheet parser.
  jeff-cohere
  committed
  c4fd773
- Loosening constraints on storage gateways.
  jeff-cohere
  committed
  9792079
- Trying another thing.
  jeff-cohere
  committed
  cff0f17
- Trying again.
  jeff-cohere
  committed
  358d2e7
- Trying again.
  jeff-cohere
  committed
  beb2e99
- Fixing an issue with mapped usernames.
  jeff-cohere
  committed
  6ef3553
- Fixed glitch in finding existing credentials.
  jeff-cohere
  committed
  a88bcda
- Trying with hard-wired storage gateway.
  jeff-cohere
  committed
  df8b54b
- Whoops.
  jeff-cohere
  committed
  7a727f5
- Trying some other things.
  jeff-cohere
  committed
  20a882d
- A bit of cleanup, and commenting on the error.
  jeff-cohere
  committed
  9b55be5
- Adding more debugging info.
  jeff-cohere
  committed
  4f58f94
- More debugging info
  jeff-cohere
  committed
  3b17352

Commits

Commits on Aug 31, 2026

Commits on Sep 1, 2026

Commits on Sep 2, 2026

Commits on Sep 3, 2026

Commits on Sep 4, 2026

Commits on Sep 9, 2026

Commits on Sep 10, 2026

Commits on Sep 11, 2026

Commits on Sep 14, 2026

Commits on Sep 15, 2026

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 18, 2026

Commits on Sep 21, 2026

Commits on Sep 22, 2026

Commits on Sep 23, 2026

Commits on Sep 24, 2026

Commits on Sep 25, 2026

Commits on Sep 29, 2026

Commits on Sep 30, 2026