Add conditional-request support (ETag + Last-Modified) to PyPI metadata APIs for Akamai Centralized Authorization

[dkliban]

Summary

The PyPI metadata APIs (Simple API and JSON Metadata API) need conditional request support for Akamai Centralized Authorization, which requires If-Modified-Since / Last-Modified headers. This also enables client and edge cache efficiency.

Primary driver: Akamai Centralized Authorization requires If-Modified-Since for cache revalidation while Pulp retains authorization control.

Background: Akamai Centralized Authorization

Akamai Centralized Authorization allows edge caches to cache content while Pulp performs authorization on every request:

Cache hit flow:

1. Client sends GET with credentials to Akamai
2. Akamai sends If-Modified-Since conditional request to Pulp (with client's auth headers)
3. Pulp runs ContentGuard authorization
4. If authorized and content unchanged: Pulp returns 304 Not Modified (no body) → Akamai serves cached content
5. If authorized and content changed: Pulp returns 200 with new content → Akamai updates cache
6. If not authorized: Pulp returns 403 → Akamai blocks request

Key requirement: Akamai sends If-Modified-Since for revalidation, so Pulp must support Last-Modified + If-Modified-Since (not just ETag).

Current State

Simple API (/simple/{package}/)

• ✅ Has ETag and If-None-Match → 304 support
• ✅ Has Cache-Control: max-age=600, public
• ❌ Missing Last-Modified header
• ❌ Missing If-Modified-Since → 304 handling

JSON Metadata API (/pypi/{package}/json/)

• ❌ No ETag
• ❌ No Last-Modified
• ❌ No Cache-Control
• ❌ No conditional request handling

Proposed Implementation

Both SimpleView and MetadataView need:

1. Last-Modified header on all 200 responses

   • Use the same repo-version-based timestamp as the current ETag function
   • Value: timestamp when the repository version was created

2. If-Modified-Since conditional request handling:

   a. Run ContentGuard authorization (always, on every request)
   b. If not authorized → return 403 (stop here)
   c. If authorized and If-Modified-Since present:
      - If content not modified since that datetime → return 304 (no body)
      - If content modified or header invalid → return 200 with content
   d. If no If-Modified-Since header → return 200 with content
   

3. Cache-Control headers:

   • Simple index pages: public, max-age=300 (5 minutes - mutable metadata)
   • JSON metadata: public, max-age=900 (15 minutes - matches pypi.org)

4. ETag support (JSON Metadata API only, for pypi.org parity):

   • Add ETag header and If-None-Match → 304 handling
   • Use the same _etag_func already implemented for SimpleView

Security Requirement

Authorization MUST be checked on every request, including conditional requests.

Order of operations:

1. Run ContentGuard authorization using client credentials from request headers
2. If not authorized → return 403 Forbidden (stop here)
3. If authorized → check If-Modified-Since or If-None-Match
4. Return 304 or 200 based on conditional check

This ensures unauthorized clients cannot:

• Determine if content exists
• Learn modification timestamps
• Bypass authorization checks

pypi.org Behavior Comparison

For reference, pypi.org (Warehouse) uses ETag exclusively:

Endpoint	ETag	Last-Modified	If-None-Match→304	Cache-Control
/pypi/{pkg}/json	✅	❌ never sent	✅	max-age=900, public
/simple/{pkg}/	✅	❌ never sent	✅	max-age=600, public

Warehouse doesn't send Last-Modified because it doesn't need edge-cache revalidation (Fastly handles it differently). Pulp needs both validators:

• Last-Modified / If-Modified-Since for Akamai Centralized Authorization
• ETag / If-None-Match for pypi.org parity (optional enhancement)

Implementation Notes

• Both views are Django views served by pulp-api, NOT the content app
• They will NOT inherit If-Modified-Since support from pulpcore #7929 (which applies to content app binary downloads only)
• Reuse the existing _etag_func from pulp_python/app/pypi/views.py
• Use Django's @condition decorator (supports both etag_func and last_modified_func)

Testing

Add functional tests mirroring test_simple_cache_etag_conditional_request:

For SimpleView:

• Test If-Modified-Since with matching timestamp → 304
• Test If-Modified-Since with old timestamp → 200 with content
• Test no If-Modified-Since header → 200 with content
• Test Last-Modified header present on 200 responses

For MetadataView:

• Same tests as SimpleView
• Test If-None-Match with matching ETag → 304 (optional)
• Test ETag header present on 200 responses (optional)

Related Issues

• pulpcore #7929: Adds If-Modified-Since to content app for binary package downloads
• PULP-2262 (Jira): If-Modified-Since support for Akamai centralized authorization
• PULP-2125 (Jira): Epic for edge-cached content delivery

Files

• pulp_python/app/pypi/views.py - SimpleView, MetadataView
• pulp_python/tests/functional/api/test_simple_cache.py - conditional request tests