Skip to content

Trident Protect stores application resource archives (resource_backup.tar.gz) unencrypted, exposing Kubernetes Secret data #1192

Description

@sithglan

Repository: NetApp/trident (Trident Protect)
Component: Trident Protect — AppVault / Backup / Snapshot / data mover (Kopia, Restic)
Type: Security / Confidentiality (encryption not applied to all backup data)
Related: #954 (Trident Protect stores the encryption key in the repository)


Summary

Trident Protect encrypts volume data through the data-mover repository (Kopia repositories are
AES256_GCM, Restic repositories are format 2 / encrypted config). However, the application
resource archive
that is written next to the repository as resource_backup.tar.gz is a
plain gzip/tar with no encryption. It contains the full manifest of the backed-up
namespace/resources, including Secret objects whose data values are only base64-encoded.

As a result, anyone with read access to the AppVault object store can recover Secret material
(passwords, Helm release values) without the AppVault data-mover password. This means that
when an operator configures encryption via the AppVault (spec.dataMoverPasswordSecretRef, or
relies on the default data-mover password), part of every backup is still readable in plaintext.


Environment

  • Product: NetApp Trident Protect (protect.trident.netapp.io/v1)
  • CRs present: Application, Backup, Snapshot, ResourceBackup, KopiaVolumeBackup
  • Data movers observed: Kopia (buildVersion 0.18.2), Restic
  • AppVault: object store referenced as s3vault (AppVault UID 5ad31f22-4b01-4d02-930f-056bb1e3da73)
  • Applications: wordpress (namespace wordpress), 7 application instances (wordpress_<uuid>)
  • Backup objects in the AppVault:
    • 12 backups/ archives (backup.json, dataMover Kopia/Restic)
    • 12 snapshots/ archives (snapshot.json)
    • 1 replicatedSnapshots/ archive (app-mirror destination wordpress_101b9ce0-...)
    • 1 resourcebackups/amr-backup-established-... archive
    • Total: 26 resource_backup.tar.gz archives

Steps to reproduce

  1. Protect an application that contains Secret objects. (The bitnami/wordpress Helm chart
    creates wordpress, wordpress-mariadb and sh.helm.release.v1.wordpress.v1.)

  2. Run a Backup (data mover Kopia or Restic) or a Snapshot.

  3. From the AppVault, download the archive recorded in the CR status, e.g.:

    # Backup CR status.appArchivePath
    wordpress_22862ca7-2016-4e58-8419-290d8e27fe25/backups/cluster-05-202609151657_d5deb14d-d2b7-4517-b80f-0e378fdb30ce/
    
  4. Extract and read it — no key, password or tooling is required:

    $ file resource_backup.tar.gz
    resource_backup.tar.gz: gzip compressed data
    $ tar xzf resource_backup.tar.gz
    $ cat Secret/v1/namespaces/wordpress/wordpress.json
    {"apiVersion":"v1","data":{"wordpress-password":"<base64-not-encryption>"},"kind":"Secret", ...}
  5. Base64-decode data.* and the plaintext credential is recovered. The Helm release Secret
    (sh.helm.release.v1.wordpress.v1.json, gzip+base64 of the full chart values) is equally
    readable.


Actual behaviour

  • All 26 resource_backup.tar.gz archives begin with gzip magic 1f 8b and extract to
    plain-text Kubernetes JSON manifests.
  • 25 of 26 archives contain 3 Secret objects each (75 Secret manifests total) with
    readable data (base64 only).
  • No archive is encrypted, regardless of the data mover used (Kopia or Restic) or of the
    AppVault.
  • Two applications have no encrypted repository at all, so none of their backup is encrypted:
    • wordpress_d45785a4-9884-4d0a-aa8e-95f292b3a9d5 (cluster-06) — snapshots/ only
    • wordpress_101b9ce0-db7d-493d-853b-d5017d88add2 (app-mirror destination) — replicatedSnapshots/ + resourcebackups/ only

Expected behaviour

At least one of the following should hold:

  1. The resource archive should be encrypted with the same AppVault data-mover key/password and
    should not be readable without it; or
  2. Secret objects should be excluded/redacted from resource_backup.tar.gz by default (as is
    common for other backup products); or
  3. The documentation should state explicitly that resource_backup.tar.gz is never encrypted and
    that bucket-side encryption (SSE-KMS, bucket policy, IAM) is the only protection available for it.

Impact

  • Confidentiality: any principal with read access to the AppVault bucket (over-permissive bucket
    policy, leaked object-store credentials, replicated bucket, or a support bundle containing the
    archive) can read all Secret values, database passwords and Helm chart values.
  • It bypasses the expectation created by encryption features tied to the AppVault/data-mover
    password: volume data is protected, but application metadata (including credentials) is not.
  • It applies to both Backup and Snapshot archives, and even to the app-mirror replication path.

Evidence: volume repositories are encrypted (for contrast)

10 Kopia repositories and 2 Restic repositories were inspected:

$ python3 - <<'PY'
import json
print(json.load(open(".../kopia/wordpress/wordpress_9b302bcc-aece-4438-ac4f-c5fdb38f67e0/kopia.repository")))
PY
{'tool': 'https://github.com/kopia/kopia', 'buildVersion': '0.18.2',
 'keyAlgo': 'scrypt-65536-8-1', 'encryption': 'AES256_GCM', 'encryptedBlockFormat': '...'}
  • 10/10 kopia.repository blobs: "encryption":"AES256_GCM", "keyAlgo":"scrypt-65536-8-1",
    encryptedBlockFormat present.
  • Restic repositories use an encrypted (format-2) config.
  • No apiVersion/resource plaintext is recoverable from any Kopia/Restic blob.

So the data-mover layer is protected; only the sibling resource_backup.tar.gz is not.


Proposed fix / mitigation

  • Encrypt (or at least redact Secrets in) resource_backup.tar.gz using the AppVault data-mover
    key, ideally encrypting the whole archive rather than only selected objects.
  • If Secrets must be included in the archive for restore fidelity, encrypt the archive and/or
    document clearly that it relies solely on bucket-side encryption.
  • Interim mitigations for operators: enforce bucket-side encryption (SSE-KMS) and least-privilege
    bucket policies; use resourceFilter on the Application/Backup to exclude Secret kinds
    (verify that this is supported for the resource archive); rotate credentials contained in
    protected namespaces.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions