diff --git a/AGENTS.md b/AGENTS.md index 3d3344f..b1f068a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,7 +24,7 @@ This file is for agents changing the gatecheck repository. To use the gatecheck ## Conventions - Check `gatecheck --help` before describing a flag. `docs/cli-reference.md` matches the CLI on main. -- Report type detection is a lowercase substring of the filename. See `docs/supported-reports.md`. +- Report type is detected from file contents. A lowercase filename substring is the fallback when the contents are ambiguous. See `docs/supported-reports.md`. - Coverage thresholds and EPSS scores are ratios from 0 to 1. - Exit codes from `cmd/gatecheck`: 0 success, 1 threshold failure (`gatecheck.ErrValidationFailure`), 255 any other error (`os.Exit(-1)`). `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, and `osv-scanner` return the scanner's status. A missing scanner still exits 255. - Do not add copyright notices. diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e978ab..aadff86 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- `list` and `validate` read SARIF 2.1.0 from any tool, including KICS, Checkov, Bandit, govulncheck, and `semgrep --sarif`. Severity limits use `security-severity` when it is set, and the SARIF level otherwise. `sarif.ruleLimit` denies rule ids and `sarif.ruleRiskAcceptance` allows them. A SARIF report with no findings prints `No sarif findings`. - `gatecheck bootstrap` installs pinned Trivy 0.76.1, and `gatecheck trivy` runs it. - Trivy JSON reports work with `list`, `validate`, and `bundle`. Severity limits and CVE allow and deny lists follow the Grype rules. - `gatecheck bootstrap` installs pinned osv-scanner 2.7.0, and `gatecheck osv-scanner` runs it. @@ -16,8 +17,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- Report type comes from the file contents. The filename is used only when the contents are ambiguous. +- Any CycloneDX JSON BOM with vulnerabilities is listed and validated with the `cyclonedx` rules, whichever tool wrote it. A missing severity word falls back to the CVSS score. - `gatecheck bootstrap --list` widens the tool column so `osv-scanner` fits. +### Fixed + +- A CycloneDX file with `grype` in the name was read as Grype, so its vulnerabilities were ignored. Fixes #80. + ## [0.9.0] - 2026-10-23 ### Added diff --git a/README.md b/README.md index f6b1a41..1cb7f37 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ Generate a configuration file with the default thresholds. Every rule starts dis gatecheck config init > gatecheck.yaml ``` -Validate a report. The config is a flag. The report type comes from the filename, so a Grype report needs `grype` in the name. +Validate a report. The config is a flag. The report type comes from the file contents. The filename is used when the contents are ambiguous, so a Grype report can still be named `grype-report.json`. ```shell gatecheck validate --config gatecheck.yaml grype-report.json @@ -66,7 +66,7 @@ gatecheck validate --help ### Bundling Artifacts -Bundling artifacts produces a gzipped tarball and a manifest. The bundle filename needs to contain `bundle` so `list` and `validate` treat it as a bundle. +Bundling artifacts produces a gzipped tarball and a manifest. `list` and `validate` recognize a bundle from that manifest. A name that contains `bundle` still works when the contents are ambiguous. ```shell gatecheck bundle create gatecheck-bundle.tar.gz grype-report.json diff --git a/cmd/list.go b/cmd/list.go index 8b86ee3..7f2b36f 100644 --- a/cmd/list.go +++ b/cmd/list.go @@ -1,19 +1,17 @@ package cmd import ( + "bytes" "errors" "fmt" + "io" "log/slog" "os" - "slices" - "strings" "github.com/gatecheckdev/gatecheck/pkg/gatecheck" "github.com/spf13/cobra" ) -var supportedTypes = []string{"grype", "semgrep", "gitleaks", "syft", "cyclonedx", "bundle", "gatecheck", "trivy", "osv"} - var listCmd = &cobra.Command{ Use: "list", Short: "print a table of the findings in a report or files in a gatecheck bundle", @@ -97,17 +95,29 @@ var listAllCmd = &cobra.Command{ slog.Debug("run list all", "epss", fmt.Sprintf("%v", epss), "markdown", fmt.Sprintf("%v", markdown)) for _, filename := range args { - supportedFunc := func(s string) bool { - return strings.Contains(filename, s) - } cmd.Printf("%s\n", filename) - if !slices.ContainsFunc(supportedTypes, supportedFunc) { - slog.Warn("file not supported, skip", "filename", filename) + + src, err := os.Open(filename) + if err != nil { + if errors.Is(err, os.ErrNotExist) { + slog.Error("file not found, skip", "filename", filename) + continue + } + slog.Error("cannot open file, skip", "filename", filename, "error", err) + continue + } + content, err := io.ReadAll(src) + if closeErr := src.Close(); closeErr != nil { + slog.Debug("close file", "filename", filename, "error", closeErr) + } + if err != nil { + slog.Error("cannot read file, skip", "filename", filename, "error", err) continue } - if _, err := os.Stat(filename); errors.Is(err, os.ErrNotExist) { - slog.Error("file not found, skip", "filename", filename) + supported, withEPSS := gatecheck.ListAllAction(content, filename) + if !supported { + slog.Warn("file not supported, skip", "filename", filename) continue } @@ -121,7 +131,7 @@ var listAllCmd = &cobra.Command{ } opts = append(opts, displayOpt) - if epss && slices.ContainsFunc([]string{"grype", "cyclonedx", "trivy", "osv"}, supportedFunc) { + if epss && withEPSS { epssOpt, err := gatecheck.WithEPSS(epssFile, epssURL, epssCacheOptions()...) if err != nil { slog.Error("epss fetch failure, skip", "filename", filename, "error", err) @@ -131,13 +141,7 @@ var listAllCmd = &cobra.Command{ } dst := cmd.OutOrStdout() - src, err := os.Open(filename) - if err != nil { - slog.Error("cannot open file, skip", "filename", filename, "error", err) - continue - } - - err = gatecheck.List(dst, src, filename, opts...) + err = gatecheck.List(dst, bytes.NewReader(content), filename, opts...) if err != nil { slog.Error("cannot list report, skip", "filename", filename, "error", err) continue @@ -157,7 +161,7 @@ func newListAllCommand() *cobra.Command { } func newListCommand() *cobra.Command { - listCmd.Flags().StringP("input-type", "i", "", "the input filetype if using STDIN [grype|semgrep|gitleaks|syft|bundle|trivy|osv]") + listCmd.Flags().StringP("input-type", "i", "", "stdin type word used when the report content is ambiguous [grype|cyclonedx|semgrep|gitleaks|trivy|osv|syft|sarif|bundle|lcov]") listCmd.Flags().Bool("markdown", false, "print as a markdown table") listCmd.Flags().Bool("epss", false, "List with EPSS data") RuntimeConfig.EPSSURL.SetupCobra(listCmd) diff --git a/docs/agent-guide.md b/docs/agent-guide.md index 16e59f3..e18b44d 100644 --- a/docs/agent-guide.md +++ b/docs/agent-guide.md @@ -48,31 +48,51 @@ The managed binary wins when it is present. `gatecheck grype`, `gatecheck syft`, A platform with no Grype, Syft, Gitleaks, Trivy, or osv-scanner asset fails that tool and the command exits 255. A platform with no semgrep wheel prints a skip message on stderr and continues. Current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, and osv-scanner `2.7.0`, in `internal/bootstrap/versions.go`. -Syft is installed so it can write CycloneDX (`syft -o cyclonedx-json`). A filename containing `syft` is still recognized by `list` and `validate` and then returns an error. CycloneDX is a report format, and coverage tools are left to the language toolchain. +Syft is installed so it can write CycloneDX (`syft -o cyclonedx-json`). That BOM is listed and validated as CycloneDX. Native Syft JSON is recognized from `descriptor.name` or the schema URL, and `list` and `validate` then return an error. A filename containing `syft` does the same when the contents are ambiguous. CycloneDX is a report format, and coverage tools are left to the language toolchain. ## Report type -`list` and `validate` pick a type by the first matching lowercase word in the filename: +`list` and `validate` pick a type from the file contents. The filename is used only when the contents do not identify a type. + +Content rules, in this order: + +1. A gzipped tar that contains `gatecheck-manifest.json` is a Gatecheck bundle. +2. `bomFormat` of `CycloneDX` is CycloneDX JSON. This wins when the name contains `grype` or `syft`, and it wins when the tool that wrote the BOM was Grype, Syft, Trivy, or another tool. CycloneDX XML is not read. +3. SARIF 2.1.0. The top-level `version` is the string `2.1.0` and `runs` is an array. A `$schema` that mentions SARIF is enough when `runs` is missing or null. Another version returns an error when the name says `sarif`. +4. Grype JSON. `descriptor.name` is `grype`, or `matches` is a non-empty array of objects with `vulnerability` and `artifact`. +5. Syft JSON. `descriptor.name` is `syft`, or `schema.url` points at the Syft schema. `list` and `validate` then return an error. Syft JSON is not implemented. A Syft CycloneDX BOM is CycloneDX, not Syft. +6. Semgrep JSON. `results` is an array of objects with `check_id`, or `results` and `errors` are both arrays. Semgrep's SARIF output is SARIF, not this type. +7. Gitleaks JSON. A non-empty array of objects with `RuleID` and one of `File`, `Commit`, or `Fingerprint`. +8. Trivy JSON. `SchemaVersion` is a number, and `ArtifactType` is set or `Results` is an array. Vulnerability entries are listed and validated. Misconfigurations, secrets, and licenses are ignored. A CycloneDX BOM from Trivy is CycloneDX. +9. osv-scanner JSON. `results` is a non-empty array of objects with a `packages` array. One finding per vulnerability group. Aliases of that group are not counted again. License-only packages are skipped. Semgrep is checked first, so a `check_id` stays Semgrep. `{"results":[]}` does not identify itself. +10. LCOV text. A line starts with `SF:` and the file also has `DA:`, `LF:`, or `end_of_record`. +11. Clover XML. An XML document with `clover=` or a ``, so the same words work, including the coverage words. `list-all` and `validate` do not read standard input. `validate` requires a file path. +Coverage format when the name is the only signal, in this order: the name contains `lcov` or ends in `.info` (LCOV); the name contains `clover` (Clover XML); the name ends in `.xml` (Cobertura XML). A name that only contains `coverage`, such as `coverage.json`, is selected as coverage and then fails. -`list-all` lists a file only when the name contains `grype`, `semgrep`, `gitleaks`, `syft`, `cyclonedx`, `bundle`, `gatecheck`, `trivy`, or `osv`. Coverage files are skipped with a warning. There is no `--input-type` on `list-all`. +From standard input, `list` requires `-i` / `--input-type`, so the command does not wait on a terminal. The value is matched as the filename `stdin:`, and only when the contents are ambiguous. Clear contents win over `--input-type`. `list-all` and `validate` do not read standard input. `validate` requires a file path. -Inside a bundle, `validate` uses the stored label, which is the base name of the file that was added. Labels that do not match a type are skipped. Syft labels inside a bundle are skipped. +`list-all` lists a file when the contents or the name identify Grype, Semgrep, Gitleaks, Syft, CycloneDX, SARIF, Trivy, osv-scanner, or a bundle. Coverage files are skipped with a warning. There is no `--input-type` on `list-all`. + +Inside a bundle, `validate` uses the same content rules on each stored file, and falls back to the label. The label is the base name of the file that was added. Labels that still do not match are skipped. Syft JSON inside a bundle is skipped. ## Config @@ -86,7 +106,7 @@ gatecheck config convert --file gatecheck.yaml --output toml `config convert` reads `-f` / `--file` (default `gatecheck.yaml`). The extension selects the decoder: `.yaml`, `.yml`, `.json`, or `.toml`. -A partial file is fine. Omitted fields keep the default. Unknown keys are ignored. With no config, `validate` uses the built-in default: every rule disabled, coverage thresholds `0`. Grype, CycloneDX, Trivy, osv-scanner, Semgrep, Gitleaks, coverage, and bundle reports pass under that default. A Syft filename still returns an error. +A partial file is fine. Omitted fields keep the default. Unknown keys are ignored. With no config, `validate` uses the built-in default: every rule disabled, coverage thresholds `0`. Grype, CycloneDX, Trivy, osv-scanner, SARIF, Semgrep, Gitleaks, coverage, and bundle reports pass under that default. Syft JSON still returns an error. An old config with no `sarif` section leaves those limits disabled. `GATECHECK_CONFIG_FILENAME` is the same setting as `validate --config` / `-f`. @@ -104,6 +124,11 @@ Field names below are the ones `config init` emits. In YAML they are case sensit `gitleaks.limitEnabled: true` fails validation when the report contains any finding. There is no count. +`sarif` is for any SARIF 2.1.0 report. Semgrep's own JSON still uses `semgrep`. `semgrep --sarif` uses `sarif`. + +- `severityLimit` has `critical`, `high`, `medium`, and `low`. Each has `enabled` and `limit`, with the same count rule as Grype. The severity comes from the SARIF property `security-severity` when a result or its rule has one. The value is a number from 0 to 10, or a numeric string. 9.0 and above is critical, 7.0 to 8.9 is high, 4.0 to 6.9 is medium, above 0 and below 4.0 is low, and 0 is none. A result property wins over the same property on the rule. When neither is set, gatecheck uses the SARIF level. A missing result level uses the rule's `defaultConfiguration.level`, then `warning`. `error` is high, `warning` is medium, `note` is low, and `none` is ignored. +- `ruleLimit` and `ruleRiskAcceptance` each have `enabled` and `rules`. A rule entry is `id` plus an optional `metadata.tags` list. `id` is compared without regard to case. `ruleLimit` denies those ids. `ruleRiskAcceptance` removes them. + `coverage.lineThreshold`, `functionThreshold`, and `branchThreshold` are ratios from 0 to 1: covered lines, functions, or branches divided by the total. The report fails when its ratio is less than the threshold. A ratio equal to the threshold passes. `0.8` is 80 percent. `80` fails a report whose real line coverage is 0.8, because 0.8 is less than 80. There is no `enabled` flag. The default `0` allows any coverage, because a ratio is never less than 0. `list` prints these numbers as percents in the table footer. The config still wants the ratio. Grype, CycloneDX, Trivy, and osv-scanner rules run in this order. A failure returns immediately, except where a rule only removes findings. Trivy counts vulnerability entries only. osv-scanner counts one finding per group. @@ -116,7 +141,20 @@ Grype, CycloneDX, Trivy, and osv-scanner rules run in this order. A failure retu 6. EPSS limit. 7. Severity limit. -Semgrep drops findings whose severity limit is disabled, then applies impact risk acceptance, then the severity counts. Coverage runs all three checks and reports each failure. A bundle is validated when its filename contains `bundle`. Each label is checked on its own. The command exits 1 if any label fails a threshold. +Semgrep drops findings whose severity limit is disabled, then applies impact risk acceptance, then the severity counts. That impact rule does not apply to `semgrep --sarif`. That file uses the `sarif` rules. + +SARIF runs in this order. + +1. Rule limit, when enabled. A denied rule fails even when its severity limit is disabled. +2. Disabled severities are dropped. `none` has no limit field and stays in the report, but the severity rule does not count it. +3. Rule risk acceptance removes matching ids. +4. Severity limit. + +These SARIF results are not counted: `kind` of `pass` or `notApplicable`, `level` of `none`, a non-empty `suppressions` array, and `baselineState` of `absent`. SARIF has no EPSS or KEV rules. + +Any CycloneDX JSON BOM uses the `cyclonedx` rules, including a BOM from Grype, Trivy, Syft, or another tool. The severity is the highest rating. A rating with a score and no severity word uses CVSS ranges. A CVSS v2 method does not produce critical. `important` counts as high, and `moderate` counts as medium. `negligible` counts as none. Empty ratings use a top-level `severity` string when the producer set one, otherwise `unknown`. `unknown`, `none`, and `info` are not counted by the critical, high, medium, or low limits. The id is the vulnerability `id`, or the `bom-ref` when `id` is empty. + +Coverage runs all three checks and reports each failure. A bundle is validated when its contents are a Gatecheck bundle, or when the filename contains `bundle` and the contents do not identify another type. Each stored file is checked on its own. The command exits 1 if any file fails a threshold. ## validate @@ -148,13 +186,13 @@ grype image:tag -o json | gatecheck ls --input-type grype gatecheck list-all grype-report.json semgrep-report.json ``` -`list` opens the first filename only. Further filenames are ignored. `--markdown` prints a markdown table. A Grype, CycloneDX, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table, with or without `--markdown`: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `--epss` adds EPSS score and percentile columns for Grype, CycloneDX, Trivy, and osv-scanner. Other report types still list, and those columns are omitted. `--epss-url` empty uses `https://epss.cyentia.com`. `--epss-filename` reads that CSV instead of calling the API. +`list` opens the first filename only. Further filenames are ignored. `--markdown` prints a markdown table. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table, with or without `--markdown`: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `--epss` adds EPSS score and percentile columns for Grype, CycloneDX, Trivy, and osv-scanner. Other report types still list, and those columns are omitted. `--epss-url` empty uses `https://epss.cyentia.com`. `--epss-filename` reads that CSV instead of calling the API. -`list-all` prints each filename, then its table, or that sentence when the report has no findings. A file whose name is not a supported `list-all` type is skipped with a warning. A missing file is skipped. A list error is logged and the command continues. The command exits 0. `--epss` adds EPSS data only for filenames that contain `grype`, `cyclonedx`, `trivy`, or `osv`. There is no `--epss-filename` flag, and `GATECHECK_EPSS_FILENAME` is not read. `GATECHECK_EPSS_URL` overrides the API. `--epss-cache-dir` and `--epss-no-cache` are accepted. +`list-all` prints each filename, then its table, or that sentence when the report has no findings. A file whose contents and name are not a supported `list-all` type is skipped with a warning. A missing file is skipped. A list error is logged and the command continues. The command exits 0. `--epss` adds EPSS data for Grype, CycloneDX, Trivy, and osv-scanner, including a file whose name does not contain those words when the contents do. There is no `--epss-filename` flag, and `GATECHECK_EPSS_FILENAME` is not read. `GATECHECK_EPSS_URL` overrides the API. `--epss-cache-dir` and `--epss-no-cache` are accepted. ## bundle -Use a name that contains `bundle`, such as `gatecheck-bundle.tar.gz`. `validate` treats the file as a bundle only when the name contains `bundle`. `list` does that too, and also when the name contains `gatecheck`. A bundle is a gzipped tar plus a JSON manifest. +A bundle is a gzipped tar plus a JSON manifest named `gatecheck-manifest.json`. `list` and `validate` treat the file as a bundle when the contents are that archive. The filename is the fallback: `validate` requires the word `bundle`, and `list` also accepts a name containing `gatecheck`. A name such as `project.gcb` still works when the file is a real bundle. A name that contains `bundle`, such as `gatecheck-bundle.tar.gz`, still works when the contents are ambiguous. ```shell gatecheck bundle create gatecheck-bundle.tar.gz grype-report.json --tag ci @@ -163,7 +201,7 @@ gatecheck bundle remove gatecheck-bundle.tar.gz semgrep-report.json gatecheck list gatecheck-bundle.tar.gz ``` -The stored label is the base name of the target file (`reports/grype-report.json` is stored as `grype-report.json`). Later validation uses that label to pick the type, so the file name needs a word such as `grype` or `semgrep`. Repeat `-t` / `--tag` for more than one tag. When the flag is omitted, `GATECHECK_BUNDLE_TAG` is read as a comma-separated list. `bundle add` requires an existing bundle. `bundle remove` takes the label, not a path that is opened. A label that is not in the bundle is logged and the command exits 0. There is no `bundle list` subcommand. `gatecheck list` prints label, SHA-256 digest, tags, and size. +The stored label is the base name of the target file (`reports/grype-report.json` is stored as `grype-report.json`). Later validation uses the file contents to pick the type, and the label when the contents are ambiguous. Repeat `-t` / `--tag` for more than one tag. When the flag is omitted, `GATECHECK_BUNDLE_TAG` is read as a comma-separated list. `bundle add` requires an existing bundle. `bundle remove` takes the label, not a path that is opened. A label that is not in the bundle is logged and the command exits 0. There is no `bundle list` subcommand. `gatecheck list` prints label, SHA-256 digest, tags, and size. ## download @@ -208,11 +246,11 @@ Create the config with `gatecheck config init > gatecheck.yaml` and set real thr ## Common mistakes -- The filename has no lowercase type word (`scan.json`, `Grype.json`, `project.gcb`), so the type cannot be detected. Use names such as `grype-report.json`, and a bundle name that contains `bundle`. +- The contents do not identify a type and the filename has no lowercase type word (`scan.json`, `Grype.json`). A real bundle named `project.gcb` is still recognized from its contents. An empty document such as `{"matches":[]}` still needs a name such as `grype-report.json`, or `--input-type grype` on standard input. - Coverage thresholds are ratios. Set `coverage.lineThreshold` to `0.8` for 80 percent, not `80` or `80.0`. - `validate` takes the report as its positional argument. The config is `--config` / `-f`. `gatecheck validate gatecheck.yaml grype-report.json` tries to open the config as a report. `config convert` uses `--file` / `-f`, which is a different flag. -- `list` on a pipe needs `--input-type`. `validate` needs a filesystem path. +- `list` on a pipe needs `--input-type`. Clear contents still win over that flag. `validate` needs a filesystem path. - `list` ignores every filename after the first. Use `list-all` for several files. `list-all` skips coverage files such as `lcov.info`. List those with `gatecheck list lcov.info`. -- A Syft filename is recognized and then returns an error. Use CycloneDX output from Syft when you want `validate` to score it. +- Native Syft JSON is recognized and then returns an error. Use CycloneDX output from Syft when you want `validate` to score it. That BOM is CycloneDX even when the filename contains `syft`. - `gatecheck semgrep --version` reaches osemgrep without `--experimental`, and osemgrep then tries to run `pysemgrep`. Pass the flag: `gatecheck semgrep --experimental --version`. - `--audit` still exits 255 when the report, the config, or a named EPSS or KEV file cannot be opened. diff --git a/docs/bootstrap.md b/docs/bootstrap.md index 5d10074..5a4d5e1 100644 --- a/docs/bootstrap.md +++ b/docs/bootstrap.md @@ -129,10 +129,10 @@ The name in that line is the command you typed. Coverage files (LCOV, Clover, Cobertura) come from the language toolchain. Bootstrap does not install those tools. -CycloneDX is a report format. Syft writes it with `-o cyclonedx-json`, and Grype can write a CycloneDX vulnerability report. Those are the files `list` and `validate` score. There is no separate CycloneDX scanner. +CycloneDX is a report format. Syft writes it with `-o cyclonedx-json`, and Grype can write a CycloneDX vulnerability report. Those are the files `list` and `validate` score. Any other tool's CycloneDX JSON BOM is scored the same way. There is no separate CycloneDX scanner. -Syft's own JSON is still rejected. A filename containing `syft` is recognized, and `list` and `validate` then return an error. Syft is installed so it can write CycloneDX. +Syft's own JSON is still rejected. `list` and `validate` return an error for that document. A filename that contains `syft` does the same when the contents are ambiguous. Syft is installed so it can write CycloneDX. -Trivy JSON (`trivy fs --format json`, `trivy image --format json`, and the other Trivy scanners' JSON output) is a report type. A filename containing `trivy` is scored. `cyclonedx` is checked first, so `cyclonedx-trivy-sbom.json` stays CycloneDX. +Trivy JSON (`trivy fs --format json`, `trivy image --format json`, and the other Trivy scanners' JSON output) is a report type. The contents identify it. A filename containing `trivy` is the fallback when the contents are ambiguous. A CycloneDX BOM stays CycloneDX even when the name contains `trivy`. Misconfigurations, secrets, and licenses are ignored. Trivy SARIF is a SARIF report. -osv-scanner JSON (`osv-scanner scan source --format json`) is a report type. A filename containing `osv` is scored, so `osv-scanner-report.json` works. Each vulnerability group is one finding. Aliases of that group are not counted again. `max_severity` is a CVSS base score. License-only packages are skipped. +osv-scanner JSON (`osv-scanner scan source --format json`) is a report type. The contents identify it when `results` holds objects with a `packages` array. A filename containing `osv` is the fallback, so `osv-scanner-report.json` still works when the document is only `{"results":[]}`. Each vulnerability group is one finding. Aliases of that group are not counted again. `max_severity` is a CVSS base score. License-only packages are skipped. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 3e2a130..13f4cb2 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -67,11 +67,13 @@ gatecheck list [FILE] Aliases: `ls`, `print`. -Prints a table for one report. The first filename is the one that is opened. Further filenames are ignored. Use `list-all` for more than one file. A Grype, CycloneDX, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `--markdown` prints that same sentence. +Prints a table for one report. The first filename is the one that is opened. Further filenames are ignored. Use `list-all` for more than one file. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `--markdown` prints that same sentence. + +The report type comes from the file contents. See [Supported Reports](./supported-reports.md). With no filename, the command reads standard input and `--input-type` is required. -- `-i`, `--input-type`: type word used when reading standard input. It is matched with the same rules as a filename. `grype`, `cyclonedx`, `semgrep`, `gitleaks`, `syft`, `bundle`, `trivy`, `osv`, and the coverage words all work. See [Supported Reports](./supported-reports.md). +- `-i`, `--input-type`: type word used when reading standard input and the contents are ambiguous. It is matched with the same rules as a filename. `grype`, `cyclonedx`, `sarif`, `semgrep`, `gitleaks`, `trivy`, `osv`, `syft`, `bundle`, and the coverage words all work. Clear contents win over this flag. See [Supported Reports](./supported-reports.md). - `--markdown`: print a markdown table. - `--epss`: add EPSS score and percentile columns. This changes the table for Grype, CycloneDX, Trivy, and osv-scanner. Other report types still list, and the EPSS data is not shown. With no local file, gatecheck downloads the current EPSS CSV and caches it until the published date changes. Requests send a User-Agent with the gatecheck version and project URL. - `--epss-filename`: read that CSV instead of calling the API. `GATECHECK_EPSS_FILENAME` is the same setting. @@ -91,10 +93,10 @@ grype bkimminich/juice-shop:latest -o json | gatecheck ls --input-type grype gatecheck list-all [FILE...] ``` -Prints each filename, then its table, or one sentence when that report has no findings. A file whose name is not a supported `list-all` type is skipped with a warning. A missing file is skipped. A list error is logged and the command continues. The command exits 0 after the last file. +Prints each filename, then its table, or one sentence when that report has no findings. A file whose contents and name are not a supported `list-all` type is skipped with a warning. Coverage files are in that skipped set. A missing file is skipped. A list error is logged and the command continues. The command exits 0 after the last file. - `--markdown`: print markdown tables. -- `--epss`: download EPSS data and add it for filenames that contain `grype`, `cyclonedx`, `trivy`, or `osv`. The download happens per file and is cached until the published date changes. There is no `--epss-filename` flag. `GATECHECK_EPSS_URL` overrides the API. `GATECHECK_EPSS_FILENAME` is not read by this command. Requests send a User-Agent with the gatecheck version and project URL. +- `--epss`: download EPSS data and add it for Grype, CycloneDX, Trivy, and osv-scanner reports. The type comes from the file contents, then the filename. The download happens per file and is cached until the published date changes. There is no `--epss-filename` flag. `GATECHECK_EPSS_URL` overrides the API. `GATECHECK_EPSS_FILENAME` is not read by this command. Requests send a User-Agent with the gatecheck version and project URL. - `--epss-cache-dir`: cache directory for that download. `GATECHECK_EPSS_CACHE_DIR` is the same setting. - `--epss-no-cache`: download on every run. `GATECHECK_EPSS_CACHE=off` does the same. diff --git a/docs/configuration.md b/docs/configuration.md index 56e0380..68d2f93 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -154,6 +154,26 @@ semgrep: low: false gitleaks: limitEnabled: false +sarif: + severityLimit: + critical: + enabled: false + limit: 0 + high: + enabled: false + limit: 0 + medium: + enabled: false + limit: 0 + low: + enabled: false + limit: 0 + ruleLimit: + enabled: false + rules: [] + ruleRiskAcceptance: + enabled: false + rules: [] coverage: lineThreshold: 0 functionThreshold: 0 @@ -206,6 +226,49 @@ grype: `limitEnabled: true` fails validation when the report contains any finding. `false` skips the check. There is no count. +## SARIF + +`sarif` is for any SARIF 2.1.0 report. Semgrep's own JSON still uses `semgrep`. `semgrep --sarif` uses `sarif`. + +`severityLimit` has `critical`, `high`, `medium`, and `low`. Each has `enabled` and `limit`, with the same count rule as Grype. + +The severity comes from the SARIF property `security-severity` when a result or its rule has one. The value is a number from 0 to 10, or a numeric string. The ranges are the ones GitHub code scanning uses: + +- 9.0 and above: critical +- 7.0 to 8.9: high +- 4.0 to 6.9: medium +- above 0 and below 4.0: low +- 0: none, and the severity limit does not count it + +A result property wins over the same property on the rule. When neither is set, gatecheck uses the SARIF level. A missing result level uses the rule's `defaultConfiguration.level`, then `warning`. + +- `error`: high +- `warning`: medium +- `note`: low +- `none`: the result is ignored + +`ruleLimit` is the deny list. `ruleRiskAcceptance` is the allow list. Each has `enabled` and `rules`. A rule entry is `id` plus an optional `metadata.tags` list. `id` is compared without regard to case. + +```yaml +sarif: + ruleLimit: + enabled: true + rules: + - id: CKV_AWS_21 + metadata: + tags: + - denied + ruleRiskAcceptance: + enabled: true + rules: + - id: B101 + metadata: + tags: + - accepted +``` + +See [Validation](./validation.md) for which results are counted and the order of the rules. + ## Coverage `lineThreshold`, `functionThreshold`, and `branchThreshold` are ratios from 0 to 1: diff --git a/docs/examples.md b/docs/examples.md index 17b6a58..d66eb0c 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -18,7 +18,7 @@ gatecheck list --epss --epss-filename epss.csv grype-report.json ## Bundle -The bundle name contains `bundle`. Each report name contains its type, and that base name becomes the label inside the bundle. +The bundle name contains `bundle`. Each report's contents identify its type. The base name becomes the label inside the bundle and is used only when the contents are ambiguous. ```shell gatecheck bundle create gatecheck-bundle.tar.gz grype-report.json --tag ci diff --git a/docs/gatecheck-bundle.md b/docs/gatecheck-bundle.md index 4645ecc..2b51a82 100644 --- a/docs/gatecheck-bundle.md +++ b/docs/gatecheck-bundle.md @@ -1,6 +1,6 @@ # Gatecheck Bundle -A bundle is a gzipped tar with a JSON manifest. It holds the report bytes, a SHA-256 digest, and any tags you set. The suggested name is `gatecheck-bundle.tar.gz`. `validate` treats the file as a bundle when the name contains `bundle`. `list` does that too, and also when the name contains `gatecheck`. `project.gcb` is rejected by both commands. +A bundle is a gzipped tar with a JSON manifest. It holds the report bytes, a SHA-256 digest, and any tags you set. The suggested name is `gatecheck-bundle.tar.gz`. `list` and `validate` treat a file as a bundle when the gzipped tar contains `gatecheck-manifest.json`. When the contents are ambiguous, `validate` still requires `bundle` in the name, and `list` also accepts `gatecheck`. ## Create @@ -8,7 +8,7 @@ A bundle is a gzipped tar with a JSON manifest. It holds the report bytes, a SHA gatecheck bundle create gatecheck-bundle.tar.gz grype-report.json --tag ci ``` -`bundle init` is the same command. The label stored in the manifest is the base name of the target file. That label is what `validate` uses later to pick the report type, so the file needs a word such as `grype`, `semgrep`, `trivy`, or `osv` in its name. +`bundle init` is the same command. The label stored in the manifest is the base name of the target file. `validate` reads each stored file's contents to pick the type, and uses the label only when the contents are ambiguous. A CycloneDX, Trivy, or osv-scanner file is still validated as that type when the stored name would suggest something else. Repeat `--tag` for more than one tag. When the flag is omitted, `GATECHECK_BUNDLE_TAG` is read as a comma-separated list. @@ -45,4 +45,4 @@ The table columns are label, digest, tags, and size. gatecheck validate --config gatecheck.yaml gatecheck-bundle.tar.gz ``` -Each label is checked on its own with the same rules as a standalone file. Labels that do not match a supported type are skipped, including Syft. A Grype failure and a Semgrep failure in the same bundle are both reported, and the command exits 1. +Each stored file is checked on its own with the same rules as a standalone file. Files that do not match a supported type are skipped, including Syft JSON. A Grype failure and a Semgrep failure in the same bundle are both reported, and the command exits 1. diff --git a/docs/library-usage.md b/docs/library-usage.md index 6c054b3..f516491 100644 --- a/docs/library-usage.md +++ b/docs/library-usage.md @@ -9,9 +9,9 @@ import "github.com/gatecheckdev/gatecheck/pkg/gatecheck" - `NewDefaultConfig` returns the same defaults as `gatecheck config init`: rules disabled, coverage thresholds `0`. - `NewConfigDecoder` reads `.yaml`, `.yml`, `.json`, or `.toml`. The extension selects the format. - `NewConfigEncoder` writes a config. Pass `.yaml`, `.yml`, `.json`, or `.toml`, including the dot. -- `Validate` takes the config, a reader, and the filename. The filename selects the report type with the same words as the CLI. -- `List` writes a table. A Grype, CycloneDX, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings writes one sentence instead, such as `No gitleaks findings` or `No osv-scanner vulnerabilities`. `WithDisplayFormat("markdown")` matches `--markdown` and writes that same sentence when the report is empty. `WithEPSS` matches `--epss` and takes an open CSV file or a URL. +- `Validate` takes the config, a reader, and the filename. The bytes select the report type. The filename is the fallback, with the same words as the CLI. +- `List` writes a table. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings writes one sentence instead, such as `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `WithDisplayFormat("markdown")` matches `--markdown` and writes that same sentence when the report is empty. `WithEPSS` matches `--epss` and takes an open CSV file or a URL. - `CreateBundle`, `AppendToBundle`, and `RemoveFromBundle` are `bundle create`, `bundle add`, and `bundle remove`. - `DownloadEPSS` and `DownloadKEV` write the API payloads. `WithEPSSURL`, `WithKEVURL`, `WithEPSSFile`, and `WithKEVFile` override the defaults. -Report types and the filename words are described in [Supported Reports](./supported-reports.md). Config fields are described in [Configuration](./configuration.md). +Report types are described in [Supported Reports](./supported-reports.md). Config fields are described in [Configuration](./configuration.md). diff --git a/docs/list-reports.md b/docs/list-reports.md index ddaf4f3..76166c5 100644 --- a/docs/list-reports.md +++ b/docs/list-reports.md @@ -20,7 +20,7 @@ gatecheck ls grype-scan-report.json ![Screenshot Example List](assets/screenshot-grype-list.png) -The filename (or `--input-type`) has to contain the lowercase report word. `grype-scan-report.json` works. See [Supported Reports](./supported-reports.md). +The report type comes from the file contents. The filename, or `--input-type` on standard input, is used when the contents are ambiguous. `grype-scan-report.json` still works. See [Supported Reports](./supported-reports.md). `--markdown` prints a markdown table instead of the plain text table. @@ -37,14 +37,15 @@ Columns: - CycloneDX: CVE id, severity, package, link. With `--epss`: CVE id, severity, EPSS score, EPSS percentile, affected packages, link. - Semgrep: check id, OWASP ids, severity, impact, link. Runtime errors from the report are logged as warnings. - Gitleaks: rule id, file, commit, start line. +- SARIF: rule id, severity, location, tool. A result that passed, is suppressed, or has level `none` is left out. - Trivy: severity, package, version, link. With `--epss`: CVE id, severity, EPSS score, EPSS percentile, package, version, link. Misconfigurations are omitted. - osv-scanner: severity, package, version, link. With `--epss`: id, severity, EPSS score, EPSS percentile, package, version, link. The id is the first CVE alias, or the OSV id when there is no CVE. Groups are listed once. - Coverage: lines, functions, and branches covered, with a percent in the footer. The percent is only for display. The config file uses a ratio. See [Configuration](./configuration.md). - Bundle: label, SHA-256 digest, tags, size. -A Grype, CycloneDX, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table. The lines are `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, and `No osv-scanner vulnerabilities`. `--markdown` prints that same line. +A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table. The lines are `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, and `No osv-scanner vulnerabilities`. `--markdown` prints that same line. -`list` opens the first filename only. `list-all` walks every filename, prints the name, then the table or that sentence, and skips files it does not support. Coverage files are in that skipped set. `--epss` on `list-all` always uses the API (`GATECHECK_EPSS_URL` to override it). +`list` opens the first filename only. `list-all` walks every filename, prints the name, then the table or that sentence, and skips files it does not support. Coverage files are in that skipped set. `--epss` on `list-all` adds EPSS columns for Grype, CycloneDX, Trivy, and osv-scanner, including a file whose name does not contain those words when the contents do. It always uses the API (`GATECHECK_EPSS_URL` to override it). ```shell gatecheck list-all grype-scan-report.json semgrep-report.json diff --git a/docs/supported-reports.md b/docs/supported-reports.md index 6fbd5ff..ca44856 100644 --- a/docs/supported-reports.md +++ b/docs/supported-reports.md @@ -1,49 +1,70 @@ # Supported Reports -`list` and `validate` pick a report type by looking for a lowercase word in the filename. The first match in this list wins: +`list` and `validate` pick a report type from the file contents. The filename is used only when the contents do not identify a type. -1. `grype`: Anchore Grype JSON. -2. `cyclonedx`: CycloneDX JSON that includes vulnerabilities. -3. `semgrep`: Semgrep JSON. -4. `gitleaks`: Gitleaks JSON (a list of findings). -5. `trivy`: Trivy JSON (`--format json`). Vulnerability entries are listed and validated. Misconfigurations, secrets, and licenses in the same file are ignored. -6. `osv`: osv-scanner JSON (`osv-scanner scan source --format json`). One finding per vulnerability group. Aliases of that group are not counted again. License-only packages are skipped. -7. `syft`: the name is recognized. `list` and `validate` both return an error. Syft is not implemented. -8. `bundle`: a Gatecheck bundle. `list` also accepts a name containing `gatecheck`. `validate` requires the word `bundle`. `project.gcb` matches neither rule. -9. Coverage, when the name contains `lcov`, `clover`, `cobertura`, or `coverage`, or when it ends in `.info`. +Content rules, in this order: -The match is case sensitive. `Grype-report.json`, `CycloneDX.json`, and `OSV-scanner.json` are rejected. +1. A gzipped tar that contains `gatecheck-manifest.json` is a Gatecheck bundle. +2. `bomFormat` of `CycloneDX` is CycloneDX JSON. This wins when the name contains `grype`, `syft`, `trivy`, or `osv`, and it wins when the tool that wrote the BOM was Grype, Syft, Trivy, or another tool. +3. SARIF 2.1.0. The top-level `version` is the string `2.1.0` and `runs` is an array. A `$schema` that mentions SARIF is enough when `runs` is missing or null. +4. Grype JSON. `descriptor.name` is `grype`, or `matches` is a non-empty array of objects with `vulnerability` and `artifact`. +5. Syft JSON. `descriptor.name` is `syft`, or `schema.url` points at the Syft schema. `list` and `validate` then return an error. Syft JSON is not implemented. A Syft CycloneDX BOM is CycloneDX, not Syft. +6. Semgrep JSON. `results` is an array of objects with `check_id`, or `results` and `errors` are both arrays. Semgrep's SARIF output is SARIF, not this type. +7. Gitleaks JSON. A non-empty array of objects with `RuleID` and one of `File`, `Commit`, or `Fingerprint`. +8. Trivy JSON (`--format json`). `SchemaVersion` is a number, and `ArtifactType` is set or `Results` is an array. Vulnerability entries are listed and validated. Misconfigurations, secrets, and licenses in the same file are ignored. +9. osv-scanner JSON (`osv-scanner scan source --format json`). `results` is a non-empty array of objects with a `packages` array. One finding per vulnerability group. Aliases of that group are not counted again. License-only packages are skipped. Semgrep is checked first, so a `check_id` stays Semgrep. +10. LCOV text. A line starts with `SF:` and the file also has `DA:`, `LF:`, or `end_of_record`. +11. Clover XML. An XML document with `clover=` or a ``. +From standard input, `list` still requires `--input-type`, so the command does not wait on a terminal. The value is matched as if it were a filename called `stdin:`, and only when the contents are ambiguous. Clear contents win over `--input-type`. ```shell grype bkimminich/juice-shop:latest -o json | gatecheck list --input-type grype ``` -`list-all` is stricter. It lists a file only when the name contains `grype`, `semgrep`, `gitleaks`, `syft`, `cyclonedx`, `bundle`, `gatecheck`, `trivy`, or `osv`. Coverage files are skipped with a warning. There is no `--input-type` flag on `list-all`. - -`cyclonedx` is checked before `trivy`, so `cyclonedx-trivy-sbom.json` is CycloneDX. +`list-all` lists a file when the contents or the name identify Grype, Semgrep, Gitleaks, Syft, CycloneDX, SARIF, Trivy, osv-scanner, or a bundle. Coverage files are skipped with a warning. There is no `--input-type` flag on `list-all`. -Inside a bundle, `validate` uses the stored label, which is the base name of the file you added. A label has to contain the same words. Labels that do not match are skipped. Syft labels inside a bundle are skipped. +Inside a bundle, `validate` uses the same content rules on each stored file, and falls back to the label. The label is the base name of the file you added. Labels that still do not match are skipped. Syft JSON inside a bundle is skipped. What each command can do: | Report | list | validate | bundle | | --- | --- | --- | --- | | Grype | yes | yes | yes | -| CycloneDX | yes | yes | yes | -| Semgrep | yes | yes | yes | +| CycloneDX JSON | yes | yes | yes | +| SARIF 2.1.0 | yes | yes | yes | +| Semgrep JSON | yes | yes | yes | | Gitleaks | yes | yes | yes | | Trivy | yes | yes | yes | | osv-scanner | yes | yes | yes | | LCOV, Clover, Cobertura | yes | yes | yes | -| Syft | error | error | the file is stored, and validation skips that label | -| Bundle | when the name contains `bundle` or `gatecheck` | when the name contains `bundle` | yes, as an ordinary file | +| Syft JSON | error | error | the file is stored, and validation skips that label | +| Bundle | when the contents are a bundle, or the name contains `bundle` or `gatecheck` | when the contents are a bundle, or the name contains `bundle` | yes, as an ordinary file | diff --git a/docs/usage.md b/docs/usage.md index b993677..3295c1b 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -17,7 +17,7 @@ Commands: - `gatecheck agent-guide` (alias `llm`) prints a markdown guide for coding agents. - `gatecheck completion` prints a shell completion script. `gatecheck help [command]` prints help for that command. -`list` and `validate` decide the report type from the filename. The words are lowercase. See [Supported Reports](./supported-reports.md). +`list` and `validate` decide the report type from the file contents. The filename is the fallback when the contents are ambiguous. See [Supported Reports](./supported-reports.md). Logs go to standard error. Tables, downloaded data, and `config init` go to standard output. diff --git a/docs/validation.md b/docs/validation.md index 9c5506d..bbd29b1 100644 --- a/docs/validation.md +++ b/docs/validation.md @@ -7,9 +7,9 @@ gatecheck validate --config gatecheck.yaml gatecheck-bundle.tar.gz Pass the config with `--config` or `-f`, or with `GATECHECK_CONFIG_FILENAME`. The only positional argument is the report path, and it is required. -The report type comes from the filename. See [Supported Reports](./supported-reports.md). +The report type comes from the file contents, and from the filename when the contents are ambiguous. See [Supported Reports](./supported-reports.md). -With no config, gatecheck uses the default: every rule disabled, coverage thresholds `0`. Grype, CycloneDX, Trivy, osv-scanner, Semgrep, Gitleaks, coverage, and bundle reports pass under that default. A Syft filename still returns an error. +With no config, gatecheck uses the default: every rule disabled, coverage thresholds `0`. Grype, CycloneDX, Trivy, osv-scanner, SARIF, Semgrep, Gitleaks, coverage, and bundle reports pass under that default. Syft JSON still returns an error. An old config with no `sarif` section leaves those limits disabled. ## Exit status @@ -17,7 +17,7 @@ With no config, gatecheck uses the default: every rule disabled, coverage thresh - `1`: a threshold failed. The error text starts with `validation failure:`. - `255`: the report or config could not be opened, the config extension is not supported, or the command line was rejected. -`--audit` (`-a`) prints whatever error the check returns and exits 0. That covers a failed threshold, an unknown filename, a Syft file, a decode error, and a failed EPSS or KEV fetch. Opening the report, the config, and any `--epss-filename` or `--kev-filename` happens before the check. A missing file still exits 255. +`--audit` (`-a`) prints whatever error the check returns and exits 0. That covers a failed threshold, an unknown report, a Syft file, a decode error, and a failed EPSS or KEV fetch. Opening the report, the config, and any `--epss-filename` or `--kev-filename` happens before the check. A missing file still exits 255. ## Grype, CycloneDX, Trivy, and osv-scanner @@ -33,7 +33,9 @@ Rules run in this order. A failure returns immediately, except where a rule only 4. **KEV limit.** When `kevLimitEnabled` is true, a CVE that is in the catalog fails the report. The catalog is `--kev-filename` or, with no file, the CISA feed. `GATECHECK_KEV_URL` overrides the feed. There is no `--kev-url` flag on `validate`. 5. **EPSS risk acceptance.** A CVE is removed when the configured `score` is greater than that CVE's EPSS score. A CVE with no EPSS row is kept. A CVE whose score is equal to the configured score is kept. 6. **EPSS limit.** A CVE fails when its EPSS score is greater than `score`. A CVE with no EPSS row passes this rule. A score equal to the limit passes. Data comes from `--epss-filename` or `GATECHECK_EPSS_FILENAME`, or from the API. `GATECHECK_EPSS_URL` overrides the API. There is no `--epss-url` flag on `validate`. -7. **Severity limit.** For each enabled severity, the report fails when the count is greater than `limit`. `limit: 0` allows none. Counts that are equal to the limit pass. Grype and CycloneDX severities are `critical`, `high`, `medium`, and `low`. `negligible` and `unknown` have no limit field. +7. **Severity limit.** For each enabled severity, the report fails when the count is greater than `limit`. `limit: 0` allows none. Counts that are equal to the limit pass. Grype, CycloneDX, Trivy, and osv-scanner severities are `critical`, `high`, `medium`, and `low`. `negligible` and `unknown` have no limit field. + +Any CycloneDX JSON BOM uses the `cyclonedx` rules, including a BOM from Grype, Trivy, Syft, or another tool. The severity is the highest rating. A rating with a score and no severity word uses CVSS ranges. A CVSS v2 method does not produce critical. `important` counts as high, and `moderate` counts as medium. `negligible` counts as none. Empty ratings use a top-level `severity` string when the producer set one, otherwise `unknown`. `unknown`, `none`, and `info` are not counted by the critical, high, medium, or low limits. The id is the vulnerability `id`, or the `bom-ref` when `id` is empty. EPSS scores in the config are ratios from 0 to 1. @@ -45,6 +47,19 @@ When a rule needs EPSS scores and `--epss-filename` is not set, gatecheck downlo 2. **Impact risk acceptance.** When enabled, a finding is removed when its impact is `high`, `medium`, or `low` and that level's boolean is true. The three booleans are independent. 3. **Severity limit.** Severities are `error`, `warning`, and `info`. The report fails when an enabled severity has more findings than its `limit`. +## SARIF + +Rules run in this order. A failure returns immediately, except where a rule only removes findings. + +1. **Rule limit.** `ruleLimit`, when enabled, fails if any listed rule id is in the report. Ids are compared without regard to case. This runs before disabled severities are dropped, so a denied rule fails even when its severity limit is disabled. +2. **Disabled severities.** Findings whose severity limit is disabled are dropped. `none` has no limit field and stays in the report, but the severity rule does not count it. +3. **Rule risk acceptance.** Matching rule ids are removed and skip the remaining rules. +4. **Severity limit.** For each enabled severity, the report fails when the count is greater than `limit`. `limit: 0` allows none. Counts that are equal to the limit pass. Severities are `critical`, `high`, `medium`, and `low`. + +The severity mapping is in [Configuration](./configuration.md). These results are not counted: `kind` of `pass` or `notApplicable`, `level` of `none`, a non-empty `suppressions` array, and `baselineState` of `absent`. + +SARIF has no EPSS or KEV rules. A SARIF version other than 2.1.0 returns an error. + ## Gitleaks When `limitEnabled` is true, any finding in the report fails validation. There is no numeric limit and no separate allow list. @@ -59,4 +74,4 @@ Line, function, and branch coverage are each `covered / total`. The config thres ## Bundles -A bundle is validated when the bundle filename contains `bundle`. Each stored label is validated with the rules above. Labels that do not match are skipped. Failures are combined, and the command exits 1 if any label fails a threshold. +A bundle is validated when its contents are a Gatecheck bundle, or when the filename contains `bundle` and the contents do not identify another type. Each stored file is validated with the rules above. The file contents pick the type, and the label is the fallback. Labels that still do not match are skipped, including Syft JSON. Failures are combined, and the command exits 1 if any label fails a threshold. diff --git a/pkg/artifacts/cyclonedx.go b/pkg/artifacts/cyclonedx.go index 2a830b7..82c83ff 100644 --- a/pkg/artifacts/cyclonedx.go +++ b/pkg/artifacts/cyclonedx.go @@ -1,9 +1,10 @@ package artifacts import ( - "cmp" + "bytes" + "encoding/json" "fmt" - "slices" + "strconv" "strings" ) @@ -23,11 +24,21 @@ type CyclonedxComponent struct { type CyclonedxVulnerability struct { ID string `json:"id"` + BOMRef string `json:"bom-ref"` + Severity string `json:"severity"` Advisories []CyclonedxAdvisory `json:"advisories"` Affects []CyclondexAffectedPackage `json:"affects"` Ratings []CyclonedxRating `json:"ratings"` } +// Identity is the vulnerability id, or the bom-ref when the producer left id empty. +func (v CyclonedxVulnerability) Identity() string { + if strings.TrimSpace(v.ID) != "" { + return v.ID + } + return v.BOMRef +} + type CyclondexAffectedPackage struct { Ref string `json:"ref"` } @@ -39,6 +50,42 @@ type CyclonedxAdvisory struct { type CyclonedxRating struct { Source CyclonedxSource `json:"source"` Severity string `json:"severity"` + Score cyclonedxScore `json:"score"` + Method string `json:"method"` +} + +// cyclonedxScore accepts a JSON number or a numeric string. +// A missing score leaves Valid false so it is not treated as zero. +type cyclonedxScore struct { + Value float64 + Valid bool +} + +func (s *cyclonedxScore) UnmarshalJSON(data []byte) error { + data = bytes.TrimSpace(data) + if len(data) == 0 || bytes.Equal(data, []byte("null")) { + return nil + } + if data[0] == '"' { + var text string + if err := json.Unmarshal(data, &text); err != nil { + return nil + } + parsed, err := strconv.ParseFloat(strings.TrimSpace(text), 64) + if err != nil { + return nil + } + s.Value = parsed + s.Valid = true + return nil + } + var number float64 + if err := json.Unmarshal(data, &number); err != nil { + return nil + } + s.Value = number + s.Valid = true + return nil } type CyclonedxSource struct { @@ -57,11 +104,93 @@ func (r *CyclonedxReportMin) SelectBySeverity(severity string) []CyclonedxVulner } func (r *CyclonedxVulnerability) HighestSeverity() string { - order := map[string]int{"none": 0, "low": 1, "medium": 2, "high": 3, "critical": 4} - rating := slices.MaxFunc(r.Ratings, func(a, b CyclonedxRating) int { - return cmp.Compare(order[a.Severity], order[b.Severity]) - }) - return rating.Severity + best := "" + bestRank := -1 + consider := func(severity string) { + if severity == "" { + return + } + rank := cyclonedxSeverityRank(severity) + if rank > bestRank { + best = severity + bestRank = rank + } + } + + for _, rating := range r.Ratings { + severity := normalizeCyclonedxSeverity(rating.Severity) + if severity == "" && rating.Score.Valid { + severity = severityFromCVSS(rating.Score.Value, rating.Method) + } + consider(severity) + } + if best == "" { + consider(normalizeCyclonedxSeverity(r.Severity)) + } + if best == "" { + return "unknown" + } + return best +} + +func normalizeCyclonedxSeverity(raw string) string { + switch strings.ToLower(strings.TrimSpace(raw)) { + case "": + return "" + case "critical": + return "critical" + case "high", "important": + return "high" + case "medium", "moderate": + return "medium" + case "low": + return "low" + case "info", "informational", "note": + return "info" + case "none", "negligible": + return "none" + case "unknown", "unspecified": + return "unknown" + default: + return strings.ToLower(strings.TrimSpace(raw)) + } +} + +func cyclonedxSeverityRank(severity string) int { + switch severity { + case "critical": + return 7 + case "high": + return 6 + case "medium": + return 5 + case "low": + return 4 + case "info": + return 3 + case "none": + return 2 + case "unknown": + return 1 + default: + return 0 + } +} + +func severityFromCVSS(score float64, method string) string { + v2 := strings.Contains(strings.ToLower(method), "cvssv2") + switch { + case score <= 0: + return "none" + case score < 4: + return "low" + case score < 7: + return "medium" + case v2 || score < 9: + return "high" + default: + return "critical" + } } func (r CyclonedxReportMin) AffectedPackages(vulnerabilityIndex int) string { @@ -80,6 +209,9 @@ func (r CyclonedxReportMin) AffectedPackages(vulnerabilityIndex int) string { } } } + if len(pkgs) == 0 { + return strings.Join(refs, ", ") + } return strings.Join(pkgs, ", ") } diff --git a/pkg/artifacts/cyclonedx_test.go b/pkg/artifacts/cyclonedx_test.go new file mode 100644 index 0000000..fe8043f --- /dev/null +++ b/pkg/artifacts/cyclonedx_test.go @@ -0,0 +1,113 @@ +package artifacts + +import ( + "encoding/json" + "os" + "testing" +) + +func osRead(path string) ([]byte, error) { + return os.ReadFile(path) +} + +func TestCyclonedxHighestSeverity(t *testing.T) { + score := func(n float64) cyclonedxScore { + return cyclonedxScore{Value: n, Valid: true} + } + cases := []struct { + name string + vuln CyclonedxVulnerability + want string + }{ + { + name: "empty ratings", + vuln: CyclonedxVulnerability{}, + want: "unknown", + }, + { + name: "critical word", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Severity: "critical"}}}, + want: "critical", + }, + { + name: "capitalized", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Severity: "HIGH"}}}, + want: "high", + }, + { + name: "highest of several", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Severity: "low"}, {Severity: "medium"}, {Severity: "info"}}}, + want: "medium", + }, + { + name: "score only v3", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Score: score(9.8), Method: "CVSSv31"}}}, + want: "critical", + }, + { + name: "score only v2", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Score: score(9), Method: "CVSSv2"}}}, + want: "high", + }, + { + name: "explicit word beats score", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Severity: "low", Score: score(9.8), Method: "CVSSv31"}}}, + want: "low", + }, + { + name: "top level severity", + vuln: CyclonedxVulnerability{Severity: "Medium"}, + want: "medium", + }, + { + name: "important alias", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Severity: "important"}}}, + want: "high", + }, + { + name: "negligible", + vuln: CyclonedxVulnerability{Ratings: []CyclonedxRating{{Severity: "negligible"}}}, + want: "none", + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := tc.vuln.HighestSeverity(); got != tc.want { + t.Fatalf("got %q want %q", got, tc.want) + } + }) + } +} + +func TestCyclonedxIdentityAndPackages(t *testing.T) { + report := CyclonedxReportMin{ + Components: []CyclonedxComponent{{BOMRef: "pkg:npm/left-pad@1.3.0", Name: "left-pad", Version: "1.3.0"}}, + Vulnerabilities: []CyclonedxVulnerability{ + {ID: "CVE-1", Affects: []CyclondexAffectedPackage{{Ref: "pkg:npm/left-pad@1.3.0"}}}, + {BOMRef: "vuln-without-id", Affects: []CyclondexAffectedPackage{{Ref: "missing"}}}, + }, + } + if report.Vulnerabilities[0].Identity() != "CVE-1" { + t.Fatal(report.Vulnerabilities[0].Identity()) + } + if report.Vulnerabilities[1].Identity() != "vuln-without-id" { + t.Fatal(report.Vulnerabilities[1].Identity()) + } + if got := report.AffectedPackages(0); got != "left-pad [1.3.0]" { + t.Fatal(got) + } + if got := report.AffectedPackages(1); got != "missing" { + t.Fatal(got) + } +} + +func TestCyclonedxStringScore(t *testing.T) { + var report CyclonedxReportMin + src := []byte(`{"vulnerabilities":[{"id":"CVE-1","ratings":[{"score":"9.8","method":"CVSSv31"}]}]}`) + if err := json.Unmarshal(src, &report); err != nil { + t.Fatal(err) + } + if got := report.Vulnerabilities[0].HighestSeverity(); got != "critical" { + t.Fatalf("got %q", got) + } +} diff --git a/pkg/artifacts/detect.go b/pkg/artifacts/detect.go new file mode 100644 index 0000000..e9cd5f0 --- /dev/null +++ b/pkg/artifacts/detect.go @@ -0,0 +1,262 @@ +package artifacts + +import ( + "bytes" + "encoding/json" + "strings" +) + +// ReportKind is a report type gatecheck can list or validate. +type ReportKind string + +const ( + KindUnknown ReportKind = "" + KindGrype ReportKind = "grype" + KindCycloneDX ReportKind = "cyclonedx" + KindSemgrep ReportKind = "semgrep" + KindGitleaks ReportKind = "gitleaks" + KindSyft ReportKind = "syft" + KindTrivy ReportKind = "trivy" + KindOSV ReportKind = "osv" + KindSARIF ReportKind = "sarif" + KindBundle ReportKind = "bundle" + KindLCOV ReportKind = "lcov" + KindClover ReportKind = "clover" + KindCobertura ReportKind = "cobertura" + // KindCoverage means the filename selected coverage, and the format is not known yet. + KindCoverage ReportKind = "coverage" +) + +// IsCoverage reports whether the kind is a coverage file. +func (k ReportKind) IsCoverage() bool { + switch k { + case KindLCOV, KindClover, KindCobertura, KindCoverage: + return true + default: + return false + } +} + +// Detect identifies a report from its bytes. +// KindUnknown means the contents are ambiguous and the caller should use the filename. +func Detect(content []byte) ReportKind { + content = bytes.TrimSpace(bytes.TrimPrefix(content, []byte{0xEF, 0xBB, 0xBF})) + if len(content) == 0 { + return KindUnknown + } + // Gzip is a bundle, or it is not a report. Bundle detection lives with the caller. + if bytes.HasPrefix(content, []byte{0x1f, 0x8b}) { + return KindUnknown + } + if content[0] == '{' || content[0] == '[' { + return detectJSON(content) + } + if looksXML(content) { + return detectXMLCoverage(content) + } + if looksLCOV(content) { + return KindLCOV + } + return KindUnknown +} + +func detectJSON(content []byte) ReportKind { + if content[0] == '[' { + return detectGitleaks(content) + } + + var head sniff + if err := json.Unmarshal(content, &head); err != nil { + return KindUnknown + } + return classifyObject(head) +} + +type sniff struct { + BomFormat string `json:"bomFormat"` + SchemaURI string `json:"$schema"` + Version any `json:"version"` + Runs json.RawMessage `json:"runs"` + Matches json.RawMessage `json:"matches"` + Descriptor struct { + Name string `json:"name"` + } `json:"descriptor"` + Results json.RawMessage `json:"results"` + Errors json.RawMessage `json:"errors"` + Paths json.RawMessage `json:"paths"` + SchemaVersion any `json:"SchemaVersion"` + ArtifactType string `json:"ArtifactType"` + TrivyResults json.RawMessage `json:"Results"` + Schema struct { + URL string `json:"url"` + } `json:"schema"` +} + +func classifyObject(s sniff) ReportKind { + if strings.EqualFold(strings.TrimSpace(s.BomFormat), "CycloneDX") { + return KindCycloneDX + } + if isSarif(s) { + return KindSARIF + } + switch strings.ToLower(strings.TrimSpace(s.Descriptor.Name)) { + case "grype": + return KindGrype + case "syft": + return KindSyft + } + if strings.Contains(strings.ToLower(s.Schema.URL), "anchore/syft") { + return KindSyft + } + if isSemgrep(s) { + return KindSemgrep + } + if isGrypeMatches(s.Matches) { + return KindGrype + } + if isTrivy(s) { + return KindTrivy + } + if isOSV(s) { + return KindOSV + } + return KindUnknown +} + +func isTrivy(s sniff) bool { + if _, ok := s.SchemaVersion.(float64); !ok { + return false + } + if strings.TrimSpace(s.ArtifactType) != "" { + return true + } + return isJSONArray(s.TrivyResults) +} + +func isOSV(s sniff) bool { + if !isJSONArray(s.Results) || jsonArrayEmpty(s.Results) { + return false + } + var rows []struct { + Packages json.RawMessage `json:"packages"` + } + if err := json.Unmarshal(s.Results, &rows); err != nil || len(rows) == 0 { + return false + } + return isJSONArray(rows[0].Packages) +} + +func isSarif(s sniff) bool { + version, ok := s.Version.(string) + if !ok || version != "2.1.0" { + return false + } + if isJSONArray(s.Runs) { + return true + } + schema := strings.ToLower(s.SchemaURI) + if !strings.Contains(schema, "sarif") { + return false + } + trimmed := bytes.TrimSpace(s.Runs) + return len(trimmed) == 0 || bytes.Equal(trimmed, []byte("null")) +} + +func isSemgrep(s sniff) bool { + if !isJSONArray(s.Results) { + return false + } + if !jsonArrayEmpty(s.Results) { + var rows []struct { + CheckID string `json:"check_id"` + } + if err := json.Unmarshal(s.Results, &rows); err != nil || len(rows) == 0 { + return false + } + return rows[0].CheckID != "" + } + // An empty results array is Semgrep when the rest of the document has Semgrep's other keys. + return isJSONArray(s.Errors) +} + +func isGrypeMatches(raw json.RawMessage) bool { + if !isJSONArray(raw) || jsonArrayEmpty(raw) { + return false + } + var rows []struct { + Vulnerability json.RawMessage `json:"vulnerability"` + Artifact json.RawMessage `json:"artifact"` + } + if err := json.Unmarshal(raw, &rows); err != nil || len(rows) == 0 { + return false + } + return len(bytes.TrimSpace(rows[0].Vulnerability)) > 0 && len(bytes.TrimSpace(rows[0].Artifact)) > 0 +} + +func detectGitleaks(content []byte) ReportKind { + dec := json.NewDecoder(bytes.NewReader(content)) + tok, err := dec.Token() + if err != nil || tok != json.Delim('[') || !dec.More() { + return KindUnknown + } + var first map[string]json.RawMessage + if err := dec.Decode(&first); err != nil { + return KindUnknown + } + if _, ok := first["RuleID"]; !ok { + return KindUnknown + } + _, file := first["File"] + _, commit := first["Commit"] + _, fingerprint := first["Fingerprint"] + if file || commit || fingerprint { + return KindGitleaks + } + return KindUnknown +} + +func looksXML(content []byte) bool { + return bytes.HasPrefix(content, []byte(" 32*1024 { + scanned = scanned[:32*1024] + } + hasSF := false + hasRecord := false + for _, line := range bytes.Split(scanned, []byte("\n")) { + line = bytes.TrimSpace(line) + if bytes.HasPrefix(line, []byte("SF:")) { + hasSF = true + } + if bytes.Equal(line, []byte("end_of_record")) || bytes.HasPrefix(line, []byte("DA:")) || bytes.HasPrefix(line, []byte("LF:")) { + hasRecord = true + } + if hasSF && hasRecord { + return true + } + } + return false +} + +func isJSONArray(raw json.RawMessage) bool { + trimmed := bytes.TrimSpace(raw) + return len(trimmed) > 0 && trimmed[0] == '[' +} + +func jsonArrayEmpty(raw json.RawMessage) bool { + return bytes.Equal(bytes.TrimSpace(raw), []byte("[]")) +} diff --git a/pkg/artifacts/detect_test.go b/pkg/artifacts/detect_test.go new file mode 100644 index 0000000..893b1a0 --- /dev/null +++ b/pkg/artifacts/detect_test.go @@ -0,0 +1,174 @@ +package artifacts + +import ( + "os" + "testing" +) + +func TestDetect(t *testing.T) { + cases := []struct { + name string + src string + want ReportKind + }{ + {name: "empty", src: "", want: KindUnknown}, + {name: "ambiguous object", src: `{}`, want: KindUnknown}, + {name: "ambiguous grype matches", src: `{"matches":[]}`, want: KindUnknown}, + {name: "ambiguous semgrep results", src: `{"results":[]}`, want: KindUnknown}, + {name: "empty gitleaks", src: `[]`, want: KindUnknown}, + { + name: "grype descriptor", + src: `{"descriptor":{"name":"grype","version":"0.121.0"},"matches":[]}`, + want: KindGrype, + }, + { + name: "grype matches", + src: `{"matches":[{"vulnerability":{"id":"CVE-1","severity":"High"},"artifact":{"name":"pkg","version":"1"}}]}`, + want: KindGrype, + }, + { + name: "cyclonedx bom", + src: `{"bomFormat":"CycloneDX","specVersion":"1.5","vulnerabilities":[]}`, + want: KindCycloneDX, + }, + { + name: "cyclonedx case", + src: `{"bomFormat":"cyclonedx","specVersion":"1.6"}`, + want: KindCycloneDX, + }, + { + name: "cyclonedx written by grype", + src: `{"bomFormat":"CycloneDX","metadata":{"tools":[{"name":"grype"}]},"vulnerabilities":[{"id":"CVE-1","ratings":[{"severity":"critical"}]}]}`, + want: KindCycloneDX, + }, + { + name: "sarif", + src: `{"version":"2.1.0","runs":[]}`, + want: KindSARIF, + }, + { + name: "sarif schema null runs", + src: `{"$schema":"https://json.schemastore.org/sarif-2.1.0.json","version":"2.1.0","runs":null}`, + want: KindSARIF, + }, + { + name: "sarif 2.0", + src: `{"version":"2.0.0","runs":[]}`, + want: KindUnknown, + }, + { + name: "syft descriptor", + src: `{"descriptor":{"name":"syft"},"artifacts":[]}`, + want: KindSyft, + }, + { + name: "syft schema", + src: `{"artifacts":[],"schema":{"url":"https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-16.0.0.json"}}`, + want: KindSyft, + }, + { + name: "semgrep", + src: `{"version":"1.182.1","results":[{"check_id":"x","extra":{"severity":"ERROR"}}],"errors":[]}`, + want: KindSemgrep, + }, + { + name: "semgrep empty results", + src: `{"results":[],"errors":[]}`, + want: KindSemgrep, + }, + { + name: "gitleaks", + src: `[{"RuleID":"jwt","File":"a.ts","Commit":"abcdef0123456789","StartLine":1}]`, + want: KindGitleaks, + }, + { + name: "lcov", + src: "SF:main.go\nDA:1,1\nLF:1\nLH:1\nend_of_record\n", + want: KindLCOV, + }, + { + name: "clover", + src: ``, + want: KindClover, + }, + { + name: "cobertura", + src: ``, + want: KindCobertura, + }, + { + name: "gzip is not sniffed as text", + src: string([]byte{0x1f, 0x8b, 0x08, 'S', 'F', ':', '\n', 'D', 'A', ':'}), + want: KindUnknown, + }, + { + name: "trivy schema and type", + src: `{"SchemaVersion":2,"ArtifactType":"filesystem","Results":[]}`, + want: KindTrivy, + }, + { + name: "trivy results only", + src: `{"SchemaVersion":2,"Results":[{"Target":"alpine","Class":"os-pkgs"}]}`, + want: KindTrivy, + }, + { + name: "schema version alone is ambiguous", + src: `{"SchemaVersion":2}`, + want: KindUnknown, + }, + { + name: "osv packages", + src: `{"results":[{"source":{"path":"go.mod","type":"lockfile"},"packages":[]}]}`, + want: KindOSV, + }, + { + name: "empty results stay ambiguous", + src: `{"results":[]}`, + want: KindUnknown, + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := Detect([]byte(tc.src)) + if got != tc.want { + t.Fatalf("got %q want %q", got, tc.want) + } + }) + } +} + +func TestDetectFixtures(t *testing.T) { + cases := []struct { + file string + want ReportKind + }{ + {file: "../../test/grype-report.json", want: KindGrype}, + {file: "../../test/cyclonedx-grype-sbom.json", want: KindCycloneDX}, + {file: "../../test/cyclonedx-syft-sbom.json", want: KindCycloneDX}, + {file: "../../test/cyclonedx-trivy-sbom.json", want: KindCycloneDX}, + {file: "../../test/cyclonedx-mixed.json", want: KindCycloneDX}, + {file: "../../test/semgrep-sast-report.json", want: KindSemgrep}, + {file: "../../test/semgrep-osemgrep-report.json", want: KindSemgrep}, + {file: "../../test/gitleaks-report.json", want: KindGitleaks}, + {file: "../../test/sarif-kics.json", want: KindSARIF}, + {file: "../../test/sarif-semgrep.json", want: KindSARIF}, + {file: "../../test/sarif-checkov.json", want: KindSARIF}, + {file: "../../test/sarif-govulncheck.json", want: KindSARIF}, + {file: "../../test/sarif-bandit.json", want: KindSARIF}, + {file: "../../test/trivy-report.json", want: KindTrivy}, + {file: "../../test/osv-scanner-report.json", want: KindOSV}, + } + for _, tc := range cases { + t.Run(tc.file, func(t *testing.T) { + body, err := os.ReadFile(tc.file) + if err != nil { + t.Fatal(err) + } + got := Detect(body) + if got != tc.want { + t.Fatalf("got %q want %q", got, tc.want) + } + }) + } +} diff --git a/pkg/artifacts/sarif.go b/pkg/artifacts/sarif.go new file mode 100644 index 0000000..03f741e --- /dev/null +++ b/pkg/artifacts/sarif.go @@ -0,0 +1,272 @@ +package artifacts + +import ( + "encoding/json" + "fmt" + "strconv" + "strings" +) + +// SarifReport is the subset of SARIF 2.1.0 used for listing and validation. +type SarifReport struct { + Schema string `json:"$schema"` + Version string `json:"version"` + Runs []SarifRun `json:"runs"` +} + +// SarifRun is one tool invocation inside a SARIF report. +type SarifRun struct { + Tool SarifTool `json:"tool"` + Results []SarifResult `json:"results"` +} + +// SarifTool is the tool that produced a run. +type SarifTool struct { + Driver SarifDriver `json:"driver"` + Extensions []SarifDriver `json:"extensions"` +} + +// SarifDriver is a SARIF tool component that owns rules. +type SarifDriver struct { + Name string `json:"name"` + Rules []SarifRule `json:"rules"` +} + +// SarifRule is a SARIF reporting descriptor. +type SarifRule struct { + ID string `json:"id"` + DefaultConfiguration *SarifConfig `json:"defaultConfiguration"` + Properties map[string]any `json:"properties"` +} + +// SarifConfig is a rule's default reporting configuration. +type SarifConfig struct { + Level string `json:"level"` +} + +// SarifResult is one SARIF result. +type SarifResult struct { + RuleID string `json:"ruleId"` + RuleIndex *int `json:"ruleIndex"` + Level string `json:"level"` + Kind string `json:"kind"` + Message SarifMessage `json:"message"` + Locations []SarifLocation `json:"locations"` + Properties map[string]any `json:"properties"` + Suppressions []json.RawMessage `json:"suppressions"` + BaselineState string `json:"baselineState"` +} + +// SarifMessage is a SARIF message object. +type SarifMessage struct { + Text string `json:"text"` +} + +// SarifLocation is a SARIF location. +type SarifLocation struct { + PhysicalLocation struct { + ArtifactLocation struct { + URI string `json:"uri"` + } `json:"artifactLocation"` + Region struct { + StartLine int `json:"startLine"` + } `json:"region"` + } `json:"physicalLocation"` +} + +// SarifFinding is one result that list and validate count. +type SarifFinding struct { + RuleID string + Severity string + URI string + Line int + Tool string +} + +// Location is the file and start line, or "-" when the result has neither. +func (f SarifFinding) Location() string { + switch { + case f.URI == "" && f.Line <= 0: + return "-" + case f.Line <= 0: + return f.URI + default: + return fmt.Sprintf("%s:%d", f.URI, f.Line) + } +} + +// Findings returns the results that list and validate count. +// Suppressed results, passes, and results marked absent in a baseline are omitted. +// The report version must be 2.1.0. +func (r *SarifReport) Findings() ([]SarifFinding, error) { + if r.Version != "2.1.0" { + return nil, fmt.Errorf("unsupported SARIF version %q", r.Version) + } + + findings := make([]SarifFinding, 0) + for _, run := range r.Runs { + rules := run.rules() + tool := run.Tool.Driver.Name + for _, result := range run.Results { + if !result.counted() { + continue + } + rule := findRule(rules, result) + findings = append(findings, SarifFinding{ + RuleID: result.ruleID(rule), + Severity: result.severity(rule), + URI: result.uri(), + Line: result.line(), + Tool: tool, + }) + } + } + return findings, nil +} + +func (run SarifRun) rules() []SarifRule { + rules := make([]SarifRule, 0, len(run.Tool.Driver.Rules)) + rules = append(rules, run.Tool.Driver.Rules...) + for _, ext := range run.Tool.Extensions { + rules = append(rules, ext.Rules...) + } + return rules +} + +func findRule(rules []SarifRule, result SarifResult) *SarifRule { + if result.RuleID != "" { + for i := range rules { + if rules[i].ID == result.RuleID { + return &rules[i] + } + } + for i := range rules { + if strings.EqualFold(rules[i].ID, result.RuleID) { + return &rules[i] + } + } + } + if result.RuleIndex != nil && *result.RuleIndex >= 0 && *result.RuleIndex < len(rules) { + return &rules[*result.RuleIndex] + } + return nil +} + +func (result SarifResult) ruleID(rule *SarifRule) string { + if result.RuleID != "" { + return result.RuleID + } + if rule != nil { + return rule.ID + } + return "" +} + +func (result SarifResult) uri() string { + if len(result.Locations) == 0 { + return "" + } + return result.Locations[0].PhysicalLocation.ArtifactLocation.URI +} + +func (result SarifResult) line() int { + if len(result.Locations) == 0 { + return 0 + } + return result.Locations[0].PhysicalLocation.Region.StartLine +} + +// counted reports whether list and validate include the result. +func (result SarifResult) counted() bool { + switch strings.ToLower(result.Kind) { + case "pass", "notapplicable": + return false + } + if strings.EqualFold(result.Level, "none") { + return false + } + if len(result.Suppressions) > 0 { + return false + } + if strings.EqualFold(result.BaselineState, "absent") { + return false + } + return true +} + +// severity maps security-severity, then SARIF level, onto critical, high, medium, low, or none. +// security-severity on the result wins, then security-severity on the rule. +// A missing level uses the rule default, then "warning", which is the SARIF default. +func (result SarifResult) severity(rule *SarifRule) string { + if score, ok := securitySeverity(result.Properties); ok { + return severityFromSecurityScore(score) + } + if rule != nil { + if score, ok := securitySeverity(rule.Properties); ok { + return severityFromSecurityScore(score) + } + } + level := result.Level + if level == "" && rule != nil && rule.DefaultConfiguration != nil { + level = rule.DefaultConfiguration.Level + } + if level == "" { + level = "warning" + } + switch strings.ToLower(level) { + case "error": + return "high" + case "warning": + return "medium" + case "note": + return "low" + case "none": + return "none" + default: + return "medium" + } +} + +func securitySeverity(props map[string]any) (float64, bool) { + if len(props) == 0 { + return 0, false + } + value, ok := props["security-severity"] + if !ok || value == nil { + return 0, false + } + switch n := value.(type) { + case float64: + return n, true + case string: + parsed, err := strconv.ParseFloat(strings.TrimSpace(n), 64) + if err != nil { + return 0, false + } + return parsed, true + case json.Number: + parsed, err := n.Float64() + if err != nil { + return 0, false + } + return parsed, true + default: + return 0, false + } +} + +// severityFromSecurityScore uses the GitHub code scanning ranges for security-severity. +func severityFromSecurityScore(score float64) string { + switch { + case score >= 9: + return "critical" + case score >= 7: + return "high" + case score >= 4: + return "medium" + case score > 0: + return "low" + default: + return "none" + } +} diff --git a/pkg/artifacts/sarif_test.go b/pkg/artifacts/sarif_test.go new file mode 100644 index 0000000..9e06a18 --- /dev/null +++ b/pkg/artifacts/sarif_test.go @@ -0,0 +1,88 @@ +package artifacts + +import ( + "encoding/json" + "testing" +) + +func TestSarifFindings(t *testing.T) { + report := &SarifReport{} + if err := json.Unmarshal(mustRead(t, "../../test/sarif-kics.json"), report); err != nil { + t.Fatal(err) + } + findings, err := report.Findings() + if err != nil { + t.Fatal(err) + } + if len(findings) != 3 { + t.Fatalf("got %d findings, want 3 (pass and suppressed omitted)", len(findings)) + } + if findings[0].Severity != "critical" || findings[0].RuleID != "611ab018" || findings[0].Location() != "main.tf:12" { + t.Fatalf("first finding: %+v", findings[0]) + } + if findings[1].Severity != "medium" || findings[1].RuleID != "4a8c5d22" { + t.Fatalf("security-severity on the rule should win: %+v", findings[1]) + } + if findings[2].Severity != "low" || findings[2].Tool != "KICS" { + t.Fatalf("note level should be low: %+v", findings[2]) + } +} + +func TestSarifLevelAndVersion(t *testing.T) { + semgrep := &SarifReport{} + if err := json.Unmarshal(mustRead(t, "../../test/sarif-semgrep.json"), semgrep); err != nil { + t.Fatal(err) + } + findings, err := semgrep.Findings() + if err != nil { + t.Fatal(err) + } + if len(findings) != 2 || findings[0].Severity != "high" || findings[1].Severity != "medium" { + t.Fatalf("%+v", findings) + } + + bandit := &SarifReport{} + if err := json.Unmarshal(mustRead(t, "../../test/sarif-bandit.json"), bandit); err != nil { + t.Fatal(err) + } + findings, err = bandit.Findings() + if err != nil { + t.Fatal(err) + } + if len(findings) != 2 || findings[0].Severity != "high" || findings[1].Severity != "low" { + t.Fatalf("bandit: %+v", findings) + } + + old := &SarifReport{Version: "2.0.0", Runs: []SarifRun{}} + if _, err := old.Findings(); err == nil { + t.Fatal("expected unsupported version") + } +} + +func TestSecuritySeverityRanges(t *testing.T) { + cases := []struct { + score float64 + want string + }{ + {9, "critical"}, + {8.9, "high"}, + {7, "high"}, + {4, "medium"}, + {0.1, "low"}, + {0, "none"}, + } + for _, tc := range cases { + if got := severityFromSecurityScore(tc.score); got != tc.want { + t.Fatalf("score %v got %s want %s", tc.score, got, tc.want) + } + } +} + +func mustRead(t *testing.T, path string) []byte { + t.Helper() + body, err := osRead(path) + if err != nil { + t.Fatal(err) + } + return body +} diff --git a/pkg/gatecheck/config.go b/pkg/gatecheck/config.go index f847cd0..5062291 100644 --- a/pkg/gatecheck/config.go +++ b/pkg/gatecheck/config.go @@ -30,6 +30,7 @@ type Config struct { OSVScanner reportWithCVEs `json:"osv-scanner" toml:"osv-scanner" yaml:"osv-scanner"` Semgrep configSemgrepReport `json:"semgrep" toml:"semgrep" yaml:"semgrep"` Gitleaks configGitleaksReport `json:"gitleaks" toml:"gitleaks" yaml:"gitleaks"` + Sarif configSarifReport `json:"sarif" toml:"sarif" yaml:"sarif"` Coverage configCoverageReport `json:"coverage" toml:"coverage" yaml:"coverage"` } @@ -67,6 +68,29 @@ type configGitleaksReport struct { LimitEnabled bool `json:"limitEnabled" toml:"limitEnabled" yaml:"limitEnabled"` } +type configSarifReport struct { + SeverityLimit configServerityLimit `json:"severityLimit" toml:"severityLimit" yaml:"severityLimit"` + RuleLimit configRuleLimit `json:"ruleLimit" toml:"ruleLimit" yaml:"ruleLimit"` + RuleRiskAcceptance configRuleRiskAcceptance `json:"ruleRiskAcceptance" toml:"ruleRiskAcceptance" yaml:"ruleRiskAcceptance"` +} + +type configRuleLimit struct { + Enabled bool `json:"enabled" toml:"enabled" yaml:"enabled"` + Rules []configRule `json:"rules" toml:"rules" yaml:"rules"` +} + +type configRuleRiskAcceptance struct { + Enabled bool `json:"enabled" toml:"enabled" yaml:"enabled"` + Rules []configRule `json:"rules" toml:"rules" yaml:"rules"` +} + +type configRule struct { + ID string `json:"id" toml:"id" yaml:"id"` + Metadata struct { + Tags []string `json:"tags" toml:"tags" yaml:"tags"` + } `json:"metadata" toml:"metadata" yaml:"metadata"` +} + type configSemgrepReport struct { SeverityLimit configSemgrepSeverityLimit `json:"severityLimit" toml:"severityLimit" yaml:"severityLimit"` ImpactRiskAcceptance configSemgrepImpactRiskAcceptance `json:"impactRiskAcceptance" toml:"impactRiskAcceptance" yaml:"impactRiskAcceptance"` @@ -186,6 +210,34 @@ func NewDefaultConfig() *Config { Gitleaks: configGitleaksReport{ LimitEnabled: false, }, + Sarif: configSarifReport{ + SeverityLimit: configServerityLimit{ + Critical: configLimit{ + Enabled: false, + Limit: 0, + }, + High: configLimit{ + Enabled: false, + Limit: 0, + }, + Medium: configLimit{ + Enabled: false, + Limit: 0, + }, + Low: configLimit{ + Enabled: false, + Limit: 0, + }, + }, + RuleLimit: configRuleLimit{ + Enabled: false, + Rules: make([]configRule, 0), + }, + RuleRiskAcceptance: configRuleRiskAcceptance{ + Enabled: false, + Rules: make([]configRule, 0), + }, + }, Coverage: configCoverageReport{ LineThreshold: 0, FunctionThreshold: 0, @@ -274,6 +326,7 @@ func (d *ConfigDecoder) Decode(config *Config) error { if err != nil { return err } + defer func() { _ = f.Close() }() var decoder interface { Decode(any) error diff --git a/pkg/gatecheck/kind.go b/pkg/gatecheck/kind.go new file mode 100644 index 0000000..f45b16b --- /dev/null +++ b/pkg/gatecheck/kind.go @@ -0,0 +1,124 @@ +package gatecheck + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "io" + "strings" + + "github.com/easy-up/go-coverage" + + "github.com/gatecheckdev/gatecheck/pkg/archive" + "github.com/gatecheckdev/gatecheck/pkg/artifacts" +) + +// KindForList is the report type used by list and list-all. +func KindForList(content []byte, filename string) artifacts.ReportKind { + return resolveKind(content, filename, true) +} + +func resolveKind(content []byte, filename string, listRules bool) artifacts.ReportKind { + if isGatecheckBundle(content) { + return artifacts.KindBundle + } + if kind := artifacts.Detect(content); kind != artifacts.KindUnknown { + return kind + } + return kindFromFilename(filename, listRules) +} + +// ListAllAction reports whether list-all should print the file, and whether EPSS columns apply. +// Coverage files are skipped. list still prints them. +func ListAllAction(content []byte, filename string) (supported bool, withEPSS bool) { + kind := KindForList(content, filename) + if kind == artifacts.KindUnknown || kind.IsCoverage() { + return false, false + } + switch kind { + case artifacts.KindGrype, artifacts.KindCycloneDX, artifacts.KindTrivy, artifacts.KindOSV: + return true, true + default: + return true, false + } +} + +func kindFromFilename(name string, listRules bool) artifacts.ReportKind { + switch { + case strings.Contains(name, "grype"): + return artifacts.KindGrype + case strings.Contains(name, "cyclonedx"): + return artifacts.KindCycloneDX + case strings.Contains(name, "semgrep"): + return artifacts.KindSemgrep + case strings.Contains(name, "gitleaks"): + return artifacts.KindGitleaks + case strings.Contains(name, "trivy"): + return artifacts.KindTrivy + case strings.Contains(name, "osv"): + return artifacts.KindOSV + case strings.Contains(name, "syft"): + return artifacts.KindSyft + case strings.Contains(name, "sarif"): + return artifacts.KindSARIF + case strings.Contains(name, "bundle") || (listRules && strings.Contains(name, "gatecheck")): + return artifacts.KindBundle + case artifacts.IsCoverageReport(name): + return coverageKindFromName(name) + default: + return artifacts.KindUnknown + } +} + +func coverageKindFromName(name string) artifacts.ReportKind { + switch { + case strings.Contains(name, "lcov") || strings.HasSuffix(name, ".info"): + return artifacts.KindLCOV + case strings.Contains(name, "clover"): + return artifacts.KindClover + case strings.HasSuffix(name, ".xml"): + return artifacts.KindCobertura + default: + return artifacts.KindCoverage + } +} + +func isGatecheckBundle(content []byte) bool { + if len(content) < 2 || content[0] != 0x1f || content[1] != 0x8b { + return false + } + gzipReader, err := gzip.NewReader(bytes.NewReader(content)) + if err != nil { + return false + } + defer func() { _ = gzipReader.Close() }() + + tarReader := tar.NewReader(gzipReader) + for { + header, err := tarReader.Next() + if err != nil { + return false + } + if header.Name == archive.ManifestFilename || strings.HasSuffix(header.Name, "/"+archive.ManifestFilename) { + return true + } + } +} + +func coverageMode(kind artifacts.ReportKind, filename string) (coverage.CoverageMode, error) { + switch kind { + case artifacts.KindLCOV: + return coverage.LCOV, nil + case artifacts.KindClover: + return coverage.CLOVER, nil + case artifacts.KindCobertura: + return coverage.COBERTURA, nil + default: + return artifacts.GetCoverageMode(filename) + } +} + +// readReport buffers a report so the type can be detected and the bytes decoded. +func readReport(src io.Reader) ([]byte, error) { + return io.ReadAll(src) +} diff --git a/pkg/gatecheck/list.go b/pkg/gatecheck/list.go index 969e128..6a1d7c0 100644 --- a/pkg/gatecheck/list.go +++ b/pkg/gatecheck/list.go @@ -1,10 +1,10 @@ package gatecheck import ( + "bytes" "encoding/json" "errors" "fmt" - "github.com/easy-up/go-coverage" "io" "log/slog" "os" @@ -12,11 +12,13 @@ import ( "strconv" "strings" + "github.com/easy-up/go-coverage" + "github.com/olekukonko/tablewriter" + "github.com/gatecheckdev/gatecheck/pkg/archive" "github.com/gatecheckdev/gatecheck/pkg/artifacts" "github.com/gatecheckdev/gatecheck/pkg/epss" "github.com/gatecheckdev/gatecheck/pkg/format" - "github.com/olekukonko/tablewriter" ) // emptyReport means the report listed successfully and has no findings. @@ -70,54 +72,57 @@ func List(dst io.Writer, src io.Reader, inputFilename string, options ...ListOpt f(o) } - switch { - case strings.Contains(inputFilename, "grype"): - slog.Debug("list", "filename", inputFilename, "filetype", "grype") + content, err := readReport(src) + if err != nil { + return err + } + kind := resolveKind(content, inputFilename, true) + src = bytes.NewReader(content) + slog.Debug("list", "filename", inputFilename, "filetype", kind) + + switch kind { + case artifacts.KindGrype: if o.epssData != nil { err = listGrypeWithEPSS(table, src, o.epssData) } else { err = ListGrypeReport(table, src) } - case strings.Contains(inputFilename, "cyclonedx"): - slog.Debug("list", "filename", inputFilename, "filetype", "cyclonedx") + case artifacts.KindCycloneDX: if o.epssData != nil { err = listCyclonedxWithEPSS(table, src, o.epssData) } else { err = ListCyclonedx(table, src) } - case strings.Contains(inputFilename, "semgrep"): - slog.Debug("list", "filename", inputFilename, "filetype", "semgrep") + case artifacts.KindSemgrep: err = ListSemgrep(table, src) - case strings.Contains(inputFilename, "gitleaks"): - slog.Debug("list", "filename", inputFilename, "filetype", "gitleaks") + case artifacts.KindGitleaks: err = listGitleaks(table, src) - case strings.Contains(inputFilename, "trivy"): - slog.Debug("list", "filename", inputFilename, "filetype", "trivy") + case artifacts.KindTrivy: if o.epssData != nil { err = listTrivyWithEPSS(table, src, o.epssData) } else { err = ListTrivyReport(table, src) } - case strings.Contains(inputFilename, "osv"): - slog.Debug("list", "filename", inputFilename, "filetype", "osv-scanner") + case artifacts.KindOSV: if o.epssData != nil { err = listOSVWithEPSS(table, src, o.epssData) } else { err = ListOSVReport(table, src) } - case strings.Contains(inputFilename, "syft"): - slog.Debug("list", "filename", inputFilename, "filetype", "syft") + case artifacts.KindSyft: slog.Warn("syft decoder is not supported yet") return errors.New("syft not implemented yet") - case strings.Contains(inputFilename, "bundle") || strings.Contains(inputFilename, "gatecheck"): - slog.Debug("list", "filename", inputFilename, "filetype", "bundle") + case artifacts.KindSARIF: + err = ListSarif(table, src) + + case artifacts.KindBundle: bundle := archive.NewBundle() if err = archive.UntarGzipBundle(src, bundle); err != nil { return err @@ -125,12 +130,10 @@ func List(dst io.Writer, src io.Reader, inputFilename string, options ...ListOpt _, err = fmt.Fprintln(dst, bundle.Content()) return err - case artifacts.IsCoverageReport(inputFilename): - slog.Debug("list", "filename", inputFilename, "filetype", "coverage") - - err = listCoverage(table, inputFilename, src) + case artifacts.KindLCOV, artifacts.KindClover, artifacts.KindCobertura, artifacts.KindCoverage: + err = listCoverage(table, kind, inputFilename, src) default: - slog.Error("unsupported file type, cannot be determined from filename", "filename", inputFilename) + slog.Error("unsupported file type, cannot be determined from content or filename", "filename", inputFilename) return errors.New("failed to list artifact content") } @@ -155,8 +158,8 @@ func List(dst io.Writer, src io.Reader, inputFilename string, options ...ListOpt return nil } -func listCoverage(table *tablewriter.Table, inputFilename string, src io.Reader) error { - coverageFormat, err := artifacts.GetCoverageMode(inputFilename) +func listCoverage(table *tablewriter.Table, kind artifacts.ReportKind, inputFilename string, src io.Reader) error { + coverageFormat, err := coverageMode(kind, inputFilename) if err != nil { return err } @@ -273,7 +276,7 @@ func ListCyclonedx(table *tablewriter.Table, src io.Reader) error { return err } - catLess := format.NewCatagoricLess([]string{"critical", "high", "medium", "low", "none"}) + catLess := format.NewCatagoricLess([]string{"critical", "high", "medium", "low", "info", "none", "unknown"}) matrix := format.NewSortableMatrix(make([][]string, 0), 1, catLess) link := "-" @@ -284,7 +287,7 @@ func ListCyclonedx(table *tablewriter.Table, src io.Reader) error { link = vul.Advisories[0].URL } // get the affected vulnerability - matrix.Append([]string{vul.ID, severity, pkgs, link}) + matrix.Append([]string{vul.Identity(), severity, pkgs, link}) } if len(report.Vulnerabilities) == 0 { @@ -313,7 +316,7 @@ func listCyclonedxWithEPSS(table *tablewriter.Table, src io.Reader, epssData *ep matrix := format.NewSortableMatrix(make([][]string, 0), 1, catLess) for idx, item := range report.Vulnerabilities { - cve, ok := epssData.CVEs[item.ID] + cve, ok := epssData.CVEs[item.Identity()] score := "-" prctl := "-" if ok { @@ -325,7 +328,7 @@ func listCyclonedxWithEPSS(table *tablewriter.Table, src io.Reader, epssData *ep link = item.Advisories[0].URL } row := []string{ - item.ID, + item.Identity(), item.HighestSeverity(), score, prctl, @@ -395,6 +398,38 @@ func ListSemgrep(table *tablewriter.Table, src io.Reader) error { return nil } +func ListSarif(table *tablewriter.Table, src io.Reader) error { + report := &artifacts.SarifReport{} + if err := json.NewDecoder(src).Decode(report); err != nil { + return err + } + findings, err := report.Findings() + if err != nil { + return err + } + if len(findings) == 0 { + return emptyReport("No sarif findings") + } + + catLess := format.NewCatagoricLess([]string{"critical", "high", "medium", "low", "none", "unknown"}) + matrix := format.NewSortableMatrix(make([][]string, 0), 1, catLess) + for _, finding := range findings { + ruleID := finding.RuleID + if ruleID == "" { + ruleID = "-" + } + tool := finding.Tool + if tool == "" { + tool = "-" + } + matrix.Append([]string{ruleID, finding.Severity, finding.Location(), tool}) + } + sort.Sort(matrix) + + table.Header([]string{"SARIF Rule ID", "Severity", "Location", "Tool"}) + return matrix.Table(table) +} + func listGitleaks(table *tablewriter.Table, src io.Reader) error { report := artifacts.GitLeaksReportMin{} if err := json.NewDecoder(src).Decode(&report); err != nil { diff --git a/pkg/gatecheck/list_test.go b/pkg/gatecheck/list_test.go index ed07526..2e41ce2 100644 --- a/pkg/gatecheck/list_test.go +++ b/pkg/gatecheck/list_test.go @@ -50,6 +50,10 @@ func TestEmptyReportPrintsALine(t *testing.T) { {name: "semgrep markdown", filename: "semgrep.json", src: `{"results":[]}`, markdown: true, want: "No semgrep findings\n"}, {name: "gitleaks", filename: "gitleaks.json", src: `[]`, want: "No gitleaks findings\n"}, {name: "gitleaks markdown", filename: "gitleaks.json", src: `[]`, markdown: true, want: "No gitleaks findings\n"}, + {name: "sarif", filename: "report.json", src: `{"version":"2.1.0","runs":[]}`, want: "No sarif findings\n"}, + {name: "sarif markdown", filename: "grype.json", src: `{"version":"2.1.0","runs":[]}`, markdown: true, want: "No sarif findings\n"}, + {name: "trivy", filename: "grype.json", src: `{"SchemaVersion":2,"ArtifactType":"filesystem","Results":[]}`, want: "No trivy vulnerabilities\n"}, + {name: "osv", filename: "notes.json", src: `{"results":[{"source":{"path":"go.mod","type":"lockfile"},"packages":[]}]}`, want: "No osv-scanner vulnerabilities\n"}, } for _, tc := range cases { diff --git a/pkg/gatecheck/osv_test.go b/pkg/gatecheck/osv_test.go index 03d8d13..a7472f7 100644 --- a/pkg/gatecheck/osv_test.go +++ b/pkg/gatecheck/osv_test.go @@ -56,10 +56,10 @@ func TestOSVListAndValidate(t *testing.T) { } }) - t.Run("case sensitive filename", func(t *testing.T) { - err := List(&bytes.Buffer{}, bytes.NewReader(report), "OSV-scanner.json") - if err == nil { - t.Fatal("expected an unsupported filename") + t.Run("content wins over filename case", func(t *testing.T) { + got := renderList(t, "OSV-scanner.json", report, nil) + if !strings.Contains(got, "OSV SEVERITY") { + t.Fatalf("contents should win over filename case\n%s", got) } }) diff --git a/pkg/gatecheck/report_kind_test.go b/pkg/gatecheck/report_kind_test.go new file mode 100644 index 0000000..a6d3195 --- /dev/null +++ b/pkg/gatecheck/report_kind_test.go @@ -0,0 +1,354 @@ +package gatecheck + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "errors" + "os" + "strings" + "testing" + + "github.com/gatecheckdev/gatecheck/pkg/archive" + "github.com/gatecheckdev/gatecheck/pkg/artifacts" +) + +func TestKindFallsBackToFilename(t *testing.T) { + if got := KindForList([]byte(`{"matches":[]}`), "grype.json"); got != artifacts.KindGrype { + t.Fatalf("got %s", got) + } + if got := KindForList([]byte(`{"results":[]}`), "semgrep.json"); got != artifacts.KindSemgrep { + t.Fatalf("got %s", got) + } + if got := KindForList([]byte(`[]`), "gitleaks.json"); got != artifacts.KindGitleaks { + t.Fatalf("got %s", got) + } + if got := KindForList([]byte(`{"SchemaVersion":2}`), "trivy-report.json"); got != artifacts.KindTrivy { + t.Fatalf("got %s", got) + } + if got := KindForList([]byte(`{"results":[]}`), "osv-scanner-report.json"); got != artifacts.KindOSV { + t.Fatalf("got %s", got) + } + if got := KindForList([]byte(`{"results":[]}`), "semgrep.json"); got != artifacts.KindSemgrep { + t.Fatalf("semgrep should stay ahead of osv in the filename fallback: %s", got) + } + if got := KindForList([]byte(`not a report`), "scan.json"); got != artifacts.KindUnknown { + t.Fatalf("got %s", got) + } + if got := resolveKind([]byte(`not a report`), "gatecheck-reports.tar.gz", false); got != artifacts.KindUnknown { + t.Fatalf("validate filename fallback should ignore gatecheck without bundle: %s", got) + } + if got := KindForList([]byte(`not a report`), "gatecheck-reports.tar.gz"); got != artifacts.KindBundle { + t.Fatalf("list filename fallback should accept gatecheck: %s", got) + } +} + +func TestContentWinsOverFilename(t *testing.T) { + cdx := []byte(`{"bomFormat":"CycloneDX","specVersion":"1.5","vulnerabilities":[{"id":"CVE-2024-1111","ratings":[{"severity":"critical"}]}]}`) + got := renderList(t, "cyclonedx-grype-sbom.json", cdx, nil) + if !strings.Contains(got, "No cyclonedx vulnerabilities") && !strings.Contains(got, "CVE-2024-1111") { + t.Fatalf("%s", got) + } + if strings.Contains(got, "No grype") || strings.Contains(got, "GRYPE SEVERITY") { + t.Fatalf("listed as grype:\n%s", got) + } + + config := NewDefaultConfig() + config.Grype.SeverityLimit.Critical.Enabled = true + config.Cyclonedx.SeverityLimit.Critical.Enabled = true + config.Cyclonedx.SeverityLimit.Critical.Limit = 0 + err := Validate(config, bytes.NewReader(cdx), "cyclonedx-grype-sbom.json") + if !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "CycloneDx") { + t.Fatalf("got %v", err) + } +} + +func TestCycloneDXProducersListTheSameWay(t *testing.T) { + mixed := mustRead(t, "../../test/cyclonedx-mixed.json") + for _, filename := range []string{"cyclonedx-mixed.json", "grype-report.json", "trivy.json", "dependency-track.json"} { + t.Run(filename, func(t *testing.T) { + got := renderList(t, filename, mixed, nil) + for _, needle := range []string{"CYCLONEDX CVE ID", "CVE-2024-0001", "critical", "left-pad [1.3.0]", "requests [2.28.0]", "vuln-without-id", "missing-component"} { + if !strings.Contains(got, needle) { + t.Fatalf("missing %q\n%s", needle, got) + } + } + if strings.Contains(got, "No grype") { + t.Fatal(got) + } + }) + } + + config := NewDefaultConfig() + config.Cyclonedx.SeverityLimit.Critical.Enabled = true + config.Cyclonedx.SeverityLimit.Critical.Limit = 0 + err := Validate(config, bytes.NewReader(mixed), "scan.json") + if !errors.Is(err, ErrValidationFailure) { + t.Fatal(err) + } + + config = NewDefaultConfig() + config.Cyclonedx.SeverityLimit.Critical.Enabled = true + config.Cyclonedx.CVERiskAcceptance.Enabled = true + config.Cyclonedx.CVERiskAcceptance.CVEs = []configCVE{{ID: "cve-2024-0001"}} + config.Cyclonedx.SeverityLimit.High.Enabled = true + config.Cyclonedx.SeverityLimit.High.Limit = 0 + err = Validate(config, bytes.NewReader(mixed), "scan.json") + if !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "Severity") { + t.Fatalf("high findings should still fail after the critical is allowed: %v", err) + } +} + +func TestRepoCycloneDXReports(t *testing.T) { + grype := mustRead(t, "../../test/cyclonedx-grype-sbom.json") + got := renderList(t, "cyclonedx-grype-sbom.json", grype, nil) + if strings.Contains(got, "No grype") || strings.Contains(got, "GRYPE SEVERITY") { + t.Fatal("grype cyclonedx bom was listed as grype") + } + if !strings.Contains(got, "CVE-2016-2781") || !strings.Contains(got, "CYCLONEDX CVE ID") { + t.Fatalf("missing cyclonedx row:\n%.400s", got) + } + config := NewDefaultConfig() + config.Cyclonedx.SeverityLimit.Low.Enabled = true + config.Cyclonedx.SeverityLimit.Low.Limit = 0 + if err := Validate(config, bytes.NewReader(grype), "cyclonedx-grype-sbom.json"); !errors.Is(err, ErrValidationFailure) { + t.Fatalf("low limit: %v", err) + } + + trivy := mustRead(t, "../../test/cyclonedx-trivy-sbom.json") + got = renderList(t, "trivy-report.json", trivy, nil) + if !strings.Contains(got, "CVE-2023-29383") || !strings.Contains(got, "CYCLONEDX CVE ID") { + t.Fatalf("trivy bom was not listed as cyclonedx:\n%.400s", got) + } + config = NewDefaultConfig() + config.Cyclonedx.SeverityLimit.High.Enabled = true + if err := Validate(config, bytes.NewReader(trivy), "trivy-report.json"); !errors.Is(err, ErrValidationFailure) { + t.Fatalf("trivy high limit: %v", err) + } + + syft := mustRead(t, "../../test/cyclonedx-syft-sbom.json") + got = renderList(t, "image.syft.json", syft, nil) + if got != "No cyclonedx vulnerabilities\n" { + t.Fatalf("syft cyclonedx bom: %q", got) + } + if err := Validate(NewDefaultConfig(), bytes.NewReader(syft), "image.syft.json"); err != nil { + t.Fatal(err) + } +} + +func TestSarifListAndValidate(t *testing.T) { + kics := mustRead(t, "../../test/sarif-kics.json") + assertGolden(t, "notes.json", kics, nil, "testdata/sarif-kics.golden") + + for _, file := range []string{ + "../../test/sarif-semgrep.json", + "../../test/sarif-checkov.json", + "../../test/sarif-govulncheck.json", + "../../test/sarif-bandit.json", + } { + t.Run(file, func(t *testing.T) { + body := mustRead(t, file) + got := renderList(t, "scan.json", body, nil) + if !strings.Contains(got, "SARIF RULE ID") { + t.Fatalf("%s", got) + } + if err := Validate(NewDefaultConfig(), bytes.NewReader(body), "scan.json"); err != nil { + t.Fatal(err) + } + }) + } + + semgrep := mustRead(t, "../../test/sarif-semgrep.json") + if strings.Contains(renderList(t, "semgrep.sarif", semgrep, nil), "SEMGREP CHECK ID") { + t.Fatal("semgrep sarif was listed as semgrep json") + } + + config := NewDefaultConfig() + config.Sarif.SeverityLimit.Critical.Enabled = true + config.Sarif.SeverityLimit.Critical.Limit = 0 + err := Validate(config, bytes.NewReader(kics), "kics.sarif") + if !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "SARIF") { + t.Fatalf("%v", err) + } + + config = NewDefaultConfig() + config.Sarif.RuleLimit.Enabled = true + config.Sarif.RuleLimit.Rules = []configRule{{ID: "b602"}} + bandit := mustRead(t, "../../test/sarif-bandit.json") + err = Validate(config, bytes.NewReader(bandit), "bandit.json") + if !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "denied") { + t.Fatalf("%v", err) + } + + config = NewDefaultConfig() + config.Sarif.SeverityLimit.High.Enabled = true + config.Sarif.SeverityLimit.High.Limit = 0 + config.Sarif.RuleRiskAcceptance.Enabled = true + config.Sarif.RuleRiskAcceptance.Rules = []configRule{{ID: "B602"}} + if err := Validate(config, bytes.NewReader(bandit), "bandit.json"); err != nil { + t.Fatal(err) + } + + // A denied rule fails even when that severity limit is disabled. + config = NewDefaultConfig() + config.Sarif.RuleLimit.Enabled = true + config.Sarif.RuleLimit.Rules = []configRule{{ID: "B101"}} + err = Validate(config, bytes.NewReader(bandit), "bandit.json") + if !errors.Is(err, ErrValidationFailure) { + t.Fatal(err) + } + + old := []byte(`{"version":"2.0.0","runs":[]}`) + if err := Validate(NewDefaultConfig(), bytes.NewReader(old), "out.sarif"); err == nil { + t.Fatal("expected sarif 2.0 to fail") + } +} + +func TestBundleUsesStoredContent(t *testing.T) { + kics := mustRead(t, "../../test/sarif-kics.json") + cdx := []byte(`{"bomFormat":"CycloneDX","specVersion":"1.5","vulnerabilities":[{"id":"CVE-9","ratings":[{"severity":"high"}]}]}`) + bundle := archive.NewBundle() + bundle.Add(kics, "notes.json", []string{"ci"}) + bundle.Add(cdx, "grype.json", nil) + var buf bytes.Buffer + if _, err := archive.TarGzipBundle(&buf, bundle); err != nil { + t.Fatal(err) + } + raw := append([]byte(nil), buf.Bytes()...) + + listed := renderList(t, "scan.tar.gz", raw, nil) + if !strings.Contains(listed, "notes.json") || !strings.Contains(listed, "grype.json") { + t.Fatalf("%s", listed) + } + + config := NewDefaultConfig() + config.Sarif.SeverityLimit.Critical.Enabled = true + config.Sarif.SeverityLimit.Critical.Limit = 0 + config.Cyclonedx.SeverityLimit.High.Enabled = true + config.Cyclonedx.SeverityLimit.High.Limit = 0 + err := Validate(config, bytes.NewReader(raw), "scan.tar.gz") + if !errors.Is(err, ErrValidationFailure) { + t.Fatal(err) + } + if !strings.Contains(err.Error(), "SARIF") || !strings.Contains(err.Error(), "CycloneDx") { + t.Fatalf("%v", err) + } +} + +func TestListAllAction(t *testing.T) { + supported, epss := ListAllAction([]byte(`{"bomFormat":"CycloneDX"}`), "scan.json") + if !supported || !epss { + t.Fatalf("supported %v epss %v", supported, epss) + } + supported, epss = ListAllAction([]byte("SF:main.go\nDA:1,1\nLF:1\nLH:1\nend_of_record\n"), "out.txt") + if supported || epss { + t.Fatal("coverage should stay out of list-all") + } + lcov := []byte("SF:main.go\nDA:1,1\nLF:1\nLH:1\nend_of_record\n") + got := renderList(t, "out.txt", lcov, nil) + if !strings.Contains(got, "LINES COVERED") { + t.Fatalf("%s", got) + } +} + +func TestGatecheckBundleMagic(t *testing.T) { + bundle := archive.NewBundle() + bundle.Add([]byte(`{"matches":[]}`), "grype.json", nil) + var buf bytes.Buffer + if _, err := archive.TarGzipBundle(&buf, bundle); err != nil { + t.Fatal(err) + } + if !isGatecheckBundle(buf.Bytes()) { + t.Fatal("expected bundle") + } + plain := gzipBytes(t, []byte("hello")) + if isGatecheckBundle(plain) { + t.Fatal("plain gzip is not a bundle") + } +} + +func gzipBytes(t *testing.T, payload []byte) []byte { + t.Helper() + var buf bytes.Buffer + zw := gzip.NewWriter(&buf) + tw := tar.NewWriter(zw) + if err := tw.WriteHeader(&tar.Header{Name: "a.txt", Size: int64(len(payload)), Mode: 0o644}); err != nil { + t.Fatal(err) + } + if _, err := tw.Write(payload); err != nil { + t.Fatal(err) + } + if err := tw.Close(); err != nil { + t.Fatal(err) + } + if err := zw.Close(); err != nil { + t.Fatal(err) + } + return buf.Bytes() +} + +func TestTrivyAndOSVContentWinsOverFilename(t *testing.T) { + trivy := mustRead(t, "../../test/trivy-report.json") + got := renderList(t, "grype-report.json", trivy, nil) + if !strings.Contains(got, "TRIVY SEVERITY") || !strings.Contains(got, "libssl3") { + t.Fatalf("trivy json named grype was not listed as trivy:\n%.400s", got) + } + if strings.Contains(got, "GRYPE SEVERITY") { + t.Fatal("trivy json was listed as grype") + } + config := NewDefaultConfig() + config.Trivy.SeverityLimit.Critical.Enabled = true + config.Trivy.SeverityLimit.Critical.Limit = 0 + if err := Validate(config, bytes.NewReader(trivy), "scan.json"); !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "Trivy") { + t.Fatalf("trivy critical limit: %v", err) + } + + osv := mustRead(t, "../../test/osv-scanner-report.json") + got = renderList(t, "notes.json", osv, nil) + if !strings.Contains(got, "OSV SEVERITY") || !strings.Contains(got, "GO-2021-0053") { + t.Fatalf("osv json was not listed:\n%.400s", got) + } + config = NewDefaultConfig() + config.OSVScanner.SeverityLimit.High.Enabled = true + config.OSVScanner.SeverityLimit.High.Limit = 0 + if err := Validate(config, bytes.NewReader(osv), "scan.json"); !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "osv-scanner") { + t.Fatalf("osv high limit: %v", err) + } + + supported, epss := ListAllAction(trivy, "scan.json") + if !supported || !epss { + t.Fatalf("trivy list-all supported %v epss %v", supported, epss) + } + supported, epss = ListAllAction(osv, "scan.json") + if !supported || !epss { + t.Fatalf("osv list-all supported %v epss %v", supported, epss) + } +} + +func TestSyftJSONStillErrors(t *testing.T) { + src := []byte(`{"descriptor":{"name":"syft"},"artifacts":[]}`) + if err := List(&bytes.Buffer{}, bytes.NewReader(src), "scan.json"); err == nil || !strings.Contains(err.Error(), "syft") { + t.Fatal(err) + } + if err := Validate(NewDefaultConfig(), bytes.NewReader(src), "bom.syft.json"); err == nil || !strings.Contains(err.Error(), "syft") { + t.Fatal(err) + } +} + +func TestPartialConfigIgnoresMissingSarif(t *testing.T) { + raw := []byte("version: \"1\"\ngrype:\n severityLimit:\n critical:\n enabled: true\n limit: 0\n") + path := t.TempDir() + "/gatecheck.yaml" + if err := os.WriteFile(path, raw, 0o644); err != nil { + t.Fatal(err) + } + config := &Config{} + if err := NewConfigDecoder(path).Decode(config); err != nil { + t.Fatal(err) + } + if config.Sarif.SeverityLimit.Critical.Enabled || config.Sarif.RuleLimit.Enabled { + t.Fatal("missing sarif section should stay disabled") + } + if !config.Grype.SeverityLimit.Critical.Enabled { + t.Fatal("existing grype config was not decoded") + } +} diff --git a/pkg/gatecheck/testdata/sarif-kics.golden b/pkg/gatecheck/testdata/sarif-kics.golden new file mode 100644 index 0000000..e70353f --- /dev/null +++ b/pkg/gatecheck/testdata/sarif-kics.golden @@ -0,0 +1,7 @@ ++---------------+----------+------------+------+ +| SARIF RULE ID | SEVERITY | LOCATION | TOOL | ++---------------+----------+------------+------+ +| 611ab018 | critical | main.tf:12 | KICS | +| 4a8c5d22 | medium | main.tf:30 | KICS | +| note-rule | low | main.tf:2 | KICS | ++---------------+----------+------------+------+ diff --git a/pkg/gatecheck/trivy_test.go b/pkg/gatecheck/trivy_test.go index 9b5cccb..5b3f50b 100644 --- a/pkg/gatecheck/trivy_test.go +++ b/pkg/gatecheck/trivy_test.go @@ -52,10 +52,10 @@ func TestTrivyListAndValidate(t *testing.T) { } }) - t.Run("case sensitive filename", func(t *testing.T) { - err := List(&bytes.Buffer{}, bytes.NewReader(report), "Trivy-report.json") - if err == nil { - t.Fatal("expected an unsupported filename") + t.Run("content wins over filename case", func(t *testing.T) { + got := renderList(t, "Trivy-report.json", report, nil) + if !strings.Contains(got, "TRIVY SEVERITY") { + t.Fatalf("contents should win over filename case\n%s", got) } }) diff --git a/pkg/gatecheck/validate.go b/pkg/gatecheck/validate.go index f151a05..d944fb1 100644 --- a/pkg/gatecheck/validate.go +++ b/pkg/gatecheck/validate.go @@ -32,45 +32,47 @@ func Validate(config *Config, reportSrc io.Reader, targetFilename string, option f(options) } - switch { - case strings.Contains(targetFilename, "grype"): - slog.Debug("validate grype report", "filename", targetFilename) + content, err := readReport(reportSrc) + if err != nil { + return err + } + kind := resolveKind(content, targetFilename, false) + reportSrc = bytes.NewReader(content) + slog.Debug("validate", "filename", targetFilename, "filetype", kind) + + switch kind { + case artifacts.KindGrype: return validateGrypeReportWithFetch(reportSrc, config, options) - case strings.Contains(targetFilename, "cyclonedx"): - slog.Debug("validate", "filename", targetFilename, "filetype", "cyclonedx") + case artifacts.KindCycloneDX: return validateCyclonedxReportWithFetch(reportSrc, config, options) - case strings.Contains(targetFilename, "semgrep"): - slog.Debug("validate", "filename", targetFilename, "filetype", "semgrep") + case artifacts.KindSemgrep: return validateSemgrepReport(reportSrc, config) - case strings.Contains(targetFilename, "gitleaks"): - slog.Debug("validate", "filename", targetFilename, "filetype", "gitleaks") + case artifacts.KindGitleaks: return validateGitleaksReport(reportSrc, config) - case strings.Contains(targetFilename, "trivy"): - slog.Debug("validate", "filename", targetFilename, "filetype", "trivy") + case artifacts.KindTrivy: return validateTrivyReportWithFetch(reportSrc, config, options) - case strings.Contains(targetFilename, "osv"): - slog.Debug("validate", "filename", targetFilename, "filetype", "osv-scanner") + case artifacts.KindOSV: return validateOSVReportWithFetch(reportSrc, config, options) - case strings.Contains(targetFilename, "syft"): - slog.Debug("validate", "filename", targetFilename, "filetype", "syft") + case artifacts.KindSyft: return errors.New("syft validation not supported yet") - case strings.Contains(targetFilename, "bundle"): - slog.Debug("validate", "filename", targetFilename, "filetype", "bundle") + case artifacts.KindSARIF: + return validateSarifReport(reportSrc, config) + + case artifacts.KindBundle: return validateBundle(reportSrc, config, options) - case artifacts.IsCoverageReport(targetFilename): - slog.Debug("validate", "filename", targetFilename, "filetype", "coverage") - return validateCoverage(reportSrc, targetFilename, config) + case artifacts.KindLCOV, artifacts.KindClover, artifacts.KindCobertura, artifacts.KindCoverage: + return validateCoverage(reportSrc, kind, targetFilename, config) default: - slog.Error("unsupported file type, cannot be determined from filename", "filename", targetFilename) + slog.Error("unsupported file type, cannot be determined from content or filename", "filename", targetFilename) return errors.New("failed to validate artifact, See log for details") } } @@ -191,7 +193,7 @@ func ruleCyclonedxCVEDeny(config *Config, report *artifacts.CyclonedxReportMin) } for _, cve := range config.Cyclonedx.CVELimit.CVEs { contains := slices.ContainsFunc(report.Vulnerabilities, func(vulerability artifacts.CyclonedxVulnerability) bool { - return strings.EqualFold(vulerability.ID, cve.ID) + return strings.EqualFold(vulerability.Identity(), cve.ID) }) if contains { @@ -238,11 +240,11 @@ func ruleCyclonedxCVEAllow(config *Config, report *artifacts.CyclonedxReportMin) vulnerabilities := slices.DeleteFunc(report.Vulnerabilities, func(vulnerability artifacts.CyclonedxVulnerability) bool { allowed := slices.ContainsFunc(config.Cyclonedx.CVERiskAcceptance.CVEs, func(cve configCVE) bool { - return strings.EqualFold(cve.ID, vulnerability.ID) + return strings.EqualFold(cve.ID, vulnerability.Identity()) }) if allowed { slog.Info("CVE explicitly allowed, removing from subsequent rules", - "id", vulnerability.ID, "severity", vulnerability.HighestSeverity()) + "id", vulnerability.Identity(), "severity", vulnerability.HighestSeverity()) } return allowed }) @@ -294,13 +296,13 @@ func ruleCyclonedxKEVLimit(config *Config, report *artifacts.CyclonedxReportMin, // Check if vulnerability is in the KEV Catalog for _, vulnerability := range report.Vulnerabilities { inKEVCatalog := slices.ContainsFunc(catalog.Vulnerabilities, func(kevVul kev.Vulnerability) bool { - return strings.EqualFold(kevVul.CveID, vulnerability.ID) + return strings.EqualFold(kevVul.CveID, vulnerability.Identity()) }) if inKEVCatalog { - badCVEs = append(badCVEs, vulnerability.ID) + badCVEs = append(badCVEs, vulnerability.Identity()) slog.Warn("cve found in kev catalog", - "cve_id", vulnerability.ID) + "cve_id", vulnerability.Identity()) } } if len(badCVEs) > 0 { @@ -364,16 +366,16 @@ func ruleCyclonedxEPSSAllow(config *Config, report *artifacts.CyclonedxReportMin "epss_risk_acceptance_score", config.Cyclonedx.EPSSRiskAcceptance.Score, ) vulnerabilities := slices.DeleteFunc(report.Vulnerabilities, func(vulnerability artifacts.CyclonedxVulnerability) bool { - epssCVE, ok := data.CVEs[vulnerability.ID] + epssCVE, ok := data.CVEs[vulnerability.Identity()] if !ok { - slog.Debug("no epss score", "cve_id", vulnerability.ID, "severity", vulnerability.HighestSeverity()) + slog.Debug("no epss score", "cve_id", vulnerability.Identity(), "severity", vulnerability.HighestSeverity()) return false } riskAccepted := config.Cyclonedx.EPSSRiskAcceptance.Score > epssCVE.EPSSValue() if riskAccepted { slog.Info( "risk accepted reason: epss score", - "cve_id", vulnerability.ID, + "cve_id", vulnerability.Identity(), "severity", vulnerability.HighestSeverity(), "epss_score", epssCVE.EPSS, ) @@ -447,7 +449,7 @@ func ruleCyclonedxEPSSLimit(config *Config, report *artifacts.CyclonedxReportMin ) for _, vulnerability := range report.Vulnerabilities { - epssCVE, ok := data.CVEs[vulnerability.ID] + epssCVE, ok := data.CVEs[vulnerability.Identity()] if !ok { continue } @@ -456,7 +458,7 @@ func ruleCyclonedxEPSSLimit(config *Config, report *artifacts.CyclonedxReportMin badCVEs = append(badCVEs, epssCVE) slog.Warn( "epss score limit violation", - "cve_id", vulnerability.ID, + "cve_id", vulnerability.Identity(), "severity", vulnerability.HighestSeverity(), "epss_score", epssCVE.EPSS, ) @@ -696,8 +698,8 @@ func validateGitleaksReport(r io.Reader, config *Config) error { return validateGitleaksRules(config, report) } -func validateCoverage(src io.Reader, targetFilename string, config *Config) error { - coverageFormat, err := artifacts.GetCoverageMode(targetFilename) +func validateCoverage(src io.Reader, kind artifacts.ReportKind, targetFilename string, config *Config) error { + coverageFormat, err := coverageMode(kind, targetFilename) if err != nil { return err } @@ -760,28 +762,33 @@ func validateBundle(r io.Reader, config *Config, options *fetchOptions) error { var errs error for fileLabel, descriptor := range bundle.Manifest().Files { - slog.Info("gatecheck bundle validation", "file_label", fileLabel, "digest", descriptor.Digest) - switch { - case strings.Contains(fileLabel, "grype"): - err := validateGrypeFrom(bytes.NewBuffer(bundle.FileBytes(fileLabel)), config, catalog, epssData) + content := bundle.FileBytes(fileLabel) + kind := resolveKind(content, fileLabel, false) + slog.Info("gatecheck bundle validation", "file_label", fileLabel, "digest", descriptor.Digest, "filetype", kind) + switch kind { + case artifacts.KindGrype: + err := validateGrypeFrom(bytes.NewBuffer(content), config, catalog, epssData) errs = errors.Join(errs, err) - case strings.Contains(fileLabel, "cyclonedx"): - err := validateCyclonedxFrom(bytes.NewBuffer(bundle.FileBytes(fileLabel)), config, catalog, epssData) + case artifacts.KindCycloneDX: + err := validateCyclonedxFrom(bytes.NewBuffer(content), config, catalog, epssData) errs = errors.Join(errs, err) - case strings.Contains(fileLabel, "semgrep"): - err := validateSemgrepReport(bytes.NewBuffer(bundle.FileBytes(fileLabel)), config) + case artifacts.KindSemgrep: + err := validateSemgrepReport(bytes.NewBuffer(content), config) errs = errors.Join(errs, err) - case strings.Contains(fileLabel, "gitleaks"): - err := validateGitleaksReport(bytes.NewBuffer(bundle.FileBytes(fileLabel)), config) + case artifacts.KindGitleaks: + err := validateGitleaksReport(bytes.NewBuffer(content), config) errs = errors.Join(errs, err) - case strings.Contains(fileLabel, "trivy"): - err := validateTrivyFrom(bytes.NewBuffer(bundle.FileBytes(fileLabel)), config, catalog, epssData) + case artifacts.KindTrivy: + err := validateTrivyFrom(bytes.NewBuffer(content), config, catalog, epssData) errs = errors.Join(errs, err) - case strings.Contains(fileLabel, "osv"): - err := validateOSVFrom(bytes.NewBuffer(bundle.FileBytes(fileLabel)), config, catalog, epssData) + case artifacts.KindOSV: + err := validateOSVFrom(bytes.NewBuffer(content), config, catalog, epssData) errs = errors.Join(errs, err) - case artifacts.IsCoverageReport(fileLabel): - err := validateCoverage(bytes.NewBuffer(bundle.FileBytes(fileLabel)), fileLabel, config) + case artifacts.KindSARIF: + err := validateSarifReport(bytes.NewBuffer(content), config) + errs = errors.Join(errs, err) + case artifacts.KindLCOV, artifacts.KindClover, artifacts.KindCobertura, artifacts.KindCoverage: + err := validateCoverage(bytes.NewBuffer(content), kind, fileLabel, config) errs = errors.Join(errs, err) } } @@ -892,6 +899,121 @@ func validateSemgrepRules(config *Config, report *artifacts.SemgrepReportMin) er return nil } +func validateSarifReport(r io.Reader, config *Config) error { + slog.Debug("validate sarif report") + report := &artifacts.SarifReport{} + if err := json.NewDecoder(r).Decode(report); err != nil { + slog.Error("decode sarif report for validation", "error", err) + return errors.New("cannot run SARIF validation: Report decoding failed, See log for details") + } + findings, err := report.Findings() + if err != nil { + slog.Error("read sarif report for validation", "error", err) + return errors.New("cannot run SARIF validation: Report decoding failed, See log for details") + } + return validateSarifRules(config, findings) +} + +func removeIgnoredSarifFindings(config *Config, findings []artifacts.SarifFinding) []artifacts.SarifFinding { + limits := map[string]configLimit{ + "critical": config.Sarif.SeverityLimit.Critical, + "high": config.Sarif.SeverityLimit.High, + "medium": config.Sarif.SeverityLimit.Medium, + "low": config.Sarif.SeverityLimit.Low, + } + return slices.DeleteFunc(findings, func(finding artifacts.SarifFinding) bool { + limit, ok := limits[finding.Severity] + return ok && !limit.Enabled + }) +} + +func ruleSarifDeny(config *Config, findings []artifacts.SarifFinding) bool { + if !config.Sarif.RuleLimit.Enabled { + slog.Debug("rule id limits not enabled", "artifact", "sarif", "count_denied", len(config.Sarif.RuleLimit.Rules)) + return true + } + for _, rule := range config.Sarif.RuleLimit.Rules { + if rule.ID == "" { + continue + } + contains := slices.ContainsFunc(findings, func(finding artifacts.SarifFinding) bool { + return strings.EqualFold(finding.RuleID, rule.ID) + }) + if contains { + slog.Error("rule matched to Deny List", "artifact", "sarif", "id", rule.ID, "metadata", fmt.Sprintf("%+v", rule)) + return false + } + } + return true +} + +func ruleSarifAllow(config *Config, findings []artifacts.SarifFinding) []artifacts.SarifFinding { + slog.Debug("rule id risk acceptance rule", "artifact", "sarif", + "enabled", config.Sarif.RuleRiskAcceptance.Enabled, + "risk_accepted_rules", len(config.Sarif.RuleRiskAcceptance.Rules), + ) + if !config.Sarif.RuleRiskAcceptance.Enabled { + return findings + } + return slices.DeleteFunc(findings, func(finding artifacts.SarifFinding) bool { + allowed := slices.ContainsFunc(config.Sarif.RuleRiskAcceptance.Rules, func(rule configRule) bool { + return rule.ID != "" && strings.EqualFold(rule.ID, finding.RuleID) + }) + if allowed { + slog.Info("rule explicitly allowed, removing from subsequent rules", + "id", finding.RuleID, "severity", finding.Severity) + } + return allowed + }) +} + +func ruleSarifSeverityLimit(config *Config, findings []artifacts.SarifFinding) bool { + validationPass := true + limits := map[string]configLimit{ + "critical": config.Sarif.SeverityLimit.Critical, + "high": config.Sarif.SeverityLimit.High, + "medium": config.Sarif.SeverityLimit.Medium, + "low": config.Sarif.SeverityLimit.Low, + } + for _, severity := range []string{"critical", "high", "medium", "low"} { + configuredLimit := limits[severity] + matches := make([]artifacts.SarifFinding, 0) + for _, finding := range findings { + if strings.EqualFold(finding.Severity, severity) { + matches = append(matches, finding) + } + } + matchCount := len(matches) + if !configuredLimit.Enabled { + slog.Debug("severity limit not enabled", "artifact", "sarif", "severity", severity, "reported", matchCount) + continue + } + if matchCount > int(configuredLimit.Limit) { + slog.Error("severity limit exceeded", "artifact", "sarif", "severity", severity, "report", matchCount, "limit", configuredLimit.Limit) + for _, match := range matches { + slog.Info("finding detected", "rule_id", match.RuleID, "severity", match.Severity, "location", match.Location()) + } + validationPass = false + continue + } + slog.Info("severity limit valid", "artifact", "sarif", "severity", severity, "reported", matchCount, "limit", configuredLimit.Limit) + } + return validationPass +} + +func validateSarifRules(config *Config, findings []artifacts.SarifFinding) error { + slog.Info("validating sarif rules", "findings", len(findings)) + if !ruleSarifDeny(config, findings) { + return newValidationErr("SARIF: Rule explicitly denied") + } + findings = removeIgnoredSarifFindings(config, findings) + findings = ruleSarifAllow(config, findings) + if !ruleSarifSeverityLimit(config, findings) { + return newValidationErr("SARIF: Severity Limit Exceeded") + } + return nil +} + func validateGitleaksRules(config *Config, report *artifacts.GitLeaksReportMin) error { // 1. Limit Secrets - fail if !ruleGitLeaksLimit(config, report) { diff --git a/skills/gatecheck/SKILL.md b/skills/gatecheck/SKILL.md index 1ad08b5..d001075 100644 --- a/skills/gatecheck/SKILL.md +++ b/skills/gatecheck/SKILL.md @@ -7,7 +7,7 @@ description: Use when running gatecheck to list, validate, or bundle scanner rep Run `gatecheck agent-guide` and follow that output. The alias is `gatecheck llm`. -`gatecheck bootstrap` can install Trivy and osv-scanner. A filename containing `trivy` is a Trivy JSON report, and a filename containing `osv` is an osv-scanner JSON report. Severity limits and CVE allow and deny lists use the same fields as Grype. +`gatecheck bootstrap` can install Trivy and osv-scanner. Trivy JSON and osv-scanner JSON are recognized from their contents. A filename containing `trivy` or `osv` is the fallback when the contents are ambiguous. Severity limits and CVE allow and deny lists use the same fields as Grype. The command prints `docs/agent-guide.md`. That file is embedded in the binary, so the command and the file are the same text. Prefer the command when the `gatecheck` binary is installed. diff --git a/test/cyclonedx-mixed.json b/test/cyclonedx-mixed.json new file mode 100644 index 0000000..f61e5f7 --- /dev/null +++ b/test/cyclonedx-mixed.json @@ -0,0 +1,100 @@ +{ + "bomFormat": "CycloneDX", + "specVersion": "1.5", + "version": 1, + "metadata": { + "tools": [ + { + "vendor": "example", + "name": "dependency-track", + "version": "4.12.0" + } + ] + }, + "components": [ + { + "bom-ref": "pkg:npm/left-pad@1.3.0", + "type": "library", + "name": "left-pad", + "version": "1.3.0" + }, + { + "bom-ref": "pkg:pypi/requests@2.28.0", + "type": "library", + "name": "requests", + "version": "2.28.0" + } + ], + "vulnerabilities": [ + { + "id": "CVE-2024-0001", + "ratings": [ + { + "source": { + "name": "nvd" + }, + "score": 9.8, + "method": "CVSSv31" + } + ], + "advisories": [ + { + "url": "https://example.com/CVE-2024-0001" + } + ], + "affects": [ + { + "ref": "pkg:npm/left-pad@1.3.0" + } + ] + }, + { + "id": "CVE-2024-0002", + "ratings": [ + { + "source": { + "name": "ubuntu" + }, + "severity": "HIGH" + } + ], + "affects": [ + { + "ref": "pkg:pypi/requests@2.28.0" + } + ] + }, + { + "id": "CVE-2024-0003", + "ratings": [ + { + "source": { + "name": "nvd" + }, + "score": "9.0", + "method": "CVSSv2" + } + ], + "affects": [ + { + "ref": "missing-component" + } + ] + }, + { + "bom-ref": "vuln-without-id", + "severity": "Medium", + "ratings": [], + "affects": [] + }, + { + "id": "CVE-2024-0004", + "ratings": [ + { + "severity": "info" + } + ], + "affects": [] + } + ] +} diff --git a/test/sarif-bandit.json b/test/sarif-bandit.json new file mode 100644 index 0000000..7b62fdb --- /dev/null +++ b/test/sarif-bandit.json @@ -0,0 +1,92 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Bandit", + "rules": [ + { + "id": "B602", + "name": "subprocess_popen_with_shell_equals_true", + "shortDescription": { + "text": "subprocess call with shell=True" + }, + "defaultConfiguration": { + "level": "error" + } + }, + { + "id": "B101", + "name": "assert_used", + "defaultConfiguration": { + "level": "note" + } + } + ] + } + }, + "results": [ + { + "ruleId": "B602", + "level": "error", + "message": { + "text": "subprocess call with shell=True identified, security issue." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "app.py" + }, + "region": { + "startLine": 4 + } + } + } + ] + }, + { + "ruleId": "B101", + "level": "note", + "message": { + "text": "Use of assert detected." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "app.py" + }, + "region": { + "startLine": 11 + } + } + } + ] + }, + { + "ruleId": "B602", + "level": "none", + "kind": "fail", + "message": { + "text": "Level none is suppressed by the SARIF spec" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "app.py" + }, + "region": { + "startLine": 20 + } + } + } + ] + } + ] + } + ] +} diff --git a/test/sarif-checkov.json b/test/sarif-checkov.json new file mode 100644 index 0000000..35d1fb7 --- /dev/null +++ b/test/sarif-checkov.json @@ -0,0 +1,101 @@ +{ + "version": "2.1.0", + "$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json", + "runs": [ + { + "tool": { + "driver": { + "name": "Checkov", + "rules": [ + { + "id": "CKV_AWS_21", + "defaultConfiguration": { + "level": "error" + }, + "properties": { + "security-severity": "5.0" + } + }, + { + "id": "CKV_AWS_18", + "defaultConfiguration": { + "level": "error" + }, + "properties": { + "security-severity": "3.0" + } + }, + { + "id": "CKV2_AWS_6", + "defaultConfiguration": { + "level": "warning" + }, + "properties": { + "security-severity": "7.4" + } + } + ] + } + }, + "results": [ + { + "ruleId": "CKV_AWS_21", + "level": "error", + "message": { + "text": "Ensure all data stored in the S3 bucket have versioning enabled" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "s3.tf" + }, + "region": { + "startLine": 1 + } + } + } + ] + }, + { + "ruleId": "CKV_AWS_18", + "level": "error", + "message": { + "text": "Ensure the S3 bucket has access logging enabled" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "s3.tf" + }, + "region": { + "startLine": 9 + } + } + } + ] + }, + { + "ruleId": "CKV2_AWS_6", + "level": "warning", + "message": { + "text": "Ensure that S3 bucket has a Public Access block" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "s3.tf" + }, + "region": { + "startLine": 20 + } + } + } + ] + } + ] + } + ] +} diff --git a/test/sarif-govulncheck.json b/test/sarif-govulncheck.json new file mode 100644 index 0000000..c458202 --- /dev/null +++ b/test/sarif-govulncheck.json @@ -0,0 +1,48 @@ +{ + "version": "2.1.0", + "$schema": "https://json.schemastore.org/sarif-2.1.0.json", + "runs": [ + { + "tool": { + "driver": { + "name": "govulncheck", + "rules": [ + { + "id": "GO-2024-2687", + "shortDescription": { + "text": "HTTP/2 CONTINUATION flood in net/http" + }, + "defaultConfiguration": { + "level": "error" + }, + "properties": { + "security-severity": "7.5" + } + } + ] + } + }, + "results": [ + { + "ruleId": "GO-2024-2687", + "level": "error", + "message": { + "text": "Vulnerability in net/http called from example.com/app.Serve" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "go.mod" + }, + "region": { + "startLine": 5 + } + } + } + ] + } + ] + } + ] +} diff --git a/test/sarif-kics.json b/test/sarif-kics.json new file mode 100644 index 0000000..a3643b9 --- /dev/null +++ b/test/sarif-kics.json @@ -0,0 +1,150 @@ +{ + "$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "KICS", + "version": "1.7.12", + "rules": [ + { + "id": "611ab018", + "name": "S3 Bucket Without Versioning", + "defaultConfiguration": { + "level": "error" + }, + "properties": { + "security-severity": "9.1" + } + }, + { + "id": "4a8c5d22", + "name": "S3 Bucket Logging Disabled", + "defaultConfiguration": { + "level": "warning" + }, + "properties": { + "security-severity": "5.3" + } + }, + { + "id": "note-rule", + "defaultConfiguration": { + "level": "note" + } + } + ] + } + }, + "results": [ + { + "ruleId": "611ab018", + "ruleIndex": 0, + "level": "error", + "kind": "fail", + "message": { + "text": "S3 bucket does not have versioning enabled" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "main.tf" + }, + "region": { + "startLine": 12 + } + } + } + ], + "properties": { + "security-severity": "9.1" + } + }, + { + "ruleId": "4a8c5d22", + "level": "warning", + "message": { + "text": "S3 bucket logging is disabled" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "main.tf" + }, + "region": { + "startLine": 30 + } + } + } + ] + }, + { + "ruleId": "note-rule", + "level": "note", + "message": { + "text": "Consider a description" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "main.tf" + }, + "region": { + "startLine": 2 + } + } + } + ] + }, + { + "ruleId": "611ab018", + "level": "error", + "kind": "pass", + "message": { + "text": "Passed and should be ignored" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "ok.tf" + }, + "region": { + "startLine": 1 + } + } + } + ] + }, + { + "ruleId": "611ab018", + "level": "error", + "suppressions": [ + { + "kind": "inSource" + } + ], + "message": { + "text": "Suppressed and should be ignored" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "skip.tf" + }, + "region": { + "startLine": 4 + } + } + } + ] + } + ] + } + ] +} diff --git a/test/sarif-semgrep.json b/test/sarif-semgrep.json new file mode 100644 index 0000000..ef19007 --- /dev/null +++ b/test/sarif-semgrep.json @@ -0,0 +1,70 @@ +{ + "$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Semgrep OSS", + "rules": [ + { + "id": "python.lang.security.audit.eval-detected", + "shortDescription": { + "text": "Detected the use of eval" + }, + "defaultConfiguration": { + "level": "error" + } + }, + { + "id": "python.lang.security.audit.logging-clear", + "defaultConfiguration": { + "level": "warning" + } + } + ] + } + }, + "results": [ + { + "ruleId": "python.lang.security.audit.eval-detected", + "level": "error", + "message": { + "text": "Detected the use of eval. eval is dangerous." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "app.py" + }, + "region": { + "startLine": 8 + } + } + } + ] + }, + { + "ruleId": "python.lang.security.audit.logging-clear", + "level": "warning", + "message": { + "text": "Logger cleared" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "app.py" + }, + "region": { + "startLine": 14 + } + } + } + ] + } + ] + } + ] +} diff --git a/website/llms-full.txt b/website/llms-full.txt index 60ceefe..e18b44d 100644 --- a/website/llms-full.txt +++ b/website/llms-full.txt @@ -1,6 +1,6 @@ # gatecheck -Gatecheck reads a scanner report, compares it to thresholds in a config file, and exits non-zero when a threshold fails. It also prints a table of a report, packs reports into one bundle, and downloads EPSS and KEV data. `gatecheck bootstrap` installs Grype, Syft, Gitleaks, and semgrep's experimental osemgrep CLI, and the matching subcommands run them. Coverage reports still come from the language toolchain. +Gatecheck reads a scanner report, compares it to thresholds in a config file, and exits non-zero when a threshold fails. It also prints a table of a report, packs reports into one bundle, and downloads EPSS and KEV data. `gatecheck bootstrap` installs Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner, and the matching subcommands run them. Coverage reports still come from the language toolchain. `gatecheck` with no arguments prints nothing and exits 0. `gatecheck --help` lists commands. `gatecheck agent-guide` (alias `llm`) prints this file. @@ -14,7 +14,7 @@ Logs go to stderr. Tables, `config init`, downloads, and this guide go to stdout - `validate` compares one report, or one bundle, to a config. - `bundle create` (alias `init`), `bundle add`, and `bundle remove` (alias `rm`) manage a bundle. - `download epss` and `download kev` write upstream data to stdout. -- `bootstrap` installs Grype, Syft, Gitleaks, and osemgrep. `grype`, `syft`, `gitleaks`, `semgrep`, and `osemgrep` run those binaries. +- `bootstrap` installs Grype, Syft, Gitleaks, osemgrep, Trivy, and osv-scanner. `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, and `osv-scanner` run those binaries. - `update` (aliases `self-update`, `upgrade`) replaces this binary with a GitHub release after checking `checksums.txt`. `update --check` only prints the current and latest versions. - `completion bash|fish|powershell|zsh` prints a shell completion script. - `help [command]` prints the same text as ` --help`. @@ -25,6 +25,10 @@ Global flags, on every command: `-v` / `--verbose` (debug logs) and `--silent` ( `gatecheck bootstrap` downloads pinned releases into the gatecheck data directory and checks each file's SHA-256. Grype, Syft, and Gitleaks come from their GitHub release assets and checksum files. Semgrep publishes no GitHub release assets. Bootstrap downloads the PyPI wheel for this platform, checks the digest in the PyPI JSON, and installs the wheel's native `semgrep-core` binary as `osemgrep`. That binary is semgrep's experimental native CLI. On Linux the wheel's `libs` directory is installed beside it. On Windows the file is `osemgrep.exe`. +Trivy comes from the `aquasecurity/trivy` GitHub release. The asset names look like `trivy_0.76.1_Linux-64bit.tar.gz`, `trivy_0.76.1_macOS-ARM64.tar.gz`, and `trivy_0.76.1_windows-64bit.zip`. The checksum file is `trivy__checksums.txt`. Linux arm is the armv7 archive. Windows arm64 has no published asset. + +osv-scanner comes from the `google/osv-scanner` GitHub release as a raw binary, not an archive. Asset names look like `osv-scanner_linux_amd64`, `osv-scanner_darwin_arm64`, and `osv-scanner_windows_amd64.exe`. The checksum file is `osv-scanner_SHA256SUMS`. Published builds are amd64 and arm64 on Linux, macOS, and Windows. The installed name is `osv-scanner`, or `osv-scanner.exe` on Windows. + The directory is `$GATECHECK_HOME/bin` when `GATECHECK_HOME` is set. Otherwise it is `$XDG_DATA_HOME/gatecheck/bin`, or `~/.local/share/gatecheck/bin` on Linux, `~/Library/Application Support/gatecheck/bin` on macOS, and `%LOCALAPPDATA%\gatecheck\bin` on Windows. A tool already at the pinned version is left in place. `--force` downloads it again. `--tools grype,syft` limits the set. `osemgrep` in `--tools` means semgrep. `--list` prints pinned versions, installed versions, and paths, and does not download. @@ -34,37 +38,61 @@ gatecheck bootstrap gatecheck bootstrap --list gatecheck grype dir:latest -o json > grype-report.json gatecheck semgrep scan --json --config auto . +gatecheck trivy fs --format json --output trivy-report.json . +gatecheck osv-scanner scan source --format json . > osv-scanner-report.json ``` `gatecheck semgrep scan` and `gatecheck osemgrep scan` insert `--experimental` after `scan` when that flag is absent. Other osemgrep commands need `--experimental` from the caller. `gatecheck semgrep --experimental --version` prints the osemgrep version. When the program that runs is the Python `semgrep` on `PATH`, the flag is left alone. `gatecheck osemgrep` looks for the managed binary, then `osemgrep` on `PATH`. -The managed binary wins when it is present. `gatecheck grype`, `gatecheck syft`, and `gatecheck gitleaks` otherwise run that name on `PATH`. `gatecheck semgrep` tries `semgrep` and then `osemgrep` on `PATH`. If none exist, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 255. A scanner's own exit status is returned as-is. +The managed binary wins when it is present. `gatecheck grype`, `gatecheck syft`, `gatecheck gitleaks`, `gatecheck trivy`, and `gatecheck osv-scanner` otherwise run that name on `PATH`. `gatecheck semgrep` tries `semgrep` and then `osemgrep` on `PATH`. If none exist, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 255. A scanner's own exit status is returned as-is. -A platform with no Grype, Syft, or Gitleaks asset fails that tool and the command exits 255. A platform with no semgrep wheel prints a skip message on stderr and continues. Current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, and semgrep `1.182.1`, in `internal/bootstrap/versions.go`. +A platform with no Grype, Syft, Gitleaks, Trivy, or osv-scanner asset fails that tool and the command exits 255. A platform with no semgrep wheel prints a skip message on stderr and continues. Current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, and osv-scanner `2.7.0`, in `internal/bootstrap/versions.go`. -Syft is installed so it can write CycloneDX (`syft -o cyclonedx-json`). A filename containing `syft` is still recognized by `list` and `validate` and then returns an error. CycloneDX is a report format, and coverage tools are left to the language toolchain. +Syft is installed so it can write CycloneDX (`syft -o cyclonedx-json`). That BOM is listed and validated as CycloneDX. Native Syft JSON is recognized from `descriptor.name` or the schema URL, and `list` and `validate` then return an error. A filename containing `syft` does the same when the contents are ambiguous. CycloneDX is a report format, and coverage tools are left to the language toolchain. ## Report type -`list` and `validate` pick a type by the first matching lowercase word in the filename: +`list` and `validate` pick a type from the file contents. The filename is used only when the contents do not identify a type. + +Content rules, in this order: + +1. A gzipped tar that contains `gatecheck-manifest.json` is a Gatecheck bundle. +2. `bomFormat` of `CycloneDX` is CycloneDX JSON. This wins when the name contains `grype` or `syft`, and it wins when the tool that wrote the BOM was Grype, Syft, Trivy, or another tool. CycloneDX XML is not read. +3. SARIF 2.1.0. The top-level `version` is the string `2.1.0` and `runs` is an array. A `$schema` that mentions SARIF is enough when `runs` is missing or null. Another version returns an error when the name says `sarif`. +4. Grype JSON. `descriptor.name` is `grype`, or `matches` is a non-empty array of objects with `vulnerability` and `artifact`. +5. Syft JSON. `descriptor.name` is `syft`, or `schema.url` points at the Syft schema. `list` and `validate` then return an error. Syft JSON is not implemented. A Syft CycloneDX BOM is CycloneDX, not Syft. +6. Semgrep JSON. `results` is an array of objects with `check_id`, or `results` and `errors` are both arrays. Semgrep's SARIF output is SARIF, not this type. +7. Gitleaks JSON. A non-empty array of objects with `RuleID` and one of `File`, `Commit`, or `Fingerprint`. +8. Trivy JSON. `SchemaVersion` is a number, and `ArtifactType` is set or `Results` is an array. Vulnerability entries are listed and validated. Misconfigurations, secrets, and licenses are ignored. A CycloneDX BOM from Trivy is CycloneDX. +9. osv-scanner JSON. `results` is a non-empty array of objects with a `packages` array. One finding per vulnerability group. Aliases of that group are not counted again. License-only packages are skipped. Semgrep is checked first, so a `check_id` stays Semgrep. `{"results":[]}` does not identify itself. +10. LCOV text. A line starts with `SF:` and the file also has `DA:`, `LF:`, or `end_of_record`. +11. Clover XML. An XML document with `clover=` or a ``, so the same words work, including the coverage words. `list-all` and `validate` do not read standard input. `validate` requires a file path. +From standard input, `list` requires `-i` / `--input-type`, so the command does not wait on a terminal. The value is matched as the filename `stdin:`, and only when the contents are ambiguous. Clear contents win over `--input-type`. `list-all` and `validate` do not read standard input. `validate` requires a file path. -`list-all` lists a file only when the name contains `grype`, `semgrep`, `gitleaks`, `syft`, `cyclonedx`, `bundle`, or `gatecheck`. Coverage files are skipped with a warning. There is no `--input-type` on `list-all`. +`list-all` lists a file when the contents or the name identify Grype, Semgrep, Gitleaks, Syft, CycloneDX, SARIF, Trivy, osv-scanner, or a bundle. Coverage files are skipped with a warning. There is no `--input-type` on `list-all`. -Inside a bundle, `validate` uses the stored label, which is the base name of the file that was added. Labels that do not match a type are skipped. Syft labels inside a bundle are skipped. +Inside a bundle, `validate` uses the same content rules on each stored file, and falls back to the label. The label is the base name of the file that was added. Labels that still do not match are skipped. Syft JSON inside a bundle is skipped. ## Config @@ -78,36 +106,55 @@ gatecheck config convert --file gatecheck.yaml --output toml `config convert` reads `-f` / `--file` (default `gatecheck.yaml`). The extension selects the decoder: `.yaml`, `.yml`, `.json`, or `.toml`. -A partial file is fine. Omitted fields keep the default. Unknown keys are ignored. With no config, `validate` uses the built-in default: every rule disabled, coverage thresholds `0`. Grype, CycloneDX, Semgrep, Gitleaks, coverage, and bundle reports pass under that default. A Syft filename still returns an error. +A partial file is fine. Omitted fields keep the default. Unknown keys are ignored. With no config, `validate` uses the built-in default: every rule disabled, coverage thresholds `0`. Grype, CycloneDX, Trivy, osv-scanner, SARIF, Semgrep, Gitleaks, coverage, and bundle reports pass under that default. Syft JSON still returns an error. An old config with no `sarif` section leaves those limits disabled. `GATECHECK_CONFIG_FILENAME` is the same setting as `validate --config` / `-f`. Field names below are the ones `config init` emits. In YAML they are case sensitive. `version` (currently `"1"`) and `metadata.tags` do not change validation. -`grype` and `cyclonedx` use the same fields. +`grype`, `cyclonedx`, `trivy`, and `osv-scanner` use the same fields. The config key is `osv-scanner`. - `severityLimit` has `critical`, `high`, `medium`, and `low`. Each has `enabled` and `limit`. `limit` is a count. The report fails when the count is greater than `limit`. A count equal to `limit` passes. `enabled: true` and `limit: 0` allows none. `enabled: false` skips that severity. `negligible` and `unknown` have no limit field. - `epssLimit.score` and `epssRiskAcceptance.score` are EPSS ratios from 0 to 1, the same scale as the FIRST CSV. The limit fails when a CVE's score is greater than `score`. A score equal to the limit passes. A CVE with no EPSS row passes the limit. Risk acceptance removes a CVE when `score` is greater than that CVE's score. An equal score is kept. A CVE with no EPSS row is kept. - `kevLimitEnabled` fails the report when any remaining CVE is in the CISA catalog. -- `cveLimit` and `cveRiskAcceptance` each have `enabled` and `cves`. A CVE entry is `id` plus an optional `metadata.tags` list. `id` is compared without regard to case. +- `cveLimit` and `cveRiskAcceptance` each have `enabled` and `cves`. A CVE entry is `id` plus an optional `metadata.tags` list. `id` is compared without regard to case. Trivy matches `VulnerabilityID` and each `VendorIDs` entry, so a GHSA primary id still matches a CVE in `VendorIDs`. osv-scanner matches the OSV id and each alias. EPSS and KEV use the first id that starts with `CVE-`. +- osv-scanner `max_severity` is a CVSS base score: `9.0` and above is critical, `7.0` high, `4.0` medium, `0.1` low, and anything lower is unknown. The words `CRITICAL`, `HIGH`, `MEDIUM`, `LOW`, `MODERATE`, and `UNKNOWN` are accepted. `MODERATE` is medium. A CVSS vector with no numeric score is unknown. `database_specific.severity` is used when `max_severity` is empty. `semgrep.severityLimit` has `error`, `warning`, and `info`, each with `enabled` and `limit`, with the same count rule. `impactRiskAcceptance` has `enabled`, `high`, `medium`, and `low`. When `enabled` is true, a finding is removed when its impact matches a level whose value is true. `gitleaks.limitEnabled: true` fails validation when the report contains any finding. There is no count. +`sarif` is for any SARIF 2.1.0 report. Semgrep's own JSON still uses `semgrep`. `semgrep --sarif` uses `sarif`. + +- `severityLimit` has `critical`, `high`, `medium`, and `low`. Each has `enabled` and `limit`, with the same count rule as Grype. The severity comes from the SARIF property `security-severity` when a result or its rule has one. The value is a number from 0 to 10, or a numeric string. 9.0 and above is critical, 7.0 to 8.9 is high, 4.0 to 6.9 is medium, above 0 and below 4.0 is low, and 0 is none. A result property wins over the same property on the rule. When neither is set, gatecheck uses the SARIF level. A missing result level uses the rule's `defaultConfiguration.level`, then `warning`. `error` is high, `warning` is medium, `note` is low, and `none` is ignored. +- `ruleLimit` and `ruleRiskAcceptance` each have `enabled` and `rules`. A rule entry is `id` plus an optional `metadata.tags` list. `id` is compared without regard to case. `ruleLimit` denies those ids. `ruleRiskAcceptance` removes them. + `coverage.lineThreshold`, `functionThreshold`, and `branchThreshold` are ratios from 0 to 1: covered lines, functions, or branches divided by the total. The report fails when its ratio is less than the threshold. A ratio equal to the threshold passes. `0.8` is 80 percent. `80` fails a report whose real line coverage is 0.8, because 0.8 is less than 80. There is no `enabled` flag. The default `0` allows any coverage, because a ratio is never less than 0. `list` prints these numbers as percents in the table footer. The config still wants the ratio. -Grype and CycloneDX rules run in this order. A failure returns immediately, except where a rule only removes findings. +Grype, CycloneDX, Trivy, and osv-scanner rules run in this order. A failure returns immediately, except where a rule only removes findings. Trivy counts vulnerability entries only. osv-scanner counts one finding per group. 1. CVE limit, when enabled. -2. Disabled Grype severities are dropped. If the EPSS limit is also enabled, a Grype finding at a disabled severity stays when its EPSS score is greater than or equal to that limit. CycloneDX leaves findings at a disabled severity in the report. +2. Disabled Grype, Trivy, and osv-scanner severities are dropped. If the EPSS limit is also enabled, a finding at a disabled severity stays when its EPSS score is greater than or equal to that limit. CycloneDX leaves findings at a disabled severity in the report. 3. CVE risk acceptance removes matching ids. 4. KEV limit, when `kevLimitEnabled` is true. 5. EPSS risk acceptance. 6. EPSS limit. 7. Severity limit. -Semgrep drops findings whose severity limit is disabled, then applies impact risk acceptance, then the severity counts. Coverage runs all three checks and reports each failure. A bundle is validated when its filename contains `bundle`. Each label is checked on its own. The command exits 1 if any label fails a threshold. +Semgrep drops findings whose severity limit is disabled, then applies impact risk acceptance, then the severity counts. That impact rule does not apply to `semgrep --sarif`. That file uses the `sarif` rules. + +SARIF runs in this order. + +1. Rule limit, when enabled. A denied rule fails even when its severity limit is disabled. +2. Disabled severities are dropped. `none` has no limit field and stays in the report, but the severity rule does not count it. +3. Rule risk acceptance removes matching ids. +4. Severity limit. + +These SARIF results are not counted: `kind` of `pass` or `notApplicable`, `level` of `none`, a non-empty `suppressions` array, and `baselineState` of `absent`. SARIF has no EPSS or KEV rules. + +Any CycloneDX JSON BOM uses the `cyclonedx` rules, including a BOM from Grype, Trivy, Syft, or another tool. The severity is the highest rating. A rating with a score and no severity word uses CVSS ranges. A CVSS v2 method does not produce critical. `important` counts as high, and `moderate` counts as medium. `negligible` counts as none. Empty ratings use a top-level `severity` string when the producer set one, otherwise `unknown`. `unknown`, `none`, and `info` are not counted by the critical, high, medium, or low limits. The id is the vulnerability `id`, or the `bom-ref` when `id` is empty. + +Coverage runs all three checks and reports each failure. A bundle is validated when its contents are a Gatecheck bundle, or when the filename contains `bundle` and the contents do not identify another type. Each stored file is checked on its own. The command exits 1 if any file fails a threshold. ## validate @@ -128,7 +175,7 @@ Opening the report, the config, and any `--epss-filename` or `--kev-filename` ha `--epss-filename` is the EPSS CSV used when an EPSS rule is enabled. `--kev-filename` is the CISA KEV JSON used when `kevLimitEnabled` is true. `--epss-cache-dir` and `--epss-no-cache` apply when that CSV is downloaded. `validate` has no `--epss-url` or `--kev-url` flag. `GATECHECK_EPSS_URL` and `GATECHECK_KEV_URL` still override those APIs when the matching file is not set. The defaults are the same URLs `download` uses. -EPSS data is loaded only when an EPSS limit or EPSS risk acceptance rule is enabled. KEV data is loaded only when `kevLimitEnabled` is true for Grype or CycloneDX. With no local EPSS file, gatecheck downloads the current CSV and caches it until the published date changes. +EPSS data is loaded only when an EPSS limit or EPSS risk acceptance rule is enabled. KEV data is loaded only when `kevLimitEnabled` is true for Grype, CycloneDX, Trivy, or osv-scanner. With no local EPSS file, gatecheck downloads the current CSV and caches it until the published date changes. ## list @@ -139,13 +186,13 @@ grype image:tag -o json | gatecheck ls --input-type grype gatecheck list-all grype-report.json semgrep-report.json ``` -`list` opens the first filename only. Further filenames are ignored. `--markdown` prints a markdown table. A Grype, CycloneDX, Semgrep, or Gitleaks report with no findings prints one sentence and no table, with or without `--markdown`: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No semgrep findings`, or `No gitleaks findings`. `--epss` adds EPSS score and percentile columns for Grype and CycloneDX. Other report types still list, and those columns are omitted. `--epss-url` empty uses `https://epss.cyentia.com`. `--epss-filename` reads that CSV instead of calling the API. +`list` opens the first filename only. Further filenames are ignored. `--markdown` prints a markdown table. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table, with or without `--markdown`: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `--epss` adds EPSS score and percentile columns for Grype, CycloneDX, Trivy, and osv-scanner. Other report types still list, and those columns are omitted. `--epss-url` empty uses `https://epss.cyentia.com`. `--epss-filename` reads that CSV instead of calling the API. -`list-all` prints each filename, then its table, or that sentence when the report has no findings. A file whose name is not a supported `list-all` type is skipped with a warning. A missing file is skipped. A list error is logged and the command continues. The command exits 0. `--epss` adds EPSS data only for filenames that contain `grype` or `cyclonedx`. There is no `--epss-filename` flag, and `GATECHECK_EPSS_FILENAME` is not read. `GATECHECK_EPSS_URL` overrides the API. `--epss-cache-dir` and `--epss-no-cache` are accepted. +`list-all` prints each filename, then its table, or that sentence when the report has no findings. A file whose contents and name are not a supported `list-all` type is skipped with a warning. A missing file is skipped. A list error is logged and the command continues. The command exits 0. `--epss` adds EPSS data for Grype, CycloneDX, Trivy, and osv-scanner, including a file whose name does not contain those words when the contents do. There is no `--epss-filename` flag, and `GATECHECK_EPSS_FILENAME` is not read. `GATECHECK_EPSS_URL` overrides the API. `--epss-cache-dir` and `--epss-no-cache` are accepted. ## bundle -Use a name that contains `bundle`, such as `gatecheck-bundle.tar.gz`. `validate` treats the file as a bundle only when the name contains `bundle`. `list` does that too, and also when the name contains `gatecheck`. A bundle is a gzipped tar plus a JSON manifest. +A bundle is a gzipped tar plus a JSON manifest named `gatecheck-manifest.json`. `list` and `validate` treat the file as a bundle when the contents are that archive. The filename is the fallback: `validate` requires the word `bundle`, and `list` also accepts a name containing `gatecheck`. A name such as `project.gcb` still works when the file is a real bundle. A name that contains `bundle`, such as `gatecheck-bundle.tar.gz`, still works when the contents are ambiguous. ```shell gatecheck bundle create gatecheck-bundle.tar.gz grype-report.json --tag ci @@ -154,7 +201,7 @@ gatecheck bundle remove gatecheck-bundle.tar.gz semgrep-report.json gatecheck list gatecheck-bundle.tar.gz ``` -The stored label is the base name of the target file (`reports/grype-report.json` is stored as `grype-report.json`). Later validation uses that label to pick the type, so the file name needs a word such as `grype` or `semgrep`. Repeat `-t` / `--tag` for more than one tag. When the flag is omitted, `GATECHECK_BUNDLE_TAG` is read as a comma-separated list. `bundle add` requires an existing bundle. `bundle remove` takes the label, not a path that is opened. A label that is not in the bundle is logged and the command exits 0. There is no `bundle list` subcommand. `gatecheck list` prints label, SHA-256 digest, tags, and size. +The stored label is the base name of the target file (`reports/grype-report.json` is stored as `grype-report.json`). Later validation uses the file contents to pick the type, and the label when the contents are ambiguous. Repeat `-t` / `--tag` for more than one tag. When the flag is omitted, `GATECHECK_BUNDLE_TAG` is read as a comma-separated list. `bundle add` requires an existing bundle. `bundle remove` takes the label, not a path that is opened. A label that is not in the bundle is logged and the command exits 0. There is no `bundle list` subcommand. `gatecheck list` prints label, SHA-256 digest, tags, and size. ## download @@ -182,7 +229,7 @@ A non-empty flag wins. An empty flag falls through to the environment variable, | `GATECHECK_EPSS_CACHE` | `download epss`, `list`, `list-all`, `validate` | `--epss-no-cache` when the value is `off` | | `GATECHECK_KEV_URL` | `download kev`, `validate` | `--kev-url` on `download kev` | | `GATECHECK_BUNDLE_TAG` | `bundle create`, `bundle add` | `--tag`, as a comma-separated list | -| `GATECHECK_HOME` | `bootstrap`, `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep` | data directory; binaries are in `bin` under it | +| `GATECHECK_HOME` | `bootstrap`, `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner` | data directory; binaries are in `bin` under it | `GATECHECK_EPSS_CACHE` also disables the cache for `0`, `false`, `no`, `disabled`, `disable`, and `none`. @@ -199,11 +246,11 @@ Create the config with `gatecheck config init > gatecheck.yaml` and set real thr ## Common mistakes -- The filename has no lowercase type word (`scan.json`, `Grype.json`, `project.gcb`), so the type cannot be detected. Use names such as `grype-report.json`, and a bundle name that contains `bundle`. +- The contents do not identify a type and the filename has no lowercase type word (`scan.json`, `Grype.json`). A real bundle named `project.gcb` is still recognized from its contents. An empty document such as `{"matches":[]}` still needs a name such as `grype-report.json`, or `--input-type grype` on standard input. - Coverage thresholds are ratios. Set `coverage.lineThreshold` to `0.8` for 80 percent, not `80` or `80.0`. - `validate` takes the report as its positional argument. The config is `--config` / `-f`. `gatecheck validate gatecheck.yaml grype-report.json` tries to open the config as a report. `config convert` uses `--file` / `-f`, which is a different flag. -- `list` on a pipe needs `--input-type`. `validate` needs a filesystem path. +- `list` on a pipe needs `--input-type`. Clear contents still win over that flag. `validate` needs a filesystem path. - `list` ignores every filename after the first. Use `list-all` for several files. `list-all` skips coverage files such as `lcov.info`. List those with `gatecheck list lcov.info`. -- A Syft filename is recognized and then returns an error. Use CycloneDX output from Syft when you want `validate` to score it. +- Native Syft JSON is recognized and then returns an error. Use CycloneDX output from Syft when you want `validate` to score it. That BOM is CycloneDX even when the filename contains `syft`. - `gatecheck semgrep --version` reaches osemgrep without `--experimental`, and osemgrep then tries to run `pysemgrep`. Pass the flag: `gatecheck semgrep --experimental --version`. - `--audit` still exits 255 when the report, the config, or a named EPSS or KEV file cannot be opened. diff --git a/website/llms.txt b/website/llms.txt index f06d5b6..0571782 100644 --- a/website/llms.txt +++ b/website/llms.txt @@ -2,15 +2,15 @@ > Gatecheck compares a scanner report to thresholds in a config file and exits non-zero when a limit is crossed. One binary. Stateless. -Report type comes from a lowercase word in the filename. Coverage thresholds are ratios: 0.8 is 80 percent. Pass the config with `gatecheck validate --config`. Print the agent guide with `gatecheck agent-guide`. +Report type comes from the file contents. The filename is the fallback when the contents are ambiguous. Coverage thresholds are ratios: 0.8 is 80 percent. Pass the config with `gatecheck validate --config`. Print the agent guide with `gatecheck agent-guide`. ## Docs - [Installation](https://docs.gatecheck.dev/installation.html): install a release binary or build from source - [Usage](https://docs.gatecheck.dev/usage.html): commands, logs, and exit status -- [Bootstrap](https://docs.gatecheck.dev/bootstrap.html): install Grype, Syft, Gitleaks, and osemgrep, then run them +- [Bootstrap](https://docs.gatecheck.dev/bootstrap.html): install Grype, Syft, Gitleaks, osemgrep, Trivy, and osv-scanner, then run them - [CLI reference](https://docs.gatecheck.dev/cli-reference.html): flags, subcommands, and environment variables -- [Supported reports](https://docs.gatecheck.dev/supported-reports.html): filename rules that select the report type +- [Supported reports](https://docs.gatecheck.dev/supported-reports.html): how file contents and filenames select the report type - [Configuration](https://docs.gatecheck.dev/configuration.html): config keys and threshold units - [Validation](https://docs.gatecheck.dev/validation.html): rule order and exit codes - [List reports](https://docs.gatecheck.dev/list-reports.html): tables, standard input, and EPSS columns