Repository navigation
Batch: 2 fixes, one commit each - #58
Merged
Merged
Conversation
…h "bufio.Scanner: token too long" (#53) The go.mod reader read the file with a bufio.Scanner and its default token limit of 64 KiB. A longer line stopped it with "bufio.Scanner: token too long", the parser returned that error, and the file was refused as a whole: a directory whose go.mod has a comment line of 70,003 characters on line 5 and "require golang.org/x/crypto v0.17.0" on line 7 failed analyze --offline with "found 1 manifest file(s) but none could be read" and exit code 2. go mod edit -json from Go 1.26.3 reads that file and lists the require. The scanner's token limit is now math.MaxInt, as the requirements.txt reader's is. That scan now reports golang.org/x/crypto@v0.17.0 at go.mod:7 with 2 vulnerable and 3 safe findings, and exits 1. A comment line inside a require block and a comment at the end of a require entry, each longer than 64 KiB, are read with the entries around them. For testdata/fixtures/go-project and examples/vulnerable-demo, analyze --offline prints the same table and stderr before and after, its JSON output differs only in timestamps, and the exit code is 1 for both. Fixes #53
…ndency's version, so "cryptography==42.0.0 xxx" is reported as cryptography 42.0.0 (#54) The requirements.txt parser matches a requirement line with a pattern whose last group takes the rest of the line, and removes only a comment and an environment marker from that group before using it as the version. Text after the version specifier stayed in the version: a requirements.txt holding "cryptography==42.0.0 xxx", which pip 26.0 rejects with "Invalid requirement: 'cryptography==42.0.0 xxx': Expected comma (within version specifier), semicolon (after version specifier) or end", was reported by analyze --offline --format json as "version": "42.0.0 xxx", and the table printed cryptography@42.0.0 xxx. With 70,000 "x" characters after the version, each of the 5 table rows naming cryptography carried all of them. A requirement line is now cut where its version specifier ends, before that pattern reads it. The end is found as pip's requirement parser finds it: after the name, the extras and one or more clauses, each an operator and a version, joined by commas, with whitespace allowed around each part and the whole optionally in parentheses. The first clause needs no operator, as "cryptography 42.0.0" has been read as version 42.0.0. The name is read with its dots, so "zope.interface >= 5.0" is kept whole, and a name @ path line, which has no version, is not cut. For the file in the report, the JSON output differs from before only in "version": "42.0.0" and the timestamps, the table prints cryptography@42.0.0, and the exit code is 1 before and after. With 70,000 characters after the version, no table row carries them, the table output is 8,545 bytes where it was 638,554, and the scan reports 9 quantum-vulnerable findings before and after. For a file of 21 requirement lines, among them "cryptography>=41.0 , < 43", "cryptography>=41.0,", "pyopenssl (>=23.0)", "pycryptodome [extra] >= 3.20", "ecdsa==0.18.0;python_version<'3.12'", "zope.interface >= 5.0" and "asn1crypto @ ./vendor/asn1crypto-1.5.1.tar.gz", the dependency names and versions are the same before and after. With " xxx" added to 19 of those lines, each is reported with the version of its line without it, where 17 of the 19 versions held "xxx" before. For testdata/fixtures/python-project the table output and stderr are the same before and after, the JSON output differs only in timestamps, and the exit code is 1 for both. Fixes #54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request lands 2 commits in internal/manifest and CHANGELOG.md.
Included
Verified on this exact tree
go build ./...(go1.26.3): exit 0go vet ./...(go1.26.3): exit 0golangci-lint run ./...(golangci-lint 2.12.2): exit 3,can't load config: unsupported version of the configuration: ""golangci-lint run ./...in a worktree oforigin/main(1101eaf): exit 3,can't load config: unsupported version of the configuration: ""git diff --name-only origin/main...HEAD -- .golangci.yml: no outputgolangci-lint run --config <version 2 translation of .golangci.yml> --max-same-issues 0 --max-issues-per-linter 0 ./...: exit 0,0 issues.go test -race -count=1 -p 4 -json -coverprofile=coverage.out ./...(go1.26.3): exit 0; packages ok=16 failed=0; tests and subtests passed=1625 failed=0 skipped=1; top-level tests passed=526 failed=0 skipped=1TestIntegration_FetchPyPIPackage(pip not available, skipping test)go vet ./internal/manifest/(go1.21.13): exit 0go test -race -count=1 -p 4 ./internal/manifest/(go1.21.13): exit 0,okforgithub.com/csnp/qramm-cryptodeps/internal/manifestNot included
None
Closes
Fixes #53
Fixes #54
How to merge
Rebase and merge, so that each change stays its own commit on
main.