Skip to content

Break immidiately after 507 - #241

Open
leborchuk wants to merge 1 commit into
masterfrom
Fix507
Open

leborchuk wants to merge 1 commit into
masterfrom
Fix507

Conversation

@leborchuk

Copy link
Copy Markdown
Contributor

507 from S3 means object does not exists for sure.

So there are no need to read it - break immidiately and return error.

Copilot AI balanced review requested due to automatic review settings October 16, 2026 10:48
Comment thread pkg/proc/yio/yrreader.go
err := y.underlying.Restart(y.offsetReached)

if err != nil {
if storage.IsNonRetryableError(err) {
Comment thread pkg/proc/yio/yrreader.go
metrics.ReadReqErrors.Inc()
ylogger.Zero.Error().Err(err).Int64("offset reached", y.offsetReached).Int("bytes half-read", n).Int("retry count", int(retry)).Msg("encounter read error")
if err != nil {
if storage.IsNonRetryableError(err) {
"testing"

"github.com/stretchr/testify/assert"
"github.com/yezzey-gp/aws-sdk-go/aws/awserr"
ycl := mockcl.NewMockYproxyClient(ctrl)
yr := yio.NewYRetryReader(rr, ycl)

err507 := awserr.NewRequestFailure(
yr := yio.NewYRetryReader(rr, ycl)

err507 := awserr.NewRequestFailure(
awserr.New("InsufficientStorage", "object does not exist", nil),
ycl := mockcl.NewMockYproxyClient(ctrl)
yr := yio.NewYRetryReader(rr, ycl)

err507 := awserr.NewRequestFailure(
yr := yio.NewYRetryReader(rr, ycl)

err507 := awserr.NewRequestFailure(
awserr.New("InsufficientStorage", "object does not exist", nil),
Comment thread pkg/storage/utils.go
* Storage returns 507 for objects that do not exist.
* Retrying such requests is pointless: the object will not appear.
*/
const nonRetryableStatusCode = 507
Comment thread pkg/storage/utils.go
*/
const nonRetryableStatusCode = 507

func IsNonRetryableError(err error) bool {
Comment thread pkg/storage/utils.go
func IsNonRetryableError(err error) bool {
var reqErr awserr.RequestFailure
if errors.As(err, &reqErr) {
return reqErr.StatusCode() == nonRetryableStatusCode
@github-actions

Copy link
Copy Markdown

@check-spelling-bot Report

🔴 Please review

See the 📂 files view, the 📜action log, 👼 SARIF report, or 📝 job summary for details.

Unrecognized words (2)

awserr
Retryable

These words are not needed and should be removed fuzz myminio yml

To accept these unrecognized words as correct and remove the previously acknowledged and now absent words, you could run the following commands

... in a clone of the git@github.com:open-gpdb/yproxy.git repository
on the Fix507 branch (ℹ️ how do I use this?):

curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' |
perl - 'https://github.com/open-gpdb/yproxy/actions/runs/38889638631/attempts/1' &&
git commit -m 'Update check-spelling metadata'

OR

To have the bot accept them for you, comment in the PR quoting the following line:
@check-spelling-bot apply updates.

Available 📚 dictionaries could cover words (expected and unrecognized) not in the 📘 dictionary

This includes both expected items (287) from .github/actions/spelling/expect.txt and unrecognized words (2)

Dictionary Entries Covers Uniquely
cspell:golang/dict/go.txt 2099 15 5
cspell:node/dict/node.txt 891 15 2
cspell:php/dict/php.txt 1689 15 2
cspell:python/src/python/python-lib.txt 2417 14 2
cspell:dotnet/dict/dotnet.txt 405 7
cspell:fullstack/dict/fullstack.txt 419 5 1
cspell:java/src/java.txt 2464 5 2
cspell:python/src/python/python.txt 392 5
cspell:typescript/dict/typescript.txt 1098 4 1
cspell:k8s/dict/k8s.txt 153 4
cspell:filetypes/filetypes.txt 264 4
cspell:r/src/r.txt 543 4
cspell:django/dict/django.txt 393 3 2
cspell:aws/aws.txt 218 3 1
cspell:cpp/src/stdlib-c.txt 278 3 1
cspell:rust/dict/rust.txt 30 3
cspell:scala/dict/scala.txt 153 3
cspell:cpp/src/stdlib-cpp.txt 252 3
cspell:npm/dict/npm.txt 302 3
cspell:lua/dict/lua.txt 190 2 1

Consider adding them (in .github/workflows/spelling.yaml) in jobs:/spelling: for uses: check-spelling/check-spelling@main in its with to extra_dictionaries:

            cspell:golang/dict/go.txt
            cspell:node/dict/node.txt
            cspell:php/dict/php.txt
            cspell:python/src/python/python-lib.txt
            cspell:dotnet/dict/dotnet.txt
            cspell:fullstack/dict/fullstack.txt
            cspell:java/src/java.txt
            cspell:python/src/python/python.txt
            cspell:typescript/dict/typescript.txt
            cspell:k8s/dict/k8s.txt
            cspell:filetypes/filetypes.txt
            cspell:r/src/r.txt
            cspell:django/dict/django.txt
            cspell:aws/aws.txt
            cspell:cpp/src/stdlib-c.txt
            cspell:rust/dict/rust.txt
            cspell:scala/dict/scala.txt
            cspell:cpp/src/stdlib-cpp.txt
            cspell:npm/dict/npm.txt
            cspell:lua/dict/lua.txt

To stop checking additional dictionaries, add (in .github/workflows/spelling.yaml) for uses: check-spelling/check-spelling@main in its with:

check_extra_dictionaries: ""
Pattern suggestions ✂️ (13)

You could add these patterns to .github/actions/spelling/patterns.txt:

# Automatically suggested patterns

# hit-count: 13 file-count: 8
# https/http/file urls
(?:\b(?:https?|ftp|file)://)[-A-Za-z0-9+&@#/*%?=~_|!:,.;]+[-A-Za-z0-9+&@#/*%=~_|]

# hit-count: 8 file-count: 2
# version suffix <word>v#
(?:(?<=[A-Z]{2})V|(?<=[a-z]{2}|[A-Z]{2})v)\d+(?:\b|(?=[a-zA-Z_]))

# hit-count: 5 file-count: 5
# uuencoded
[!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_]{40,}

# hit-count: 5 file-count: 1
# in check-spelling@v0.0.22+, printf markers aren't automatically consumed
# printf markers
(?<!\\)\\[nrt](?=[a-z]{2,})

# hit-count: 5 file-count: 1
# alternate printf markers if you run into latex and friends
(?<!\\)\\[nrt](?=[a-z]{2,})(?=.*['"`])

# hit-count: 4 file-count: 2
# C network byte conversions
(?:\d|\bh)to(?!ken)(?=[a-z])|to(?=[adhiklpun]\()

# hit-count: 3 file-count: 1
# container images
image: [-\w./:@]+

# hit-count: 1 file-count: 1
# hex digits including css/html color classes:
(?:[\\0][xX]|\\u|[uU]\+|#x?|%23|&H)[0-9_a-fA-FgGrR]*?[a-fA-FgGrR]{2,}[0-9_a-fA-FgGrR]*(?:[uUlL]{0,3}|[iu]\d+)\b

# hit-count: 1 file-count: 1
# scala imports
^import (?:[\w.]|\{\w*?(?:,\s*(?:\w*|\*))+\})+

# hit-count: 1 file-count: 1
# Debian changelog severity
[-\w]+ \(.*\) (?:\w+|baseline|unstable|experimental); urgency=(?:low|medium|high|emergency|critical)\b

# hit-count: 1 file-count: 1
# base64 encoded content, possibly wrapped in mime
(?:^|[\s=;:?])[-a-zA-Z=;:/0-9+]{50,}(?:[\s=;:?]|$)

# hit-count: 1 file-count: 1
# Compiler flags (Unix, Java/Scala)
# Use if you have things like `-Pdocker` and want to treat them as `docker`
(?:^|[\t ,>"'`=(])-(?:(?:J-|)[DPWXY]|[Llf])(?=[A-Z]{2,}|[A-Z][a-z]|[a-z]{2,})

# hit-count: 1 file-count: 1
# Compiler flags (Windows / PowerShell)
# This is a subset of the more general compiler flags pattern.
# It avoids matching `-Path` to prevent it from being treated as `ath`
(?:^|[\t ,"'`=(])-(?:[DPL](?=[A-Z]{2,})|[WXYlf](?=[A-Z]{2,}|[A-Z][a-z]|[a-z]{2,}))

Alternatively, if a pattern suggestion doesn't make sense for this project, add a # to the beginning of the line in the candidates file with the pattern to stop suggesting it.

Warnings and Notices ⚠️ (4)

See the 📂 files view, the 📜action log, 👼 SARIF report, or 📝 job summary for details.

⚠️ Warnings and Notices Count
ℹ️ candidate-pattern 19
⚠️ ignored-expect-variant 2
⚠️ non-alpha-in-dictionary 1
ℹ️ unused-config-file 1

See ⚠️ Event descriptions for more information.

If the flagged items are 🤯 false positives

If items relate to a ...

  • binary file (or some other file you wouldn't want to check at all).

    Please add a file path to the excludes.txt file matching the containing file.

    File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

    ^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

  • well-formed pattern.

    If you can write a pattern that would match it,
    try adding it to the patterns.txt file.

    Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

    Note that patterns can't match multiline strings.

🚂 If you're seeing this message and your PR is from a branch that doesn't have check-spelling,
please merge to your PR's base branch to get the version configured for your repository.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Both new error paths return negative byte counts, violating the reader contract and potentially panicking buffered consumers.

2 open findings
What changed in this PR

Stops storage read retries immediately on HTTP 507 errors.

Changes:

  • Adds a helper to identify non-retryable 507 errors.
  • Applies early returns during restart and read operations.
  • Adds tests for both failure paths.
File Description
pkg/​storage/​utils.go Classifies 507 request failures.
pkg/​proc/​yio/​yrreader.go Stops retries on classified errors.
pkg/​proc/​yio/​yrreader_test.go Tests immediate failure without retries.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pkg/proc/yio/yrreader.go
if err != nil {
if storage.IsNonRetryableError(err) {
ylogger.Zero.Error().Err(err).Int64("offset reached", y.offsetReached).Msg("object does not exist in external storage, giving up")
return -1, err

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered.

Source: https://pkg.go.dev/io#Reader

Comment thread pkg/proc/yio/yrreader.go
if err != nil {
if storage.IsNonRetryableError(err) {
ylogger.Zero.Error().Err(err).Int64("offset reached", y.offsetReached).Msg("object does not exist in external storage, giving up")
return -1, err
@Vlasdislav

Copy link
Copy Markdown
Contributor

I think to need addition to description 507 code is not standard, but contract from s3 (s3 use free code and assign a meaning)

yr := yio.NewYRetryReader(rr, ycl)

err507 := awserr.NewRequestFailure(
awserr.New("InsufficientStorage", "object does not exist", nil),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Message incorrect. Return code from s3 use other semantic not equal http standard.

Comment thread pkg/storage/utils.go
Comment on lines +25 to +33
const nonRetryableStatusCode = 507

func IsNonRetryableError(err error) bool {
var reqErr awserr.RequestFailure
if errors.As(err, &reqErr) {
return reqErr.StatusCode() == nonRetryableStatusCode
}
return false
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name IsNonRetryableError has broad meaning, but return code means "object does not exists for sure". We use const only into function I think we need const move to function body.

Suggested change
const nonRetryableStatusCode = 507
func IsNonRetryableError(err error) bool {
var reqErr awserr.RequestFailure
if errors.As(err, &reqErr) {
return reqErr.StatusCode() == nonRetryableStatusCode
}
return false
}
func IsMissingObjectError(err error) bool {
const missingObjectStatusCode = 507
var reqErr awserr.RequestFailure
return errors.As(err, &reqErr) &&
reqErr.StatusCode() == missingObjectStatusCode
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe without const better. We see code number and function name, and know s3 contract.

Suggested change
const nonRetryableStatusCode = 507
func IsNonRetryableError(err error) bool {
var reqErr awserr.RequestFailure
if errors.As(err, &reqErr) {
return reqErr.StatusCode() == nonRetryableStatusCode
}
return false
}
func IsMissingObjectError(err error) bool {
var reqErr awserr.RequestFailure
return errors.As(err, &reqErr) &&
reqErr.StatusCode() == 507
}

507, "request-id")

/* no retry on 507: single Restart attempt, then error is returned */
rr.EXPECT().Restart(int64(0)).Return(err507).Times(1)

@Vlasdislav Vlasdislav Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we want to compare the case with a repeat and without a repeat, but with error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants