Repository navigation
Conversation
… result
P='a b c'; set -- ${P:-x}; echo $# # hellish 1, bash 3
set -- ${u:-*.c} # hellish: the word *.c, no glob
POSIX (2.6.2, 2.6.5, 2.6.6): the result of an unquoted parameter
expansion is field-split and pathname-expanded, whichever operator made
it. hellish kept the whole result as one field whenever the operator
word's TEXT held no unquoted blank:
- it looked at the word even when the word was not used, so the
variable's own value was never split (${P:-x}, ${P-x}, ${P:+$P},
${P:?e}, ${U:="a b"});
- it could not see inside expansions, so ${u:-$P} never split and
${u:-$Q}, ${u:-a*} never globbed;
- a word with a blank was split flat, quotes and all:
${u:-"a b"c d} gave three fields, a / bc / d, where bash gives
two, "a bc" / d; and ${u:-a\ b} gave two.
Root cause: expand_op_token retyped the token to TT_DQWORD (one field,
no glob) based on opword_no_split(word text). The quoting of each part
of the word was gone by the time the result reached the splitter.
Now:
- the variable's value, and the new value of ${p:=w}, stay a plain
unquoted expansion: split and globbed exactly like $p;
- a used word of - or + is expanded by pf_op_word_segments
(expand_param_opword.c) into segments that keep their quoting:
'q' for '...', "...", $'...', a backslash escape and a tilde prefix
(POSIX 2.6.1: ${u:-~} is one field even when HOME has a blank);
'u' for everything else, literal text included. The segments are
parked behind a SEG_MAGIC marker, the same way ${a[@]} parks its
elements. The splitter (emit_op_segments) IFS-splits the 'u' ones
into the current field and appends the 'q' ones verbatim, as
TT_DQENVVAR children, so the glob stage escapes them.
- empty results are unchanged: an unquoted empty result is no field
(git-completion's `${a:+"${a[@]}"} ${d:+--git-dir="$d"}`), and
${x:-""} is still one empty field.
opword_no_split is gone; nothing else used it.
tests/scripts/57_opword_fields.sh, graded against bash --posix, covers:
- the value of each operator;
- used words: unquoted, quoted, backslash-escaped, $'...', mixed;
- globs, command substitution, $@ / $* / "$@" in the word;
- empty results next to literal text, :=, tilde with a blank HOME;
- IFS=:, set -f, and the contexts that do not split (quotes,
assignment, case, for).
The binary without this fix diverges on 25 of its 65 lines; this
branch matches bash byte for byte.
Built on fix/quoted-at-no-glob: the quoted segments use its
push_new_dq_child.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
POSIX (2.6.2, 2.6.5, 2.6.6) says the result of an unquoted parameter expansion is field-split and pathname-expanded, whichever operator produced it. hellish kept the whole result as one field whenever the text of the operator word had no unquoted blank. That rule was wrong in three ways:
${P:-x},${P-x},${P:+$P},${P:?e},${U:="a b"}.${u:-$P}never split, and${u:-$Q}and${u:-a*}never globbed.${u:-"a b"c d}gave three fields (a/bc/d) where bash gives two (a bc/d), and${u:-a\ b}gave two.Root cause.
expand_op_tokenretyped the token toTT_DQWORD(one field, no glob) based onopword_no_split(word text). By the time the result reached the splitter, the quoting of each part of the word was gone.Change.
${p:=w}, stay a plain unquoted expansion, split and globbed exactly like$p.-or+is expanded bypf_op_word_segments(expand_param_opword.c) into segments that keep their quoting:qfor'...',"...",$'...', a backslash escape, and a tilde prefix. POSIX 2.6.1 makes${u:-~}one field even when HOME contains a blank.ufor everything else, literal text included.SEG_MAGICmarker, the same way${a[@]}parks its elements. The splitter (emit_op_segments) IFS-splits theusegments into the current field and appends theqsegments verbatim asTT_DQENVVARchildren, so the glob stage escapes them.${a:+"${a[@]}"} ${d:+--git-dir="$d"};${x:-""}is still one empty field.opword_no_splitis gone; nothing else used it.How I verified it
Reproduced first.
tests/scripts/57_opword_fields.shis graded againstbash --posix5.3.9. It covers:$'...', and mixed;$@/$*/"$@"in the word;:=, and a tilde with a blank HOME;IFS=:,set -f, and the contexts that do not split (quotes, assignment,case,for).The binary without this fix diverges on 25 of its 65 lines; this branch matches bash byte for byte.
Local gates on this commit (ASan debug build):
tests/tester: 5363/5363;tests/run_scripts.shagainstbash --posix: 126/126;verify_alloc.sh: identical output on both heaps;alloc_stress.sh: all clean;tests/pty_suite.sh: 111 ok, 6 skipped, 4 failed.plugin_corpus_test, which loads git-completion and its${a:+"${a[@]}"}idioms, passes. None of the four failures is this change:prompt_compat_matrix,prompt_drift_matrixandprompt_jobs_badgeexpect the non-root%/$prompt and get#, because the container runs as root. They fail the same way on develop, and CI runs them as a normal user.hxp_framework_testhit the 420 s per-file limit on this 4-core container. develop's binary takes 440 s for it on the same machine, and CI's runners finish it inside the limit.norminetteis OK on every touched file.Notes / trade-offs
${p:-a b}unquoted. The new result is what bash, dash and POSIX give; quoting the expansion keeps one field in every shell.🤖 Generated with Claude Code
https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
Generated by Claude Code