Skip to content

fix: correct Bedrock us. regional premium & cache-read rates - #34

Merged
samuelboland merged 2 commits into
mainfrom
dux-rubyllm2-w1-07a-regional-prices
Oct 14, 2026
Merged

samuelboland merged 2 commits into
mainfrom
dux-rubyllm2-w1-07a-regional-prices

Conversation

@staging-supernova-dx-appf-io

@staging-supernova-dx-appf-io staging-supernova-dx-appf-io Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

What this does

Two commits.

1. Price us. Claude ids at the regional rate and fix Sonnet 5.5 cache reads

Supernova prices every LLM call from this gem's lib/ruby_llm/models.json, and calls Claude on Bedrock only through us. inference profiles. Anthropic's pricing page (read 14 Oct 2026) says regional and multi-region endpoints for Claude 4.5 and later carry a 10% premium over global endpoints. The registry priced the us. ids at the global rate (the eu. ids already had the premium), so Supernova's Claude costs ran 10% low. The same page prices Sonnet 5.5 and Opus 5.5 cache hits at 0.05x input; the registry had Sonnet 5.5 at 0.1x.

Changes to lib/ruby_llm/models.json, in both pricing.text_tokens.standard and metadata.cost, written as exact decimal literals:

  • 12 us. ids set to global price x 1.1 (input / output / cache read / cache write per million):
    • Sonnet 4.5, Sonnet 4.6: 3.3 / 16.5 / 0.33 / 4.125
    • Opus 4.5, 4.6, 4.7, 4.8, 5: 5.5 / 27.5 / 0.55 / 6.875
    • Sonnet 5: 2.2 / 11 / 0.22 / 2.75
    • Sonnet 5.5: 2.2 / 11 / 0.11 / 2.75
    • Opus 5.5: 4.4 / 22 / 0.22 / 5.5
    • Fable 5: 11 / 55 / 1.1 / 13.75
    • Fable 5.1: 11 / 55 / 0.275 / 13.75
  • Sonnet 5.5 cache read on the other Bedrock entries Supernova can route to: eu. 0.22 to 0.11, global. and bare anthropic.claude-sonnet-5-5 0.2 to 0.1.

Opus 5.5 and Fable 5.1 already had the right cache multipliers (0.05x and 0.025x); only their us. rows gained the 10%.

Left unchanged on purpose:

  • Haiku 4.5: the AWS bill for 5 to 12 Oct 2026 shows it at 94% of the regional rate, for a reason not yet known, so its global price is closer to what is billed.
  • Opus 4.1 (older than Claude 4.5, no regional premium) and Sonnet 4 (no pricing today).
  • Sonnet 5.5 on au., jp., and the anthropic and vertexai providers. Supernova does not call them.

The AWS bill for 5 to 12 Oct 2026 (Cost Explorer, USW2-MP:USW2_* usage types) shows Sonnet 5 on us. at 2.20 / 11 / 0.22 / 2.75 and Sonnet 5.5 at 2.20 / 11 / 0.11 / 2.75, matching these prices.

Specs: models_sonnet_5_5_spec.rb, models_opus_5_5_spec.rb and models_fable_5_1_spec.rb updated; new models_bedrock_regional_pricing_spec.rb checks both price copies for the 12 us. ids and that Haiku 4.5, Opus 4.1 and Sonnet 4 keep today's values.

How "nothing else changed" was checked: a script parsed models.json before and after and compared every entry field by field. Both sides have 1,238 entries in the same order; exactly 15 bedrock entries and 102 fields changed (12 entries x 8 fields, 3 entries x 2 fields), and every new value matches the list above.

2. Exclude Faraday 2.0.0, which has no default adapter

Every CI test job failed with :net_http is not registered on Faraday::Adapter, on main as well as this branch. faraday-net_http 3.5.0 was released on 2026-10-11. Faraday 2.0.1 to 2.14.4 all require faraday-net_http >= 2.0, < 3.5, but Faraday 2.0.0 declares no faraday-net_http dependency, so with no lockfile Bundler installed Faraday 2.0.0 with faraday-net_http 3.5.0. The gemspec now requires ['>= 1.10.0', '!= 2.0.0'], the same change as upstream crmne/ruby_llm ef2bebdb. spec/ruby_llm_gemspec_spec.rb checks the requirement accepts 1.10.3 and 2.14.4 and rejects 2.0.0.

A companion PR in appfolio/supernova (RubyLLM2 W1-07b) pins Supernova to the commit this PR merges. No other Ruby code changed.

Testing

  • The four pricing specs, models_json_validation_spec.rb and ruby_llm_gemspec_spec.rb pass locally.
  • The full bundle exec rspec suite passes locally on a fresh resolve, and all 21 CI test jobs pass.

Type of change

  • Bug fix
  • No API changes

models.json was edited by hand on purpose: this is a correction to its price data.

🤖 Generated with Claude Code

@staging-supernova-dx-appf-io

Copy link
Copy Markdown
Author

🤖 Supernova Code Review — View trace

🔄 Re-run this review

— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

@staging-supernova-dx-appf-io staging-supernova-dx-appf-io Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the models.json changes and the new and updated specs. I checked every repriced us. entry by hand: input/output/cache-write are each 1.1x the global price, and cache-read uses the right multiplier (0.05x for Sonnet 5.5 and Opus 5.5, 0.025x for Fable 5.1, 0.1x for the rest). Each one matches its spec table, and metadata.cost agrees with pricing.text_tokens.standard everywhere.

One finding (inline): the Sonnet 5.5 cache-read fix only covers the Bedrock bare, global., eu. and us. entries. The anthropic and vertexai entries are still at 0.1x (0.20), even though the PR says that bug isn't region-specific. The same goes for the au./jp. Bedrock entries, which also lack the 10% regional premium. spec/ruby_llm/models_sonnet_5_5_spec.rb lines 27–39 now assert those known-wrong values.

🔄 Re-run this review

— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

Comment on lines 27 to 39
standard = { input: 2, output: 10, cache_read: 0.20, cache_write: 2.5 }
eu = { input: 2.2, output: 11, cache_read: 0.22, cache_write: 2.75 }
global = { input: 2, output: 10, cache_read: 0.1, cache_write: 2.5 }
regional = { input: 2.2, output: 11, cache_read: 0.11, cache_write: 2.75 }

{
%w[anthropic claude-sonnet-5-5] => standard,
%w[vertexai claude-sonnet-5-5] => standard,
%w[bedrock anthropic.claude-sonnet-5-5] => standard,
%w[bedrock global.anthropic.claude-sonnet-5-5] => standard,
%w[bedrock us.anthropic.claude-sonnet-5-5] => standard,
%w[bedrock eu.anthropic.claude-sonnet-5-5] => eu,
%w[bedrock anthropic.claude-sonnet-5-5] => global,
%w[bedrock global.anthropic.claude-sonnet-5-5] => global,
%w[bedrock us.anthropic.claude-sonnet-5-5] => regional,
%w[bedrock eu.anthropic.claude-sonnet-5-5] => regional,
%w[bedrock au.anthropic.claude-sonnet-5-5] => standard,
%w[bedrock jp.anthropic.claude-sonnet-5-5] => standard

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spec locks in Sonnet 5.5 cache-read prices that the PR's own source says are wrong. The anthropic, vertexai, au. and jp. entries stay at cache_read: 0.20, which is the 0.1x multiplier, but the footnote quoted on lines 12–13 says Sonnet 5.5 cache hits are 0.05x on every endpoint. The PR description also says the bug "wasn't region-specific." So anyone using the first-party Anthropic or Vertex provider gets cache reads costed at 2x the real price. au./jp. are regional endpoints too, so they should be at 2.2 / 11 / 0.11 / 2.75, the same as us./eu.. "Not called by Supernova" is a reason about one consumer, but this registry ships to every RubyLLM user. Fix: set cache_read to 0.1 on the anthropic/vertexai entries, both metadata.cost and pricing.text_tokens.standard. Then either move au./jp. to the regional rate or leave them out of this spec, so it doesn't assert values we know are wrong.

Anthropic's pricing page (read 14 Oct 2026) says regional and multi-region
Bedrock endpoints carry a 10% premium over global endpoints, and that
Sonnet 5.5 and Opus 5.5 cache hits are 0.05x input. The registry priced
us. ids at the global rate and Sonnet 5.5 cache reads at 0.1x. The AWS bill
for 5-12 Oct 2026 shows us. Sonnet 5 at 2.20/11/0.22/2.75 and us. Sonnet 5.5
at 2.20/11/0.11/2.75, which matches the regional rule.

Set 12 us. Claude entries to the regional price in both pricing fields, and
set the eu., global. and bare bedrock Sonnet 5.5 cache reads to 0.05x input.
Haiku 4.5, Opus 4.1 and Sonnet 4 are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-authored-by: Sam Boland <sam.boland@appfolio.com>
@staging-supernova-dx-appf-io
staging-supernova-dx-appf-io Bot force-pushed the dux-rubyllm2-w1-07a-regional-prices branch from 4c2ad34 to 0d10799 Compare October 14, 2026 19:21
@samuelboland

Copy link
Copy Markdown

Review of 4c2ad34. The data and specs are right. One change is needed before this can merge, because every test job fails for a reason outside this PR.

  1. CI: exclude Faraday 2.0.0 in ruby_llm.gemspec (separate commit). All 21 test jobs fail with :net_http is not registered on Faraday::Adapter (432 of 1,029 examples in the 3.2 / rails-7.1 job). faraday-net_http 3.5.0 was released on 2026-10-11. Every Faraday from 2.0.1 to 2.14.4 requires faraday-net_http >= 2.0, < 3.5, but Faraday 2.0.0 declares no faraday-net_http dependency. With no lockfile under gemfiles/, Bundler keeps the newest faraday-net_http (3.5.0) and falls back to Faraday 2.0.0, which has no adapter registered. The job logs show Installing faraday 2.0.0 and Installing faraday-net_http 3.5.0. The last Main CI run on main was 2026-10-06, before the release, so main is broken too and nobody has seen it yet.

    Upstream fixed this in crmne/ruby_llm ef2bebdb ("Exclude Faraday 2.0.0, which has no default adapter"). Apply the same change:

    • ruby_llm.gemspec line 61:
      spec.add_dependency 'faraday', ENV['FARADAY_VERSION'] || ['>= 1.10.0', '!= 2.0.0']
    • spec/ruby_llm_gemspec_spec.rb: the first example asserts requirement.to_s equals '>= 1.10.0', which will no longer hold. Replace its body with expect(runtime_dependency('faraday').requirement).to be_satisfied_by(Gem::Version.new('1.10.3')), as upstream did, and add:
      it 'excludes Faraday 2.0.0, which ships without its default adapter' do
        requirement = runtime_dependency('faraday').requirement
      
        expect(requirement).not_to be_satisfied_by(Gem::Version.new('2.0.0'))
        expect(requirement).to be_satisfied_by(Gem::Version.new('2.14.4'))
      end
    • Make it its own commit, with a message explaining the faraday-net_http 3.5.0 cause and naming upstream ef2bebdb. Run spec/ruby_llm_gemspec_spec.rb plus the four pricing specs.
  2. Bot inline note at spec/ruby_llm/models_sonnet_5_5_spec.rb:39 (anthropic, vertexai, au., jp. still at 0.1x): overruled, no change. Leaving those entries alone was decided for this brief. Supernova calls none of them, and the brief keeps them out of scope. The spec comment at lines 17–20 already says they keep their earlier values on purpose.

Verified and fine:

  • I diffed the parsed models.json at dcfc5492 and at this head: 1,238 entries on both sides, with no entries added or removed. Exactly 15 (id, provider) entries changed, all bedrock: the 12 us. ids in the table (8 fields each) plus eu., global. and bare anthropic.claude-sonnet-5-5 (cache read only, 2 fields each), for 102 fields in total. Every value matches the table, in both pricing.text_tokens.standard and metadata.cost.
  • Haiku 4.5, Opus 4.1 and Sonnet 4 are unchanged.
  • The new models_bedrock_regional_pricing_spec.rb covers the 12 ids and the 3 unchanged ones, looks entries up by raw id and provider == 'bedrock', and carries the source quotes. The three updated specs and their comments match the brief.
  • RuboCop, CodeQL and gitleaks pass.

I will correct the PR description myself after your push (it says 13 repriced us. ids where there are 12, and it says Opus 5.5 and Fable 5.1 cache-read multipliers were fixed when only their 1.1x changed).

@staging-supernova-dx-appf-io

Copy link
Copy Markdown
Author

@samuelboland

Review of 4c2ad34. The data and specs are right. One change is needed before this can merge, because every test job fails for a reason outside this PR.

  1. CI: exclude Faraday 2.0.0 in ruby_llm.gemspec (separate commit). All 21 test jobs fail with :net_http is not registered on Faraday::Adapter (432 of 1,029 examples in the 3.2 / rails-7.1 job). faraday-net_http 3.5.0 was released on 2026-10-11. Every Faraday from 2.0.1 to 2.14.4 requires faraday-net_http >= 2.0, < 3.5, but Faraday 2.0.0 declares no faraday-net_http dependency. With no lockfile under gemfiles/, Bundler keeps the newest faraday-net_http (3.5.0) and falls back to Faraday 2.0.0, which has no adapter registered. The job logs show Installing faraday 2.0.0 and Installing faraday-net_http 3.5.0. The last Main CI run on main was 2026-10-06, before the release, so main is broken too and nobody has seen it yet.

    Upstream fixed this in crmne/ruby_llm ef2bebdb ("Exclude Faraday 2.0.0, which has no default adapter"). Apply the same change:

    • ruby_llm.gemspec line 61:
      spec.add_dependency 'faraday', ENV['FARADAY_VERSION'] || ['>= 1.10.0', '!= 2.0.0']
    • spec/ruby_llm_gemspec_spec.rb: the first example asserts requirement.to_s equals '>= 1.10.0', which will no longer hold. Replace its body with expect(runtime_dependency('faraday').requirement).to be_satisfied_by(Gem::Version.new('1.10.3')), as upstream did, and add:
      it 'excludes Faraday 2.0.0, which ships without its default adapter' do
        requirement = runtime_dependency('faraday').requirement
      
        expect(requirement).not_to be_satisfied_by(Gem::Version.new('2.0.0'))
        expect(requirement).to be_satisfied_by(Gem::Version.new('2.14.4'))
      end
    • Make it its own commit, with a message explaining the faraday-net_http 3.5.0 cause and naming upstream ef2bebdb. Run spec/ruby_llm_gemspec_spec.rb plus the four pricing specs.
  2. Bot inline note at spec/ruby_llm/models_sonnet_5_5_spec.rb:39 (anthropic, vertexai, au., jp. still at 0.1x): overruled, no change. Leaving those entries alone was decided for this brief. Supernova calls none of them, and the brief keeps them out of scope. The spec comment at lines 17–20 already says they keep their earlier values on purpose.

Verified and fine:

  • I diffed the parsed models.json at dcfc5492 and at this head: 1,238 entries on both sides, with no entries added or removed. Exactly 15 (id, provider) entries changed, all bedrock: the 12 us. ids in the table (8 fields each) plus eu., global. and bare anthropic.claude-sonnet-5-5 (cache read only, 2 fields each), for 102 fields in total. Every value matches the table, in both pricing.text_tokens.standard and metadata.cost.
  • Haiku 4.5, Opus 4.1 and Sonnet 4 are unchanged.
  • The new models_bedrock_regional_pricing_spec.rb covers the 12 ids and the 3 unchanged ones, looks entries up by raw id and provider == 'bedrock', and carries the source quotes. The three updated specs and their comments match the brief.
  • RuboCop, CodeQL and gitleaks pass.

I will correct the PR description myself after your push (it says 13 repriced us. ids where there are 12, and it says Opus 5.5 and Fable 5.1 cache-read multipliers were fixed when only their 1.1x changed).

I've queued the Faraday fix as a separate commit and am working on it now. It makes three changes:

  • Gemspec: ruby_llm.gemspec line 61 becomes ['>= 1.10.0', '!= 2.0.0'].
  • Specs: in spec/ruby_llm_gemspec_spec.rb, the first example now asserts that 1.10.3 satisfies the requirement. A new example asserts that 2.0.0 is excluded and 2.14.4 is allowed.
  • Commit message: it explains the faraday-net_http 3.5.0 cause and names upstream ef2bebdb.

I'll run only the gemspec spec and the four pricing specs.

I'm leaving the au., jp., Anthropic and Vertex AI entries alone, as you decided.


🔍 View trace

faraday-net_http 3.5.0 was released on 2026-10-11. No Faraday from 2.0.1
to 2.14.4 accepts it: they all require faraday-net_http >= 2.0, < 3.5.
Faraday 2.0.0 declares no faraday-net_http dependency at all, so Bundler
keeps faraday-net_http 3.5.0 and falls back to Faraday 2.0.0. That pair
fails at connection setup with ":net_http is not registered on
Faraday::Adapter", which breaks every CI test job.

Exclude Faraday 2.0.0 in the gemspec so Bundler picks a Faraday 2.x that
pins a compatible faraday-net_http. This is the same change as upstream
crmne/ruby_llm commit ef2bebd ("Exclude Faraday 2.0.0, which has no
default adapter"). The gemspec spec now checks that the requirement accepts
Faraday 1.10.3 and 2.14.4 and rejects 2.0.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@staging-supernova-dx-appf-io

Copy link
Copy Markdown
Author

@samuelboland

Review of 4c2ad34. The data and specs are right. One change is needed before this can merge, because every test job fails for a reason outside this PR.

  1. CI: exclude Faraday 2.0.0 in ruby_llm.gemspec (separate commit). All 21 test jobs fail with :net_http is not registered on Faraday::Adapter (432 of 1,029 examples in the 3.2 / rails-7.1 job). faraday-net_http 3.5.0 was released on 2026-10-11. Every Faraday from 2.0.1 to 2.14.4 requires faraday-net_http >= 2.0, < 3.5, but Faraday 2.0.0 declares no faraday-net_http dependency. With no lockfile under gemfiles/, Bundler keeps the newest faraday-net_http (3.5.0) and falls back to Faraday 2.0.0, which has no adapter registered. The job logs show Installing faraday 2.0.0 and Installing faraday-net_http 3.5.0. The last Main CI run on main was 2026-10-06, before the release, so main is broken too and nobody has seen it yet.

    Upstream fixed this in crmne/ruby_llm ef2bebdb ("Exclude Faraday 2.0.0, which has no de...

Pushed a1d2f6d.

Changed 2 files (+9 −2): ruby_llm.gemspec, spec/ruby_llm_gemspec_spec.rb


View the full session

@samuelboland samuelboland left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: parsed models.json diff shows exactly 15 bedrock entries / 102 fields changed, all matching the regional price table in both copies; Faraday != 2.0.0 matches upstream ef2bebd; all CI test jobs green.

@samuelboland
samuelboland merged commit c273946 into main Oct 14, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants