Skip to content

feat: add ItemBank for storing and reusing calibrated items - #25

Merged
kholdrex merged 3 commits into
mainfrom
feat/item-bank
Oct 11, 2026
Merged

kholdrex merged 3 commits into
mainfrom
feat/item-bank

Conversation

@kholdrex

Copy link
Copy Markdown
Member

Why

Fit results are positional arrays tied to one in-memory model. An app that wants to work with new respondents (for example, a Rails assessment app) has to either recalibrate or invent its own storage for item parameters. This is step 2 of the roadmap and the base for the next step: scoring new respondents (EAP/MAP) against fixed items.

What

IrtRuby::ItemBank is an immutable, id-keyed set of calibrated items. It uses the logistic metric without a scaling constant, the same as R's mirt: P = c + (1 - c) / (1 + exp(-a (theta - b))).

bank = IrtRuby::ItemBank.from_fit(results, item_ids: %w[q1 q2 q3 q4], metadata: { source: "spring pilot" })
File.write("item_bank.json", bank.to_json)
bank = IrtRuby::ItemBank.from_json(File.read("item_bank.json"))
bank.fetch("q2")             # Item: id, a, b, c
bank.probability("q2", 0.5)
  • Versioned JSON format. format: "irt_ruby.item_bank", format_version: 1, model, items {id, a, b, c}, optional ability_distribution: {mean, sd}, and free-form metadata. Unknown formats and newer versions are rejected.
  • from_fit. Infers the model from the result keys. Records provenance (gem version, scale convention, converged, iterations) and the calibration sample's ability distribution. It never invents values for estimated parameters. Items with nil estimates raise an error unless exclude_unestimated: true is passed, in which case they are listed under metadata["unestimated_items"].
  • Per-model validation. Rasch requires a = 1 (stored exactly), c = 0. 2PL requires c = 0. 3PL requires 0 ≤ c < 1. Numbers must be finite and ids must be unique non-empty strings.
  • Immutability and equality. The bank, its items and its metadata are deep-frozen and defensively copied. Floats round-trip through JSON exactly. ==, eql? and hash are consistent.
  • README. New "Item Banks" section documenting the format, plus an export recipe for R's mirt: it uses coef(..., IRTpars = TRUE, simplify = TRUE) and jsonlite with digits = I(17) for exact doubles, and exports the latent mean and variance.

Verification

  • 326 specs pass on Ruby 2.6.10 (json 2.1), 3.3 and 4.0.6. RuboCop 1.92.0 is clean.
  • New spec/irt_ruby/item_bank_spec.rb covers building from all three models, JSON round-trip and format, validation, immutability, equality and probabilities. A README example spec is also added.
  • The R recipe was checked against the mirt and jsonlite documentation and source, but has not been run in R. A parity check against mirt is the next roadmap step.

Fit results are positional arrays tied to one in-memory model, so an app
that wants to work with new respondents has to recalibrate or invent its
own storage. ItemBank is an immutable, id-keyed set of item parameters
with a versioned JSON format, per-model validation, recorded calibration
provenance, and response probabilities, so a bank calibrated once (here
or in R's mirt) can be loaded anywhere. It is the base for scoring new
respondents against fixed items.
Review found that callers could mutate the model name, from_fit invented
a = 1 / c = 0 for missing estimates, eql? disagreed with hash, huge
integers became Infinity after validation, excluded items skipped id
checks, and Rasch slopes near 1 were stored unnormalized. Scoring new
respondents will also need the calibration ability distribution, which
the format could not carry, so add an optional ability_distribution now
rather than bump format_version later. The mirt recipe now writes 17
significant digits and exports the latent mean and variance.
Integer abilities were averaged with integer division, non-finite ones
silently dropped the ability distribution, and a non-Array difficulties
value raised NoMethodError instead of ArgumentError.
@kholdrex
kholdrex merged commit 43550e3 into main Oct 11, 2026
5 checks passed
@kholdrex
kholdrex deleted the feat/item-bank branch October 11, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant