Return a different exit code for all commands when the cli fails to locate the metaschema

[Karan-Palan]

ref #152

Screencast.from.2026-01-28.23-32-51.mp4

[RitoG09]

Should lint,format still work even if the metaschema itself is invalid?
And currently the metaschema command returns no exit code when the metaschema is invalid:

@jviotti @Karan-Palan Could you elaborate on how you’d like to handle this?

[jviotti]

So this is an issue we need to fix at the CLI level and it applies to most commands. The idea is that there are different categories of reasons why commands fail:

• Is the actual operation reporting "failures" even though its execution was fine? i.e. the linter displaying real linting errors is an error, but it is an expected error, in that the linter did what it was supposed to do
• Did the operation fail because a pre-requisite wasn't met? For example, if you trying to bundle a schema but one of the references are wrong, that's some sort of logical error. The tool didn't break, and the schemas are not "incorrect", but maybe you forgot to --resolve a schema, or enable --http or whatever.
• Did the operation fail because it couldn't even start because of a major error? This is where i.e. not finding the meta-schema means we can't really even start to think how to proceed with the operation
• Did the operation fail because of a bug in the tool? Like it crashed, or segfaulted or whatever? This is clearly our fault

For all of those cases, we want the CLI to exit with different exit codes, so that the VS Code extension can then handle them accordingly.

If you guys want to help with this, maybe the first step is to map out the various exit paths from all the commands, categorise them a bit, see what exit codes we return right now, and try to find the delta. Once we have a plan for which case should return what exit code for each command, updating the CLI is probably trivial.

[RitoG09]

I go through lint, fmt, validate, and metaschema all four commands already encode the same core idea via the final throw Fail{2} block :

Category 1 -> Expected operational failures (command ran correctly)

These are result-level failures where execution succeeded, but the outcome is negative.

• lint:
  • Lint rules executed, diagnostics produced
  • Exit code: 2
• fmt:
  • --check mode, formatting differences detected
  • Exit code: 2
• validate:
  • Instance does not conform to schema
  • Exit code: 2
• metaschema:
  • Schema does not validate against its metaschema
  • Exit code: 2

This category is already consistently implemented across all four commands.

Category 2 -> Pre-requisite failures

The command cannot proceed due to invalid usage or unsupported inputs.

Examples:

• invalid or missing positional arguments
• unsupported input types (e.g. YAML in fmt)
• invalid option combinations

Current behavior (all commands):

• Exception thrown
• Exit code: 1

Category 3 -> Can't start failures

The command cannot even start its core operation due to logical preconditions not being met.

Examples:

• metaschema cannot be located or resolved
• metaschema itself is invalid or cannot be compiled
• $ref resolution failures
• unknown or undetectable base dialect

Current behavior (all commands):

• Exception thrown
• Exit code: 1

Semantically, these are distinct from both validation/linting failures and from tool bugs, but they are currently collapsed into the same exit code.

Category 4 -> Tool bugs failures

Internal errors where the tool itself fails (asserts, unexpected states).

Current behavior:

• Exit code: 1

All four commands already consistently distinguish Category 1 using exit code 2. The remaining gap is that Category 2 and Category 3 are currently indistinguishable at the exit-code level.

[jviotti]

Good work!

The remaining gap is that Category 2 and Category 3 are currently indistinguishable at the exit-code level.

That seems correct. Feel free to send a PR correcting this on the CLI. You can use the style we do for returning code 2 for the category 1.