Add unit testing guide

docs/build-for-developers/cli-intro.md

@@ -14,6 +14,7 @@ developer experience with OpenFn. You can use the OpenFn CLI to:
 - Sync workflows between OpenFn and a local filesystem or GitHub
 - Securely run OpenFn workflows
 - Troubleshoot and debug OpenFn steps
+- [Unit testing job code](/documentation/jobs/unit-testing-jobs) in a standard JavaScript
 - Read and write Collections data
 
 ---

docs/build-for-developers/cli-usage.md

@@ -4,14 +4,16 @@ sidebar_label: Basic usage
 slug: /cli-usage
 ---
 
-This page shows examples for some of the most common usages of the CLI, including:
+This page shows examples for some of the most common usages of the CLI,
+including:
 
 - get help
 - run a job
 - saving the state
 - adjust logging level
 - maintain adaptors repo
 - run a workflow
+- compile job code for unit testing
 - load adaptor documentation
 
 ---
@@ -30,7 +32,8 @@ openfn deploy --help
 
 ### Run a job
 
-To run a single job, you must explicitly specify which adaptor to use - see the [publicly available adaptors](/adaptors).
+To run a single job, you must explicitly specify which adaptor to use - see the
+[publicly available adaptors](/adaptors).
 
 Adaptors are automatically installed if the specified version is not detected.
 
@@ -222,6 +225,64 @@ running workflows via the CLI.
 
 ---
 
+### Compile job code for unit testing
+
+So you want to write unit tests against your job code? See
+[Writing unit tests for your jobs](/documentation/jobs/unit-testing-jobs) for
+the full guide.
+
+#### What "unit testing a job" means
+
+A job is made of two different kinds of code, and only one of them is unit
+testable:
+
+- **Pure JavaScript functions you write and export** - These take input and
+  return output without calling adaptor or library code.
+- **Operations** - Such as `fn`, `http.get`, `each` and the rest of the adaptor
+  API. These need a runtime, a state object and often a live connection to run.
+  You cannot unit test these.
+
+Unit testing a job means taking the pure functions of your job cod and testing
+that they return the correct output for a given input. It does not mean
+_running_ the job and testing the final output.
+
+#### How it works
+
+1. **Compile your project** using `openfn compile`. This compiles your workflows
+   and writes them out as ordinary ES modules.
+2. **Import the compiled functions** into your test file, just like any other
+   native JS module.
+3. **Write tests as usual** against those pure functions.
+
+Compilation is what makes this possible: a job expression is not valid
+JavaScript on its own, so it can't be imported by a test runner until it has
+been compiled. See [Compilation](/documentation/jobs/compilation) for why.
+
+**Compile every workflow in the project, keeping only exported declarations:**
+
+```bash
+openfn compile --exports-only
+```
+
+Compiled files are written to `dist/` as `.mjs` files. Operations are stripped
+out entirely, so what's left is only the helper functions you exported - which
+is exactly the part you can test. Anything you don't export is dropped too, so
+export every helper you want a test to reach.
+
+**Compile a single workflow by name:**
+
+```bash
+openfn compile my-workflow --exports-only
+```
+
+**Recompile whenever a job code changes:**
+
+```bash
+openfn compile --exports-only --watch
+```
+
+Without `--exports-only` you get the full compiled output of the step.
+
 ### Load adaptor documentation
 
 The CLI can list adaptor documentation in the terminal. Note that it has to

docs/jobs/best-practices.md

@@ -76,3 +76,27 @@ individual items and write them to state. That way one bad item won't ruin a
 whole batch, and you know which items succeeded and which failed. You can then
 throw an exception to recognise that the job has failed.
 
+
+## Writing Testable Functions
+
+To make functions in an OpenFn workflow testable, move the real logic (mapping, validation, formatting, filtering) out of operation blocks and into pure helper functions that take plain inputs and return plain outputs, with no dependence on `state`, globals or external systems. Export each helper, and let the operations do nothing more than pass data from state into them:
+
+```js
+// workflows/patient-sync/transform.js
+export function isValid(record) {
+  return Boolean(record.first_name && record.birth_date);
+}
+
+export const mapPatient = record => ({
+  name: `${record.first_name} ${record.last_name ?? ''}`.trim(),
+  dob: record.birth_date,
+  sex: record.gender?.toLowerCase() === 'f' ? 'female' : 'male',
+});
+
+fn(state => ({
+  ...state,
+  data: state.data.filter(isValid).map(mapPatient),
+}));
+```
+
+To test the export helpers [See Unit testing jobs](/documentation/jobs/unit-testing-jobs)

docs/jobs/job-writing-guide.md

@@ -55,6 +55,12 @@ If you're ready to start using the app, take a look at this guide to
 Workflow design is a non-trivial problem, so you might also like to review the
 Workflow [Design Process docs](/documentation/design/design-overview).
 
+As your jobs grow, you'll start writing helper functions inside them - parsing,
+mapping, reformatting. Those helpers can be unit tested like any other
+JavaScript: export them at top level, compile your project with the CLI, and point a test
+runner at the output. See
+[Writing unit tests for your jobs](/documentation/jobs/unit-testing-jobs).
+
 :::info Questions?
 
 If you have any job-writing questions, ask on

docs/jobs/unit-testing-jobs.md

@@ -0,0 +1,211 @@
+---
+sidebar_label: Unit Testing Jobs
+title: Writing unit tests for your jobs
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+Most job code goes like this: fetch some records, reshape them, send them
+somewhere else. But the reshaping bit often grows into complex logic - parsing
+a string into a structured record, mapping local codes onto DHIS2 data
+elements, normalising a dozen date formats into one.
+
+Unit testing that logic helps to validate that the code runs correctly, and
+helps to prevent errors occurring when the code is modified later.
+
+:::info Requirements
+
+You need `@openfn/cli` v1.39.0 or later to compile your job code for testing.
+Check your version with `openfn -v`, and upgrade with
+`npm install -g @openfn/cli`.
+
+You also need an OpenFn project checked out locally:
+
+```bash
+openfn project pull <uuid>
+```
+
+That gives you a folder with `openfn.yaml`, a `workflows/` directory, and one
+`.js` file per step. See [OpenFn Sync](/documentation/sync) for pulling,
+checking out and deploying projects.
+
+:::
+
+
+## Step 1: Export any helper you want to test
+
+The functions that you want to test must be exported:
+```js title="testable code"
+export const FIELDS = ['id', 'name', 'dob'];
+
+export const parseSms = text =>
+  Object.fromEntries(FIELDS.map((f, i) => [f, text.split('#')[i]]));
+
+fn(state => ({ ...state, data: state.data.messages.map(parseSms) }));
+```
+## Step 2: Compile your workflows
+
+From your project root (the folder with `openfn.yaml`):
+
+```bash
+openfn compile --exports-only
+```
+
+```
+[CLI] ✔ Compiled 1 step(s) to /path/to/project/dist
+```
+
+Compiled files land in `dist/`, mirroring your workflow structure. **Output files use the `.mjs` extension.** Node always treats `.mjs` as an ES
+  module, so you don't need `"type": "module"` in your `package.json` for the
+  compiled code to import cleanly.
+
+:::warning Don't commit the generated `.mjs` files
+
+The CLI does not add a `.gitignore` for the compiled directory, so add one
+yourself before your first commit:
+
+```title=".gitignore"
+dist/
+```
+
+The `.mjs` files are build output, derived entirely from your `.js` steps.
+Tracking them gives you noisy diffs and merge conflicts on every edit, and lets
+`dist/` drift out of sync with `workflows/`.
+
+:::
+
+Other useful flags:
+
+```bash
+# Write somewhere other than dist/
+openfn compile --exports-only -o workflows
+
+# Wipe the output folder first
+openfn compile --exports-only --clean
+
+# Just one workflow, by name
+openfn compile sms-parser --exports-only
+```
+Run `openfn compile --help` for the complete list.
+
+You can also set the output folder permanently in `openfn.yaml`:
+
+```yaml title="openfn.yaml"
+dirs:
+  workflows: workflows
+  compiled: workflows
+```
+
+## Step 3: Write a test
+
+We recommend Node's built-in test runner here because it needs no dependencies, but
+nothing about this is Node-specific. You can use any test runner that can import an ES module.
+
+:::tip Name your test files `.test.mjs`
+
+The compiled output is `.mjs` and needs no configuration. Your _test_ files are
+yours, though - if you name them `.js` in a project without `"type": "module"`,
+Node will warn about reparsing them as ES modules. Naming them `.test.mjs`
+avoids the warning without touching your `package.json`.
+
+:::
+<Tabs groupId="write-a-test">
+  <TabItem value="source" label="The job code">
+    ```js title="workflows/sms-parser/parse-message.js"
+    export const FIELDS = ['id', 'name', 'dob', 'weight'];
+
+    export const parseSms = text => {
+      const parts = text.trim().split('#');
+      return FIELDS.reduce((record, field, i) => {
+        record[field] = parts[i]?.trim() ?? null;
+        return record;
+      }, {});
+    };
+
+    fn(state => ({
+      ...state,
+      data: state.data.messages.map(parseSms),
+    }));
+    ```
+  </TabItem>
+  <TabItem value="output" label="The compiled output">
+
+    After `openfn compile --exports-only`:
+
+    ```js title="dist/sms-parser/parse-message.mjs"
+    export const FIELDS = ['id', 'name', 'dob', 'weight'];
+
+    export const parseSms = text => {
+      const parts = text.trim().split('#');
+      return FIELDS.reduce((record, field, i) => {
+        record[field] = parts[i]?.trim() ?? null;
+        return record;
+      }, {});
+    };
+    ```
+    The `fn(...)` operation is gone. Both exports survived.
+  </TabItem>
+  <TabItem value="test" label="The test">
+    Note the import path: it points at `dist/`, **not** at your source file.
+
+    ```js title="test/parse-message.test.mjs"
+    import { test } from 'node:test';
+    import assert from 'node:assert/strict';
+
+    import { parseSms } from '../dist/sms-parser/parse-message.mjs';
+
+    test('parses a well-formed message into a record', () => {
+      assert.deepEqual(parseSms('P-001#Ada Lovelace#1815-12-10#3.2'), {
+        id: 'P-001',
+        name: 'Ada Lovelace',
+        dob: '1815-12-10',
+        weight: '3.2',
+      });
+    });
+    ```
+  </TabItem>
+</Tabs>
+
+
+### Running the test
+
+```bash
+openfn compile --exports-only && node --test
+```
+
+```
+✔ parses a well-formed message into a record (0.9ms)
+✔ trims whitespace around each field (0.1ms)
+✔ fills missing trailing fields with null (0.1ms)
+ℹ tests 1
+ℹ pass 1
+ℹ fail 0
+```
+
+
+## Step 4: Running test in watch mode
+
+Run the compiler in watch mode in one terminal:
+
+```bash
+openfn compile --exports-only --watch
+```
+
+And your test runner in watch mode in another:
+
+```bash
+node --test --watch
+```
+
+Now editing a step recompiles it, which changes a file in `dist/`, which re-runs
+your tests.
+
+## Related pages
+
+- [Compilation](/documentation/jobs/compilation) - what the compiler does and
+  why
+- [Best Practices](/documentation/jobs/best-practices) - writing job code that's
+  worth testing
+- [CLI basic usage](/documentation/cli-usage) - running workflows locally
+- [OpenFn Sync](/documentation/sync) - pulling a project down so you have an
+  `openfn.yaml` to compile against

sidebars-main.js

@@ -52,6 +52,7 @@ module.exports = {
         'jobs/image-handling',
         'jobs/best-practices',
         'jobs/compilation',
+        'jobs/unit-testing-jobs',
         'build-for-developers/security-for-devs',
       ],
     },