Skip to content

feat(cli): report the real exit code and really stop what uipath server runs - #1842

Draft
robert-ursu wants to merge 2 commits into
mainfrom
feat/python-job-cancellation
Draft

robert-ursu wants to merge 2 commits into
mainfrom
feat/python-job-cancellation

Conversation

@robert-ursu

@robert-ursu robert-ursu commented Aug 4, 2026 •

Copy link
Copy Markdown
Collaborator

Pairs with UiPath/hdens#7868, which makes the handler send the stop, act on the answer, and kill and restart the server when a job cannot be stopped.

Why

Exit codes

A failed execution inside uipath server was reported as a success, on both transports:

  1. ConsoleLogger.error ends in click.get_current_context().exit(1). Under standalone_mode=False click returns that code instead of raising (click/core.py, except Exit). _run_command_isolated took the return value as the result and hard-coded ExitCode: 0. Every console.error path (runtime errors, unexpected exceptions, bad --simulation) came back as exit code 0.
  2. The HTTP /jobs/{key}/start reply had no exitCode. The .NET PythonRuntimeClient reads exitCode into an int and treats a missing one as 0, so every HTTP 200 counted as a success, even when the body said success: false.

The handler usually overrides the exit code with the status in output.json or the IPC result. The exit code decides the outcome when neither exists: failures before or outside UiPathRuntimeContext, and crashes.

Stop

uipath server could not stop what it was running:

  • IPC StopJob was a no-op that answered True.
  • HTTP had no stop route at all.
  • Cancelling the awaiting task was worse than doing nothing. When the handler's IPC connection drops, uipath-ipc cancels every in-flight handler (uipath_ipc/client/connection.py, _teardown). await asyncio.to_thread(...) raised, the lock was released and env/cwd were restored while the execution's thread kept running. The next execution then ran concurrently in a mutated process.

What: exit codes

  • _server_core: an int returned by cmd.main is the exit code. run, debug and eval never return an int of their own.
  • HTTP start reply: exitCode is added to every response. An unchanged handler becomes correct against this runtime with no .NET change.
  • _job_api: a result status the sink does not know is reported as Faulted, not Successful. "stopped" maps to the wire's Stopped (4), which the handler already accepts.
  • uipath run / uipath eval: entrypoint or eval-set discovery failure exits 1 instead of 0.

What: how an execution is stopped

run, debug and eval each drive their own event loop on the worker thread, so an execution is an event loop. They now call run_execution_loop instead of asyncio.run. It publishes the loop and its root task to the ExecutionControl carried on a ContextVar. Outside the server (uipath run on a terminal) it is asyncio.run unchanged.

stop_execution(job_key, resume_version, force) is shared by IPC StopJob and the new POST /jobs/{key}/stop (200 {"stopped": bool}).

Both a stop and a kill are cooperative and run the same sequence; a kill (forceStop: true) only has tighter windows:

  1. Cancel the execution's root task. The runtime unwinds cooperatively, and UiPathRuntimeContext.__exit__ still writes the result. Wait 30 s, or 5 s for a kill.
  2. Cancel every task on the execution's loop. This gives up on a clean cleanup. Wait 10 s, or 5 s for a kill.
  3. Answer False if the execution still runs. It is typically blocked inside a call that cancellation cannot reach (a sync socket read, a lock wait, a busy loop with no await), and only ending the process stops it. Whether to do that is the handler's call, since the process and the execution instance belong to it.

So a stop answers within about 40 s and a kill within about 10 s, both within the handler's 60 s StopJobTimeout.

Cases that answer True:

  • A queued execution (waiting for the lock) is dropped before it runs.
  • An unknown job key, or a live run with a different resume version, is not running. Answering False there would invite the handler to kill a resumed run for a stop aimed at the previous one.

The lock waits for the thread

_run_command_isolated now returns or raises only once the worker thread has exited. Cancelling the caller, for example on a dropped connection, stops the execution. The CancelledError is re-raised only after that, so the lock, env and cwd are never handed on while the job still runs.

The job-scope teardown also completes before the env is restored, so the xfail in test_server_job_core.py is removed.

Outcomes

How the execution ended ExitCode Error
Stopped on request 143 Stopped on request
Its own code let a CancelledError escape 1 (unexpected) The execution cancelled itself
Otherwise click's exit code, 0 on success Exit code: N

A stopped execution's result document still says Faulted/ERROR_CancelledError; that is written by uipath-runtime. The handler reports Stopped for an Orchestrator stop regardless of what the runtime wrote.

Worth a careful look

  • The worker is a plain run_in_executor future, not a task. A task re-raises the execution's SystemExit into the server loop, and _run_command_isolated turns SystemExit into an exit code.
  • An execution waiting on its own to_thread work (sync LangGraph nodes, sync HTTP clients) runs its finally at once. But the execution's asyncio.Runner does not close until that call returns. run_execution_loop withdraws the loop before the runner closes, so the sweep in step 2 cannot cancel the runner's wait on those threads, which would leak them. Such an execution answers False if the call outlives the grace.
  • ExecutionControl.cancel() delivers at most once. A stop followed by a force stop is ordinary, and a second delivery would land inside the cleanup that writes output.json.

Testing

  • uv run pytest tests/cli: 1521 passed, 1 skipped, with coverage on as in CI.
  • ruff check, ruff format --check and mypy are clean.
  • New tests are in tests/cli/test_server_cancellation.py (18). They use click commands shaped like the real ones and cover:
    • a running execution stops, its cleanup runs and the lock is freed;
    • a stop that arrives before the execution has a loop is applied when it gets one;
    • a repeated or forced stop does not abort a slow cleanup;
    • a force stop gives up within its own, shorter window;
    • an execution waiting on a nested thread stops once that call returns;
    • an uninterruptible execution answers False and keeps the lock until its thread exits;
    • a self-inflicted cancel is a fault;
    • a queued execution is dropped, and cancelling a queued execution's caller does not leak the lock;
    • a stop for another resume version leaves the live run alone;
    • a cancelled caller stops the execution and waits for its thread, keeping the lock and the execution's env meanwhile;
    • IPC StopJob stops a running RunJob;
    • the HTTP stop route stops a running /start and validates its body.
  • Exit-code tests:
    • a real click command that calls ConsoleLogger.error reports exit code 1 through _run_command_isolated;
    • an execution started over HTTP whose entrypoint raises replies success: false, exitCode: 1;
    • an unknown result status maps to Faulted, and "stopped" to Stopped;
    • discovery failures exit 1.
  • Version 2.14.25 → 2.14.26.

🤖 Generated with Claude Code

@github-actions github-actions Bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-integrations labels Aug 4, 2026
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch 2 times, most recently from ffcf810 to bca74f9 Compare August 4, 2026 11:09
@robert-ursu
robert-ursu force-pushed the feat/async-job-dispatch-and-result-push branch from 114a204 to 71c81e9 Compare August 4, 2026 11:13
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from bca74f9 to b05d195 Compare August 4, 2026 11:18
@robert-ursu robert-ursu changed the title feat(cli): really stop a job that is already executing feat(cli): really stop a job that is already executing [PC-4873] Aug 4, 2026
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from b05d195 to 1df44eb Compare August 4, 2026 16:00
@robert-ursu
robert-ursu force-pushed the feat/async-job-dispatch-and-result-push branch from 6c8af15 to e509941 Compare August 19, 2026 19:13
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from 1df44eb to e14f4b2 Compare August 19, 2026 19:13
@robert-ursu
robert-ursu force-pushed the feat/async-job-dispatch-and-result-push branch from e509941 to db4653c Compare September 24, 2026 15:12
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from e14f4b2 to 50e0cc7 Compare September 24, 2026 15:26
@robert-ursu robert-ursu changed the title feat(cli): really stop a job that is already executing [PC-4873] feat(cli): really stop a job that uipath server is running [PC-4873] Sep 24, 2026
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from 50e0cc7 to 0654cec Compare September 28, 2026 08:18
@robert-ursu robert-ursu changed the title feat(cli): really stop a job that uipath server is running [PC-4873] feat(cli): report a job's real exit code and really stop running jobs [PC-4873] Sep 28, 2026
@robert-ursu
robert-ursu changed the base branch from feat/async-job-dispatch-and-result-push to main September 28, 2026 08:19
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from 0654cec to 7e4af34 Compare September 28, 2026 08:29
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from 7e4af34 to 5c4ea34 Compare September 28, 2026 11:33
@robert-ursu robert-ursu changed the title feat(cli): report a job's real exit code and really stop running jobs [PC-4873] feat(cli): report a job's real exit code and really stop running jobs Sep 28, 2026
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from 5c4ea34 to 37cc492 Compare September 28, 2026 11:46
@github-actions

Copy link
Copy Markdown

🚨 Heads up: uipath-integrations cross-tests are FAILING 🚨

Your changes may break one or more integrations in uipath-integrations-python:

  • uipath-openai-agents
  • uipath-google-adk
  • uipath-agent-framework
  • uipath-llamaindex
  • uipath-pydantic-ai

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →

Under standalone_mode=False click returns ctx.exit(N)'s code instead of
raising, so every ConsoleLogger.error path came back as a result and
_run_command_isolated reported ExitCode 0. Map a returned int to the exit
code.

The HTTP start reply now carries exitCode, the field the .NET HTTP client
already reads (it defaulted a missing one to 0, so every 200 was a success).

StopJob returns False: this server cannot stop what it runs, and the handler
must not be told it did.

A result status the IPC sink does not know is reported as Faulted instead of
Successful, and "stopped" maps to the wire's Stopped.

Entrypoint discovery failures in `uipath run` and `uipath eval` exit 1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
run/debug/eval drive their own event loop on the server's worker thread. They
now publish it through run_execution_loop to an ExecutionControl, so a stop
cancels the execution's root task and the runtime unwinds cooperatively,
still writing its result.

stop_execution backs both IPC StopJob and the new POST /jobs/{key}/stop. It
cancels the root task, waits a grace period, cancels every task on the
execution's loop, and answers False if it is still running, because it is
blocked in a call that only ending the process can interrupt. forceStop
shortens the waits. A queued execution is dropped before it runs. A stop that
targets another resume version, or an unknown key, answers True: that run is
not running.

The lock, env and cwd are no longer handed on while the execution's thread
still runs. A cancelled caller (a dropped IPC connection, a shutdown) stops
the execution and re-raises only once the thread has exited, and the job
scope's teardown completes before the env is restored.

A stopped execution ends with exit code 143 and "Stopped on request"; a
CancelledError it raised on its own is reported as an unexpected failure.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from 195d328 to 3c195d6 Compare September 28, 2026 15:05
@robert-ursu robert-ursu changed the title feat(cli): report a job's real exit code and really stop running jobs feat(cli): report the real exit code and really stop what uipath server runs Sep 28, 2026
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-integrations test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant