Skip to content

FEAT: Add conda packaging recipes and publish tooling for mssql-python - #720

Draft
Jahnvi Thakkar (jahnvi480) wants to merge 69 commits into
mainfrom
jahnvi/conda-onboarding
Draft

FEAT: Add conda packaging recipes and publish tooling for mssql-python#720
Jahnvi Thakkar (jahnvi480) wants to merge 69 commits into
mainfrom
jahnvi/conda-onboarding

Conversation

@jahnvi480

@jahnvi480 Jahnvi Thakkar (jahnvi480) commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Work Item / Issue Reference

AB#47315


Summary

This pull request introduces a complete, standalone Conda build and release pipeline for the mssql-python package, decoupling Conda packaging from the wheel/PyPI release process. The new pipelines ensure that Conda packages are built, validated, and published independently, with robust audit and gating steps to prevent regressions and incomplete releases.

Key changes include:

1. New Conda Build and Release Pipelines

  • Added OneBranchPipelines/conda-build-pipeline.yml: Implements a full, per-platform Conda build pipeline, repackaging prebuilt wheels into Conda packages for all supported OS/architectures, with validation and artifact consolidation.
  • Added OneBranchPipelines/conda-release-pipeline.yml: Introduces a dedicated release pipeline to validate the consolidated Conda packages and optionally publish them to Anaconda.org, with strict gating to ensure completeness and version consistency.

2. Automated Auditing on Pull Requests

  • Added .github/workflows/conda-audit.yml: Sets up a GitHub Actions workflow that builds and audits a Linux Conda package on every relevant PR, automatically failing the PR if regressions are detected in binary dependencies or audit checks.

3. Improved Separation and Safety

  • The Conda pipelines are now fully decoupled from the wheel/PyPI release pipeline, ensuring that Conda artifacts are not mixed or confused with wheel artifacts and that each release path is independently validated and gated.

4. Platform Coverage and Validation

  • The build pipeline covers all major platforms (Windows, macOS Intel/ARM, Linux x86_64/aarch64), using native and cross-builds as appropriate, and includes runtime validation (e.g., import tests, TLS probes) to ensure package integrity.

5. Secure, Gated Publishing

  • Publishing to Anaconda.org is strictly gated behind successful validation and is only enabled when explicitly requested, with secrets handled securely and never exposed in logs.

These changes significantly improve the reliability, auditability, and maintainability of the Conda packaging and release process for mssql-python.

Adds conda recipes for mssql-python and its mssql-python-odbc companion, a local build/test-before-live harness, and a draft OneBranch conda publish stage. Azure SDK deps resolve from the lean 'microsoft' Anaconda channel under --strict-channel-priority so conda-forge's azure-core recipe (which over-declares flask/six -> celery/boto3/botocore) does not bloat the environment. See conda-forge/azure-core-feedstock#71.
Copilot AI lite review requested due to automatic review settings August 14, 2026 08:50
@github-actions github-actions Bot added the pr-size: medium Moderate update size label Aug 14, 2026
Comment thread conda/mssql-python-odbc/meta.yaml Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds initial conda packaging assets for mssql-python (and its mssql-python-odbc companion) so the project can be published to the Microsoft-owned microsoft Anaconda channel, alongside the existing PyPI wheel distribution.

Changes:

  • Adds conda recipes for mssql-python and mssql-python-odbc that repackage existing wheels (no compilation).
  • Adds a local PowerShell harness to build both recipes, create a local channel, and validate import / optional live-connect.
  • Adds a draft OneBranch stage to build/test/publish conda artifacts from signed release artifacts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
conda/onebranch-publish-conda-stage.yml Draft OneBranch stage to build from signed artifacts, gate on smoke tests, and publish to the microsoft Anaconda channel.
conda/mssql-python/meta.yaml Conda recipe for repackaging the mssql-python wheel and depending on a version-locked mssql-python-odbc.
conda/mssql-python-odbc/meta.yaml Conda recipe for repackaging the proprietary driver wheel into a companion conda package.
conda/build_and_test_local.ps1 Local “test-before-live” harness to build, index, install, and smoke test the conda packages.
Suppressed comments (1)

conda/onebranch-publish-conda-stage.yml:133

  • Same string-vs-boolean condition issue here: quoting the template expression turns it into a string, which can cause the publish step to be skipped unexpectedly even when publishToConda is true.
          - task: PowerShell@2
            displayName: 'Publish to anaconda.org/microsoft'
            condition: and(succeeded(), eq('${{ parameters.publishToConda }}', true))

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread conda/mssql-python/meta.yaml Outdated
Comment thread conda/mssql-python-odbc/meta.yaml Outdated
Comment thread conda/build_and_test_local.ps1 Outdated
Comment thread conda/onebranch-publish-conda-stage.yml Outdated
Comment thread conda/mssql-python-odbc/meta.yaml Outdated
Comment thread conda/build_and_test_local.ps1 Outdated
Recipes now honor ARTIFACTS_PY/ARTIFACTS_ODBC to install signed wheels offline (--no-index --find-links) instead of always hitting PyPI. build_and_test_local.ps1 adds --override-channels for reproducible solves and its header no longer claims an offline/no-microsoft-channel run. Draft publish stage condition compares the boolean parameter directly (eq(param, true)) instead of a quoted string.
Comment thread conda/mssql-python-odbc/meta.yaml Fixed
- Single-source versions via MSSQL_PYTHON_VERSION/MSSQL_ODBC_VERSION env (wired from the publish-stage params) so the package version and the companion pin can't drift.

- Point mssql-python-odbc license_file at the actual ODBC Driver 18 EULA + VC++ license (was MIT-primary root LICENSE); remove the resolved TODO.

- Drop the no-op azure-identity >=1.12.0 floor (microsoft channel ships CalVer).

- Re-assert the wheel platform floor via __glibc/__osx virtual-package run constraints.

- Add conda/driver_load_probe.py + run it in the gate and local harness so we prove the native ODBC driver loads, not just the Python shim.

- Publish stage: require signed wheels (no PyPI fallback), publish companion-first with --skip-existing and a #706 pair guard, and document the required resources.pipelines declaration.
Comment thread conda/driver_load_probe.py Fixed
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

82%


📈 Total Lines Covered: 7783 out of 9455
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 75.5%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 84.3%
mssql_python.logging.py: 85.5%
mssql_python.connection.py: 85.9%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@jahnvi480
Jahnvi Thakkar (jahnvi480) marked this pull request as draft August 18, 2026 08:57
Port the productionized conda pipeline from the ADO conda-publish-pipeline branch onto GitHub conda-onboarding, replacing the earlier draft prototype:

- conda recipes: meta.yaml + build.sh + bld.bat for mssql-python and mssql-python-odbc, vendored ODBC/VC++ EULA text, and .gitattributes (LF for shell scripts).

- OneBranchPipelines conda glue: scripts/build-conda-packages.{sh,ps1}, steps/conda-build-validate-step{,-posix}.yml, steps/conda-publish-step.yml, steps/conda-release-step.yml, jobs/consolidate-conda-artifacts-job.yml.

- Wire conda legs into build/release pipelines (buildConda params on win-64/osx/linux legs + ConsolidateConda stage), preserving GitHub-only signWindowsBinaries content.

- Remove superseded draft prototype: conda/onebranch-publish-conda-stage.yml and conda/build_and_test_local.ps1.
@github-actions github-actions Bot added pr-size: large Substantial code update and removed pr-size: medium Moderate update size labels Aug 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

Suppressed comments (4)

OneBranchPipelines/scripts/build-conda-packages.sh:173

  • The verify env name only includes the Python version. On macOS this script is invoked twice on the same agent (native osx-arm64, then cross-target osx-64), so the second conda create -n verify_... can fail with an existing prefix or reuse the wrong-arch env. Include the effective subdir (native vs $CONDA_SUBDIR) in the env name to avoid collisions.
for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="

OneBranchPipelines/stages/build-linux-single-stage.yml:116

  • This apt install runs before the later apt-get update in the job, so it can fail on agents with stale apt indices (common on hosted Ubuntu). Add an apt-get update (and ideally noninteractive) before installing qemu-user-static/libc6-arm64-cross.
              # qemu-user-static: run aarch64 ELF binaries on the x86_64 host.
              # libc6-arm64-cross: the aarch64 glibc runtime (loader + libc/libm/...)
              # under /usr/aarch64-linux-gnu so qemu can resolve /lib/ld-linux-aarch64.so.1
              # for the emulated aarch64 conda build/verify (QEMU_LD_PREFIX points here).
              sudo apt-get install -y qemu-user-static libc6-arm64-cross

OneBranchPipelines/steps/conda-publish-step.yml:91

  • Installing anaconda-client from PyPI without pinning a version can make releases non-reproducible and may unexpectedly break publishing if a new release introduces behavioral changes. Consider pinning to a vetted version (or using a constraints file) so publish behavior is stable.
        $ErrorActionPreference = 'Stop'
        python -m pip install --upgrade pip
        python -m pip install anaconda-client
        # anaconda-client installs the `anaconda` console script onto PATH.
        anaconda --version

OneBranchPipelines/steps/conda-build-validate-step.yml:126

  • The error message references $links (the wheel find-links directory), but conda packages are searched under ${{ parameters.outputDir }}/bld. If this trips, the message will mislead troubleshooting.
      if (-not $built) { Write-Error "No conda packages were produced under $($links)"; exit 1 }

Comment thread conda/mssql-python/meta.yaml Outdated
Comment thread OneBranchPipelines/scripts/build-conda-packages.sh
Comment thread OneBranchPipelines/scripts/build-conda-packages.ps1
…uildAll on Windows

The Windows mssql-python-odbc companion conda is now built ONCE as a Python-agnostic package in the ODBC_BuildAll stage (no python in host, wheel extracted via tar in bld.bat), mirroring the single py3-none-win_* PyPI wheel, instead of once per Python on every binding leg. The per-Python mssql-python binding legs seed that prebuilt companion into their local channel (-Package binding -DriverCondaDir) so the version-locked dependency still resolves. ConsolidateConda now also pulls the ODBC_BuildAll companion, and the #706 release/publish gates use presence-pairing (keeping strict 1:1 only for per-Python companions, c>1). macOS/Linux stay per-Python (unchanged).
P0-1: invert conda/driver_load_probe.py from a fail-OPEN denylist to a
fail-CLOSED allowlist. A repackaged native ODBC driver that fails to load now
FAILS the DB-less pre-publish gate instead of passing on any unrecognized
exception. Only a clean connect or a connection-stage diagnostic the loaded
msodbcsql driver alone can emit (ODBC branding, network provider, TLS, auth)
counts as PASS. Defer 'import mssql_python' into main() so the classifier is
unit-testable without the compiled extension. Adds tests/test_026 (22 no-DB
tests).

P0-3: gate the conda release on package METADATA, not folder names/counts.
New conda/validate_conda_release.py reads each package's authoritative
info/index.json (zstd) and validates real subdir == folder, allowed subdirs,
the full (subdir x Python) binding matrix, exact/consistent versions, and #706
binding<->companion pairing. Catches a mislabeled subdir and the 8e7f217
dropped-win-64-variant regression the count gate missed. Rewires
OneBranchPipelines/steps/conda-release-step.yml to call it. Adds tests/test_027
(11 tests incl. a real .conda round-trip).
Comment thread conda/driver_load_probe.py Fixed
…P0-2)

macos-latest is an Intel Mac and the arm64 Python cannot execute there (no
reverse Rosetta), so the previous osx-arm64 leg silently built NATIVE osx-64
packages and staged them under osx-arm64 (mislabeled). Cross-build for real:

- conda/*/build.sh: when the host-env Python is not executable (non-emulated
  cross-build), extract the universal2 wheel into \ with unzip instead of
  pip -- mirrors the Windows bld.bat tar path; the arm64 slice comes from the
  universal2 wheel. Native + QEMU-emulated legs keep the pip install.
- conda/*/meta.yaml: skip_compile_pyc on macOS so conda-build does not run the
  non-runnable arm64 Python for .pyc byte-compilation (Python regenerates it).
- build-conda-packages.sh: section-7 verify auto-skips the runtime import when
  the target Python can't run on the host (osx-arm64 on Intel); the static
  arm64-slice audit is the stand-in. Native/QEMU legs still import for real.
- build-macos-single-stage.yml: add condaTargetSubdir: osx-arm64 +
  continueOnError to the arm64 leg; mark osx-64 as the native/blocking leg; add
  a BLOCKING static arm64 slice audit (lipo/otool/file) asserting the shipped
  ddbc_bindings ext and macos/arm64 dylibs really contain arm64 Mach-O. Fix the
  misleading 'Apple Silicon, native' comments.
The vendored ODBC Driver 18 links crypto/auth libs that must receive conda's
security updates instead of being frozen into the payload -- mirroring
conda-forge libpq (declares openssl + krb5, vendors neither):

- openssl # [not win]: the driver dlopen's libssl/libcrypto for TLS; it is not
  an ELF NEEDED entry so overlinking can't see it. Windows uses SChannel.
- krb5 # [linux]: libmsodbcsql NEEDs libkrb5.so.3 + libgssapi_krb5.so.2 (not
  bundled). macOS uses Kerberos.framework, Windows uses SSPI.
- vc14_runtime # [win]: msodbcsql18.dll imports VCRUNTIME140.dll but the
  vendored vcredist ships only msvcp140.dll; declare the serviced conda runtime.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated no new comments.

Suppressed comments (2)

OneBranchPipelines/scripts/build-conda-packages.sh:184

  • The verify env name is only based on the Python version (e.g. "verify_310"). On macOS this script is invoked twice on the same agent (osx-64 and osx-arm64) sharing the same Miniforge install/outputDir, so the second invocation will fail when it tries to create an env that already exists. Include the target subdir in the env name (and/or delete any existing env before creating it) so the two runs don’t collide.
for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="
  # -c microsoft (ahead of conda-forge) so azure-core/azure-identity/msal resolve from the
  # lean `microsoft` channel, NOT conda-forge whose azure-core recipe over-declares flask/six
  # -> celery/boto3/botocore (~9 MB); see conda-forge/azure-core-feedstock#71.
  # --strict-channel-priority keeps the freshly built local companion + binding authoritative.
  "$conda" create -y -n "$envName" -c "$bld" -c microsoft -c conda-forge --strict-channel-priority --override-channels "python=$py" mssql-python

conda/validate_conda_release.py:67

  • read_index_json() uses next(...) to locate the info-*.tar.zst member inside a .conda. If the archive is malformed (missing that member), this will raise StopIteration and produce a stack trace rather than a clear validation failure message. Handle the empty case and raise a ValueError with a helpful message instead.
        with zipfile.ZipFile(path) as zf:
            info_name = next(
                n for n in zf.namelist() if n.startswith("info-") and n.endswith(".tar.zst")
            )
            info_blob = zf.read(info_name)

libodbcinst.so.2 has NEEDED libltdl.so.7 but no RUNPATH, so a minimal glibc

Linux base throws 'OSError: libltdl.so.7: cannot open shared object file' on

import. macOS already vendors libltdl.7.dylib; Linux was the inconsistent

outlier. Two parts:

1) eng/scripts/patch-linux-odbc-libs.sh (new): maintainer/CI tool, run in a

   manylinux_2_28 container, that sources the glibc libltdl.so.7, copies it

   next to libodbcinst.so.2, and patchelf --set-rpath '\' so the driver

   resolves it from its own dir. Skips Alpine/musl by design.

2) build-odbc-all-stage.yml: the wheel content verifier now REQUIRES

   libltdl.so.7 in both manylinux_2_28 payloads (fail-closed). This gate stays

   red until a maintainer runs the patch script and commits the produced

   libltdl.so.7 + rpath-patched libodbcinst.so.2 under

   mssql_python_odbc/libs/linux/debian_ubuntu/<arch>/lib/ (cannot be produced

   on the Windows-only odbc build host).
Decision 2(b): Alpine is a supported, PR-tested platform, so its wheels must be

fixed too (not dropped, not documented-as-limitation). Extends the libltdl

self-contained-payload work to musl:

- eng/scripts/patch-linux-odbc-libs.sh now auto-detects libc + arch and patches

  the matching distro subtrees: a glibc-built libltdl (manylinux, dnf) for

  debian_ubuntu/rhel/suse, a musl-built libltdl (Alpine, apk add libtool) for

  alpine. One build per (libc, arch) serves all that arch's distro subtrees. It

  also drops a per-dir LIBLTDL_LGPL_LICENSE.txt notice.

- mssql_python_odbc/libs/LICENSING: document libltdl (GNU Libtool, LGPL-2.1-or-

  later, dynamically linked) covering the existing macOS libltdl.7.dylib and the

  new Linux libltdl.so.7 (compliance; precedent = macOS already vendors it).

- build-odbc-all-stage.yml: musllinux_1_2_{x86_64,aarch64} wheels now also REQUIRE

  libltdl.so.7 (fail-closed), since the Alpine test leg's system libltdl masks

  the OSError today.

- eng/scripts/audit_bundled_binaries.py (new): allowlist-driven ELF/Mach-O/PE

  dependency audit (gate step 3). Every dep must be BUNDLED, BASE (OS/libc), or

  DECLARED (openssl/krb5 [linux], vc14_runtime [win]); anything else fails. ELF

  binaries that need a bundled sibling must carry an \ RUNPATH; macOS

  absolute non-system install names (e.g. /opt/homebrew) fail regardless of

  basename; --require-arch asserts Mach-O slices (the Intel-agent substitute for

  the arm64 runtime import). Validated locally: Windows PASS, Linux FAIL on the

  missing libltdl.so.7, macOS FAIL on libodbc.2.dylib's /opt/homebrew libltdl.
The gate now builds the package + patches the climb correctly (RUNPATH is
\ + the exact climb), but the audit failed: the payload bundles ALL distros
including alpine/musl, whose libodbcinst NEEDs only libc.musl* (it statically links
ltdl -- no libltdl DT_NEEDED). There is no musl conda subdir (conda Linux is
glibc-only), so the glibc DT_NEEDED requirements don't apply to musl variants. Skip
them for musl (detected via NEEDED libc.musl*); the climb / per-subdir presence /
no-vendored checks still apply. +1 test (21 audit tests).
…nux climb)

- Remove the redundant standalone 'conda index' from both orchestrators. conda-build --output-folder already writes each subdir repodata.json, and the subcommand was split into a separate conda-index package (absent in miniforge), failing the macOS legs with 'invalid choice: index'.

- Run the section-7 verify imports from a neutral cwd (\) so the repo's in-tree mssql_python source (no compiled extension) cannot shadow the conda-installed package on win-64 ('No ddbc_bindings module found').

- Patch the \ climb by default instead of assertion-only: the Linux ODBC .so are NOT ESRP code-signed (odbc pipeline only malware-scans them; only Windows .dll / macOS .dylib are code-signed, which this recipe never touches), so stamping the RPATH breaks no signature. Removes the now-inverted assertion-only guard from conda-build-pipeline.yml.
…valid

Removing 'conda index' left the output folder without noarch/repodata.json. conda-build --output-folder writes only the platform subdir's repodata, but conda rejects any channel lacking noarch/repodata.json (UnavailableInvalidChannel), which failed the linux-64 verify solve. Create the empty noarch repodata directly (miniforge has no 'conda index' subcommand). Reproduced in condaforge/miniforge3: without it the channel is invalid, with it the channel validates and solves.
conda's channel URL parser strips any path component equal to a known subdir token (osx-arm64/linux-64/...). The posix build+validate step isolates each leg under a subdir-named dir (OutputDir=.../<subdir>), so 'conda create -c \' (bld=\/bld) resolved to the token-stripped path .../conda-bld/bld, which has no repodata -> UnavailableInvalidChannel on the macOS/Linux legs. Windows passes because its step does not append the subdir. Copy the built channel into a token-free, per-leg-unique dir and verify against that. Reproduced+proven in condaforge/miniforge3: -c .../osx-arm64/bld resolves to .../bld (INVALID); -c .../verifychan_osx_arm64 resolves verbatim (VALID).
…d glob

The minimal-base reachability gate globbed libs/linux/*/*/lib/libmsodbcsql* and took [0] = the alphabetically-first 'alpine' (musl) variant. On the glibc conda leg that variant needs libc.musl (absent) and its libodbcinst does NOT link libltdl, so the gate failed 'libltdl absent' even though the driver the loader actually binds (debian_ubuntu) resolves libltdl fine. Mirror GetDriverPathCpp's exact /etc/*-release probe (alpine/rhel/suse/debian_ubuntu) + arch so the gate inspects the SAME variant the driver loads.

Proven end-to-end in continuumio/miniconda3 against the real mssql_python 1.13.0 + mssql_python_odbc 18.6.2.1 wheels: full build -> climb -> audit -> noarch -> token-free channel -> import -> driver-load -> reachability gate exits 0 (REACHABILITY_OK). Verified debian_ubuntu/rhel/suse (x86_64 AND arm64) link libltdl; alpine does not.
force_tls used regex substitution that could hand mssql_python's parser a malformed string: a valueless Encrypt, a DUPLICATE Encrypt (only the first rewritten, a Duplicate keyword error), and shapes like Encrypt=;yes becoming Encrypt=yes;yes (a bare yes, so keyword yes has no value), which failed the linux-64 TLS gate. Rebuild from brace-aware tokens: drop any existing Encrypt/TrustServerCertificate (incl. valueless or duplicate), re-append the canonical pair once. Add a redacted structure log so a future malformed string is self-explaining, plus 11 hardening unit tests.

Proven end-to-end in Docker against a live SQL Server 2022: build, climb, audit, noarch, token-free channel, import, driver-load, reachability gate, and TLS_OK (real Encrypt=yes handshake), exit 0.
Comment thread tests/test_028_tls_connect_probe.py Fixed
Comment thread tests/test_028_tls_connect_probe.py Fixed
Comment thread tests/test_028_tls_connect_probe.py Fixed
Comment thread tests/test_028_tls_connect_probe.py Fixed
Comment thread tests/test_028_tls_connect_probe.py Fixed
Comment thread tests/test_028_tls_connect_probe.py Fixed
Comment thread tests/test_028_tls_connect_probe.py Fixed
…n string

A queue-time value of yes (the connection-string field mistaken for a yes/no toggle) was fed to the driver as a connection string, failing the linux-64 leg with a keyword yes has no value parse error. The probe now recognises a non-connection-string value (no key=value pair) and skips loudly with guidance instead of failing the leg; the static RUNPATH audit still guards OpenSSL layout. Clarified the pipeline parameter label. Plus 9 unit tests.

Proven in Docker: with CONDA_TLS_PROBE_CONN=yes the full leg reaches SCRIPT_EXIT=0 with TLS_PROBE_SKIPPED; a real connection string still runs the Encrypt=yes gate.
Comment thread tests/test_028_tls_connect_probe.py Fixed
Comment thread tests/test_028_tls_connect_probe.py Fixed
…an mirror

The literal Server;Uid;Pwd example in the pipeline displayName and the probe skip message tripped ADO credential scanning on the mirrored push (harmless placeholder, but blocked). Replaced the connection-string examples with prose; no behaviour change, 41 tests pass.
…an and policheck

linux-aarch64 is a QEMU-binfmt cross leg: qemu-user runs the Python binding but cannot init the native unixODBC environment (SQLAllocEnv fails with Failed to allocate environment handle). Reproduced exactly (x86_64 host with CONDA_SUBDIR=linux-aarch64); the SAME driver loads under full-arch emulation and on the native leg, and the masking-immune static audit proves self-containment. So on an emulated cross leg the runtime driver probes (driver-load, ldd reachability, TLS) are best-effort; build, audit and import stay blocking.

test_028: drop the User Login Credentials (Uid plus Pwd pairs) that CredScan breaks the build on, and replace localhost with a neutral placeholder host to clear the DevSkim notes. Reword whitelist to allow-list for PoliCheck.

Validated in Docker: native linux-64 stays DRIVER_LOADED plus REACHABILITY_OK and exit 0; the aarch64 cross leg now skips the qemu-incompatible probes and exits 0; 41 unit tests pass.
…ecipe copies

The conda recipe kept its own copies of the ODBC driver and VC++ runtime EULAs. Point license_file at the canonical mssql_python_odbc/licenses/ copies (via ../../, the same climb ../../LICENSE already uses) and remove the duplicates. Validated in Docker: conda-build resolves and packages them (Packaged license file/s, CONDA_BUILD_OK, exit 0). NOTE: the wheel-build stages rm -rf mssql_python_odbc as part of the ODBC split, so if that tree is ever removed from the conda pipeline checkout too, this reference must be revisited.
Cross-build the self-contained mssql-python conda package for win-arm64 on the native x64 Windows agent via CONDA_SUBDIR, mirroring the existing osx-arm64 cross leg. bld.bat now extracts the cp<CONDA_PY>-win_arm64 code wheel WITHOUT running Python when the arm64 host Python cannot execute on x64 (build.sh already does this for osx-arm64); meta.yaml skips pyc on win too. build-conda-packages.ps1 adds Anaconda defaults for the win-arm64 build host env and the verify solve (win-arm64 python 3.12/3.13 and cryptography live only on defaults, azure-identity/msal are noarch on microsoft), makes the arm64 verify best-effort with an import auto-skip, and auto-accepts the defaults ToS. The pipeline adds a continueOnError win-arm64 leg scoped to Python 3.12-3.14 (the intersection of available win_arm64 wheels and win-arm64 conda python).

Validated locally: real cross-build produced win-arm64/mssql-python-1.13.0-py314_0.conda (subdir=win-arm64, arm64 ddbc_bindings.pyd + arm64 driver, depends=azure-identity/python3.14/vc14_runtime); the full transitive dep closure resolves for 3.12, 3.13 and 3.14 (mssql-python -> azure-identity/msal -> cryptography 49.0 + cffi + vc14_runtime).
…onPreference=Stop

The verify loop's conda env remove (on an absent env -- fresh agent / first run) and conda run (when the cross-built arm64 Python cannot execute on the x64 host) write to stderr on their NON-FATAL paths. Under the script-wide ErrorActionPreference=Stop, PowerShell escalated that to a terminating NativeCommandError, aborting BOTH the win-64 (blocking) and win-arm64 (best-effort) legs at conda env remove before any package was staged -- even though every conda package built successfully. Switch the verify section to Continue and gate control flow on LASTEXITCODE / Assert-LastExit, the same intent as the bash port's || true. Reproduced the NativeCommandError under Stop and confirmed no-abort under Continue; the native win-64 create + import path completes.
…leg passes + stages

The win-arm64 verify leg's last native command is the runnable-check conda run python that INTENTIONALLY fails (arm64 Python cannot launch on the x64 host -> exit 216 = ERROR_EXE_MACHINE_TYPE_MISMATCH); the script then skips the import and completes normally (CONDA_BUILD_OK, all three packages built and their deps resolved from pkgs/main win-arm64 + microsoft). But the validate-step .yml calls this script with the call operator in-session, so the lingering LASTEXITCODE=216 propagated to the caller's exit-code gate and failed the leg BEFORE staging. Add a trailing cmd /c exit 0 to reset LASTEXITCODE without PowerShell exit (which, under the in-session call, would terminate the whole step before it stages the packages). Verified end-to-end: & script leaves the caller at LASTEXITCODE=0 with the reset (216 without) and control returns so staging runs. Real failures still exit 1 via Assert-LastExit.
…sh plan)

Conda analogue of dummy-release-pipeline.yml (the PyPI dummy). Always NonOfficial, manual trigger, [TEST]-labelled. Downloads the consolidated conda packages from the conda-build pipeline, runs the SAME readiness gate the real conda-release-pipeline uses (conda-release-step with labelPrefix=[TEST]), then performs a DUMMY publish: a DRY-RUN that enumerates every package and prints the exact anaconda upload plan (per package/subdir/channel/label) WITHOUT uploading. Unlike PyPI -- where the dummy releases via ESRP with the harmless Maven ContentType -- anaconda-client has no safe alternate target, so the dummy never runs anaconda upload; a real staged test to a non-production org goes through conda-release-pipeline.yml with a test condaChannel + publishToConda=true. YAML validated locally.
…ch assert, exact matrix/RUNPATH, publish version pin); drop dead CONDA_ALLOW_UNSIGNED_PATCH
Comment thread OneBranchPipelines/scripts/build-conda-packages.sh Fixed
Comment thread OneBranchPipelines/scripts/build-conda-packages.sh Fixed
Comment thread OneBranchPipelines/scripts/build-conda-packages.sh Fixed
Comment thread OneBranchPipelines/scripts/build-conda-packages.sh Fixed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: large Substantial code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants