Skip to content

CI + build/release script hardening - #37

Closed
dowjames wants to merge 7 commits into
ARahim3:mainfrom
dowjames:fix/tooling
Closed

CI + build/release script hardening#37
dowjames wants to merge 7 commits into
ARahim3:mainfrom
dowjames:fix/tooling

Conversation

@dowjames

@dowjames dowjames commented Oct 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Six commits from the repo review, all tooling (no Swift changes):

  • New CI workflow: swift build + swift test on macOS 14 for PRs and main (Vendor/llama cached by the same stamp build-llama.sh records; ReferenceVectorTests skipped on PRs since it needs the model), plus shellcheck + ruff on Ubuntu.
  • build-llama.sh: a no-logits patch that does not apply (after a TAG bump) is now a hard error instead of a silent unpatched build that still gets stamped as patched.
  • build.sh: Sparkle.framework located by pattern with a clear error; blank .sign-id treated as absent; --deep dropped from release verification (nested code verified explicitly); Sparkle key checked before the long build; notarytool logs kept as evidence; APPCAST required (or SKIP_APPCAST=1).
  • appcast.py: generated feed validated as XML before writing, ]]> escaped in notes, download base derived from the origin remote so forks get their own URLs.
  • testbed/dev-run: nullglob for empty globs, say voice fallback, non-APFS cp fallback.
  • homebrew.yml: commits only Casks/digup.rb.
  • worker: text() retries failed batches per item (matches the protocol docstring), quiet exit on a broken pipe.

Built and installed successfully on a MacBook Pro (Xcode 26.3) as part of the combined fix/all branch.

Closes #44.

@dowjames dowjames mentioned this pull request Oct 12, 2026
Closed
AmirHossein-84 added a commit to AmirHossein-84/DigUp that referenced this pull request Oct 18, 2026
@AmirHossein-84
feat: integrate upstream PRs, engine optimizations, bugfixes, and CJK…
454e78d 
… localization

- SQLite & Search Layer (PRs ARahim3#16, ARahim3#38, ARahim3#82, ARahim3#85, ARahim3#94, ARahim3#111):
  * StatementCache with sqlite3_reset statement recycling and reentrant transactions
  * PRAGMA mmap_size = 1GB, synchronous = NORMAL, cache_size = -64000
  * Zero-copy row.copyBlob direct vector loading into searcher matrix halves
  * Unicode NFC normalization for cross-platform and CJK/accented file paths
  * Chunked IN queries with UNION ALL to eliminate N+1 roundtrips
  * CJK Han character detection, Han text budgeting, and Han-aware snippet generation
  * SIMD8<Float> acceleration in Math+Portable.swift without heap allocations
  * Float16Shim for Intel Mac x86_64 IEEE 754 binary16 compatibility

- Crawler, Rules, Path Normalization & Extractors (PRs ARahim3#33, ARahim3#42, ARahim3#56, ARahim3#60, ARahim3#79, ARahim3#80, ARahim3#86, ARahim3#88, ARahim3#91, ARahim3#101, ARahim3#107, ARahim3#109, Issues ARahim3#67, ARahim3#97, ARahim3#105, ARahim3#127):
  * Fix Windows path slash/drive normalization in DirectoryWalk preventing 0-file crawl
  * Fix case-insensitive folder matching on Windows in FolderSelection
  * Fix Crawler to never treat user-selected search roots (e.g. ~) as project roots
  * Exclude secrets/keys (.env*, .pem, .kdbx, SSH keys, passwd) in Rules
  * Honor standard CACHEDIR.TAG 43-byte signature when crawling
  * Skip build/intermediate directories (.venv, target, .build, .next, dist, bin, obj)
  * Drop FileProvider download status lookups for 10-50x faster cloud folder crawls
  * OfficeText extractor for .xlsx, .pptx, .ods, .odp XML documents
  * Headless LibreOffice .doc extractor on Linux and local Notion SQLite workspace extractor
  * Safe typed CGPDFDictionaryGetString for PDF metadata preventing crashes
  * Preserve OCR keywords on image-only PDFs
  * Outlook PST reader utility crate in pst/

- Indexing Pipeline & Runtime (PRs ARahim3#49, ARahim3#53, ARahim3#62, ARahim3#64, ARahim3#71, ARahim3#72, ARahim3#114, Issue ARahim3#3):
  * Batched document text embeddings (up to 16 docs per pass)
  * Cumulative IndexProgress tracking across multi-pass indexing
  * DocumentCache for extracted chunks to prevent redundant file disk reads
  * Linux URLSession SIGABRT fix in FileFetcher
  * ImageLoader parallel prefetching and batched image embeddings
  * Apple Neural Engine CoreML video frame encoding support
  * Scoped incremental subtree sync when changed paths are known
  * Autoreleasepool wrapping in background worker extraction to eliminate memory leaks

- Daemon, Lifecycle & Policies (PRs ARahim3#24, ARahim3#27, ARahim3#31, ARahim3#36, ARahim3#51, ARahim3#82, Issues ARahim3#22, ARahim3#68):
  * HelperRestartPolicy for clean stop/resume transitions
  * Backlog starvation recovery and --resume-pending in IndexService
  * Configurable indexing hours window policy
  * Opt-in Low Power Mode indexing to protect laptop thermals and battery
  * Win32 OpenProcess(SYNCHRONIZE) parent monitor to prevent zombie helpers on Windows
  * Correct 0-hit search behavior when all search roots are removed

- UI, Cross-Platform Shell, Localization & Build (PRs ARahim3#15, ARahim3#20, ARahim3#37, ARahim3#39, ARahim3#58, ARahim3#65, ARahim3#74, ARahim3#81, ARahim3#84, ARahim3#112):
  * DigUpLocalization framework with en, zh-Hans, zh-Hant string catalogs
  * AppSearch engine and launcher for macOS App and Tauri desktop shells
  * KeyCommands and QuickLookController shared handling
  * Preserved search field text during IME composition
  * Search panel fade transitions and @observable code editor selection
  * Hardened build scripts with pinned SHA-256 vendor checksum verification
@dowjames
Add CI: build + engine tests on macOS, shellcheck + ruff on Ubuntu
c8c7516 
Nothing ran swift build/swift test on PRs, so regressions merged unverified
(CONTRIBUTING.md asks for passing tests). Vendor/llama is cached by the same
stamp build-llama.sh records, so PR jobs reuse it instead of rebuilding
llama.cpp; ReferenceVectorTests (needs the model) is skipped on PRs.
@dowjames
Fail fast and loudly in the build and release scripts
580db80 
- build-llama.sh: a no-logits patch that does not apply (after a TAG bump) was
  silently skipped while BUILD still stamped it as patched, caching an
  unpatched build forever; now it is an error (already-applied is tolerated).
- build.sh: locate Sparkle.framework by pattern with a clear error instead of
  one hardcoded slice path; treat a blank .sign-id as absent; build both
  products in one swift build invocation.
- release.sh: check the Sparkle key before the multi-minute build instead of
  after notarization; drop deprecated --deep and verify nested code explicitly;
  keep notarytool logs as release evidence; require APPCAST or SKIP_APPCAST so
  a DMG is never silently left off the update feed.
@dowjames
appcast.py: validate the feed, derive the download base from origin
ece3386 
The f-string feed could contain unparseable XML (]]> in notes, raw <title>);
it is now parsed before writing, ]]> is escaped, and DOWNLOAD_BASE defaults to
the origin remote so forks get their own URLs. Read the feed with a context
manager.
@dowjames
Testbed and dev-run: survive empty globs, missing voices, non-APFS
be0252a 
An empty screenshots glob handed xattr a literal pattern and aborted the
script (nullglob now); make_media.sh falls back from Samantha to any en_US say
voice; cp -c (APFS clones) falls back to plain cp elsewhere.
@dowjames
homebrew.yml: commit only Casks/digup.rb
b7142ca 
git commit -am would sweep any unrelated dirty file in the tap into the release
commit.
@dowjames
worker: per-item text errors, quiet exit on a broken pipe
754d312 
text() failed the whole request on one bad input, contradicting the protocol
docstring; it now retries the batch item by item like media(). A dead app now
ends the worker quietly instead of with a BrokenPipeError traceback.
@dowjames
build.sh: keep the two product builds separate (swift build honours o…
730c08f 
…nly one --product per run; the merged call built digup but not DigUpApp)
@ARahim3

ARahim3 commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Thanks for this. I've added a CI workflow of my own and I'm taking three of the script fixes (the patch check, the key check, the appcast escaping) separately, so I'm closing the PR rather than merging the bundle.

@ARahim3 ARahim3 closed this Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

No CI runs the Swift build or tests on pull requests

2 participants

@dowjames @ARahim3