From 905092957bdc8ba84d7871d31f4fb1b8e82cacad Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Sun, 11 Oct 2026 21:49:30 +0100 Subject: [PATCH 1/3] Keep panels still as the macOS menu bar and Dock show and hide NSScreen.visibleFrame leaves out the menu bar and Dock only while they show, so in a full-screen app, or with either set to hide, the work area changed whenever the pointer reached a screen edge, and the keyboard and mouse panel jumped to fit it. On macOS, panels now keep the area their scan started with while it stays on the same display, and point scanning no longer ends for that change alone. Closes #1023. Co-Authored-By: Claude Opus 5.5 --- docs/point-scan.md | 2 +- docs/qwerty-keyboard.md | 2 +- docs/scanner-architecture.md | 2 +- src-tauri/src/point_scan_runtime.rs | 70 ++++++++++++++++++++++++++++- 4 files changed, 72 insertions(+), 4 deletions(-) diff --git a/docs/point-scan.md b/docs/point-scan.md index 92872c1b..b45a8137 100644 --- a/docs/point-scan.md +++ b/docs/point-scan.md @@ -48,7 +48,7 @@ Drag retains the source and scans a destination on the same display without hold `point_workflow` separates point selection, menu navigation and typed action requests. Its current selection policy always opens the menu; a future auto-select policy can reuse `default_click` without changing the executor. No auto-select timer or setting exists. The reusable `scan_menu` defines stable action IDs and rows. Shared frames carry menu tiles, labels and highlight strips; all native windows remain click-through and nonactivating. -The desktop view adds menu, menuSuspended, dragDestination, dragConfirmation and executing phases. Existing point phases, saved settings and Bluetooth commands retain their shapes. Mobile connections, switch editing/learning and shutdown cancel the workflow. A change of display or work area cancels point scanning and its menus. So does a change of the window in front, once a switch has moved or chosen anything in the scan, so that a point is never used on a window it was not chosen over. Until then a point scan follows whichever window is in front, and can run with none in front. If no window is in front at the moment of the first choice and one appears afterwards, the scan is cancelled. The keyboard and the mouse panel stay open through both kinds of change; see [the keyboard](qwerty-keyboard.md). Targets and foreground identity remain in memory and are never logged or emitted. Automated action tests use fake input only; physical Windows/macOS focus, scaling, target, scrolling and drag checks remain required for hardware qualification. +The desktop view adds menu, menuSuspended, dragDestination, dragConfirmation and executing phases. Existing point phases, saved settings and Bluetooth commands retain their shapes. Mobile connections, switch editing/learning and shutdown cancel the workflow. A change of display or work area cancels point scanning and its menus. On macOS, the menu bar and Dock showing or hiding does not count as a work area change; see [the keyboard](qwerty-keyboard.md). So does a change of the window in front, once a switch has moved or chosen anything in the scan, so that a point is never used on a window it was not chosen over. Until then a point scan follows whichever window is in front, and can run with none in front. If no window is in front at the moment of the first choice and one appears afterwards, the scan is cancelled. The keyboard and the mouse panel stay open through both kinds of change; see [the keyboard](qwerty-keyboard.md). Targets and foreground identity remain in memory and are never logged or emitted. Automated action tests use fake input only; physical Windows/macOS focus, scaling, target, scrolling and drag checks remain required for hardware qualification. The action menu uses a fixed grid of square icon tiles with labels beneath the artwork, on the same dark rounded panel chrome as the scanning keyboard (`TileRole::Panel`). A yellow border and amber background identify the current row or item. The same artwork is drawn on Windows and macOS. Menu titles stay above the shared panel when a scan opens a menu or changes submenus, without taking keyboard focus. diff --git a/docs/qwerty-keyboard.md b/docs/qwerty-keyboard.md index 986dc0ce..65cf4432 100644 --- a/docs/qwerty-keyboard.md +++ b/docs/qwerty-keyboard.md @@ -16,7 +16,7 @@ The bottom control row starts with Close keyboard, followed by Letters, Navigati Modifier keys are pressed only around each emitted shortcut and immediately released. Selecting a locked modifier does not hold that operating-system key while the scanner runs. Ordinary characters use text injection; command combinations and navigation use native key events. Native shortcuts retain the operating system's layout semantics. -The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation. +The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit. On macOS the work area leaves out the menu bar and Dock only while they show, which in a full-screen app, or with either set to hide automatically, is only while the pointer is at the screen edge. So there, on the same display, the keyboard and mouse panel keep the area they opened with and do not jump as the bars come and go. They stay above both and never take clicks. Moving to another display, or this display changing size, still fits them again; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation. ## Manual validation diff --git a/docs/scanner-architecture.md b/docs/scanner-architecture.md index 332e6330..6db00dbe 100644 --- a/docs/scanner-architecture.md +++ b/docs/scanner-architecture.md @@ -110,7 +110,7 @@ Rules that hold whatever is chosen: - `wait` has no effect in an area that is not automatic, except that point scanning always waits unless `automatic` is chosen. - A point scan that starts by itself still stops at the pass limit. -Every point scan takes up whichever window is in front, or none, until a switch moves or chooses something in it. A click may have brought another window forward, and a panel may just have closed. From then on, a change of window ends the scan as usual. A change of display or work area ends a point scan at any time. The keyboard and the mouse panel are not tied to any window or display; see [the keyboard](qwerty-keyboard.md). +Every point scan takes up whichever window is in front, or none, until a switch moves or chooses something in it. A click may have brought another window forward, and a panel may just have closed. From then on, a change of window ends the scan as usual. A change of display or work area ends a point scan at any time, except on macOS when only the menu bar or Dock shows or hides. The keyboard and the mouse panel are not tied to any window or display; see [the keyboard](qwerty-keyboard.md). `keyboardWaitAfterTyping`, saved by earlier versions, still makes the keyboard wait. It applies only while the keyboard has no `nextScan` of its own and the shared value is `standard`. Settings no longer offers or changes it; the saved value is kept. diff --git a/src-tauri/src/point_scan_runtime.rs b/src-tauri/src/point_scan_runtime.rs index a337f76b..207e77a8 100644 --- a/src-tauri/src/point_scan_runtime.rs +++ b/src-tauri/src/point_scan_runtime.rs @@ -35,6 +35,25 @@ fn keyboard_display<'a>( .or_else(|| displays.iter().find(|display| display.name == current.name)) .or_else(|| display_navigation::current_display(cursor, displays)) } +/// On macOS the work area leaves out the menu bar and Dock only while they +/// show, so it changes whenever an auto-hidden one comes and goes, as in a +/// full-screen app. Windows keeps its work area when the taskbar hides. +const WORK_AREA_FOLLOWS_AUTO_HIDE: bool = cfg!(target_os = "macos"); +/// The area for panels on `display`. Where the work area follows auto-hidden +/// bars, the area this scan already has is kept while it stays on the same +/// display, so panels never jump as the bars show and hide. +fn steady_area( + follows_auto_hide: bool, + environment: &Environment, + display: &Display, + area: Rect, +) -> Rect { + if follows_auto_hide && environment.display == *display { + environment.keyboard_area + } else { + area + } +} fn bounds(display: &Display) -> Rect { Rect { x: display.x.into(), @@ -275,6 +294,8 @@ impl Adapter for PointScan { Some((cursor, displays.len(), current, area)) }); if let Some((cursor, displays, current, area)) = placed { + let area = + steady_area(WORK_AREA_FOLLOWS_AUTO_HIDE, environment, ¤t, area); technique.set_mouse_area( area, bounds(¤t), @@ -333,6 +354,8 @@ impl Adapter for PointScan { }) .and_then(|display| { let area = crate::scan_host::work_area(bounds(&display)).ok()?; + let area = + steady_area(WORK_AREA_FOLLOWS_AUTO_HIDE, environment, &display, area); Some((display, area)) }); if let Some((current, area)) = fitted.filter(|(display, area)| { @@ -431,7 +454,14 @@ fn validate_display(app: &AppHandle, display: Option<&Environment>) -> Result<() if !displays.contains(&expected.display) { return Err("Display geometry changed. Scanning restarts.".into()); } - if crate::scan_host::work_area(bounds(&expected.display))? != expected.keyboard_area { + let area = crate::scan_host::work_area(bounds(&expected.display))?; + if steady_area( + WORK_AREA_FOLLOWS_AUTO_HIDE, + expected, + &expected.display, + area, + ) != expected.keyboard_area + { return Err("Display work area changed. Scanning restarts.".into()); } } @@ -568,6 +598,44 @@ mod tests { } } #[test] + fn panels_keep_their_area_while_auto_hidden_bars_show_and_hide() { + let one = display("one", 0, 1920); + // Opened in a full-screen app, with the menu bar and Dock hidden. + let environment = Environment { + display: one.clone(), + foreground: None, + panel: true, + following: false, + keyboard_area: bounds(&one), + }; + let menu_bar_showing = Rect { + y: 24.0, + height: 1056.0, + ..bounds(&one) + }; + assert_eq!( + steady_area(true, &environment, &one, menu_bar_showing), + bounds(&one) + ); + // Windows keeps its work area when the taskbar hides, so a change is real. + assert_eq!( + steady_area(false, &environment, &one, menu_bar_showing), + menu_bar_showing + ); + // Another display, or this one at a new size, is fitted straight away. + for moved in [display("two", 1920, 1280), display("one", 0, 2560)] { + let area = Rect { + y: 24.0, + height: 1056.0, + ..bounds(&moved) + }; + for follows in [true, false] { + assert_eq!(steady_area(follows, &environment, &moved, area), area); + } + } + assert_eq!(WORK_AREA_FOLLOWS_AUTO_HIDE, cfg!(target_os = "macos")); + } + #[test] fn the_keyboard_stays_on_its_display_while_that_exists() { let one = display("one", 0, 1920); let two = display("two", 1920, 1280); From 096d2d4c386de72b766ad3ad71f3c795144fa3b8 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Sun, 11 Oct 2026 21:57:24 +0100 Subject: [PATCH 2/3] Read the macOS work area again for each Space Keeping a scan's work area for as long as it stayed on one display left panels sized for full screen after leaving it, covering a permanent Dock or menu bar. Panels now read the work area again when the active Space changes, following it for a second while it settles, and keep it until the next change. Point scanning keeps the area current for a keyboard it opens, and the macOS work-area check no longer pretends to compare anything. Co-Authored-By: Claude Opus 5.5 --- docs/point-scan.md | 2 +- docs/qwerty-keyboard.md | 2 +- docs/scanner-architecture.md | 2 +- src-tauri/src/macos.rs | 17 +++- src-tauri/src/point_scan_runtime.rs | 132 ++++++++++++++++++++++++---- 5 files changed, 133 insertions(+), 22 deletions(-) diff --git a/docs/point-scan.md b/docs/point-scan.md index b45a8137..4cb387e2 100644 --- a/docs/point-scan.md +++ b/docs/point-scan.md @@ -48,7 +48,7 @@ Drag retains the source and scans a destination on the same display without hold `point_workflow` separates point selection, menu navigation and typed action requests. Its current selection policy always opens the menu; a future auto-select policy can reuse `default_click` without changing the executor. No auto-select timer or setting exists. The reusable `scan_menu` defines stable action IDs and rows. Shared frames carry menu tiles, labels and highlight strips; all native windows remain click-through and nonactivating. -The desktop view adds menu, menuSuspended, dragDestination, dragConfirmation and executing phases. Existing point phases, saved settings and Bluetooth commands retain their shapes. Mobile connections, switch editing/learning and shutdown cancel the workflow. A change of display or work area cancels point scanning and its menus. On macOS, the menu bar and Dock showing or hiding does not count as a work area change; see [the keyboard](qwerty-keyboard.md). So does a change of the window in front, once a switch has moved or chosen anything in the scan, so that a point is never used on a window it was not chosen over. Until then a point scan follows whichever window is in front, and can run with none in front. If no window is in front at the moment of the first choice and one appears afterwards, the scan is cancelled. The keyboard and the mouse panel stay open through both kinds of change; see [the keyboard](qwerty-keyboard.md). Targets and foreground identity remain in memory and are never logged or emitted. Automated action tests use fake input only; physical Windows/macOS focus, scaling, target, scrolling and drag checks remain required for hardware qualification. +The desktop view adds menu, menuSuspended, dragDestination, dragConfirmation and executing phases. Existing point phases, saved settings and Bluetooth commands retain their shapes. Mobile connections, switch editing/learning and shutdown cancel the workflow. A change of display or work area cancels point scanning and its menus (on macOS, a change of display only; see [the keyboard](qwerty-keyboard.md)). So does a change of the window in front, once a switch has moved or chosen anything in the scan, so that a point is never used on a window it was not chosen over. Until then a point scan follows whichever window is in front, and can run with none in front. If no window is in front at the moment of the first choice and one appears afterwards, the scan is cancelled. The keyboard and the mouse panel stay open through both kinds of change; see [the keyboard](qwerty-keyboard.md). Targets and foreground identity remain in memory and are never logged or emitted. Automated action tests use fake input only; physical Windows/macOS focus, scaling, target, scrolling and drag checks remain required for hardware qualification. The action menu uses a fixed grid of square icon tiles with labels beneath the artwork, on the same dark rounded panel chrome as the scanning keyboard (`TileRole::Panel`). A yellow border and amber background identify the current row or item. The same artwork is drawn on Windows and macOS. Menu titles stay above the shared panel when a scan opens a menu or changes submenus, without taking keyboard focus. diff --git a/docs/qwerty-keyboard.md b/docs/qwerty-keyboard.md index 65cf4432..a7dad761 100644 --- a/docs/qwerty-keyboard.md +++ b/docs/qwerty-keyboard.md @@ -16,7 +16,7 @@ The bottom control row starts with Close keyboard, followed by Letters, Navigati Modifier keys are pressed only around each emitted shortcut and immediately released. Selecting a locked modifier does not hold that operating-system key while the scanner runs. Ordinary characters use text injection; command combinations and navigation use native key events. Native shortcuts retain the operating system's layout semantics. -The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit. On macOS the work area leaves out the menu bar and Dock only while they show, which in a full-screen app, or with either set to hide automatically, is only while the pointer is at the screen edge. So there, on the same display, the keyboard and mouse panel keep the area they opened with and do not jump as the bars come and go. They stay above both and never take clicks. Moving to another display, or this display changing size, still fits them again; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation. +The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit. On macOS the work area leaves out the menu bar and Dock only while they show. In a full-screen app, or with either set to hide automatically, that is only while the pointer is at the screen edge. So on macOS the keyboard and mouse panel read the work area again only when the Space changes (entering or leaving a full-screen app, or choosing another desktop), following it for a second while it settles. They keep that area on the same display and Space, so they do not jump as the bars come and go. They stay above both and never take clicks. Moving the Dock or changing its size on the same Space takes effect when the panel next opens or the Space changes. Moving to another display, or this display changing size, still fits them again; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation. ## Manual validation diff --git a/docs/scanner-architecture.md b/docs/scanner-architecture.md index 6db00dbe..d6c2eec7 100644 --- a/docs/scanner-architecture.md +++ b/docs/scanner-architecture.md @@ -110,7 +110,7 @@ Rules that hold whatever is chosen: - `wait` has no effect in an area that is not automatic, except that point scanning always waits unless `automatic` is chosen. - A point scan that starts by itself still stops at the pass limit. -Every point scan takes up whichever window is in front, or none, until a switch moves or chooses something in it. A click may have brought another window forward, and a panel may just have closed. From then on, a change of window ends the scan as usual. A change of display or work area ends a point scan at any time, except on macOS when only the menu bar or Dock shows or hides. The keyboard and the mouse panel are not tied to any window or display; see [the keyboard](qwerty-keyboard.md). +Every point scan takes up whichever window is in front, or none, until a switch moves or chooses something in it. A click may have brought another window forward, and a panel may just have closed. From then on, a change of window ends the scan as usual. A change of display ends a point scan at any time, and so does a change of work area except on macOS, where the work area is read again for each Space instead. The keyboard and the mouse panel are not tied to any window or display; see [the keyboard](qwerty-keyboard.md). `keyboardWaitAfterTyping`, saved by earlier versions, still makes the keyboard wait. It applies only while the keyboard has no `nextScan` of its own and the shared value is `standard`. Settings no longer offers or changes it; the saved value is kept. diff --git a/src-tauri/src/macos.rs b/src-tauri/src/macos.rs index 5e084d1e..c48ebffe 100644 --- a/src-tauri/src/macos.rs +++ b/src-tauri/src/macos.rs @@ -10,7 +10,8 @@ use enigo::{Enigo, Settings}; use objc2::rc::Retained; use objc2::runtime::{AnyObject, NSObjectProtocol, ProtocolObject}; use objc2_app_kit::{ - NSWorkspace, NSWorkspaceDidWakeNotification, NSWorkspaceWillSleepNotification, + NSWorkspace, NSWorkspaceActiveSpaceDidChangeNotification, NSWorkspaceDidWakeNotification, + NSWorkspaceWillSleepNotification, }; #[allow(deprecated)] use objc2_foundation::{NSHost, NSNotification, NSNotificationCenter, NSString, NSURL}; @@ -543,8 +544,20 @@ impl MacRuntime { &wake_block, ) }; + // Scanning panels read the work area again for each Space. + let space_block = RcBlock::new(|_notification: NonNull| { + crate::point_scan_runtime::space_changed(); + }); + let space_observer = unsafe { + center.addObserverForName_object_queue_usingBlock( + Some(NSWorkspaceActiveSpaceDidChangeNotification), + None, + None, + &space_block, + ) + }; self.notification_center = Some(center); - self.power_observers = vec![sleep_observer, wake_observer]; + self.power_observers = vec![sleep_observer, wake_observer, space_observer]; } fn handle_system_suspend(&mut self) -> Result<(), String> { diff --git a/src-tauri/src/point_scan_runtime.rs b/src-tauri/src/point_scan_runtime.rs index 207e77a8..b8e6190f 100644 --- a/src-tauri/src/point_scan_runtime.rs +++ b/src-tauri/src/point_scan_runtime.rs @@ -6,6 +6,13 @@ use crate::{ scanning::Rect, scanning_runtime::{self, Adapter}, }; +use std::{ + sync::{ + atomic::{AtomicU64, Ordering}, + Mutex, + }, + time::{Duration, Instant}, +}; use tauri::{AppHandle, Manager}; #[derive(Clone)] pub struct Environment { @@ -17,6 +24,8 @@ pub struct Environment { /// Nothing is chosen yet in the point scan, which takes up any window. following: bool, keyboard_area: Rect, + /// The Space `keyboard_area` was read on, counted by `space_changed`. + space: u64, } /// A point is chosen over one window and must not be used on another. fn window_changed(environment: &Environment, foreground: Option) -> bool { @@ -39,16 +48,46 @@ fn keyboard_display<'a>( /// show, so it changes whenever an auto-hidden one comes and goes, as in a /// full-screen app. Windows keeps its work area when the taskbar hides. const WORK_AREA_FOLLOWS_AUTO_HIDE: bool = cfg!(target_os = "macos"); +/// How long the work area may still be moving after the Space changes. +const SPACE_SETTLING: Duration = Duration::from_millis(1000); +static SPACE_CHANGES: AtomicU64 = AtomicU64::new(0); +static SPACE_CHANGED_AT: Mutex> = Mutex::new(None); +/// The active macOS Space changed: a full-screen app was entered or left, or +/// another desktop was chosen. Its menu bar and Dock may differ. +#[cfg_attr(not(target_os = "macos"), allow(dead_code))] +pub fn space_changed() { + *SPACE_CHANGED_AT.lock().unwrap_or_else(|p| p.into_inner()) = Some(Instant::now()); + SPACE_CHANGES.fetch_add(1, Ordering::SeqCst); +} +#[derive(Debug, Clone, Copy, PartialEq)] +struct Space { + changes: u64, + /// Just changed, so the work area is still read as it moves. + settling: bool, +} +fn space() -> Space { + let changed_at = *SPACE_CHANGED_AT.lock().unwrap_or_else(|p| p.into_inner()); + Space { + changes: SPACE_CHANGES.load(Ordering::SeqCst), + settling: changed_at.is_some_and(|at| at.elapsed() < SPACE_SETTLING), + } +} /// The area for panels on `display`. Where the work area follows auto-hidden -/// bars, the area this scan already has is kept while it stays on the same -/// display, so panels never jump as the bars show and hide. +/// bars, the area this scan already has is kept on the same display and +/// Space, so panels never jump as the bars show and hide. A new Space is read +/// afresh, since entering or leaving full screen changes what the bars take. fn steady_area( follows_auto_hide: bool, environment: &Environment, display: &Display, area: Rect, + space: Space, ) -> Rect { - if follows_auto_hide && environment.display == *display { + if follows_auto_hide + && environment.display == *display + && environment.space == space.changes + && !space.settling + { environment.keyboard_area } else { area @@ -279,6 +318,7 @@ impl Adapter for PointScan { technique: Option<&mut Workflow>, ) -> Result { if let (Some(environment), Some(technique)) = (environment, technique) { + let space = space(); environment.following = false; if technique.mouse_open() { environment.foreground = crate::scan_host::foreground().ok(); @@ -294,8 +334,13 @@ impl Adapter for PointScan { Some((cursor, displays.len(), current, area)) }); if let Some((cursor, displays, current, area)) = placed { - let area = - steady_area(WORK_AREA_FOLLOWS_AUTO_HIDE, environment, ¤t, area); + let area = steady_area( + WORK_AREA_FOLLOWS_AUTO_HIDE, + environment, + ¤t, + area, + space, + ); technique.set_mouse_area( area, bounds(¤t), @@ -354,8 +399,13 @@ impl Adapter for PointScan { }) .and_then(|display| { let area = crate::scan_host::work_area(bounds(&display)).ok()?; - let area = - steady_area(WORK_AREA_FOLLOWS_AUTO_HIDE, environment, &display, area); + let area = steady_area( + WORK_AREA_FOLLOWS_AUTO_HIDE, + environment, + &display, + area, + space, + ); Some((display, area)) }); if let Some((current, area)) = fitted.filter(|(display, area)| { @@ -388,6 +438,20 @@ impl Adapter for PointScan { } technique.set_pointer(None); } + // Point scanning ignores the work area, but a keyboard it opens + // fits the one for the Space it is on now. + if !environment.panel + && WORK_AREA_FOLLOWS_AUTO_HIDE + && (environment.space != space.changes || space.settling) + { + if let Ok(area) = crate::scan_host::work_area(bounds(&environment.display)) { + environment.keyboard_area = area; + technique.set_keyboard_area(area); + } + } + if !space.settling { + environment.space = space.changes; + } } Ok(true) } @@ -440,6 +504,7 @@ fn new_engine(app: &AppHandle, config: Config) -> Result<(Workflow, Environment) panel: false, following: false, keyboard_area, + space: space().changes, }, )) } @@ -454,13 +519,10 @@ fn validate_display(app: &AppHandle, display: Option<&Environment>) -> Result<() if !displays.contains(&expected.display) { return Err("Display geometry changed. Scanning restarts.".into()); } - let area = crate::scan_host::work_area(bounds(&expected.display))?; - if steady_area( - WORK_AREA_FOLLOWS_AUTO_HIDE, - expected, - &expected.display, - area, - ) != expected.keyboard_area + // On macOS the work area changes as the menu bar and Dock show, and + // the environment reads it again for each Space instead. + if !WORK_AREA_FOLLOWS_AUTO_HIDE + && crate::scan_host::work_area(bounds(&expected.display))? != expected.keyboard_area { return Err("Display work area changed. Scanning restarts.".into()); } @@ -585,6 +647,7 @@ mod tests { panel, following, keyboard_area: bounds(&display("one", 0, 1920)), + space: 0, }; for now in [None, Some(1), Some(2)] { for held in [None, Some(1)] { @@ -607,21 +670,43 @@ mod tests { panel: true, following: false, keyboard_area: bounds(&one), + space: 4, }; let menu_bar_showing = Rect { y: 24.0, height: 1056.0, ..bounds(&one) }; + let same_space = Space { + changes: 4, + settling: false, + }; assert_eq!( - steady_area(true, &environment, &one, menu_bar_showing), + steady_area(true, &environment, &one, menu_bar_showing, same_space), bounds(&one) ); // Windows keeps its work area when the taskbar hides, so a change is real. assert_eq!( - steady_area(false, &environment, &one, menu_bar_showing), + steady_area(false, &environment, &one, menu_bar_showing, same_space), menu_bar_showing ); + // Leaving full screen, or another desktop, is read afresh, including + // while its bars are still moving into place. + for space in [ + Space { + changes: 5, + settling: false, + }, + Space { + changes: 4, + settling: true, + }, + ] { + assert_eq!( + steady_area(true, &environment, &one, menu_bar_showing, space), + menu_bar_showing + ); + } // Another display, or this one at a new size, is fitted straight away. for moved in [display("two", 1920, 1280), display("one", 0, 2560)] { let area = Rect { @@ -630,12 +715,25 @@ mod tests { ..bounds(&moved) }; for follows in [true, false] { - assert_eq!(steady_area(follows, &environment, &moved, area), area); + assert_eq!( + steady_area(follows, &environment, &moved, area, same_space), + area + ); } } assert_eq!(WORK_AREA_FOLLOWS_AUTO_HIDE, cfg!(target_os = "macos")); } #[test] + fn a_space_change_is_counted_and_settles() { + let before = space(); + space_changed(); + let changed = space(); + assert!(changed.changes > before.changes); + assert!(changed.settling); + *SPACE_CHANGED_AT.lock().unwrap() = Some(Instant::now() - SPACE_SETTLING); + assert!(!space().settling); + } + #[test] fn the_keyboard_stays_on_its_display_while_that_exists() { let one = display("one", 0, 1920); let two = display("two", 1920, 1280); From 274aa4276bca6cc96cb2879b989cd904387db633 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 07:25:20 +0100 Subject: [PATCH 3/3] Settle a new Space only once its work area is read If the displays or work area could not be read on the first update after the settling window, the Space was still recorded and the old area kept until the next Space change. The Space is now recorded only on an update that read the area. Co-Authored-By: Claude Opus 5.5 --- docs/qwerty-keyboard.md | 2 +- src-tauri/src/point_scan_runtime.rs | 13 +++++++++++-- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/docs/qwerty-keyboard.md b/docs/qwerty-keyboard.md index a7dad761..b92a5315 100644 --- a/docs/qwerty-keyboard.md +++ b/docs/qwerty-keyboard.md @@ -16,7 +16,7 @@ The bottom control row starts with Close keyboard, followed by Letters, Navigati Modifier keys are pressed only around each emitted shortcut and immediately released. Selecting a locked modifier does not hold that operating-system key while the scanner runs. Ordinary characters use text injection; command combinations and navigation use native key events. Native shortcuts retain the operating system's layout semantics. -The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit. On macOS the work area leaves out the menu bar and Dock only while they show. In a full-screen app, or with either set to hide automatically, that is only while the pointer is at the screen edge. So on macOS the keyboard and mouse panel read the work area again only when the Space changes (entering or leaving a full-screen app, or choosing another desktop), following it for a second while it settles. They keep that area on the same display and Space, so they do not jump as the bars come and go. They stay above both and never take clicks. Moving the Dock or changing its size on the same Space takes effect when the panel next opens or the Space changes. Moving to another display, or this display changing size, still fits them again; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation. +The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit. On macOS the work area leaves out the menu bar and Dock only while they show. In a full-screen app, or with either set to hide automatically, that is only while the pointer is at the screen edge. So on macOS the keyboard and mouse panel read the work area again only when the Space changes (entering or leaving a full-screen app, or choosing another desktop), following it for a second while it settles. They keep that area on the same display and Space, so they do not jump as the bars come and go. They stay above both and never take clicks. Moving the Dock or changing its size on the same Space takes effect when the Space next changes or scanning starts again. Moving to another display, or this display changing size, still fits them again; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation. ## Manual validation diff --git a/src-tauri/src/point_scan_runtime.rs b/src-tauri/src/point_scan_runtime.rs index b8e6190f..5fe2cc9d 100644 --- a/src-tauri/src/point_scan_runtime.rs +++ b/src-tauri/src/point_scan_runtime.rs @@ -319,6 +319,9 @@ impl Adapter for PointScan { ) -> Result { if let (Some(environment), Some(technique)) = (environment, technique) { let space = space(); + // Whether this update read the work area, so a new Space is only + // settled once its area is known. + let mut read = false; environment.following = false; if technique.mouse_open() { environment.foreground = crate::scan_host::foreground().ok(); @@ -334,6 +337,7 @@ impl Adapter for PointScan { Some((cursor, displays.len(), current, area)) }); if let Some((cursor, displays, current, area)) = placed { + read = true; let area = steady_area( WORK_AREA_FOLLOWS_AUTO_HIDE, environment, @@ -408,6 +412,7 @@ impl Adapter for PointScan { ); Some((display, area)) }); + read = fitted.is_some(); if let Some((current, area)) = fitted.filter(|(display, area)| { environment.display != *display || environment.keyboard_area != *area }) { @@ -445,11 +450,12 @@ impl Adapter for PointScan { && (environment.space != space.changes || space.settling) { if let Ok(area) = crate::scan_host::work_area(bounds(&environment.display)) { + read = true; environment.keyboard_area = area; technique.set_keyboard_area(area); } } - if !space.settling { + if read && !space.settling { environment.space = space.changes; } } @@ -730,7 +736,10 @@ mod tests { let changed = space(); assert!(changed.changes > before.changes); assert!(changed.settling); - *SPACE_CHANGED_AT.lock().unwrap() = Some(Instant::now() - SPACE_SETTLING); + let Some(settled) = Instant::now().checked_sub(SPACE_SETTLING) else { + return; + }; + *SPACE_CHANGED_AT.lock().unwrap() = Some(settled); assert!(!space().settling); } #[test]