Keep panels still as the macOS menu bar and Dock show and hide

docs/point-scan.md

@@ -48,7 +48,7 @@ Drag retains the source and scans a destination on the same display without hold
 
 `point_workflow` separates point selection, menu navigation and typed action requests. Its current selection policy always opens the menu; a future auto-select policy can reuse `default_click` without changing the executor. No auto-select timer or setting exists. The reusable `scan_menu` defines stable action IDs and rows. Shared frames carry menu tiles, labels and highlight strips; all native windows remain click-through and nonactivating.
 
-The desktop view adds menu, menuSuspended, dragDestination, dragConfirmation and executing phases. Existing point phases, saved settings and Bluetooth commands retain their shapes. Mobile connections, switch editing/learning and shutdown cancel the workflow. A change of display or work area cancels point scanning and its menus. So does a change of the window in front, once a switch has moved or chosen anything in the scan, so that a point is never used on a window it was not chosen over. Until then a point scan follows whichever window is in front, and can run with none in front. If no window is in front at the moment of the first choice and one appears afterwards, the scan is cancelled. The keyboard and the mouse panel stay open through both kinds of change; see [the keyboard](qwerty-keyboard.md). Targets and foreground identity remain in memory and are never logged or emitted. Automated action tests use fake input only; physical Windows/macOS focus, scaling, target, scrolling and drag checks remain required for hardware qualification.
+The desktop view adds menu, menuSuspended, dragDestination, dragConfirmation and executing phases. Existing point phases, saved settings and Bluetooth commands retain their shapes. Mobile connections, switch editing/learning and shutdown cancel the workflow. A change of display or work area cancels point scanning and its menus (on macOS, a change of display only; see [the keyboard](qwerty-keyboard.md)). So does a change of the window in front, once a switch has moved or chosen anything in the scan, so that a point is never used on a window it was not chosen over. Until then a point scan follows whichever window is in front, and can run with none in front. If no window is in front at the moment of the first choice and one appears afterwards, the scan is cancelled. The keyboard and the mouse panel stay open through both kinds of change; see [the keyboard](qwerty-keyboard.md). Targets and foreground identity remain in memory and are never logged or emitted. Automated action tests use fake input only; physical Windows/macOS focus, scaling, target, scrolling and drag checks remain required for hardware qualification.
 
 The action menu uses a fixed grid of square icon tiles with labels beneath the artwork, on the same dark rounded panel chrome as the scanning keyboard (`TileRole::Panel`). A yellow border and amber background identify the current row or item. The same artwork is drawn on Windows and macOS. Menu titles stay above the shared panel when a scan opens a menu or changes submenus, without taking keyboard focus.
 

docs/qwerty-keyboard.md

@@ -16,7 +16,7 @@ The bottom control row starts with Close keyboard, followed by Letters, Navigati
 
 Modifier keys are pressed only around each emitted shortcut and immediately released. Selecting a locked modifier does not hold that operating-system key while the scanner runs. Ordinary characters use text injection; command combinations and navigation use native key events. Native shortcuts retain the operating system's layout semantics.
 
-The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation.
+The keyboard and the mouse panel do not depend on other windows. They open and keep working when no window is in front. When the foreground application changes the keyboard stays open and keeps its modifiers, Caps and highlight, and sends subsequent keys to the new foreground application. Only what belongs to the text of one window is let go: the suggestions and prediction context, a space or capital the keyboard owns, and any input still held. Suggestions are unavailable while no window is in front. When its display or work area changes, for example when a taskbar hides, the keyboard moves to fit. On macOS the work area leaves out the menu bar and Dock only while they show. In a full-screen app, or with either set to hide automatically, that is only while the pointer is at the screen edge. So on macOS the keyboard and mouse panel read the work area again only when the Space changes (entering or leaving a full-screen app, or choosing another desktop), following it for a second while it settles. They keep that area on the same display and Space, so they do not jump as the bars come and go. They stay above both and never take clicks. Moving the Dock or changing its size on the same Space takes effect when the Space next changes or scanning starts again. Moving to another display, or this display changing size, still fits them again; if its display is removed it moves to the display under the pointer, and while the displays cannot be read it stays where it is. The mouse panel does the same on the display under the pointer. It closes when its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation.
 
 ## Manual validation
 

docs/scanner-architecture.md

@@ -110,7 +110,7 @@ Rules that hold whatever is chosen:
 - `wait` has no effect in an area that is not automatic, except that point scanning always waits unless `automatic` is chosen.
 - A point scan that starts by itself still stops at the pass limit.
 
-Every point scan takes up whichever window is in front, or none, until a switch moves or chooses something in it. A click may have brought another window forward, and a panel may just have closed. From then on, a change of window ends the scan as usual. A change of display or work area ends a point scan at any time. The keyboard and the mouse panel are not tied to any window or display; see [the keyboard](qwerty-keyboard.md).
+Every point scan takes up whichever window is in front, or none, until a switch moves or chooses something in it. A click may have brought another window forward, and a panel may just have closed. From then on, a change of window ends the scan as usual. A change of display ends a point scan at any time, and so does a change of work area except on macOS, where the work area is read again for each Space instead. The keyboard and the mouse panel are not tied to any window or display; see [the keyboard](qwerty-keyboard.md).
 
 `keyboardWaitAfterTyping`, saved by earlier versions, still makes the keyboard wait. It applies only while the keyboard has no `nextScan` of its own and the shared value is `standard`. Settings no longer offers or changes it; the saved value is kept.
 

src-tauri/src/macos.rs

@@ -10,7 +10,8 @@ use enigo::{Enigo, Settings};
 use objc2::rc::Retained;
 use objc2::runtime::{AnyObject, NSObjectProtocol, ProtocolObject};
 use objc2_app_kit::{
-    NSWorkspace, NSWorkspaceDidWakeNotification, NSWorkspaceWillSleepNotification,
+    NSWorkspace, NSWorkspaceActiveSpaceDidChangeNotification, NSWorkspaceDidWakeNotification,
+    NSWorkspaceWillSleepNotification,
 };
 #[allow(deprecated)]
 use objc2_foundation::{NSHost, NSNotification, NSNotificationCenter, NSString, NSURL};
@@ -543,8 +544,20 @@ impl MacRuntime {
                 &wake_block,
             )
         };
+        // Scanning panels read the work area again for each Space.
+        let space_block = RcBlock::new(|_notification: NonNull<NSNotification>| {
+            crate::point_scan_runtime::space_changed();
+        });
+        let space_observer = unsafe {
+            center.addObserverForName_object_queue_usingBlock(
+                Some(NSWorkspaceActiveSpaceDidChangeNotification),
+                None,
+                None,
+                &space_block,
+            )
+        };
         self.notification_center = Some(center);
-        self.power_observers = vec![sleep_observer, wake_observer];
+        self.power_observers = vec![sleep_observer, wake_observer, space_observer];
     }
 
     fn handle_system_suspend(&mut self) -> Result<(), String> {

src-tauri/src/point_scan_runtime.rs

@@ -6,6 +6,13 @@ use crate::{
     scanning::Rect,
     scanning_runtime::{self, Adapter},
 };
+use std::{
+    sync::{
+        atomic::{AtomicU64, Ordering},
+        Mutex,
+    },
+    time::{Duration, Instant},
+};
 use tauri::{AppHandle, Manager};
 #[derive(Clone)]
 pub struct Environment {
@@ -17,6 +24,8 @@ pub struct Environment {
     /// Nothing is chosen yet in the point scan, which takes up any window.
     following: bool,
     keyboard_area: Rect,
+    /// The Space `keyboard_area` was read on, counted by `space_changed`.
+    space: u64,
 }
 /// A point is chosen over one window and must not be used on another.
 fn window_changed(environment: &Environment, foreground: Option<usize>) -> bool {
@@ -35,6 +44,55 @@ fn keyboard_display<'a>(
         .or_else(|| displays.iter().find(|display| display.name == current.name))
         .or_else(|| display_navigation::current_display(cursor, displays))
 }
+/// On macOS the work area leaves out the menu bar and Dock only while they
+/// show, so it changes whenever an auto-hidden one comes and goes, as in a
+/// full-screen app. Windows keeps its work area when the taskbar hides.
+const WORK_AREA_FOLLOWS_AUTO_HIDE: bool = cfg!(target_os = "macos");
+/// How long the work area may still be moving after the Space changes.
+const SPACE_SETTLING: Duration = Duration::from_millis(1000);
+static SPACE_CHANGES: AtomicU64 = AtomicU64::new(0);
+static SPACE_CHANGED_AT: Mutex<Option<Instant>> = Mutex::new(None);
+/// The active macOS Space changed: a full-screen app was entered or left, or
+/// another desktop was chosen. Its menu bar and Dock may differ.
+#[cfg_attr(not(target_os = "macos"), allow(dead_code))]
+pub fn space_changed() {
+    *SPACE_CHANGED_AT.lock().unwrap_or_else(|p| p.into_inner()) = Some(Instant::now());
+    SPACE_CHANGES.fetch_add(1, Ordering::SeqCst);
+}
+#[derive(Debug, Clone, Copy, PartialEq)]
+struct Space {
+    changes: u64,
+    /// Just changed, so the work area is still read as it moves.
+    settling: bool,
+}
+fn space() -> Space {
+    let changed_at = *SPACE_CHANGED_AT.lock().unwrap_or_else(|p| p.into_inner());
+    Space {
+        changes: SPACE_CHANGES.load(Ordering::SeqCst),
+        settling: changed_at.is_some_and(|at| at.elapsed() < SPACE_SETTLING),
+    }
+}
+/// The area for panels on `display`. Where the work area follows auto-hidden
+/// bars, the area this scan already has is kept on the same display and
+/// Space, so panels never jump as the bars show and hide. A new Space is read
+/// afresh, since entering or leaving full screen changes what the bars take.
+fn steady_area(
+    follows_auto_hide: bool,
+    environment: &Environment,
+    display: &Display,
+    area: Rect,
+    space: Space,
+) -> Rect {
+    if follows_auto_hide
+        && environment.display == *display
+        && environment.space == space.changes
+        && !space.settling
+    {
+        environment.keyboard_area
+    } else {
+        area
+    }
+}
 fn bounds(display: &Display) -> Rect {
     Rect {
         x: display.x.into(),
@@ -260,6 +318,10 @@ impl Adapter for PointScan {
         technique: Option<&mut Workflow>,
     ) -> Result<bool, String> {
         if let (Some(environment), Some(technique)) = (environment, technique) {
+            let space = space();
+            // Whether this update read the work area, so a new Space is only
+            // settled once its area is known.
+            let mut read = false;
             environment.following = false;
             if technique.mouse_open() {
                 environment.foreground = crate::scan_host::foreground().ok();
@@ -275,6 +337,14 @@ impl Adapter for PointScan {
                             Some((cursor, displays.len(), current, area))
                         });
                 if let Some((cursor, displays, current, area)) = placed {
+                    read = true;
+                    let area = steady_area(
+                        WORK_AREA_FOLLOWS_AUTO_HIDE,
+                        environment,
+                        &current,
+                        area,
+                        space,
+                    );
                     technique.set_mouse_area(
                         area,
                         bounds(&current),
@@ -333,8 +403,16 @@ impl Adapter for PointScan {
                     })
                     .and_then(|display| {
                         let area = crate::scan_host::work_area(bounds(&display)).ok()?;
+                        let area = steady_area(
+                            WORK_AREA_FOLLOWS_AUTO_HIDE,
+                            environment,
+                            &display,
+                            area,
+                            space,
+                        );
                         Some((display, area))
                     });
+                read = fitted.is_some();
                 if let Some((current, area)) = fitted.filter(|(display, area)| {
                     environment.display != *display || environment.keyboard_area != *area
                 }) {
@@ -365,6 +443,21 @@ impl Adapter for PointScan {
                 }
                 technique.set_pointer(None);
             }
+            // Point scanning ignores the work area, but a keyboard it opens
+            // fits the one for the Space it is on now.
+            if !environment.panel
+                && WORK_AREA_FOLLOWS_AUTO_HIDE
+                && (environment.space != space.changes || space.settling)
+            {
+                if let Ok(area) = crate::scan_host::work_area(bounds(&environment.display)) {
+                    read = true;
+                    environment.keyboard_area = area;
+                    technique.set_keyboard_area(area);
+                }
+            }
+            if read && !space.settling {
+                environment.space = space.changes;
+            }
         }
         Ok(true)
     }
@@ -417,6 +510,7 @@ fn new_engine(app: &AppHandle, config: Config) -> Result<(Workflow, Environment)
             panel: false,
             following: false,
             keyboard_area,
+            space: space().changes,
         },
     ))
 }
@@ -431,7 +525,11 @@ fn validate_display(app: &AppHandle, display: Option<&Environment>) -> Result<()
         if !displays.contains(&expected.display) {
             return Err("Display geometry changed. Scanning restarts.".into());
         }
-        if crate::scan_host::work_area(bounds(&expected.display))? != expected.keyboard_area {
+        // On macOS the work area changes as the menu bar and Dock show, and
+        // the environment reads it again for each Space instead.
+        if !WORK_AREA_FOLLOWS_AUTO_HIDE
+            && crate::scan_host::work_area(bounds(&expected.display))? != expected.keyboard_area
+        {
             return Err("Display work area changed. Scanning restarts.".into());
         }
     }
@@ -555,6 +653,7 @@ mod tests {
             panel,
             following,
             keyboard_area: bounds(&display("one", 0, 1920)),
+            space: 0,
         };
         for now in [None, Some(1), Some(2)] {
             for held in [None, Some(1)] {
@@ -568,6 +667,82 @@ mod tests {
         }
     }
     #[test]
+    fn panels_keep_their_area_while_auto_hidden_bars_show_and_hide() {
+        let one = display("one", 0, 1920);
+        // Opened in a full-screen app, with the menu bar and Dock hidden.
+        let environment = Environment {
+            display: one.clone(),
+            foreground: None,
+            panel: true,
+            following: false,
+            keyboard_area: bounds(&one),
+            space: 4,
+        };
+        let menu_bar_showing = Rect {
+            y: 24.0,
+            height: 1056.0,
+            ..bounds(&one)
+        };
+        let same_space = Space {
+            changes: 4,
+            settling: false,
+        };
+        assert_eq!(
+            steady_area(true, &environment, &one, menu_bar_showing, same_space),
+            bounds(&one)
+        );
+        // Windows keeps its work area when the taskbar hides, so a change is real.
+        assert_eq!(
+            steady_area(false, &environment, &one, menu_bar_showing, same_space),
+            menu_bar_showing
+        );
+        // Leaving full screen, or another desktop, is read afresh, including
+        // while its bars are still moving into place.
+        for space in [
+            Space {
+                changes: 5,
+                settling: false,
+            },
+            Space {
+                changes: 4,
+                settling: true,
+            },
+        ] {
+            assert_eq!(
+                steady_area(true, &environment, &one, menu_bar_showing, space),
+                menu_bar_showing
+            );
+        }
+        // Another display, or this one at a new size, is fitted straight away.
+        for moved in [display("two", 1920, 1280), display("one", 0, 2560)] {
+            let area = Rect {
+                y: 24.0,
+                height: 1056.0,
+                ..bounds(&moved)
+            };
+            for follows in [true, false] {
+                assert_eq!(
+                    steady_area(follows, &environment, &moved, area, same_space),
+                    area
+                );
+            }
+        }
+        assert_eq!(WORK_AREA_FOLLOWS_AUTO_HIDE, cfg!(target_os = "macos"));
+    }
+    #[test]
+    fn a_space_change_is_counted_and_settles() {
+        let before = space();
+        space_changed();
+        let changed = space();
+        assert!(changed.changes > before.changes);
+        assert!(changed.settling);
+        let Some(settled) = Instant::now().checked_sub(SPACE_SETTLING) else {
+            return;
+        };
+        *SPACE_CHANGED_AT.lock().unwrap() = Some(settled);
+        assert!(!space().settling);
+    }
+    #[test]
     fn the_keyboard_stays_on_its_display_while_that_exists() {
         let one = display("one", 0, 1920);
         let two = display("two", 1920, 1280);