Repository navigation
Agent's browser live in the right panel - #221
Merged
Merged
Conversation
…gent presses announced
…-agent browsers, user control
…ser-panel feature Review | Browser switch in the panel header, remembered per session. The view lists the session's browsers from BrowsersChanged events, watches the active tab (or a picked one) through a BrowserView, decodes frames off the main thread and pulses where the agent clicks. The panel switches to the browser when the agent opens its first one.
…ext and editing commands, navigation
While the user has control the panel forwards mouse, wheel and keys to the page (macOS editing shortcuts carry their command, an input method composes through EntityInputHandler, copy/cut/paste use the system clipboard), and the toolbar navigates: back, forward, reload and an editable address. Hand back returns the browser to the agent.
…debug logs for the panel's frames
GPUI's event loop ran inside the #[tokio::main] future, in one poll that never returns, so tokio's cooperation budget on the main thread was never renewed. After about 128 tokio channel operations there, every tokio watch or broadcast awaited on the main thread stayed pending: the browser panel stopped showing frames while the screencast went on. main is synchronous now; the async modes run on a runtime of their own.
Each launched browser carries an instance number; the listing reports it and the panel watches key, instance and tab, so a browser closed and opened again (with tab t1 again) no longer leaves the panel on the dead tab.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Shows the agent's browser live in the right panel, next to the review view (steps 1–4 of
docs/browser-panel-plan.md). Chrome keeps running headless in its own process; the panel mirrors it over a CDP screencast. The GPUI part sits behind the Cargo featurebrowser-panel, which is on by default.What the panel does
EntityInputHandler, and copy, cut and paste use the system clipboard.Layers
web:record, running only while someone watches;code_assistant_core:SessionBrowserscovers the agent's browsers plus those of running sub-agents;UiEvent::BrowsersChangedcarries metadata only, never frames;SessionService::{watch_browsers, browser_view, set_browser_control};BrowserViewwith an ordered input queue.ui_gpui:right_panel/browser/(view, toolbar, input, key mapping, geometry). The view switch of the left sidebar became the sharedsegmented_switch.Fixes found on the way
mainis no longer#[tokio::main]. GPUI's event loop ran inside one never-ending poll of tokio'sblock_on, so tokio's cooperation budget on the main thread was never renewed. After about 128 operations, every tokiowatch/broadcastawaited in a GPUI foreground task stayed pending. The panel froze while the screencast went on. This may explain other stalls seen on the main thread.Cannot find context with specified id) is retried.Testing
web: tests against real Chrome for live frames, recording during a live view, a start during navigation, tab state and popups, viewer lifetime, user control and raw input.ui_gpui: tests for the geometry mapping, the key mapping and address schemes.cargo clippy --all-targets --all-features -D warningsis clean, and so is a build with--no-default-features --features gpui-frontend.Not in this PR
<select>overlay and file chooser (step 5) wait for the CEF decision.a_hung_page_fails_fastfailed twice under load early on this branch and could not be reproduced afterwards.