fix(eventbus): prevent cross-loop contamination in bubus EventBus - #5512
fix(eventbus): prevent cross-loop contamination in bubus EventBus#5512shobhitsahani wants to merge 1 commit into
Conversation
…owser-use#5509) ## Summary - Add `browser_use/bubus_compat.py`: an idempotent compatibility patch for the pinned `bubus==1.5.6` that stops `BaseEvent.__await__` from draining EventBuses owned by a different event loop. - Record each bus's owning loop in `_start`, and in `__await__`'s drain/polling loop skip any bus whose `_loop` is not the current running loop (and any stopped bus). Every other bus is already drained by its own `_run_loop` on its own loop. - Apply the patch at import time in `browser_use/__init__.py`, guarded by `try/except ImportError` so a bare `import browser_use` can never hard-fail on a missing or partially-installed bubus. - Add `tests/ci/test_bubus_cross_loop_isolation.py` with a regression test that fails on unpatched bubus 1.5.6 and passes with the fix. ## Why `bubus==1.5.6`'s `BaseEvent.__await__` iterates `EventBus.all_instances` — the process-global WeakSet of every bus — and calls `await bus.process_event(...)` on any bus with queued events, regardless of which event loop each bus belongs to. With parallel agent sessions on separate event loops (one EventBus per agent), one bus's handlers get executed on another bus's loop, where they hang forever, pile up, and eventually trip the 100-event capacity guard in `dispatch()`: ```text RuntimeError: EventBus at capacity: 100 pending events (100 max). Queue: 50, Processing: 50. Cannot accept new events until some complete. ``` The fix mirrors the canonical upstream change in browser-use/bubus#30 (not yet released), which makes the drain loop only touch buses started on the current running loop. The patch auto-no-ops via `hasattr(EventBus, '_loop')` once a fixed bubus is released. ## Reproduction Before the fix, a standalone repro with a bus on a second event loop showed the probe event run on the wrong (awaiting) loop: ```text probe ran on loop A (BUG): True RESULT: BUG REPRODUCED - cross-loop contamination ``` With the patch active the probe stays on its owning loop: ```text probe ran on loop A (BUG): False probe ran on loop B : True RESULT: OK - no cross-loop contamination ``` ## Tests - `uv run pytest tests/ci/test_bubus_cross_loop_isolation.py -q` — 2 passed - `uv run pytest tests/ci/test_event_bus_resilience.py -q` — 5 passed (existing warm-resume/Restart event-bus flows remain green) - Same-loop nested-await drain (the legit deadlock-avoidance path) still processes queued child events correctly. - `ruff check` and `ruff format --check` pass on the new/changed source; `pyright` reports 0 errors / 0 warnings; `codespell` clean.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3c657dd362
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # haven't started (_loop is None) or have been stopped (_is_running is | ||
| # False) — a stopped bus can keep _loop set with events still queued, and | ||
| # nothing should run its handlers after stop(). | ||
| if not bus._is_running or getattr(bus, '_loop', None) is not current_loop: |
There was a problem hiding this comment.
Handle buses that started before the patch
If an application has already started an EventBus before its first import browser_use, that bus never passes through _patched_start and therefore has no _loop. A nested await in one of its handlers will now skip its own queued child events here, spin through the 1,000-iteration fallback, and return before the child event completes. Since this monkeypatch changes the global BaseEvent behavior, initialize existing running buses from their run-loop task (or retain the prior behavior for untagged buses) rather than treating them as foreign-loop buses.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
4 issues found across 3 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="browser_use/bubus_compat.py">
<violation number="1" location="browser_use/bubus_compat.py:63">
P1: When an already-running bus receives `dispatch()` from another event loop, `_original_start()` is a no-op but this wrapper overwrites `_loop` with the caller's loop. Record ownership only when `_start()` transitions the bus from stopped to running.</violation>
<violation number="2" location="browser_use/bubus_compat.py:113">
P2: When an `EventBus` started before this patch is still running, it has no `_loop`, so this condition skips its queued child events even when it is the current loop's bus. A nested `await` can therefore hit the 1,000-iteration limit and return before the child completes; preserve the pre-patch fallback for untagged buses or initialize their owning loop.</violation>
<violation number="3" location="browser_use/bubus_compat.py:144">
P3: The `if iterations >= max_iterations:` block does nothing — its body is a commented-out `logger.error` and a `pass`. Since the drain loop can legitimately exhaust 1000 iterations, this placeholder silently swallows the case instead of reporting it. Remove the block, or replace it with a real `_logger.warning(...)` so hitting the iteration cap is observably logged.</violation>
</file>
<file name="tests/ci/test_bubus_cross_loop_isolation.py">
<violation number="1" location="tests/ci/test_bubus_cross_loop_isolation.py:133">
P3: loop_b is never close()d and thread.join(timeout=5) is silently ignored, so a failing run can leave the background thread running loop_b and leak into later tests, contradicting the finally block's comment. Also, `bus_b = ...result(timeout=5)` sits outside the try, so any failure before it (build_bus_b raising, or the 5s timeout) skips the finally entirely and leaks the still-spinning loop and thread unconditionally. Move loop_b/thread creation inside the guarded region and call loop_b.close() once the thread has exited.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
|
|
||
| def _patched_start(self: EventBus) -> None: | ||
| """Record the owning event loop so BaseEvent.__await__ can stay on-loop.""" | ||
| _original_start(self) |
There was a problem hiding this comment.
P1: When an already-running bus receives dispatch() from another event loop, _original_start() is a no-op but this wrapper overwrites _loop with the caller's loop. Record ownership only when _start() transitions the bus from stopped to running.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At browser_use/bubus_compat.py, line 63:
<comment>When an already-running bus receives `dispatch()` from another event loop, `_original_start()` is a no-op but this wrapper overwrites `_loop` with the caller's loop. Record ownership only when `_start()` transitions the bus from stopped to running.</comment>
<file context>
@@ -0,0 +1,160 @@
+
+ def _patched_start(self: EventBus) -> None:
+ """Record the owning event loop so BaseEvent.__await__ can stay on-loop."""
+ _original_start(self)
+ try:
+ loop = asyncio.get_running_loop()
</file context>
| # haven't started (_loop is None) or have been stopped (_is_running is | ||
| # False) — a stopped bus can keep _loop set with events still queued, and | ||
| # nothing should run its handlers after stop(). | ||
| if not bus._is_running or getattr(bus, '_loop', None) is not current_loop: |
There was a problem hiding this comment.
P2: When an EventBus started before this patch is still running, it has no _loop, so this condition skips its queued child events even when it is the current loop's bus. A nested await can therefore hit the 1,000-iteration limit and return before the child completes; preserve the pre-patch fallback for untagged buses or initialize their owning loop.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At browser_use/bubus_compat.py, line 113:
<comment>When an `EventBus` started before this patch is still running, it has no `_loop`, so this condition skips its queued child events even when it is the current loop's bus. A nested `await` can therefore hit the 1,000-iteration limit and return before the child completes; preserve the pre-patch fallback for untagged buses or initialize their owning loop.</comment>
<file context>
@@ -0,0 +1,160 @@
+ # haven't started (_loop is None) or have been stopped (_is_running is
+ # False) — a stopped bus can keep _loop set with events still queued, and
+ # nothing should run its handlers after stop().
+ if not bus._is_running or getattr(bus, '_loop', None) is not current_loop:
+ continue
+
</file context>
| _logger.debug(f'Polling loop cancelled for {self}') | ||
| raise | ||
|
|
||
| if iterations >= max_iterations: |
There was a problem hiding this comment.
P3: The if iterations >= max_iterations: block does nothing — its body is a commented-out logger.error and a pass. Since the drain loop can legitimately exhaust 1000 iterations, this placeholder silently swallows the case instead of reporting it. Remove the block, or replace it with a real _logger.warning(...) so hitting the iteration cap is observably logged.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At browser_use/bubus_compat.py, line 144:
<comment>The `if iterations >= max_iterations:` block does nothing — its body is a commented-out `logger.error` and a `pass`. Since the drain loop can legitimately exhaust 1000 iterations, this placeholder silently swallows the case instead of reporting it. Remove the block, or replace it with a real `_logger.warning(...)` so hitting the iteration cap is observably logged.</comment>
<file context>
@@ -0,0 +1,160 @@
+ _logger.debug(f'Polling loop cancelled for {self}')
+ raise
+
+ if iterations >= max_iterations:
+ # logger.error(f'Max iterations reached while waiting for {self}')
+ pass
</file context>
| if bus_a is not None: | ||
| await bus_a.stop() | ||
| loop_b.call_soon_threadsafe(loop_b.stop) | ||
| thread.join(timeout=5) |
There was a problem hiding this comment.
P3: loop_b is never close()d and thread.join(timeout=5) is silently ignored, so a failing run can leave the background thread running loop_b and leak into later tests, contradicting the finally block's comment. Also, bus_b = ...result(timeout=5) sits outside the try, so any failure before it (build_bus_b raising, or the 5s timeout) skips the finally entirely and leaks the still-spinning loop and thread unconditionally. Move loop_b/thread creation inside the guarded region and call loop_b.close() once the thread has exited.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/ci/test_bubus_cross_loop_isolation.py, line 133:
<comment>loop_b is never close()d and thread.join(timeout=5) is silently ignored, so a failing run can leave the background thread running loop_b and leak into later tests, contradicting the finally block's comment. Also, `bus_b = ...result(timeout=5)` sits outside the try, so any failure before it (build_bus_b raising, or the 5s timeout) skips the finally entirely and leaks the still-spinning loop and thread unconditionally. Move loop_b/thread creation inside the guarded region and call loop_b.close() once the thread has exited.</comment>
<file context>
@@ -0,0 +1,133 @@
+ if bus_a is not None:
+ await bus_a.stop()
+ loop_b.call_soon_threadsafe(loop_b.stop)
+ thread.join(timeout=5)
</file context>
|
Thank you so much for your fix! |
Fixes #5509
This PR fixes a cross-loop bug in bubus==1.5.6 that causes parallel agent sessions to fail with:
RuntimeError: EventBus at capacity: 100 pending events (100 max).
Queue: 50, Processing: 50. Cannot accept new events until some complete.
Root cause
BaseEvent.await drains every EventBus in the process (EventBus.all_instances) regardless of which event loop each bus belongs to. When handlers await child events while holding the global lock, the drain loop runs other buses' handlers on the wrong event loop — where they hang, pile up, and overflow the bus capacity.
The fix
Tests
Summary by cubic
Prevents cross-loop event processing in
bubusso parallel agent sessions no longer hang and hit the 100-event capacity. Old:BaseEvent.__await__drained every process-wideEventBus, running handlers on the wrong loop; New: it only drains buses on the current loop, leaving others to their own run loops.browser_use/bubus_compat.py: records anEventBus’s owning loop in_startand patchesBaseEvent.__await__to skip buses not on the current loop or that are stopped; idempotent and auto no-op once upstreambubusexposes_loop.browser_use/__init__.py, guarded withtry/except ImportErrorsoimport browser_usenever hard-fails.tests/ci/test_bubus_cross_loop_isolation.pyto lock the behavior; existing resilience tests remain green.Written for commit 3c657dd. Summary will update on new commits.