Skip to content

[Dev]Speed up _wait_for_stable_network by using Playwright’s built-in load/networkidle #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

HongxiaoN
Copy link
Collaborator

Pull Request: Speed up _wait_for_stable_network by using Playwright’s built-in load/networkidle

Summary

Our replay runner was spending up to 20 s per step waiting for “network idle” even on fast pages, resulting in 160 s total for 11 steps. By switching to Playwright’s built-in wait_for_load_state('load') + wait_for_load_state('networkidle') with our configured timeouts, we cut total runtime down to 60 s — a 62 % improvement!

Benchmark

  • Before: 160 s total (≈ 14.5 s/step)
  • After: 60 s total (≈ 5.5 s/step)
  • Improvement: 62 % faster

🎥 Run command:

python run_replay.py \
  --model-provider openai \
  --wap_replay_list data_samples/replay_list_samples/wap_exact_replay_list_GqMnZeKFxvePGKGA.json \
  --max-concurrent 1

The default Playwright “networkidle” can wait up to 20 s (4 s × 5 polling) per call. We were repeatedly hitting our maximum_wait_page_load_time each step, so every step often paid the full timeout.

Changes

  • Replace custom request/response listeners + manual sleeps with:
    1. await page.wait_for_load_state('load', timeout=…)
    2. await page.wait_for_load_state('networkidle', timeout=…)
  • Honor our maximum_wait_page_load_time and wait_for_network_idle_page_load_time directly in ms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant