Skip to content

Conversation

Kobzol
Copy link
Member

@Kobzol Kobzol commented Aug 20, 2025

This PR adds the main logic required to execute job benchmarks in the collector.

It handles:

  • Updating the collector's heartbeat periodically
  • Quick loading of already downloaded sysroots, to avoid redownloading them in-between jobs and also collector restarts (useful for local testing)
  • Dequeing jobs, including in-progress jobs and expanding benchmark sets
  • Distinguishing between transient and permanent job errors, storing job errors into the DB, and marking jobs as failed or successful
  • Marking jobs that have been dequeued too many times as failed
  • Reconnecting to the DB if a transient I/O/network/DB error happens, to try to refresh the DB connection

@Kobzol Kobzol requested a review from Jamesbarford August 20, 2025 10:32
Copy link
Contributor

@Jamesbarford Jamesbarford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@Kobzol
Copy link
Member Author

Kobzol commented Aug 26, 2025

Ok, let's try.

@Kobzol Kobzol merged commit 7658080 into rust-lang:master Aug 26, 2025
11 checks passed
@Kobzol Kobzol deleted the benchmark-job-queue branch August 26, 2025 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants