Skip to content

[bugfix] Introduce optional file locking for appending to perflogs #3523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

vkarak
Copy link
Contributor

@vkarak vkarak commented Jul 17, 2025

This PR introduces two new configuration variables (locking_enable and locking_file_mode) for the filelog handler. Setting this will cause the filelog handler to use file locking when appending to the perflog files.

This PR also introduces the following improvements:

  • Introduces a new utility function osext.flock() to handle the creation of the lock file for the different Python versions; this function is now used also for the DB.
  • Make sure the logging mechanism is shut down correctly when switching configurations and when the reframe exits.
  • Use Python syntax for true and false in the configuration reference to be consistent with the use of None instead of the JSON's null.

Closes #3521.

@vkarak vkarak added this to the ReFrame 4.8.3 milestone Jul 17, 2025
@vkarak vkarak requested review from ekouts and teojgo July 17, 2025 13:16
@vkarak vkarak self-assigned this Jul 17, 2025
@vkarak vkarak changed the title Introduce optional file locking for appending to perflogs [bugfix] Introduce optional file locking for appending to perflogs Jul 17, 2025
@vkarak vkarak force-pushed the bugfix/lock-filelog-append branch 2 times, most recently from a10625f to 353c6d8 Compare July 21, 2025 14:30
Also:

- Introduce an `osext.flock()` utility function to handle the underlying
  locking for the different Python versions.
- Use Python syntax for ``true`` and ``false`` in the configuration
  reference to be consistent with the use of ``None`` instead of the
  JSON's ``null``.
@vkarak vkarak force-pushed the bugfix/lock-filelog-append branch from 353c6d8 to fac5691 Compare July 22, 2025 07:11
@vkarak vkarak marked this pull request as draft July 23, 2025 14:16
@vkarak
Copy link
Contributor Author

vkarak commented Jul 23, 2025

I'm converting this to draft since the fix has to be designed in accordance with #3532.

@vkarak vkarak marked this pull request as ready for review July 24, 2025 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

Perflogs on NFS get corrupted when updated by multiple reframe processes
1 participant