Skip to content

[feat] Log all tests in perfloggers, not only performance tests #3516

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: develop
Choose a base branch
from
4 changes: 4 additions & 0 deletions docs/tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2072,6 +2072,10 @@ Let's walk briefly through the most important parts of its configuration:

Let's rerun our STREAM example using the new configuration:

.. note::

After Reframe 4.9 ReFrame will add entries in the perflogs for all tests, not only performance tests.

.. code-block:: bash
:caption: Run with the Docker compose setup.

Expand Down
16 changes: 14 additions & 2 deletions reframe/core/logging.py
Original file line number Diff line number Diff line change
Expand Up @@ -881,8 +881,10 @@ def _update_check_extras(self):
'%FT%T%:z'
)

def log_performance(self, level, task, msg=None, multiline=False):
if self.check is None or not self.check.is_performance_check():
def log_result(
self, level, task, msg=None, multiline=False
):
if self.check is None:
return

_, part, env = task.testcase
Expand Down Expand Up @@ -911,6 +913,16 @@ def log_performance(self, level, task, msg=None, multiline=False):
self.extra['check_perf_unit'] = unit
self.extra['check_perf_result'] = result
self.log(level, msg)

if not self.check.perfvalues:
self.extra['check_perf_var'] = "$sanity_dummy"
self.extra['check_perf_value'] = None
self.extra['check_perf_ref'] = None
self.extra['check_perf_lower_thres'] = None
self.extra['check_perf_upper_thres'] = None
self.extra['check_perf_unit'] = None
self.extra['check_perf_result'] = None
self.log(level, msg)
else:
self.log(level, msg)

Expand Down
8 changes: 4 additions & 4 deletions reframe/frontend/executors/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -525,8 +525,8 @@ def finalize(self):
self._current_stage = 'finalize'
self._notify_listeners('on_task_success')
try:
self._perflogger.log_performance(logging.INFO, self,
multiline=self._perflog_compat)
self._perflogger.log_result(logging.INFO, self,
multiline=self._perflog_compat)
except LoggingError as e:
getlogger().warning(
f'could not log performance data for {self.testcase}: {e}'
Expand All @@ -550,8 +550,8 @@ def _wait_job(job):
self._exc_info = exc_info or sys.exc_info()
self._notify_listeners(callback)
try:
self._perflogger.log_performance(logging.INFO, self,
multiline=self._perflog_compat)
self._perflogger.log_result(logging.INFO, self,
multiline=self._perflog_compat)
except LoggingError as e:
getlogger().warning(
f'could not log performance data for {self.testcase}: {e}'
Expand Down
Loading