Skip to content

Remove output of module commands in log files #4937

@Flamefire

Description

@Flamefire

The output of the lmod command in the log makes the test report file miss (almost) all important info. E.g.: https://gist.github.com/Flamefire/704e041f5696e98b6b8ecb0ef98ec02f

I.e. there is a large chunk of those:

del os.environ["_ModuleTable071_"]
os.environ["_ModuleTable072_"] = ''
del os.environ["_ModuleTable072_"]
<etc>
...
_mlstatus = True

== 2025-06-25 17:00:38,652 run.py:611 INFO Warnings and errors of '/usr/share/lmod/lmod/libexec/lmod ...' shell command (stderr only):

== 2025-06-25 17:00:38,652 run.py:616 INFO Shell command completed successfully (see output above): /usr/share/lmod/lmod/libexec/lmod python unload TensorFlow/2.18.1-CUDA-12.6.0
== 2025-06-25 17:00:38,655 environment.py:93 INFO Environment variable LD_LIBRARY_PATH set to /software/protobuf/28.0-GCCcore-13.3.0/lib:/software/Abseil/20240722.0-GCCcore-13.3.0/lib:/software/snappy/1.2.1-GCCcore-13.3.0/lib:/software/libpng/1.6.43-GCCcore-13.3.0/lib:/software/nsync/1.29.2-GCCcore-13.3.0/lib:/software/libjpeg-turbo/3.0.1-GCCcore-13.3.0/lib:/software/JsonCpp/1.9.5-GCCcore-13.3.0/lib:/software/giflib/5.2.1-GCCcore-13.3.0/lib:/software/flatbuffers/24.3.25-GCCcore-13.3.0/lib:/software/double-conversion/3.3.0-GCCcore-13.3.0/lib:/software/HDF5/1.14.5-gompi-2024a/lib:/software/Szip/2.1.1-GCCcore-13.3.0/lib:/software/SciPy-bundle/2024.05-gfbf-2024a/lib/python3.12/site-packages/numpy/core/lib:/software/genoa/r24.04/NCCL/2.22.3-GCCcore-13.3.0-CUDA-12.6.0/lib:/software/genoa/r24.04/GDRCopy/2.4.1-GCCcore-13.3.0/lib:/software/cuDNN/9.5.0.50-CUDA-12.6.0/lib:/software/CUDA/12.6.0/nvvm/lib64:/software/CUDA/12.6.0/extras/CUPTI/lib64:/software/CUDA/12.6.0/targets/x86_64-linux/lib:/software/Boost/1.85.0-GCC-13.3.0/lib:/software/ICU/75.1-GCCcore-13.3.0/lib:/software/zstd/1.5.6-GCCcore-13.3.0/lib:/software/lz4/1.9.4-GCCcore-13.3.0/lib:/software/gettext/0.22.5-GCCcore-13.3.0/lib:/software/libiconv/1.17-GCCcore-13.3.0/lib:/software/genoa/r24.04/expat/2.6.2-GCCcore-13.3.0/lib:/software/genoa/r24.04/cURL/8.7.1-GCCcore-13.3.0/lib:/software/genoa/r24.04/Java/11.0.20/lib:/software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/lib:/software/genoa/r24.04/libffi/3.4.5-GCCcore-13.3.0/lib64:/software/genoa/r24.04/SQLite/3.45.3-GCCcore-13.3.0/lib:/software/genoa/r24.04/Tcl/8.6.14-GCCcore-13.3.0/lib:/software/genoa/r24.04/libreadline/8.2-GCCcore-13.3.0/lib:/software/genoa/r24.04/ncurses/6.5-GCCcore-13.3.0/lib:/software/genoa/r24.04/bzip2/1.0.8-GCCcore-13.3.0/lib:/software/genoa/r24.04/ScaLAPACK/2.2.0-gompi-2024a-fb/lib:/software/genoa/r24.04/FFTW.MPI/3.3.10-gompi-2024a/lib:/software/genoa/r24.04/FFTW/3.3.10-GCC-13.3.0/lib:/software/genoa/r24.04/FlexiBLAS/3.4.4-GCC-13.3.0/lib:/software/genoa/r24.04/OpenBLAS/0.3.27-GCC-13.3.0/lib:/software/genoa/r24.04/OpenMPI/5.0.3-GCC-13.3.0/lib:/software/genoa/r24.04/UCC/1.3.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/PRRTE/3.0.5-GCCcore-13.3.0/lib:/software/genoa/r24.04/PMIx/5.0.2-GCCcore-13.3.0/lib:/software/genoa/r24.04/libfabric/1.21.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/UCX/1.16.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/libevent/2.1.12-GCCcore-13.3.0/lib:/software/genoa/r24.04/OpenSSL/3/lib:/software/genoa/r24.04/hwloc/2.10.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/libpciaccess/0.18.1-GCCcore-13.3.0/lib:/software/genoa/r24.04/libxml2/2.12.7-GCCcore-13.3.0/lib:/software/genoa/r24.04/XZ/5.4.5-GCCcore-13.3.0/lib:/software/genoa/r24.04/numactl/2.0.18-GCCcore-13.3.0/lib:/software/binutils/2.42-GCCcore-13.3.0/lib:/software/zlib/1.3.1-GCCcore-13.3.0/lib:/software/genoa/r24.04/GCCcore/13.3.0/lib64 (previously undefined)
== 2025-06-25 17:00:38,657 filetools.py:403 INFO Path /tmp/easybuild-tmp/eb-gehzfysz/tmp7vzo8jyc successfully removed.
== 2025-06-25 17:00:38,657 environment.py:93 INFO Environment variable _ModuleTable002_ set to IjAyMDI0MDcyMi4qemZpbmFsIiwKfSwKQmF6ZWwgPSB7CmZuID0gIi9kYXRhL2hvcnNlL3dzL3MzMjQ4OTczLUVhc3lCdWlsZC9lYXN5YnVpbGQtZ2Vub2EvbW9kdWxlcy9hbGwvQ29tcGlsZXIvR0NDY29yZS8xMy4zLjAvQmF6ZWwvNi41LjAtSmF2YS0xMS5sdWEiLApmdWxsTmFtZSA9ICJCYXplbC82LjUuMC1KYXZhLTExIiwKbG9hZE9yZGVyID0gMjcsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAiQmF6ZWwvNi41LjAtSmF2YS0xMSIsCndWID0gIjAwMDAwMDAwNi4wMDAwMDAwMDUuKmphdmEuKnpmaW5hbC0uMDAwMDAwMDExLip6ZmluYWwiLAp9LApCb29zdCA9IHsKZm4gPSAiL2RhdGEvaG9yc2Uvd3MvczMyNDg5 (previous value: 'LjAwMDAwMDAwNS4qamF2YS4qemZpbmFsLS4wMDAwMDAwMTEuKnpmaW5hbCIsCn0sCkJvb3N0ID0gewpmbiA9ICIvZGF0YS9ob3JzZS93cy9zMzI0ODk3My1FYXN5QnVpbGQvZWFzeWJ1aWxkLWdlbm9hL21vZHVsZXMvYWxsL0NvbXBpbGVyL0dDQy8xMy4zLjAvQm9vc3QvMS44NS4wLmx1YSIsCmZ1bGxOYW1lID0gIkJvb3N0LzEuODUuMCIsCmxvYWRPcmRlciA9IDQ2LApwcm9wVCA9IHt9LApyZWZfY291bnQgPSAxLApzdGFja0RlcHRoID0gMSwKc3RhdHVzID0gImFjdGl2ZSIsCnVzZXJOYW1lID0gIkJvb3N0LzEuODUuMCIsCndWID0gIjAwMDAwMDAwMS4wMDAwMDAwODUuKnpmaW5hbCIsCn0sCkN5dGhvbiA9IHsKZm4gPSAiL2RhdGEvaG9yc2Uvd3MvczMyNDg5NzMtRWFzeUJ1')
== 2025-06-25 17:00:38,657 environment.py:93 INFO Environment variable __LMOD_REF_COUNT_MODULEPATH set to /modules/all:1;/modules/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:1;/software/modules/genoa/r24.04/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:1;/modules/all/Compiler/GCC/13.3.0:1;/software/modules/genoa/r24.04/all/Compiler/GCC/13.3.0:1;/modules/all/Compiler/GCCcore/13.3.0:1;/software/modules/genoa/r24.04/all/Compiler/GCCcore/13.3.0:1;/modules/all/Core:1;/home/s3248973/.local/EasyBuildDev/modules:1;/software/modules/genoa/r24.04/all:1;/software/modules/genoa/r24.04/all/Core:1;/software/modules/releases/genoa:1 (previous value: '/modules/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:2;/software/modules/genoa/r24.04/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:2;/modules/all/Compiler/GCC/13.3.0:2;/software/modules/genoa/r24.04/all/Compiler/GCC/13.3.0:2;/modules/all/Compiler/GCCcore/13.3.0:2;/software/modules/genoa/r24.04/all/Compiler/GCCcore/13.3.0:2;/modules/all/Core:1;/tmp/easybuild-tmp/eb-gehzfysz/tmp7vzo8jyc/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:1;/modules/all:1;/home/s3248973/.local/EasyBuildDev/modules:1;/software/modules/genoa/r24.04/all:1;/software/modules/genoa/r24.04/all/Core:1;/software/modules/releases/genoa:1')
<etc>
...

Do we really need the output of the module commands or can we silence them completely?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions