change ModulesTool.load to not prepend path to $MODULEPATH if that path is already listed in $MODULEPATH
#4987
Conversation
Update `load` method of ModuleTool class to avoid prepending a modulepath passed as argument if it is already enabled. This change avoid setting a modulepath at top priority that could lead to wrong module dependency resolution. Fixes easybuilders#4986 Fixes easybuilders/easybuild-easyconfigs#23675 Fixes easybuilders/easybuild-easyconfigs#17407 Fixes easybuilders/easybuild-easyconfigs#17368 Fixes easybuilders/easybuild-easyconfigs#23579 Signed-off-by: Xavier Delaruelle <xavier.delaruelle@cea.fr>
xdelaruelle
force-pushed
the
envmodules_hmns_build
branch
from
5370ea3 to
ecb6e2a
Compare
Create a test that demonstrates built module test issue with HMNS naming scheme and Environment Modules. Such issue occurs when built easyconfig has dependencies available in the toolchain-specific modulepath but also in the Core modulepath but with conflicting requirements (here ncurses 6.4 is required by toolchain-specific gettext and util-linux modules whereas ncurses 6.3 is required by Core gettext module). Issue comes from the Core modulepath being put at the top priority in the MODULEPATH environment variable. Thus wrong gettext module is loaded (from Core modulepath). Signed-off-by: Xavier Delaruelle <xavier.delaruelle@cea.fr>
xdelaruelle
force-pushed
the
envmodules_hmns_build
branch
from
ecb6e2a to
58eceea
Compare
boegel
changed the title
ModulesTool.load to not prepend path to $MODULEPATH if that path is already listed in $MODULEPATH
| for mod_path in mod_paths: | ||
| full_mod_path = os.path.join(install_path('mod'), build_option('suffix_modules_path'), mod_path) | ||
| if os.path.exists(full_mod_path): | ||
| if os.path.exists(full_mod_path) and full_mod_path not in curr_mod_paths: |
There was a problem hiding this comment.
This is potentially problematic I think...
Imagine that EasyBuild is configured to install modules into /apps/modules/all, and that eb is launched in an environment in which $MODULEPATH is set to /apps/manual-modules:/apps/modules/all.
In that case, we would expect that the modules that EasyBuild loads as dependencies for an installation come from /apps/modules/all, and would only be picked up from /apps/manual-modules if the required module is not found in /apps/modules/all.
Or at least, that's the current behavior: giving precedence to modules that were (assumed to be) installed with EasyBuild.
This changes that, since the order in $MODULEPATH as it was before launching EasyBuild would be retained, and that may surprise people depending on their setup, since it's a change in behavior...
There was a problem hiding this comment.
In the situation you describe, eb is changing MODULEPATH to make its modulepaths at the top. eb does not go through ModuleTool.load() to do that.
I have made the following test:
export MODULEPATH=/usr/share/Modules/modulefiles:/home/user/.local/easybuild/modules/all/Core:/home/user/.local/easybuild/modules/all
rm -rf ~/.local/easybuild
eb --module-naming-scheme=HierarchicalMNS --skip-sanity-check --robot --rebuild --module-only GLib-2.77.1-GCCcore-12.3.0.eb
Then looking at the produced log file for instrumented ModuleTool.set_mod_paths():
$ grep MODULEPATH /home/user/.local/easybuild/software/GLib/2.77.1-GCCcore-12.3.0/easybuild/easybuild-GLib-2.77.1-20250827.211211.log
os.environ['MODULEPATH'] = '/home/user/.local/easybuild/modules/all:/usr/share/Modules/modulefiles'
os.environ['MODULEPATH'] = '/home/user/.local/easybuild/modules/all/Core:/home/user/.local/easybuild/modules/all:/usr/share/Modules/modulefiles'
== 2025-08-27 21:12:09,913 modules.py:794 INFO $MODULEPATH after set_mod_paths: /home/user/.local/easybuild/modules/all/Core:/home/user/.local/easybuild/modules/all:/usr/share/Modules/modulefilesSo my external modulepath is moved at the back by eb.
Update
loadmethod of ModuleTool class to avoid prepending a modulepath passed as argument if it is already enabled.This change avoid setting a modulepath at top priority that could lead to wrong module dependency resolution.
Fixes #4986
Fixes easybuilders/easybuild-easyconfigs#23675
Fixes easybuilders/easybuild-easyconfigs#17407
Fixes easybuilders/easybuild-easyconfigs#17368
Fixes easybuilders/easybuild-easyconfigs#23579