Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 19 additions & 42 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,7 @@ The only strict requirements are:
* a **GNU/Linux** distribution as operating system
* some common shell tools are expected to be available, see [Required shell tools][required_shell_tools]
* [Python](https://python.org):
* Python 2.7, or Python 3.x (>= 3.5);
* since [Python 2 is end-of-life](https://www.python.org/doc/sunset-python-2/) we strongly recommend
using Python 3 if it is available;
* only EasyBuild v4.0 (or newer) is compatible with Python 3, earlier EasyBuild releases require Python 2;
* EasyBuild 5.0 will require Python >= 3.6;
* Python >= 3.6 is required for EasyBuild 5.0;
* no third-party Python packages are strictly required (the Python standard library is sufficient);
* for some *specific* EasyBuild features additional Python packages are required however, see [Optional Python packages][optional_python_packages];
* a **modules tool**: Tcl(/C) environment modules or Lmod
Expand Down Expand Up @@ -58,23 +54,10 @@ Compare the version of `eb`, the main EasyBuild command, with the version of the
For example::

``` console
$ module load EasyBuild
$ module list

Currently Loaded Modules:
1) EasyBuild/4.8.0

$ eb --version
This is EasyBuild 4.8.0 (framework: 4.8.0, easyblocks: 4.8.0) on host example.local
This is EasyBuild 5.0.0 (framework: 5.0.0, easyblocks: 5.0.0) on host example.local
```

!!! tip
The Tcl/C Environment Modules (version <= 3.2.10) does its default sorting differently than
newer versions of Environment Modules and Lmod. The former will normally sort in the
lexicographic order, while the latters follow an approach that is closer to Python's construct
`LooseVersion` way of ordering. Such aspects may make a big difference, if you have installed
both versions 1.9.0 and 1.15.2, with respect to what is the version being loaded by default.

You can also run `eb --show-system-info` to see system information relevant to EasyBuild,
or run`eb --show-config` to see the default EasyBuild configuration (see also [Configuring EasyBuild][configuring_easybuild]).

Expand All @@ -90,12 +73,6 @@ To upgrade to a newer EasyBuild version than the one currently installed:

For the `pip` install, you may wish to slightly change this command depending on the context and your personal preferences:

* to install EasyBuild *system-wide*, you can use `sudo` (if you have admin privileges):

``` shell
sudo pip install easybuild
```

* To install EasyBuild *in your personal home directory*, you can use the `--user` option:

``` shell
Expand Down Expand Up @@ -131,7 +108,7 @@ above:
pip3 install easybuild
```

If you want to ensure that you are using the `pip` installation that corresponds to the Python 3 installation
If you want to ensure that you are using the `pip` installation that corresponds a specific Python installation
that you intend to use, you can use `python3 -m pip` rather than `pip3`.

``` shell
Expand Down Expand Up @@ -225,10 +202,10 @@ you can define the `$EB_VERBOSE` environment variable. For example:
``` console
$ EB_VERBOSE=1 eb --version
>> Considering 'python3.6'...
>> 'python3' version: 3.6.8, which matches Python 3 version requirement (>= 3.5)
>> 'python3' version: 3.6.8, which matches Python 3 version requirement (>= 3.6)
>> Selected Python command: python3 (/usr/bin/python3.6)
>> python3.6 -m easybuild.main --version
This is EasyBuild 4.8.0 (framework: 4.8.0, easyblocks: 4.8.0) on host example
This is EasyBuild 5.0.0 (framework: 5.0.0, easyblocks: 5.0.0) on host example
```


Expand Down Expand Up @@ -299,7 +276,14 @@ module use _PREFIX_/modules/all
Then, load the EasyBuild module to update your environment and make EasyBuild available for use:

``` shell
module load EasyBuild
$ module load EasyBuild
$ module list

Currently Loaded Modules:
1) EasyBuild/5.0.0

$ eb --version
This is EasyBuild 5.0.0 (framework: 5.0.0, easyblocks: 5.0.0) on host example.local
```

!!! note
Expand Down Expand Up @@ -340,16 +324,14 @@ Hence, a modules tool must be available to consume module files with.

Supported module tools:

* [Tcl/C environment-modules](https://modules.sourceforge.net/) (version >= 3.2.10)
* [Tcl-only variant of environment modules](https://sourceforge.net/projects/modules/files/Modules-Tcl)
* [Lmod](https://lmod.sourceforge.net) (version >= 6.5.1), *highly recommended*
* [Tcl/C environment-modules](https://modules.sourceforge.net/) (version >= 4.3.0)
* [Lmod](https://lmod.sourceforge.net) (version >= 8.0.0)

!!! note
The path to the actual modules tool binary/script used *must* be included in `$PATH`,
to make it readily available to EasyBuild.

* for Tcl/C environment modules: `modulecmd`
* for Tcl-only environment modules: `modulecmd.tcl`
* for Lmod: `lmod`

The path where the modules tool binary/script is located can be determined via the definition of
Expand All @@ -359,9 +341,6 @@ Supported module tools:
For Lmod, EasyBuild will try to fall back to finding the `lmod` binary via the `$LMOD_CMD`
environment variable, in case `lmod` is not available in `$PATH`.

In EasyBuild versions *prior* to 2.1.1, the path specified by `$LMOD_CMD` was (erroneously) preferred over the
(first) `lmod` binary available via `$PATH`.

For modern Tcl-only environment modules (version >= 4.0.0), EasyBuild will try to fall back to finding the
`modulecmd.tcl` binary via the `$MODULES_CMD` environment variable, in case `modulecmd.tcl` is not available
in `$PATH`.
Expand All @@ -382,7 +361,7 @@ Additional notes:

### Required Python packages {: #required_python_packages }

Since EasyBuild v4.0, *no* Python packages outside of the Python standard library are required.
EasyBuild requires *no* Python packages outside of the Python standard library.

## Optional dependencies

Expand All @@ -391,8 +370,6 @@ Some dependencies are optional and are only required to support certain features

### Optional Python packages {: #optional_python_packages }

* [GC3Pie](https://pypi.org/project/gc3pie), only needed when using `GC3Pie` as a backend for `--job`,
see also [Submitting jobs using --job][submitting_jobs];
* [GitPython](https://github.com/gitpython-developers/GitPython), only needed if
EasyBuild is hosted in a git repository or if you’re using a git
repository for easyconfig files (.eb);
Expand All @@ -412,13 +389,13 @@ Some dependencies are optional and are only required to support certain features
EasyBuild is split up into three different packages, which are available
from the Python Package Index (PyPi):

* [easybuild-framework](http://pypi.python.org/pypi/easybuild-framework) - the EasyBuild framework, which includes the
* [easybuild-framework](https://pypi.python.org/pypi/easybuild-framework) - the EasyBuild framework, which includes the
easybuild.framework and easybuild.tools Python packages that provide
general support for building and installing software
* [easybuild-easyblocks](http://pypi.python.org/pypi/easybuild-easyblocks) - a collection of easyblocks that implement
* [easybuild-easyblocks](https://pypi.python.org/pypi/easybuild-easyblocks) - a collection of easyblocks that implement
support for building and installing (collections of) software
packages
* [easybuild-easyconfigs](http://pypi.python.org/pypi/easybuild-easyconfigs) - a collection of example easyconfig files
* [easybuild-easyconfigs](https://pypi.python.org/pypi/easybuild-easyconfigs) - a collection of example easyconfig files
that specify which software to build, and using which build options;
these easyconfigs will be well tested with the latest compatible
versions of the easybuild-framework and easybuild-easyblocks packages
Expand Down
146 changes: 13 additions & 133 deletions docs/python-2-3-compatibility.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
# Compatibility with Python 2 and Python 3 {: #py2_py3_compatibility }
# Compatibility with Python 3 {: #py2_py3_compatibility }

Starting with EasyBuild v4.0, the EasyBuild framework and easyblocks are
compatible with both Python versions 2 and 3. More specifically, the
following Python versions are currently supported:
Starting with EasyBuild v5.0, the EasyBuild framework and easyblocks require Python >= 3.6.

More specifically, the following Python versions are currently supported:

- Python 2.6.x (support was removed in EasyBuild v4.4.0)
- Python 2.7.x ([support will be removed in EasyBuild v5.0.0](easybuild-v5/index.md))
- Python 3.5.x ([support will be removed in EasyBuild v5.0.0](easybuild-v5/index.md))
- Python 3.6.x
- Python 3.7.x
- Python 3.8.x (requires EasyBuild v4.1.0)
- Python 3.9.x (requires EasyBuild v4.3.1)
- Python 3.10.x (requires EasyBuild v4.5.2)
- Python 3.11.x (requires EasyBuild v4.5.2)
- Python 3.12.x (requires EasyBuild v5.0.0)
- Python 3.13.x (requires EasyBuild v5.0.0)

## Determining which Python version EasyBuild is using via `$EB_VERBOSE` {: #py2_py3_compatibility_EB_VERBOSE }

Expand Down Expand Up @@ -41,7 +40,6 @@ The commands considered are (in order):

- `python`
- `python3`
- `python2`

Before considering these commands, `eb` will first consider the command
name specified via the `$EB_PYTHON` environment variable (if defined),
Expand All @@ -62,130 +60,12 @@ This is EasyBuild 4.0.0 (framework: 4.0.0, easyblocks: 4.0.0) on host example.

## The `easybuild.tools.py2vs3` package

To facilitate this, the `easybuild.tools.py2vs3` package was introduced
in EasyBuild v4.0. When importing a function from this package, you will
automatically get a version of the function that is compatible with the
Python version being used to run EasyBuild.

Through this approach we can hide subtle differences between Python 2
and 3, while avoiding code duplication and Python version checks
throughout the codebase, as well as avoid requiring packages like `six`
or `future` (which facilitate maintaining compatibility with Python 2
and 3, but are also a bit of a burden).

The `easybuild.tools.py2vs3` package provides two major classes of items
(listed below in alphabetical order):

- functions from the Python standard library which should be imported
from different locations in Python 2 and 3
- wrappers for functionality in the Python standard library which
behaves differently in Python 2 and 3

## `ascii_letters` {: #py2vs3_ascii_letters }

- Python 2: corresponds to `string.letters`
- Python 3: corresponds to `string.ascii_letters`

## `ascii_lowercase` {: #py2vs3_ascii_lowercase }

- Python 2: corresponds to `string.lowercase`
- Python 3: corresponds to `string.ascii_lowercase`

## `build_opener` {: #py2vs3_build_opener }

- Python 2: corresponds to `urllib2.build_opener` function
- Python 3: corresponds to `urllib.request.build_opener` function

## `configparser` {: #py2vs3_configparser }

- Python 2: corresponds to `ConfigParser.configparser` module
- Python 3: corresponds to `configparser` module

## `create_base_metaclass` {: #py2vs3_create_base_metaclass }

Function to create a metaclass that can be used as a base class,
implemented in a way that is compatible with both Python 2 and 3.

## `extract_method_name` {: #py2vs3_extract_method_name }

Function to method name from lambda function, implemented in a way that
is compatible with both Python 2 and 3.

## `HTTPError` {: #py2vs3_HTTPError }

- Python 2: corresponds to `urllib2.HTTPError`
- Python 3: corresponds to `urllib.request.HTTPError`

## `HTTPSHandler` {: #py2vs3_HTTPSHandler }

- Python 2: corresponds to `urllib2.HTTPSHandler`
- Python 3: corresponds to `urllib.request.HTTPSHandler`

## `json_loads` {: #py2vs3_json_loads }

- Python 2: wraps `json.loads` function

- Python 3: wraps `json.loads` function, taking into account that for Python versions older than 3.6
- a value of type `string` (rather than `bytes`) is required as argument

## `mk_wrapper_baseclass` {: #py2vs3_mk_wrapper_baseclass }

Function to create a wrapper base class using the specified metaclass,
implemented in a way that is compatible with both Python 2 and 3.

## `OrderedDict` {: #py2vs3_OrderedDict }

- Python 2.6: corresponds to `easybuild.tools.ordereddict.OrderedDict`
- Python 2.7: corresponds to `collections.OrderedDict`
- Python 3: corresponds to `collections.OrderedDict`

## `reload` {: #py2vs3_reload }

- Python 2: corresponds to `reload` built-in function
- Python 3: corresponds to `importlib.reload` function

## `raise_with_traceback` {: #py2vs3_raise_with_traceback }

Function to raise an error with specified message and traceback,
implemented in a way that is compatible with both Python 2 and 3.

## `Request` {: #py2vs3_Request }

- Python 2: corresponds to `urllib2.Request`
- Python 3: corresponds to `urllib.request.Request`

## `subprocess_popen_text` {: #py2vs3_subprocess_popen_text }

- Python 2: wrapper for `subprocess.Popen`
- Python 3: wrapper for `subprocess.Popen` while forcing text mode
(using `universal_newlines=True`)

## `std_urllib` {: #py2vs3_std_urllib }

- Python 2: corresponds to `urllib` package
- Python 3: corresponds to `urllib.request` package

## `string_type` {: #py2vs3_string_type }

- Python 2: corresponds to `basestring` built-in string type
- Python 3: corresponds to `str` built-in string type

## `StringIO` {: #py2vs3_StringIO }

- Python 2: corresponds to `StringIO.StringIO` class
- Python 3: corresponds to `io.StringIO` class

## `urlencode` {: #py2vs3_urlencode }

- Python 2: corresponds to `urllib.urlencode` function
- Python 3: corresponds to `urllib.parse.urlencode` function

## `URLError` {: #py2vs3_URLError }

- Python 2: corresponds to `urllib2.URLError`
- Python 3: corresponds to `urllib.request.URLError`
Support for Python 2.7, and Python 3.5, was removed in EasyBuild v5.0.

## `urlopen` {: #py2vs3_urlopen }
The `easybuild.tools.py2vs3` package was introduced in EasyBuild v4.0 so
that when importing a function from this package, you would automatically
get a version of the function that is compatible with the Python version
being used to run EasyBuild.

- Python 2: corresponds to `urllib2.urlopen`
- Python 3: corresponds to `urllib.request.urlopen`
Use of `easybuild.tools.py2vs3` is deprecated in EasyBuild v5 and will be
removed in the future.
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ nav:
- Locks: locks.md
- Manipulating dependencies: manipulating-dependencies.md
- Partial installations: partial-installations.md
- Compatibility with Python 2 + 3: python-2-3-compatibility.md
- Compatibility with Python 3: python-2-3-compatibility.md
- Progress bars: progress-bars.md
- Search index for easyconfigs: easyconfigs-search-index.md
- System toolchain: system-toolchain.md
Expand Down