Installation¶
Downloading¶
esm_tools
is hosted on https://github.com/esm-tools. To get access to the software you need to be able to log into GitHub.
Then you can start by cloning the repository esm_tools.git
:
$ git clone https://github.com/esm-tools/esm_tools.git
This gives you a collection of yaml configuration files containing all the information on models, coupled setups, machines, etc. in the subfolder config
, default namelists in the folder namelists
, example runscripts for a large number of models on different HPC systems in subfolder runscripts
, and this documention in docs
. Also you will find the installer install.sh
used to install the python packages.
Installing in an encapuslated environment¶
Based on an alternative installation procedure, that provides an esm-tools installation employing direnv (https://direnv.net/), you can now install various encapsulated versions of esm-tools alongside each other. These different installations do not impact each others’ configuration. Consequently, they can coexist in peaceful harmony. In the suggested alternative installation method all configurations will reside within the base folder of a specific esm-tools version that you install. There is no dependency on configurations outside the installation directory of a specific esm-tools version, mitigating the potential for side effects if another version of esm tools is installed in parallel. To install esm-tools as suggested here, just follow the procedure outlined below. The steps to create the installation involve preparation of direnv, including setting up an environment that encapsulates potentially version-specific settings, creating a dedicated directory to which a specific version of esm-tools will be installed, and installing the the esm-tools via pip. The commands to be executed are (note comments for further explanation):
$ curl -sfL https://direnv.net/install.sh | bash # install direnv if not yet done - this enables encapsulation and parallel use of different esm-tools versions
$ mkdir esm_tools_v6.1.10 #adjust version number as appropriate
$ cd esm_tools_v6.1.10/
#create .envrc (e.g. via direnv edit .) and add information matching the result of the cat command below
$ cat .envrc
module load python3
layout python
module load gcc
$ pip install -U pip wheel
$ pip install esm-tools
Accessing components in DKRZ server¶
Some of the esm_tools
components are hosted in the gitlab.dkrz.de servers. To be able to reach these components you will need:
A DKRZ account (https://www.dkrz.de/up/my-dkrz/getting-started/account/DKRZ-user-account).
Become a member of the group
esm_tools
. Either look for the group and request membership, or directly contact dirk.barbi@awi.de.Request access from the corresponding author of the component. Feel free to contact us if you don’t know who the model developers are or check the Supported Models section.
ESM Tools¶
For our complete documentation, please check https://esm-tools.readthedocs.io/en/latest/index.html.
Before you continue¶
You will need python 3 (possibly version 3.6 or newer), a version of git that is not ancient (everything newer than 2.10 should be good), and up-to-date pip (pip install -U pip
) to install the esm_tools. That means that on the supported machines, you could for example use the following settings:
ollie.awi.de:
$ module load git
$ module load python3
levante.dkrz.de:
$ module load git
$ module load python3
glogin.hlrn.de / blogin.hlrn.de:
$ module load git
$ module load anaconda3
juwels.fz-juelich.de:
$ module load git
$ module load Python-3.6.8
aleph:
$ module load git
$ module load python
Note that some machines might raise an error conflict netcdf_c
when loading anaconda3
. In that case you will need to swap netcdf_c
with anaconda3
:
$ module unload netcdf_c
$ module load anaconda3
Installing¶
First, make sure you add the following lines to one of your login or profile files, i.e.
~/.bash_profile
,~/.bashrc
,~/.profile
, etc.:$ export PATH=$PATH:~/.local/bin $ export LC_ALL=en_US.UTF-8 $ export LANG=en_US.UTF-8
Inside the same login or profile file, add also the
module
commands necessary for the HPC system you are using (find the lines in the section above).You can choose to source now your login or profile file, so that the
module
andexport
commands are run (e.g.$ source ~/.bash_profile
).To use the new version of the ESM-Tools, now rewritten in Python, clone this repository:
$ git clone https://github.com/esm-tools/esm_tools.git
Then, run the
install.sh
:$ ./install.sh
You should now have the command line tools esm_master
and esm_runscripts
, which replace the old version.
Upgrade ESM-Tools¶
To upgrade all the ESM-Tools packages you can run:
$ esm_versions upgrade
This will only upgrade the packages that are not installed in editable mode. Those,
installed in editable mode will need to be upgraded using git
.
You can also choose to upgrade specific packages by adding the package name to the
previous command, i.e. to upgrade esm_master
:
$ esm_versions upgrade esm_parser
Note
If there are version conflicts reported back at this point with some of the
Python modules (i.e. pkg_resources.ContextualVersionConflict: (<package name>)
),
try reinstalling that package:
pip install <package> --upgrade --ignore-installed
.
Uninstall ESM-Tools¶
To uninstall your current installation make sure you have the most recent version of
pip
available for your system:
$ python3 -m pip install -U pip
Then, you can use use the following command to uninstall all ESM-Tools packages:
$ esm_versions clean
You can also choose to manually uninstall. In order to do that, remove the installed Python packages and delete the esm_*
executables. The following commands will do the trick if you installed with the install.sh
script or installed using pip
with user
mode
$ rm -ri ~/.local/bin/esm*
$ rm -ri ~/.local/lib/python3.<version>/site-packages/esm*
Note that you may have a different Python version, so the second command might need to be adapted. You may also use pip
to uninstall any of the packages:
$ pip uninstall [--user] esm-tools
The --user
flag may be required when using pip
.