Installation

Downloading

esm_tools is hosted on https://github.com/esm-tools. To get access to the software you need to be able to log into GitHub.

Then you can start by cloning the repository esm_tools.git:

$ git clone https://github.com/esm-tools/esm_tools.git

This gives you a collection of yaml configuration files containing all the information on models, coupled setups, machines, etc. in the subfolder config, default namelists in the folder namelists, example runscripts for a large number of models on different HPC systems in subfolder runscripts, and this documention in docs. Also you will find the installer install.sh used to install the python packages.

Installing in an encapuslated environment

Based on an alternative installation procedure, that provides an esm-tools installation employing direnv (https://direnv.net/), you can now install various encapsulated versions of esm-tools alongside each other. These different installations do not impact each others’ configuration. Consequently, they can coexist in peaceful harmony. In the suggested alternative installation method all configurations will reside within the base folder of a specific esm-tools version that you install. There is no dependency on configurations outside the installation directory of a specific esm-tools version, mitigating the potential for side effects if another version of esm tools is installed in parallel. To install esm-tools as suggested here, just follow the procedure outlined below. The steps to create the installation involve preparation of direnv, including setting up an environment that encapsulates potentially version-specific settings, creating a dedicated directory to which a specific version of esm-tools will be installed, and installing the the esm-tools via pip. The commands to be executed are (note comments for further explanation):

$ curl -sfL https://direnv.net/install.sh | bash # install direnv if not yet done - this enables encapsulation and parallel use of different esm-tools versions
$ mkdir esm_tools_v6.1.10 #adjust version number as appropriate
$ cd esm_tools_v6.1.10/
  #create .envrc (e.g. via direnv edit .) and add information matching the result of the cat command below
$ cat .envrc
  module load python3
  layout python
  module load gcc
$ pip install -U pip wheel
$ pip install esm-tools

Accessing components in DKRZ server

Some of the esm_tools components are hosted in the gitlab.dkrz.de servers. To be able to reach these components you will need:

  1. A DKRZ account (https://www.dkrz.de/up/my-dkrz/getting-started/account/DKRZ-user-account).

  2. Become a member of the group esm_tools. Either look for the group and request membership, or directly contact dirk.barbi@awi.de.

  3. Request access from the corresponding author of the component. Feel free to contact us if you don’t know who the model developers are or check the Supported Models section.

ESM Tools

https://readthedocs.org/projects/esm-tools/badge/?version=latest

For our complete documentation, please check https://esm-tools.readthedocs.io/en/latest/index.html.

Before you continue

You will need python 3 (possibly version 3.6 or newer), a version of git that is not ancient (everything newer than 2.10 should be good), and up-to-date pip (pip install -U pip) to install the esm_tools. That means that on the supported machines, you could for example use the following settings:

ollie.awi.de:

$ module load git
$ module load python3

levante.dkrz.de:

$ module load git
$ module load python3

glogin.hlrn.de / blogin.hlrn.de:

$ module load git
$ module load anaconda3

juwels.fz-juelich.de:

$ module load git
$ module load Python-3.6.8

aleph:

$ module load git
$ module load python

Note that some machines might raise an error conflict netcdf_c when loading anaconda3. In that case you will need to swap netcdf_c with anaconda3:

$ module unload netcdf_c
$ module load anaconda3

Installing

  1. First, make sure you add the following lines to one of your login or profile files, i.e. ~/.bash_profile, ~/.bashrc, ~/.profile, etc.:

    $ export PATH=$PATH:~/.local/bin
    $ export LC_ALL=en_US.UTF-8
    $ export LANG=en_US.UTF-8
    
  2. Inside the same login or profile file, add also the module commands necessary for the HPC system you are using (find the lines in the section above).

  3. You can choose to source now your login or profile file, so that the module and export commands are run (e.g. $ source ~/.bash_profile).

  4. To use the new version of the ESM-Tools, now rewritten in Python, clone this repository:

    $ git clone https://github.com/esm-tools/esm_tools.git
    
  5. Then, run the install.sh:

    $ ./install.sh
    

You should now have the command line tools esm_master and esm_runscripts, which replace the old version.

Upgrade ESM-Tools

To upgrade all the ESM-Tools packages you can run:

$ esm_versions upgrade

This will only upgrade the packages that are not installed in editable mode. Those, installed in editable mode will need to be upgraded using git.

You can also choose to upgrade specific packages by adding the package name to the previous command, i.e. to upgrade esm_master:

$ esm_versions upgrade esm_parser

Note

If there are version conflicts reported back at this point with some of the Python modules (i.e. pkg_resources.ContextualVersionConflict: (<package name>)), try reinstalling that package: pip install <package> --upgrade --ignore-installed.

Uninstall ESM-Tools

We are sorry to see you go! To uninstall your current installation make sure you have the most recent version of pip available for your system:

$ python3 -m pip install -U pip

Then, you can use use the following command to uninstall all ESM-Tools packages:

$ pip freeze | grep esm | xargs pip uninstall -y

You can also choose to manually uninstall. In order to do that, remove the installed Python packages and delete the esm_* executables. The following commands will do the trick if you installed with the install.sh script or installed using pip with user mode

$ rm -ri ~/.local/bin/esm*
$ rm -ri ~/.local/lib/python3.<version>/site-packages/esm*

Note that you may have a different Python version, so the second command might need to be adapted. You may also use pip to uninstall any of the packages:

$ pip uninstall [--user] esm-tools

The --user flag may be required when using pip if you are not uninstalling in either a virtual environment or a global install (you would need to be root in that case).