===============
ESM Environment
===============

The package ``esm_environment`` takes care of generating the environments for the
different HPCs supported by `ESM-Tools`. This is done through the use of the
``EnvironmentInfos`` class inside the different `ESM-Tools` packages.

For the correct definition of an environment for an HPC a `yaml` file for that
system needs to be included inside the ``esm_tools`` package inside the
``configs/machines/`` folder (e.g. ``ollie.yaml``). This file should contain all
the required preset variables for that system and the environment variables
``module_actions`` and ``export_vars``.

Environment variables
=====================

module_actions (list)
    A list of ``module`` actions to be included in the compilation and run scripts
    generated by ``esm_master`` and ``esm_runscripts`` respectively, such as
    ``module load netcdf``, ``module unload netcdf``, ``module purge``, etc. The
    syntax of this list is such as that of the command that would be normally used
    in shell, but omitting the ``module`` word, for example:

    .. code-block:: yaml

       module_actions:
           - "purge"
           - "load netcdf"

    This variable also allows for sourcing files by adding a member to the list
    such as ``source <file_to_be_sourced>``.

export_vars (dict)
    A dictionary containing all the variables (and their values) to be exported.
    The syntax is as follows:

    .. code-block:: yaml

       export_vars:
           A_VAR_TO_BE_EXPORTED: the_value

    The previous example will result in the following export in the script produced
    by ``esm_master`` or ``esm_runscripts``::

        export A_VAR_TO_BE_EXPORTED=the_value

    As a dictionary, ``export_vars`` is not allowed to have repeated keys. This
    could be a problem when environments are required to redefine a variable at
    different points of the script. To overcome this limitation, repetitions of the
    same variable are allowed if the key is followed by an integer contained inside
    ``[(int)]``:

    .. code-block:: yaml

       export_vars:
           A_VAR_TO_BE_EXPORTED: the_value
           A_VAR_TO_BE_EXPORTED[(1)]: $A_VAR_TO_BE_EXPORTED:another_value

    The resulting script will contain the following exports::

        export A_VAR_TO_BE_EXPORTED=the_value
        export A_VAR_TO_BE_EXPORTED=$A_VAR_TO_BE_EXPORTED:another_value

    Note that the index is removed once the exports are transferred into the script.

Modification of the environment through the model/setup files
=============================================================

As previously mentioned, the default environment for a HPC system is defined inside
its machine file (in ``esm_tools/machines/<machine_name>.yaml``). However, it is
possible to modify this environment through the model and/or coupled setup files
(or even inside the runscript) to adjust to the model/setup requirements. For this
purpose, the variables ``environment_changes``, ``compiletime_environment_changes``
and ``runtime_environment_changes`` can be used.

environment_changes (dict)
    Allows for modifications of the machine ``module_actions`` and ``export_vars``,
    both during compilation and runtime.
compiletime_environment_changes (dict)
    Allows for modifications of the machine ``module_actions`` and ``export_vars``,
    only applied during compilation time.
compiletime_environment_changes (dict)
    Allows for modifications of the machine ``module_actions`` and ``export_vars``,
    only applied during run time.

The syntax for this dictionary is the same as that defined in
:ref:`Environment variables`, but using ``add_`` in front of the environment
variables (``add_module_actions`` and ``add_export_vars``). Furthermore, the
environment variables can be nested inside ``choose_`` blocks:

.. code-block:: yaml

   environment_changes:
           choose_computer:
                   ollie:
                           add_export_vars:
                                   COMPUTER_VAR: 'ollie'
                   juwels:
                           add_export_vars:
                                   COMPUTER_VAR: 'mistral'

.. Note::
   This changes are model-specific for compilation, meaning that **the changes will
   only occur for the compilation script of the model containing those changes**.
   For runtime, all the environments of the components will be added together into
   the same ``.run`` script. Please, refer to
   `Coupled setup environment control` for an explanation on how to control
   environments for a whole setup.

Coupled setup environment control
=================================

There are two ways in which the environments for the coupled setups can be
modified: defining ``environment_changes`` for each component or defining a general
``environment_changes`` for the whole setup:

Component-by-component
~~~~~~~~~~~~~~~~~~~~~~

The ``environment_changes`` are taken **from the standalone component files**. It
is possible to modify these ``environment_changes`` through the setup file by
including ``environment_changes`` **inside the chapter of that component**.

.. Warning::
   Handling ``environment_changes`` in this fashion implies that compilation
   scripts can potentially end up containing different environments.

General environment for setups
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To define a general ``environment_changes`` **for all the components of a setup**,
include the ``environment_changes`` inside the ``general`` section of the setup
file. **This will ignore all the** ``environment_changes`` **defined by the standalone
files**. It is still possible to add component-specific ``environment_changes`` **from
the component chapter inside the setup file**.