How to get FALL3D
FALL3D is maintained in a public git repository, which contains the stable releases and the current working code. Stable version releases of the FALL3D source code are available as a tarball or zip file and can be downloaded from the Releases section.
However, we strongly recommend to clone the git repository especially if you want to update the source code or select different versions in case of problems. In order to obtain the software from the gitlab repository, you must first download and install the git software.
Then clone the repository using the following command line:
> git clone git@gitlab.com:fall3d-suite/fall3d.git
After cloning, a directory fall3d
should have been created.
Code changes are made fairly frequently in the gitlab repository. It is recommended to update the code periodically so that you have the latest version of FALL3D available on your computer. In order to update your local version of the source code, enter the FALL3D directory and pull all changes from a remote repository:
> cd fall3d
> git pull
Notes:
- Every time the source code is updated with
git pull
, the compilation process detailed below must be repeated.
Installation
FALL3D has been successfully built and tested on various clusters and distinct desktop computers using linux based operating systems using both CMake or Meson build systems. These build systems streamline the process, generating appropriate build files for diverse platforms and compilers. The following table details the EuroHPC systems and environments where FALL3D has been tested, including NVIDIA/AMD GPU acceleration support. It indicates the compilers used and compatibility with CMake and Meson build tools.
Leonardo@CIN | MN5-ACC@BSC | LUMI@CSC | Linux | MacOS | |
---|---|---|---|---|---|
Cmake | ✅ GNU & NVHPC | ✅ GNU & NVHPC | ✅ CRAY | ✅ GNU & NVHPC | ✅ GNU |
Meson | ✅ GNU & NVHPC | ✅ GNU & NVHPC | ❌ Cray compiler not supported by Meson | ✅ GNU & NVHPC | ✅ GNU |
MPI | ✅ NVHPC | ✅ NVHPC | ✅ CRAY | ✅ GNU & NVHPC | ✅ GNU |
MPI + OpenACC | 🚀 NVHPC | 🚀 NVHPC | 🚀 CRAY | 🚀 NVHPC | ❌ |
Notes:
It is essential that the required libraries or modules (compatible between them) are present during the configuration and compilation process. To install FALL3D in Leonardo, Lumi or Mare Nostrum 5 you can follow this FALL3D HPC guide
The use of autotools has been deprecated.
Compilation using CMake
Expand section for CMake installation instructions
CMake is a versatile build system that streamlines the compilation and installation of software projects across various platforms. It simplifies the process, making the projects more portable and easier to manage. This section covers the basic and advanced installation procedures for FALL3D using CMake. For users requiring a highly customized FALL3D installation, the Advanced installation process provides more flexibility and control.
CMake Prerequisites
Verify that installation program prerequisites are present on your system before proceeding. See section Installation requirements for further details.
- A Fortran compiler
- CMake > 3.1
- NetCDF Fortran
- Optional:
- MPI library
- Hardware specific drivers (GPU)
CMake Configure
Notes:
- Please ensure that the required libraries (NetCDF, MPI, etc.) are either installed on your system or loaded in your working environment.
For the basic installation (serial code without MPI or OpenACC), simply type the following commands from the FALL3D root folder:
Notes:
- The build directory name can be customized as long as it's consistently used in all commands.
> mkdir build
> cd build
> cmake ..
CMake Configure options
To customize the compilation of FALL3D, CMake allows you to use various configuration options. These options are passed to the cmake ..
command using the -D
prefix. This provides you with the flexibility to modify various aspects of the build process. A detailed table of these options and their effects is provided below.
CMake basic configuration
Option | Comment |
---|---|
-DWITH-R4 | Compiles using single precision |
-DWITH-MPI | Enables parallel using MPI (domain decomposition) |
-DWITH-ACC | Compiles the code using OpenACC (GPU and driver required) |
-DDETAIL_BIN | Adds to the binary name the compiler and the selected options |
-DCMAKE_INSTALL_PREFIX | Path to installation folder |
Expand the section to see some basic configuration examples
Basic configuration examples
For a multi-GPU configuration using MPI and OpenACC, execute the following
> cmake .. -DDETAIL_BIN=YES -DWITH-MPI=YES -DWITH-ACC=YES -DWITH-R4=NO
For a single GPU configuration, you can use
> cmake .. -DDETAIL_BIN=YES -DWITH-MPI=NO -DWITH-ACC=YES -DWITH-R4=NO
Notes
- The basic options set to
NO
(-DWITH-xxx=NO
) are shown here for clarity, it's not strictly necessary. If omitted, the default behavior of NO is applied ALL basic options. The options requiring some text defaults to empty if anything is defined by the user.
For a serial and single precission binary named Fall3d.x you can configure
> cmake .. -DWITH-R4=YES
Basic configuration summary
When executing the command, you'll see a lot of information displayed in the terminal. However, the summary section at the end will be most important.
-- Summary
-- Binary name: Fall3d.NVHPC.r8.mpi.acc.x
-- Build path: /fall3d/bbuild/bin
-- Binary name: Detailed
-- Precision: Double (R8)
-- MPI: Enabled
-- OpenACC: Enabled
-- Compiler flags: -DWITH_MPI;-O3;-acc;-DWITH_ACC
-- Linker flags: -acc
Once you've reviewed the summary and confirmed that there are no errors or warnings, you're ready to start the Build process. If you encounter any issues, please refer to the Troubleshooting section.
CMake advanced installation
For users looking for a fine grained control over the build process, the advanced installation of FALL3D provides the flexibility to customize default Fortran compiler and linker flags. In addition to the basic options, you can also use the following ones.
Advanced options
Option | Comment |
---|---|
-DCMAKE_BUILD_TYPE | CMake build type [Debug | Release | RelWithDebInfo | MinSizeRel] |
-DCUSTOM_COMPILER_FLAGS | Flags sent directly to default system Fortran compiler/wrapper |
-DCUSTOM_LINKER_FLAGS | Flags sent directly to default system Fortran linker/wrapper |
Any predefined CMake variable can be used to customize the compilation process. Refer to CMake variables for a complete list.
Expand the section to see some advanced configuration examples
Advanced configuration examples
To use the default CMake buildtypes you can configure the build script with this options to create a binary named Fall3d.x using double precission, serial code and without MPI using the debug buildtype
> cmake .. -DDETAIL_BIN=NO -DWITH-MPI=NO -DWITH-ACC=NO -DWITH-R4=NO \
-DCMAKE_BUILD_TYPE=debug
Notes
- The basic options set to
NO
(-DWITH-xxx=NO
) are shown here for clarity, it's not strictly necessary. If omitted, the default behavior of NO is applied ALL basic options. The options requiring some text defaults to empty if anything is defined by the user.
For a multi-GPU configuration using MPI and OpenACC using double precission and compiled with nvfortran using the debug flag -g
, the -O0
optimization, print in the terminal the acceleration information using the flag -Minfo=accel
, and adding to the linker the debug flag -g
execute the following command
> cmake .. -DDETAIL_BIN=YES -DWITH-MPI=YES -DWITH-ACC=YES \
-DCUSTOM_COMPILER_FLAGS="-g -O0 -Minfo=accel" -DCUSTOM_LINKER_FLAGS="-g"
Advanced configuration summary
When executing the command, you'll see a lot of information displayed in the terminal. However, the summary section at the end will be most important.
-- Summary
-- Binary name: Fall3d.NVHPC.r8.mpi.acc.x
-- Build path: /fall3d/bbuild/bin
-- Binary name: Detailed
-- Precision: Double (R8)
-- MPI: Enabled
-- OpenACC: Enabled
-- Compiler flags: -DWITH_MPI;-g;-O0;-Minfo=accel;-acc;-DWITH_ACC
-- Linker flags: -g;-acc
The final summary explicitly lists the flags that were added to the compiler and linker during the process (not the CMAKE_BUILD_TYPE ones). Once you've reviewed the summary and confirmed that there are no errors or warnings, you're ready to start the Build process. If you encounter any issues, please refer to the Troubleshooting section
Cmake build
With the configuration complete, the compilation process can be started by executing the following command
> make -j16
In the example, the number 16 has been used. This number indicates the number of CPUs that will be used to compile the code in parallel. If you don't know the number of available CPUs, you can safely use 16 as CMake will use the maximum amount of available CPUs.
Assuming no other output directory was set and the compilation was successful,The compilation process generates a new bin
folder containing the executable file ready to run into current working environment. You can link, copy, or move the folder or the binary to your desired location. For detailed instructions, refer to the RUN SECTION. If you encounter any issues, consult the troubleshooting section where common problems are commented.
Modifying the code
If any changes are made to the source code, you need to recompile. To do this, simply go back to the folder where you compiled in the previous steps and run make -j16
. CMake will detect the changes and compile only the necessary files, overwriting the old binary or creating it if it has been moved.
Compilation using Meson
Expand section for Meson installation instructions
Meson is a modern and efficient build system that simplifies the compilation and installation of software projects like FALL3D. It provides a flexible and intuitive interface for configuring and building projects across various platforms and compilers. This section covers the basic and advanced installation procedures for FALL3D using Meson. For users requiring a highly customized FALL3D installation, the Advanced installation process provides more flexibility and control.
Meson Prerequisites
Verify that installation program prerequisites are present on your system before proceeding. See section Installation requirements for further details.
- Fortran compiler
- Meson > 1.1
- NetCDF Fortran
- Optional:
- MPI library
- Hardware specific drivers (GPU)
Meson Configure
Notes:
- Please ensure that the required libraries (NetCDF, MPI, etc.) are either installed on your system or loaded in your working environment.
For the basic installation (serial code without MPI or OpenACC), simply type the following commands from the FALL3D root folder:
> meson setup build [ options ]
If some change in the options is required after the first option setting, just type:
> meson setup --reconfigure build [ options ]
Notes:
- The build directory name can be customized as long as it's consistently used in all commands.
Meson Configure options
To customize the compilation of FALL3D, Meson allows you to use various configuration options. These options are passed to the meson
command using the -D
prefix. This provides you with the flexibility to modify various aspects of the build process. A detailed table of these options and their effects is provided below.
Meson basic configuration
Option | Comment |
---|---|
-DWITH-R4 | Compiles using single precision |
-DWITH-MPI | Enables parallel using MPI (domain decomposition) |
-DWITH-ACC | Compiles the code using OpenACC (GPU and driver required) |
-DDETAIL_BIN | Adds to the binary name the compiler and the selected options |
Expand the section to see some basic configuration examples
Basic configuration examples
For a multi-GPU configuration using MPI and OpenACC, execute the following
> meson setup build -DDETAIL_BIN=true -DWITH-MPI=true -DWITH-ACC=true -DWITH-R4=false
For a single GPU configuration, you can use
> meson setup build -DDETAIL_BIN=true -DWITH-MPI=false -DWITH-ACC=true -DWITH-R4=false
Notes
- The basic options set to
NO
(-DWITH-xxx=NO
) are shown here for clarity, it's not strictly necessary. If omitted, the default behavior of NO is applied ALL basic options. The options requiring some text defaults to empty if anything is defined by the user.
For a serial and single precission binary named Fall3d.x you can configure
> meson setup build -DWITH-R4=YES
Basic configuration summary
When executing the command, you'll see a lot of information displayed in the terminal. However, the options section at the end will be most important.
Options
MPI support : YES
Double precision : YES
OpenACC : YES
Custom compiler flags: NO
Custom link flags : NO
All compiler flags : -DWITH_MPI
-DWITH_ACC
-acc
All linker flags : -acc
Once you've reviewed the summary and confirmed that there are no errors or warnings, you're ready to start the Build process. If you encounter any issues, please refer to the Troubleshooting section.
Meson advanced installation
For users looking for a fine grained control over the build process, the advanced installation of FALL3D provides the flexibility to customize default Fortran compiler and linker flags. In addition to the basic options, you can also use the following ones.
Advanced options
Option | Comment |
---|---|
-Dbuildtype | Meson build type [Plain | Debug | Release | RelWithDebInfo | MinSizeRel] |
-DCUSTOM_COMPILER_FLAGS | Flags sent directly to default system Fortran compiler/wrapper |
-DCUSTOM_LINKER_FLAGS | Flags sent directly to default system Fortran linker/wrapper |
Any predefined Meson variable can be used to customize the compilation process. Refer to Meson variables for a complete list.
Expand the section to see some advanced configuration examples
Advanced configuration examples
To use the default Meson buildtypes you can configure the build script with this options to create a binary named Fall3d.x using double precission, serial code and without MPI using the debug buildtype
> meson setup build -DDETAIL_BIN=false -DWITH-MPI=false -DWITH-ACC=false -DWITH-R4=false \
-Dbuildtype=debug
Notes
- The basic options set to
NO
(-DWITH-xxx=NO
) are shown here for clarity, it's not strictly necessary. If omitted, the default behavior of NO is applied ALL basic options. The options requiring some text defaults to empty if anything is defined by the user.
For a multi-GPU configuration using MPI and OpenACC using double precission and compiled with nvfortran using the debug flag -g
, the -O0
optimization, print in the terminal the acceleration information using the flag -Minfo=accel
, and adding to the linker the debug flag -g
execute the following command
> meson setup build -D DETAIL_BIN=true -D WITH-R4=false -D WITH-ACC=true \
-D WITH-MPI=true -D CUSTOM_COMPILER_FLAGS="-fast -g -Minfo=accel" \
-D CUSTOM_LINKER_FLAGS="-g" -D buildtype=plain
Advanced configuration summary
When executing the command, you'll see a lot of information displayed in the terminal. However, the summary section at the end will be most important.
Options
MPI support : YES
Double precision : YES
OpenACC : YES
Custom compiler flags: -fast -g -Minfo=accel
Custom link flags : -g
All compiler flags : -DWITH_MPI
-DWITH_ACC
-acc
-fast
-g
-Minfo=accel
All linker flags : -acc
-g
The final summary explicitly lists the flags that were added to the compiler and linker (not ingludig the buildtype ones) during the process. Once you've reviewed the summary and confirmed that there are no errors or warnings, you're ready to start the Build process. If you encounter any issues, please refer to the [Troubleshooting section]
Meson build
With the configuration complete, the compilation process can be started by executing the following command
> meson compile -C build/
Assuming no other output directory was set and the compilation was successful, the configuration and compilation process generate a new build
(if you used the commands as ahown in this readme) folder containing the executable file ready to run into current working environment. You can link, copy, or move the the binary to your desired location. For detailed instructions, refer to the TODO ### RUN SECTION ###. If you encounter any issues, consult the troubleshooting section where common problems are commented.
Modifying the code
If any changes are made to the source code, you need to recompile. To do this, simply go back to the project root folder where you configured meson in the previous steps and run meson compile -C build/
. Meson will detect the changes and compile only the necessary files, overwriting the old binary or creating it if it has been moved.
Uninstalling
To uninstall the project, you can choose from the following options:
-
Partial cleanup with make clean:
- By running
make clean
into the build folder for CMake, or using Mesonmeson compile --clean -C build
in the project's root folder, you will remove object files, libraries, and other temporary files generated during compilation, effectively removing the executable binary and any intermediate files. However, configuration files will generally remain in the build folder.
- By running
-
Complete removal of the build directory:
- If you want to remove all files related to the compilation, including the binary (if not moved), intermediate and configuration files, simply delete the build directory from project's root folder
rm -rf build
- If you want to remove all files related to the compilation, including the binary (if not moved), intermediate and configuration files, simply delete the build directory from project's root folder
-
Removing the binary:
- If you moved the binary just remove the executable binary, you can delete it directly:
rm Fall3d.x
.
- If you moved the binary just remove the executable binary, you can delete it directly:
Notes
- If you have installed the project in a system directory, make sure to follow the specific installation instructions to remove it correctly.
Troubleshooting
This section aims to provide a detailed guide to resolve the most common issues that may arise during the FALL3D compilation process. This documentation is periodically updated to incorporate solutions to the user's common issues.
NetCDF Fortran Library not detected by build tool
Problem:
- The build tool fails to automatically detect the NetCDFF library, even though it is installed.
- CMAKE ERROR:
NetCDFF not found. Set the path in the FALL3D_NETCDF_DIR environment variable.
- MESON ERROR:
ERROR: Assert failed: All netCDF detection tests failed, configure meson with -Dnetcdf-path=<path>
Causes:
- The environment maybe is not set correctly or has special settings.
Solution:
- Ensure the netCDFF library is installed or loaded.
- Ensure that the libraries are compatible with each other, with the compiler and with the system requirements.
- In case of some change in the environment start the installation process from scratch
rm -rf build
- If the problem persists, force the path for the netCDF Fortran library.
- CMAKE
# Set the FALL3D_NETCDF_DIR
> export FALL3D_NETCDF_DIR=<path_to_netcdff>
- MESON
> meson setup --reconfigure build [ options ] -Dnetcdf-path=<path_to_netcdff>
- Find netcdff in unknown environments
- Sometimes in unknown enviroments, specially in HPC, is not easy to define where are the libraries are located. To help with this, once the library is loaded you can use the command
env | grep -i netcdf
and check the output for the folder that contains the subfodersbin include lib share
and under the subfolderlib
the filelibnetcdff.so
can be found.
NetCDF errors compiling the code
Problem:
- The build process gives errors related with netCDF library.
- Compile errors directly related with netCDF fuctions.
- Corrupt or old module detected for netCDF.
Causes:
- In some cases, particularly in HPC clusters using module systems, changes in the building configuration over existent folder can lead to module conflicts due to cached remnants.
Solution:
- To address this, the most straightforward solution is to delete the build folder from the project's root folder
rm -rf build
, or empty it if you're into the build folderrm -rf .
and begin the compilation process from scratch. - If the problem persists even after verifying that dependencies are correctly detected, a thorough examination of library compatibility is necessary. Note that a NetCDF library compiled with a GNU compiler may not work when is used by an NVHPC compiler.