School of GeoSciences

School of GeoSciences

Scientific I/O Libraries

Contents

Overview

This page describes the plethora of Scientific I/O libraries available on the system and how to access them. This is not entirely straight forward since these libraries also contain Fortran bindings which are compiler dependent. This problem is solved by using modules.

If you want to use the system compiler, i.e. gcc, g++ or gfortran, you do not need to load a module and the libraries are installed in the system path. To compile a F90 (or newer) fortran program you need to specify the module include path:

gfortran -o test_gfortran test.f90 -I/usr/lib64/gfortran/modules/ -lnetcdff

If you want to use modules from within a script you need to first setup your environment by adding

source /etc/profile.d/modules.sh

to your shell script.

You can get a list of available modules by running

module avail

and you can load a module using

module load NAME_OF_MODULE

First of all you will need to select a compiler you want to use. This affects mostly the Fortran compiler since (where available) the C compilers are generally produce interchangable object code. The following compilers are currently supported

  • compiler/intel/11.0 The Intel Professional Compiler Suite v 11.0 which includes C, C++ and Fortran95 compilers and various libraries. Here is the documentation.
  • compiler/intel/12.0 The Intel Professional Compiler Suite v 12.0
  • compiler/intel/2013.1 The Intel Professional Compiler Suite v 2013.1
Although Intel supplies C and C++ compilers the system compilers where used Once you have selected a compiler module more modules will become available. Select any number of the Scientific I/O modules and use them from your own software.

HDF5

HDF5 is a very comprehensive albeit complicated to uses scientific data I/O library. It will store about anything you can think off (which is what makes it so complex to use). The library cannot be built with the NAG f95 compiler. So if you need to use HDF5 you cannot use the NAG compiler.

The module will set various environment variables. In particular it will set HDF_PREFIX to be the path to where the module is installed. You can use this for your compiler settings.

netCDF-4

netCDF-4 is less complex than HDF5 and thus not support all the wonderful stuff HDF5 does but it is a lot easier to use. netCDF-4 supports a number of netCDF formats. By default you will get the classic format which means your datafiles will be most portable. You can also specify to use the 64bit version to be able to write larger files or the new HDF5 based file format which offers new features.

Except for the NAG compiler all netCDF packages depend on the HDF5 module. Load it first. Then load the netCDF module using

module load hdf5/1.8.5.patch1 netcdf/4.1.1

You can leave the version out if you are not bothered about the particular version. The module with the highest version number will be used

Compile your code using

${FC} -O2 -o foo foo.f -L${NETCDF_PREFIX}/lib -L${HDF5_PREFIX}/lib -I${NETCDF_PREFIX}/include -lnetcdf -lnetcdff -lhdf5 -lhdf5_hl

where ${FC} is your Fortran compiler. Leave out the hdf5 libraries if you are using the NAG compiler.

Checkout the netCDF documentation. I recommend the use of the Fortran90 interface. It hides a lot of the type-dependent pain of the API.

silo

silo is another scientific I/O library which uses HDF5 as its backend data store. I have not used it (yet) but it looks very exciting. You can use it to store all kinds of field data defined on structured and unstructured grids. Load it using

module load silo/4.8 netcdf/4.1.1

VisIt

VisIt is a 3D visualisation package that can read data in all sorts of formats, including HDF4, HDF5, netCDF and silo. Before you can run visit you need to load the module

module load applications/visit

You can find some example data files in the directory /usr/geos/VisIt/visit_data_files. Documentation can be found online.