HDF5

unique technology suite that makes possible the management of large and complex data

Introduction

Documentation for this package can be found at: https://www.hdfgroup.org/HDF5/doc/index.html

From the ( https://www.hdfgroup.org/HDF5/doc/UG/UG_frame03DataModel.html ):

"The Hierarchical Data Format (HDF) implements a model for managing and storing data. The model includes an abstract data model and an abstract storage model (the data format), and libraries to implement the abstract model and to map the storage model to different storage mechanisms. The HDF5 library provides a programming interface to a concrete implementation of the abstract models. The library also implements a model of data transfer, i.e., efficient movement of data from one stored representation to another stored representation. The figure below illustrates the relationships between the models and implementations. This chapter explains these models in detail."

Using HDF5 on RCC Resources

On HPC hdf5 can be accessed and used for many of these functions. We will consider an example which can be found on at the homepage ( https://www.hdfgroup.org/HDF5/examples/ ) compiling and linking hdf5 libraries on HPC using the gnu compiler,

[fsuID_hpc-login-40]$ gcc -c test_hdf5.c -I/opt/hpc/gnu/include/
[fsuID_hpc-login-40]$ gcc test_hdf5.o -L/opt/hpc/gnu/lib64 -lhdf5

Using gnu is not necessary. These libraries are also available to other compilers. Using a different compiler will simply mean linking and including a different path, for example:

-I/opt/hpc/<compiler>/include/
-L/opt/hpc/<compiler>/lib64/ -lhdf5