IRATE Format Specification

This document describes the basic structure of the IRATE format.

All IRATE files are HDF5 files, and hence usually have either a .h5 or .hdf5 extension.

IRATE Data File Format

The main data file for an IRATE format is referred to as simply an “IRATE file”. These files store the actual outputs of a simulation (typically a snapshot at a single time). To conform to the IRATE standard, such a file must satisfy the following conditions:

  • At the root of the file, there must be a Group ‘Header’. This Group should only contain other Groups, and the subgroups (typically 1 per file, although that’s not required) should have HDF5 attributes that specify information such as smoothing lengths, the sim’s name, cosmological parameters, and so on. The exact format of these Groups is undefined here and is left up to the source of the simulation.

  • Also at the root of the file, there must be a Group ‘Analysis’. This Group can contain any number of other Groups that store information and data (e.g. sub-sampled or smoothed versions of the base data) used for analysis of the data set. Each of these subgroups should have a name matching the name of the analysis tool, although the subgroups’ content is undefined here.

  • The root should also contain three more groups with names ‘Dark’, ‘Star’, and ‘Gas’. These contain the data for dark matter, stars, and gas, respectively. All three share a common format: They may either contain only data sets, only groups, or be empty (if no particles of that type are present).

    If they are groups, those groups are used to sub-divide the data in whatever way the simulation likes. In a simulation of a single galaxy, the stars might include groups for Bulge, Thin Disk, Thick Disk, and Halo stars, with no seperate groups for the dark matter and gas. A cosmological simulation, on the other hand, might sub-divide all three classes into Cluster, Group, and Field based on some overdensity criterion.

    Regardless of whether the three classes are subdivided into groups, all groups of datasets must have a certain format. For particle data, the following Dataset objects must be present, even if they have 0 particles:

    • ‘Position’ (N x d)
    • ‘Velocity’ (N x d)
    • ‘Mass’ (N)

    where d is the dimensionality (presumably pretty much always 3) and N is the total number of particles. Additional data sets (e.g. ‘Metallicity’,’Entropy’, ‘Density’, etc.) may be present, but the above 3 are the minimum required. Any other data sets are encouraged to either be shape N for scalar data, or N x d for vector data.

    Note

    Particle IDs are not specifically required in the format. The recommended convention (unless particle number changes over multiple timesteps) is that the implicit IDs match the index of the particle in the dataset. The standard order should be Dark, Star, Gas and indecies should be 0-based. Thus, if there are 100 dark particles 50 star particles and 50 gas particles, the last dark particle has id 99, the first and last star particles are 100 and 149, and the first gas particle is 150. If particle IDs are not contiguous (e.g. datasets from multiple timesteps over which particles are removed or added), the above convention should be used when possible for the initial dataset (e.g. the first timestep).

    For grid data:

    • The grid data specification has not yet been defined.
  • There must not be spaces in any group names so as not to confuse some HDF5 tools that don’t play well with spaces.

  • To conform to the “strict” format, there must also be no other groups in the root beyond the five previously mentioned, aside from optionally two more that match the halo catalog format described in the next section. Further, there can be at most one nested group in the ‘Dark’,’Gas’, and ‘Star’ groups, and all datasets in those groups must have the same first dimension. In the non-strict format, deeper nested groups, mis-matched datsets and additional groups at the root are allowed.

IRATE Halo Catalog File Format

IRATE also includes a specification of the file format for halo catalogs that are typically derived from large cosmological simulations. Note that these files may either be stand-alone HDF5 files, or the necessary groups can be added to an IRATE format data file. The halo catalog format includes the follow conditions:

  • At the root of the file, there must be a Group ‘CatalogHeader’. This Group should only contain other Groups, and the subgroups (typically 1 per file, although that’s not required) should have HDF5 attributes that specify information such as linking lengths, the simulation used to generate the halo catalog, or other gnereal info about the halo-finder. The exact format of these Groups is undefined here and is left up to the source of the halo catalog.
  • The root should also contain a Group ‘Catalog’. This group must contain a Dataset with the name ‘Center’ that has shape N x d, where N is the number of halos in the catalog, and d is the dimensionality (typically 3). All other datasets in the catalog must have a matching first dimension, and should be in the same order. That is, the ith entry in ‘Center’ should correspond to the same halo as the ith entry in any of the other datasets. Each item in this group must be a dataset, unless it is a Group named either ‘RadialProfiles’ or ‘ParticleData.’ If either of these exist, they must contain only datasets; however, the datasets in ‘ParticleData’ are expected to be organized differently and need not be N x d.
  • There must not be spaces in any group names so as not to confuse some HDF5 tools that don’t play well with spaces.

Examples

A sample IRATE Format file might have the following structure. Note that the ‘ThinDisk’,’ThickDisk’,’Bulge’, and ‘Halo’ groups are not actually a part of the specification, but are examples of possible ways one might wish to sub-divide the particle data:

/
/Header
/Header/GadgetHeader (contains attributes with simulation info but no datasets)
/Analysis
/Analysis/Tipsy
/Analysis/Tipsy/TipGrid (a dataset with gridded data used by tipsy)
/Analysis/SomeOtherAnalysisProgram
...
/Dark
/Dark/Position (dataset)
/Dark/Velocity (dataset)
/Dark/Mass (dataset)
/Star
/Star/ThinDisk
/Star/ThinDisk/Position (dataset)
/Star/ThinDisk/Velocity (dataset)
/Star/ThinDisk/Mass (dataset)
/Star/ThickDisk
/Star/ThickDisk/Position (dataset)
/Star/ThickDisk/Velocity (dataset)
/Star/ThickDisk/Mass (dataset)
/Star/Bulge
/Star/Bulge/Position (dataset)
/Star/Bulge/Velocity (dataset)
/Star/Bulge/Mass (dataset)
/Star/Halo
/Star/Halo/Position (dataset)
/Star/Halo/Velocity (dataset)
/Star/Halo/Mass (dataset)
/Gas
/Gas/Position (dataset)
/Gas/Velocity (dataset)
/Gas/Mass (dataset)

Below is an example of an IRATE Halo Catalog file:

/CatalogHeader
/CatalogHeader/FOFHeader (contains attributes with halo-finder run info but no datasets)
/Catalog
/Catalog/Center (dataset)
/Catalog/Subhalo (dataset)
/Catalog/Velocity (dataset)
/Catalog/NParticles (dataset)
/Catalog/RadialProfiles/r  (dataset)
/Catalog/RadialProfiles/dens  (dataset)
...

Table Of Contents

Previous topic

Getting and Installing IRATE

Next topic

Command Line Scripts and Conversion Tools

This Page