13  File formats

The choice of a format will determine how data can be accessed throughout its lifecycle, including sharing and reuse. In general, we need to consider long-term preservation when choosing a format. Files in formats with the following features will be more likely to be accessible in the future:

• Non-proprietary

• Open, documented standard

• Popular format

• Standard representation

• Unencrypted

• Uncompressed

We recommend selecting file formats based on these features and to follow existing guidelines for general and domain-specific file formats. The section below summarizes the file formats recommended by various institutions.

13.1 File formats in the general domain

There are many institutions providing recommendations on file formats in general. This section shows a couple of representative examples.

Recommendations from the Faculty of Biology and Medicine, University of Lausanne (Medical Library):

Category	Recommended Formats
Text	PDF/A – PDF/X, Plain text (.txt), Open Office (.odt), XML / HTML (with schema), Word XML (.docx), RTF, LaTeX
Images	Bitmap: TIFF (uncompressed), PNG, JPEG2000, (GIF). Vector: SVG
Tabular Data	CSV (comma, tab, semi-colon), Open Office (.ods), XML / HTML (with schema), Excel (.xlsx), .SQL
Video	MPEG-4 (H.264) (~ MP4), Motion JPEG 2000, MPEG-1/2
Audio	WAV (preferably Broadcast Wave Format, LPCM), AIFF (LPCM), OGG Vorbis, MP3 (MPEG Layer III), AAC (MPEG-4)

Recommendations from the open data publishing platform (Dryad):

Category	Recommended formats
README files	Markdown (MD) or text (TXT)
Tabular data	Comma-separated values (CSV)
Non-tabular data	Semi-structured plain text (e.g., protein sequences)
Structured plain text	XML, JSON (e.g., metadata companion files)
Images	PDF, JPEG, PNG, TIFF, SVG
Audio	FLAC, AIFF, WAV, MP3, OGG
Video	AVI, MPEG, MP4
Compressed file archives	TAR.GZ, 7Z, ZIP

13.2 Interoperable formats in neuroimaging

There are are many modalities of neuroimaging research and therefore many different file formats used in this field. This section covers some of the most popular formats in neuroimaging. The Brain Imaging Data Structure - BIDS standards also provides recommendations for some of these formats in the different data modalities available.

Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Positron Emission Tomography (PET) and similar

• Neuroimaging Informatics Technology Initiative (NIfTI). It is an open file format commonly used to stored imaging data obtained with magnetic resonance imaging methods. The associated file extensions are: .nii, .nii.gz, .hdr/.img.

• Digital Imaging and Communication in Medicine (DICOM) is a technical standard for digital storage and transmission of biomedical images. It was developed in the 80s and it is widely in use today. The DICOM standard is widely adopted by hospitals and medical software. The files extension is usually .dcm.

Microscopy

• OME data model. OME Model is an specification from the Open Microscopy Environent for storing data on biological imaging. The model includes parameter and extensive metadata. OME-XML is a file format used to store data according ot the model. OME-TIFF is a multiplane tiff file with OME metadata in the header (as OME-XML). See more details in their publication (Goldberg et al. 2005). Further, OME-ZARR (sometimes referred to as OME-NGFF or NGFF) is a cloud-optimized format developed to improve access and storage for large data (Moore et al. 2023).

• In BIDS (v1.10.0) microscopy raw data MUST be stored in one of the following formats:

  • Portable Network Graphics (.png)

  • Tag Image File Format (.tif)

  • OME-TIFF (.ome.tif for standard TIFF files or .ome.btf for BigTIFF files)

  • OME-ZARR/NGFF (.ome.zarr directories)

Electroencephalography (EEG) and intracraneal electroencephalography (iEEG)

There is no single standard. The recommendations from BIDS for raw data are the following:

• European data format (.edf). Each recording is a single file. edf+ files are permitted. The capital .EDF extension MUST NOT be used.

• BrainVision Core Data Format (.vhdr, .vmrk, .eeg). Each dataset recording consists of a file triplet.

• EEGLAB (.set, .fdt). The format used by the MATLAB toolbox EEGLAB. Each recording consists of a .set file with an OPTIONAL .fdt file.

• Neurodata Without Borders (.nwb). For iEEG data only. Each recording consists of a single .nwb file.

• MEF3 (.mefd). For iEEG data only. Each recording consists of a .mefd directory.

• Biosemi (.bdf). For EEG data only. Each recording consists of a single .bdf file. bdf+ files are permitted. The capital .BDF extension MUST NOT be used.

13.3 Metadata, markup and data serialization

For more information about metadata see the section Metadata companion files. Markup languages specify the structure and formatting of a document. Data serialization languages are used to create configuration files in plain text and compatible between systems.

• JavaScript Object Notation (JSON). (.json In the BIDS specification, JSON files are primarily used as “sidecar” files accompanying the data files. There are also a few special cases of JSON files being first-order data files, such as genetic_info.json.

• Extensible Markup Language (XML) (.xml) is a markup language and file format for storing, transmitting, and reconstructing data.

• YAML Ain’t Markup Language (.yaml,.yml) is a human-friendly data serialization language for all programming languages which is often used for configuration files.

• Markdown, a lightweight language with simple syntax that will add formatting elements to plain text. This format is widely used for technical documentation online.

13.4 File format conversion

There are multiple reasons why researchers often have to convert files to a different format. For example, imaging acquisition hardware may generate files in a proprietary format. Researchers will often convert these files to a standardized format like NIFTI to be able to analyze it, share and publish the images in a ‘more universally’ interoperable format. File compression is also often done to facilitate sharing.

These are some important considerations when converting files:

• Quality should be preserved: there should be no data loss (e.g., lossy compression methods where some data is lost may have an impact when reusing the data) or data corruption (data that is still present but unreadable).

• When possible, the source and converted files should be kept, in case errors in the process were to be found in the future.

• Conversion is better done automatically with a script logging any conversion parameters.

• Long-time preservation and reusability should be considered when choosing the conversion format.

Goldberg, Ilya G, Chris Allan, Jean-Marie Burel, Doug Creager, Andrea Falconi, Harry Hochheiser, Josiah Johnston, Jeff Mellen, Peter K Sorger, and Jason R Swedlow. 2005. “The Open Microscopy Environment (OME) Data Model and XML File: Open Tools for Informatics and Quantitative Analysis in Biological Imaging.” Genome Biology 6 (5): R47. https://doi.org/10.1186/gb-2005-6-5-r47.

Moore, Josh, Daniela Basurto-Lozada, Sébastien Besson, John Bogovic, Jordão Bragantini, Eva M. Brown, Jean-Marie Burel, et al. 2023. “OME-Zarr: A Cloud-Optimized Bioimaging File Format with International Community Support.” Histochemistry and Cell Biology 160 (3): 223–51. https://doi.org/10.1007/s00418-023-02209-1.