11  Metadata companion files

Metadata are essential information about data. It is as important as the data itself, it provides the context to the data. Metadata include documents intended for human use, consisting mainly of text and images, but mostly, when refering to metadata we refer to machine-readable information in the form of tables (see tables) and structured files that accompany data. ‘Machine-readable’ means that it can be easily processed by computers without human intervention. These files are also referred sometimes as ‘sidecar’ or ‘companion’ files.

We recommend thinking about content and format of these files early on in a project. When possible, it is preferable to adopt or adjust discipline-specific standards (see standards) instead of creating new standards.

11.1 Format: JavaScript Object Notation (JSON)

A popular format for metadata companion files is JavaScript Object Notation (JSON). JSON is an open and text-based standard format for data interchange. It is often recommended as a format for metadata files because it is easy to read by humans and by many programming languages. JSON is also a good format for larger (metadata) data that have a hierarchical structured relationship. The structure of a JSON object is:

  • The data are in name/value pairs and data objects are separated by commas

  • Curly braces {} hold objects

  • Square brackets [] hold arrays

  • Each data element is enclosed with quotes “” if it is a character, or without quotes if it is a numeric value

The ‘fields’ or name and value pairs in the JSON files can vary widely depending on the data and file type they are accompanying. The examples are provided by the Brain Imaging Data Standards for microscopy data:


Example of sidecar JSON file (*_<suffix>.json)
{
        "Manufacturer": "Hamamatsu",
        "ManufacturersModelName": "C9600-12",
        "PixelSize": [0.23, 0.23],
        "PixelSizeUnits": "um",
        "Magnification": 40,
        "BodyPart": "BRAIN",
        "BodyPartDetails": "corpus callosum",
        "SampleEnvironment": "ex vivo",
        "SampleFixation": "4% paraformaldehyde, 2% glutaraldehyde",
        "SampleStaining": "LFB",
        "SliceThickness": 5,
        "TissueDeformationScaling": 97
}

Example of participants.json:

{
    "species": {
        "Description": "binomial species name from the NCBI Taxonomy 
        (https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi)"
    },
    "strain": {
        "Description": "name of the strain of the species"
    },
    "strain_rrid": {
        "Description": "research resource identifier (RRID) of the strain 
        (https://rrid.site/data/source/nlx_154697-1/search)"
    }
}

11.2 Creating metadata files from tables

Data acquisition instruments often generate metadata files automatically. However, this is not always the case and sometimes researchers need to customize the content of these files. Adopting existing metadata standards is usually recommended. Some projects may required their own JSON-schemas, although developing these schema may be labor intensive and require data stewards or expert data managers. Often, simpler solutions are sufficient, and researchers may, for example, generate a JSON file from the content of a table (see Table 11.1). This can be done using many programming languages, for example, the R package jsonlite is simple to use and offers several encoding formatting options. Automatically generating these files is recommended, rather than resorting to manual edits that can lead to errors that go unnoticed. An example of R code to create a JSON file from a table is provided in Appendix Code Examples.

Table 11.1: Example of metadata in tabular format.
id img_location sex condition treatment
subject_04 dummyfolder/subject_04_imgfile.tiff female A control
subject_93 dummyfolder/subject_93_imgfile.tiff female B control
subject_12 dummyfolder/subject_12_imgfile.tiff female B treat_3
subject_65 dummyfolder/subject_65_imgfile.tiff male B control
subject_98 dummyfolder/subject_98_imgfile.tiff female A treat_1
subject_71 dummyfolder/subject_71_imgfile.tiff male A treat_3
subject_24 dummyfolder/subject_24_imgfile.tiff female B control

The first row of Table 11.1 can represented in a JSON file would look like this:

 
{
  "id": "subject_04",
  "img_location": "dummyfolder/subject_04_imgfile.tiff",
  "sex": "female",
  "condition": "A",
  "treatment": "control"
}