ScanImage Tiff Reader

About

This is a command line tool and library for extracting data from Tiff and BigTiff files recorded using ScanImage. It is a very fast tiff reader and provides access to ScanImage-specific metadata. Interfaces to the library are provided for Python, Matlab and Julia . This library should actually work with most tiff files, but as of now we don’t support compressed or tiled data.

Binaries can be downloaded from scanimage.org for 64-bit versions of Windows, OS X, or Linux.

Both ScanImage and this reader are products of Vidrio Technologies. If you have questions or need support feel free to email.

Examples

Each example shows how to extract raw image data from the file.

Command Line Interface

ScanImageTiffReader image data my.tif > my.raw

Python Interface

from ScanImageTiffReader import ScanImageTiffReader
vol=ScanImageTiffReader("my.tif").data();

Matlab Interface

vol=ScanImageTiffReader('my.tif').data();

Julia Interface

include("ScanImageTiffReader.jl")
vol=ScanImageTiffReader.open("my.tif",ScanImageTiffReader.data)

C Library

#include <tiff.reader.api.h>
const size_t bytes_per_pixel[]={1,2,4,8,1,2,4,8,4,8};
ScanImageTiffReader reader=ScanImageTiffReader_Open("my.tif");
struct nd shape=ScanImageTiffReader_GetShape(reader);
size_t nbytes=bytes_per_pixel[shape->type]*shape.strides[shape.ndim];
void *data=malloc(nbytes);
ScanImageTiffReader_GetData(reader,data,nbytes);
ScanImageTiffReader_Close(reader);

ScanImage Tiff Files

ScanImage records images, volumes and video as Tiff or BigTiff files. For the most part Tiff and BigTiff are similar, but they are not the same; BigTiff enables storage of larger (>4GB) data sets. In addition to the image data, ScanImage stores metadata describing the microscope configuration and settings used during the acquisition. These are stored in the file itself.

Some of this metadata is accessible using standard Tiff readers. The tiff format provides for data fields (tags) which can be used to attach data to describing each frame. Past versions of ScanImage would store a copy the metadata in “image description” tag for each frame. Much of the metadata is redundant, and this can lead to longer load times and significant storage overhead in some cases.

Fortunately, the tiff format is very flexible and allows us to easily store the part of the metadata that doesn’t change over time in a dedicated block of the file. However, this metadata block is only accessible if you know where to look. The tiff files can still be read by any reader conforming to the baseline tiff specification, but those readers will not be aware of the ScanImage metadata block. The ScanImageTiffReader knows how to extract this metadata and also provides fast access to the images themselves.

ScanImage tiff files store the following kinds of data:

Kind Description
data The image data itself
metadata Frame-invariant metadata such as the microscope configuration and region of interest descriptions.
descriptions Frame-varying metadata such as timestamps and I2C data.

The metadata sections themselves are encoded as either matlab evaluable strings or json.

The ScanImage documentation has a very detailed description of how data is stored in a ScanImage Tiff.

Performance

The ScanImageTiffReader is fast.

Compared to other common BigTiff readers, the ScanImageTiffReader is fast. When done right, reading a Tiff file can have very low overhead. One should expect that the read performance is roughly the same as the bandwidth-limiting bottleneck, usually the hard-drive used for storage.

Thanks to solid-state storage and aggressive caching of files by the operating systems, read speeds of 500 MB/s or greater are very acheivable. Realizing this bandwidth improvement can reduce read times by an order of magnitude or more.

Reading from an SSD drive (Samsung 840 Pro),

_images/drive_stats.png

Benchmark speeds for the storage drive used here.

_images/file_explorer.png

A 6GB BigTiff file produced by ScanImage.

A roughly 6GB file should take ~11.4 seconds to read. Using the ScanImageTiffReader:

_images/julia_si.png

Time to read a stack using the ScanImageTiffReader API in Julia. The choice of language doesn’t make a significant difference.

We can compute the effective read bandwidth by dividing the total byte size of the file (6.277 GB) by the amount of time it took to read the file. In the Julia example above, this comes out to 430 MB/s. The effective bandwidth can be compared with other BigTiff readers:

_images/bandwidth.png

Effective read bandwidth measured for different readers. “Cached” indicates the file had been accessed once before the benchmark was run. “Uncached” benchmarks were measured by reseting the computer between measurements.

Due to caching by the operating system, we sometimes exceed the expected speed of the hard drive. The behavior of this kind of file-system caching might vary between operating systems.