Implementationstooling reviewed 2026-08
3 Sequencing Data and File Formats: Implementations
Everything on this page is generated from tools/formats.json. If something here is wrong or out of date, the fix belongs in that file, not in this prose.
For what these formats are and why their edges are sharp, see Section 2.1.
3.1 Start here
The advice this chapter exists to give is short: do not write a parser. Read formats through a library that has already settled the ambiguities described in Section 2.1, and use the command-line tools built on that library rather than reimplementing their operations.
What samtools is to alignments, this is to variants: reading, filtering, annotating, merging and subsetting VCF and BCF, released in lockstep with samtools and htslib. Its filtering expression language is the part worth learning first, because it replaces a great deal of fragile text processing. (Danecek et al. 2021, 2011)
Dual-licensed MIT or GPL-3.0. Building against the GNU Scientific Library — optional and off by default — commits you to the GPL branch.
Interval arithmetic as a command-line algebra: intersect, merge, subtract, closest, complement. A surprising share of genomics questions that are not alignment or counting turn out to be an interval operation, and expressing one here is shorter and harder to get wrong than the equivalent script. (Quinlan and Hall 2010)
Releases have slowed — the most recent is from November 2023 — although fixes continue to land on the default branch. The status field here is the thing to check before assuming it is still moving.
The reference implementation of SAM, BAM, CRAM, VCF and BCF, and the layer almost everything else on this page stands on: samtools and bcftools are command-line front ends over it, and pysam, Rsamtools and rust-htslib are bindings to it. Reading a format through htslib is how you inherit the specification’s edge cases instead of rediscovering them. (Bonfield et al. 2021)
Licensing is split: the cram/ subdirectory is 3-clause BSD and everything else is MIT/Expat, which matters if you vendor it.
Not an aligner, but every alignment step ends in it: sorting, indexing and converting SAM to BAM or CRAM. In practice the aligner and the sort are one piped command, so the unsorted intermediate never reaches disk. (Danecek et al. 2021)
The FASTA and FASTQ workhorse: subsetting, filtering, deduplicating, translating and reformatting, with consistent flags across every subcommand. It ships as a single static binary with no runtime dependencies, which makes it the easiest tool on this page to get onto a cluster you do not administer. (Shen et al. 2016)
3.2 Interval operations
Interval arithmetic is the operation most often rewritten by hand and most often rewritten wrongly, because it is where the coordinate conventions of Section 2.3 collide. A tool that already knows BED is half-open and GFF is closed will convert correctly at the boundary; a script that treats both as numbers will not.
The choice between the two tools here is about memory, not capability.
| Tool | Upstream | Languages | Why / why not |
|---|---|---|---|
| BEDOPS | active | C++ |
A second interval algebra, built around a strict sort order that lets it stream operations rather than hold inputs in memory; the paper makes its case on exactly that. Reach for it when a bedtools command runs out of memory on a large input. Otherwise bedtools has the larger ecosystem and the gentler learning curve. (Neph et al. 2012) |
3.3 FASTA and FASTQ utilities
These are the everyday operations on unaligned sequence — subsetting, filtering, converting, sampling. Both entries do the job; the difference is breadth against minimalism, and how easily each gets onto a machine you do not control.
| Tool | Upstream | Languages | Why / why not |
|---|---|---|---|
| seqtk | active | C |
Minimal and dependency-free: subsample, trim, convert, reverse-complement. Where SeqKit is a toolkit, this is a small amount of C that compiles anywhere, which is why it keeps turning up inside other people’s pipelines and container images. |
3.4 Reading formats from your own language
This is where the chapter’s advice becomes concrete. Every entry below is a way to open these files in-process, without shelling out and parsing text.
| Tool | Upstream | Languages | Why / why not |
|---|---|---|---|
| Biopython | active | py |
The general-purpose Python library, and the one that reads the widest range of formats behind a single SeqIO and AlignIO interface. For alignment files pysam is the better choice because it is htslib underneath; this is what you want for the long tail of sequence and annotation formats that htslib does not cover. (Cock et al. 2009) |
| FASTX.jl | dormant | jl |
FASTA and FASTQ for Julia, with the type discipline BioJulia is built around: records parse into typed structures rather than strings, so a malformed record fails at read time instead of somewhere downstream. Paired with XAM.jl it covers most of what a Julia analysis will open. |
| noodles | active | rs |
A native Rust implementation of the formats — SAM, BAM, CRAM, VCF, BCF, FASTA, FASTQ, GFF, BED — rather than a binding to htslib. The independence is the point and also the risk: it is the only entry here reimplementing the specifications instead of inheriting htslib’s reading of them. |
| rtracklayer | active | R |
R’s route into the annotation formats — BED, GFF, WIG and bigWig — returning GRanges objects the rest of Bioconductor already understands, which is what makes it worth using over a bare reader. For alignment files R goes through Rsamtools instead, which is htslib underneath. (Lawrence et al. 2009) |
| XAM.jl | active | jl |
SAM and BAM for Julia, again written natively rather than bound to htslib. It is the reason a Julia pipeline can read alignments in-process, which is precisely the gap the read alignment chapter reports for every other stage of that work. |
3.5 Language coverage: a better story than alignment
The read alignment chapter had to report that outside C and C++ there is almost nothing but bindings. File handling is different, and the difference is worth stating plainly.
Python reaches these formats two ways: through a binding to the reference implementation, and through a native library that covers a wider range of formats less deeply. R’s route is Bioconductor, which is a binding for alignments and native code for annotations, returning objects the rest of that ecosystem already understands. Rust and Julia both have native implementations — not wrappers — which is a genuine change from the picture in the previous part.
That native code is also the honest caveat. A binding inherits the reference implementation’s twenty years of argument about edge cases. A reimplementation starts those arguments again. Both entries reimplementing the specifications here are young by the standards of the formats they read, and one of them is already marked dormant above.
NoteWhat this means in practice
For alignment and variant files, prefer whatever route reaches the reference implementation in your language — that is the path with the fewest disagreements. For annotation and sequence formats, native libraries are on much safer ground, because the formats are simpler and the ambiguities fewer.