22 Metagenomics: Concepts

Concepts

22.1 The problem

Everywhere else in this book the sample is one organism and the reference is its genome. Here the sample is a community — a gram of soil, a stool sample, a litre of seawater — and there is no reference, because you do not know what is in it. That is not a detail of the protocol. It is the whole difficulty, and almost every idea in this chapter is a consequence of it.

Two things follow immediately. Membership is unknown, so any method that matches against a database is answering a question about the database as much as about the sample. And proportions are unknown and enormously unequal: a community routinely spans several orders of magnitude in abundance, so the depth at which the commonest organism is oversampled is the depth at which the rarest is represented by a handful of reads, or by none.

The unit of a finding is also unsettled, in a way it is not elsewhere. In a human study the unit is a variant or a gene. Here it might be a species, a genome reconstructed from the data, a gene family, or a pathway — and these are not different resolutions of one measurement. They are different measurements, answered by different methods, with different failure modes. Deciding which one the question actually needs is the first analytical choice, and it is made before any software is chosen.

Metagenomics also has an unusually good record of knowing how well its methods work, because the field has repeatedly benchmarked them against simulated communities of known composition (Meyer et al. 2022). That work is the source of most defensible statements about relative performance, and it is worth reading before believing any single paper’s own comparison.

22.2 Two routes through the data

There are two ways to get from reads to an answer, and they are so different that they have different vocabularies and largely different failure modes.

Reference-based profiling compares reads to a database of known sequence and reports what matched. It is fast, it works at any depth, and it needs no assembly. Its ceiling is the database: an organism with no sequenced relative cannot be reported, and the fraction of a sample that is unclassifiable is routinely large and is not evenly distributed across environments.

Genome-resolved metagenomics assembles the reads into contigs, groups the contigs into putative genomes, and then asks what those genomes are. It can recover organisms nobody has ever isolated, which is its entire justification. It needs depth, it needs computation, and it yields incomplete genomes of uncertain purity rather than clean answers.

These are not competitors so much as different questions. “How does the composition of this community differ between my two conditions” is a profiling question. “What is this organism that dominates my samples and is not in any database” is not answerable by profiling at all.

The routes fail in opposite directions, which is why running both is more informative than it sounds. Profiling is precise about things that are already known and silent about everything else. Assembly is the only way to see the unknown, and it preferentially recovers whatever was abundant enough to assemble — so the organisms it misses and the organisms profiling misses are different sets.

22.3 Classification is a reference problem

A taxonomic classifier does not identify organisms. It reports, for each read or each sample, the best explanation available within its database, and every property of the answer follows from that.

What is absent cannot be reported, and its reads do not disappear. They are either left unclassified or, worse, assigned to whatever relative is closest, which produces a confident name for an organism that is not there. The second failure is far more damaging than the first because nothing in the output distinguishes it from a real detection.

Shared sequence forces a choice of rank. A read from a conserved gene is consistent with many genomes at once. The standard response is to report it at the lowest common ancestor of everything it matches, which is the honest answer and means that a large share of reads legitimately stop at genus, family or higher. A method that pushed all of them down to species would be more precise and less true.

The threshold is a dial, not a fact. Classifiers expose a parameter controlling how much evidence a read needs, and comparisons across tools and databases find that false positives at permissive settings are substantial and tool-dependent (Ye et al. 2019). A species list is therefore always a list of things above some threshold, and the long tail of low-abundance detections is mostly not real.

A taxonomic name is not a stable identifier

Taxonomy is a hypothesis about relationships, and it is actively revised. A genome-based taxonomy that reorganizes ranks so they are consistent across the tree (Parks et al. 2022) disagrees with the traditional literature-derived taxonomy in a great many places — names moved between genera, ranks renormalized, polyphyletic groups split. Neither is wrong; they are answering different questions.

The practical consequence is that a species name means nothing without the taxonomy it came from. Two tables that both say “Clostridium” may not be about the same organisms, and comparing them without a mapping produces a result that is impossible to debug later because everything looks fine.

22.4 What abundance is relative to

“Relative abundance” is the most-reported quantity in this field and one of the most misread, because three different things are called by that name.

Fraction of reads. The direct output of read classification. It is proportional to the amount of DNA an organism contributed, so a bacterium with a 6 Mb genome contributes roughly twice the reads of an equally numerous one with a 3 Mb genome. Read fractions describe the DNA in the tube.

Fraction of cells. Usually what people mean, and what they use when they say one organism is “twice as common”. Getting from reads to cells requires dividing by genome size — or measuring in a way that already is per-cell, which is what marker-gene methods do by counting sequences that occur once per genome rather than counting the whole genome.

Copies per unit sample. Not obtainable from sequencing at all without an external quantification such as a spike-in or a cell count, because sequencing depth is set by the machine and not by the sample.

Whichever is chosen, the result is compositional: the values are constrained to sum to one, so nothing in the table can move without something else moving. An organism that has genuinely not changed will appear to decrease whenever another organism blooms, and a test that treats the columns as independent measurements will report that decrease as a finding.

This is the same structural problem as the normalization discussion in Section 16.5, and it is worse here, because the total is not merely unknown but is frequently the biologically interesting quantity. Two samples with identical relative composition can differ enormously in absolute microbial load, and the sequencing cannot tell.

22.5 Assembly, binning and what a MAG is

The genome-resolved route runs in three steps, and each one introduces a distinct kind of error.

Assembly. The problem of Section 8.1, with its most useful assumption removed. A single-genome assembler can treat coverage as roughly uniform and read a departure from it as evidence of a repeat; here coverage varies by orders of magnitude for reasons that are purely about abundance. So the graph must be traversed without that signal, and the result is a set of contigs, generally not scaffolded, whose depth ranges over the whole community. The repeat problem of Section 8.2 is also worse, because sequence shared between two organisms behaves like a repeat and there is no single genome for it to belong to.

Binning. Contigs are grouped into putative genomes using two signals that have nothing to do with the reference databases. Composition — k-mer frequency profiles are characteristic of a genome and reasonably constant along it. And coverage — fragments of the same genome are present in the same proportion in every sample, so their depths covary across samples in a way that fragments of different genomes do not. The second signal is much stronger than the first and gets stronger with more samples, which is why binning many libraries together recovers genomes that per-sample binning cannot.

Quality assessment. The output is a bin: a set of contigs somebody’s algorithm believes came from one organism. It is a claim, and the claim is checked by asking whether the genes expected to be present exactly once in a genome are present exactly once. Missing ones suggest the bin is incomplete; duplicated ones suggest it contains more than one organism.

The resulting object — a metagenome-assembled genome — is not a genome in the sense the rest of this book uses the word. It is a draft consensus over a population, assembled from a mixture, of estimated completeness and estimated purity, with the parts that were hard to assemble missing rather than randomly sampled. Community standards exist that define what may be reported at what quality tier and what metadata must accompany it (Bowers et al. 2017), and they exist precisely because “we recovered a genome” had been meaning too many different things.

Completeness and contamination are estimates of themselves

Both numbers come from counting marker genes, so both inherit whatever is wrong with the marker set. A genome from a lineage with few sequenced relatives gets a marker set that fits it poorly — and that is exactly the novel organism the study was run to find, so the estimator is least reliable on the cases that matter most.

The failure is also asymmetric in a way that flatters the result. A bin missing a whole region that happens to carry few markers scores as complete. Two closely related strains binned together score as low contamination, because their single-copy genes are similar enough to be counted once. Neither number is a measurement of the thing it is named after; both are useful anyway, which is a sentence worth holding in mind rather than resolving.

22.6 Strains, and why they break everything

Almost every difficulty above gets sharper when a community contains several closely related populations of the same species, which is the normal case rather than the exception.

For assembly, related strains produce a graph with bubbles everywhere the strains differ. An assembler must either collapse them into a consensus that matches no actual organism, or fragment the assembly at every difference. Both happen, and which one happens is a property of the assembler’s parameters rather than of the biology.

For binning, strains share composition and covary in coverage, which is precisely the evidence binning uses. They are therefore grouped together by design, and the resulting bin is a chimera whose contamination estimate does not flag it, for the reason in the callout above.

For profiling, reads from a strain not in the database match its relatives ambiguously, and the multi-mapping problem of Section 16.3 arrives without the resolution that a single reference genome affords: there is no correct answer to assign the read to.

This is the field’s live frontier rather than a solved problem, and it is worth being clear about why. The species boundary is a convention — a percentage identity chosen because it is useful — and below it there is a continuum of population structure that the boundary was never meant to describe. Methods that resolve it exist, they disagree with each other, and their claims are strongest in the well-sampled environments where the reference genomes already are.

22.7 Contamination is part of the measurement

Every sequencing experiment in this book has a contamination problem. This one has it worst, because the contaminants are the same kind of thing as the signal.

Reagents are not sterile. DNA extraction kits, polymerases and water carry bacterial DNA, and it is reproducible enough that particular taxa recur across unrelated studies using the same kits (Salter et al. 2014). The critical property is that reagent contamination is roughly constant in absolute terms, so its relative contribution grows as the biomass of the real sample falls. In a low-biomass sample — a skin swab, a tissue biopsy, a deep-sea sample, most negative-result clinical specimens — it can dominate.

The host is often most of the library. A stool metagenome carries a modest host fraction; a tissue biopsy or a swab can be almost entirely host. Removing those reads is standard and is itself a source of bias, because the removal is done by mapping to a host reference and will also take real microbial reads from regions conserved enough to map.

Index hopping and cross-contamination between samples put reads from one library into another at low rates. That rate is negligible against a dominant organism and is not negligible against the rare tail, which is exactly where the interesting detections in a clinical or environmental study tend to be.

A negative control is not optional here

Sequencing a blank — extraction reagents and no sample, carried through the entire protocol — is the only way to know which taxa in your results came from the kit. It costs one library per batch and it is the difference between a finding and an artifact.

It has to be a processed blank, run alongside the samples in the same batch with the same reagents, because contamination varies between kit lots. A control from a different batch answers a different question.

22.8 What a functional profile does and does not say

The second question asked of a community is not who is there but what they can do, and the gap between those two verbs is the whole caution of this section.

Genes are found either by translating reads and matching them to reference protein families, or by predicting genes on assembled contigs and annotating those. Both produce a table of gene families or pathways with abundances, and both are describing genetic potential: the DNA was present in some cell in the sample. Nothing about a DNA measurement says the gene was transcribed, translated, or that its product did anything. A pathway reported as abundant may be carried by an organism that is dormant.

Two further limits are structural rather than technical.

Annotation is transfer, and transfer is an inference. A gene is annotated by resemblance to something already annotated, so the result is only as good as the reference and inherits every error in it. A large fraction of genes in any environmental sample resemble nothing characterized — the proportion is high enough that “unknown” is one of the largest categories in most functional profiles, and it is not distributed evenly across environments.

A pathway is not present because its genes are. Pathway-level summaries count genes mapped to a pathway definition, and a community-level total can be assembled from genes carried by organisms that never co-occur in a cell. That is why attributing function to the organisms contributing it — rather than reporting only the community total — is what turns a description into something that can be checked.

Everything in this section describes what could happen, and the measurements that describe what is happening are different experiments: sequencing RNA to see transcription, or proteins and metabolites to see the rest. They also have their own reference problems, which are harder. Functional potential is a real and useful measurement, and reading it as activity is the most common overstatement in this field’s literature.