Benchmarking in Taxonomy: A Commentary on the Role of the Holotype

1. Introduction

In its original usage 200 years ago a ‘benchmark’ was a surveying term that referred to a mark in the ground which enabled instruments to be re-established in their original location: it was a reliable point of reference, a ground truth location. Although this usage continues, it has evolved in the digital age to include processes seeking to optimize performance (Bartz-Beielstein, 2020, Moriarty & Smallman, 2014) and is widely used in science and technology. A principal application in the biological sciences has been in computational genomics where it refers to optimization of processes for decoding molecular data.

Considered as a point of reference, as in its original use, ‘benchmark’ is seldom used in Linnaean taxonomy either by practicing taxonomists or by philosophers of science and semantics yet, as identified by ICZN (1999), the holotype specimen appears to be a prime example. Examined here is its use by Brummer & Kucera (2022) in their review of the taxonomy of living planktonic foraminifera. They aim to benchmark species concepts, noting that the taxonomy should be robust and operational. Their usage of ‘benchmark’ is simple: for species, they confirm that a holotype specimen exists and that the name is not a subjective synonym (ICZN, 1999 art. 61). Otherwise expressed, they assess whether the species name been validly typified. While this is a strict interpretation of a benchmark specimen in Linnaean taxonomy, it overlooks its role in the recognition of such taxa, either as new species or as specimens of already named species. Several aspects of these roles are considered here.

2. Morphospecies: Their Recognition

In cognitive neuroscience ‘categorization’ (more generally ‘classification’) denotes a process whereby a group of objects is picked out by an individual’s mental representation (concept) of the group. It allows for determination of objects belonging to the group (Medin & Coley, 1998); visual similarity is a primary determinant of membership. Current research indicates that the mental representation may be based on exemplars (members) of the group already stored in visual memory (Smith, 2016; Ashby & Rosedahl, 2017), or on prototypes formed from summed similarities of exemplars. These processes are located separately in the brain but may operate in tandem (Blank & Bayer, 2022). Category learning is aided by the extent of similarity among exemplars and may be assisted by textual data (Hughes & Thomas, 2021). Importantly, all of these processes are personal to each observer.

In practice, building knowledge of a morphospecies begins with a taxonomist recognizing a new (=unnamed, i.e., no imagery recall) group within his collections or in the literature, depositing a specimen (holotype) in a recognized repository, and publishing a Linnaean name and description for it. This makes the species available in the public domain as a named taxon, as Brummer & Kucera (2022) showed. The holotype is the name bearer and as such qualifies as a benchmark, or equivalently as a voucher specimen in a machine learning application. That the species name is bound to that specimen creates a widely discussed problem in its use (Michaelson, 2024) and is a focus of this commentary.

How the author’s concept is to be promulgated is not addressed by ICZN (1999) and seldom is in the wider literature, yet is vital for its use as a referent. The following representation seeks to mirror conventional practice for referencing the name of a newly recognized taxon as a visual entity. In it ‘labelled’ is used for specimens, including images thereof, to which the Linnaean name is attached, say as a label on the specimen or on its mountant, or in captions for illustrations of the specimen. In this sense the holotype is the primary labelled specimen (PLS). Knowledge of the species is incremented and disseminated as other taxonomists view the deposited museum specimen, or its published image, and recognize similar specimens in their collections. They might isolate and label such specimens, subjectively determined (SLS), with that name for their reference or publish illustrations, thereby extending the set of labelled specimens. Some may have inspected the holotype (Figure 1) but commonly this is infeasible. Increasingly, available sources of visual knowledge of species are SLS specimens shown in print or digital publications. They form an essential guide for today’s taxonomists to develop their personal concept of a species and also serve as an important adjunct to the limited capacity of our short term memory (Brady et al., 2011).

For a given species, labelled specimens form an incremental record of its identification that expands out from its original publication. Ostensibly, the expansion is an historical record of specimens identified by later taxonomists labelled using the name of the holotype specimen: it is termed a labelled specimen knowledge base (LSKB). For a recently described species individual entries in the LSKB are readily traceable online and the history of usage is viewable. However, as taxonomists seldom record which prior labelled specimens they inspected when forming their concept of a species, for many taxa their LKSB resembles a virtual entity.

3. Discussion

3.1. Conventional Model

This model (CM) relies on the operations of our vision system for construction of a LSKB and inherits its defects for recognition of groups based on a primary labelled specimen. Nominally, a LSKB is an archive of images whose name is a homonym of the holotype: it uses the name of the holotype but references different specimens. This referencing problem is generally ignored by taxonomists (e.g., Braby 2024; Ruedas 2024) but is widely discussed in the philosophy of the natural sciences (e.g., Ereshefsky & Reydon, 2015 and their citations; Michaelson, 2024). Of relevance here is Crane (2021, p. 12195) who regards the holotype as a sample and argues that “One needs a particular explanatory purpose to pick out a particular population by a species baptism. If that is left open, the name may not latch onto any determinate population”. Although this is not an operational solution, it recognizes the need to relate a holotype to its source population and is consistent with the following model.

3.2. Population Model

This model (PM) seeks to remedy problems raised by the conventional model. It accepts that ‘species’ is a group concept for biological specimens recognized as similar tangible objects, a view that compares with Darwin (1861 p. 159) “To sum up, I believe that species come to be tolerably well-defined objects ...”. To expand this definition, I follow Simpson (1961, p. 65) that “... Populations, not individuals, are the units of systematics ...” and Beckner’s (1959) view that species are polytypic populations in space/time. The problem then for a species proposal is to identify a specimen that to serve as the primary benchmark for that population, or as a benchmark for a local named population. It leads to a quite different, probabilistic, search strategy wherein the opaque operations of the personal perception process of CM are supplemented by strategies that create probabilistic, contestable data about the position of the holotype or other labelled specimen in its source population. The schema of the population model used here follows these steps. 1. The taxonomist uses their visual perception to select specimens they provisionally identify as species A or similar thereto. 2. Morphometric data on a functional trait are gathered. 3. Statistical analyses provide probabilities of the selected specimens belonging to one population. 4. The taxonomist evaluates this statistical grouping against their knowledge of the functional significance of the trait. 5. A benchmark or other voucher specimens are selected, informed by the probability that they belong to Species A.

The trait in the examples of this study is the axial outline of planktonic foraminifera shells. Experimental neuroscience shows that, subliminally, the outline is an important guide to object recognition (Sprote et al, 2016; Elder, 2018). Here it captures most of a specimen’s ontogeny and is regarded as serving hydrodynamic and trophic functions. The normal model is preferred for analysis because it provides a suitable measure of central tendency, which might be associated with adaptation towards an optimum form in the shape trait.

The primary focus of PM is to demonstrate the probability that specimens in a sample are drawn from one population (Figure 2). Its value for accessing the relation of benchmark specimens to other populations is shown in Figure 3. Theoretically, PM might be viewed as an optimizing strategy for selection of representative specimens such as benchmarks. Nevertheless, the statistical data remain open to interpretation, biologically: the taxonomist retains discretionary judgment when, for example (Figure 2 #30), a specimen is flagged as unlikely to belong to the population when it is interpreted as a juvenile (Scott, 2025).

Operationally, PM requires that the population from which the holotype was sourced is analysed and that data from that analysis are publicly available to allow them to be plotted in an analysis of a local population that a taxonomist considers to represent the holotype population (e.g., Figure 3). This is a project that organizations like that represented by Brummer & Kucera (2022) might consider.

4. Conclusion

Brummer & Kucera (2022) offer an account of the application of benchmarking modern planktonic taxa that is limited to verifying the existence of a formally named specimen, its subjective synonyms and specification of diagnostic characters, but without imagery thereof. They do not address how an operational taxonomy is to be developed from these data.

ICZN (1999) rules relate only to the formation of a valid name for a species and the curation of one specimen, the holotype or benchmark specimen. They do not provide guidance on usage of the name of that specimen: its role as a referent. In conventional practice taxonomists identify individual specimens using their personal concept of a species, which is informed by their stored mental representations of previously viewed specimens, or their images, to which the name has been applied. Collectively, the named images form a database of referents, which, excepting the holotype, are subjective interpretations of the name and personal to each taxonomist’s species concept.

On the workbench, morphospecies comprise tangible specimens primarily grouped by their common visual attributes. Although conventional practice, reliant on the autonomous processes of visual memory, is highly efficient for specimen recognition and grouping of such objects, its mechanisms are cryptic, and outcomes are personal to the viewer. They do not provide objective data about the relation of a labelled specimen to the name bearing specimen (benchmark) and are unsuitable for species when viewed as polytypic populations. Contrarily, for the population model the taxonomic problem is to identify their populations and generate contestable, objective data about specimen relationships to facilitate selection of representative specimens.

Expectedly, because of its history of usage, efficiency and general acceptance, the conventional model will remain normative into the future and accusations of “stamp collecting” (Johnson, 2007) will continue. Acceptance of the population, or similar models, will be contingent on wider appreciation of the constraints imposed on recognition of a natural group when its name is bound to one specimen. Impeding wide usage of the population model for many cohorts is the present limited availability and cost of laboratory equipment for rapid specimen positioning and capture of 2-3D image data.