A Genomic Perspective on Cryptic Species Reveals Complex Evolutionary Dynamics in the Gray Zone of the Speciation Continuum

Abstract

The evolutionary dynamics of cryptic species remain poorly understood, and their detection relies primarily on methods that quantify divergence, assuming that gene flow is absent. Here, we examine how gene flow shapes the evolutionary trajectories and species boundaries in Bornean Fanged Frogs, a renowned example of cryptic diversity where a single species has been split into 18 genetically divergent yet morphologically indistinguishable species. We employed target-capture data from over 13,000 loci to assess lineage independence of 14 nominal species distributed across Malaysian Borneo by evaluating both divergence and cohesion using network multispecies coalescent (NMSC) and MSC + migration approaches. Under the Unified Species Concept, only 6 of the 14 nominal species unambiguously form independently evolving lineages; the remainder represent cohesive metapopulation lineages nested within those six species. While mitochondrial p-distances varied substantially (up to 10%), genome-wide net divergences (Da) were surprisingly consistent, ranging from 0.5 to 2%, placing all the hypothesized “cryptic species” within the empirical gray zone of the speciation continuum. We show that diversification in the gray zone is unpredictable and heavily impacted by gene flow, leading to two key phenomena that confound species delimitation: (1) the artifactual branch effect, where admixed lineages are inferred as long, early-diverging branches, creating an illusion of deep divergence; and (2) the species-definition anomaly zone, where intraspecific pairwise sequence distances exceed interspecific ones. We further demonstrate that divergence in the gray zone varies substantially among metrics and genomic regions, reflecting heterogeneity in evolutionary dynamics across the genome. Different genomic markers also vary considerably in phylogenetic discordance and their ability to retain signatures of gene flow. Loci from anchored hybrid enrichment and ultraconserved elements produced less phylogenetic discordance and retained signals of older introgression but failed to detect recent migration, making them suitable for phylogenetic reconstruction and inferring ancient introgression, but not ongoing gene flow. Recognizing the central role of gene flow reframes our understanding of cryptic species; rather than being considered as genetically distinct units that failed to evolve morphological differentiation, they can be manifestations of continuous diversification in the gray zone. This shift in perspective offers a new and dynamic evolutionary framework for identifying and interpreting cryptic biodiversity across the Tree of Life.