Mathematical Constraints on FST: Biallelic Markers in Arbitrarily Many Populations. Issue 3 (1st July 2017)

• Record Type: Journal Article
• Authors: Alcala, Nicolas; Rosenberg, Noah A
• Is Part Of: Genetics. Volume 206:Issue 3(2017)
• Page Start: 1581
• Page End: 1600
• Publication Date: 2017-07-01
• Subjects: allele frequency -- FST -- genetic differentiation -- migration -- population structure; Genetics -- Periodicals; 576.5
• DOI: 10.1534/genetics.116.199141
• Languages: English
• ISSNs: 0016-6731

Abstract

Abstract: F ST is one of the most widely used statistics in population genetics. Recent mathematical studies have identified constraints that challenge interpretations of F ST as a measure with potential to range from 0 for genetically similar populations to 1 for divergent populations. We generalize results obtained for population pairs to arbitrarily many populations, characterizing the mathematical relationship between F ST, the frequency M of the more frequent allele at a polymorphic biallelic marker, and the number of subpopulations K . We show that for fixed K, F ST has a peculiar constraint as a function of M, with a maximum of 1 only if M = i / K, for integers i with ⌈ K / 2 ⌉ ≤ i ≤ K − 1. For fixed M, as K grows large, the range of F ST becomes the closed or half-open unit interval. For fixed K, however, some M < ( K − 1 ) / K always exists at which the upper bound on F ST lies below 2 2 − 2 ≈ 0.8284. We use coalescent simulations to show that under weak migration, F ST depends strongly on M when K is small, but not when K is large. Finally, examining data on human genetic variation, we use our results to explain the generally smaller F ST values between pairs of continents relative to global F ST values. We discuss implications for the interpretation and use of F ST .