Transcript Alpha Diversity Indices

Alpha Diversity Indices
James A. Danoff-Burg
Dept. Ecol., Evol., & Envir. Biol.
Columbia University
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Diversity of Diversities
Difference between the diversities is usually one of
relative emphasis of two main envir. aspects
Two key features


Richness
Abundance – our emphasis today
Each index differs in the mathematical method of
relating these features


One is often given greater prominence than the other
Formulae significantly differ between indices
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Diversity Levels
Progress from local to regional levels

Point: diversity at a single point or microenvironment
• Our emphasis thus far

Alpha: within habitat diversity
• Usually consists of several subsamples in a habitat

Beta: species diversity along transects & gradients
• High Beta indicates number of spp increases rapidly with
additional sampling sites along the gradient


Gamma: diversity of a larger geographical unit (island)
Epsilon: regional diversity
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Q Statistic Introduction
A bridge between the abundance models &
diversity indices

Does not involve fitting a model
• as in the abundance models
Provides an indication of community diversity

No weighting towards very abundant or rare species
• They are excluded from the analysis
• Whittaker (1972) created earlier analysis including these
– Thereby more influenced by the few rare / abundant species

Proposed by Kempton & Taylor (1976)
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Q Statistic Visually
Measures “inter-quartile slope” on the cumulative
species abundance curve
S = 250
S/4 = 62.5
250
200
R2 = 187.5 = 0.75*S
1st = 62.5
2nd = 125
3rd = 187.5
Cumulative 150
Species
100
Q = slope
50
R1 = 62.5 = 0.25*S
0
10
100
1,000
10,000
Species Abundance
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Q Statistic
Relationship to other indices


Similar to the a value in the log series model
Q = (0.371)(S*) / s
Biases in Q

May be biased in small samples
• Because we are including more of the rare and abundant
species in the calculation
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Calculating Q
- Worked Example #6
Assemble table with 3 columns

# Individuals, # Species, Summed # species
Determine R1 and R2


R1 should be > or = 0.25 * S
R2 should be > or = 0.75 * S
Calculate Q

Q = [((nR1)/2) + Snr + ((nR2)/2)] / [ln(R2/R1)]
• nR1 and nR2 = # species in each quartile class
 Snr = total number of species between the quartiles
• R1 and R2 = # of individuals at each quartile break point
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
All based on proportional species abundances

Species abundance models have drawbacks
• Tedious and repetitive
• Problems if the data do not violate more than one model
– How to choose between?
Building upon the species abundance models

Allows for formal comparisons between sites /
treatments
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
“Heterogeneity Indices”


Consider both evenness AND richness
Species abundance models only consider evenness
No assumptions made about species abundance
distributions


Cause of distribution
Shape of curve
“Non-parametric”

Free of assumptions of normality
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Two General Categories
Information Theory (complicated computation)


Diversity (or information) of a natural system is similar to
info in a code or message
Examples: Shannon-Wiener and Brillouin Indices
Species Dominance Measures (simple comput.)



Weighted towards abundance of the commonest
species
Total species richness is downweighted relative to
evenness
Examples: Simpson, McIntosh, and Berger-Parker
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Simpson Index Values
Derived by Simpson (1949)
Basis


Probability of 2 individuals being conspecifics
If drawn randomly from an infinitely large community
Summarized by letter D, 1-D, or 1/D

D decreases with increasing diversity
• Can go from 1 – 30+
• Probability that two species are conspecifics  with  diversity

1-D and 1/D increases with increasing diversity
• 0.0 < 1-D < 1.0
• 0.0 < 1/D < 10+
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Simpson Index
Heavily weighted towards most abundant species



Less sensitive to changes in species richness
Once richness > 10  underlying species abundance is
important in determining the index value
Inappropriate for some models
• Log Series & Geometric

Best for Log-Normal
• Possibly Broken Stick
Log Series
10000
1000
100
Number of
Species 10
Lecture 4 – Alpha Diversity Indices
Log Normal
Series
Broken Stick
Series
D value 10
20
30
© 2003 Dr. James A. Danoff-Burg, [email protected]
Simpson Index
When would this weight towards most abundant
species be desired?

Not just when the abundance model fits the Log-Normal
Conservation implications of index use?
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Simpson Calculation –
Worked Example 9
Calculate N and S
Calculate D


D = S (ni(ni-1)) / (N(N-1)
Solve and then sum for all species in the sample
Calculate 1/D

Increases with increasing diversity
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
McIntosh Index
Proposed by McIntosh (1967)
Community is a point in an S dimensional
hypervolume whose Euclidean distance from the
origin is a measure of diversity


Paraphrased from Magurran
Origin is no diversity, distances from origin are more
diverse
Not strictly a dominance index

Needs conversion to dominance index
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
McIntosh Manipulations
Base calculations (U metric)

Strongly influenced by sample size
Conversion to a dominance measure (D)


Use Dm for our class
Makes value independent of sample size
Derive a simple evenness index using McIntosh

Most often used contribution of McIntosh
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
McIntosh Calculation – Worked
Example 10
Base calculations

U = (Sni2)
• ni = abundance of ith species
• Different from Magurran’s definition
Conversion to a dominance measure

Dm = (N-U) / (N-N)
Derive evenness index

Em = (N-U) / ((N-(N/S))
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Berger-Parker
Proposed by Berger and Parker (1970) and
developed by May (1975)
Simple calculation = d
Expresses proportional importance of most
abundant species
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Berger-Parker
Decreasing d values  increasing diversity

Often use 1 / d
• Increasing 1 / d  increasing diversity
• And reduction in dominance of one species
Independent of S, influenced by sample size

Comparability between sites if sampling efforts
standardized
Question may lead to use of Berger-Parker

Example: Change in dominant species in diet?
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Berger-Parker – An Example
Dominant species in flounder (Platichys flesus) diet
across an Irish estuary (Wirjoatmodjo 1980)
River Mouth #1
Intertidal #2
Intertidal #3
+ Sewage #4
Fresh & Hot #5
Nereis
394
1642
90
126
32
Corophium
3487
5681
320
17
0
Gammarus
275
196
180
115
0
Tubifex
683
1348
46
436
5
Chironomids
22
12
2
27
0
Insect larvae
1
0
0
0
0
Arachnid
0
1
0
0
0
Carcinus
4
48
1
3
0
Cragnon
6
21
0
1
13
Neomysis
8
1
0
0
9
Sphaeroma
1
5
2
0
0
Flounder
1
7
1
1
0
Other fish
2
3
5
0
4
d
0.714
0.634
0.495
0.601
0.508
1/d
1.4
1.58
2.02
1.67
1.96
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Berger-Parker Calculations –
Worked Example 11
Calculate N, S, Nmax
Calculate d and 1/d
Very simple
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
“Heterogeneity Indices”


Consider both evenness AND richness
Species abundance models only consider evenness
No assumptions made about species abundance
distributions


Cause of distribution
Shape of curve
“Non-parametric”

Free of assumptions of normality
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Two General Categories
Information Theory (complicated computation)


Diversity (or information) of a natural system is similar to
info in a code or message
Examples: Shannon-Wiener and Brillouin Indices
Species Dominance Measures (simple comput.)



Weighted towards abundance of the commonest
species
Total species richness is downweighted relative to
evenness
Examples: Simpson, McIntosh, and Berger-Parker
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Information Theory
Information Theory, described (read more here)

A system contains more information when it has many
possible states
• E.g., large numbers of species, or high species richness

Also contains more information when the probability of
encountering each state is high
• E.g., all species are equally abundant or have high evenness
Indices derived from this simple relationship
between richness and evenness

Examples
• Shannon-Wiener and Brillouin
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Shannon-Wiener Index
Derived by Claude Shannon and Warren Weaver
in late 40s


Developed a general model of communication and
information theory
Initially developed to separate noise from information
carrying signals
Subsequently

mathematician Norbert Wiener contributed to the model
as part of his work in developing cybernetic technology
Called alternatively Shannon-Weaver, ShannonWiener, or Shannon Index – more info here
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Shannon-Wiener
Assumptions



All individuals are randomly sampled
Population is indefinitely large, or effectively infinite
All species in the community are represented
Result: difficult to justify for many communities


Particularly very diverse communities, guilds, functional
groups
Incomplete sampling  significant error & bias
• Increasingly important as proportion of species sampled
declines
• Simple mathematical consequence – see next slide
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Shannon-Wiener Mathematics
Equation

H’ = -S pi ln pi
• pi = proportion of individuals found in the ith species
• Unknowable, estimated using ni / N
– Flawed estimation, need more sophisticated equation (2.18 in
Magurran)

Error
• Mostly from inadequate sampling
• Flawed estimate of pi is negligible in most instances from this
simple estimate
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Shannon-Wiener Mathematics
Need to convert data


Log2 was historically used
Any Log base is acceptable
• Need consistency across samples

Currently, Ln is used more commonly
• What we will use
Range of S-W index



Usually between 1.5 and 3.5
Rarely surpasses 4.5
If underlying distribution is log-normal
• Need 100,000 species to have a H’ > 5.0
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Building on H’
Can also use Exp H’

= Number of equally common species required to
produce a given H’ value
• Reduces S from the observed value
• Allows for an estimation of departures from maximal evenness
and diversity

We won’t explore this here
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Building on H’
Evenness measure (E)

Useful for determining the departure from maximal
evenness and diversity
• Similar to the Exp H’



Hmax = maximal diversity which could occur if all species
collected were equally abundant
E = H’ / Hmax = H’ / ln S
0<E<1
• H’ will always be less than Hmax


Assumes all species have been sampled
Some have criticized this as being biologically
unrealistic
• Argue for best fit to the Broken Stick model
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Comparing H’ Values
Using Shannon for a t-test


Can use a simple t-test for differences between two
samples
Need variance in H’ (Var H’) and to know the df
• Both have complicated equations (2.19, 2.21 in Magurran)
Shannon and ANOVA


H’ values tend to be normally distributed
Can use ANOVAs for differences between multiple sites
• Need to have real replication to do this
• Pseudoreplication introduces error, particularly in parametric
statistics
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Shannon-Wiener Calculation –
Worked Example 7
 Calculate proportion of individuals in each species (pi) and
ln pi
 Sum all (pi)(ln pi) values
 Calculate E

E = H’ / ln S
 Calculate Var H’

Var H’= ([S (pi)(ln pi)2 – S ((pi)(ln pi))2] / N) – ((S-1)/(2N2))
 Calculate t

t = (H’1 - H’2) / (Var H’1 + Var H’2)1/2
 Calculate df

df = (Var H’1 + Var H’2)2 / ([(Var H’1)2 / N1] + [(Var H’2)2 / N2])
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Brillouin Index
Useful when

The randomness of a sample is not guaranteed
• Light traps, baited traps, attractive traps in general

Community is completely (thoroughly) censused
• Similar to Shannon-Wiener index
Assumes


Community is completely sampled
Does not assume:
• Randomness of sampling
• Equal attractiveness of traps
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Brillouin Mathematics
HB


Rarely larger than 4.5
Ranges between 1 and 4 most commonly
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Brillouin vs. Shannon-Wiener
 Give similar values – significantly correlated
 Brillouin < Shannon-Wiener
 Brillouin has no uncertainty about all species present in
sample

Does not estimate those that were not sampled, as in Shannon
 When relative proportions of spp are consistent, totals
differ


Shannon stays constant
Brillouin will decrease with fewer total individuals
 Brillouin is more sensitive to overall sample size
 Collections are compared, not samples

Disallows statistical comparisons, as all collections are different
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Brillouin Mathematics
Uses factorials throughout
Equation

HB = (ln N! – S ln ni!) / N
Evenness

E = HB / HBmax
HBmax

HBmax = [(1/n)][(ln {((N!) / (((N/S)!)s-r)*((((N/S)+1)!)r)}]
r

r = N – S (N/S)
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Brillouin Calculations –
Worked Example 8
Calculate HB
Calculate r
Calculate HBmax
Calculate E
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Jack-Knifing Diversity Indices
Improves the accuracy of any estimate
First proposed in 1956 (Quenouille) and refined by
Tukey in 1958


Theoretical biostatisticians
First applied to diversity by Zahl in 1977
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Jack-Knifing
Assumptions:


None made about underlying distribution
Does not attempt to estimate actual number of species
present
• As in Shannon-Wiener
Random sampling is not necessary


Repeated measures overcome the biases
Jack-Knifing can determine the impact of biased
sampling
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Data for Jack-Knifing
Need multiple samples to conduct this procedure

Some debate exists about this, may be able to do a
single sample
For our data


Can use each tray  to create an estimate for what?
Can use each garden  to create an estimate for what?
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Jack-Knifing Procedure
Procedure





Create the overall pooled index estimate (
Subsamples with replacement from the actual data
Creates pseudovalues of the statistic
Pseudovalues are normally distributed about the mean
Mean value is best estimate of the statistics
Confidence limits


Also possible to attach these to the estimate
Consequence of normal distribution of pseudovalues
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Applications of the Jack-Knife
Most commonly used for the most common
indices


Shannon and Simpson in particular
Also useful for other indices
Variance in the pseudovalues


More useful than the Var H’ of the Shannon
Gives a better estimate of the accuracy and impact of
non-random sampling
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Jack-Knifing –
Worked Example 12
Overall diversity index including all data (V)
Recalculate, excluding each sample in turn

Creates n number of VJi estimates
Convert VJi to pseudovalues VPi


Use VPi = (nV) – [(n-1) (VJi)]
n = number of samples
Calculate mean VP value
Calculate Sample Influence Function

SIF = V – VP
Calculate standard error VP = stand dev Vpis / n
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Alpha Diversity Indices
Q-Statistic
Intro to Alpha Diversity Indices
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
Jack-Knifing Diversity Indices
Pielou’s Hierarchical Diversity Index
Lecture 4 – Alpha Diversity Indices
Week 1
Week 2
© 2003 Dr. James A. Danoff-Burg, [email protected]
Pielou’s Pooled Quadrat
Method
Similar to Jack-Knifing

Improves the estimate of diversity
Also not influenced by non-random sampling

Provides the best estimate of the value, given the data
Can be calculated using either of the information
statistic indexes


Shannon
Brillouin
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Pielou’s Pooled Quadrat
Outputs


A graph that levels off when diversity has been best
estimated in the community (Hpop)
Determine the minimal number of samples to achieve
maximal diversity (t)
3
Diversity
Hpop
2
1
t
0
Quadrats
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Pielou’s Method Utility
Stability of an index

Evaluating the stability of a diversity index and its
relationship to sample size
Determining an adequate sample size


Produces a graph of the indices
When the line levels out, you have adequate samples
• Adequately estimated biodiversity locally
Can create confidence limits


Then, can compare values between habitats
Use standard parametric statistics
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Pielou’s Pooled Quadrat –
Worked Example 13
Using Brillouin index, calculate all HBk




From k = 0  k = z
k = number of samples
z = total samples
Mk = total abundance in k number of samples
Estimate t

t = Point at which HBk levels off
Calculate Hpop



Using k+t number of samples
Calculate mean Hpop
Calculate standard deviation
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Next week(s):
Continuing Alpha Diversity Indices
Read


Magurran Ch 2, pages 32-45
Magurran Worked Examples 6-13
We will continue conducting alpha diversity
analyses next week
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]
Hypothetical Model Curves
100
10
Broken Stick Model
1
Per
Species
Abundance 0.1
Log-Normal Series
0.01
Log Series
0.001
Geometric Series
10
20
30
40
Species Addition Sequence
Lecture 4 – Alpha Diversity Indices
© 2003 Dr. James A. Danoff-Burg, [email protected]