Ryeland et al (2017) studied the roosting behaviour of several species of shorebirds. They recorded the proportion of time (number of minutes as a proportion of total minutes in a video bout) individuals of various species spent in the backrest position while roosting. They used a binomial model with a logit link for proportions with four fixed predictors recorded for each video bout: ambient temperature, wind speed, size of group focal bird was in, and distance focal bird was from the observer. We will analyze the data for a single species, the sharp-tailed sandpiper (Calidris acuminata). This would be a standard binomial GLM except that more than one bird was sometimes recorded in each bout so bout was included as a random effect since birds closer together may be correlated in their behaviour. The resulting model is a binomial GLMM.

Sharp-tailed sandpiper. patrickkavanagh, , via Wikimedia Commons

The paper is here

Ryeland, J., Weston, M. A., Symonds, M. R. E. & Overgaard, J. (2017). Bill size mediates behavioural thermoregulation in birds. Functional Ecology, 31, 885-93.

Preliminaries

First, load the required packages (car, performance, MuMIn, lme4, glmmTMB, lmtest, ggplot2)

Import ryeland data file (ryeland.csv)

ryeland <- read.csv("../data/ryeland.csv")
head(ryeland)

Create proportion of time facing back

ryeland$prophb<-ryeland$timehb/ryeland$filmp

Create success and fail columns

ryeland$success<-as.integer(ryeland$timehb)
ryeland$fail<-as.integer(ryeland$filmp-ryeland$timehb)

Create response variable

ryeland.prop<-cbind(ryeland$success,ryeland$fail)

Preliminary checks

Scatterplots

plot(prophb~wind, data=ryeland)

plot(prophb~dist, data=ryeland)

plot(prophb~groupsize, data=ryeland)

plot(prophb~temp, data=ryeland)

Boxplots

boxplot(ryeland$dist)

boxplot(log10(ryeland$dist))

boxplot(ryeland$groupsize)

boxplot(ryeland$temp)

boxplot(ryeland$wind)

boxplot(log10(ryeland$wind))

Check collinearity

vif(lm(ryeland$prophb ~ dist+groupsize+temp+wind,data=ryeland))
     dist groupsize      temp      wind 
     1.23      2.03      1.85      1.16 
cor(ryeland[,c('dist','groupsize','temp','wind')])
             dist groupsize    temp   wind
dist       1.0000    -0.203  0.0162 -0.274
groupsize -0.2030     1.000  0.6525 -0.157
temp       0.0162     0.652  1.0000 -0.047
wind      -0.2738    -0.157 -0.0470  1.000

Fit GLMM with binomial distribution and test predictor effects

ryeland1.glmer<-glmer(ryeland.prop~dist+groupsize+temp+wind+(1|vbout),data=ryeland,family=binomial)
summary(ryeland1.glmer)
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
 Family: binomial  ( logit )
Formula: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout)
   Data: ryeland

     AIC      BIC   logLik deviance df.resid 
     224      234     -106      212       36 

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-3.529 -0.697 -0.221  0.408  3.338 

Random effects:
 Groups Name        Variance Std.Dev.
 vbout  (Intercept) 1.43     1.19    
Number of obs: 42, groups:  vbout, 21

Fixed effects:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  4.46783    1.23192    3.63  0.00029 ***
dist         0.00364    0.00770    0.47  0.63666    
groupsize    0.00316    0.01311    0.24  0.80938    
temp        -0.24929    0.05531   -4.51  6.6e-06 ***
wind        -0.03106    0.09391   -0.33  0.74084    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
          (Intr) dist   gropsz temp  
dist      -0.375                     
groupsize -0.155  0.453              
temp      -0.551 -0.362 -0.597       
wind      -0.395  0.387  0.345 -0.200
AIC(ryeland1.glmer)
[1] 224
AICc(ryeland1.glmer)
[1] 226
anova(ryeland1.glmer, type="lrtest")
Warning: additional arguments ignored: ‘type’
Analysis of Variance Table
          npar Sum Sq Mean Sq F value
dist         1   0.08    0.08    0.08
groupsize    1   8.33    8.33    8.33
temp         1  22.00   22.00   22.00
wind         1   0.11    0.11    0.11

Test wind

ryeland2.glmer<-glmer(ryeland.prop~dist+groupsize+temp+(1|vbout),data=ryeland,family=binomial)
lrtest(ryeland1.glmer, ryeland2.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout)
Model 2: ryeland.prop ~ dist + groupsize + temp + (1 | vbout)
  #Df LogLik Df Chisq Pr(>Chisq)
1   6   -106                    
2   5   -106 -1  0.11       0.74

Test temp

ryeland3.glmer<-glmer(ryeland.prop~dist+groupsize+wind+(1|vbout),data=ryeland,family=binomial)
lrtest(ryeland1.glmer, ryeland3.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout)
Model 2: ryeland.prop ~ dist + groupsize + wind + (1 | vbout)
  #Df LogLik Df Chisq Pr(>Chisq)    
1   6   -106                        
2   5   -115 -1  17.4      3e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Test groupsize

ryeland4.glmer<-glmer(ryeland.prop~dist+temp+wind+(1|vbout),data=ryeland,family=binomial)
lrtest(ryeland1.glmer, ryeland4.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout)
Model 2: ryeland.prop ~ dist + temp + wind + (1 | vbout)
  #Df LogLik Df Chisq Pr(>Chisq)
1   6   -106                    
2   5   -106 -1  0.06       0.81

Test dist

ryeland5.glmer<-glmer(ryeland.prop~groupsize+wind+temp+wind+(1|vbout),data=ryeland,family=binomial)
lrtest(ryeland1.glmer, ryeland5.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout)
Model 2: ryeland.prop ~ groupsize + wind + temp + wind + (1 | vbout)
  #Df LogLik Df Chisq Pr(>Chisq)
1   6   -106                    
2   5   -106 -1  0.22       0.64

Check residuals

plot(ryeland1.glmer, resid(., type = "deviance") ~ fitted(.))

residuals(ryeland1.glmer, type="deviance")
      1       2       3       4       5       6       7       8       9      10      11      12      13      14      15      16      17      18      19      20      21      22 
-3.7634  2.8380  0.8204 -0.6598 -0.5124 -0.5124  3.8535 -3.4811 -1.7306  2.0263  2.7415 -1.5641  0.2951  0.1486 -0.9874 -0.9874  0.0823 -0.4955  0.4147 -1.1464  2.3104 -1.8960 
     23      24      25      26      27      28      29      30      31      32      33      34      35      36      37      38      39      40      41      42 
-0.5897 -0.5897 -1.1729  1.3381  0.0043  0.0043 -0.2598 -0.2598 -0.3127 -0.3127 -0.2564 -0.2564 -1.8043  2.4378 -0.3901  0.5396  0.4258 -1.1398 -0.6372 -0.6372 

Check overdispersion

(code from Zuur et al 2013)

presid1 <- resid(ryeland1.glmer, type="pearson")
ssize1 <- nrow(ryeland)
params1 <- length(fixef(ryeland1.glmer)+1)
disp1 <- sum(presid1^2)/(ssize1-params1)
disp1
[1] 2.23

Add Observation Level Random Effect into model and retest the predictors

ryeland6.glmer<-glmer(ryeland.prop~dist+groupsize+temp+wind+(1|vbout)+(1|olre),data=ryeland,family=binomial)
Warning: Model failed to converge with max|grad| = 0.00479232 (tol = 0.002, component 1)
summary(ryeland6.glmer)
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
 Family: binomial  ( logit )
Formula: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout) +      (1 | olre)
   Data: ryeland

     AIC      BIC   logLik deviance df.resid 
     176      188      -81      162       35 

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-1.129 -0.365 -0.110  0.146  0.806 

Random effects:
 Groups Name        Variance Std.Dev.
 olre   (Intercept) 2.282    1.511   
 vbout  (Intercept) 0.653    0.808   
Number of obs: 42, groups:  olre, 42; vbout, 21

Fixed effects:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  5.03538    1.39144    3.62   0.0003 ***
dist         0.00488    0.00859    0.57   0.5695    
groupsize    0.00664    0.01469    0.45   0.6511    
temp        -0.28931    0.06361   -4.55  5.4e-06 ***
wind        -0.02193    0.10482   -0.21   0.8343    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
          (Intr) dist   gropsz temp  
dist      -0.355                     
groupsize -0.125  0.448              
temp      -0.567 -0.368 -0.607       
wind      -0.387  0.391  0.344 -0.203
optimizer (Nelder_Mead) convergence code: 0 (OK)
Model failed to converge with max|grad| = 0.00479232 (tol = 0.002, component 1)
AIC(ryeland6.glmer)
[1] 176
AICc(ryeland6.glmer)
[1] 179

Test wind

ryeland7.glmer<-glmer(ryeland.prop~dist+groupsize+temp+(1|vbout)+(1|olre),data=ryeland,family=binomial)
lrtest(ryeland6.glmer, ryeland7.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout) + 
    (1 | olre)
Model 2: ryeland.prop ~ dist + groupsize + temp + (1 | vbout) + (1 | olre)
  #Df LogLik Df Chisq Pr(>Chisq)
1   7    -81                    
2   6    -81 -1  0.04       0.83

Test temp

ryeland8.glmer<-glmer(ryeland.prop~dist+groupsize+wind+(1|vbout)+(1|olre),data=ryeland,family=binomial)
lrtest(ryeland6.glmer, ryeland8.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout) + 
    (1 | olre)
Model 2: ryeland.prop ~ dist + groupsize + wind + (1 | vbout) + (1 | olre)
  #Df LogLik Df Chisq Pr(>Chisq)    
1   7  -81.0                        
2   6  -90.1 -1  18.2      2e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Test groupsize

ryeland9.glmer<-glmer(ryeland.prop~dist+temp+wind+(1|vbout)+(1|olre),data=ryeland,family=binomial)
lrtest(ryeland6.glmer, ryeland9.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout) + 
    (1 | olre)
Model 2: ryeland.prop ~ dist + temp + wind + (1 | vbout) + (1 | olre)
  #Df LogLik Df Chisq Pr(>Chisq)
1   7  -81.0                    
2   6  -81.1 -1   0.2       0.65

Test dist

ryeland10.glmer<-glmer(ryeland.prop~groupsize+wind+temp+wind+(1|vbout)+(1|olre),data=ryeland,family=binomial)
lrtest(ryeland6.glmer, ryeland10.glmer)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout) + 
    (1 | olre)
Model 2: ryeland.prop ~ groupsize + wind + temp + wind + (1 | vbout) + 
    (1 | olre)
  #Df LogLik Df Chisq Pr(>Chisq)
1   7  -81.0                    
2   6  -81.1 -1  0.32       0.57

Check residuals

residuals(ryeland6.glmer, type="deviance")
        1         2         3         4         5         6         7         8         9        10        11        12        13        14        15        16        17        18 
-1.544739  0.517286  0.119681 -0.032263 -0.468487 -0.468487  0.532565 -0.413694 -0.252082  0.424475  0.888607  0.099123  0.205676  0.194426 -0.863413 -0.863413 -0.092399 -0.205755 
       19        20        21        22        23        24        25        26        27        28        29        30        31        32        33        34        35        36 
 0.129297 -0.810856  0.465602  0.081393 -0.519161 -0.519161 -0.556220  0.670993  0.008321  0.008321 -0.193177 -0.193177 -0.229764 -0.229764 -0.180583 -0.180583  0.021693  0.501009 
       37        38        39        40        41        42 
 0.000936  0.219131  0.147782 -0.798193 -0.502843 -0.502843 
plot(ryeland6.glmer, resid(., type = "deviance") ~ fitted(.))

Fit model with only temp to match paper

ryeland11.glmer<-glmer(ryeland.prop~temp+(1|vbout)+(1|olre),data=ryeland,family=binomial)
summary(ryeland11.glmer)
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
 Family: binomial  ( logit )
Formula: ryeland.prop ~ temp + (1 | vbout) + (1 | olre)
   Data: ryeland

     AIC      BIC   logLik deviance df.resid 
   170.6    177.5    -81.3    162.6       38 

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-1.128 -0.350 -0.108  0.163  0.812 

Random effects:
 Groups Name        Variance Std.Dev.
 olre   (Intercept) 2.264    1.505   
 vbout  (Intercept) 0.781    0.884   
Number of obs: 42, groups:  olre, 42; vbout, 21

Fixed effects:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)   5.1380     1.2625    4.07  4.7e-05 ***
temp         -0.2701     0.0508   -5.31  1.1e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
     (Intr)
temp -0.953
AIC(ryeland11.glmer)
[1] 171
AICc(ryeland11.glmer)
[1] 172

Plot temp to match paper Need to add fit logistic curve?

ggplot(ryeland, aes(temp, prophb)) + geom_point() +
  geom_smooth(method = "loess", span = 0.5, se = FALSE) 

Rerun original GLMM with glmmTMB

ryeland.glmmb <- glmmTMB(ryeland.prop~groupsize+wind+temp+wind+(1|vbout), family=binomial, data=ryeland)
summary(ryeland.glmmb)
 Family: binomial  ( logit )
Formula:          ryeland.prop ~ groupsize + wind + temp + wind + (1 | vbout)
Data: ryeland

     AIC      BIC   logLik deviance df.resid 
     222      231     -106      212       37 

Random effects:

Conditional model:
 Groups Name        Variance Std.Dev.
 vbout  (Intercept) 1.45     1.2     
Number of obs: 42, groups:  vbout, 21

Conditional model:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  4.69865    1.14625    4.10  4.1e-05 ***
groupsize    0.00034    0.01174    0.03     0.98    
wind        -0.04793    0.08722   -0.55     0.58    
temp        -0.24056    0.05164   -4.66  3.2e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Now fit betabinomial to compare to OLRE

ryeland.glmmbb <- glmmTMB(ryeland.prop~dist+groupsize+temp+wind+(1|vbout),data=ryeland,family=betabinomial(link="logit"))
summary(ryeland.glmmbb)
 Family: betabinomial  ( logit )
Formula:          ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout)
Data: ryeland

     AIC      BIC   logLik deviance df.resid 
   178.7    190.8    -82.3    164.7       35 

Random effects:

Conditional model:
 Groups Name        Variance Std.Dev.
 vbout  (Intercept) 0.124    0.353   
Number of obs: 42, groups:  vbout, 21

Dispersion parameter for betabinomial family ():  2.6 

Conditional model:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  3.39744    1.02782    3.31  0.00095 ***
dist         0.00215    0.00601    0.36  0.72045    
groupsize    0.00685    0.00984    0.70  0.48647    
temp        -0.19311    0.04846   -3.98  6.8e-05 ***
wind        -0.01632    0.06696   -0.24  0.80738    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
AICc(ryeland.glmmbb)
[1] 182

Test temperature

ryeland1.glmmbb <- glmmTMB(ryeland.prop~dist+groupsize+wind+(1|vbout),data=ryeland,family=betabinomial(link="logit"))
lrtest(ryeland.glmmbb, ryeland1.glmmbb)
Likelihood ratio test

Model 1: ryeland.prop ~ dist + groupsize + temp + wind + (1 | vbout)
Model 2: ryeland.prop ~ dist + groupsize + wind + (1 | vbout)
  #Df LogLik Df Chisq Pr(>Chisq)    
1   7  -82.3                        
2   6  -91.9 -1  19.1    1.2e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
LS0tCnRpdGxlOiAiUSAmIEsgQm94IDEzLjgiCm91dHB1dDoKICBodG1sX25vdGVib29rCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkKYGBgCgpSeWVsYW5kIGV0IGFsICgyMDE3KSBzdHVkaWVkIHRoZSByb29zdGluZyBiZWhhdmlvdXIgb2Ygc2V2ZXJhbCBzcGVjaWVzIG9mIHNob3JlYmlyZHMuIFRoZXkgcmVjb3JkZWQgdGhlIHByb3BvcnRpb24gb2YgdGltZSAobnVtYmVyIG9mIG1pbnV0ZXMgYXMgYSBwcm9wb3J0aW9uIG9mIHRvdGFsIG1pbnV0ZXMgaW4gYSB2aWRlbyBib3V0KSBpbmRpdmlkdWFscyBvZiB2YXJpb3VzIHNwZWNpZXMgc3BlbnQgaW4gdGhlIGJhY2tyZXN0IHBvc2l0aW9uIHdoaWxlIHJvb3N0aW5nLiBUaGV5IHVzZWQgYSBiaW5vbWlhbCBtb2RlbCB3aXRoIGEgbG9naXQgbGluayBmb3IgcHJvcG9ydGlvbnMgd2l0aCBmb3VyIGZpeGVkIHByZWRpY3RvcnMgcmVjb3JkZWQgZm9yIGVhY2ggdmlkZW8gYm91dDogYW1iaWVudCB0ZW1wZXJhdHVyZSwgd2luZCBzcGVlZCwgc2l6ZSBvZiBncm91cCBmb2NhbCBiaXJkIHdhcyBpbiwgYW5kIGRpc3RhbmNlIGZvY2FsIGJpcmQgd2FzIGZyb20gdGhlIG9ic2VydmVyLiBXZSB3aWxsIGFuYWx5emUgdGhlIGRhdGEgZm9yIGEgc2luZ2xlIHNwZWNpZXMsIHRoZSBzaGFycC10YWlsZWQgc2FuZHBpcGVyICgqQ2FsaWRyaXMgYWN1bWluYXRhKikuIFRoaXMgd291bGQgYmUgYSBzdGFuZGFyZCBiaW5vbWlhbCBHTE0gZXhjZXB0IHRoYXQgbW9yZSB0aGFuIG9uZSBiaXJkIHdhcyBzb21ldGltZXMgcmVjb3JkZWQgaW4gZWFjaCBib3V0IHNvIGJvdXQgd2FzIGluY2x1ZGVkIGFzIGEgcmFuZG9tIGVmZmVjdCBzaW5jZSBiaXJkcyBjbG9zZXIgdG9nZXRoZXIgbWF5IGJlIGNvcnJlbGF0ZWQgaW4gdGhlaXIgYmVoYXZpb3VyLiBUaGUgcmVzdWx0aW5nIG1vZGVsIGlzIGEgYmlub21pYWwgR0xNTS4KClshW10oaW1hZ2VzL1NoYXJwLXRhaWxlZF9TYW5kaXBlcl8oQ2FsaWRyaXNfYWN1bWluYXRhKV8oNDA3MzM1ODcxNzQpLmpwZyldKGh0dHBzOi8vY29tbW9ucy53aWtpbWVkaWEub3JnL3dpa2kvRmlsZTpTaGFycC10YWlsZWRfU2FuZGlwZXJfKENhbGlkcmlzX2FjdW1pbmF0YSlfKDQwNzMzNTg3MTc0KS5qcGcpCgpTaGFycC10YWlsZWQgc2FuZHBpcGVyLiBwYXRyaWNra2F2YW5hZ2gsIFshW10oaW1hZ2VzL2J5LTAxLnBuZyl7d2lkdGg9IjU3In1dKGh0dHBzOi8vY3JlYXRpdmVjb21tb25zLm9yZy9saWNlbnNlcy9ieS80LjApLCB2aWEgV2lraW1lZGlhIENvbW1vbnMKClRoZSBwYXBlciBpcyBbaGVyZV0oaHR0cHM6Ly9kb2kub3JnLzEwLjExMTEvMTM2NS0yNDM1LjEyODE0KQoKUnllbGFuZCwgSi4sIFdlc3RvbiwgTS4gQS4sIFN5bW9uZHMsIE0uIFIuIEUuICYgT3ZlcmdhYXJkLCBKLiAoMjAxNykuIEJpbGwgc2l6ZSBtZWRpYXRlcyBiZWhhdmlvdXJhbCB0aGVybW9yZWd1bGF0aW9uIGluIGJpcmRzLiAqRnVuY3Rpb25hbCBFY29sb2d5KiwgMzEsIDg4NS05My4KCiMjIyBQcmVsaW1pbmFyaWVzCgpGaXJzdCwgbG9hZCB0aGUgcmVxdWlyZWQgcGFja2FnZXMgKGNhciwgcGVyZm9ybWFuY2UsIE11TUluLCBsbWU0LCBnbG1tVE1CLCBsbXRlc3QsIGdncGxvdDIpCgpgYGB7ciBpbmNsdWRlPUZBTFNFLCByZXN1bHRzPSdoaWRlJ30Kc291cmNlKCIuLi9SL2xpYnJhcmllcy5SIikgICAjVGhpcyBpcyB0aGUgY29tbW9uIGxpYnJhcnkKbGlicmFyeShwZXJmb3JtYW5jZSkKbGlicmFyeShnbG1tVE1CKQpgYGAKCkltcG9ydCByeWVsYW5kIGRhdGEgZmlsZSAoW3J5ZWxhbmQuY3N2XSguLi9kYXRhL3J5ZWxhbmQuY3N2KSkKCmBgYHtyfQpyeWVsYW5kIDwtIHJlYWQuY3N2KCIuLi9kYXRhL3J5ZWxhbmQuY3N2IikKaGVhZChyeWVsYW5kKQpgYGAKCkNyZWF0ZSBwcm9wb3J0aW9uIG9mIHRpbWUgZmFjaW5nIGJhY2sKCmBgYHtyIH0KcnllbGFuZCRwcm9waGI8LXJ5ZWxhbmQkdGltZWhiL3J5ZWxhbmQkZmlsbXAKYGBgCgpDcmVhdGUgc3VjY2VzcyBhbmQgZmFpbCBjb2x1bW5zCgpgYGB7ciB9CnJ5ZWxhbmQkc3VjY2VzczwtYXMuaW50ZWdlcihyeWVsYW5kJHRpbWVoYikKcnllbGFuZCRmYWlsPC1hcy5pbnRlZ2VyKHJ5ZWxhbmQkZmlsbXAtcnllbGFuZCR0aW1laGIpCmBgYAoKQ3JlYXRlIHJlc3BvbnNlIHZhcmlhYmxlCgpgYGB7ciB9CnJ5ZWxhbmQucHJvcDwtY2JpbmQocnllbGFuZCRzdWNjZXNzLHJ5ZWxhbmQkZmFpbCkKYGBgCgojIyMgUHJlbGltaW5hcnkgY2hlY2tzCgpTY2F0dGVycGxvdHMKCmBgYHtyIH0KcGxvdChwcm9waGJ+d2luZCwgZGF0YT1yeWVsYW5kKQpwbG90KHByb3BoYn5kaXN0LCBkYXRhPXJ5ZWxhbmQpCnBsb3QocHJvcGhifmdyb3Vwc2l6ZSwgZGF0YT1yeWVsYW5kKQpwbG90KHByb3BoYn50ZW1wLCBkYXRhPXJ5ZWxhbmQpCmBgYAoKQm94cGxvdHMKCmBgYHtyIH0KYm94cGxvdChyeWVsYW5kJGRpc3QpCmJveHBsb3QobG9nMTAocnllbGFuZCRkaXN0KSkKYm94cGxvdChyeWVsYW5kJGdyb3Vwc2l6ZSkKYm94cGxvdChyeWVsYW5kJHRlbXApCmJveHBsb3QocnllbGFuZCR3aW5kKQpib3hwbG90KGxvZzEwKHJ5ZWxhbmQkd2luZCkpCmBgYAoKQ2hlY2sgY29sbGluZWFyaXR5CgpgYGB7ciB9CnZpZihsbShyeWVsYW5kJHByb3BoYiB+IGRpc3QrZ3JvdXBzaXplK3RlbXArd2luZCxkYXRhPXJ5ZWxhbmQpKQpjb3IocnllbGFuZFssYygnZGlzdCcsJ2dyb3Vwc2l6ZScsJ3RlbXAnLCd3aW5kJyldKQpgYGAKCiMjIyBGaXQgR0xNTSB3aXRoIGJpbm9taWFsIGRpc3RyaWJ1dGlvbiBhbmQgdGVzdCBwcmVkaWN0b3IgZWZmZWN0cwoKYGBge3IgfQpyeWVsYW5kMS5nbG1lcjwtZ2xtZXIocnllbGFuZC5wcm9wfmRpc3QrZ3JvdXBzaXplK3RlbXArd2luZCsoMXx2Ym91dCksZGF0YT1yeWVsYW5kLGZhbWlseT1iaW5vbWlhbCkKc3VtbWFyeShyeWVsYW5kMS5nbG1lcikKQUlDKHJ5ZWxhbmQxLmdsbWVyKQpBSUNjKHJ5ZWxhbmQxLmdsbWVyKQphbm92YShyeWVsYW5kMS5nbG1lciwgdHlwZT0ibHJ0ZXN0IikKYGBgCgpUZXN0IHdpbmQKCmBgYHtyIH0KcnllbGFuZDIuZ2xtZXI8LWdsbWVyKHJ5ZWxhbmQucHJvcH5kaXN0K2dyb3Vwc2l6ZSt0ZW1wKygxfHZib3V0KSxkYXRhPXJ5ZWxhbmQsZmFtaWx5PWJpbm9taWFsKQpscnRlc3QocnllbGFuZDEuZ2xtZXIsIHJ5ZWxhbmQyLmdsbWVyKQpgYGAKClRlc3QgdGVtcAoKYGBge3IgfQpyeWVsYW5kMy5nbG1lcjwtZ2xtZXIocnllbGFuZC5wcm9wfmRpc3QrZ3JvdXBzaXplK3dpbmQrKDF8dmJvdXQpLGRhdGE9cnllbGFuZCxmYW1pbHk9Ymlub21pYWwpCmxydGVzdChyeWVsYW5kMS5nbG1lciwgcnllbGFuZDMuZ2xtZXIpCmBgYAoKVGVzdCBncm91cHNpemUKCmBgYHtyIH0KcnllbGFuZDQuZ2xtZXI8LWdsbWVyKHJ5ZWxhbmQucHJvcH5kaXN0K3RlbXArd2luZCsoMXx2Ym91dCksZGF0YT1yeWVsYW5kLGZhbWlseT1iaW5vbWlhbCkKbHJ0ZXN0KHJ5ZWxhbmQxLmdsbWVyLCByeWVsYW5kNC5nbG1lcikKYGBgCgpUZXN0IGRpc3QKCmBgYHtyIH0KcnllbGFuZDUuZ2xtZXI8LWdsbWVyKHJ5ZWxhbmQucHJvcH5ncm91cHNpemUrd2luZCt0ZW1wK3dpbmQrKDF8dmJvdXQpLGRhdGE9cnllbGFuZCxmYW1pbHk9Ymlub21pYWwpCmxydGVzdChyeWVsYW5kMS5nbG1lciwgcnllbGFuZDUuZ2xtZXIpCmBgYAoKQ2hlY2sgcmVzaWR1YWxzCgpgYGB7ciB9CnBsb3QocnllbGFuZDEuZ2xtZXIsIHJlc2lkKC4sIHR5cGUgPSAiZGV2aWFuY2UiKSB+IGZpdHRlZCguKSkKcmVzaWR1YWxzKHJ5ZWxhbmQxLmdsbWVyLCB0eXBlPSJkZXZpYW5jZSIpCmBgYAoKIyMjIENoZWNrIG92ZXJkaXNwZXJzaW9uCgooY29kZSBmcm9tIFp1dXIgZXQgYWwgMjAxMykKCmBgYHtyIH0KcHJlc2lkMSA8LSByZXNpZChyeWVsYW5kMS5nbG1lciwgdHlwZT0icGVhcnNvbiIpCnNzaXplMSA8LSBucm93KHJ5ZWxhbmQpCnBhcmFtczEgPC0gbGVuZ3RoKGZpeGVmKHJ5ZWxhbmQxLmdsbWVyKSsxKQpkaXNwMSA8LSBzdW0ocHJlc2lkMV4yKS8oc3NpemUxLXBhcmFtczEpCmRpc3AxCmBgYAoKIyMjIEFkZCBPYnNlcnZhdGlvbiBMZXZlbCBSYW5kb20gRWZmZWN0IGludG8gbW9kZWwgYW5kIHJldGVzdCB0aGUgcHJlZGljdG9ycwoKYGBge3IgfQpyeWVsYW5kNi5nbG1lcjwtZ2xtZXIocnllbGFuZC5wcm9wfmRpc3QrZ3JvdXBzaXplK3RlbXArd2luZCsoMXx2Ym91dCkrKDF8b2xyZSksZGF0YT1yeWVsYW5kLGZhbWlseT1iaW5vbWlhbCkKc3VtbWFyeShyeWVsYW5kNi5nbG1lcikKQUlDKHJ5ZWxhbmQ2LmdsbWVyKQpBSUNjKHJ5ZWxhbmQ2LmdsbWVyKQpgYGAKClRlc3Qgd2luZAoKYGBge3IgfQpyeWVsYW5kNy5nbG1lcjwtZ2xtZXIocnllbGFuZC5wcm9wfmRpc3QrZ3JvdXBzaXplK3RlbXArKDF8dmJvdXQpKygxfG9scmUpLGRhdGE9cnllbGFuZCxmYW1pbHk9Ymlub21pYWwpCmxydGVzdChyeWVsYW5kNi5nbG1lciwgcnllbGFuZDcuZ2xtZXIpCmBgYAoKVGVzdCB0ZW1wCgpgYGB7ciB9CnJ5ZWxhbmQ4LmdsbWVyPC1nbG1lcihyeWVsYW5kLnByb3B+ZGlzdCtncm91cHNpemUrd2luZCsoMXx2Ym91dCkrKDF8b2xyZSksZGF0YT1yeWVsYW5kLGZhbWlseT1iaW5vbWlhbCkKbHJ0ZXN0KHJ5ZWxhbmQ2LmdsbWVyLCByeWVsYW5kOC5nbG1lcikKYGBgCgpUZXN0IGdyb3Vwc2l6ZQoKYGBge3IgfQpyeWVsYW5kOS5nbG1lcjwtZ2xtZXIocnllbGFuZC5wcm9wfmRpc3QrdGVtcCt3aW5kKygxfHZib3V0KSsoMXxvbHJlKSxkYXRhPXJ5ZWxhbmQsZmFtaWx5PWJpbm9taWFsKQpscnRlc3QocnllbGFuZDYuZ2xtZXIsIHJ5ZWxhbmQ5LmdsbWVyKQpgYGAKClRlc3QgZGlzdAoKYGBge3IgfQpyeWVsYW5kMTAuZ2xtZXI8LWdsbWVyKHJ5ZWxhbmQucHJvcH5ncm91cHNpemUrd2luZCt0ZW1wK3dpbmQrKDF8dmJvdXQpKygxfG9scmUpLGRhdGE9cnllbGFuZCxmYW1pbHk9Ymlub21pYWwpCmxydGVzdChyeWVsYW5kNi5nbG1lciwgcnllbGFuZDEwLmdsbWVyKQpgYGAKCkNoZWNrIHJlc2lkdWFscwoKYGBge3IgfQpyZXNpZHVhbHMocnllbGFuZDYuZ2xtZXIsIHR5cGU9ImRldmlhbmNlIikKcGxvdChyeWVsYW5kNi5nbG1lciwgcmVzaWQoLiwgdHlwZSA9ICJkZXZpYW5jZSIpIH4gZml0dGVkKC4pKQpgYGAKCiMjIyBGaXQgbW9kZWwgd2l0aCBvbmx5IHRlbXAgdG8gbWF0Y2ggcGFwZXIKCmBgYHtyIH0KcnllbGFuZDExLmdsbWVyPC1nbG1lcihyeWVsYW5kLnByb3B+dGVtcCsoMXx2Ym91dCkrKDF8b2xyZSksZGF0YT1yeWVsYW5kLGZhbWlseT1iaW5vbWlhbCkKc3VtbWFyeShyeWVsYW5kMTEuZ2xtZXIpCkFJQyhyeWVsYW5kMTEuZ2xtZXIpCkFJQ2MocnllbGFuZDExLmdsbWVyKQpgYGAKClBsb3QgdGVtcCB0byBtYXRjaCBwYXBlciAqKk5lZWQgdG8gYWRkIGZpdCBsb2dpc3RpYyBjdXJ2ZT8qKgoKYGBge3IgfQpnZ3Bsb3QocnllbGFuZCwgYWVzKHRlbXAsIHByb3BoYikpICsgZ2VvbV9wb2ludCgpICsKICBnZW9tX3Ntb290aChtZXRob2QgPSAibG9lc3MiLCBzcGFuID0gMC41LCBzZSA9IEZBTFNFKSAKYGBgCgojIyMgUmVydW4gb3JpZ2luYWwgR0xNTSB3aXRoIGdsbW1UTUIKCmBgYHtyfQpyeWVsYW5kLmdsbW1iIDwtIGdsbW1UTUIocnllbGFuZC5wcm9wfmdyb3Vwc2l6ZSt3aW5kK3RlbXArd2luZCsoMXx2Ym91dCksIGZhbWlseT1iaW5vbWlhbCwgZGF0YT1yeWVsYW5kKQpzdW1tYXJ5KHJ5ZWxhbmQuZ2xtbWIpCmBgYAoKIyMjIE5vdyBmaXQgYmV0YWJpbm9taWFsIHRvIGNvbXBhcmUgdG8gT0xSRQoKYGBge3J9CnJ5ZWxhbmQuZ2xtbWJiIDwtIGdsbW1UTUIocnllbGFuZC5wcm9wfmRpc3QrZ3JvdXBzaXplK3RlbXArd2luZCsoMXx2Ym91dCksZGF0YT1yeWVsYW5kLGZhbWlseT1iZXRhYmlub21pYWwobGluaz0ibG9naXQiKSkKc3VtbWFyeShyeWVsYW5kLmdsbW1iYikKQUlDYyhyeWVsYW5kLmdsbW1iYikKYGBgCgpUZXN0IHRlbXBlcmF0dXJlCgpgYGB7cn0KcnllbGFuZDEuZ2xtbWJiIDwtIGdsbW1UTUIocnllbGFuZC5wcm9wfmRpc3QrZ3JvdXBzaXplK3dpbmQrKDF8dmJvdXQpLGRhdGE9cnllbGFuZCxmYW1pbHk9YmV0YWJpbm9taWFsKGxpbms9ImxvZ2l0IikpCmxydGVzdChyeWVsYW5kLmdsbW1iYiwgcnllbGFuZDEuZ2xtbWJiKQpgYGAK