Q & K Box 13.2

Mayekar et al (2017) studied some potential determinants of colour (green vs brown) of a tropical butterfly species. A laboratory population was established, and newly hatched larvae were placed in a growth chamber set at either 60% or 85% relative humidity. Resulting pupae were recorded for colour, time to pupation, pupal weight and sex. We will use their data to model green vs brown pupae against the two continuous predictors (time to pupation, pupal weight) and the one categorical predictor (sex). We will only use data from the low humidity treatment as brown pupae were very uncommon at high humidity.

Mayekar, H. V. & Kodandaramaiah, U. (2017). Pupal colour plasticity in a tropical butterfly, Mycalesis mineus (Nymphalidae: Satyrinae). PLoS One, 12, e0171482.

The paper is here, and Figure 1 is an excellent image of the divergent colour phenotypes.

Preliminaries

First, load the required packages (car, performance, MuMIn)

Import mayekar data file (mayekar.csv)

mayekar <- read.csv("../data/mayekar.csv")
mayekar

Create subset of the low-humidity treatment

#select rh = low
mayekar1 <- subset(mayekar, rh=="low")

Do simple plots for cont. predictors

plot(colnum~timepup, data=mayekar1)

plot(colnum~weight, data=mayekar1)

Fit full glm

mayekar1.glm <- glm(colnum ~ timepup*weight*sex,data=mayekar1,family=binomial)
tidy(mayekar1.glm)

glance(mayekar1.glm)

AICc(mayekar1.glm)

[1] 197

Check collinearity

vif(lm(colnum ~ timepup*weight*sex, data=mayekar1))

there are higher-order terms (interactions) in this model
consider setting type = 'predictor'; see ?vif

           timepup             weight                sex     timepup:weight        timepup:sex         weight:sex timepup:weight:sex 
              27.9               36.3             1056.6               42.5              964.7              952.8              863.2

cor(mayekar1$timepup,mayekar1$weight)

[1] -0.321

aov1 <-aov(timepup~sex, data=mayekar1)
summary(aov1)

             Df Sum Sq Mean Sq F value Pr(>F)
sex           1     50    49.6    2.33   0.13
Residuals   386   8195    21.2

aov2 <- aov(weight~sex, data=mayekar1)
summary(aov2)

             Df Sum Sq Mean Sq F value  Pr(>F)    
sex           1  0.046  0.0459    38.1 1.7e-09 ***
Residuals   386  0.465  0.0012                    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Try centering continuous predictors to reduce collinearity

mayekar1$ctimepup <- scale(mayekar1$timepup, center=TRUE, scale=FALSE)
mayekar1$cweight <- scale(mayekar1$weight, center=TRUE, scale=FALSE)
vif(lm(colnum ~ ctimepup*cweight*sex, data=mayekar1))

there are higher-order terms (interactions) in this model
consider setting type = 'predictor'; see ?vif

            ctimepup              cweight                  sex     ctimepup:cweight         ctimepup:sex          cweight:sex ctimepup:cweight:sex 
                1.18                 1.42                 1.23                 1.09                 1.17                 1.30                 1.18

Centred predictors

Fit full model

mayekar2.glm <- glm(colnum ~ ctimepup*cweight*sex,data=mayekar1,family=binomial)
summary(mayekar2.glm)


Call:
glm(formula = colnum ~ ctimepup * cweight * sex, family = binomial, 
    data = mayekar1)

Coefficients:
                      Estimate Std. Error z value Pr(>|z|)    
(Intercept)            2.98785    0.30434    9.82   <2e-16 ***
ctimepup               0.22009    0.06747    3.26   0.0011 ** 
cweight                0.67112    8.01128    0.08   0.9332    
sex1                  -0.04966    0.30434   -0.16   0.8704    
ctimepup:cweight      -1.46097    1.76341   -0.83   0.4074    
ctimepup:sex1          0.00968    0.06747    0.14   0.8859    
cweight:sex1          -7.77651    8.01128   -0.97   0.3317    
ctimepup:cweight:sex1  0.97277    1.76341    0.55   0.5812    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 201.14  on 387  degrees of freedom
Residual deviance: 180.92  on 380  degrees of freedom
AIC: 196.9

Number of Fisher Scoring iterations: 6

AICc(mayekar2.glm)

[1] 197

Fit no interaction model

mayekar3.glm <- glm(colnum ~ ctimepup+cweight+sex,data=mayekar1,family=binomial)
summary(mayekar3.glm)


Call:
glm(formula = colnum ~ ctimepup + cweight + sex, family = binomial, 
    data = mayekar1)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)    2.921      0.267   10.93  < 2e-16 ***
ctimepup       0.221      0.063    3.50  0.00046 ***
cweight        0.440      6.352    0.07  0.94476    
sex1          -0.128      0.217   -0.59  0.55502    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 201.14  on 387  degrees of freedom
Residual deviance: 183.93  on 384  degrees of freedom
AIC: 191.9

Number of Fisher Scoring iterations: 6

AICc(mayekar3.glm)

[1] 192

Compare the two models

anova(mayekar2.glm, mayekar3.glm, test = 'Chisq')

Analysis of Deviance Table

Model 1: colnum ~ ctimepup * cweight * sex
Model 2: colnum ~ ctimepup + cweight + sex
  Resid. Df Resid. Dev Df Deviance Pr(>Chi)
1       380        181                     
2       384        184 -4    -3.01     0.56

For interest, see if not centering would have changed our conclusions

Fit uncentered no interaction model

mayekar4.glm <- glm(colnum ~ timepup+weight+sex,data=mayekar1,family=binomial)
summary(mayekar4.glm)


Call:
glm(formula = colnum ~ timepup + weight + sex, family = binomial, 
    data = mayekar1)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)   -3.095      2.254   -1.37  0.16980    
timepup        0.221      0.063    3.50  0.00046 ***
weight         0.440      6.352    0.07  0.94476    
sex1          -0.128      0.217   -0.59  0.55502    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 201.14  on 387  degrees of freedom
Residual deviance: 183.93  on 384  degrees of freedom
AIC: 191.9

Number of Fisher Scoring iterations: 6

AICc(mayekar4.glm)

[1] 192

Compare to full model (uncentered)

anova(mayekar1.glm, mayekar4.glm, test = 'Chisq')

Analysis of Deviance Table

Model 1: colnum ~ timepup * weight * sex
Model 2: colnum ~ timepup + weight + sex
  Resid. Df Resid. Dev Df Deviance Pr(>Chi)
1       380        181                     
2       384        184 -4    -3.01     0.56

Same results as for centered analysis

Focus on fit of no interaction model with uncentered data

summary(mayekar4.glm)


Call:
glm(formula = colnum ~ timepup + weight + sex, family = binomial, 
    data = mayekar1)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)   -3.095      2.254   -1.37  0.16980    
timepup        0.221      0.063    3.50  0.00046 ***
weight         0.440      6.352    0.07  0.94476    
sex1          -0.128      0.217   -0.59  0.55502    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 201.14  on 387  degrees of freedom
Residual deviance: 183.93  on 384  degrees of freedom
AIC: 191.9

Number of Fisher Scoring iterations: 6

AICc(mayekar4.glm)

[1] 192

confint(mayekar4.glm)

Waiting for profiling to be done...

              2.5 % 97.5 %
(Intercept)  -7.576  1.299
timepup       0.103  0.350
weight      -12.028 12.963
sex1         -0.563  0.297

anova(mayekar4.glm, test = 'Chisq')

Analysis of Deviance Table

Model: binomial, link: logit

Response: colnum

Terms added sequentially (first to last)

        Df Deviance Resid. Df Resid. Dev Pr(>Chi)    
NULL                      387        201             
timepup  1    16.84       386        184  4.1e-05 ***
weight   1     0.02       385        184     0.88    
sex      1     0.35       384        184     0.55    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Create reduced models for lrtests

Remove each of the individual predictors

Note slight differences in P values for continuous predictors - lrtest more reliable

mayekar5.glm <- glm(colnum ~ timepup+weight,data=mayekar1,family=binomial)
mayekar6.glm <- glm(colnum ~ timepup+sex,data=mayekar1,family=binomial)
mayekar7.glm <- glm(colnum ~ weight+sex,data=mayekar1,family=binomial)
lrtest(mayekar5.glm, mayekar4.glm)

Likelihood ratio test

Model 1: colnum ~ timepup + weight
Model 2: colnum ~ timepup + weight + sex
  #Df LogLik Df Chisq Pr(>Chisq)
1   3  -92.1                    
2   4  -92.0  1  0.35       0.55

lrtest(mayekar6.glm, mayekar4.glm)

Likelihood ratio test

Model 1: colnum ~ timepup + sex
Model 2: colnum ~ timepup + weight + sex
  #Df LogLik Df Chisq Pr(>Chisq)
1   3    -92                    
2   4    -92  1     0       0.94

lrtest(mayekar7.glm, mayekar4.glm)

Likelihood ratio test

Model 1: colnum ~ weight + sex
Model 2: colnum ~ timepup + weight + sex
  #Df LogLik Df Chisq Pr(>Chisq)    
1   3  -99.4                        
2   4  -92.0  1  14.8    0.00012 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Check assumptions for the simple (no interaction) model

Get and plot residuals - focus on deviance residuals plot

residuals(mayekar4.glm, type="deviance")

      1       2       3       4       5       6       7       8       9      10      11      12      13      14      15      16      17      18      19      20      21      22 
 0.6959  0.6290  0.6864  0.5655  0.5745  0.6315  0.6303  0.5655  0.5671  0.6908  0.6903  0.6903  0.7811  0.6947  0.9218  0.4628  0.4622  0.5669  0.6317  0.4623  0.5167  0.5176 
     23      24      25      26      27      28      29      30      31      32      33      34      35      36      37      38      39      40      41      42      43      44 
 0.4610  0.4615  0.4626  0.4589  0.5211  0.4600  0.4611  0.4589  0.5170  0.5098  0.3844  0.3800  0.3387  0.5068  0.5714  0.5066  0.4170  0.4698  0.4668  0.7035  0.7024  0.5726 
     45      46      47      48      49      50      51      52      53      54      55      56      57      58      59      60      61      62      63      64      65      66 
 0.5773  0.5745  0.3389  0.3796  0.3375  0.3786  0.3846  0.3825  0.4158  0.4122  0.4171  0.4132  0.4657  0.4693  0.4640  0.4626  0.4622  0.3802  0.3789  0.3776  0.3763  0.3334 
     67      68      69      70      71      72      73      74      75      76      77      78      79      80      81      82      83      84      85      86      87      88 
 0.3756  0.3350  0.3800  0.3807  0.5052  0.5071  0.5738  0.2466  0.2464  0.2192  0.1972  0.1609  0.2750  0.2448  0.2450  0.2757  0.2761  0.2438  0.2459  0.4607  0.5165  0.5174 
     89      90      91      92      93      94      95      96      97      98      99     100     101     102     103     104     105     106     107     108     109     110 
 0.4630  0.5173  0.5222  0.5214  0.5238  0.5225  0.1144  0.1288  0.4160  0.4197  0.4176  0.4197  0.4158  0.1960  0.2217  0.4701  0.4662  0.4685  0.4175  0.2231  0.2221  0.2219 
    111     112     113     114     115     116     117     118     119     120     121     122     123     124     125     126     127     128     129     130     131     132 
 0.1977  0.1964  0.2221  0.1970  0.2435  0.2428  0.2442  0.2442  0.3416  0.3034  0.3061  0.3017  0.3013  0.3417  0.3008  0.3389  0.2777  0.2771  0.2456  0.2456  0.1580  0.1572 
    133     134     135     136     137     138     139     140     141     142     143     144     145     146     147     148     149     150     151     152     153     154 
 0.1580  0.1566  0.1585  0.1581  0.1587  0.5275  0.1780  0.1806  0.1795  0.1792  0.1596  0.1812  0.1579  0.1429  0.1269  0.1433  0.1269  0.1435  0.1269  0.2768  0.2464  0.2774 
    155     156     157     158     159     160     161     162     163     164     165     166     167     168     169     170     171     172     173     174     175     176 
 0.2753  0.2781  0.2750  0.2439  0.3811  0.3814  0.3047  0.2714  0.3781  0.3353  0.3824  0.3378  0.3786  0.3711  0.4190  0.3744  0.3362  0.2184  0.2182  0.2492  0.2187  0.2185 
    177     178     179     180     181     182     183     184     185     186     187     188     189     190     191     192     193     194     195     196     197     198 
 0.2200  0.2498  0.1763  0.1771  0.3743  0.3730  0.4202  0.3732  0.1994  0.1757  0.2016  0.1986  0.1992  0.1756  0.1752  0.1773  0.1761  0.2002  0.1976  0.2001  0.1759  0.0820 
    199     200     201     202     203     204     205     206     207     208     209     210     211     212     213     214     215     216     217     218     219     220 
 0.0915  0.0932  0.1593  0.1419  0.1414  0.1403  0.1609  0.1428  0.1415  0.1421  0.2740  0.2754 -1.7458 -2.1302 -2.3094 -2.3992 -2.3077 -1.9542 -2.1442 -2.3111 -2.3158 -2.3130 
    386     387     388     389     390     391     392     393     394     395     396     397     398     399     400     401     402     403     404     405     406     407 
 0.1430  0.4604  0.4129  0.4086 -2.2326  0.3805 -2.0346  0.3772  0.3771  0.3368  0.5133  0.4114  0.2995  0.2995  0.2996  0.2989  0.0170  0.2747  0.2716  0.2752  0.2433 -2.4124 
    408     409     410     411     412     413     414     415     416     417     418     419     420     421     422     423     424     425     426     427     428     429 
 0.3724  0.3799  0.3762  0.3355  0.2716  0.2711  0.2439  0.3761  0.3354  0.3768  0.3358  0.4226  0.3049  0.3055  0.3038  0.3044  0.3368  0.5635  0.5738  0.5647  0.5703  0.5696 
    430     431     432     433     434     435     436     437     438     439     440     441     442     443     444     445     446     447     448     449     450     451 
 0.4566  0.4627  0.3379  0.4227  0.5141  0.3096  0.3750  0.3774  0.3393  0.3411  0.3851  0.3791  0.3845 -1.6588  0.3344  0.6383  0.4697  0.4694  0.4721  0.4605  0.4588  0.5155 
    452     453     454     455     456     457     458     459     460     461     462     463     464     465     466     467     468     469     470     471     472     473 
 0.4593  0.4600  0.4607  0.3805  0.3813  0.5725  0.3069  0.3059  0.3076  0.1280  0.1285  0.2484  0.5105 -2.0532  0.5110  0.5085  0.1993  0.1972  0.2001  0.2008  0.1998  0.1278 
    474     475     476     477     478     479     480     481     482     483     484     485     486     487     488     489     490     491     492     493     494     495 
-2.2341  0.4137 -2.2329  0.3745 -2.2273  0.3015  0.3016  0.2998  0.3393  0.3382  0.3003  0.2996  0.3409  0.2999  0.3021  0.3387 -1.7510  0.2195  0.2207 -2.7359  0.2201  0.1940 
    496     497     498     499     500     501     502     503     504     505     506     507     508     509     510     511     512     513     514     515     516     517 
 0.1968  0.1965  0.3041  0.2701  0.1428 -2.3070  0.0826  0.0824  0.0740  0.2742  0.4183  0.1995  0.1777  0.2759  0.3029  0.5099  0.4182  0.4154  0.3733  0.3772  0.5735  0.6319 
    518     519     520     521     522     523     524     525     526     527     528     529     530     531     532     533     534     535     536     537     538     539 
 0.5147  0.3029  0.2427 -2.5690  0.1605  0.1394  0.1599  0.3830  0.3344 -1.9371 -1.7521  0.6895  0.7739  0.6906  0.6902  0.6899 -1.8619  0.1996 -1.7530  0.5757  0.4190  0.3446 
    540     541     542     543     544     545     546     547     548     549     550     551     552     553 
 0.4662  0.3333  0.3344 -2.4122  0.3367  0.3330 -2.0346  0.5186  0.5138  0.5179  0.4610  0.2746  0.3022  0.3033

residualPlots(mayekar4.glm, type="deviance")

        Test stat Pr(>|Test stat|)
timepup      1.41             0.23
weight       0.01             0.94
sex

Check influence diagnostics

influencePlot(mayekar4.glm)

Examine some alternative assessments

Get odds ratio with CI

exp(coef(mayekar4.glm))

(Intercept)     timepup      weight        sex1 
     0.0453      1.2468      1.5530      0.8795

exp(confint.default(mayekar4.glm))

               2.5 %   97.5 %
(Intercept) 5.46e-04 3.76e+00
timepup     1.10e+00 1.41e+00
weight      6.08e-06 3.96e+05
sex1        5.74e-01 1.35e+00

Get added variable plots - hard to interpret

avPlots(mayekar4.glm)

Get H-L test and Tjur r²

performance_hosmer(mayekar4.glm, n_bins=10)

# Hosmer-Lemeshow Goodness-of-Fit Test

  Chi-squared: 6.877
           df: 8    
      p-value: 0.550

Summary: model seems to fit well.

r2_tjur(mayekar4.glm)

Tjur's R2 
   0.0433

LS0tCnRpdGxlOiAiUSAmIEsgQm94IDEzLjIiCm91dHB1dDoKICBodG1sX25vdGVib29rCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkKYGBgCgpNYXlla2FyIGV0IGFsICgyMDE3KSBzdHVkaWVkIHNvbWUgcG90ZW50aWFsIGRldGVybWluYW50cyBvZiBjb2xvdXIgKGdyZWVuIHZzIGJyb3duKSBvZiBhIHRyb3BpY2FsIGJ1dHRlcmZseSBzcGVjaWVzLiBBIGxhYm9yYXRvcnkgcG9wdWxhdGlvbiB3YXMgZXN0YWJsaXNoZWQsIGFuZCBuZXdseSBoYXRjaGVkIGxhcnZhZSB3ZXJlIHBsYWNlZCBpbiBhIGdyb3d0aCBjaGFtYmVyIHNldCBhdCBlaXRoZXIgNjAlIG9yIDg1JSByZWxhdGl2ZSBodW1pZGl0eS4gUmVzdWx0aW5nIHB1cGFlIHdlcmUgcmVjb3JkZWQgZm9yIGNvbG91ciwgdGltZSB0byBwdXBhdGlvbiwgcHVwYWwgd2VpZ2h0IGFuZCBzZXguIMKgV2Ugd2lsbCB1c2UgdGhlaXIgZGF0YSB0byBtb2RlbCBncmVlbiB2cyBicm93biBwdXBhZSBhZ2FpbnN0IHRoZSB0d28gY29udGludW91cyBwcmVkaWN0b3JzICh0aW1lIHRvIHB1cGF0aW9uLCBwdXBhbCB3ZWlnaHQpIGFuZCB0aGUgb25lIGNhdGVnb3JpY2FsIHByZWRpY3RvciAoc2V4KS4gV2Ugd2lsbCBvbmx5IHVzZSBkYXRhIGZyb20gdGhlIGxvdyBodW1pZGl0eSB0cmVhdG1lbnQgYXMgYnJvd24gcHVwYWUgd2VyZSB2ZXJ5IHVuY29tbW9uIGF0IGhpZ2ggaHVtaWRpdHkuCgpNYXlla2FyLCBILiBWLiAmIEtvZGFuZGFyYW1haWFoLCBVLiAoMjAxNykuIFB1cGFsIGNvbG91ciBwbGFzdGljaXR5IGluIGEgdHJvcGljYWwgYnV0dGVyZmx5LCAqTXljYWxlc2lzIG1pbmV1cyogKE55bXBoYWxpZGFlOiBTYXR5cmluYWUpLiAqUExvUyBPbmUqLCAxMiwgZTAxNzE0ODIuCgpUaGUgcGFwZXIgaXMgW2hlcmVdKGh0dHBzOi8vZG9pLm9yZy8xMC4xMzcxL2pvdXJuYWwucG9uZS4wMTcxNDgyKSwgYW5kIEZpZ3VyZSAxIGlzIGFuIGV4Y2VsbGVudCBpbWFnZSBvZiB0aGUgZGl2ZXJnZW50IGNvbG91ciBwaGVub3R5cGVzLgoKIyMjIFByZWxpbWluYXJpZXMKCkZpcnN0LCBsb2FkIHRoZSByZXF1aXJlZCBwYWNrYWdlcyAoY2FyLCBwZXJmb3JtYW5jZSwgTXVNSW4pCgpgYGB7ciBpbmNsdWRlPUZBTFNFLCByZXN1bHRzPSdoaWRlJ30Kc291cmNlKCIuLi9SL2xpYnJhcmllcy5SIikgICAjVGhpcyBpcyB0aGUgY29tbW9uIGxpYnJhcnkKbGlicmFyeShwZXJmb3JtYW5jZSkKYGBgCgpJbXBvcnQgbWF5ZWthciBkYXRhIGZpbGUgKFttYXlla2FyLmNzdl0oLi4vZGF0YS9tYXlla2FyLmNzdikpCgpgYGB7cn0KbWF5ZWthciA8LSByZWFkLmNzdigiLi4vZGF0YS9tYXlla2FyLmNzdiIpCm1heWVrYXIKYGBgCgpDcmVhdGUgc3Vic2V0IG9mIHRoZSBsb3ctaHVtaWRpdHkgdHJlYXRtZW50CgpgYGB7ciB9CiNzZWxlY3QgcmggPSBsb3cKbWF5ZWthcjEgPC0gc3Vic2V0KG1heWVrYXIsIHJoPT0ibG93IikKYGBgCgojIyMgRG8gc2ltcGxlIHBsb3RzIGZvciBjb250LiBwcmVkaWN0b3JzCgpgYGB7ciB9CnBsb3QoY29sbnVtfnRpbWVwdXAsIGRhdGE9bWF5ZWthcjEpCnBsb3QoY29sbnVtfndlaWdodCwgZGF0YT1tYXlla2FyMSkKYGBgCgojIyMgRml0IGZ1bGwgZ2xtCgpgYGB7ciB9Cm1heWVrYXIxLmdsbSA8LSBnbG0oY29sbnVtIH4gdGltZXB1cCp3ZWlnaHQqc2V4LGRhdGE9bWF5ZWthcjEsZmFtaWx5PWJpbm9taWFsKQp0aWR5KG1heWVrYXIxLmdsbSkKZ2xhbmNlKG1heWVrYXIxLmdsbSkKQUlDYyhtYXlla2FyMS5nbG0pCmBgYAoKIyMjIyBDaGVjayBjb2xsaW5lYXJpdHkKCmBgYHtyIH0KdmlmKGxtKGNvbG51bSB+IHRpbWVwdXAqd2VpZ2h0KnNleCwgZGF0YT1tYXlla2FyMSkpCmNvcihtYXlla2FyMSR0aW1lcHVwLG1heWVrYXIxJHdlaWdodCkKYW92MSA8LWFvdih0aW1lcHVwfnNleCwgZGF0YT1tYXlla2FyMSkKc3VtbWFyeShhb3YxKQphb3YyIDwtIGFvdih3ZWlnaHR+c2V4LCBkYXRhPW1heWVrYXIxKQpzdW1tYXJ5KGFvdjIpCmBgYAoKVHJ5IGNlbnRlcmluZyBjb250aW51b3VzIHByZWRpY3RvcnMgdG8gcmVkdWNlIGNvbGxpbmVhcml0eQoKYGBge3IgfQptYXlla2FyMSRjdGltZXB1cCA8LSBzY2FsZShtYXlla2FyMSR0aW1lcHVwLCBjZW50ZXI9VFJVRSwgc2NhbGU9RkFMU0UpCm1heWVrYXIxJGN3ZWlnaHQgPC0gc2NhbGUobWF5ZWthcjEkd2VpZ2h0LCBjZW50ZXI9VFJVRSwgc2NhbGU9RkFMU0UpCnZpZihsbShjb2xudW0gfiBjdGltZXB1cCpjd2VpZ2h0KnNleCwgZGF0YT1tYXlla2FyMSkpCmBgYAoKIyMgQ2VudHJlZCBwcmVkaWN0b3JzCgpGaXQgZnVsbCBtb2RlbAoKYGBge3IgfQptYXlla2FyMi5nbG0gPC0gZ2xtKGNvbG51bSB+IGN0aW1lcHVwKmN3ZWlnaHQqc2V4LGRhdGE9bWF5ZWthcjEsZmFtaWx5PWJpbm9taWFsKQpzdW1tYXJ5KG1heWVrYXIyLmdsbSkKQUlDYyhtYXlla2FyMi5nbG0pCmBgYAoKRml0IG5vIGludGVyYWN0aW9uIG1vZGVsCgpgYGB7ciB9Cm1heWVrYXIzLmdsbSA8LSBnbG0oY29sbnVtIH4gY3RpbWVwdXArY3dlaWdodCtzZXgsZGF0YT1tYXlla2FyMSxmYW1pbHk9Ymlub21pYWwpCnN1bW1hcnkobWF5ZWthcjMuZ2xtKQpBSUNjKG1heWVrYXIzLmdsbSkKYGBgCgpDb21wYXJlIHRoZSB0d28gbW9kZWxzCgpgYGB7ciB9CmFub3ZhKG1heWVrYXIyLmdsbSwgbWF5ZWthcjMuZ2xtLCB0ZXN0ID0gJ0NoaXNxJykKYGBgCgojIyMgRm9yIGludGVyZXN0LCBzZWUgaWYgbm90IGNlbnRlcmluZyB3b3VsZCBoYXZlIGNoYW5nZWQgb3VyIGNvbmNsdXNpb25zCgojIyMjIEZpdCB1bmNlbnRlcmVkIG5vIGludGVyYWN0aW9uIG1vZGVsCgpgYGB7ciB9Cm1heWVrYXI0LmdsbSA8LSBnbG0oY29sbnVtIH4gdGltZXB1cCt3ZWlnaHQrc2V4LGRhdGE9bWF5ZWthcjEsZmFtaWx5PWJpbm9taWFsKQpzdW1tYXJ5KG1heWVrYXI0LmdsbSkKQUlDYyhtYXlla2FyNC5nbG0pCmBgYAoKQ29tcGFyZSB0byBmdWxsIG1vZGVsICh1bmNlbnRlcmVkKQoKYGBge3IgfQphbm92YShtYXlla2FyMS5nbG0sIG1heWVrYXI0LmdsbSwgdGVzdCA9ICdDaGlzcScpCmBgYAoKU2FtZSByZXN1bHRzIGFzIGZvciBjZW50ZXJlZCBhbmFseXNpcwoKIyMjIEZvY3VzIG9uIGZpdCBvZiBubyBpbnRlcmFjdGlvbiBtb2RlbCB3aXRoIHVuY2VudGVyZWQgZGF0YQoKYGBge3IgfQpzdW1tYXJ5KG1heWVrYXI0LmdsbSkKQUlDYyhtYXlla2FyNC5nbG0pCmNvbmZpbnQobWF5ZWthcjQuZ2xtKQphbm92YShtYXlla2FyNC5nbG0sIHRlc3QgPSAnQ2hpc3EnKQpgYGAKCiMjIyBDcmVhdGUgcmVkdWNlZCBtb2RlbHMgZm9yIGxydGVzdHMKClJlbW92ZSBlYWNoIG9mIHRoZSBpbmRpdmlkdWFsIHByZWRpY3RvcnMKCk5vdGUgc2xpZ2h0IGRpZmZlcmVuY2VzIGluIFAgdmFsdWVzIGZvciBjb250aW51b3VzIHByZWRpY3RvcnMgLSBscnRlc3QgbW9yZSByZWxpYWJsZQoKYGBge3IgfQptYXlla2FyNS5nbG0gPC0gZ2xtKGNvbG51bSB+IHRpbWVwdXArd2VpZ2h0LGRhdGE9bWF5ZWthcjEsZmFtaWx5PWJpbm9taWFsKQptYXlla2FyNi5nbG0gPC0gZ2xtKGNvbG51bSB+IHRpbWVwdXArc2V4LGRhdGE9bWF5ZWthcjEsZmFtaWx5PWJpbm9taWFsKQptYXlla2FyNy5nbG0gPC0gZ2xtKGNvbG51bSB+IHdlaWdodCtzZXgsZGF0YT1tYXlla2FyMSxmYW1pbHk9Ymlub21pYWwpCmxydGVzdChtYXlla2FyNS5nbG0sIG1heWVrYXI0LmdsbSkKbHJ0ZXN0KG1heWVrYXI2LmdsbSwgbWF5ZWthcjQuZ2xtKQpscnRlc3QobWF5ZWthcjcuZ2xtLCBtYXlla2FyNC5nbG0pCmBgYAoKIyMjIyBDaGVjayBhc3N1bXB0aW9ucyBmb3IgdGhlIHNpbXBsZSAobm8gaW50ZXJhY3Rpb24pIG1vZGVsCgpHZXQgYW5kIHBsb3QgcmVzaWR1YWxzIC0gZm9jdXMgb24gZGV2aWFuY2UgcmVzaWR1YWxzIHBsb3QKCmBgYHtyIH0KcmVzaWR1YWxzKG1heWVrYXI0LmdsbSwgdHlwZT0iZGV2aWFuY2UiKQpyZXNpZHVhbFBsb3RzKG1heWVrYXI0LmdsbSwgdHlwZT0iZGV2aWFuY2UiKQpgYGAKCkNoZWNrIGluZmx1ZW5jZSBkaWFnbm9zdGljcwoKYGBge3IgfQppbmZsdWVuY2VQbG90KG1heWVrYXI0LmdsbSkKYGBgCgojIyMgRXhhbWluZSBzb21lIGFsdGVybmF0aXZlIGFzc2Vzc21lbnRzCgpHZXQgb2RkcyByYXRpbyB3aXRoIENJCgpgYGB7ciB9CmV4cChjb2VmKG1heWVrYXI0LmdsbSkpCmV4cChjb25maW50LmRlZmF1bHQobWF5ZWthcjQuZ2xtKSkKYGBgCgpHZXQgYWRkZWQgdmFyaWFibGUgcGxvdHMgLSBoYXJkIHRvIGludGVycHJldAoKYGBge3IgfQphdlBsb3RzKG1heWVrYXI0LmdsbSkKYGBgCgpHZXQgSC1MIHRlc3QgYW5kIFRqdXIgcl4yXgoKYGBge3IgfQpwZXJmb3JtYW5jZV9ob3NtZXIobWF5ZWthcjQuZ2xtLCBuX2JpbnM9MTApCnIyX3RqdXIobWF5ZWthcjQuZ2xtKQpgYGAK