Za ilustraciju jackknife procedure iskoristićemo eksperiment o uticaju socijalnog isključivanja na perifernu temperaturu. Originalna studija IJzerman et al. (2012) (https://doi.org/10.1016/j.actpsy.2012.05.002) je pokazala da telesna temperatura ispitanika (merena na prstu) opada u eksperimentalnoj situaciji socijalnog isključivanja, dok kod grupe koja je uključena nema promene u temperaturi. Trenutno je u planu velika replikaciona studija koja će pokušati da potvrdi ili opovrgne ovaj efekat u većem broju nezavisnih laboratorija (https://osf.io/v5s6d/). Za potrebe nove studije kreirali smo skript koji sadrži konfirmatorne analize, uključujući i jackknife postupak. Ovde je izdvojen jedan deo tog skripta.

Pripremne aktivnosti

Učitavanje potrebnih paketa

paket faux služi generisanju baza podataka koje imaju višeslojnu strukturu
paket dyplyr služi lakšoj manipulaciji podacima
paket lme4 služi modelovanju mešovitih modela
paket pbkrtest služi poređenju fita mešovitih modela

library(faux)
library(dplyr)
library(lme4)
library(pbkrtest)

Generisanje simuliranog uzorka koji odgovara potrebnoj analizi i računanje svih potrebnih skorova

Kreiraćemo mali uzorak da bi se analize izvele relativno brzo.
Na početku postavljamo seed da bismo uvek dobili iste rezultate kada pokrenemo skript.

set.seed(2345)

d <- add_random(lab = 3) %>%
  add_random(ID = round(runif(3, min=10, max=20)), .nested_in = "lab") %>%
  add_random(point = 15) %>%
  add_between("ID", condition = c("inclusion","exclusion")) %>%
  add_recode("condition", "Condition", inclusion = 0, exclusion = 1) %>%
  add_ranef("ID", temp_baseline = 0.5)  %>%
  mutate(temp_baseline = 35 + temp_baseline) %>%
  add_ranef("point", temp_actual = 0.5, t_slope = 0.5, .cors = 0) %>%
  add_ranef(sigma = 1) %>%
  mutate(temp_actual = 35 + temp_actual + (0 + t_slope) * Condition + sigma) %>%
  add_between("point", tik = 0:14) %>%
  add_ranef(c("ID","point"), t = 1) %>%
  mutate(t = as.numeric(tik) * 10 + t)



d$temp <- d$temp_actual - d$temp_baseline # deviation of actual temperature from a person's baseline temperature

d$time <- round(d$t/10 - 0.49, 0) # rounding time to multiple of 10 (fully elapsed) seconds
d <- mutate(d, time = ifelse(time < 0, 0, time)) # transforming negative values to 0

d$condition_ref_zero <- d$Condition - 0.5 # centering the condition variable, i.e. zero as the reference group
d$condition_ref_included <- d$Condition # using the included group as the reference group
d$condition_ref_excluded <- d$Condition - 1 # using the excluded group as the reference group

d$time_ref_0 <- d$time - 18 # centering the time variable around the midpoint (180s)
d$time_ref_max <- d$time - 36 # centering the time variable around it's maximum value (360s)

Glavni model

U pitanju je generalni mešani model (general mixed model) gde je periferna temperatura na kažiprstu zavisna varijabla, a prediktori su vreme merenja (kontinuirani prediktor), eksperimentalna manipulacija (isključenost vs. uključnenost) i njihova interakcija. Kako je struktura podataka višeslojna (multilevel) - mere temperature su ugnježdene u ispitanike, pored fiksnih faktora, u model uključujemo i slučajne faktore. Jedan slučajni faktor odnosi se na intercept za ispitanike (nemaju svi istu prosečnu temperaturu), a drugi na efekat vremena na temperaturu (kod različitih ispitanika promene u temperaturi mogu biti različite brzine). Na kraju, kako postoji još jedan nivo ugnježdavanja - ispitanici su ugnježdeni u laboratorije - uključujemo i slučajan efekat laboratorije u model.

Glavna istraživačka hipoteza tiče se efekta interakcije vremena i eksperimentalne grupe. Kako bismo procenili njenu značajnost, pravimo dva modela - glavni istraživački model i model za poređenje. Model za poređenje je isti kao i istraživački, osim što ne sadrži interkaciju. Poređenjem fita ova dva modela dobijamo procenu veličine i značajnosti efekta.

Istraživački model

m1<- lmer(temp ~  (time | ID) + time + condition_ref_zero + time:condition_ref_zero + (1 | lab), data=d)
## boundary (singular) fit: see help('isSingular')
summary(m1)
## Linear mixed model fit by REML ['lmerMod']
## Formula: 
## temp ~ (time | ID) + time + condition_ref_zero + time:condition_ref_zero +  
##     (1 | lab)
##    Data: d
## 
## REML criterion at convergence: 1912.9
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -3.0896 -0.6457  0.0280  0.6004  3.3986 
## 
## Random effects:
##  Groups   Name        Variance  Std.Dev. Corr 
##  ID       (Intercept) 2.539e-01 0.503898      
##           time        1.089e-05 0.003301 -1.00
##  lab      (Intercept) 8.336e-02 0.288722      
##  Residual             1.265e+00 1.124548      
## Number of obs: 600, groups:  ID, 40; lab, 3
## 
## Fixed effects:
##                         Estimate Std. Error t value
## (Intercept)             -0.03658    0.20691  -0.177
## time                     0.01672    0.01053   1.588
## condition_ref_zero      -0.31016    0.24348  -1.274
## time:condition_ref_zero  0.02990    0.02106   1.420
## 
## Correlation of Fixed Effects:
##             (Intr) time   cndt__
## time        -0.404              
## cndtn_rf_zr  0.001  0.006       
## tm:cndtn_r_  0.003 -0.008 -0.686
## optimizer (nloptwrap) convergence code: 0 (OK)
## boundary (singular) fit: see help('isSingular')

Model za poređenje i poređenje dva modela

m0<- lmer(temp ~  (time | ID) + time + condition_ref_zero + (1 | lab), data=d)
## boundary (singular) fit: see help('isSingular')
KRmodcomp(m1, m0)
## large : temp ~ (time | ID) + time + condition_ref_zero + time:condition_ref_zero + 
##     (1 | lab)
## small : temp ~ (time | ID) + time + condition_ref_zero + (1 | lab)
##          stat     ndf     ddf F.scaling p.value
## Ftest  2.0151  1.0000 37.8771         1  0.1639

Pošto smo generisali jako mali uzorak (za kompleksnu strukturu podataka) i slučajan - efekat nije značajan, ali to zanemarujemo.

Analiza osetljivosti na outliere - jackknife

Da bismo proverili da li su rezultati osetljivi na outliere koristimo jackknife.

Prvo, pravimo funkciju koja (iz prethodnog outputa) izvlači i prikazuje samo značajnost interakcije.

p za interakciju

get_p <- function(d) {
m1<- lmer(temp ~  (time | ID) + time + condition_ref_zero + time:condition_ref_zero + (1 | lab), data=d)
m0<- lmer(temp ~  (time | ID) + time + condition_ref_zero + (1 | lab), data=d)
k<-KRmodcomp(m1, m0)
p=k$stats['p.value']
p
}

Zatim, kreiramo funkciju koja isključuje samo po jednog ispitanika iz uzorka u datom trenutku, za sve ispitanike. Drugim rečima, kreiramo N uzoraka čija je veličina N-1 (i svi su različiti jer se svaki put isključuje drugi ispitanik iz uzorka) - jackknife postupak.

jackknife postupak

unique(d$ID) # original sample - unique participant IDs
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36"
## [37] "I37" "I38" "I39" "I40"
# jackknifed subsamples - unique participant IDs
for (i in unique(d$ID)) {  
dd <- d[which(d$ID!=i),]
print(unique(dd$ID))
}
##  [1] "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I06" "I07" "I08" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I07" "I08" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I08" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I09" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I10" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I11" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I12" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I13"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I18" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I19" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I20" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I21" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I22" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I23" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I24" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I25"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I30" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I31" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I32" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I33" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I34" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I35" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I36" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I37"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36"
## [37] "I38" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36"
## [37] "I37" "I39" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36"
## [37] "I37" "I38" "I40"
##  [1] "I01" "I02" "I03" "I04" "I05" "I06" "I07" "I08" "I09" "I10" "I11" "I12"
## [13] "I13" "I14" "I15" "I16" "I17" "I18" "I19" "I20" "I21" "I22" "I23" "I24"
## [25] "I25" "I26" "I27" "I28" "I29" "I30" "I31" "I32" "I33" "I34" "I35" "I36"
## [37] "I37" "I38" "I39"

Na kraju, koristimo jackknife postupak da procenimo da li su rezultati osetljivi na outliere tako što poredimo originalni rezultat sa rezultatima dobijenim na jackknife poduzorcima.

primena jackknife postupka

# p-value for the original sample
res <- cbind(out='none', pvalue=get_p(d))
print(res)
##         out    pvalue   
## p.value "none" 0.1639145
# p-values for the jackknifed subsamples
for (i in unique(d$ID)) {  
  dd<-d[which(d$ID!=i),]
  string <- paste('case ',i)
  res <- cbind(out=string, pvalue=get_p(dd))
  print(res)
}
##         out         pvalue   
## p.value "case  I01" 0.1654442
##         out         pvalue   
## p.value "case  I02" 0.1879651
##         out         pvalue   
## p.value "case  I03" 0.1183316
##         out         pvalue  
## p.value "case  I04" 0.133608
##         out         pvalue   
## p.value "case  I05" 0.2197063
##         out         pvalue   
## p.value "case  I06" 0.1480865
##         out         pvalue   
## p.value "case  I07" 0.1473975
##         out         pvalue  
## p.value "case  I08" 0.180371
##         out         pvalue   
## p.value "case  I09" 0.1751511
##         out         pvalue   
## p.value "case  I10" 0.2295616
##         out         pvalue   
## p.value "case  I11" 0.1418814
##         out         pvalue   
## p.value "case  I12" 0.1964088
##         out         pvalue   
## p.value "case  I13" 0.2009045
##         out         pvalue   
## p.value "case  I14" 0.2132399
##         out         pvalue   
## p.value "case  I15" 0.2396534
##         out         pvalue   
## p.value "case  I16" 0.1509657
##         out         pvalue  
## p.value "case  I17" 0.208867
##         out         pvalue   
## p.value "case  I18" 0.2181665
##         out         pvalue   
## p.value "case  I19" 0.1456761
##         out         pvalue  
## p.value "case  I20" 0.124382
##         out         pvalue   
## p.value "case  I21" 0.1119768
##         out         pvalue   
## p.value "case  I22" 0.2258116
##         out         pvalue   
## p.value "case  I23" 0.2362346
##         out         pvalue   
## p.value "case  I24" 0.1539123
##         out         pvalue   
## p.value "case  I25" 0.1695453
##         out         pvalue   
## p.value "case  I26" 0.2663642
##         out         pvalue  
## p.value "case  I27" 0.125205
##         out         pvalue   
## p.value "case  I28" 0.1119671
##         out         pvalue   
## p.value "case  I29" 0.2135189
##         out         pvalue   
## p.value "case  I30" 0.1325667
##         out         pvalue   
## p.value "case  I31" 0.1775405
##         out         pvalue   
## p.value "case  I32" 0.3678118
##         out         pvalue   
## p.value "case  I33" 0.1638288
##         out         pvalue   
## p.value "case  I34" 0.1410615
##         out         pvalue   
## p.value "case  I35" 0.1438598
##         out         pvalue   
## p.value "case  I36" 0.1928798
##         out         pvalue   
## p.value "case  I37" 0.2408546
##         out         pvalue   
## p.value "case  I38" 0.1163684
##         out         pvalue   
## p.value "case  I39" 0.1338896
##         out         pvalue    
## p.value "case  I40" 0.09754535