From b9525b57e41db9822ab250fb66bc64e3326338ca Mon Sep 17 00:00:00 2001 From: "Kyle M. Lang" Date: Mon, 16 Oct 2023 13:36:20 +0200 Subject: [PATCH] Updating week 7 at-home exercises --- docs/404.html | 6 +- docs/a3_components.html | 6 +- docs/acknowledgement.html | 6 +- docs/assignment-1-path-analysis.html | 6 +- ...gnment-2-confirmatory-factor-analysis.html | 6 +- docs/assignments.html | 6 +- docs/at-home-exercises-1.html | 6 +- docs/at-home-exercises-2.html | 6 +- docs/at-home-exercises-3.html | 6 +- docs/at-home-exercises-4.html | 6 +- docs/at-home-exercises-5.html | 6 +- docs/at-home-exercises-6.html | 797 +++++++++++++++++- docs/at-home-exercises.html | 6 +- docs/attendance.html | 6 +- docs/cfa.html | 6 +- docs/course-data.html | 6 +- docs/course-overview.html | 6 +- docs/data_options.html | 6 +- docs/efa.html | 6 +- docs/elaboration-tips.html | 6 +- docs/full-sem.html | 6 +- docs/grading-1.html | 6 +- docs/grading.html | 6 +- docs/in-class-exercises-1.html | 6 +- docs/in-class-exercises-2.html | 6 +- docs/in-class-exercises-3.html | 6 +- docs/in-class-exercises-4.html | 6 +- docs/in-class-exercises-5.html | 6 +- docs/in-class-exercises-6.html | 6 +- docs/in-class-exercises.html | 6 +- docs/index.html | 8 +- docs/installing-software.html | 6 +- docs/instructors.html | 6 +- docs/introduction-to-r.html | 6 +- docs/learning-goals.html | 6 +- docs/lecture-1.html | 6 +- docs/lecture-2.html | 6 +- docs/lecture-3.html | 6 +- docs/lecture-4.html | 6 +- docs/lecture-5.html | 6 +- docs/lecture-6.html | 6 +- docs/lecture.html | 6 +- docs/mediation-moderation.html | 6 +- docs/multiple-group-models.html | 6 +- docs/note-on-data-updates.html | 6 +- docs/procedures.html | 6 +- docs/reading-1.html | 6 +- docs/reading-2.html | 6 +- docs/reading-3.html | 6 +- docs/reading-4.html | 6 +- docs/reading-5.html | 6 +- docs/reading-6.html | 6 +- docs/reading-questions.html | 6 +- docs/reading.html | 6 +- docs/reference-keys.txt | 14 + docs/resources.html | 6 +- docs/rules.html | 6 +- docs/schedule.html | 6 +- docs/search_index.json | 2 +- docs/software-setup.html | 6 +- docs/statistical-modeling-path-analysis.html | 6 +- docs/typographic-conventions.html | 6 +- docs/weekly-preparation.html | 6 +- .../figure-html/unnamed-chunk-132-1.png | Bin 70988 -> 70188 bytes .../figure-html/unnamed-chunk-134-1.png | Bin 71546 -> 71554 bytes .../figure-html/unnamed-chunk-207-1.png | Bin 0 -> 107271 bytes docs/wrap-up.html | 6 +- ... Multigroup and Meas invar.Rmd => lab.Rmd} | 0 sections/week7/home.Rmd | 453 ++++++++++ week7.Rmd | 3 +- 70 files changed, 1570 insertions(+), 67 deletions(-) create mode 100644 docs/willy_wallaby_from_wasatch_files/figure-html/unnamed-chunk-207-1.png rename resources/lavaan_summer_school/multiple_group_models/lab/Exercise with answers - RMK/{Lavaan - Lab - Multigroup and Meas invar.Rmd => lab.Rmd} (100%) create mode 100644 sections/week7/home.Rmd diff --git a/docs/404.html b/docs/404.html index 506f7ecb..0d710c90 100644 --- a/docs/404.html +++ b/docs/404.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/a3_components.html b/docs/a3_components.html index b0956b89..5d41e5c8 100644 --- a/docs/a3_components.html +++ b/docs/a3_components.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/acknowledgement.html b/docs/acknowledgement.html index 3746fe5b..256b4598 100644 --- a/docs/acknowledgement.html +++ b/docs/acknowledgement.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/assignment-1-path-analysis.html b/docs/assignment-1-path-analysis.html index 4d9222ab..ad2b41b5 100644 --- a/docs/assignment-1-path-analysis.html +++ b/docs/assignment-1-path-analysis.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/assignment-2-confirmatory-factor-analysis.html b/docs/assignment-2-confirmatory-factor-analysis.html index c8a6edce..7503feca 100644 --- a/docs/assignment-2-confirmatory-factor-analysis.html +++ b/docs/assignment-2-confirmatory-factor-analysis.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/assignments.html b/docs/assignments.html index 3680c786..a2085f5e 100644 --- a/docs/assignments.html +++ b/docs/assignments.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/at-home-exercises-1.html b/docs/at-home-exercises-1.html index f837ccdd..bfb80acd 100644 --- a/docs/at-home-exercises-1.html +++ b/docs/at-home-exercises-1.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/at-home-exercises-2.html b/docs/at-home-exercises-2.html index a1f176ec..9c4156d6 100644 --- a/docs/at-home-exercises-2.html +++ b/docs/at-home-exercises-2.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/at-home-exercises-3.html b/docs/at-home-exercises-3.html index 8ef4da6a..f0809f40 100644 --- a/docs/at-home-exercises-3.html +++ b/docs/at-home-exercises-3.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/at-home-exercises-4.html b/docs/at-home-exercises-4.html index ac384d21..b024315f 100644 --- a/docs/at-home-exercises-4.html +++ b/docs/at-home-exercises-4.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/at-home-exercises-5.html b/docs/at-home-exercises-5.html index 990b9702..6bc098c5 100644 --- a/docs/at-home-exercises-5.html +++ b/docs/at-home-exercises-5.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/at-home-exercises-6.html b/docs/at-home-exercises-6.html index 4e3bfb87..859fe975 100644 --- a/docs/at-home-exercises-6.html +++ b/docs/at-home-exercises-6.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • @@ -463,7 +467,796 @@

    7.3 At-Home Exercises

    -

    Coming soon to a GitBook near you!

    +
    +
    +

    7.3.1 Multiple-Group Path Analysis

    +
    +

    To fix ideas, we’ll start these practical exercises by re-running part of the +moderation analysis from the Week 3 At-Home Exercises +as a multiple group model.

    +
    +
    +

    7.3.1.1

    +

    Load the Sesam2.sav data.

    +
      +
    • NOTE: Unless otherwise specified, all analyses in Section 7.3.1 +use these data.
      • +
    +
    + +Click to show code + +
    library(haven)
+
+# Read the data into an object called 'sesam2':
+sesam2 <- read_sav("Sesam2.sav")
    +
    +
    +

    VIEWCAT is a nominal grouping variable, but it is represented as a numeric +variable in the sesam2 data. The levels represent the following frequencies of +Sesame Street viewership of the children in the data:

    +
      +
    • VIEWCAT = 1: Rarely/Never
      • +
    • VIEWCAT = 2: 2–3 times a week
      • +
    • VIEWCAT = 3: 4–5 times a week
      • +
    • VIEWCAT = 4: > 5 times a week
      • +
    +

    We will use VIEWCAT as the grouping variable in our path model. To do so, we +don’t really need to convert VIEWCAT into a factor, but, if we do, lavaan +will give our groups meaningful labels in the output. That added clarity can be +pretty helpful.

    +
    +
    +
    +

    7.3.1.2

    +

    Convert VIEWCAT into a factor.

    +
      +
    • Make sure that VIEWCAT = 1 is the reference group.
      • +
    • Assign the factor labels denoted above.
      • +
    +
    + +Click to show code + +
    library(dplyr)
+
+## Store the old version for checking:
+tmp <- sesam2$VIEWCAT
+
+## Convert 'VIEWCAT' to a factor:
+ sesam2 <- mutate(sesam2,
+                  VIEWCAT = factor(VIEWCAT,
+                                   labels = c("Rarely/never",
+                                              "2-3 times per week",
+                                              "4-5 times per week",
+                                              "> 5 times per week")
+                                              )
+                  )
+
+## Check the conversion:
+table(old = tmp, new = sesam2$VIEWCAT, useNA = "always")
    +
    ##       new
+## old    Rarely/never 2-3 times per week 4-5 times per week > 5 times per week
+##   1              25                  0                  0                  0
+##   2               0                 44                  0                  0
+##   3               0                  0                 57                  0
+##   4               0                  0                  0                 53
+##   <NA>            0                  0                  0                  0
+##       new
+## old    <NA>
+##   1       0
+##   2       0
+##   3       0
+##   4       0
+##   <NA>    0
    +
    +
    +
    +
    +

    7.3.1.3

    +

    Create a conditional slopes plot to visualize the effect of AGE on POSTNUMB +within each of the VIEWCAT groups.

    +
      +
    • Based on this visualization, do you think it is reasonable to expect that +VIEWCAT moderates the effect of AGE on POSTNUMB?
      • +
    +
    + +Click to show code + +
    library(ggplot2)
+
+ggplot(sesam2, aes(AGE, POSTNUMB, color = VIEWCAT)) + 
+  geom_point() + 
+  geom_smooth(method = "lm", se = FALSE)
    +

    +
    + +Click for explanation + +

    The regression lines representing the conditional focal effects are not parallel, +so there appears to be some level of moderation. That being said, the differences +are pretty small, so the moderation may not be significant (i.e., the non-parallel +regression lines may simply be reflecting sampling variability).

    +
    +
    +
    +

    We will use path analysis to test if VIEWCAT moderates the effect of AGE on +POSTNUMB. This analysis will entail three steps:

    +
      +
    1. Estimate the unrestricted multiple-group model wherein we regress POSTNUMB +onto AGE and specify VIEWCAT as the grouping factor.
      2. +
    2. Estimate the restricted model wherein we constrain the AGE \(rightarrow\) +POSTNUMB effect to be equal in all VIEWCAT groups.
      3. +
    3. Conduct a \(\Delta \chi^2\) test to compare the fit of the two models.
      4. +
    +
    +
    +
    +

    7.3.1.4

    +

    Estimate the unrestricted path model described above.

    +
      +
    • Include the intercept term in your model.
      • +
    • Judging from the focal effects estimate in each group, do you think moderation +is plausible?
      • +
    +
    + +Click to show code + +
    library(lavaan)
+
+## Estimate the additive model a view the results:
+out_full <- sem('POSTNUMB ~ 1 + AGE', data = sesam2, group = "VIEWCAT")
+summary(out_full)
    +
    ## lavaan 0.6.16 ended normally after 1 iteration
+## 
+##   Estimator                                         ML
+##   Optimization method                           NLMINB
+##   Number of model parameters                        12
+## 
+##   Number of observations per group:                   
+##     Rarely/never                                    25
+##     4-5 times per week                              57
+##     > 5 times per week                              53
+##     2-3 times per week                              44
+## 
+## Model Test User Model:
+##                                                       
+##   Test statistic                                 0.000
+##   Degrees of freedom                                 0
+##   Test statistic for each group:
+##     Rarely/never                                 0.000
+##     4-5 times per week                           0.000
+##     > 5 times per week                           0.000
+##     2-3 times per week                           0.000
+## 
+## Parameter Estimates:
+## 
+##   Standard errors                             Standard
+##   Information                                 Expected
+##   Information saturated (h1) model          Structured
+## 
+## 
+## Group 1 [Rarely/never]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE               0.747    0.239    3.118    0.002
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        -18.721   12.142   -1.542    0.123
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB         73.285   20.728    3.536    0.000
+## 
+## 
+## Group 2 [4-5 times per week]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE               0.554    0.234    2.369    0.018
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB          4.861   12.178    0.399    0.690
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        135.923   25.461    5.339    0.000
+## 
+## 
+## Group 3 [> 5 times per week]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE               0.405    0.214    1.894    0.058
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB         15.676   11.249    1.394    0.163
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        115.942   22.523    5.148    0.000
+## 
+## 
+## Group 4 [2-3 times per week]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE               0.729    0.255    2.855    0.004
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB         -8.747   13.003   -0.673    0.501
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        112.019   23.882    4.690    0.000
    +
    + +Click for explanation + +

    There are some notable differences in the AGE \(\rightarrow\) POSTNUMB focal +effect between VIEWCAT groups. It looks like VIEWCAT could moderate the +focal effect.

    +
    +
    +
    +
    +
    +

    7.3.1.5

    +

    Estimate the restricted model described above.

    +
      +
    • Equate the focal effect across all VIEWCAT groups.
      • +
    +
    + +Click to show code + +
    ## Estimate the restricted model and view the results:
+out_res <- sem('POSTNUMB ~ 1 + c("b1", "b1", "b1", "b1") * AGE',
+               data = sesam2,
+               group = "VIEWCAT")
+summary(out_res)
    +
    ## lavaan 0.6.16 ended normally after 38 iterations
+## 
+##   Estimator                                         ML
+##   Optimization method                           NLMINB
+##   Number of model parameters                        12
+##   Number of equality constraints                     3
+## 
+##   Number of observations per group:                   
+##     Rarely/never                                    25
+##     4-5 times per week                              57
+##     > 5 times per week                              53
+##     2-3 times per week                              44
+## 
+## Model Test User Model:
+##                                                       
+##   Test statistic                                 1.486
+##   Degrees of freedom                                 3
+##   P-value (Chi-square)                           0.685
+##   Test statistic for each group:
+##     Rarely/never                                 0.413
+##     4-5 times per week                           0.027
+##     > 5 times per week                           0.760
+##     2-3 times per week                           0.287
+## 
+## Parameter Estimates:
+## 
+##   Standard errors                             Standard
+##   Information                                 Expected
+##   Information saturated (h1) model          Structured
+## 
+## 
+## Group 1 [Rarely/never]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE       (b1)    0.592    0.118    5.032    0.000
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        -10.966    6.154   -1.782    0.075
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB         74.505   21.073    3.536    0.000
+## 
+## 
+## Group 2 [4-5 times per week]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE       (b1)    0.592    0.118    5.032    0.000
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB          2.869    6.275    0.457    0.647
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        135.988   25.473    5.339    0.000
+## 
+## 
+## Group 3 [> 5 times per week]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE       (b1)    0.592    0.118    5.032    0.000
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB          5.923    6.313    0.938    0.348
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        117.616   22.848    5.148    0.000
+## 
+## 
+## Group 4 [2-3 times per week]:
+## 
+## Regressions:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##   POSTNUMB ~                                          
+##     AGE       (b1)    0.592    0.118    5.032    0.000
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB         -1.826    6.157   -0.297    0.767
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)
+##    .POSTNUMB        112.751   24.039    4.690    0.000
    +
    +
    +
    +
    +

    7.3.1.6

    +

    Test for moderation by comparing the full and restricted models from +7.3.1.4 and 7.3.1.5, respectively:

    +
      +
    • Does VIEWCAT significantly moderate the effect of AGE on POSTNUMB?
      • +
    +
    + +Click to show code + +
    ## Test for moderation:
+anova(out_full, out_res)
    +
    + +
    +
    + +Click for explanation + +

    No, VIEWCAT does not significantly moderate the effect of AGE on POSTNUMB +(\(\Delta \chi^2[3] = 1.486\), \(p = 0.685\)).

    +
    +
    +
    +
    +
    +
    +

    7.3.2 Multiple-Group CFA

    +
    +

    In the next part of these exercises, we will estimate a multiple-group CFA to +evaluate the measurement structure of a scale assessing Prolonged Grief Disorder. +The relevant data are contained in the PGDdata2.txt file. This dataset +consists of a grouping variable, Kin2 (with two levels: “partner” and “else”) +and 5 items taken from the Inventory of Complicated Grief:

    +
      +
    1. Yearning
      2. +
    2. Part of self died
      3. +
    3. Difficulty accepting the loss
      4. +
    4. Avoiding reminders of deceased
      5. +
    5. Bitterness about the loss
      6. +
    +

    You can find more information about this scale in Boelen et al. (2010).

    +
    +
    +

    7.3.2.1

    +

    Load the PGDdata2.txt data.

    +
      +

    • Use the read.table() function to load the data.

      +
        +
      • Convert the missing values to NA via the na.strings argument.
        • +
      • Retain the column labels via the header argument.
        • +
      • Specify the field delimiter as the tab character (i.e., "\t").
        • +
      • +

    • Exclude any cases with missing values on Kin2.

      • +

    • NOTE: Unless otherwise specified, all analyses in Section 7.3.2 use +these data.

      • +
    +
    + +Click to show code + +
    ## Load the data:
+pgd <- read.table("PGDdata2.txt",
+                  na.strings = "-999",
+                  header = TRUE,
+                  sep = "\t") %>%
+       filter(!is.na(Kin2))
+
+## Check the results:
+head(pgd)
+summary(pdg)
+str(pgd)
    +
    + +
    +
    ##       Kin2            b1pss1          b2pss2           b3pss3      
+##  Min.   :0.0000   Min.   :0.000   Min.   :0.0000   Min.   :0.0000  
+##  1st Qu.:0.0000   1st Qu.:0.000   1st Qu.:0.0000   1st Qu.:0.0000  
+##  Median :1.0000   Median :1.000   Median :0.0000   Median :1.0000  
+##  Mean   :0.6661   Mean   :1.236   Mean   :0.4622   Mean   :0.9771  
+##  3rd Qu.:1.0000   3rd Qu.:2.000   3rd Qu.:1.0000   3rd Qu.:1.0000  
+##  Max.   :1.0000   Max.   :5.000   Max.   :3.0000   Max.   :5.0000  
+##                                                    NA's   :1       
+##      b4pss4          b5pss5      
+##  Min.   :0.000   Min.   :0.0000  
+##  1st Qu.:0.000   1st Qu.:0.0000  
+##  Median :1.000   Median :0.0000  
+##  Mean   :1.009   Mean   :0.6761  
+##  3rd Qu.:2.000   3rd Qu.:1.0000  
+##  Max.   :3.000   Max.   :3.0000  
+##                  NA's   :1
    +
    ## 'data.frame':    569 obs. of  6 variables:
+##  $ Kin2  : int  0 0 1 1 0 1 1 1 1 1 ...
+##  $ b1pss1: int  1 1 1 1 1 2 1 3 1 1 ...
+##  $ b2pss2: int  1 0 1 0 1 2 1 2 0 0 ...
+##  $ b3pss3: int  1 0 1 1 2 2 1 2 1 1 ...
+##  $ b4pss4: int  1 1 1 1 0 2 2 3 0 1 ...
+##  $ b5pss5: int  1 0 0 0 0 1 2 3 0 0 ...
    +
    +
    +
    +
    +

    7.3.2.2

    +

    Run a single-group CFA wherein the five scale variables described above indicate +a single latent factor.

    +
      +
    • Do not include any grouping variable.
      • +
    • Use the default settings in the cfa() function.
      • +
    +
    + +Click to show code + +
    ## Define the model syntax:
+cfaMod <- 'grief =~ b1pss1 + b2pss2 + b3pss3 + b4pss4 + b5pss5'
+
+## Estimate the model:
+out0 <- cfa(cfaMod, data = pgd)
    +
    +
    +
    +
    +

    7.3.2.3

    +

    Summarize the evaluate the fitted CFA

    +
      +
    • Does the model fit well?
      • +
    • Are the items homogeneously associated with the latent factor?
      • +
    • Which item is most weakly associated with the latent factor?
      • +
    +
    + +Click to show code + +
    ## Summarize the fitted model:
+summary(out0, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE)
    +
    ## lavaan 0.6.16 ended normally after 19 iterations
+## 
+##   Estimator                                         ML
+##   Optimization method                           NLMINB
+##   Number of model parameters                        10
+## 
+##                                                   Used       Total
+##   Number of observations                           567         569
+## 
+## Model Test User Model:
+##                                                       
+##   Test statistic                                 8.110
+##   Degrees of freedom                                 5
+##   P-value (Chi-square)                           0.150
+## 
+## Model Test Baseline Model:
+## 
+##   Test statistic                               775.364
+##   Degrees of freedom                                10
+##   P-value                                        0.000
+## 
+## User Model versus Baseline Model:
+## 
+##   Comparative Fit Index (CFI)                    0.996
+##   Tucker-Lewis Index (TLI)                       0.992
+## 
+## Loglikelihood and Information Criteria:
+## 
+##   Loglikelihood user model (H0)              -3219.918
+##   Loglikelihood unrestricted model (H1)      -3215.863
+##                                                       
+##   Akaike (AIC)                                6459.836
+##   Bayesian (BIC)                              6503.240
+##   Sample-size adjusted Bayesian (SABIC)       6471.495
+## 
+## Root Mean Square Error of Approximation:
+## 
+##   RMSEA                                          0.033
+##   90 Percent confidence interval - lower         0.000
+##   90 Percent confidence interval - upper         0.073
+##   P-value H_0: RMSEA <= 0.050                    0.710
+##   P-value H_0: RMSEA >= 0.080                    0.023
+## 
+## Standardized Root Mean Square Residual:
+## 
+##   SRMR                                           0.018
+## 
+## Parameter Estimates:
+## 
+##   Standard errors                             Standard
+##   Information                                 Expected
+##   Information saturated (h1) model          Structured
+## 
+## Latent Variables:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##   grief =~                                                              
+##     b1pss1            1.000                               0.752    0.759
+##     b2pss2            0.454    0.043   10.570    0.000    0.341    0.495
+##     b3pss3            0.831    0.058   14.445    0.000    0.625    0.691
+##     b4pss4            0.770    0.055   14.010    0.000    0.579    0.667
+##     b5pss5            0.817    0.057   14.410    0.000    0.614    0.689
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##    .b1pss1            0.416    0.037   11.300    0.000    0.416    0.424
+##    .b2pss2            0.358    0.023   15.549    0.000    0.358    0.755
+##    .b3pss3            0.427    0.033   13.117    0.000    0.427    0.522
+##    .b4pss4            0.419    0.031   13.599    0.000    0.419    0.555
+##    .b5pss5            0.417    0.032   13.160    0.000    0.417    0.525
+##     grief             0.565    0.059    9.514    0.000    1.000    1.000
+## 
+## R-Square:
+##                    Estimate
+##     b1pss1            0.576
+##     b2pss2            0.245
+##     b3pss3            0.478
+##     b4pss4            0.445
+##     b5pss5            0.475
    +
    + +Click for explanation + +
      +

    • The model fits the data quite well (\(\chi^2[5] = 8.11\), +\(p = 0.15\), +\(\textit{RMSEA} = 0.033\), +\(\textit{CFI} = 0.996\), +\(\textit{SRMR} = 0.018\)).

      • +

    • All of the indicators appear to be more-or-less equally good indicators of the +latent factor except for b2pss2 which has a standardized factor loading of +\(\lambda = 0.495\) and \(R^2 = 0.245\).

      +
    + +
    +
    +
    +
    +

    7.3.2.4

    +

    Rerun the CFA from 7.3.2.2 as a multiple-group model.

    +
      +
    • Use the Kin2 variable as the grouping factor.
      • +
    • Do not place any equality constraints across groups.
      • +
    +
    + +Click to show code + +
    out1 <- cfa(cfaMod, data = pgd, group = "Kin2")
    +
    +
    +
    +
    +

    7.3.2.5

    +

    Summarize the fitted multiple-group CFA from 7.3.2.4.

    +
      +
    • Does the two-group model fit the data well?
      • +
    • Do you notice any salient differences between the two sets of within-group +estimates?
      • +
    +
    + +Click to show code + +
    summary(out1, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE)
    +
    ## lavaan 0.6.16 ended normally after 27 iterations
+## 
+##   Estimator                                         ML
+##   Optimization method                           NLMINB
+##   Number of model parameters                        30
+## 
+##   Number of observations per group:               Used       Total
+##     0                                              188         190
+##     1                                              379         379
+## 
+## Model Test User Model:
+##                                                       
+##   Test statistic                                11.317
+##   Degrees of freedom                                10
+##   P-value (Chi-square)                           0.333
+##   Test statistic for each group:
+##     0                                            8.976
+##     1                                            2.340
+## 
+## Model Test Baseline Model:
+## 
+##   Test statistic                               781.358
+##   Degrees of freedom                                20
+##   P-value                                        0.000
+## 
+## User Model versus Baseline Model:
+## 
+##   Comparative Fit Index (CFI)                    0.998
+##   Tucker-Lewis Index (TLI)                       0.997
+## 
+## Loglikelihood and Information Criteria:
+## 
+##   Loglikelihood user model (H0)              -3206.363
+##   Loglikelihood unrestricted model (H1)      -3200.705
+##                                                       
+##   Akaike (AIC)                                6472.727
+##   Bayesian (BIC)                              6602.937
+##   Sample-size adjusted Bayesian (SABIC)       6507.701
+## 
+## Root Mean Square Error of Approximation:
+## 
+##   RMSEA                                          0.022
+##   90 Percent confidence interval - lower         0.000
+##   90 Percent confidence interval - upper         0.070
+##   P-value H_0: RMSEA <= 0.050                    0.789
+##   P-value H_0: RMSEA >= 0.080                    0.018
+## 
+## Standardized Root Mean Square Residual:
+## 
+##   SRMR                                           0.017
+## 
+## Parameter Estimates:
+## 
+##   Standard errors                             Standard
+##   Information                                 Expected
+##   Information saturated (h1) model          Structured
+## 
+## 
+## Group 1 [0]:
+## 
+## Latent Variables:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##   grief =~                                                              
+##     b1pss1            1.000                               0.702    0.712
+##     b2pss2            0.372    0.076    4.922    0.000    0.261    0.410
+##     b3pss3            0.938    0.118    7.986    0.000    0.659    0.709
+##     b4pss4            0.909    0.116    7.848    0.000    0.638    0.691
+##     b5pss5            0.951    0.122    7.774    0.000    0.667    0.683
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##    .b1pss1            1.346    0.072   18.727    0.000    1.346    1.366
+##    .b2pss2            0.441    0.046    9.499    0.000    0.441    0.693
+##    .b3pss3            1.059    0.068   15.618    0.000    1.059    1.139
+##    .b4pss4            1.122    0.067   16.671    0.000    1.122    1.216
+##    .b5pss5            0.745    0.071   10.442    0.000    0.745    0.762
+##     grief             0.000                               0.000    0.000
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##    .b1pss1            0.478    0.067    7.118    0.000    0.478    0.493
+##    .b2pss2            0.338    0.037    9.205    0.000    0.338    0.832
+##    .b3pss3            0.430    0.060    7.170    0.000    0.430    0.498
+##    .b4pss4            0.445    0.060    7.408    0.000    0.445    0.522
+##    .b5pss5            0.511    0.068    7.519    0.000    0.511    0.534
+##     grief             0.493    0.098    5.007    0.000    1.000    1.000
+## 
+## R-Square:
+##                    Estimate
+##     b1pss1            0.507
+##     b2pss2            0.168
+##     b3pss3            0.502
+##     b4pss4            0.478
+##     b5pss5            0.466
+## 
+## 
+## Group 2 [1]:
+## 
+## Latent Variables:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##   grief =~                                                              
+##     b1pss1            1.000                               0.769    0.778
+##     b2pss2            0.502    0.052    9.597    0.000    0.386    0.542
+##     b3pss3            0.785    0.066   11.945    0.000    0.604    0.680
+##     b4pss4            0.708    0.062   11.497    0.000    0.544    0.652
+##     b5pss5            0.762    0.062   12.185    0.000    0.586    0.696
+## 
+## Intercepts:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##    .b1pss1            1.182    0.051   23.277    0.000    1.182    1.196
+##    .b2pss2            0.475    0.037   12.973    0.000    0.475    0.666
+##    .b3pss3            0.934    0.046   20.460    0.000    0.934    1.051
+##    .b4pss4            0.955    0.043   22.270    0.000    0.955    1.144
+##    .b5pss5            0.644    0.043   14.879    0.000    0.644    0.764
+##     grief             0.000                               0.000    0.000
+## 
+## Variances:
+##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
+##    .b1pss1            0.385    0.043    8.862    0.000    0.385    0.394
+##    .b2pss2            0.359    0.029   12.468    0.000    0.359    0.706
+##    .b3pss3            0.425    0.039   11.025    0.000    0.425    0.538
+##    .b4pss4            0.401    0.035   11.420    0.000    0.401    0.575
+##    .b5pss5            0.366    0.034   10.767    0.000    0.366    0.516
+##     grief             0.592    0.073    8.081    0.000    1.000    1.000
+## 
+## R-Square:
+##                    Estimate
+##     b1pss1            0.606
+##     b2pss2            0.294
+##     b3pss3            0.462
+##     b4pss4            0.425
+##     b5pss5            0.484
    +
    + +Click for explanation + +
      +

    • The two-group model also fits the data very well (\(\chi^2[10] = 11.32\), +\(p = 0.333\), +\(\textit{RMSEA} = 0.022\), +\(\textit{CFI} = 0.998\), +\(\textit{SRMR} = 0.017\)).

      • +

    • No, there are no striking differences between the two sets of estimates. Although, +there is certainly some variability between groups, the two sets of estimates +don’t look systematically different.

      +
    + +
    +
    +
    +
    +

    7.3.2.6

    +

    Based on the above results, what can you conclude about configural, weak, and +strong measurement invariance across the Kin2 groups?

    +
    + +Click for explanation + +
      +
    • Configural invariance holds. The unrestricted multiple-group CFA fits the data +adequately (very well, actually), and the measurement model parameters are +reasonable in both groups.
      • +
    • We cannot yet draw any conclusions about weak or strong invariance. We need to +do the appropriate model comparison tests first.
      • +
    +
    +
    +

    End of At-Home Exercises 7

    +
    +
    +
    diff --git a/docs/at-home-exercises.html b/docs/at-home-exercises.html index e053083f..a0d81369 100644 --- a/docs/at-home-exercises.html +++ b/docs/at-home-exercises.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/attendance.html b/docs/attendance.html index e8dc6d74..7e2a74bf 100644 --- a/docs/attendance.html +++ b/docs/attendance.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/cfa.html b/docs/cfa.html index 2d5b4aed..1f29f109 100644 --- a/docs/cfa.html +++ b/docs/cfa.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/course-data.html b/docs/course-data.html index 68e0f9ed..5d45cc54 100644 --- a/docs/course-data.html +++ b/docs/course-data.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/course-overview.html b/docs/course-overview.html index 120cfeed..a9f8c8d7 100644 --- a/docs/course-overview.html +++ b/docs/course-overview.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/data_options.html b/docs/data_options.html index 17164216..9c084326 100644 --- a/docs/data_options.html +++ b/docs/data_options.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/efa.html b/docs/efa.html index 1f6aac55..a8f6bb6f 100644 --- a/docs/efa.html +++ b/docs/efa.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/elaboration-tips.html b/docs/elaboration-tips.html index 8ef3b46e..7b664e75 100644 --- a/docs/elaboration-tips.html +++ b/docs/elaboration-tips.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/full-sem.html b/docs/full-sem.html index 0d2670ee..a4ba15aa 100644 --- a/docs/full-sem.html +++ b/docs/full-sem.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/grading-1.html b/docs/grading-1.html index 7013f8f1..9c17be02 100644 --- a/docs/grading-1.html +++ b/docs/grading-1.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/grading.html b/docs/grading.html index f695584e..e9b1b02f 100644 --- a/docs/grading.html +++ b/docs/grading.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/in-class-exercises-1.html b/docs/in-class-exercises-1.html index 76983644..ee2a1622 100644 --- a/docs/in-class-exercises-1.html +++ b/docs/in-class-exercises-1.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/in-class-exercises-2.html b/docs/in-class-exercises-2.html index dde403ae..744b6938 100644 --- a/docs/in-class-exercises-2.html +++ b/docs/in-class-exercises-2.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/in-class-exercises-3.html b/docs/in-class-exercises-3.html index b95b394e..35c41b12 100644 --- a/docs/in-class-exercises-3.html +++ b/docs/in-class-exercises-3.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/in-class-exercises-4.html b/docs/in-class-exercises-4.html index 68814973..46f4c4fc 100644 --- a/docs/in-class-exercises-4.html +++ b/docs/in-class-exercises-4.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/in-class-exercises-5.html b/docs/in-class-exercises-5.html index 3da2373e..a94a42b9 100644 --- a/docs/in-class-exercises-5.html +++ b/docs/in-class-exercises-5.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/in-class-exercises-6.html b/docs/in-class-exercises-6.html index 807dba07..148f8766 100644 --- a/docs/in-class-exercises-6.html +++ b/docs/in-class-exercises-6.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/in-class-exercises.html b/docs/in-class-exercises.html index d81bed72..c6b62f58 100644 --- a/docs/in-class-exercises.html +++ b/docs/in-class-exercises.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/index.html b/docs/index.html index b41a5810..93ee55d0 100644 --- a/docs/index.html +++ b/docs/index.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • @@ -464,7 +468,7 @@

    Course Information

    diff --git a/docs/installing-software.html b/docs/installing-software.html index 7be55ab7..dd2bad40 100644 --- a/docs/installing-software.html +++ b/docs/installing-software.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/instructors.html b/docs/instructors.html index a8ebd9a1..6283e7ea 100644 --- a/docs/instructors.html +++ b/docs/instructors.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/introduction-to-r.html b/docs/introduction-to-r.html index ff1b361c..ad6fa3f8 100644 --- a/docs/introduction-to-r.html +++ b/docs/introduction-to-r.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/learning-goals.html b/docs/learning-goals.html index db0b069f..e7631da8 100644 --- a/docs/learning-goals.html +++ b/docs/learning-goals.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/lecture-1.html b/docs/lecture-1.html index 5fe4af97..53dff1ab 100644 --- a/docs/lecture-1.html +++ b/docs/lecture-1.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/lecture-2.html b/docs/lecture-2.html index 6ea92973..0dcee48a 100644 --- a/docs/lecture-2.html +++ b/docs/lecture-2.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/lecture-3.html b/docs/lecture-3.html index d7fb3ece..5b66aff8 100644 --- a/docs/lecture-3.html +++ b/docs/lecture-3.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/lecture-4.html b/docs/lecture-4.html index 40ea6626..2bcf3820 100644 --- a/docs/lecture-4.html +++ b/docs/lecture-4.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/lecture-5.html b/docs/lecture-5.html index d1efae43..647d4fe0 100644 --- a/docs/lecture-5.html +++ b/docs/lecture-5.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/lecture-6.html b/docs/lecture-6.html index 870fd6d6..7cebb970 100644 --- a/docs/lecture-6.html +++ b/docs/lecture-6.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/lecture.html b/docs/lecture.html index afe5a3c5..8a633307 100644 --- a/docs/lecture.html +++ b/docs/lecture.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/mediation-moderation.html b/docs/mediation-moderation.html index c64de3c9..fc48cb1b 100644 --- a/docs/mediation-moderation.html +++ b/docs/mediation-moderation.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/multiple-group-models.html b/docs/multiple-group-models.html index 76ecac56..de92d69f 100644 --- a/docs/multiple-group-models.html +++ b/docs/multiple-group-models.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/note-on-data-updates.html b/docs/note-on-data-updates.html index 69185570..e7567613 100644 --- a/docs/note-on-data-updates.html +++ b/docs/note-on-data-updates.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/procedures.html b/docs/procedures.html index 00e72cdc..cf884aff 100644 --- a/docs/procedures.html +++ b/docs/procedures.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading-1.html b/docs/reading-1.html index b2dce3e6..b31e3bd3 100644 --- a/docs/reading-1.html +++ b/docs/reading-1.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading-2.html b/docs/reading-2.html index bb3292d6..fc641603 100644 --- a/docs/reading-2.html +++ b/docs/reading-2.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading-3.html b/docs/reading-3.html index 7819b8f7..3b28293b 100644 --- a/docs/reading-3.html +++ b/docs/reading-3.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading-4.html b/docs/reading-4.html index cbfd531c..6e64781a 100644 --- a/docs/reading-4.html +++ b/docs/reading-4.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading-5.html b/docs/reading-5.html index 72f7ffd6..8eb1e2b0 100644 --- a/docs/reading-5.html +++ b/docs/reading-5.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading-6.html b/docs/reading-6.html index 43980521..e3c78a8a 100644 --- a/docs/reading-6.html +++ b/docs/reading-6.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading-questions.html b/docs/reading-questions.html index 11debf9a..2feb86f3 100644 --- a/docs/reading-questions.html +++ b/docs/reading-questions.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reading.html b/docs/reading.html index 5319db98..0814f815 100644 --- a/docs/reading.html +++ b/docs/reading.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/reference-keys.txt b/docs/reference-keys.txt index 2e5c0097..23682c3c 100644 --- a/docs/reference-keys.txt +++ b/docs/reference-keys.txt @@ -187,5 +187,19 @@ recordings-3 slides-6 reading-6 at-home-exercises-6 +mgPathAnalysis +section-95 +section-96 +section-97 +fullPaMod +resPaMod +section-98 +mgCFA +section-99 +oneGroupCFA +section-100 +twoGroupCFA +section-101 +section-102 in-class-exercises-6 wrap-up diff --git a/docs/resources.html b/docs/resources.html index 10878e7c..9c10bb0b 100644 --- a/docs/resources.html +++ b/docs/resources.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/rules.html b/docs/rules.html index c839e617..ad7e4552 100644 --- a/docs/rules.html +++ b/docs/rules.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/schedule.html b/docs/schedule.html index 03f72421..7ded0131 100644 --- a/docs/schedule.html +++ b/docs/schedule.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/search_index.json b/docs/search_index.json index 7e49d512..f2e79daf 100644 --- a/docs/search_index.json +++ b/docs/search_index.json @@ -1 +1 @@ -[["index.html", "Theory Construction and Statistical Modeling Course Information", " Theory Construction and Statistical Modeling Kyle M. Lang Last updated: 2023-10-15 Course Information In order to test a theory, we must express the theory as a statistical model and then test this model on quantitative (numeric) data. In this course we will use datasets from different disciplines within the social sciences (educational sciences, psychology, and sociology) to explain and illustrate theories and practices that are used in all social science disciplines to statistically model social science theories. This course uses existing tutorial datasets to practice the process of translating verbal theories into testable statistical models. If you are interested in the methods of acquiring high quality data to test your own theory, we recommend following the course Conducting a Survey which is taught from November to January. Most information about the course is available in this GitBook. Course-related communication will be through https://uu.blackboard.com (Log in with your student ID and password). "],["acknowledgement.html", "Acknowledgement", " Acknowledgement This course was originally developed by dr. Caspar van Lissa. I (dr. Kyle M. Lang) have modified Caspar’s original materials and take full responsibility for any errors or inaccuracies introduced through these modifications. Credit for any particularly effective piece of pedagogy should probably go to Caspar. You can view the original version of this course here on Caspar’s GitHub page. "],["instructors.html", "Instructors", " Instructors Coordinator: dr. Kyle M. Lang Lectures: dr. Kyle M. Lang Practicals: Rianne Kraakman Daniëlle Remmerswaal Danielle McCool "],["course-overview.html", "Course overview", " Course overview This course comprises three parts: Path analysis: You will learn how to estimate complex path models of observed variables (e.g., linked linear regressions) as structural equation models. Factor analysis: You will learn different ways of defining and estimating latent (unobserved) constructs. Full structural equation modeling: You will combine the first two topics to estimate path models describing the associations among latent constructs. Each of these three themes will be evaluated with a separate assignment. The first two assignments will be graded on a pass/fail basis. Your course grade will be based on your third assignment grade. "],["schedule.html", "Schedule", " Schedule Course Week Calendar Week Lecture/Practical Topic Workgroup Activity Assignment Deadline 0 36 Pre-course preparation 1 37 Introduction to R 2 38 Statistical modeling, Path analysis 3 39 Mediation, Moderation 4 40 Exploratory factor analysis (EFA) A1 Peer-Review A1: 2023-10-04 @ 23:59 5 41 Confirmatory factor analysis (CFA) 6 42 Structural equation modeling (SEM) A2 Peer-Review A2: 2023-10-18 @ 23:59 7 43 Multiple group models 8 44 Wrap-up A3 Peer-Review 9 45 Exam week: No class meetings A3: 2023-11-10 @ 23:59 NOTE: The schedule (including topics covered and assignment deadlines) is subject to change at the instructors’ discretion. "],["learning-goals.html", "Learning goals", " Learning goals In this course you will learn how to translate a social scientific theory into a statistical model, how to analyze your data with these models, and how to interpret and report your results following APA standards. After completing the course, you will be able to: Translate a verbal theory into a conceptual model, and translate a conceptual model into a statistical model. Independently analyze data using the free, open-source statistical software R. Apply a latent variable model to a real-life problem wherein the observed variables are only indirect indicators of an unobserved construct. Use a path model to represent the hypothesized causal relations among several variables, including relationships such as mediation and moderation. Explain to a fellow student how structural equation modeling combines latent variable models with path models and the benefits of doing so. Reflect critically on the decisions involved in defining and estimating structural equation models. "],["resources.html", "Resources", " Resources Literature You do not need a separate book for this course! Most of the information is contained within this GitBook and the course readings (which you will be able to access via links in this GitBook). All literature is freely available online, as long as you are logging in from within the UU-domain (i.e., from the UU campus or through an appropriate VPN). All readings are linked in this GitBook via either direct download links or DOIs. If you run into any trouble accessing a given article, searching for the title using Google Scholar or the University Library will probably due the trick. Software You will do all of your statistical analyses with the statistical programming language/environment R and the add-on package lavaan. If you want to expand your learning, you can follow this optional lavaan tutorial. "],["reading-questions.html", "Reading questions", " Reading questions Along with every article, we will provide reading questions. You will not be graded on the reading questions, but it is important to prepare the reading questions before every lecture. The reading questions serve several important purposes: Provide relevant background knowledge for the lecture Help you recognize and understand the key terms and concepts Make you aware of important publications that shaped the field Help you extract the relevant insights from the literature "],["weekly-preparation.html", "Weekly preparation", " Weekly preparation Before every class meeting (both lectures and practicals) you need to do the assigned homework (delineated in the GitBook chapter for that week). This course follows a flipped classroom procedure, so you must complete the weekly homework to meaningfully participate in, and benefit from, the class meetings. Background knowledge We assume you have basic knowledge about multivariate statistics before entering this course. You do not need any prior experience working with R. If you wish to refresh your knowledge, we recommend the chapters on ANOVA, multiple regression, and exploratory factor analysis from Field’s Discovering Statistics using R. If you cannot access the Field book, many other introductory statistics textbooks cover these topics equally well. So, use whatever you have lying around from past statistics courses. You could also try one of the following open-access options: Applied Statistics with R Introduction to Modern Statistics Introduction to Statistical Learning "],["grading.html", "Grading", " Grading Your grade for the course is based on a “portfolio” composed of the three take-home assignments: Path modeling Deadline: Wednesday 2023-10-04 at 23:59 Group assignment Pass/Fail Confirmatory factor analysis Deadline: Wednesday 2023-10-18 at 23:59 Group assignment Pass/Fail Full structural equation modeling Deadline: Friday 2023-11-10 at 23:59 Individual assignment Comprises your entire numeric course grade The specifics of the assignments will be explicated in the Assignments chapter of this GitBook "],["attendance.html", "Attendance", " Attendance Attendance is not mandatory, but we strongly encourage you to attend all lectures and practicals. In our experience, students who actively participate tend to pass the course, whereas those who do not participate tend to drop out or fail. The lectures and practicals build on each other, so, in the unfortunate event that you have to miss a class meeting, please make sure you have caught up with the material before the next session. "],["assignments.html", "Assignments", " Assignments This chapter contains the details and binding information about the three assignments that comprise the portfolio upon which your course grade is based. Below, you can find a brief idea of what each assignment will cover. For each assignment, you will use R to analyze some real-world data, and you will write up your results in a concise report (not a full research paper). Guidelines for these analyses/reports are delineated in the following three sections. You will submit your reports via Blackboard. You will complete the first two assignments in your Assignment Groups. You will complete the third assignment individually. The first two assignments are graded as pass/fail. You must pass both of these assignments to pass the course. The third assignment constitutes your course grade. "],["assignment-1-path-analysis.html", "Assignment 1: Path Analysis", " Assignment 1: Path Analysis For the first assignment, you will work in groups to apply a path model that describes how several variables could be causally related. The components of the first assignment are described below. Choose a suitable dataset, and describe the data. You can use any of the 8 datasets linked below. State the research question; define and explicate the theoretical path model. This model must include, at least, three variables. Use a path diagram to show your theoretical model. Translate your theoretical path model into lavaan syntax, and estimate the model. Include the code used to define and estimate your model as an appendix. Explain your rationale for important modeling decisions. Discuss the conceptual fit between your theory and your model. Evaluate the model assumptions. Discuss other important decisions that could have influence your results. Report the results in APA style. Provide relevant output in a suitable format. Include measures of explained variance for the dependent variables. Discuss the results. Use your results to answer the research question. Consider the strengths and limitations of your analysis. Evaluation See the Grading section below for more information on how Assignment 1 will be evaluated. You can access an evaluation matrix for Assignment 1 here. This matrix gives an indication of what level of work constitutes insufficient, sufficient, and excellent responses to the six components described above. Submission Assignment 1 is due at 23:59 on Wednesday 4 October 2023. Submit your report via the Assignment 1 portal on Blackboard. "],["assignment-2-confirmatory-factor-analysis.html", "Assignment 2: Confirmatory Factor Analysis", " Assignment 2: Confirmatory Factor Analysis In the second assignment, you will work in groups to run a CFA wherein the observed variables are indirect indicators of the unobserved constructs you want to analyze. The components of the second assignment are described below. Choose a suitable dataset, and describe the data. Ideally, you will work with the same data that you analyzed in Assignment 1. If you want to switch, you can use any of the 8 datasets linked below. State the research question; define and explicate the theoretical CFA model. This model must include, at least, two latent constructs. Use a path diagram to represent your model. Translate your theoretical model into lavaan syntax, and estimate the model. Include the code used to define and estimate your model as an appendix. Explain your rationale for important modeling decisions. Discuss the conceptual fit between your theory and your model. Evaluate the model assumptions. Discuss other important decisions that could have influence your results. Report the results in APA style. Provide relevant output in a suitable format. Include measures of model fit. Discuss the results. Use your results to answer the research question. Consider the strengths and limitations of your analysis. Evaluation See the Grading section below for more information on how Assignment 2 will be evaluated. You can access an evaluation matrix for Assignment 2 here. This matrix gives an indication of what level of work constitutes insufficient, sufficient, and excellent responses to the six components described above. Submission Assignment 2 is due at 23:59 on Wednesday 18 October 2023. Submit your report via the Assignment 2 portal on Blackboard. "],["a3_components.html", "Assignment 3: Full Structural Equation Model", " Assignment 3: Full Structural Equation Model In the third assignment, you will work individually to apply a full SEM that describes how several (latent) variables could be causally related. The components of the third assignment are described below. Choose a suitable dataset, and describe the data. Ideally, you will work with the same data that you analyzed in Assignments 1 & 2. If you want to switch, you can use any of the 8 datasets linked below. State the research question; define and explicate the theoretical SEM. The structural component of this model must include, at least, three variables. The model must include, at least, two latent variables. Use a path diagram to represent your model. Translate your theoretical SEM into lavaan syntax, and estimate the model. Include the code used to define and estimate your model as an appendix. Explain your rationale for important modeling decisions. Discuss the conceptual fit between your theory and your model. Evaluate the model assumptions. Discuss other important decisions that could have influence your results. Report the results in APA style. Provide relevant output in a suitable format. Include measures of model fit. Include measures of explained variance for the dependent variables. Discuss the results. Use your results to answer the research question. Consider the strengths and limitations of your analysis. Evaluation See the Grading section below for more information on how the component scores represented in the rubric are combined into an overall assignment grade. You can access an evaluation matrix for Assignment 3 here. This matrix gives an indication of what level of work constitutes insufficient, sufficient, and excellent responses to the six components described above. Submission Assignment 3 is due at 23:59 on Friday 10 November 2023. Submit your report via the Assignment 3 portal on Blackboard. "],["elaboration-tips.html", "Elaboration & Tips", " Elaboration & Tips Theoretical Model & Research Question You need to provide some justification for your model and research question, but only enough to demonstrate that you’ve actually conceptualized and estimated a theoretically plausible statistical model (as opposed to randomly combining variables until lavaan returns a pretty picture). You have several ways to show that your model is plausible. Use common-sense arguments. Reference (a small number of) published papers. Replicate an existing model/research question. Don’t provide a rigorous literature-supported theoretical motivation. You don’t have the time to conduct a thorough literature review, and we don’t have the time to read such reviews when grading. Literature review is not one of the learning goals for this course, so you cannot get “bonus points” for an extensive literature review. You are free to test any plausible model that meets the size requirements. You can derive your own model/research question or you can replicate a published analysis. Model Specifications We will not cover methods for modeling categorical outcome variables. So, use only continuous variables as outcomes. DVs in path models and the structural parts of SEMs Observed indicators of latent factors in CFA/SEM NOTE: You may treat ordinal items as continuous, for the purposes of these assignments. We will not cover methods for latent variable interactions. Don’t specify a theoretical model that requires an interaction involving a latent construct. There is one exception to the above prohibition. If the moderator is an observed grouping variable, you can estimate the model as a multiple-group model. We’ll cover these methods in Week 7. Assumptions You need to show that you’re thinking about the assumptions and their impact on your results, but you don’t need to run thorough model diagnostics. Indeed, the task of checking assumptions isn’t nearly as straight forward in path analysis, CFA, and SEM as it is in linear regression modeling. You won’t be able to directly apply the methods you have learned for regression diagnostics, for example. Since all of our models are estimated with normal-theory maximum likelihood, the fundamental assumption of all the models we’ll consider in this course boils down to the following. All random variables in my model are i.i.d. multivariate normally distributed. So, you can get by with basic data screening and checking the observed random variables in your model (i.e., all variables other than fixed predictors) for normality. Since checking for multivariate normality is a bit tricky, we’ll only ask you to evaluate univariate normality. You should do these evaluations via graphical means. To summarize, we’re looking for the following. Data Consider whether the measurement level of your data matches the assumptions of your model. Check your variables for univariate outliers. If you find any outliers, either treat them in some way or explain why you are retaining them for the analysis. Check for missing data. For the purposes of the assignment, you can use complete case analysis to work around the missing data. If you’re up for more of a challenge, feel free to try multiple imputation or full information maximum likelihood. Model Evaluate the univariate normality of any random, observed variables in your model. E.g., DVs in path models, observed IVs modeled as random variables, indicators of latent factors If you fit a multiple-group model for Assignment 3, do this evaluation within groups. Use graphical tools to evaluate the normality assumption. Normal QQ-Plots Histograms Results What do we mean by reporting your results “in a suitable format”? Basically, put some effort into making your results readable, and don’t include a bunch of superfluous information. Part of demonstrating that you understand the analysis is showing that you know which pieces of output convey the important information. Tabulate your results; don’t directly copy the R output. Don’t include everything lavaan gives you. Include only the output needed to understand your results and support your conclusions. "],["data_options.html", "Data", " Data Below, you can find links to a few suitable datasets that you can use for the assignments. You must use one of the following datasets. You may not choose your own data from the wild. Coping with Covid Dataset Codebook Pre-Registration Feminist Perspectives Scale Dataset Article Hypersensitive Narcissism Scale & Dirty Dozen Dataset HSNS Article DD Article Kentucky Inventory of Mindfulness Skills Dataset Article Depression Anxiety Stress Scale Dataset DASS Information Nomophobia Dataset Recylced Water Acceptance Dataset Article "],["procedures.html", "Procedures", " Procedures Formatting You must submit your assignment reports in PDF format. Each report should include a title page. The title page should include the following information: The name of the assignment. The names of all assignment authors (i.e., all group members for Assignments 1 & 2, your name for Assignment 3). The Assignment Group number (only for Assignments 1 & 2). You must include the code used to define and run your model(s) as an appendix. Try to format the text in this appendix clearly. Use a monospace font. Length You may use as many words as necessary to adequately explain yourself; though, concision and parsimony are encouraged. Note that the assignments are not intended to be full-blown papers! The focus should be on the definition of your model, how this model relates to theory (introduction), and what you have learned from your estimated model (discussion). For each of the assignments, you should be able to get the job done in fewer than 10 pages of text (excluding title page, figures, appendices, and references). Submission You will submit your reports through Blackboard. Each assignment has a corresponding item in the “Assignments” section of the BB page through which you will submit your reports. For Assignments 1 & 2, you may only submit one report per group. Designate one group member to submit the report. The grade for this submission will apply to all group members. If something goes wrong with the submission, or you notice a mistake (before the deadline) that you want to correct, you may upload a new version of your report. We will grade the final submitted version. The submissions will be screened with Ouriginal. "],["grading-1.html", "Grading", " Grading Group Assignments Assignments 1 & 2 are simply graded as pass/fail. To pass, your submission must: Do a reasonable job of addressing the relevant components listed above Be submitted before the deadline Otherwise, you will fail the assignment. Individual Assignment Assignment 3 will be fully graded on the usual 10-point scale. Points will be allocated according to the extent to which your submission addresses the six components listed above. The evaluation matrix gives an indication of how these points will be apportioned. Further details over the grading procedures for Assignment 3 (e.g., exactly how your 10-point grade will be defined) will be provided at a later date. Assuming your group passes the first two assignments, your final course grade will simply be your Assignment 3 grade. Resits You must get a “pass” for Assignments 1 & 2 and score at least 5.5 on Assignment 3 to pass the course. If you fail any of the assignments, you will have the opportunity to resit the failed assignment(s). If you resit Assignment 3, your revised graded cannot be higher than 6. Further details on the resit procedure will be provided at a later date. Example Assignment You can find an example of a good submission (for an older version of Assignment 2) here. This example is not perfect (no paper ever is), and several points could be improved. That being said, this submission exemplifies what we’re looking for in your project reports. So, following the spirit of this example would earn you a high grade. "],["rules.html", "Rules", " Rules Resources For all three assignments, you may use any reference materials you like, including: All course materials The course GitBook Additional books and papers The internet Collaboration You will complete the first two assignments in groups. Although you will work in groups, your group may not work together with other groups. You will complete the final assignment individually. For this assignment, you may not work with anyone else. For all three assignments, you are obligated to submit original work (i.e., work conducted for this course by you or your group). Submitting an assignment that violates this condition constitutes fraud. Such cases of fraud will be addressed according to the University’s standard policy. Academic integrity Hopefully, you also feel a moral obligation to obey the rules. For this course, we have implemented an examination that allows you to showcase what you have learned in a more realistic way than a written exam would allow. This assessment format spares you the stress of long exams (the two exams for this course used to be 4 hours each) and the attendant studying/cramming. The assignments will also help you assess your ability to independently analyse data, which is important to know for your future courses and/or career. However, this format also assumes that you complete the assignments in good faith. So, I simply ask that you hold up your end of the bargain, and submit your original work to show us what you’ve learned. Strict stuff By submitting your assignments (both group and individual), you confirm the following: You have completed the assignment yourself (or with your group) You are submitting work that you have written yourself (or with your group) You are using your own UU credentials to submit the assignment You have not had outside help that violates the conditions delineated above while completing the assignment All assignments will be submitted via Ouriginal in Blackboard and, thereby, checked for plagiarism. If fraud or plagiarism is detected or suspected, we will inform the Board of Examiners in the usual manner. In the event of demonstrable fraud, the sanctions delineated in Article 5.15 of the Education and Examination Regulations (EER) will apply. "],["software-setup.html", "Software Setup", " Software Setup This chapter will help you prepare for the course by showing how to install R and RStudio on your computer. If you’re already using R, there may be nothing new for you here. That being said, you should look over this chapter to ensure that your current setup will be compatible with the course requirements. If you have never used R before, this chapter is essential! The information is this chapter will be crucial for getting your computer ready for the course. "],["typographic-conventions.html", "Typographic Conventions", " Typographic Conventions Throughout this GitBook, we (try to) use a consistent set of typographic conventions: Functions are typeset in a code font, and the name of the function is always followed by parentheses E.g., sum(), mean() Other R objects (e.g., data objects, function arguments) are in also typeset in a code font but without parentheses E.g., seTE, method.tau Sometimes, we’ll use the package name followed by two colons (::, the so-called *scope-resolution operator), like lavaan::sem(). This command is valid R code and will run if you copy it into your R console. The lavaan:: part of the command tells R that we want to use the sem() from the lavaan package. "],["installing-software.html", "Installing software", " Installing software Before we start the course, we have to install three things: R: A free program for statistical programming RStudio: An integrated development environment (IDE) which makes it easier to work with R. Several packages: Separate pieces of ‘add-on’ software for R with functions to do specific analyses. Packages also include documentation describing how to use their functions and sample data. Installing R The latest version of R is available here. Click the appropriate link for your operating system and follow the instructions for installing the latest stable release. Depending on which OS you select, you may be given an option to install different components (e.g., base, contrib, Rtools). For this course, you will only need the base package. Installing RStudio Download the Free Desktop version of RStudio from the download page of the RStudio website. Installing packages To participate in this course, you will need a few essential R packages. Here’s an overview of the packages and why we need them: Package Description lavaan A sophisticated and user-friendly package for structural equation modeling dplyr A powerful suite of data-processing tools ggplot2 A flexible and user-friendly package for making graphs tidySEM Plotting and tabulating the output of SEM-models semTools Comparing models, establishing measurement invariance across groups psych Descriptive statistics and EFA rockchalk Probing interactions foreign Loading data from SPSS ‘.sav’ files readxl Loading data from Excel ‘.xslx’ files To install these packages, we use the install.packages() function in R. Open RStudio Inside RStudio, find the window named Console on left side of the screen. Copy the following code into the console and hit Enter/Return to run the command. install.packages(c("lavaan", "dplyr", "ggplot2", "tidySEM", "semTools", "psych", "rockchalk", "foreign", "readxl"), dependencies = TRUE) "],["course-data.html", "Course Data", " Course Data All of the data files you will need for the course are available in this SurfDrive directory. Follow the link to download a ZIP archive containing the data you will need to complete the practical exercises and assignments. Extract these data files to a convenient location on your computer. "],["note-on-data-updates.html", "Note on Data Updates", " Note on Data Updates During the course, we may need to update some of these datasets and/or add some new datasets to the SurfDrive directory. If so, you will need to download the updated data. We will let you know if and when any datasets are modified. In such situations, you are responsible for updating your data. Working with outdated data will probably produce incorrect results. Your answer won’t match the solutions we expect. Your answer will be marked as incorrect, even if the code used to produce the answer is correct. Points lost on an assignment due to using outdated datasets will not be returned. "],["introduction-to-r.html", "1 Introduction to R", " 1 Introduction to R This week is all about getting up-and-running with R and RStudio. Homework before the lecture Complete the preparatory material: Read over the Course Information chapter Work through the Software Setup chapter Watch the Lecture Recording for this week. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture.html", "1.1 Lecture", " 1.1 Lecture This week, you will learn the basics of R and RStudio. Rather than re-inventing the proverbial wheel, we’re linked to existing resources developed by R-Ladies Sydney. 1.1.1 Recordings Tour of RStudio \\[\\\\[6pt]\\] R Packages \\[\\\\[6pt]\\] Data I/0 1.1.2 Slides You can access the accompanying resources on the R-Ladies Sydney website here. "],["reading.html", "1.2 Reading", " 1.2 Reading There is no official reading this week. If you’d like to deepen your dive into R, feel free to check out Hadley Wickham’s excellent book R for Data Science. Otherwise, you may want to get a jump-start on the At-Home Exercises for this week. \\[\\\\[12pt]\\] "],["at-home-exercises.html", "1.3 At-Home Exercises", " 1.3 At-Home Exercises This week is all about gaining familiarity with R and RStudio. We’ll be using the primers available on Posit Cloud to work through some basic elements of data visualization and statistical programming in R. Although you should already have R working, this week’s at-home and in-class exercises don’t require that you have R installed on your system. If following along within this GitBook doesn’t work for you, you can also find the tutorials online on the Posit Primers page. 1.3.1 Visualizations with R 1.3.2 Programming with R End of At-Home Exercises "],["in-class-exercises.html", "1.4 In-Class Exercises", " 1.4 In-Class Exercises In the practical this week, we’ll go a little further into what it’s possible with R. Don’t worry if you cannot remember everything in these primers—they’re only meant to familiarize you with what is possible and to get you some experience interacting with R and RStudio. The following primers come from Posit Cloud and were created with the learnr package. 1.4.1 Viewing Data This first primer introduces a special data format called a tibble, as well as some functions for viewing your data. 1.4.2 Dissecting Data In the next primer, we’ll explore tools to subset and rearrange you data: select(), filter(), and arrange(). 1.4.3 Grouping and Manipulating Data Advanced If you made it through the previous two sections with ease and want to challenge yourself, go ahead with this next section. If you’re running short on time, you can skip ahead to Exploratory Data Analysis. \\[\\\\[3pt]\\] 1.4.4 Exploratory Data Analysis 1.4.5 Visualizing Data Visualizing data is a great way to start understanding a data set. In this section, we’ll highlight a few examples of how you can use the ggplot2 libarary to visualize your data. Primers on many other visualizations are available on Posit Cloud. Bar Charts for Categorical Variables Scatterplots for Continuous Variables 1.4.6 Tidying Data This primer will provide an overview of what’s meant by “tidy data”. You only need to complete the Tidy Data section—the sections on Gathering and Spreading columns are useful, but we won’t ask you to apply those techniques in this course. Recap Hopefully, you now feel more comfortable using some of R’s basic functionality and packages to work with data. Here’s a brief description of the functions covered above: install.packages() for installing packages Remember to put the package names in quotes library() for loading packages View() for viewing your dataset select() for picking only certain columns filter() for picking only certain rows arrange() for changing the rows order %>% aka “the pipe” for chaining commands together In RStudio, you can hit ctrl+shift+m as a handy key combination ? for help files Logical tests and Boolean operators == equal to != not equal to < less than <= less than or equal to > greater than >= greater than or equal to is.na() is the value NA (not available) !is.na is the value not NA & and (true only if the left and right are both true) | or (true if either the left or right are true) ! not (invert true/false) %in% in (is left in the larger set of right values) any() any (true if any in the set are true) all() all (true if all in the set are true) xor() xor (true if one and only one of the set are true) ggplot2 ggplot() create the basic object from which to building a plot aes() contains the aesthetic mappings (like x and y) geom_bar() bar plots for distributions of categorical variables geom_point() scatterplots for plotting two continuous variables geom_label_repel() for plotting text facet_wrap() for creating sets of conditional plots End of In-Class Exercises "],["statistical-modeling-path-analysis.html", "2 Statistical Modeling & Path Analysis", " 2 Statistical Modeling & Path Analysis This week, we will cover statistical modeling and path analysis. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-1.html", "2.1 Lecture", " 2.1 Lecture In this lecture, we will begin by discussing the paradigm and contextualizing statistical modeling relative to other ways that we can conduct statistical analyses. We will conclude with an introduction to . 2.1.1 Recordings Statistical Reasoning Statistical Modeling Path Analysis 2.1.2 Slides You can download the lectures slides here "],["reading-1.html", "2.2 Reading", " 2.2 Reading Reference Smaldino, P. E. (2017). Models are stupid, and we need more of them. In R.R. Vallacher, S.J. Read, & A. Nowakt (Eds.), Computational Social Psychology (pp. 311–331). New York: Routledge. SKIP PAGES 322 - 327 Questions What are the differences between a “verbal model” and a “formal model”? As explained in the paragraph “A Brief Note on Statistical Models”, formal models are not the same as statistical models. Still, we can learn a lot from Smaldino’s approach. Write down three insights from this paper that you would like to apply to your statistical modeling during this course. Suggested Reading (Optional) The following paper is not required, but it’s definitely worth a read. Breiman provides a very interesting perspective on different ways to approach a modeling-based analysis. Breiman, L. (2001). Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statistical Science, 16(3) 199–231. https://doi.org/10.1214/ss/1009213726 "],["at-home-exercises-1.html", "2.3 At-Home Exercises", " 2.3 At-Home Exercises Load the LifeSat.sav data. library(dplyr) library(haven) LifeSat <- read_spss("LifeSat.sav") 2.3.1 Make a table of descriptive statistics for the variables: LifSat, educ, ChildSup, SpouSup, and age. What is the average age in the sample? What is the range (youngest and oldest child)? Hint: Use the tidySEM::descriptives() function.` Click for explanation The package tidySEM contains the descriptives() function for computing descriptive statistics. The describe() function in the psych package is a good alternative. library(tidySEM) descriptives(LifeSat[ , c("LifSat", "educ", "ChildSup", "SpouSup", "age")]) 2.3.2 Run a simple linear regression with LifSat as the dependent variable and educ as the independent variable. Hints: The lm() function (short for linear model) does linear regression. The summary() function provides relevant summary statistics for the model. It can be helpful to store the results of your analysis in an object. Click for explanation results <- lm(LifSat ~ educ, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ educ, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -43.781 -11.866 2.018 12.418 43.018 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 35.184 7.874 4.469 2.15e-05 *** ## educ 3.466 1.173 2.956 0.00392 ** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 17.64 on 96 degrees of freedom ## Multiple R-squared: 0.08344, Adjusted R-squared: 0.0739 ## F-statistic: 8.74 on 1 and 96 DF, p-value: 0.003918 2.3.3 Repeat the analysis from 2.3.2 with age as the independent variable. Click for explanation results <- lm(LifSat ~ age, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ age, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -35.321 -14.184 3.192 13.593 40.626 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 200.2302 52.1385 3.840 0.00022 *** ## age -2.0265 0.7417 -2.732 0.00749 ** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 17.75 on 96 degrees of freedom ## Multiple R-squared: 0.07215, Adjusted R-squared: 0.06249 ## F-statistic: 7.465 on 1 and 96 DF, p-value: 0.007487 2.3.4 Repeat the analysis from 2.3.2 and 2.3.3 with ChildSup as the independent variable. Click for explanation results <- lm(LifSat ~ ChildSup, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ ChildSup, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -37.32 -12.14 0.66 12.41 44.68 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 37.559 8.342 4.502 1.89e-05 *** ## ChildSup 2.960 1.188 2.492 0.0144 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 17.86 on 96 degrees of freedom ## Multiple R-squared: 0.06076, Adjusted R-squared: 0.05098 ## F-statistic: 6.211 on 1 and 96 DF, p-value: 0.01441 2.3.5 Run a multiple linear regression with LifSat as the dependent variable and educ, age, and ChildSup as the independent variables. Hint: You can use the + sign to add multiple variables to the RHS of your model formula. Click for explanation results <- lm(LifSat ~ educ + age + ChildSup, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ educ + age + ChildSup, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -32.98 -12.56 2.68 11.03 41.91 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 134.9801 53.2798 2.533 0.0130 * ## educ 2.8171 1.1436 2.463 0.0156 * ## age -1.5952 0.7188 -2.219 0.0289 * ## ChildSup 2.4092 1.1361 2.121 0.0366 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 16.92 on 94 degrees of freedom ## Multiple R-squared: 0.1741, Adjusted R-squared: 0.1477 ## F-statistic: 6.603 on 3 and 94 DF, p-value: 0.0004254 2.3.6 Compare the results from 2.3.5 with those from 2.3.2, 2.3.3, and 2.3.4. What do you notice when you compare the estimated slopes for each of the three predictors in the multiple regression model with the corresponding estimates from the simple regression models? "],["in-class-exercises-1.html", "2.4 In-Class Exercises", " 2.4 In-Class Exercises During this practical, you will work through some exercises meant to expand your statistical reasoning skills and improve your understanding of linear models. For this exercise, having some familiarity with regression will be helpful. If you feel like you need to refresh your knowledge in this area, consider the resources listed in the Background knowledge section. Data: You will use the following dataset for these exercises. Sesam.sav 2.4.1 Data Exploration Open the file “Sesam.sav” # Load `dplyr` for data processing: library(dplyr) # Load the `haven` library for reading in SPSS files: library(haven) ## Load the 'Sesam.sav' data ## Use haven::zap_formats() to remove SPSS attributes sesam <- read_sav(file = "Sesam.sav") %>% zap_formats() This file is part of a larger dataset that evaluates the impact of the first year of the Sesame Street television series. Sesame Street is mainly concerned with teaching preschool related skills to children in the 3–5 year age range. The following variables will be used in this exercise: age: measured in months prelet: knowledge of letters before watching Sesame Street (range 0–58) prenumb: knowledge of numbers before watching Sesame Street (range 0–54) prerelat: knowledge of size/amount/position relationships before watching Sesame Street (range 0–17) peabody: vocabulary maturity before watching Sesame Street (range 20–120) postnumb: knowledge of numbers after a year of Sesame Street (range 0–54) Note: Unless stated otherwise, the following questions refer to the sesam data and the above variables. 2.4.1.1 What is the type of each variable? Hint: The output of the str() function should be helpful here. Click to show code ## Examine the data structure: str(sesam) ## tibble [240 × 8] (S3: tbl_df/tbl/data.frame) ## $ id : num [1:240] 1 2 3 4 5 6 7 8 9 10 ... ## $ age : num [1:240] 66 67 56 49 69 54 47 51 69 53 ... ## $ prelet : num [1:240] 23 26 14 11 47 26 12 48 44 38 ... ## $ prenumb : num [1:240] 40 39 9 14 51 33 13 52 42 31 ... ## $ prerelat: num [1:240] 14 16 9 9 17 14 11 15 15 10 ... ## $ peabody : num [1:240] 62 80 32 27 71 32 28 38 49 32 ... ## $ postnumb: num [1:240] 44 39 40 19 54 39 44 51 48 52 ... ## $ gain : num [1:240] 4 0 31 5 3 6 31 -1 6 21 ... ## ..- attr(*, "display_width")= int 10 Click for explanation All variables are numeric. str() uses the abbreviation “num” to indicate a numeric vector. 2.4.1.2 What is the average age in the sample? What is the age range (youngest and oldest child)? Hint: Use tidySEM::descriptives() Click to show code As in the take home exercises, you can use the descriptives() function from the tidySEM package to describe the data: library(tidySEM) descriptives(sesam) Click for explanation We can get the average age from the “mean” column in the table ( 51.5), and the age range from the columns “min” and “max”, (34 and 69 respectively.) 2.4.1.3 What is the average gain in knowledge of numbers? What is the standard deviation of this gain? Hints: You will need to compute the gain and save the change score as a new object. You can then use the base-R functions mean() and sd() to do the calculations. Click to show code Create a new variable that represents the difference between pre- and post-test scores on knowledge of numbers: sesam <- mutate(sesam, ndif = postnumb - prenumb) Compute the mean and SD of the change score: sesam %>% summarise(mean(ndif), sd(ndif)) 2.4.1.4 Create an appropriate visualization of the gain scores you computed in 2.4.1.3. Justify your choice of visualization. Hint: Some applicable visualizations are explained in the Visualizations with R section. Click to show code library(ggplot2) ## Create an empty baseline plot object: p <- ggplot(sesam, aes(x = ndif)) ## Add some appropriate geoms: p + geom_histogram() p + geom_density() p + geom_boxplot() Click for explanation Because the gain score is numeric, we should use something appropriate for showing the distribution of a continuous variable. In this case, we can use either a density plot, or a histogram (remember from the lecture, this is like a density plot, but binned). We can also use a box plot, which can be a concise way to display a lot of information about a variable in a little less space. 2.4.1.5 Create a visualization that provides information about the bivariate relationship between the pre- and post-test number knowledge. Justify your choice of visualization. Describe the relationship based on what you see in your visualization. Hint: Again, the Visualizations with R section may provide some useful insights. Click to show code ## Create a scatterplot of the pre- and post-test number knowledge ggplot(sesam, aes(x = prenumb, y = postnumb)) + geom_point() Click for explanation A scatterplot is a good tool for showing patterns in the way that two continuous variables relate to each other. From it, we can quickly gather information about whether a relationship exists, its direction, its strength, how much variation there is, and whether or not a relationship might be non-linear. Based on this scatterplot, we see a positive relationship between the prior knowledge of numbers and the knowledge of numbers at the end of the study. Children who started with a higher level of numeracy also ended with a higher level of numeracy. There is a considerable amount of variance in the relationship. Not every child increases their numeracy between pre-test and post-test. Children show differing amounts of increase. 2.4.2 Linear Modeling 2.4.2.1 Are there significant, bivariate associations between postnumb and the following variables? age prelet prenumb prerelat peabody Use Pearson correlations to answer this question. You do not need to check the assumptions here (though you would in real life). Hint: The base-R cor.test() function and the corr.test() function from the psych package will both conduct hypothesis tests for a correlation coefficients (the base-R cor() function only computes the coefficients). Click to show code library(psych) ## Test the correlations using psych::corr.test(): sesam %>% select(postnumb, age, prelet, prenumb, prerelat, peabody) %>% corr.test() ## Call:corr.test(x = .) ## Correlation matrix ## postnumb age prelet prenumb prerelat peabody ## postnumb 1.00 0.34 0.50 0.68 0.54 0.52 ## age 0.34 1.00 0.33 0.43 0.44 0.29 ## prelet 0.50 0.33 1.00 0.72 0.47 0.40 ## prenumb 0.68 0.43 0.72 1.00 0.72 0.61 ## prerelat 0.54 0.44 0.47 0.72 1.00 0.56 ## peabody 0.52 0.29 0.40 0.61 0.56 1.00 ## Sample Size ## [1] 240 ## Probability values (Entries above the diagonal are adjusted for multiple tests.) ## postnumb age prelet prenumb prerelat peabody ## postnumb 0 0 0 0 0 0 ## age 0 0 0 0 0 0 ## prelet 0 0 0 0 0 0 ## prenumb 0 0 0 0 0 0 ## prerelat 0 0 0 0 0 0 ## peabody 0 0 0 0 0 0 ## ## To see confidence intervals of the correlations, print with the short=FALSE option ## OR ## library(magrittr) ## Test the correlations using multiple cor.test() calls: sesam %$% cor.test(postnumb, age) ## ## Pearson's product-moment correlation ## ## data: postnumb and age ## t = 5.5972, df = 238, p-value = 5.979e-08 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.2241066 0.4483253 ## sample estimates: ## cor ## 0.3410578 sesam %$% cor.test(postnumb, prelet) ## ## Pearson's product-moment correlation ## ## data: postnumb and prelet ## t = 8.9986, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.4029239 0.5926632 ## sample estimates: ## cor ## 0.5038464 sesam %$% cor.test(postnumb, prenumb) ## ## Pearson's product-moment correlation ## ## data: postnumb and prenumb ## t = 14.133, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.6002172 0.7389277 ## sample estimates: ## cor ## 0.6755051 sesam %$% cor.test(postnumb, prerelat) ## ## Pearson's product-moment correlation ## ## data: postnumb and prerelat ## t = 9.9857, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.4475469 0.6268773 ## sample estimates: ## cor ## 0.5433818 sesam %$% cor.test(postnumb, peabody) ## ## Pearson's product-moment correlation ## ## data: postnumb and peabody ## t = 9.395, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.4212427 0.6067923 ## sample estimates: ## cor ## 0.520128 Click for explanation Yes, based on the p-values (remember that 0 here really means very small, making it less than .05), we would say that there are significant correlations between postnumb and all other variables in the data. (In fact, all variables in the data are significantly correlated with one another.) 2.4.2.2 Do age and prenumb explain a significant proportion of the variance in postnumb? What statistic did you use to justify your conclusion? Interpret the model fit. Use the lm() function to fit your model. Click to show code lmOut <- lm(postnumb ~ age + prenumb, data = sesam) summary(lmOut) ## ## Call: ## lm(formula = postnumb ~ age + prenumb, data = sesam) ## ## Residuals: ## Min 1Q Median 3Q Max ## -38.130 -6.456 -0.456 5.435 22.568 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 7.4242 5.1854 1.432 0.154 ## age 0.1225 0.1084 1.131 0.259 ## prenumb 0.7809 0.0637 12.259 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 9.486 on 237 degrees of freedom ## Multiple R-squared: 0.4592, Adjusted R-squared: 0.4547 ## F-statistic: 100.6 on 2 and 237 DF, p-value: < 2.2e-16 Click for explanation Yes, age and prenumb explain a significant amount of variability in postnumb (\\(R^2 = 0.459\\), \\(F[2, 237] = 100.629\\), \\(p < 0.001\\)). We use the F statistic for the overall test of model fit to support this conclusion. The variables age and prenumb together explain 45.9% of the variability in postnumb. 2.4.2.3 Write the null and alternative hypotheses tested for in 2.4.2.2. Click for explanation Since we are testing for explained variance, our hypotheses concern the \\(R^2\\). \\[ \\begin{align*} H_0: R^2 = 0\\\\ H_1: R^2 > 0 \\end{align*} \\] Note that this is a directional hypotheses because the \\(R^2\\) cannot be negative. 2.4.2.4 Define the model syntax to estimate the model from 2.4.2.2 as a path analysis using lavaan. Click to show code mod <- 'postnumb ~ 1 + age + prenumb' 2.4.2.5 Estimate the path analytic model you defined above. Use the lavaan::sem() function to estimate the model. Click to show code library(lavaan) lavOut1 <- sem(mod, data = sesam) 2.4.2.6 Summarize the fitted model you estimated above. Use the summary() function to summarize the model. Click to show code summary(lavOut1) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 4 ## ## Number of observations 240 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## postnumb ~ ## age 0.123 0.108 1.138 0.255 ## prenumb 0.781 0.063 12.336 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 7.424 5.153 1.441 0.150 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 88.864 8.112 10.954 0.000 In OLS regression, the predictor variables are usually treated as fixed and do not covary. We can easily relax this assumption in path analysis. 2.4.2.7 Re-estimate the path analytic model you defined in 2.4.2.4. Specify the predictors as random, correlated variables. Hint: You can make the predictors random in, at least, two ways: Modify the model syntax to specify the correlation between age and prenumb. Add fixed.x = FALSE to your sem() call. Click to show code lavOut2 <- sem(mod, data = sesam, fixed.x = FALSE) ## OR ## mod <- ' postnumb ~ 1 + age + prenumb age ~~ prenumb ' lavOut2 <- sem(mod, data = sesam) 2.4.2.8 Summarize the fitted model you estimated above. Compare the results to those from the OLS regression in 2.4.2.2 and the path model in 2.4.2.5. Click to show code summary(lavOut2) ## lavaan 0.6.16 ended normally after 26 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 9 ## ## Number of observations 240 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## postnumb ~ ## age 0.123 0.108 1.138 0.255 ## prenumb 0.781 0.063 12.336 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## age ~~ ## prenumb 28.930 4.701 6.154 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 7.424 5.153 1.441 0.150 ## age 51.525 0.405 127.344 0.000 ## prenumb 20.896 0.688 30.359 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 88.864 8.112 10.954 0.000 ## age 39.291 3.587 10.954 0.000 ## prenumb 113.702 10.379 10.954 0.000 summary(lavOut1) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 4 ## ## Number of observations 240 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## postnumb ~ ## age 0.123 0.108 1.138 0.255 ## prenumb 0.781 0.063 12.336 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 7.424 5.153 1.441 0.150 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 88.864 8.112 10.954 0.000 summary(lmOut) ## ## Call: ## lm(formula = postnumb ~ age + prenumb, data = sesam) ## ## Residuals: ## Min 1Q Median 3Q Max ## -38.130 -6.456 -0.456 5.435 22.568 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 7.4242 5.1854 1.432 0.154 ## age 0.1225 0.1084 1.131 0.259 ## prenumb 0.7809 0.0637 12.259 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 9.486 on 237 degrees of freedom ## Multiple R-squared: 0.4592, Adjusted R-squared: 0.4547 ## F-statistic: 100.6 on 2 and 237 DF, p-value: < 2.2e-16 2.4.2.9 Consider the path model below. How many regression coefficients are estimated in this model? How many variances are estimated? How many covariances are estimated? Click for explanation Six regression coefficients (red) Four (residual) variances (blue) No covariances 2.4.2.10 Consider a multiple regression analysis with three continuous independent variables: scores on tests of language, history, and logic, and one continuous dependent variable: score on a math test. We want to know if scores on the language, history, and logic tests can predict the math test score. Sketch a path model that you could use to answer this question How many regression parameters are there? How many variances could you estimate? How many covariances could you estimate? 2.4.3 Categorical IVs Load the Drivers.sav data. # Read the data into a data frame named 'drivers': drivers <- read_sav("Drivers.sav") %>% as_factor() # This preserves the SPSS labels for nominal variables In this section, we will evaluate the following research question: Does talking on the phone interfere with people's driving skills? These data come from an experiment. The condition variable represents the three experimental conditions: Hand-held phone Hands-free phone Control (no phone) We will use condition as the IV in our models. The DV, RT, represents the participant’s reaction time (in milliseconds) during a driving simulation. 2.4.3.1 Use the package ggplot2 to create a density plot for the variable RT. What concept are we representing with this plot? Hint: Consider the lap times example from the statistical modeling section of Lecture 2. Click to show code ggplot(drivers, aes(x = RT)) + geom_density() Click for explanation This shows the distribution of all the combined reaction times from drivers in all three categories. 2.4.3.2 Modify this density plot by mapping the variable condition from your data to the fill aesthetic in ggplot. What is the difference between this plot and the previous plot? Do you think there is evidence for differences between the groups? How might we test this by fitting a model to our sample? Click to show code Hint: To modify the transparency of the densities, use the aesthetic alpha. ggplot(drivers, aes(x = RT, fill = condition)) + geom_density(alpha = .5) Click for explanation This figure models the conditional distribution of reaction time, where the type of cell phone usage is the grouping factor. Things you can look at to visually assess whether the three groups differ are the amount of overlap of the distributions, how much distance there is between the individual means, and whether the combined distribution is much different than the conditional distributions. If we are willing to assume that these conditional distributions are normally distributed and have equivalent variances, we could use a linear model with dummy-coded predictors. Aside: ANOVA vs. Linear Regression As you may know, the mathematical model underlying ANOVA is just a linear regression model with nominal IVs. So, in terms of the underlying statistical models, there is no difference between ANOVA and regression; the differences lie in the focus of the analysis. ANOVA is really a type of statistical test wherein we are testing hypotheses about the effects of some set of nominal grouping factors on some continuous outcome. When doing an ANOVA, we usually don’t interact directly with the parameter estimates from the underlying model. Regression is a type of statistical model (i.e., a way to represent a univariate distribution with a conditional mean and fixed variance). When we do a regression analysis, we primarily focus on the estimated parameters of the underling linear model. When doing ANOVA in R, we estimate the model exactly as we would for linear regression; we simply summarize the results differently. If you want to summarize your model in terms of the sums of squares table you usually see when running an ANOVA, you can supply your fitted lm object to the anova() function. This is a statistical modeling course, not a statistical testing course, so we will not consider ANOVA any further. 2.4.3.3 Estimate a linear model that will answer the research question stated in the beginning of this section. Use lm() to estimate the model. Summarize the fitted model and use the results to answer the research question. Click to show code library(magrittr) lmOut <- drivers %>% mutate(condition = relevel(condition, ref = "control")) %$% lm(RT ~ condition) summary(lmOut) ## ## Call: ## lm(formula = RT ~ condition) ## ## Residuals: ## Min 1Q Median 3Q Max ## -317.50 -71.25 2.98 89.55 243.45 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 553.75 29.08 19.042 <2e-16 *** ## conditionhand-held 100.75 41.13 2.450 0.0174 * ## conditionhands-free 63.80 41.13 1.551 0.1264 ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 130.1 on 57 degrees of freedom ## Multiple R-squared: 0.09729, Adjusted R-squared: 0.06562 ## F-statistic: 3.072 on 2 and 57 DF, p-value: 0.05408 anova(lmOut) Click for explanation The effect of condition on RT is nonsignificant (\\(F[2, 57] = 3.07\\), \\(p = 0.054\\)). Therefore, based on these results, we do not have evidence for an effect of mobile phone usage on driving performance. 2.4.3.4 Use lavaan to estimate the model from 2.4.3.3 as a path model. Hint: lavaan won’t let us use factors for our categorical predictors. So, you will need to create your own dummy codes. Click to show code mod <- 'RT ~ 1 + HH + HF' lavOut <- drivers %>% mutate(HH = ifelse(condition == "hand-held", 1, 0), # Create dummy code for "hand-held" condition HF = ifelse(condition == "hands-free", 1, 0) # Create dummy code for "hands-free" condition ) %>% sem(mod, data = .) # Estimate the model summary(lavOut) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 4 ## ## Number of observations 60 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## RT ~ ## HH 100.750 40.085 2.513 0.012 ## HF 63.800 40.085 1.592 0.111 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .RT 553.750 28.344 19.537 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .RT 16068.028 2933.607 5.477 0.000 At this point, we haven’t covered the tools you need to conduct the ANOVA-style tests with path models. So, you can’t yet answer the research question with the above model. When we discuss model comparisons, you’ll get the missing tools. End of In-Class Exercises 2 "],["mediation-moderation.html", "3 Mediation & Moderation", " 3 Mediation & Moderation In this lecture, we will discuss two particular types of processes that we can model using path analysis: mediation and moderation. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-2.html", "3.1 Lecture", " 3.1 Lecture Researchers often have theories about possible causal processes linking multiple variables. Mediation is a particularly important example of such a process in which in an input variable, X, influences the outcome, Y, through an intermediary variable, M (the mediator). For instance, psychotherapy (X), may affect thoughts (M), which in turn affects mood (Y). We can investigate mediation via a specific sequence of linear regression equations, but path modeling will make our lives much easier. We can use path models to simultaneously estimate multiple related regression equations. So, mediation analysis is an ideal application of path modeling. In this lecture, we consider both approaches and discuss their relative strengths and weaknesses. As with mediation, researchers often posit theories involving moderation. Moderation implies that the effect of X on Y depends on another variable, Z. For instance, the effect of feedback (X) on performance (Y) may depend on age (Z). Older children might process feedback more effectively than younger children. Hence, the feedback is more effective for older children than for younger children, and the effect of feedback on performance is stronger for older children than for younger children. In such a case, we would say that age moderates the effect of feedback on performance. 3.1.1 Recordings Note: In the following recordings, the slide numbers are a bit of a mess, because I made these videos by cutting together recordings that used different slide decks. My apologies to those who are particularly distracted by continuity errors. Mediation Basics Mediation Testing Bootstrapping Moderation Basics Moderation Probing 3.1.2 Slides You can download the lecture slides here "],["reading-2.html", "3.2 Reading", " 3.2 Reading Reference Baron, R. M. & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical Considerations. Journal of Personality and Individual Differences, 51(6), 1173–1182 Questions What is mediation? Give an example of mediation. According to the authors, we must satisfy four criteria to infer mediation. What are these criteria? What is “moderation”, and how is it different from “mediation”? Give an example of moderation. What are the four methods given by Baron and Kenny as suitable ways to to study interaction effects? The authors suggest that one of the most common ways to address unreliability is to use multiple indicators. Thinking back to what you’ve learned about factor analysis, briefly explain why multiple indicators can improve reliability. How can you determine whether a variable is a mediator or moderator? Reference Hayes, A. F. (2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76(4), 408–420. Questions What is an indirect or mediated effect? What is the difference between the total and direct effect? What is the main problem with the Barron & Kenny “Causal Steps Approach”? What is bootstrapping, and why is it a better way to test mediation than Sobel’s test? Explain how it is possible that “effects that don’t exist can be mediated”. "],["at-home-exercises-2.html", "3.3 At-Home Exercises", " 3.3 At-Home Exercises 3.3.1 Mediation In the first part of this practical, we will analyze the data contained in SelfEsteem.sav. These data comprise 143 observations of the following variables.1 case: Participant ID number ParAtt: Parental Attachment PeerAtt: Peer Attachment Emp: Empathy ProSoc: Prosocial behavior Aggr: Aggression SelfEst: Self-esteem 3.3.1.1 Load the SelfEsteem.sav data. Note: Unless otherwise specified, all analyses in Section 3.3.1 apply to these data. Click to show code library(haven) seData <- read_sav("SelfEsteem.sav") Suppose we are interested in the (indirect) effect of peer attachment on self-esteem, and whether empathy has a mediating effect on this relationship. We might generate the following hypotheses: Better peer relationships promote higher self esteem This effect is mediated by a student’s empathy levels, where better peer relationships increase empathy, and higher levels of empathy lead to higher self-esteem. To evaluate these hypotheses, we will use lavaan to estimate a path model. 3.3.1.2 Draw a path model (on paper) that can be used to test the above hypotheses. Label the input (X), outcome (Y), and mediator/intermediary (M). Label the paths a, b, and c’. Hint: Refer back to the Mediation Basics lecture if you need help here. Click for explanation 3.3.1.3 Specify the lavaan model syntax implied by the path diagram shown above. Save the resulting character string as an object in your environment. Hint: Refer back to the example in which opinions of systematic racism mediate the relationship between political affiliation and support for affirmative action policies from the Mediation Testing lecture this week. Click to show code mod <- ' ## Equation for outcome: SelfEst ~ Emp + PeerAtt ## Equation for the mediator: Emp ~ PeerAtt ' 3.3.1.4 Use the lavaan::sem() function to estimate the model defined in 3.3.1.3. Use the default settings in sem(). Click to show code library(lavaan) out <- sem(mod, data = seData) 3.3.1.5 Explore the summary of the fitted model. Which numbers correspond to the a, b, and c’ paths? Interpret these paths. Do the direction of the effects seem to align with our hypothesis? Click to show code summary(out) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 5 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## Emp 0.234 0.091 2.568 0.010 ## PeerAtt 0.174 0.088 1.968 0.049 ## Emp ~ ## PeerAtt 0.349 0.076 4.628 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.934 0.110 8.456 0.000 ## .Emp 0.785 0.093 8.456 0.000 Click for explanation The results show estimates of the a path (Emp ~ PeerAtt), the b path (SelfEst ~ Emp), and the c’ path (SelfEst ~ PeerAtt). All three of these effects are positive and significant, including the direct effect of PeerAtt on SelfEst (\\(\\beta = 0.174\\), \\(Z = 1.97\\), \\(p = 0.025\\)), and the parts of the indirect effect made up by the effect of PeerAtt on Emp (\\(\\beta = 0.349\\), \\(Z = 4.63\\), \\(p = 0\\)), and Emp on SelfEst (\\(\\beta = 0.234\\), \\(Z = 2.57\\), \\(p = 0.005\\)). We can see that the direction of the effects seems to support of our hypotheses, but without taking the next steps to investigate the indirect effect, we should be hesitant to say more. Remember that an indirect effect (IE) is the product of multiple regression slopes. Therefore, to estimate an IE, we must define this product in our model syntax. In lavaan, we define the new IE parameter in two steps. Label the relevant regression paths. Use the labels to define a new parameter that represent the desired IE. We can define new parameters in lavaan model syntax via the := operator. The lavaan website contains a tutorial on this procedure: http://lavaan.ugent.be/tutorial/mediation.html 3.3.1.6 Use the procedure described above to modify the model syntax from 3.3.1.3 by adding the definition of the hypothesized IE from PeerAtt to SelfEst. Click to show code mod <- ' ## Equation for outcome: SelfEst ~ b * Emp + PeerAtt ## Equation for mediator: Emp ~ a * PeerAtt ## Indirect effect: ie := a * b ' Click for explanation Notice that I only label the parameters that I will use to define the IE. You are free to label any parameter that you like, but I choose the to label only the minimally sufficient set to avoid cluttering the code/output. 3.3.1.7 Use lavaan::sem() to estimate the model with the IEs defined. Use the default settings for sem(). Is the hypothesized IE significant according to the default tests? Hint: Refer to the Mediation Testing lecture Click to show code out <- sem(mod, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 5 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## Emp (b) 0.234 0.091 2.568 0.010 ## PeerAtt 0.174 0.088 1.968 0.049 ## Emp ~ ## PeerAtt (a) 0.349 0.076 4.628 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.934 0.110 8.456 0.000 ## .Emp 0.785 0.093 8.456 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie 0.082 0.036 2.245 0.025 Click for explanation The IE of Peer Attachment on Self Esteem through Empathy is statistically significant (\\(\\hat{\\textit{IE}} = 0.082\\), \\(Z = 2.25\\), \\(p = 0.012\\)). Note: The p-value above doesn’t match the output because we’re testing a directional hypothesis, but lavaan conducts two-tailed tests for the model parameters. As we learned in the lecture, the above test of the indirect effect is equivalent to Sobel’s Z test (which we don’t really want). An appropriate, robust test of the indirect effect requires bootstrapping, which we will do later this week as part of the in-class exercises. For now, we’ll add another input variable to our model: parental attachment. We will use this model to evaluate the following research questions: Is there a direct effect of parental attachment on self-esteem, after controlling for peer attachment and empathy? Is there a direct effect of peer attachment on self-esteem, after controlling for parental attachment and empathy? Is the effect of parental attachment on self-esteem mediated by empathy, after controlling for peer attachment? Is the effect of peer attachment on self-esteem mediated by empathy, after controlling for parental attachment? 3.3.1.8 Run the path model needed to test the research questions listed above. Specify the lavaan model syntax implied by the research questions. Allow peer attachment and parental attachment to covary. Define two new parameters to represent the hypothesized indirect effects. Estimate the model using lavaan::sem(). Use the default settings in sem(). Investigate the model summary. Click to show code mod <- ' ## Equation for outcome: SelfEst ~ b * Emp + ParAtt + PeerAtt ## Equation for mediator: Emp ~ a1 * ParAtt + a2 * PeerAtt ## Covariance: ParAtt ~~ PeerAtt ie_ParAtt := a1 * b ie_PeerAtt := a2 * b ' out <- sem(mod, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 10 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## Emp (b) 0.206 0.088 2.357 0.018 ## ParAtt 0.287 0.078 3.650 0.000 ## PeerAtt 0.024 0.094 0.252 0.801 ## Emp ~ ## ParAtt (a1) 0.078 0.075 1.045 0.296 ## PeerAtt (a2) 0.306 0.086 3.557 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## ParAtt ~~ ## PeerAtt 0.537 0.103 5.215 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.854 0.101 8.456 0.000 ## .Emp 0.779 0.092 8.456 0.000 ## ParAtt 1.277 0.151 8.456 0.000 ## PeerAtt 0.963 0.114 8.456 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie_ParAtt 0.016 0.017 0.956 0.339 ## ie_PeerAtt 0.063 0.032 1.965 0.049 3.3.1.9 What can we say about the two indirect effects? Can we say that empathy mediates both paths? Click to show explanation According to the Sobel-style test, after controlling for parental attachment, the indirect effect of peer attachment on self-esteem was statistically significant (\\(\\hat{IE} = 0.063\\), \\(Z = 1.96\\), \\(p = 0.049\\)), as was the analogous direct effect (\\(\\hat{\\beta} = 0.306\\), \\(Z = 3.56\\), \\(p < 0.001\\)). After controlling for peer attachment, neither the indirect effect (\\(\\hat{IE} = 0.016\\), \\(Z = 0.96\\), \\(p = 0.339\\)) nor the direct effect (\\(\\hat{\\beta} = 0.078\\), \\(Z = 1.05\\), \\(p = 0.296\\)) of parental attachment on self-esteem was significant, though. 3.3.2 Moderation Remember that moderation attempts to describe when one variable influences another. For the home exercise, we’ll go back to the Sesame Street data we worked with for the in-class exercises last week. 3.3.2.1 Load the Sesam2.sav data.2 NOTE: Unless otherwise specified, all analyses in Section 3.3.2 use these data. Click to show code # Read the data into an object called 'sesam2': sesam2 <- read_sav("Sesam2.sav") VIEWCAT is a nominal grouping variable, but it is represented as a numeric variable in the sesam2 data. The levels represent the following frequencies of Sesame Street viewership of the children in the data: VIEWCAT = 1: Rarely/Never VIEWCAT = 2: 2–3 times a week VIEWCAT = 3: 4–5 times a week VIEWCAT = 4: > 5 times a week 3.3.2.2 Convert VIEWCAT into a factor. Make sure that VIEWCAT = 1 is the reference group. Hints: You can identify the reference group with the levels() or contrasts() functions. The reference group is the group labelled with the first level printed by levels(). When you run contrasts(), you will see a pattern matrix that defines a certain dummy coding scheme. The reference group is the group that has zeros in each column of this matrix. If you need to change the reference group, you can use the relevel() function. Click to show code library(forcats) ## Convert 'VIEWCAT' to a factor: sesam2 <- sesam2 %>% mutate(VIEWCAT = factor(VIEWCAT)) ## Optionally specify the labels # sesam2 <- # sesam2 %>% # mutate(VIEWCAT = factor(VIEWCAT, # levels = c(1, 2, 3, 4), # labels = c("Rarely/never", # "2-3 times per week", # "4-5 times per week", # "> 5 times per week"))) ## Check the reference group: levels(sesam2$VIEWCAT) ## [1] "1" "2" "3" "4" contrasts(sesam2$VIEWCAT) ## 2 3 4 ## 1 0 0 0 ## 2 1 0 0 ## 3 0 1 0 ## 4 0 0 1 ## If necessary, relevel # sesam <- # sesam2 %>% # mutate(VIEWCAT = relevel(VIEWCAT, 1)) 3.3.2.3 Use lm() to estimate a multiple regression model wherein VIEWCAT predicts POSTNUMB. Summarize the model. Interpret the estimates. Click to show code lmOut <- lm(POSTNUMB ~ VIEWCAT, data = sesam2) summary(lmOut) ## ## Call: ## lm(formula = POSTNUMB ~ VIEWCAT, data = sesam2) ## ## Residuals: ## Min 1Q Median 3Q Max ## -25.474 -7.942 0.240 8.526 25.240 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 18.760 2.316 8.102 8.95e-14 *** ## VIEWCAT2 9.331 2.900 3.218 0.00154 ** ## VIEWCAT3 14.714 2.777 5.298 3.49e-07 *** ## VIEWCAT4 18.032 2.809 6.419 1.24e-09 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 11.58 on 175 degrees of freedom ## Multiple R-squared: 0.2102, Adjusted R-squared: 0.1967 ## F-statistic: 15.53 on 3 and 175 DF, p-value: 5.337e-09 Click for explanation Viewing category explains a statistically significant proportion of the variance in the post-test score of numbers learned (\\(R^2 = 0.21\\), \\(F(3, 175) = 15.53\\), \\(p < 0.001\\)). Kids who never or rarely watched Sesame Street had an average score of 18.76 on the post-test. Kids with weekly viewing habits of 2–3, 4–5, or 5+ times per week all had significantly higher scores on the post-test than kids who never or rarely watched Sesame Street (2–3: \\(\\hat{\\beta} = 9.33\\), \\(t = 3.22\\), \\(p = 0.002\\); 4–5: \\(\\hat{\\beta} = 14.71\\), \\(t = 5.3\\), \\(p < 0.001\\); 5+: \\(\\hat{\\beta} = 18.03\\), \\(t = 6.42\\), \\(p < 0.001\\)). If we compare the box plot, kernel density plot, and model output below, the relationships between the regression coefficient estimates for the viewing categories and the group means should be evident. 3.3.2.4 Use ggplot() to make a scatterplot with AGE on the x-axis and POSTNUMB on the y-axis. Color the points according to the their VIEWCAT level. Save the plot object to a variable in your environment. Hint: You can map color to the levels of a variable on your dataset by assigning the variable names to the color argument of the aes() function in ggplot(). Click to show code library(ggplot2) ## Add aes(..., color = VIEWCAT) to get different colors for each group: p <- ggplot(sesam2, aes(x = AGE, y = POSTNUMB, color = VIEWCAT)) + geom_point() # Add points for scatterplot ## Print the plot stored as 'p': p We assigned the global color aesthetic to the VIEWCAT variable, so the points are colored based on their group. 3.3.2.5 Add linear regression lines for each group to the above scatterplot. Hints: You can add regression lines with ggplot2::geom_smooth() To get linear regression lines, set the argument method = \"lm\" To omit error envelopes, set the argument se = FALSE Click to show code ## Add OLS best-fit lines: p + geom_smooth(method = "lm", se = FALSE) The global color aesthetic assignment from above carries through to any additional plot elements that we add, including the regression lines. So, we also get a separate regression line for each VIEWCAT group. 3.3.2.6 How would you interpret the pattern of regression lines above? Click for explanation All the lines show a positive slope, so post-test number recognition appears to increase along with increasing age. The lines are not parallel, though. So VIEWCAT may be moderating the effect of AGE on POSTNUMB. Based on the figure we just created, we may want to test for moderation in our regression model. To do so, we need to add an interaction between AGE and VIEWCAT. The VIEWCAT factor is represented by 3 dummy codes in our model, though. So when we interact AGE and VIEWCAT, we will create 3 interaction terms. To test the overall moderating influence of VIEWCAT, we need to conduct a multiparameter hypothesis test of all 3 interaction terms. One way that we can go about implementing such a test is through a hierarchical regression analysis entailing three steps: Estimate the additive model wherein we regress POSTNUMB onto AGE and VIEWCAT without any interaction. Estimate the moderated model by adding the interaction between AGE and VIEWCAT into the additive model. Conduct a \\(\\Delta R^2\\) test to compare the fit of the two models. 3.3.2.7 Conduct the hierarchical regression analysis described above. Does VIEWCAT significantly moderate the effect of AGE on POSTNUMB? Provide statistical justification for your conclusion. Click to show code ## Estimate the additive model a view the results: results_add <- lm(POSTNUMB ~ VIEWCAT + AGE, data = sesam2) summary(results_add) ## ## Call: ## lm(formula = POSTNUMB ~ VIEWCAT + AGE, data = sesam2) ## ## Residuals: ## Min 1Q Median 3Q Max ## -23.680 -8.003 -0.070 8.464 22.635 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -10.1056 6.5091 -1.553 0.12235 ## VIEWCAT2 9.1453 2.7390 3.339 0.00103 ** ## VIEWCAT3 13.8602 2.6294 5.271 3.98e-07 *** ## VIEWCAT4 16.9215 2.6636 6.353 1.79e-09 *** ## AGE 0.5750 0.1221 4.708 5.08e-06 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 10.94 on 174 degrees of freedom ## Multiple R-squared: 0.2995, Adjusted R-squared: 0.2834 ## F-statistic: 18.6 on 4 and 174 DF, p-value: 9.642e-13 ## Estimate the moderated model and view the results: results_mod <- lm(POSTNUMB ~ VIEWCAT * AGE, data = sesam2) summary(results_mod) ## ## Call: ## lm(formula = POSTNUMB ~ VIEWCAT * AGE, data = sesam2) ## ## Residuals: ## Min 1Q Median 3Q Max ## -23.8371 -8.2387 0.6158 8.7988 22.5611 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -18.7211 15.5883 -1.201 0.2314 ## VIEWCAT2 9.9741 20.6227 0.484 0.6293 ## VIEWCAT3 23.5825 19.3591 1.218 0.2248 ## VIEWCAT4 34.3969 19.3600 1.777 0.0774 . ## AGE 0.7466 0.3074 2.429 0.0162 * ## VIEWCAT2:AGE -0.0175 0.4060 -0.043 0.9657 ## VIEWCAT3:AGE -0.1930 0.3782 -0.510 0.6104 ## VIEWCAT4:AGE -0.3416 0.3770 -0.906 0.3663 ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 10.99 on 171 degrees of freedom ## Multiple R-squared: 0.3046, Adjusted R-squared: 0.2762 ## F-statistic: 10.7 on 7 and 171 DF, p-value: 3.79e-11 ## Test for moderation: anova(results_add, results_mod) Click for explanation VIEWCAT does not significantly moderate the effect of AGE on POSTNUMB (\\(F[3, 171] = 0.422\\), \\(p = 0.738\\)). 3.3.2.8 Sketch the analytic path diagrams for the additive and moderated models you estimated in 3.3.2.7 (on paper). Click for explanation Additive Model Moderated Model End of At-Home Exercises 3 These data were simulated from the covariance matrix provided in Laible, D. J., Carlo, G., & Roesch, S. C. (2004). Pathways to self-esteem in late adolescence: The role of parent and peer attachment, empathy, and social behaviours. Journal of adolescence, 27(6), 703-716.↩︎ These data are from the very interesting study: Ball, S., & Bogatz, G. A. (1970). A Summary of the Major Findings in” The First Year of Sesame Street: An Evaluation”.↩︎ "],["in-class-exercises-2.html", "3.4 In-Class Exercises", " 3.4 In-Class Exercises 3.4.1 Mediation In this practical, we’ll go back to the data from the at-home exercises, SelfEsteem.sav. Recall that these data comprise 143 observations of the following variables. case: Participant ID number ParAtt: Parental Attachment PeerAtt: Peer Attachment Emp: Empathy ProSoc: Prosocial behavior Aggr: Aggression SelfEst: Self-esteem When we last worked with the data, we built a model with one mediator (Emp), creating indirect effects between our predictors ParAtt and PeerAtt, and our outcome variable SelfEst. Below, you will estimate a more complex, multiple-mediator model. 3.4.1.1 Load the data into the object seData using haven::read_sav() Click to show code library(haven) seData <- read_sav("SelfEsteem.sav") For this analysis, we are interested in the (indirect) effects of parental and peer attachment on self-esteem. Furthermore, we want to evaluate the mediating roles of empathy and social behavior (i.e., prosocial behavior and aggression). Specifically, we have the following hypotheses. Better peer relationships will promote higher self-esteem via a three-step indirect process. Better peer relationships will increase empathy levels. Higher empathy will increase prosocial behavior and decrease aggressive behavior. More prosocial behaviors and less aggressive behavior will both produce higher self-esteem. Better relationships with parents directly increase self-esteem. To evaluate these hypotheses, we will use lavaan to estimate the following multiple mediator model as a path model. 3.4.1.2 Specify the lavaan model syntax implied by the path diagram shown above. Save the resulting character string as an object in your environment. Click to show code mod0 <- ' ## Equation for outcome: SelfEst ~ ProSoc + Aggr + Emp + ParAtt + PeerAtt ## Equations for stage 2 mediators: ProSoc ~ PeerAtt + ParAtt + Emp Aggr ~ PeerAtt + ParAtt + Emp ## Equation for stage 1 mediator: Emp ~ ParAtt + PeerAtt ## Covariances: ProSoc ~~ Aggr ParAtt ~~ PeerAtt ' 3.4.1.3 Use the lavaan::sem() function to estimate the model defined in 3.4.1.2. Use the default settings in sem(). Summarize the fitted model. Click to show code library(lavaan) out <- sem(mod0, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## ProSoc 0.252 0.096 2.634 0.008 ## Aggr 0.185 0.085 2.172 0.030 ## Emp 0.143 0.098 1.460 0.144 ## ParAtt 0.244 0.078 3.133 0.002 ## PeerAtt 0.051 0.091 0.555 0.579 ## ProSoc ~ ## PeerAtt -0.037 0.080 -0.469 0.639 ## ParAtt 0.193 0.067 2.886 0.004 ## Emp 0.477 0.074 6.411 0.000 ## Aggr ~ ## PeerAtt -0.095 0.090 -1.055 0.291 ## ParAtt -0.034 0.075 -0.454 0.650 ## Emp -0.309 0.084 -3.697 0.000 ## Emp ~ ## ParAtt 0.078 0.075 1.045 0.296 ## PeerAtt 0.306 0.086 3.557 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .ProSoc ~~ ## .Aggr -0.086 0.058 -1.476 0.140 ## ParAtt ~~ ## PeerAtt 0.537 0.103 5.215 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.796 0.094 8.456 0.000 ## .ProSoc 0.618 0.073 8.456 0.000 ## .Aggr 0.777 0.092 8.456 0.000 ## .Emp 0.779 0.092 8.456 0.000 ## ParAtt 1.277 0.151 8.456 0.000 ## PeerAtt 0.963 0.114 8.456 0.000 3.4.1.4 Considering the parameter estimates from 3.4.1.3, what can you say about the hypotheses? Click for explanation Notice that all of the hypotheses stated above are explicitly directional. Hence, when evaluating the significance of the structural paths that speak to these hypotheses, we should use one-tailed tests. We cannot ask lavaan to return one-tailed p-values, but we have no need to do so. We can simply divide the two-tailed p-values in half. The significant direct effect of ParAtt on SelfEst (\\(\\beta = 0.244\\), \\(Z = 3.13\\), \\(p = 0.001\\)) and the lack of a significant direct effect of PeerAtt on SelfEst (\\(\\beta = 0.051\\), \\(Z = 0.555\\), \\(p = 0.29\\)) align with our hypotheses. The remaining patterns of individual estimates also seem to conform to the hypotheses (e.g., all of the individual paths comprising the indirect effects of PeerAtt on SelfEst are significant). We cannot make any firm conclusions until we actually estimate and test the indirect effects, though. 3.4.1.5 Modify the model syntax from 3.4.1.2 by adding definitions of the two hypothesized IEs from PeerAtt to SelfEst. Click to show code You can use any labeling scheme that makes sense to you, but I recommend adopting some kind of systematic rule. Here, I will label the individual estimates in terms of the short variable names used in the path diagram above. mod <- ' ## Equation for outcome: SelfEst ~ y_m21 * ProSoc + y_m22 * Aggr + Emp + ParAtt + PeerAtt ## Equations for stage 2 mediators: ProSoc ~ m21_x2 * PeerAtt + ParAtt + m21_m1 * Emp Aggr ~ m22_x2 * PeerAtt + ParAtt + m22_m1 * Emp ## Equation for stage 1 mediator: Emp ~ ParAtt + m1_x2 * PeerAtt ## Covariances: ProSoc ~~ Aggr ParAtt ~~ PeerAtt ## Indirect effects: ie_pro := m1_x2 * m21_m1 * y_m21 ie_agg := m1_x2 * m22_m1 * y_m22 ' 3.4.1.6 Use lavaan::sem() to estimate the model with the IEs defined. Use the default settings for sem(). Are the hypothesized IEs significant according to the default tests? Click to show code out <- sem(mod, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## ProSoc (y_21) 0.252 0.096 2.634 0.008 ## Aggr (y_22) 0.185 0.085 2.172 0.030 ## Emp 0.143 0.098 1.460 0.144 ## ParAtt 0.244 0.078 3.133 0.002 ## PerAtt 0.051 0.091 0.555 0.579 ## ProSoc ~ ## PerAtt (m21_2) -0.037 0.080 -0.469 0.639 ## ParAtt 0.193 0.067 2.886 0.004 ## Emp (m21_1) 0.477 0.074 6.411 0.000 ## Aggr ~ ## PerAtt (m22_2) -0.095 0.090 -1.055 0.291 ## ParAtt -0.034 0.075 -0.454 0.650 ## Emp (m22_1) -0.309 0.084 -3.697 0.000 ## Emp ~ ## ParAtt 0.078 0.075 1.045 0.296 ## PerAtt (m1_2) 0.306 0.086 3.557 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .ProSoc ~~ ## .Aggr -0.086 0.058 -1.476 0.140 ## ParAtt ~~ ## PeerAtt 0.537 0.103 5.215 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.796 0.094 8.456 0.000 ## .ProSoc 0.618 0.073 8.456 0.000 ## .Aggr 0.777 0.092 8.456 0.000 ## .Emp 0.779 0.092 8.456 0.000 ## ParAtt 1.277 0.151 8.456 0.000 ## PeerAtt 0.963 0.114 8.456 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie_pro 0.037 0.018 2.010 0.044 ## ie_agg -0.017 0.011 -1.657 0.098 Click for explanation The IE of Peer Attachment on Self Esteem through Empathy and Prosocial Behavior is significant (\\(\\hat{\\textit{IE}} = 0.037\\), \\(Z = 2.01\\), \\(p = 0.022\\)), as is the analogous IE through Aggressive Behavior (\\(\\hat{\\textit{IE}} = -0.017\\), \\(Z = -1.66\\), \\(p = 0.049\\)). Though, this latter effect is just barely significant at the \\(\\alpha = 0.05\\) level. The tests we used to evaluate the significance of the IEs in 3.4.1.6 are flawed because they assume normal sampling distributions for the IEs. However the IEs are defined as products of multiple, normally distributed, regression slopes. So the IEs themselves cannot be normally distributed (at least in finite samples), and the results of the normal-theory significance tests may be misleading. To get an accurate test of the IEs, we should use bootstrapping to generate an empirical sampling distribution for each IE. In lavaan, we implement bootstrapping by specifying the se = \"bootstrap\" option in the fitting function (i.e., the cfa() or sem() function) and specifying the number of bootstrap samples via the bootstrap option. Workflow Tip To draw reliable conclusions from bootstrapped results, we need many bootstrap samples (i.e., B > 1000), but we must estimate the full model for each of these samples, so the estimation can take a long time. To avoid too much frustration, you should first estimate the model without bootstrapping to make sure everything is specified correctly. Only after you are certain that your code is correct do you want to run the full bootstrapped version. 3.4.1.7 Re-estimate the model from 3.4.1.6 using 1000 bootstrap samples. Other than the se and bootstrap options, use the defaults. Are the hypothesized IEs significant according to the bootstrap-based test statistics? Click to show code ## Set a seed to get replicable bootstrap samples: set.seed(235711) ## Estimate the model with bootstrapping: out_boot <- sem(mod, data = seData, se = "bootstrap", bootstrap = 1000) ## Summarize the model: summary(out_boot) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Bootstrap ## Number of requested bootstrap draws 1000 ## Number of successful bootstrap draws 1000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## ProSoc (y_21) 0.252 0.100 2.529 0.011 ## Aggr (y_22) 0.185 0.085 2.174 0.030 ## Emp 0.143 0.095 1.507 0.132 ## ParAtt 0.244 0.079 3.089 0.002 ## PerAtt 0.051 0.095 0.530 0.596 ## ProSoc ~ ## PerAtt (m21_2) -0.037 0.082 -0.456 0.648 ## ParAtt 0.193 0.068 2.831 0.005 ## Emp (m21_1) 0.477 0.078 6.092 0.000 ## Aggr ~ ## PerAtt (m22_2) -0.095 0.087 -1.093 0.275 ## ParAtt -0.034 0.076 -0.448 0.654 ## Emp (m22_1) -0.309 0.092 -3.356 0.001 ## Emp ~ ## ParAtt 0.078 0.072 1.092 0.275 ## PerAtt (m1_2) 0.306 0.079 3.896 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .ProSoc ~~ ## .Aggr -0.086 0.058 -1.493 0.135 ## ParAtt ~~ ## PeerAtt 0.537 0.128 4.195 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.796 0.082 9.698 0.000 ## .ProSoc 0.618 0.068 9.114 0.000 ## .Aggr 0.777 0.104 7.476 0.000 ## .Emp 0.779 0.090 8.651 0.000 ## ParAtt 1.277 0.197 6.473 0.000 ## PeerAtt 0.963 0.105 9.203 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie_pro 0.037 0.019 1.891 0.059 ## ie_agg -0.017 0.011 -1.638 0.101 Click for explanation As with the normal-theory tests, the hypothesized IE of Peer Attachment on Self Esteem was significant (\\(\\hat{\\textit{IE}} = 0.037\\), \\(Z = 1.89\\), \\(p = 0.029\\)), but the IE of Aggressive Behavior has crossed into nonsignificant territory (\\(\\hat{\\textit{IE}} = -0.017\\), \\(Z = -1.64\\), \\(p = 0.051\\)). Note: Bootstrapping is a stochastic method, so each run can provide different results. Since the indirect effect of aggressive behavior is so close to the critical value, you may come to a different conclusions vis-á-vis statistical significance if you run this analysis with a different random number seed or a different number of bootstrap samples. When you use the summary() function to summarize the bootstrapped model from 3.4.1.7, the output will probably look pretty much the same as it did in 3.4.1.6, but it’s not. The standard errors and test statistics in the bootstrapped summary are derived from empirical sampling distributions, whereas these values are based on an assumed normal sampling distribution in 3.4.1.6. The standard method of testing IEs with bootstrapping is to compute confidence intervals (CIs) from the empirical sampling distribution of the IEs. In lavaan, we can compute basic (percentile, 95%) CIs by adding the ci = TRUE option to the summary() function. To evaluate our directional hypotheses at an \\(\\alpha = 0.05\\) level, however, we need to compute 90% CIs. We can get more control over the summary statistics (include the CIs) with the parameterEstimates() function. 3.4.1.8 Check the documentation for lavaan::parameterEstimates(). Click to show code ?parameterEstimates 3.4.1.9 Use the parameterEstimates() function to compute bootstrapped CIs for the hypothesized IEs. Compute percentile CIs. Are the IEs significant according to the bootstrapped CIs? Click to show code parameterEstimates(out_boot, ci = TRUE, level = 0.9) Click for explanation When evaluating a directional hypothesis with a CI, we only consider one of the interval’s boundaries. For a hypothesized positive effect, we check only if the lower boundary is greater than zero. For a hypothesized negative effect, we check if the upper boundary is less than zero. As with the previous tests, the IE of Peer Attachment on Self Esteem through Empathy and Prosocial Behavior is significant (\\(\\hat{\\textit{IE}} = 0.037\\), \\(95\\% ~ CI = [0.009; \\infty]\\)), but the analogous IE through Aggressive Behavior is not quite significant (\\(\\hat{\\textit{IE}} = -0.017\\), \\(95\\% ~ CI = [-\\infty; -0.003]\\)). 3.4.1.10 Based on the analyses you’ve conducted here, what do you conclude vis-à-vis the original hypotheses? Click for explanation When using normal-theory tests, both hypothesized indirect effects between Peer Attachment and Self Esteem were supported in that the IE through Empathy and Prosocial Behavior as well as the IE through Empathy and Aggressive Behavior were both significant. The hypothesized direct effect of Parent Attachment on Self Esteem was also born out via a significant direct effect in the model. When testing the indirect effects with bootstrapping, however, the effect through Aggressive Behavior was nonsignificant. Since bootstrapping gives a more accurate test of the indirect effect, we should probably trust these results more than the normal-theory results. We should not infer a significant indirect effect of Peer Attachment on Self Esteem transmitted through Empathy and Aggressive Behavior. These results may not tell the whole story, though. We have not tested for indirect effects between Parent Attachment and Self Esteem, and we have not evaluated simpler indirect effects between Peer Attachment and Self Esteem (e.g., PeerAtt \\(\\rightarrow\\) Emp \\(\\rightarrow\\) SelfEst). 3.4.2 Moderation We will first analyze a synthetic version of the Outlook on Life Survey data. The original data were collected in the United States in 2012 to measure, among other things, attitudes about racial issues, opinions of the Federal government, and beliefs about the future. We will work with a synthesized subset of the original data. You can access these synthetic data as outlook.rds. This dataset comprises 2288 observations of the following 13 variables. d1:d3: Three observed indicators of a construct measuring disillusionment with the US Federal government. Higher scores indicate more disillusionment s1:s4: Four observed indicators of a construct measuring the perceived achievability of material success. Higher scores indicate greater perceived achievability progress: A single item assessing perceived progress toward achieving the “American Dream” Higher scores indicate greater perceived progress merit: A single item assessing endorsement of the meritocratic ideal that hard work leads to success. Higher scores indicate stronger endorsement of the meritocratic ideal lib2Con: A single item assessing liberal-to-conservative orientation Lower scores are more liberal, higher scores are more conservative party: A four-level factor indicating self-reported political party affiliation disillusion: A scale score representing disillusionment with the US Federal government Created as the mean of d1:d3 success: A scale score representing the perceived achievability of material success Created as the mean of s1:s4 To satisfy the access and licensing conditions under which the original data are distributed, the data contained in outlook.rds were synthesized from the original variables using the methods described by Volker and Vink (2021). You can access the original data here, and you can access the code used to process the data here. 3.4.2.1 Read in the outlook.rds dataset. Hint: An RDS file is an R object that’s been saved to a file. To read in this type of file, we use readRDS() from base R. Click to show code outlook <- readRDS("outlook.rds") 3.4.2.2 Summarize the outlook data to get a sense of their characteristics. Click to show code head(outlook) summary(outlook) ## d1 d2 d3 s1 ## Min. :1.000 Min. :1.000 Min. :1.000 Min. :1.000 ## 1st Qu.:3.000 1st Qu.:2.000 1st Qu.:3.000 1st Qu.:2.000 ## Median :4.000 Median :3.000 Median :4.000 Median :2.000 ## Mean :3.642 Mean :3.218 Mean :3.629 Mean :2.288 ## 3rd Qu.:4.000 3rd Qu.:4.000 3rd Qu.:4.000 3rd Qu.:3.000 ## Max. :5.000 Max. :5.000 Max. :5.000 Max. :4.000 ## s2 s3 s4 progress ## Min. :1.000 Min. :1.000 Min. :1.000 Min. : 1.000 ## 1st Qu.:1.000 1st Qu.:1.000 1st Qu.:2.000 1st Qu.: 5.000 ## Median :2.000 Median :2.000 Median :2.000 Median : 7.000 ## Mean :1.922 Mean :2.012 Mean :2.469 Mean : 6.432 ## 3rd Qu.:2.000 3rd Qu.:3.000 3rd Qu.:3.000 3rd Qu.: 8.000 ## Max. :4.000 Max. :4.000 Max. :4.000 Max. :10.000 ## merit lib2Con party disillusion ## Min. :1.000 Min. :1.000 republican : 332 Min. :1.000 ## 1st Qu.:4.000 1st Qu.:3.000 democrat :1264 1st Qu.:3.000 ## Median :5.000 Median :4.000 independent: 576 Median :3.667 ## Mean :4.826 Mean :3.998 other : 116 Mean :3.497 ## 3rd Qu.:6.000 3rd Qu.:5.000 3rd Qu.:4.000 ## Max. :7.000 Max. :7.000 Max. :5.000 ## success ## Min. :1.000 ## 1st Qu.:1.750 ## Median :2.000 ## Mean :2.173 ## 3rd Qu.:2.500 ## Max. :4.000 str(outlook) ## 'data.frame': 2288 obs. of 13 variables: ## $ d1 : num 4 4 4 5 5 4 5 4 4 4 ... ## $ d2 : num 4 2 4 4 3 5 4 2 4 5 ... ## $ d3 : num 4 4 4 5 4 4 4 3 3 4 ... ## $ s1 : num 3 3 4 2 2 2 2 1 3 3 ... ## $ s2 : num 2 2 2 1 1 2 1 1 2 2 ... ## $ s3 : num 3 2 4 1 2 1 1 1 3 2 ... ## $ s4 : num 3 3 3 1 2 3 3 2 2 2 ... ## $ progress : num 8 4 6 1 6 5 7 6 9 7 ... ## $ merit : num 6 5 5 4 3 4 2 5 5 5 ... ## $ lib2Con : num 5 6 4 1 4 4 4 4 4 5 ... ## $ party : Factor w/ 4 levels "republican","democrat",..: 1 3 3 2 2 2 2 2 4 1 ... ## $ disillusion: num 4 3.33 4 4.67 4 ... ## $ success : num 2.75 2.5 3.25 1.25 1.75 2 1.75 1.25 2.5 2.25 ... We will first use OLS regression to estimate a model encoding the following relations: Belief in the achievability of success, success, predicts perceived progress toward the American Dream, progress, as the focal effect. Disillusionment with the US Federal government, disillusion moderates the success \\(\\rightarrow\\) progress effect. Placement on the liberal-to-conservative continuum, lib2Con is partialed out as a covariate. 3.4.2.3 Draw the conceptual path diagram for the model described above. Click for explanation 3.4.2.4 Write out the regression equation necessary to evaluate the moderation hypothesis described above. Click for explanation \\[ Y_{progress} = \\beta_0 + \\beta_1 W_{lib2Con} + \\beta_2 X_{success} + \\beta_3 Z_{disillusion} + \\beta_4 XZ + \\varepsilon \\] 3.4.2.5 Use lm() to estimate the moderated regression model via OLS regression. Click to show code olsFit <- lm(progress ~ lib2Con + success * disillusion, data = outlook) 3.4.2.6 Summarize the fitted model and interpret the results. Is the moderation hypothesis supported? How does disillusionment level affect the focal effect? Click to show code summary(olsFit) ## ## Call: ## lm(formula = progress ~ lib2Con + success * disillusion, data = outlook) ## ## Residuals: ## Min 1Q Median 3Q Max ## -7.4315 -1.2525 0.1307 1.4369 5.6717 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 6.81128 0.62073 10.973 < 2e-16 *** ## lib2Con 0.03052 0.03040 1.004 0.3155 ## success 0.42360 0.25853 1.638 0.1015 ## disillusion -0.78002 0.16864 -4.625 3.95e-06 *** ## success:disillusion 0.17429 0.07273 2.396 0.0166 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 2.041 on 2283 degrees of freedom ## Multiple R-squared: 0.1385, Adjusted R-squared: 0.137 ## F-statistic: 91.74 on 4 and 2283 DF, p-value: < 2.2e-16 Click for explanation Yes, disillusion significantly moderates the relation between success and progress (\\(\\beta = 0.174\\), \\(t[2283] = 2.396\\), \\(p = 0.017\\)) such that the effect of success on progress increases as levels of disillusion increase, after controlling for lib2Con. The rockchalk package contains some useful routines for probing interactions estimated via lm(). Specifically, the plotslopes() function will estimate and plot simple slopes, and the testSlopes() function tests the simple slopes estimated by plotSlopes(). 3.4.2.7 Probe the interaction. Use the plotSlopes() and testSlopes() functions from the rockchalk package to conduct a simple slopes analysis for the model from 3.4.2.5. Click to show code library(rockchalk) ## Estimate and plot simple slopes: psOut <- plotSlopes(olsFit, plotx = "success", modx = "disillusion", modxVals = "std.dev") ## Test the simple slopes: tsOut <- testSlopes(psOut) ## Values of disillusion OUTSIDE this interval: ## lo hi ## -28.9332857 0.2672244 ## cause the slope of (b1 + b2*disillusion)success to be statistically significant ## View the results: tsOut$hypotests Note: The message printed by testSlopes() gives the boundaries of the Johnson-Neyman Region of Significance (Johnson & Neyman, 1936). Johnson-Neyman analysis is an alternative method of probing interactions that we have not covered in this course. For more information, check out Preacher, et al. (2006). We will now use lavaan to estimate the moderated regression model from above as a path analysis. 3.4.2.8 Define the model syntax for the path analytic version of the model described above. Parameterize the model as in the OLS regression. Use only observed items and scale scores. Click to show code pathMod <- ' progress ~ 1 + lib2Con + success + disillusion + success:disillusion ' 3.4.2.9 Estimate the path model on the outlook data. Click to show code pathFit <- sem(pathMod, data = outlook) 3.4.2.10 Summarize the fitted path model and interpret the results. Do the results match the OLS regression results? What proportion of the variability in progress is explained by this model? Hint: the function lavInspect() can be used to extract information from models Click to show code summary(pathFit) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 6 ## ## Number of observations 2288 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## progress ~ ## lib2Con 0.031 0.030 1.005 0.315 ## success 0.424 0.258 1.640 0.101 ## disillusion -0.780 0.168 -4.630 0.000 ## success:dsllsn 0.174 0.073 2.399 0.016 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .progress 6.811 0.620 10.985 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .progress 4.157 0.123 33.823 0.000 lavInspect(pathFit, "r2") ## progress ## 0.138 Click for explanation Yes, the estimates and inferential conclusions are all the same as in the OLS regression model. The model explains 13.85% of the variability in progress. The semTools package contains some helpful routines for probing interactions estimated via the lavaan() function (or one of it’s wrappers). Specifically, the probe2WayMC() and plotProbe() functions will estimate/test simple slopes and plot the estimated simple slopes, respectively. 3.4.2.11 Probe the interaction from 3.4.2.9 using semTools utilities. Use probe2WayMC() to estimate and test the simple slopes. Use plotProbe() to visualize the simple slopes. Define the simple slopes with the same conditional values of disillusion that you used in 3.4.2.7. Which simple slopes are significant? Do these results match the results from 3.4.2.7? Click to show code library(semTools) ## Define the conditional values at which to calculate simple slopes: condVals <- summarise(outlook, "m-sd" = mean(disillusion) - sd(disillusion), mean = mean(disillusion), "m+sd" = mean(disillusion) + sd(disillusion) ) %>% unlist() ## Compute simple slopes and intercepts: ssOut <- probe2WayMC(pathFit, nameX = c("success", "disillusion", "success:disillusion"), nameY = "progress", modVar = "disillusion", valProbe = condVals) ## Check the results: ssOut ## $SimpleIntcept ## disillusion est se z pvalue ## m-sd 2.719 4.690 0.231 20.271 0 ## mean 3.497 4.084 0.190 21.508 0 ## m+sd 4.274 3.477 0.230 15.122 0 ## ## $SimpleSlope ## disillusion est se z pvalue ## m-sd 2.719 0.897 0.083 10.792 0 ## mean 3.497 1.033 0.065 15.994 0 ## m+sd 4.274 1.169 0.088 13.223 0 ## Visualize the simple slopes: plotProbe(ssOut, xlim = range(outlook$success), xlab = "Ease of Personal Success", ylab = "Progress toward American Dream", legendArgs = list(legend = names(condVals)) ) Click for explanation Each of the simple slopes is significant. As level of disillusionment increases, the effect of success on progress also increases, and this effect is significant for all levels of disillusion considered here. These results match the simple slopes from the OLS regression analysis. End of In-Class Exercises 3 "],["efa.html", "4 EFA", " 4 EFA This week will be a general introduction to latent variables and scaling procedures. We will discuss several different aspects of exploratory factor analysis (EFA). Most notably: The differences between Principal Component Analyses (PCA) and Factor Analysis Model estimation and factor extraction methods Factor rotations You will have to make decisions regarding each of these aspects when conducting a factor analysis. We will also discuss reliability and factor scores as means of evaluating the properties of a scale. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-3.html", "4.1 Lecture", " 4.1 Lecture How do you know if you have measured the putative hypothetical construct that you intend to measure? The methods introduced in this lecture (namely, latent variables, factor analysis, and reliability analysis) can shed empirical light on this issue. In the social and behavioral sciences we’re often forced to measure key concepts indirectly. For example, we have no way of directly quantifying a person’s current level of depression, or their innate motivation, or their risk-aversion, or any of the other myriad psychological features that comprise the human mental state. In truth, we cannot really measure these hypothetical constructs at all, we must estimate latent representations thereof (though, psychometricians still use the language of physical measurement to describe this process). Furthermore, we can rarely estimate an adequate representation with only a single observed variable (e.g., question on a survey, score on a test, reading from a sensor). We generally need several observed variables to reliably represent a single hypothetical construct. For example, we cannot accurately determine someone’s IQ or socio-economic status based on their response to a single question; we need several questions that each tap into slightly different aspects of IQ or SES. Given multiple items measuring the same construct, we can use the methods discussed in this lecture (i.e., factor analysis and reliability analysis) to evaluate the quality of our measurement (i.e., how well we have estimated the underlying hypothetical construct). If we do well enough in this estimation task, we will be able to combine these estimated latent variables with the path analysis methods discussed in previous two weeks to produce the full structural equation models that we will cover at the end of this course. 4.1.1 Recording Notes: This week (and next), we’ll be re-using Caspar van Lissa’s old slides and lecture recording. So, you’ll see Caspar in the following video, and the slides will have a notably different flavor than our usual materials. Don’t be confused by any mention of “model fit” in the lecture. We haven’t covered model fit yet, but we will do so next week. 4.1.2 Slides You can download the lecture slides here. "],["reading-3.html", "4.2 Reading", " 4.2 Reading This week, you will read two papers. Reference 1 Preacher, K. J., & MacCullum, R. C. (2003). Repairing Tom Swift’s electric factor analysis machine, Understanding Statistics 2(1) 13–43. Questions 1 What is a latent variable? Give an example of a latent variable. What is factor analysis, and what can you investigate using this method? In the introduction, Preacher and Maccallum describe a “little jiffy” method of doing factor analysis. Briefly describe this little jiffy—or bad practice—method. Briefly explain the key differences between Principal Component Analyses (PCA) and Exploratory Factor Analyses (EFA). What is the purpose of factor rotation? Reference 2 Kestilä, E. (2006). Is there demand for radical right populism in the Finnish electorate? Scandinavian Political Studies 29(3), 169–191. Questions 2 What is the research question that the author tries to answer? Briefly describe the characteristics of the Radical Right Parties (RRP) in Europe. What are the two main explanations of support for RRP upon which this paper focuses? Does the empirical part of the paper reflect the theoretical framework well? Why or why not? According to the author, is Finland very different from other European countries on the main dependent variables? What is the author’s conclusion (i.e., how does the author answer the research question)? "],["at-home-exercises-3.html", "4.3 At-Home Exercises", " 4.3 At-Home Exercises In these exercises, you will attempt to replicate some of the analyses from the second reading for this week: Kestilä, E. (2006). Is there demand for radical right populism in the Finnish electorate? Scandinavian Political Studies 29(3), 169–191. The data for this practical were collected during the first round of the European Social Survey (ESS). The ESS is a repeated cross-sectional survey administered in 32 European countries. The first wave was collected in 2002, and two new waves have been collected each year since. You can find more info and access the data at https://www.europeansocialsurvey.org. The data we will analyze for this practical are contained in the file named ESSround1-a.sav. This file contains data for all respondents, but only includes those variables that you will need to complete the following exercises. 4.3.1 Load the ESSround1-a.sav dataset into R. Inspect the data after loading to make sure everything went well. Click to show code ## Load the 'haven' package: library(haven) library(tidySEM) ## Read the 'ESSround1-a.sav' data into a data frame called 'ess': ess <- read_spss("ESSround1-a.sav") ## Inspect the result: dim(ess) head(ess) descriptives(ess) ## [1] 42359 50 Click here for a description of the variables. Variable Description name Title of dataset essround ESS round edition Edition proddate Production date cntry Country idno Respondent’s identification number trstlgl Trust in the legal system trstplc Trust in the police trstun Trust in the United Nations trstep Trust in the European Parliament trstprl Trust in country’s parliament stfhlth State of health services in country nowadays stfedu State of education in country nowadays stfeco How satisfied with present state of economy in country stfgov How satisfied with the national government stfdem How satisfied with the way democracy works in country pltinvt Politicians interested in votes rather than peoples opinions pltcare Politicians in general care what people like respondent think trstplt Trust in politicians imsmetn Allow many/few immigrants of same race/ethnic group as majority imdfetn Allow many/few immigrants of different race/ethnic group from majority eimrcnt Allow many/few immigrants from richer countries in Europe eimpcnt Allow many/few immigrants from poorer countries in Europe imrcntr Allow many/few immigrants from richer countries outside Europe impcntr Allow many/few immigrants from poorer countries outside Europe qfimchr Qualification for immigration: christian background qfimwht Qualification for immigration: be white imwgdwn Average wages/salaries generally brought down by immigrants imhecop Immigrants harm economic prospects of the poor more than the rich imtcjob Immigrants take jobs away in country or create new jobs imbleco Taxes and services: immigrants take out more than they put in or less imbgeco Immigration bad or good for country’s economy imueclt Country’s cultural life undermined or enriched by immigrants imwbcnt Immigrants make country worse or better place to live imwbcrm Immigrants make country’s crime problems worse or better imrsprc Richer countries should be responsible for accepting people from poorer countries pplstrd Better for a country if almost everyone share customs and traditions vrtrlg Better for a country if a variety of different religions shrrfg Country has more than its fair share of people applying refugee status rfgawrk People applying refugee status allowed to work while cases considered gvrfgap Government should be generous judging applications for refugee status rfgfrpc Most refugee applicants not in real fear of persecution own countries rfggvfn Financial support to refugee applicants while cases considered rfgbfml Granted refugees should be entitled to bring close family members gndr Gender yrbrn Year of birth edulvl Highest level of education eduyrs Years of full-time education completed polintr How interested in politics lrscale Placement on left right scale One thing you might notice when inspecting the ess data is that most of the variables are stored as labelled vectors. When loading SPSS data, haven will use these labelled vectors to preserve the metadata associated with SPSS scale variables (i.e., variable labels and value labels). While it’s good to have this metadata available, we want to analyze these items as numeric variables and factors, so the value labels are only going to make our lives harder. Thankfully, the labelled package contains many routines for manipulating labelled vectors. We’ll deal with the numeric variables in just a bit, but our first task will be to covert grouping variables to factors. 4.3.2 Convert the cntry, gndr, edulvl, and polintr variables into factors. Use the as_factor() function to do the conversion. Convert edulvl and polintr to ordered factors. Click to see code library(dplyr) ess <- mutate(ess, country = as_factor(cntry), sex = as_factor(gndr), edulvl = as_factor(edulvl, ordered = TRUE), polintr = as_factor(polintr, ordered = TRUE) ) The ess dataset contains much more information than Kestilä (2006) used. Kestilä only analyzed data from the following ten countries: Austria Belgium Denmark Finland France Germany Italy Netherlands Norway Sweden So, our next task is to subset the data to only the relevant population. When we apply logical subsetting, we can select rows from a dataset based on logical conditions. In this case, we want to select only rows from the 10 countries listed above. 4.3.3 Subset the data to include only the 10 countries analyzed by Kestilä (2006). Inspect the subsetted data to check that everything went well. Hints: Use the %in% operator to create a logical vector that indicates which elements of the cntry variable are in the set of target counties. Use the droplevels() levels function to clean up empty factor levels. Click to show code ## Create a character vector naming the target countries: targets <- c("Austria", "Belgium", "Denmark", "Finland", "France", "Germany", "Italy", "Netherlands", "Norway", "Sweden") ## Select only those rows that come from a target country: ess <- filter(ess, country %in% targets) %>% # Subset rows droplevels() # Drop empty factor levels ## Inspect the result: dim(ess) ## [1] 19690 52 table(ess$country) ## ## Austria Belgium Germany Denmark Finland France ## 2257 1899 2919 1506 2000 1503 ## Italy Netherlands Norway Sweden ## 1207 2364 2036 1999 In keeping with common practice, we will treat ordinal Likert-type rating scales with five or more levels as continuous. Since some R routines will treat labelled vectors as discrete variables, we can make things easier for ourselves by converting all the labelled vectors in our data to numeric vectors. We can use the labelled::remove_val_labels() function to strip the value labels and convert all of the labelled vectors to numeric vectors. 4.3.4 Convert the remaining labelled vectors to numeric vectors. Click to see code ## If necessary, install the labelled package: # install.packages("labelled", repos = "https://cloud.r-project.org") ## Load the labelled package: library(labelled) ## Strip the value labels: ess <- remove_val_labels(ess) ## Check the effects: str(ess) ## tibble [19,690 × 52] (S3: tbl_df/tbl/data.frame) ## $ name : chr [1:19690] "ESS1e06_1" "ESS1e06_1" "ESS1e06_1" "ESS1e06_1" ... ## ..- attr(*, "label")= chr "Title of dataset" ## ..- attr(*, "format.spss")= chr "A9" ## ..- attr(*, "display_width")= int 14 ## $ essround: num [1:19690] 1 1 1 1 1 1 1 1 1 1 ... ## ..- attr(*, "label")= chr "ESS round" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 10 ## $ edition : chr [1:19690] "6.1" "6.1" "6.1" "6.1" ... ## ..- attr(*, "label")= chr "Edition" ## ..- attr(*, "format.spss")= chr "A3" ## ..- attr(*, "display_width")= int 9 ## $ proddate: chr [1:19690] "03.10.2008" "03.10.2008" "03.10.2008" "03.10.2008" ... ## ..- attr(*, "label")= chr "Production date" ## ..- attr(*, "format.spss")= chr "A10" ## ..- attr(*, "display_width")= int 12 ## $ cntry : num [1:19690] 1 18 1 1 18 1 2 18 1 18 ... ## ..- attr(*, "label")= chr "Country" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 7 ## $ idno : num [1:19690] 1 1 2 3 3 4 4 4 6 6 ... ## ..- attr(*, "label")= chr "Respondent's identification number" ## ..- attr(*, "format.spss")= chr "F9.0" ## ..- attr(*, "display_width")= int 11 ## $ trstlgl : num [1:19690] 10 6 8 4 8 10 9 7 7 7 ... ## ..- attr(*, "label")= chr "Trust in the legal system" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ trstplc : num [1:19690] 10 8 5 8 8 9 8 9 4 9 ... ## ..- attr(*, "label")= chr "Trust in the police" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ trstun : num [1:19690] 9 8 6 NA 5 8 NA 7 5 7 ... ## ..- attr(*, "label")= chr "Trust in the United Nations" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ trstep : num [1:19690] NA 3 0 7 3 7 0 3 4 6 ... ## ..- attr(*, "label")= chr "Trust in the European Parliament" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ trstprl : num [1:19690] 9 7 0 6 8 8 10 2 6 8 ... ## ..- attr(*, "label")= chr "Trust in country's parliament" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ stfhlth : num [1:19690] 10 4 0 7 6 8 NA 6 3 5 ... ## ..- attr(*, "label")= chr "State of health services in country nowadays" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ stfedu : num [1:19690] 8 7 7 5 8 7 NA 7 6 7 ... ## ..- attr(*, "label")= chr "State of education in country nowadays" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ stfeco : num [1:19690] 7 6 0 7 8 6 NA 9 8 9 ... ## ..- attr(*, "label")= chr "How satisfied with present state of economy in country" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ stfgov : num [1:19690] 7 7 0 7 6 3 NA 5 5 7 ... ## ..- attr(*, "label")= chr "How satisfied with the national government" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ stfdem : num [1:19690] 8 5 5 5 7 7 NA 7 7 9 ... ## ..- attr(*, "label")= chr "How satisfied with the way democracy works in country" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ pltinvt : num [1:19690] 1 3 1 1 4 1 1 3 2 3 ... ## ..- attr(*, "label")= chr "Politicians interested in votes rather than peoples opinions" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ pltcare : num [1:19690] 1 4 1 1 4 3 2 5 2 3 ... ## ..- attr(*, "label")= chr "Politicians in general care what people like respondent think" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ trstplt : num [1:19690] 0 5 0 2 5 4 8 2 4 6 ... ## ..- attr(*, "label")= chr "Trust in politicians" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imsmetn : num [1:19690] 4 3 2 3 2 1 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants of same race/ethnic group as majority" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imdfetn : num [1:19690] 3 3 2 3 2 2 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants of different race/ethnic group from majority" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ eimrcnt : num [1:19690] 4 2 2 2 3 1 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from richer countries in Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ eimpcnt : num [1:19690] 3 2 2 2 2 2 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from poorer countries in Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imrcntr : num [1:19690] 3 3 2 2 2 1 NA 2 NA 2 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from richer countries outside Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ impcntr : num [1:19690] 3 2 2 3 2 1 NA 2 NA 2 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from poorer countries outside Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ qfimchr : num [1:19690] 4 2 0 6 2 0 99 0 1 2 ... ## ..- attr(*, "label")= chr "Qualification for immigration: christian background" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ qfimwht : num [1:19690] 1 0 0 0 0 0 99 0 0 1 ... ## ..- attr(*, "label")= chr "Qualification for immigration: be white" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imwgdwn : num [1:19690] 3 4 2 2 3 3 NA 4 NA 4 ... ## ..- attr(*, "label")= chr "Average wages/salaries generally brought down by immigrants" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imhecop : num [1:19690] 2 2 1 4 3 2 NA 3 NA 2 ... ## ..- attr(*, "label")= chr "Immigrants harm economic prospects of the poor more than the rich" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imtcjob : num [1:19690] 7 5 6 5 7 10 NA 8 NA 4 ... ## ..- attr(*, "label")= chr "Immigrants take jobs away in country or create new jobs" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imbleco : num [1:19690] 9 4 2 NA 3 10 NA 9 NA 6 ... ## ..- attr(*, "label")= chr "Taxes and services: immigrants take out more than they put in or less" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imbgeco : num [1:19690] 4 3 10 7 5 10 NA 8 NA 5 ... ## ..- attr(*, "label")= chr "Immigration bad or good for country's economy" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imueclt : num [1:19690] 9 4 10 5 4 10 NA 9 NA 3 ... ## ..- attr(*, "label")= chr "Country's cultural life undermined or enriched by immigrants" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imwbcnt : num [1:19690] 7 3 5 5 5 10 NA 8 NA 5 ... ## ..- attr(*, "label")= chr "Immigrants make country worse or better place to live" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imwbcrm : num [1:19690] 3 3 5 2 3 5 NA 5 NA 3 ... ## ..- attr(*, "label")= chr "Immigrants make country's crime problems worse or better" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imrsprc : num [1:19690] 2 2 1 4 1 2 NA 1 1 3 ... ## ..- attr(*, "label")= chr "Richer countries should be responsible for accepting people from poorer countries" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ pplstrd : num [1:19690] 2 4 2 2 3 4 NA 4 4 2 ... ## ..- attr(*, "label")= chr "Better for a country if almost everyone share customs and traditions" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ vrtrlg : num [1:19690] 3 5 3 2 4 1 NA 4 2 3 ... ## ..- attr(*, "label")= chr "Better for a country if a variety of different religions" ## ..- attr(*, "format.spss")= chr "F1.0" ## $ shrrfg : num [1:19690] 3 2 1 1 3 3 NA 3 4 3 ... ## ..- attr(*, "label")= chr "Country has more than its fair share of people applying refugee status" ## ..- attr(*, "format.spss")= chr "F1.0" ## $ rfgawrk : num [1:19690] 2 2 1 2 2 2 NA 2 1 2 ... ## ..- attr(*, "label")= chr "People applying refugee status allowed to work while cases considered" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ gvrfgap : num [1:19690] 4 3 2 4 2 2 NA 3 2 4 ... ## ..- attr(*, "label")= chr "Government should be generous judging applications for refugee status" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ rfgfrpc : num [1:19690] 4 3 2 4 4 4 NA 4 3 4 ... ## ..- attr(*, "label")= chr "Most refugee applicants not in real fear of persecution own countries" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ rfggvfn : num [1:19690] 2 3 2 4 3 2 NA 2 2 2 ... ## ..- attr(*, "label")= chr "Financial support to refugee applicants while cases considered" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ rfgbfml : num [1:19690] 2 3 1 2 2 1 NA 4 2 3 ... ## ..- attr(*, "label")= chr "Granted refugees should be entitled to bring close family members" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ gndr : num [1:19690] 1 2 1 2 2 1 NA 2 2 1 ... ## ..- attr(*, "label")= chr "Gender" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 6 ## $ yrbrn : num [1:19690] 1949 1978 1953 1940 1964 ... ## ..- attr(*, "label")= chr "Year of birth" ## ..- attr(*, "format.spss")= chr "F4.0" ## ..- attr(*, "display_width")= int 7 ## $ edulvl : Ord.factor w/ 7 levels "Not completed primary education"<..: NA 4 NA NA 4 NA NA 7 NA 6 ... ## $ eduyrs : num [1:19690] 11 16 14 9 12 18 NA 17 15 17 ... ## ..- attr(*, "label")= chr "Years of full-time education completed" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ polintr : Ord.factor w/ 4 levels "Very interested"<..: 3 3 1 2 3 2 1 4 3 3 ... ## $ lrscale : num [1:19690] 6 7 6 5 8 5 NA 8 5 7 ... ## ..- attr(*, "label")= chr "Placement on left right scale" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ country : Factor w/ 10 levels "Austria","Belgium",..: 1 9 1 1 9 1 2 9 1 9 ... ## $ sex : Factor w/ 2 levels "Male","Female": 1 2 1 2 2 1 NA 2 2 1 ... descriptives(ess) Click for explanation Note that the numeric variables are now simple numeric vectors, but the variable labels have been retained as column attributes (which is probably useful). If we want to completely nuke the labelling information, we can use the labelled::remove_labels() function to do so. In addition to screening with summary statistics, we can also visualize the variables’ distributions. You have already created a few such visualizations for single variables. Now, we will use a few tricks to efficiently plot each of our target variables. The first step in this process will be to convert the interesting part of our data from “wide format” (one column per variable) into “long format” (one column of variable names, one column of data values). The pivot_longer() function from the tidyr package provides a convenient way to execute this conversion. 4.3.5 Use tidyr::pivot_longer() to create a long-formatted data frame from the target variables in ess. The target variables are all columns from trstlgl to rfgbfml. Click to show code ## Load the tidyr package: library(tidyr) ## Convert the target variables into a long-formatted data frame: ess_plot <- pivot_longer(ess, cols = trstlgl:rfgbfml, # Which columns to convert names_to = "variable", # Name for the new grouping variable values_to = "value") # Name for the column of stacked values The next step in the process will be to plot the variables using ggplot(). In the above code, I’ve named the new grouping variable variable and the new stacked data variable value. So, to create one plot for each (original, wide-format) variable, we will use the facet_wrap() function to facet the plots of value on the variable column (i.e., create a separate conditional plot of value for each unique value in variable). 4.3.6 Use ggplot() with an appropriate geom (e.g., geom_histogram(), geom_density(), geom_boxplot()) and facet_wrap() to visualize each of the target variables. Hint: To implement the faceting, simply add facet_wrap(~ variable, scales = \"free_x\") to the end of your ggplot() call (obviously, replacing “variable” with whatever you named the grouping variable in your pivot_longer() call). Click to show code library(ggplot2) ggplot(ess_plot, aes(x = value)) + geom_histogram() + # Create a histogram facet_wrap(~ variable, scales = "free_x") # Facet on 'variable' Click for explanation Notice that the variables are actually discrete (i.e., each variable takes only a few integer values). However, most variables look relatively normal despite being categorical. So, we’ll bend the rules a bit and analyze these variables as continuous. It also looks like there’s something weird going on with qfimchr and qfimwht. More on that below. 4.3.7 Check the descriptives for the target variables again. Do you see any remaining issues? Click to show code select(ess, trstlgl:rfgbfml) %>% descriptives() Click for explanation The variables qfimchr and qfimwht both contain values that fall outside the expected range for our survey responses: 77, 88, and 99. In SPSS, these were labeled as “Refusal” “Don’t know” and “No answer” respectively, and would not have contributed to the analysis. 4.3.8 Correct any remaining issues you found above. Click to show code ess <- ess %>% mutate(across(c(qfimchr, qfimwht), na_if, 77)) %>% mutate(across(c(qfimchr, qfimwht), na_if, 88)) %>% mutate(across(c(qfimchr, qfimwht), na_if, 99)) ## Check the results: select(ess, trstlgl:rfgbfml) %>% descriptives() Click to show explanation Here, we need to tell R that these values should be considered missing, or NA. Otherwise they will contribute the numeric value to the analysis, as though someone had provided an answer of 77 on a 10-point scale. We’ve done quite a bit of data processing, and we’ll continue to use these data for several future practicals, so it would be a good idea to save the processed dataset for later use. When saving data that you plan to analyze in R, you will usually want to use the R Data Set (RDS) format. Datasets saved in RDS format retain all of their attributes and formatting (e.g., factor are still factors, missing values are coded as NA, etc.). So, you don’t have to redo any data processing before future analyses. 4.3.9 Use the saveRDS() function to save the processed dataset. Click to show code ## Save the processed data: saveRDS(ess, "ess_round1.rds") Now, we’re ready to run the analyses and see if we can replicate the Kestilä (2006) results. 4.3.10 Run two principal component analyses (PCA): one for trust in politics, one for attitudes towards immigration. Use the principal() function from the psych package. Use exactly the same specifications as Kestilä (2006) concerning the estimation method, rotation, number of components extracted, etc. Hints: Remember that you can view the help file for psych::principal() by running ?psych::principal or, if the psych package already loaded, simply running ?principal. When you print the output from psych::principal(), you can use the cut option to hide any factor loadings smaller than a given threshold. You could consider hiding any loadings smaller than those reported by Kestilä (2006) to make the output easier to interpret. Click to show code Trust in politics Kestilä extracted three components with VARIMAX rotation. ## Load the psych package: library(psych) ## Run the PCA: pca_trust <- select(ess, trstlgl:trstplt) %>% principal(nfactors = 3, rotate = "varimax") ## Print the results: print(pca_trust, cut = 0.3, digits = 3) ## Principal Components Analysis ## Call: principal(r = ., nfactors = 3, rotate = "varimax") ## Standardized loadings (pattern matrix) based upon correlation matrix ## RC3 RC2 RC1 h2 u2 com ## trstlgl 0.779 0.669 0.331 1.21 ## trstplc 0.761 0.633 0.367 1.18 ## trstun 0.675 0.556 0.444 1.44 ## trstep 0.651 0.332 0.549 0.451 1.57 ## trstprl 0.569 0.489 0.650 0.350 2.49 ## stfhlth 0.745 0.567 0.433 1.04 ## stfedu 0.750 0.603 0.397 1.14 ## stfeco 0.711 0.300 0.616 0.384 1.44 ## stfgov 0.634 0.377 0.587 0.413 1.88 ## stfdem 0.369 0.568 0.325 0.564 0.436 2.38 ## pltinvt 0.817 0.695 0.305 1.08 ## pltcare 0.811 0.695 0.305 1.11 ## trstplt 0.510 0.611 0.716 0.284 2.40 ## ## RC3 RC2 RC1 ## SS loadings 2.942 2.668 2.490 ## Proportion Var 0.226 0.205 0.192 ## Cumulative Var 0.226 0.432 0.623 ## Proportion Explained 0.363 0.329 0.307 ## Cumulative Proportion 0.363 0.693 1.000 ## ## Mean item complexity = 1.6 ## Test of the hypothesis that 3 components are sufficient. ## ## The root mean square of the residuals (RMSR) is 0.07 ## with the empirical chi square 15240.94 with prob < 0 ## ## Fit based upon off diagonal values = 0.967 Attitudes toward immigration Kestilä extracted five components with VARIMAX rotation. pca_att <- select(ess, imsmetn:rfgbfml) %>% principal(nfactors = 5, rotate = "varimax") print(pca_att, cut = 0.3, digits = 3) ## Principal Components Analysis ## Call: principal(r = ., nfactors = 5, rotate = "varimax") ## Standardized loadings (pattern matrix) based upon correlation matrix ## RC2 RC1 RC5 RC3 RC4 h2 u2 com ## imsmetn 0.797 0.725 0.275 1.30 ## imdfetn 0.775 0.794 0.206 1.70 ## eimrcnt 0.827 0.715 0.285 1.09 ## eimpcnt 0.800 0.789 0.211 1.49 ## imrcntr 0.835 0.747 0.253 1.15 ## impcntr 0.777 0.782 0.218 1.63 ## qfimchr 0.813 0.688 0.312 1.08 ## qfimwht 0.752 0.637 0.363 1.26 ## imwgdwn 0.807 0.712 0.288 1.19 ## imhecop 0.747 0.669 0.331 1.42 ## imtcjob 0.569 0.334 0.484 0.516 1.99 ## imbleco 0.703 0.554 0.446 1.25 ## imbgeco 0.698 0.605 0.395 1.52 ## imueclt 0.568 -0.340 0.545 0.455 2.43 ## imwbcnt 0.673 0.633 0.367 1.87 ## imwbcrm 0.655 0.478 0.522 1.23 ## imrsprc 0.614 0.440 0.560 1.34 ## pplstrd 0.324 -0.551 0.468 0.532 2.11 ## vrtrlg -0.345 0.471 0.419 0.581 2.67 ## shrrfg 0.365 -0.352 0.418 0.582 4.16 ## rfgawrk 0.614 0.396 0.604 1.10 ## gvrfgap 0.691 0.559 0.441 1.35 ## rfgfrpc -0.387 0.327 0.673 3.34 ## rfggvfn 0.585 0.417 0.583 1.46 ## rfgbfml 0.596 0.460 0.540 1.61 ## ## RC2 RC1 RC5 RC3 RC4 ## SS loadings 4.374 3.393 2.774 2.199 1.723 ## Proportion Var 0.175 0.136 0.111 0.088 0.069 ## Cumulative Var 0.175 0.311 0.422 0.510 0.579 ## Proportion Explained 0.302 0.235 0.192 0.152 0.119 ## Cumulative Proportion 0.302 0.537 0.729 0.881 1.000 ## ## Mean item complexity = 1.7 ## Test of the hypothesis that 5 components are sufficient. ## ## The root mean square of the residuals (RMSR) is 0.05 ## with the empirical chi square 29496.06 with prob < 0 ## ## Fit based upon off diagonal values = 0.976 Feature engineering (i.e., creating new variables by combining and/or transforming existing variables) is one of the most common applications of PCA. PCA is a dimension reduction technique that distills the most salient information from a set of variables into a (smaller) set of component scores. Hence, PCA can be a good way of creating aggregate items (analogous to weighted scale scores) when the data are not collected with validated scales. Principal component scores are automatically generated when we run the PCA. If we want to use these scores in subsequent analyses (e.g., as predictors in a regression model), we usually add them to our dataset as additional columns. 4.3.11 Add the component scores produced by the analyses you ran above to the ess data frame. Give each component score an informative name, based on your interpretation of the factor loading matrix I.e., What hypothetical construct do you think each component represents given the items that load onto it? Hints: You can use the data.frame() function to join multiple objects into a single data frame. You can use the colnames() function to assign column names to a matrix or data frame. 1. Extract the component scores Click to show code ## Save the component scores in stand-alone matrices: trust_scores <- pca_trust$scores att_scores <- pca_att$scores ## Inspect the result: head(trust_scores) ## RC3 RC2 RC1 ## [1,] NA NA NA ## [2,] 0.09755193 -0.01552183 0.994954 ## [3,] 0.23069626 -1.53162604 -2.022642 ## [4,] NA NA NA ## [5,] -0.21112678 0.84370377 1.200007 ## [6,] 1.86596955 0.31083233 -1.062603 summary(trust_scores) ## RC3 RC2 RC1 ## Min. :-4.035 Min. :-3.706 Min. :-3.139 ## 1st Qu.:-0.527 1st Qu.:-0.652 1st Qu.:-0.649 ## Median : 0.155 Median : 0.094 Median : 0.092 ## Mean : 0.055 Mean : 0.015 Mean : 0.049 ## 3rd Qu.: 0.727 3rd Qu.: 0.742 3rd Qu.: 0.742 ## Max. : 3.302 Max. : 3.452 Max. : 3.539 ## NA's :4912 NA's :4912 NA's :4912 head(att_scores) ## RC2 RC1 RC5 RC3 RC4 ## [1,] 1.9873715 1.3233586 -0.8382499 -0.02172765 -0.0908143 ## [2,] 0.1692841 -1.2178436 -0.5016936 -0.21749066 0.6758844 ## [3,] -0.3630480 0.3260383 -1.5133423 -0.51405480 -2.2071787 ## [4,] NA NA NA NA NA ## [5,] -0.1137484 -0.7891232 -1.4732563 -0.05843873 0.4110692 ## [6,] -0.9195530 2.8231404 -0.3480398 -0.75699796 -1.3230602 summary(att_scores) ## RC2 RC1 RC5 RC3 ## Min. :-3.660 Min. :-3.929 Min. :-3.824 Min. :-2.764 ## 1st Qu.:-0.616 1st Qu.:-0.585 1st Qu.:-0.656 1st Qu.:-0.748 ## Median :-0.085 Median : 0.062 Median :-0.008 Median :-0.121 ## Mean :-0.013 Mean : 0.012 Mean : 0.021 Mean : 0.014 ## 3rd Qu.: 0.680 3rd Qu.: 0.654 3rd Qu.: 0.652 3rd Qu.: 0.698 ## Max. : 3.743 Max. : 4.584 Max. : 4.108 Max. : 4.084 ## NA's :5447 NA's :5447 NA's :5447 NA's :5447 ## RC4 ## Min. :-3.784 ## 1st Qu.:-0.683 ## Median : 0.046 ## Mean : 0.003 ## 3rd Qu.: 0.717 ## Max. : 3.254 ## NA's :5447 Click for explanation The object produced by psych::principal() is simply list, and the component scores are already stored therein. So, to extract the component scores, we simply use the $ operator to extract them. 2. Name the component scores Click to show code ## Check names (note the order): colnames(trust_scores) ## [1] "RC3" "RC2" "RC1" colnames(att_scores) ## [1] "RC2" "RC1" "RC5" "RC3" "RC4" ## Give informative names: colnames(trust_scores) <- c("Trust_Institutions", "Satisfaction", "Trust_Politicians") colnames(att_scores) <- c("Quantity", "Effects", "Refugees", "Diversity", "Economic") 3. Add the component scores to the dataset Click to show code # Add the component scores to the 'ess' data: ess <- data.frame(ess, trust_scores, att_scores) 4.3.12 Were you able to replicate the results of Kestilä (2006)? Click for explanation Yes, more-or-less. Although the exact estimates differ somewhat, the general pattern of factor loadings in Kestilä (2006) matches what we found here. End of At-Home Exercises "],["in-class-exercises-3.html", "4.4 In-Class Exercises", " 4.4 In-Class Exercises In these exercises, we will continue with our re-analysis/replication of the Kestilä (2006) results. Rather than attempting a direct replication, we will now redo the analysis using exploratory factor analysis (EFA). 4.4.1 Load the ess_round1.rds dataset. These are the data that we saved after the data processing in the At-Home Exercises. Click to show code ess <- readRDS("ess_round1.rds") 4.4.2 Kestilä (2006) claimed that running a PCA is a good way to test if the questions in the ESS measure attitudes towards immigration and trust in politics. Based on what you’ve learned from the readings and lectures, do you agree with this position? Click for explanation Hopefully not. PCA is not a method for estimating latent measurement structure; PCA is a dimension reduction technique that tries to summarize a set of data with a smaller set of component scores. If we really want to estimate the factor structure underlying a set of observed variables, we should use EFA. 4.4.3 Suppose you had to construct the trust in politics and attitude towards immigration scales described by Kestilä (2006) based on the theory and background information presented in that article. What type of analysis would you choose? What key factors would influence your decision? Click for explanation We are trying to estimate meaningful latent factors, so EFA would be an appropriate method. The theory presented by Kestilä (2006) did not hypothesize a particular number of factors, so we would need to use appropriate techniques to estimate the best number. In particular, combining information from: Scree plots Parallel analysis Substantive interpretability of the (rotated) factor loadings Since the factors are almost certainly correlated, we should apply an oblique rotation. We will now rerun the two PCAs that you conducted for the At-Home Exercises using EFA. We will estimate the EFA models using the psych::fa() function, but we need to know how many factors to extract. We could simply estimate a range of solutions and compare the results. We can restrict the range of plausible solutions and save some time by first checking/plotting the eigenvalues and running parallel analysis. 4.4.4 Estimate the number of latent factors underlying the Trust items based on the eigenvalues, the scree plot, and parallel analysis. How many factors are suggested by each method? 1. Eigenvalue estimation Click to show code ## Load the psych package: library(psych) ## Run a trivial EFA on the 'trust' items efa_trust0 <- select(ess, trstlgl:trstplt) %>% fa(nfactors = 1, rotate = "none") Click for explanation (EFA) First, we run a trivial EFA using the psych::fa() function to estimate the eigenvalues. We don’t care about the factors yet, so we can extract a single factor. We also don’t care about interpretable solutions, so we don’t need rotation. ## View the estimated eigenvalues: round(efa_trust0$values, digits = 3) ## [1] 4.980 0.716 0.482 0.165 0.069 0.014 -0.066 -0.092 -0.182 -0.207 ## [11] -0.284 -0.296 -0.319 Click for explanation (eigenvalue extraction) We can check the eigenvalues to see what proportion of the observed variance is accounted for by each additional factor we may extract. Since only one eigenvalue is greater than one, the so-called “Kaiser Criterion” would suggest extracting a single factor. The Kaiser Criterion is not a valid way to select the number of factors in EFA. So, we don’t want to rely on this information alone. We can still use the eigenvalues to help us with factor enumeration, though. One way to do so is by plotting the eigenvalues in a scree plot. 2. Scree plot Click to show code Given a vector of estimated eigenvalues, we can create a scree plot using ggplot() and the geom_line() or geom_path() geometry. library(ggplot2) library(magrittr) efa_trust0 %$% data.frame(y = values, x = 1:length(values)) %>% ggplot(aes(x, y)) + geom_line() + xlab("No. of Factors") + ylab("Eigenvalues") We can also use the psych::scree() function to create a scree plot directly from the data. select(ess, trstlgl:trstplt) %>% scree(pc = FALSE) Click for explanation (scree plot) Although the scree plot provides useful information, we need to interpret that information subjectively, and the conclusions are sometimes ambiguous, in this case. In this case, the plot seems to suggest either one or three components, depending on where we consider the “elbow” to lie. As recommended in the lecture, we can also use “parallel analysis” (Horn, 1965) to provide more objective information about the number of factors. We’ll use the psych::fa.parallel() function to implement parallel analysis. Parallel analysis relies on randomly simulated/permuted data, so we should set a seed to make sure our results are reproducible. We can set the fa = \"fa\" option to get only the results for EFA. 3. Parallel Analysis Click to show code ## Set the random number seed: set.seed(235711) ## Run the parallel analysis: pa_trust <- select(ess, trstlgl:trstplt) %>% fa.parallel(fa = "fa") ## Parallel analysis suggests that the number of factors = 6 and the number of components = NA Click for explanation The results of the parallel analysis suggest 6 factors. If you’ve been paying close attention, you may have noticed that we need to compute the eigenvalues from the original data to run parallel analysis. Hence, we don’t actually need to run a separate EFA to estimate the eigenvalues. ## View the eigenvalues estimated during the parallel analysis: pa_trust$fa.values ## [1] 4.97995262 0.71644127 0.48201040 0.16517645 0.06885820 0.01422241 ## [7] -0.06606777 -0.09225113 -0.18231333 -0.20740917 -0.28415857 -0.29573407 ## [13] -0.31877470 ## Compare to the version from the EFA: pa_trust$fa.values - efa_trust0$values ## [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 ## Recreate the scree plot from above: pa_trust %$% data.frame(y = fa.values, x = 1:length(fa.values)) %>% ggplot(aes(x, y)) + geom_line() + xlab("No. of Factors") + ylab("Eigenvalues") Of course, we also see the same scree plot printed as part of the parallel analysis. So, there’s really no reason to create a separate scree plot, at all, if we’re doing parallel analysis. 4. Conclusion Click for explanation The different criteria disagree on how many factors we should extract, but we have narrowed the range. Based on the scree plot and parallel analysis, we should consider solutions for 3 to 6 factors. We need to examine the factor loadings to see which solution makes the most substantive sense. 4.4.5 Do the same analysis for the attitudes toward immigration items. Click to show code This time, we’ll start by running the parallel analysis and get the eigenvalues and scree plot from psych::fa.parallel(). ## Set the seed: set.seed(235711) ## Run parallel analysis on the 'attitudes' items: pa_att <- select(ess, imsmetn:rfgbfml) %>% fa.parallel(fa = "fa") ## Parallel analysis suggests that the number of factors = 7 and the number of components = NA ## Check the eigenvalues: round(pa_att$fa.values, digits = 3) ## [1] 7.895 1.449 0.734 0.533 0.313 0.156 0.121 0.019 -0.001 -0.064 ## [11] -0.083 -0.103 -0.119 -0.131 -0.150 -0.175 -0.185 -0.200 -0.212 -0.233 ## [21] -0.239 -0.247 -0.334 -0.422 -0.427 Click for explanation For the attitudes toward immigration analysis, the results are even more ambiguous than they were for the trust items. The Kaiser Criterion suggests 2 factors. The scree plot is hopelessly ambiguous. At least 3 factors? No more than 9 factors? Parallel analysis suggests 7 factors Based on the scree plot and parallel analysis, it seems reasonable to consider solutions for 3 to 7 factors. Again, we need to check the substantive interpretation to choose the most reasonable solution. To evaluate the substantive interpretability of the different solutions, we need to estimate the full EFA models for each candidate number of factors. We then compare the factor loadings across solutions to see which set of loadings define the most reasonable set of latent variables. 4.4.6 For the trust items, estimate the EFA models for each plausible number of components that you identified above. Use the psych::fa() function to estimate the models. You will need to specify a few key options. The data (including only the variables you want to analyze) The number of factors that you want to extract The rotation method The estimation method The method of estimating factor scores Hint: You can save yourself a lot of typing/copy-pasting (and the attendant chances of errors) by using a for() loop to iterate through numbers of factors. Click to show code ## Define an empty list to hold all of our fitted EFA objects: efa_trust <- list() ## Loop through the interesting numbers of factors and estimate an EFA for each: for(i in 3:6) efa_trust[[as.character(i)]] <- ess %>% select(trstlgl:trstplt) %>% fa(nfactors = i, # Number of factors = Loop index rotate = "promax", # Oblique rotation scores = "Bartlett") # Estimate factor scores with WLS 4.4.7 Repeat the above analysis for the attitudes items. Click to show code efa_att <- list() for(i in 3:7) efa_att[[as.character(i)]] <- ess %>% select(imsmetn:rfgbfml) %>% fa(nfactors = i, rotate = "promax", scores = "Bartlett") 4.4.8 Compare the factor loading matrices from the models estimated from the Trust items, and select the best solution. Hints: The factor loadings are stored in the loadings slot of the object returned by psych::fa(). Looping can also be useful here. Click to show code for(x in efa_trust) print(x$loadings) ## ## Loadings: ## MR3 MR2 MR1 ## trstlgl 0.839 -0.115 ## trstplc 0.763 -0.218 ## trstun 0.579 0.161 ## trstep 0.554 0.198 ## trstprl 0.444 0.342 ## stfhlth 0.656 -0.125 ## stfedu 0.695 -0.157 ## stfeco -0.102 0.704 0.146 ## stfgov 0.593 0.226 ## stfdem 0.183 0.476 0.150 ## pltinvt 0.813 ## pltcare 0.808 ## trstplt 0.330 0.526 ## ## MR3 MR2 MR1 ## SS loadings 2.299 2.016 1.970 ## Proportion Var 0.177 0.155 0.152 ## Cumulative Var 0.177 0.332 0.483 ## ## Loadings: ## MR2 MR1 MR4 MR3 ## trstlgl 0.797 ## trstplc 0.725 ## trstun 0.656 0.113 ## trstep 1.003 -0.175 ## trstprl 0.121 0.455 0.200 0.112 ## stfhlth 0.663 -0.106 ## stfedu 0.704 -0.110 0.100 ## stfeco 0.729 ## stfgov 0.631 0.175 -0.149 ## stfdem 0.501 0.107 0.115 ## pltinvt 0.855 ## pltcare -0.103 0.863 ## trstplt 0.479 0.340 ## ## MR2 MR1 MR4 MR3 ## SS loadings 2.161 1.952 1.722 1.239 ## Proportion Var 0.166 0.150 0.132 0.095 ## Cumulative Var 0.166 0.316 0.449 0.544 ## ## Loadings: ## MR1 MR4 MR5 MR3 MR2 ## trstlgl 0.935 ## trstplc 0.810 ## trstun 0.505 0.168 ## trstep -0.138 1.128 -0.108 -0.154 ## trstprl 0.359 0.250 0.140 0.201 -0.104 ## stfhlth 0.557 ## stfedu 0.752 ## stfeco 0.710 -0.118 0.172 ## stfgov 0.973 -0.132 ## stfdem 0.556 0.153 ## pltinvt 0.882 ## pltcare 0.855 ## trstplt 0.288 0.308 0.313 ## ## MR1 MR4 MR5 MR3 MR2 ## SS loadings 2.019 1.716 1.655 1.674 0.936 ## Proportion Var 0.155 0.132 0.127 0.129 0.072 ## Cumulative Var 0.155 0.287 0.415 0.543 0.615 ## ## Loadings: ## MR5 MR1 MR4 MR3 MR2 MR6 ## trstlgl 0.980 ## trstplc 0.655 ## trstun 0.911 ## trstep -0.116 0.739 0.163 ## trstprl 0.197 0.577 0.138 ## stfhlth 0.614 ## stfedu 0.771 ## stfeco 0.689 -0.123 0.144 ## stfgov 0.891 ## stfdem 0.513 0.144 ## pltinvt 0.816 ## pltcare 0.778 ## trstplt 0.706 0.193 ## ## MR5 MR1 MR4 MR3 MR2 MR6 ## SS loadings 1.606 1.417 1.442 1.327 1.014 0.879 ## Proportion Var 0.124 0.109 0.111 0.102 0.078 0.068 ## Cumulative Var 0.124 0.233 0.343 0.446 0.524 0.591 Click for explanation Note: Any factor loadings with magnitude lower than 0.1 are suppressed in above output. The factor loadings matrix indicates how strongly each latent factor (columns) associates with the observed items (rows). We can interpret these factor loadings in the same way that we would interpret regression coefficients (indeed, a factor analytic model can be viewed as a multivariate regression model wherein the latent factors are the predictors and the observed items are the outcomes). A higher factor loading indicates a stronger association between the item and factor linked by that loading. Items with high factor loadings are “good” indicators of the respective factors. Items with only very low loadings do not provide much information about any factor. You may want to exclude such items from your analysis. Note that the size of the factor loadings depends on the number of factors. So, you should only consider excluding an observed item after you have chosen the number of latent factors. When we print the loading matrix, we see additional information printed below the factor loadings. Proportion Var: What proportion of the items’ variance is explained by each of the factors. Cumulative Var: How much variance the factors explain, in total. If you estimated as many factors as items, then the Cumulative Var for the final factor would be 1.00 (i.e., 100%). 4.4.9 Compare the factor loading matrices from the models estimated from the Attitudes items, and select the best solution. Click to show code for(x in efa_att) print(x$loadings) ## ## Loadings: ## MR1 MR2 MR3 ## imsmetn 0.802 ## imdfetn 0.754 0.106 ## eimrcnt 0.843 ## eimpcnt 0.814 ## imrcntr 0.857 ## impcntr 0.769 ## qfimchr 0.235 0.858 ## qfimwht 0.132 0.719 ## imwgdwn 0.293 -0.181 ## imhecop 0.371 -0.162 ## imtcjob 0.619 ## imbleco 0.702 ## imbgeco 0.687 ## imueclt 0.561 -0.207 ## imwbcnt 0.732 ## imwbcrm 0.637 ## imrsprc -0.494 -0.125 ## pplstrd 0.249 -0.413 ## vrtrlg -0.275 0.240 ## shrrfg 0.514 -0.111 ## rfgawrk -0.386 ## gvrfgap -0.601 -0.148 ## rfgfrpc 0.432 ## rfggvfn -0.489 ## rfgbfml -0.545 ## ## MR1 MR2 MR3 ## SS loadings 4.819 3.950 1.683 ## Proportion Var 0.193 0.158 0.067 ## Cumulative Var 0.193 0.351 0.418 ## ## Loadings: ## MR2 MR4 MR1 MR3 ## imsmetn 0.788 ## imdfetn 0.731 0.153 0.110 ## eimrcnt 0.855 -0.143 ## eimpcnt 0.790 0.165 ## imrcntr 0.860 ## impcntr 0.743 0.182 ## qfimchr -0.122 0.853 ## qfimwht 0.723 ## imwgdwn 0.638 0.264 ## imhecop 0.680 0.217 ## imtcjob 0.633 0.136 ## imbleco 0.563 -0.212 0.153 ## imbgeco 0.604 -0.168 ## imueclt 0.392 -0.236 -0.168 ## imwbcnt 0.526 -0.282 ## imwbcrm 0.397 -0.292 ## imrsprc 0.616 ## pplstrd 0.231 -0.378 ## vrtrlg 0.279 0.264 ## shrrfg 0.299 -0.271 ## rfgawrk 0.452 ## gvrfgap 0.123 0.774 ## rfgfrpc 0.193 -0.281 ## rfggvfn 0.467 ## rfgbfml 0.619 ## ## MR2 MR4 MR1 MR3 ## SS loadings 3.828 2.778 2.570 1.602 ## Proportion Var 0.153 0.111 0.103 0.064 ## Cumulative Var 0.153 0.264 0.367 0.431 ## ## Loadings: ## MR2 MR1 MR5 MR3 MR4 ## imsmetn 0.792 ## imdfetn 0.728 0.169 0.113 ## eimrcnt 0.910 -0.150 -0.237 ## eimpcnt 0.779 0.126 0.213 ## imrcntr 0.910 -0.128 -0.187 ## impcntr 0.731 0.131 0.236 ## qfimchr 0.109 -0.156 0.882 ## qfimwht 0.139 0.736 ## imwgdwn 0.740 ## imhecop 0.700 ## imtcjob 0.543 0.124 0.182 ## imbleco 0.682 0.135 ## imbgeco 0.799 ## imueclt 0.572 -0.202 ## imwbcnt 0.712 ## imwbcrm 0.545 -0.124 ## imrsprc 0.620 ## pplstrd 0.207 -0.396 ## vrtrlg -0.198 0.151 0.285 0.116 ## shrrfg 0.208 -0.263 0.139 ## rfgawrk 0.457 ## gvrfgap 0.783 ## rfgfrpc -0.338 0.156 ## rfggvfn 0.477 ## rfgbfml -0.125 0.538 ## ## MR2 MR1 MR5 MR3 MR4 ## SS loadings 3.970 2.790 2.215 1.693 1.166 ## Proportion Var 0.159 0.112 0.089 0.068 0.047 ## Cumulative Var 0.159 0.270 0.359 0.427 0.473 ## ## Loadings: ## MR2 MR1 MR6 MR3 MR5 MR4 ## imsmetn 0.705 0.166 ## imdfetn 0.833 ## eimrcnt 0.249 0.859 ## eimpcnt 0.946 ## imrcntr 0.456 0.517 ## impcntr 0.951 ## qfimchr 0.134 -0.122 0.875 ## qfimwht 0.151 0.725 ## imwgdwn 0.748 ## imhecop 0.678 ## imtcjob 0.566 0.123 0.175 ## imbleco 0.753 0.144 ## imbgeco 0.822 ## imueclt 0.580 -0.201 ## imwbcnt 0.751 ## imwbcrm 0.597 ## imrsprc 0.146 0.527 ## pplstrd 0.204 -0.392 ## vrtrlg -0.204 0.143 0.281 0.115 ## shrrfg 0.198 -0.275 0.141 ## rfgawrk 0.517 ## gvrfgap 0.784 ## rfgfrpc -0.294 0.144 ## rfggvfn 0.512 ## rfgbfml 0.596 ## ## MR2 MR1 MR6 MR3 MR5 MR4 ## SS loadings 3.304 3.013 1.994 1.649 1.065 1.133 ## Proportion Var 0.132 0.121 0.080 0.066 0.043 0.045 ## Cumulative Var 0.132 0.253 0.332 0.398 0.441 0.486 ## ## Loadings: ## MR2 MR1 MR6 MR3 MR5 MR7 MR4 ## imsmetn 0.700 0.162 ## imdfetn 0.821 ## eimrcnt 0.245 0.879 ## eimpcnt 0.935 ## imrcntr 0.452 0.523 ## impcntr 0.938 ## qfimchr 0.751 ## qfimwht 0.720 ## imwgdwn 0.700 ## imhecop 0.172 0.624 ## imtcjob 0.574 -0.120 0.174 ## imbleco 0.679 0.108 ## imbgeco 0.832 -0.145 ## imueclt 0.531 -0.191 ## imwbcnt 0.649 0.138 ## imwbcrm 0.464 0.131 0.290 ## imrsprc 0.146 0.440 -0.100 ## pplstrd -0.274 0.392 ## vrtrlg -0.121 0.190 -0.297 0.115 ## shrrfg -0.124 0.437 0.131 ## rfgawrk 0.538 ## gvrfgap 0.616 -0.237 ## rfgfrpc -0.131 0.437 0.135 ## rfggvfn 0.504 ## rfgbfml 0.526 ## ## MR2 MR1 MR6 MR3 MR5 MR7 MR4 ## SS loadings 3.224 2.467 1.456 1.305 1.105 0.901 0.984 ## Proportion Var 0.129 0.099 0.058 0.052 0.044 0.036 0.039 ## Cumulative Var 0.129 0.228 0.286 0.338 0.382 0.418 0.458 It is very possible that you selected a different numbers of factors than Kestilä (2006). We need to keep these exercises consistent, though. So, the remaining questions will all assume you have extract three factors from the Trust items and five factors from the Attitudes items, to parallel the Kestilä (2006) results. ## Select the three-factor solution for 'trust': efa_trust <- efa_trust[["3"]] ## Select the five-factor solution for 'attitudes': efa_att <- efa_att[["5"]] 4.4.10 Give the factor scores meaningful names, and add the scores to the ess dataset as new columns. Hint: If you’re not sure of what do to, check 4.3.11. Click to show code ## Rename the factor scores: colnames(efa_trust$scores) <- c("trust_inst", "satisfy", "trust_pol") colnames(efa_att$scores) <- c("effects", "allowance", "refugees", "ethnic", "europe") ## Add factor scores to the dataset as new columns: ess <- data.frame(ess, efa_trust$scores, efa_att$scores) Kestilä (2006) used the component scores to descriptively evaluate country-level differences in Attitudes toward Immigration and Political Trust. So, now it’s time to replicate those analyses. 4.4.11 Repeat the Kestilä (2006) between-country comparison using the factor scores you created in 4.4.10 and an appropriate statistical test. Click to show code Here, we’ll only demonstrate a possible approach to analyzing one of the Trust dimensions. We can use a linear model to test whether the countries differ in average levels of Trust in Institutions (as quantified by the relevant factor score). ## Estimate the model: out <- lm(trust_inst ~ country, data = ess) ## View the regression-style summary: summary(out) ## ## Call: ## lm(formula = trust_inst ~ country, data = ess) ## ## Residuals: ## Min 1Q Median 3Q Max ## -4.2295 -0.6226 0.1171 0.7194 3.3061 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -0.09028 0.02445 -3.692 0.000224 *** ## countryBelgium -0.28923 0.03642 -7.942 2.12e-15 *** ## countryGermany -0.05966 0.03211 -1.858 0.063205 . ## countryDenmark 0.75509 0.03882 19.452 < 2e-16 *** ## countryFinland 0.59235 0.03439 17.224 < 2e-16 *** ## countryItaly 0.10991 0.04071 2.700 0.006939 ** ## countryNetherlands -0.05357 0.03379 -1.585 0.112893 ## countryNorway 0.36922 0.03493 10.570 < 2e-16 *** ## countrySweden 0.28560 0.03613 7.904 2.89e-15 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 1.029 on 14769 degrees of freedom ## (4912 observations deleted due to missingness) ## Multiple R-squared: 0.082, Adjusted R-squared: 0.0815 ## F-statistic: 164.9 on 8 and 14769 DF, p-value: < 2.2e-16 ## View the results as an ANOVA table: anova(out) ## Post-hoc tests out %>% aov() %>% TukeyHSD() ## Tukey multiple comparisons of means ## 95% family-wise confidence level ## ## Fit: aov(formula = .) ## ## $country ## diff lwr upr p adj ## Belgium-Austria -0.289225482 -0.40219224 -0.17625873 0.0000000 ## Germany-Austria -0.059655996 -0.15926604 0.03995405 0.6429963 ## Denmark-Austria 0.755089552 0.63466911 0.87551000 0.0000000 ## Finland-Austria 0.592348290 0.48565882 0.69903776 0.0000000 ## Italy-Austria 0.109910185 -0.01636587 0.23618624 0.1476635 ## Netherlands-Austria -0.053567808 -0.15838407 0.05124846 0.8131104 ## Norway-Austria 0.369224250 0.26085692 0.47759158 0.0000000 ## Sweden-Austria 0.285601197 0.17350905 0.39769334 0.0000000 ## Germany-Belgium 0.229569486 0.12386351 0.33527546 0.0000000 ## Denmark-Belgium 1.044315033 0.91880537 1.16982470 0.0000000 ## Finland-Belgium 0.881573772 0.76917165 0.99397589 0.0000000 ## Italy-Belgium 0.399135667 0.26799745 0.53027389 0.0000000 ## Netherlands-Belgium 0.235657673 0.12503199 0.34628336 0.0000000 ## Norway-Belgium 0.658449732 0.54445381 0.77244566 0.0000000 ## Sweden-Belgium 0.574826679 0.45728417 0.69236918 0.0000000 ## Denmark-Germany 0.814745547 0.70110863 0.92838247 0.0000000 ## Finland-Germany 0.652004286 0.55303505 0.75097352 0.0000000 ## Italy-Germany 0.169566181 0.04974170 0.28939066 0.0003895 ## Netherlands-Germany 0.006088188 -0.09085878 0.10303516 0.9999999 ## Norway-Germany 0.428880246 0.32810453 0.52965596 0.0000000 ## Sweden-Germany 0.345257193 0.24048642 0.45002796 0.0000000 ## Finland-Denmark -0.162741262 -0.28263218 -0.04285034 0.0008579 ## Italy-Denmark -0.645179366 -0.78279052 -0.50756821 0.0000000 ## Netherlands-Denmark -0.808657360 -0.92688442 -0.69043030 0.0000000 ## Norway-Denmark -0.385865301 -0.50725174 -0.26447886 0.0000000 ## Sweden-Denmark -0.469488354 -0.59421139 -0.34476531 0.0000000 ## Italy-Finland -0.482438105 -0.60820928 -0.35666693 0.0000000 ## Netherlands-Finland -0.645916098 -0.75012357 -0.54170862 0.0000000 ## Norway-Finland -0.223124040 -0.33090264 -0.11534544 0.0000000 ## Sweden-Finland -0.306747093 -0.41827017 -0.19522402 0.0000000 ## Netherlands-Italy -0.163477993 -0.28766412 -0.03929186 0.0014719 ## Norway-Italy 0.259314065 0.13211649 0.38651164 0.0000000 ## Sweden-Italy 0.175691012 0.04530545 0.30607657 0.0009794 ## Norway-Netherlands 0.422792059 0.31686740 0.52871671 0.0000000 ## Sweden-Netherlands 0.339169005 0.22943659 0.44890142 0.0000000 ## Sweden-Norway -0.083623053 -0.19675232 0.02950622 0.3462227 Click for explanation According to the omnibus F-test, average levels of Trust in Institutions significantly differ between countries, but this test cannot tell us between which countries the differences lie. Similarly, the t statistics associated with each dummy code in the regression-style summary only tell us if that country differs significantly from the reference country (i.e., Austria), but we cannot see, for example, if there is a significant difference in average trust levels between Belgium and the Netherlands. One way to test for differences between the individual countries would be a post hoc test of all pairwise comparisons. Since we’ll be doing 45 tests, we need to apply a correction for repeated testing. Above, we use the TukeyHSD() function to conduct all pairwise comparisons while applying Tukey’s HSD correction. The TukeyHSD() function only accepts models estimated with the aov() function, so we first pass our fitted lm object through aov(). The second part of the Kestilä (2006) analysis was to evaluate how socio-demographic characteristics affected attitudes towards immigrants and trust in politics among the Finnish electorate. Before we can replicate this part of the analysis, we need to subset the data to only the Finnish cases. 4.4.12 Create a new data frame that contains only the Finnish cases from ess. Hint: You can use logical indexing based on the country variable. Click to show code ess_finland <- filter(ess, country == "Finland") We still have one more step before we can estimate any models. We must prepare our variables for analysis. Our dependent variables will be the factor scores generated above. So, we do not need to apply any further processing. We have not yet used any of the independent variables, though. So, we should inspect those variables to see if they require any processing. In our processed ess data, the relevant variables have the following names: sex yrbrn eduyrs polintr lrscale 4.4.13 Inspect the independent variables listed above. Click to show code library(tidySEM) select(ess_finland, sex, yrbrn, eduyrs, polintr, lrscale) %>% descriptives() Click for explanation It looks like we still need some recoding. 4.4.14 Apply any necessary recoding/transformations. 1. Age Click to show code ess_finland <- mutate(ess_finland, age = 2002 - yrbrn) Click for explanation The data contain the participants’ years of birth instead of their age, but Kestilä analyzed age. Fortunately, we know that the data were collected in 2002, so we can simply subtract each participant’s value of yrbrn from the 2002 to compute their age. 2. Political Interest Click to show code First, we’ll transform polintr. ## Recode the four factor levels into two factor levels: ess_finland <- mutate(ess_finland, polintr_bin = recode_factor(polintr, "Not at all interested" = "Low Interest", "Hardly interested" = "Low Interest", "Quite interested" = "High Interest", "Very interested" = "High Interest") ) ## Check the conversion: with(ess_finland, table(old = polintr, new = polintr_bin, useNA = "always")) ## new ## old Low Interest High Interest <NA> ## Very interested 0 144 0 ## Quite interested 0 785 0 ## Hardly interested 842 0 0 ## Not at all interested 228 0 0 ## <NA> 0 0 1 Click for explanation Kestilä (2006) dichotomized polintr by combining the lowest two and highest two categories. So, we don’t actually want to convert the polint variable into a numeric, Likert-type variable. We want polint to be a binary factor. The recode_factor() function from dplyr() will automatically convert our result into a factor. As with the ess_round1.rds data, we will be coming back to this Finnish subsample data in future practical exercises. So, we should save our work by writing the processed dataset to disk. 4.4.15 Use the saveRDS() function to save the processed Finnish subsample data. Click to see code ## Save the processed Finnish data: saveRDS(ess_finland, "ess_finland.rds") Now, we’re finally ready to replicate the regression analysis from Kestilä (2006). Creating a single aggregate score by summing the individual component scores is a pretty silly thing to do, though. So, we won’t reproduce that aspect of the analysis. 4.4.16 Run a series of multiple linear regression analyses with the factor scores you created in 4.4.10 as the dependent variables and the same predictors used by Kestilä (2006). Do your results agree with those reported by Kestilä (2006)? Click to show code ## Predicting 'Trust in Institutions': out_trust_inst <- lm(trust_inst ~ sex + age + eduyrs + polintr_bin + lrscale, data = ess_finland) summary(out_trust_inst) ## ## Call: ## lm(formula = trust_inst ~ sex + age + eduyrs + polintr_bin + ## lrscale, data = ess_finland) ## ## Residuals: ## Min 1Q Median 3Q Max ## -3.9499 -0.5102 0.1337 0.6638 2.5919 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -0.057518 0.124294 -0.463 0.643595 ## sexFemale 0.004091 0.045170 0.091 0.927849 ## age -0.003071 0.001380 -2.225 0.026219 * ## eduyrs 0.023223 0.006388 3.635 0.000286 *** ## polintr_binHigh Interest 0.166860 0.046448 3.592 0.000337 *** ## lrscale 0.058951 0.011232 5.249 1.72e-07 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 0.9321 on 1734 degrees of freedom ## (260 observations deleted due to missingness) ## Multiple R-squared: 0.04155, Adjusted R-squared: 0.03879 ## F-statistic: 15.03 on 5 and 1734 DF, p-value: 1.78e-14 ## Predicting 'Trust in Politicians': out_trust_pol <- lm(trust_pol ~ sex + age + eduyrs + polintr_bin + lrscale, data = ess_finland) summary(out_trust_pol) ## ## Call: ## lm(formula = trust_pol ~ sex + age + eduyrs + polintr_bin + lrscale, ## data = ess_finland) ## ## Residuals: ## Min 1Q Median 3Q Max ## -3.03673 -0.67306 0.05346 0.69666 2.38771 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -0.165989 0.126840 -1.309 0.19083 ## sexFemale 0.015572 0.046095 0.338 0.73554 ## age -0.009112 0.001409 -6.469 1.28e-10 *** ## eduyrs 0.018476 0.006519 2.834 0.00465 ** ## polintr_binHigh Interest 0.463763 0.047399 9.784 < 2e-16 *** ## lrscale 0.054932 0.011462 4.793 1.79e-06 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 0.9512 on 1734 degrees of freedom ## (260 observations deleted due to missingness) ## Multiple R-squared: 0.09806, Adjusted R-squared: 0.09546 ## F-statistic: 37.71 on 5 and 1734 DF, p-value: < 2.2e-16 ## Predicting 'Attitudes toward Refugees': out_refugees <- lm(refugees ~ sex + age + eduyrs + polintr_bin + lrscale, data = ess_finland) summary(out_refugees) ## ## Call: ## lm(formula = refugees ~ sex + age + eduyrs + polintr_bin + lrscale, ## data = ess_finland) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.9118 -0.6860 -0.0594 0.6904 4.1044 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -1.690e-01 1.438e-01 -1.175 0.240080 ## sexFemale -4.828e-01 5.181e-02 -9.318 < 2e-16 *** ## age 2.903e-05 1.604e-03 0.018 0.985561 ## eduyrs -2.537e-02 7.459e-03 -3.401 0.000688 *** ## polintr_binHigh Interest -2.131e-01 5.345e-02 -3.986 6.99e-05 *** ## lrscale 9.359e-02 1.296e-02 7.223 7.65e-13 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 1.06 on 1699 degrees of freedom ## (295 observations deleted due to missingness) ## Multiple R-squared: 0.09535, Adjusted R-squared: 0.09269 ## F-statistic: 35.81 on 5 and 1699 DF, p-value: < 2.2e-16 That does it for our replication of the Kesilä (2006) analyses, but we still have one more topic to consider in this practical. One of the most common applications of EFA is scale development. Given a pool of items without a known factor structure, we try to estimate the underlying latent factors that define the (sub)scales represented by our items. In such applications, we use the factor loading matrix for our optimal solution to make “bright-line” assignments of items to putative factors according to the simple structure represented by the estimated factor loading matrix. In other words, we disregard small factor loadings and assign observed items to only the single latent factor upon which they load most strongly. We then hypothesize that those items are true indicators of that latent factor. We can use confirmatory factor analysis (which you will learn about next week) to test rigorously this hypothesis, but we can already get started by estimating the internal consistency (a type of reliability) of the hypothesized subscales. 4.4.17 Estimate the internal consistency of the three Trust subscales and five Attitudes subscales implied by your EFA solutions from above. Use Cronbach’s Alpha to quantify internal consistency. Use the alpha() function from the psych package to conduct the analysis. Run your analysis on the full ess dataset, not the Finnish subset. Are the subscales implied by your EFA reliable, in the sense of good internal consistency? Note that \\(\\alpha > 0.7\\) is generally considered acceptable, and \\(\\alpha > 0.8\\) is usually considered good. Click to show code ## Run the reliability analysis on the subscale data: ( out <- select(ess, starts_with("stf")) %>% psych::alpha() ) ## ## Reliability analysis ## Call: psych::alpha(x = .) ## ## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r ## 0.79 0.79 0.77 0.44 3.9 0.0023 5.4 1.7 0.41 ## ## 95% confidence boundaries ## lower alpha upper ## Feldt 0.79 0.79 0.8 ## Duhachek 0.79 0.79 0.8 ## ## Reliability if an item is dropped: ## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r ## stfhlth 0.78 0.78 0.73 0.47 3.5 0.0026 0.0064 0.46 ## stfedu 0.76 0.76 0.72 0.45 3.2 0.0028 0.0109 0.44 ## stfeco 0.74 0.74 0.70 0.41 2.8 0.0031 0.0069 0.39 ## stfgov 0.74 0.74 0.69 0.42 2.9 0.0030 0.0035 0.41 ## stfdem 0.75 0.75 0.71 0.43 3.0 0.0029 0.0074 0.40 ## ## Item statistics ## n raw.r std.r r.cor r.drop mean sd ## stfhlth 19481 0.69 0.69 0.56 0.50 5.8 2.3 ## stfedu 18844 0.73 0.73 0.62 0.55 5.9 2.3 ## stfeco 19211 0.78 0.78 0.70 0.63 5.0 2.4 ## stfgov 19106 0.77 0.76 0.69 0.61 4.5 2.3 ## stfdem 19106 0.75 0.75 0.67 0.59 5.7 2.3 Click for explanation Here, we estimate the reliability of the Satisfaction subscale from the Trust analysis. According to our EFA, the Satisfaction subscale should be indicated by the following five variables: stfeco stfgov stfdem stfedu stfhlth We select these variables using the tidy-select function starts_with() to extract all variables beginning with the three characters “stf”. To estimate the internal consistency of this subscale, we simply provide a data frame containing only the subscale data to the alpha() function. The raw_alpha value is the estimate of Cronbach’s Alpha. In this case \\(\\alpha = 0.794\\), so the subscale is pretty reliable. The table labeled “Reliability if an item is dropped” shows what Cronbach’s Alpha would be if each item were excluded from the scale. If this value is notably higher than the raw_alpha value, it could indicate a bad item. Note that reliability is only one aspect of scale quality, though. So, you shouldn’t throw out items just because they perform poorly in reliability analysis. End of In-Class Exercises "],["cfa.html", "5 CFA", " 5 CFA This week, we will introduce confirmatory factor analysis (CFA) and discuss how it differs from EFA. Furthermore, we will revisit the idea of model fit and introduce into the R-package lavaan. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-4.html", "5.1 Lecture", " 5.1 Lecture Often, we work with scales that have a validated or hypothesized factor structure. In the former case, the scale structure has been validated through previous psychometric studies. In the latter case, we may have conducted an EFA to estimate the factor structure on prior data, or theory/intuition may suggest a plausible structure. Regardless of how we come to expect a given factor structure, such situations represent confirmatory modeling problems, because we are attempting to empirically confirm an a priori expectation. Hence, exploratory methods like EFA are not appropriate, and we should employ confirmatory modeling techniques. This week we consider one such technique: confirmatory factor analysis (CFA). As the name suggests, CFA is related to the EFA methods we discussed last week in that both methods are flavors of factor analysis. However, the two methods address fundamentally different research questions. Rather than attempting to estimate an unknown factor structure (as in EFA), we now want to compare a hypothesized measurement model (i.e., factor structure) to observed data in order to evaluate the model’s plausibility. 5.1.1 Recording Note: When Caspar discusses the complexity of the second-order CFA model, it’s easy to misunderstand his statements. We need to be careful not to over-generalize. In general, a second-order CFA is not more complex than a first-order CFA. Actually, in most practical applications, the opposite is true. A second-order CFA is more complex than a first-order CFA, when the factors in the first-order CFA are uncorrelated. This is the situation Caspar references in the recording when claiming that the second-order model is more complex. We hardly ever want to fit such first-order CFA, though. The default CFA fully saturates the latent covariance structure. If the factors in the first-order CFA are fully correlated (according to standard practice), and we include a single second-order factor, the following statements hold. If the first-order CFA has more than three factors, the first-order model is more complex than the second-order model. If the first-order model has three or fewer factors, the first- and second-order models are equivalent (due to scaling constraints we need to impose to identify the second-order model). The second-order model cannot be more complex than the first-order model (assuming both models are correctly identified and no extra constraints are imposed). The above statements may not hold in more complex situations (e.g., more than one second-order factor, partially saturated first-order correlation structure, etc.). You can always identify the more complex model by calculating the degrees of freedom for both models. The model with fewer degrees of freedom is more complex. 5.1.2 Slides You can download the lecture slides here "],["reading-4.html", "5.2 Reading", " 5.2 Reading Reference Byrne, B. (2005). Factor analytic models: Viewing the structure of an assessment instrument from three perspectives, Journal of Personality Assessment, 85(1), 17–32. Questions What are the main differences between exploratory factor analysis (EFA) and confirmatory factor analysis (CFA)? In which circumstances should a researcher use EFA, and in which should they use CFA? What are the five main limitations of EFA that CFA overcomes? In which circumstances can a second order CFA model be useful? Consider the following four techniques: PCA, EFA, CFA, second order CFA. For each of the following three research situations, which of the above techniques would you use and why? A researcher has developed a new questionnaire that should measure personality and wants to know how many factors underlie the items in their new measure. A researcher is modeling data collected with a seven-item scale that has been used since the 1960s to measure authoritarianism. A researcher has recorded highest completed level of education, years of education, and highest level of education attempted for all respondents in a survey. The researcher wants to include some operationalization of the concept of ‘education’ in their model but is unsure of which observed variable to use. "],["at-home-exercises-4.html", "5.3 At-Home Exercises", " 5.3 At-Home Exercises This week, we will wrap up our re-analysis of the Kestilä (2006) results. During this practical, you will conduct a CFA of the Trust in Politics items and compare the results to those obtained from your previous EFA- and PCA-based replications of Kestilä (2006). 5.3.1 Load the ESS data. The relevant data are contained in the ess_round1.rds file. This file is in R Data Set (RDS) format. The dataset is already stored as a data frame with the processing and cleaning that you should have done for previous practicals completed. Click to show code ess <- readRDS("ess_round1.rds") Although you may have settled on any number of EFA solutions during the Week 4 In-Class Exercises, we are going to base the following CFA on a three-factor model of Trust in Politics similar to the original PCA results from Kestilä (2006). Note: Unless otherwise specified, all following questions refer to the Trust in Politics items. We will not consider the Attitudes toward Immigration items in these exercises. 5.3.2 Define the lavaan model syntax for the CFA implied by the three-factor EFA solution you found in the Week 4 In-Class Exercises. Covary the three latent factors. Do not specify any mean structure. Save this model syntax as an object in your environment. Click to show code mod_3f <- ' institutions =~ trstlgl + trstplc + trstun + trstep + trstprl satisfaction =~ stfhlth + stfedu + stfeco + stfgov + stfdem politicians =~ pltinvt + pltcare + trstplt ' Click for explanation We don’t have to specify the latent covariances in the model syntax, we can tell lavaan to estimate all latent covariances when we fit the model. 5.3.3 Estimate the CFA model you defined above, and summarize the results. Use the lavaan::cfa() function to estimate the model. Use the default settings for the cfa() function. Request the model fit statistics with the summary by supplying the fit.measures = TRUE argument to summary(). Request the standardized parameter estimates with the summary by supplying the standardized = TRUE argument to summary(). Check the results, and answer the following questions: Does the model fit the data well? How are the latent variances and covariances specified when using the default settings? How is the model identified when using the default settings? Click the code ## Load the lavaan package: library(lavaan) ## Estimate the CFA model: fit_3f <- cfa(mod_3f, data = ess) ## Summarize the fitted model: summary(fit_3f, fit.measures = TRUE, standardized = TRUE) ## lavaan 0.6.16 ended normally after 46 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 29 ## ## Used Total ## Number of observations 14778 19690 ## ## Model Test User Model: ## ## Test statistic 10652.207 ## Degrees of freedom 62 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 81699.096 ## Degrees of freedom 78 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.870 ## Tucker-Lewis Index (TLI) 0.837 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -371404.658 ## Loglikelihood unrestricted model (H1) -366078.555 ## ## Akaike (AIC) 742867.317 ## Bayesian (BIC) 743087.743 ## Sample-size adjusted Bayesian (SABIC) 742995.583 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.108 ## 90 Percent confidence interval - lower 0.106 ## 90 Percent confidence interval - upper 0.109 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 1.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.059 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions =~ ## trstlgl 1.000 1.613 0.677 ## trstplc 0.770 0.012 61.866 0.000 1.241 0.567 ## trstun 0.929 0.013 69.227 0.000 1.498 0.642 ## trstep 0.908 0.013 70.929 0.000 1.464 0.660 ## trstprl 1.139 0.014 84.084 0.000 1.837 0.809 ## satisfaction =~ ## stfhlth 1.000 1.173 0.521 ## stfedu 1.106 0.022 50.840 0.000 1.297 0.577 ## stfeco 1.415 0.025 57.214 0.000 1.659 0.713 ## stfgov 1.480 0.025 58.764 0.000 1.736 0.756 ## stfdem 1.384 0.024 57.904 0.000 1.623 0.731 ## politicians =~ ## pltinvt 1.000 0.646 0.613 ## pltcare 1.021 0.016 62.862 0.000 0.660 0.628 ## trstplt 3.012 0.039 76.838 0.000 1.946 0.891 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions ~~ ## satisfaction 1.391 0.032 43.206 0.000 0.736 0.736 ## politicians 0.909 0.018 49.934 0.000 0.872 0.872 ## satisfaction ~~ ## politicians 0.539 0.013 41.053 0.000 0.711 0.711 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trstlgl 3.068 0.041 75.262 0.000 3.068 0.541 ## .trstplc 3.248 0.041 80.037 0.000 3.248 0.678 ## .trstun 3.197 0.041 77.141 0.000 3.197 0.588 ## .trstep 2.776 0.036 76.243 0.000 2.776 0.564 ## .trstprl 1.776 0.029 61.361 0.000 1.776 0.345 ## .stfhlth 3.695 0.046 79.989 0.000 3.695 0.729 ## .stfedu 3.368 0.043 77.916 0.000 3.368 0.667 ## .stfeco 2.656 0.038 69.070 0.000 2.656 0.491 ## .stfgov 2.264 0.035 64.201 0.000 2.264 0.429 ## .stfdem 2.289 0.034 67.172 0.000 2.289 0.465 ## .pltinvt 0.694 0.009 78.255 0.000 0.694 0.624 ## .pltcare 0.668 0.009 77.562 0.000 0.668 0.605 ## .trstplt 0.978 0.028 34.461 0.000 0.978 0.205 ## institutions 2.601 0.059 44.198 0.000 1.000 1.000 ## satisfaction 1.375 0.044 31.407 0.000 1.000 1.000 ## politicians 0.417 0.011 38.843 0.000 1.000 1.000 Click for explanation No, the model does not seem to fit the data well. The SRMR looks good, but one good looking fit statistic is not enough. The RMSEA, TLI, and CFI are all in the “unacceptable” range. The \\(\\chi^2\\) is highly significant, but we don’t care. The cfa() function is just a wrapper for the lavaan() function with several options set at the defaults you would want for a standard CFA. By default: All latent variances and covariances are freely estimated (due to the argument auto.cov.lv.x = TRUE) The model is identified by fixing the first factor loading of each factor to 1 (due to the argument auto.fix.first = TRUE) To see a full list of the (many) options you can specify to tweak the behavior of lavaan estimation functions run ?lavOptions. Now, we will consider a couple of alternative factor structures for the Trust in Politics CFA. First, we will go extremely simple by estimating a one-factor model wherein all Trust items are explained by a single latent variable. 5.3.4 Define the lavaan model syntax for a one-factor model of the Trust items. Save this syntax as an object in your environment. Click to show code mod_1f <- ' political_trust =~ trstlgl + trstplc + trstun + trstep + trstprl + stfhlth + stfedu + stfeco + stfgov + stfdem + pltinvt + pltcare + trstplt ' 5.3.5 Estimate the one-factor model, and summarize the results. Does this model appear to fit better or worse than the three-factor model? Note: You can use the lavaan::fitMeasures() function to extract only the model fit information from a fitted lavaan object. Click to show code ## Estimate the one factor model: fit_1f <- cfa(mod_1f, data = ess) ## Summarize the results: summary(fit_1f, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 33 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 26 ## ## Used Total ## Number of observations 14778 19690 ## ## Model Test User Model: ## ## Test statistic 17667.304 ## Degrees of freedom 65 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 81699.096 ## Degrees of freedom 78 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.784 ## Tucker-Lewis Index (TLI) 0.741 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -374912.206 ## Loglikelihood unrestricted model (H1) -366078.555 ## ## Akaike (AIC) 749876.413 ## Bayesian (BIC) 750074.036 ## Sample-size adjusted Bayesian (SABIC) 749991.410 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.135 ## 90 Percent confidence interval - lower 0.134 ## 90 Percent confidence interval - upper 0.137 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 1.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.080 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## political_trust =~ ## trstlgl 1.000 ## trstplc 0.774 0.013 57.949 0.000 ## trstun 0.930 0.014 64.200 0.000 ## trstep 0.909 0.014 65.679 0.000 ## trstprl 1.182 0.015 79.401 0.000 ## stfhlth 0.615 0.013 45.947 0.000 ## stfedu 0.695 0.014 51.424 0.000 ## stfeco 0.895 0.014 62.316 0.000 ## stfgov 0.985 0.014 68.200 0.000 ## stfdem 0.998 0.014 70.899 0.000 ## pltinvt 0.382 0.006 59.215 0.000 ## pltcare 0.396 0.006 61.195 0.000 ## trstplt 1.183 0.014 81.716 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .trstlgl 3.370 0.042 79.787 0.000 ## .trstplc 3.410 0.041 82.311 0.000 ## .trstun 3.451 0.043 80.749 0.000 ## .trstep 3.019 0.038 80.272 0.000 ## .trstprl 1.938 0.027 70.878 0.000 ## .stfhlth 4.201 0.050 84.093 0.000 ## .stfedu 3.941 0.047 83.419 0.000 ## .stfeco 3.565 0.044 81.289 0.000 ## .stfgov 3.044 0.038 79.326 0.000 ## .stfdem 2.631 0.034 78.072 0.000 ## .pltinvt 0.775 0.009 82.043 0.000 ## .pltcare 0.743 0.009 81.579 0.000 ## .trstplt 1.548 0.023 67.052 0.000 ## political_trst 2.299 0.055 41.569 0.000 ## Compare fit statistics: fitMeasures(fit_3f) ## npar fmin chisq ## 29.000 0.360 10652.207 ## df pvalue baseline.chisq ## 62.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.870 ## tli nnfi rfi ## 0.837 0.837 0.836 ## nfi pnfi ifi ## 0.870 0.691 0.870 ## rni logl unrestricted.logl ## 0.870 -371404.658 -366078.555 ## aic bic ntotal ## 742867.317 743087.743 14778.000 ## bic2 rmsea rmsea.ci.lower ## 742995.583 0.108 0.106 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.109 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.255 0.255 0.059 ## srmr_bentler srmr_bentler_nomean crmr ## 0.059 0.059 0.064 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.064 0.059 0.059 ## cn_05 cn_01 gfi ## 113.901 126.971 0.897 ## agfi pgfi mfi ## 0.849 0.611 0.699 ## ecvi ## 0.725 fitMeasures(fit_1f) ## npar fmin chisq ## 26.000 0.598 17667.304 ## df pvalue baseline.chisq ## 65.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.784 ## tli nnfi rfi ## 0.741 0.741 0.741 ## nfi pnfi ifi ## 0.784 0.653 0.784 ## rni logl unrestricted.logl ## 0.784 -374912.206 -366078.555 ## aic bic ntotal ## 749876.413 750074.036 14778.000 ## bic2 rmsea rmsea.ci.lower ## 749991.410 0.135 0.134 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.137 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.364 0.364 0.080 ## srmr_bentler srmr_bentler_nomean crmr ## 0.080 0.080 0.087 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.087 0.080 0.080 ## cn_05 cn_01 gfi ## 71.949 79.980 0.825 ## agfi pgfi mfi ## 0.756 0.590 0.551 ## ecvi ## 1.199 Click for explanation The one-factor model definitely seems to fit worse than the three-factor model. A second order CFA model is another way of representing the latent structure underlying a set of items. As you read in Byrne (2005), however, the second order CFA is only appropriate in certain circumstances. 5.3.6 Given the CFA results above, would a second order CFA be appropriate for the Trust data? Why or why not? Click for explanation Yes, a second order CFA model is a theoretically appropriate representation of the Trust items. The first order latent variables in the three-factor model are all significantly correlated. The first order latent variables in the three-factor model seem to tap different aspects of some single underlying construct. 5.3.7 Define the lavaan model syntax for a second-order CFA model of the Trust items. Use the three factors defined in 5.3.2 as the first order factors. Click to show code mod_2nd <- ' institutions =~ trstlgl + trstplc + trstun + trstep + trstprl satisfaction =~ stfhlth + stfedu + stfeco + stfgov + stfdem politicians =~ pltinvt + pltcare + trstplt trust =~ politicians + satisfaction + institutions ' Click for explanation To define the second order factor, we use the same syntactic conventions that we employ to define a first order factor. The only differences is that the “indicators” of the second order factor (i.e., the variables listed on the RHS of the =~ operator) are previously defined first order latent variables. 5.3.8 Estimate the second order CFA model, and summarize the results. Does this model fit better or worse than the three-factor model? Is this model more or less complex than the three-factor model? What information can you use to quantify this difference in complexity? Click to show code fit_2nd <- cfa(mod_2nd, data = ess) summary(fit_2nd, fit.measures = TRUE, standardized = TRUE) ## lavaan 0.6.16 ended normally after 44 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 29 ## ## Used Total ## Number of observations 14778 19690 ## ## Model Test User Model: ## ## Test statistic 10652.207 ## Degrees of freedom 62 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 81699.096 ## Degrees of freedom 78 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.870 ## Tucker-Lewis Index (TLI) 0.837 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -371404.658 ## Loglikelihood unrestricted model (H1) -366078.555 ## ## Akaike (AIC) 742867.317 ## Bayesian (BIC) 743087.743 ## Sample-size adjusted Bayesian (SABIC) 742995.583 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.108 ## 90 Percent confidence interval - lower 0.106 ## 90 Percent confidence interval - upper 0.109 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 1.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.059 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions =~ ## trstlgl 1.000 1.613 0.677 ## trstplc 0.770 0.012 61.866 0.000 1.241 0.567 ## trstun 0.929 0.013 69.227 0.000 1.498 0.642 ## trstep 0.908 0.013 70.929 0.000 1.464 0.660 ## trstprl 1.139 0.014 84.084 0.000 1.837 0.809 ## satisfaction =~ ## stfhlth 1.000 1.173 0.521 ## stfedu 1.106 0.022 50.840 0.000 1.297 0.577 ## stfeco 1.415 0.025 57.214 0.000 1.659 0.713 ## stfgov 1.480 0.025 58.764 0.000 1.736 0.756 ## stfdem 1.384 0.024 57.904 0.000 1.623 0.731 ## politicians =~ ## pltinvt 1.000 0.646 0.613 ## pltcare 1.021 0.016 62.862 0.000 0.660 0.628 ## trstplt 3.012 0.039 76.838 0.000 1.946 0.891 ## trust =~ ## politicians 1.000 0.918 0.918 ## satisfaction 1.531 0.033 46.494 0.000 0.774 0.774 ## institutions 2.583 0.045 56.796 0.000 0.950 0.950 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trstlgl 3.068 0.041 75.262 0.000 3.068 0.541 ## .trstplc 3.248 0.041 80.037 0.000 3.248 0.678 ## .trstun 3.197 0.041 77.141 0.000 3.197 0.588 ## .trstep 2.776 0.036 76.243 0.000 2.776 0.564 ## .trstprl 1.776 0.029 61.361 0.000 1.776 0.345 ## .stfhlth 3.695 0.046 79.989 0.000 3.695 0.729 ## .stfedu 3.368 0.043 77.916 0.000 3.368 0.667 ## .stfeco 2.656 0.038 69.070 0.000 2.656 0.491 ## .stfgov 2.264 0.035 64.201 0.000 2.264 0.429 ## .stfdem 2.289 0.034 67.172 0.000 2.289 0.465 ## .pltinvt 0.694 0.009 78.255 0.000 0.694 0.624 ## .pltcare 0.668 0.009 77.562 0.000 0.668 0.605 ## .trstplt 0.978 0.028 34.461 0.000 0.978 0.205 ## .institutions 0.255 0.022 11.691 0.000 0.098 0.098 ## .satisfaction 0.551 0.020 27.846 0.000 0.400 0.400 ## .politicians 0.065 0.004 17.091 0.000 0.157 0.157 ## trust 0.352 0.010 35.005 0.000 1.000 1.000 ## Compare fit between the first and second order models: fitMeasures(fit_3f) ## npar fmin chisq ## 29.000 0.360 10652.207 ## df pvalue baseline.chisq ## 62.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.870 ## tli nnfi rfi ## 0.837 0.837 0.836 ## nfi pnfi ifi ## 0.870 0.691 0.870 ## rni logl unrestricted.logl ## 0.870 -371404.658 -366078.555 ## aic bic ntotal ## 742867.317 743087.743 14778.000 ## bic2 rmsea rmsea.ci.lower ## 742995.583 0.108 0.106 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.109 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.255 0.255 0.059 ## srmr_bentler srmr_bentler_nomean crmr ## 0.059 0.059 0.064 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.064 0.059 0.059 ## cn_05 cn_01 gfi ## 113.901 126.971 0.897 ## agfi pgfi mfi ## 0.849 0.611 0.699 ## ecvi ## 0.725 fitMeasures(fit_2nd) ## npar fmin chisq ## 29.000 0.360 10652.207 ## df pvalue baseline.chisq ## 62.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.870 ## tli nnfi rfi ## 0.837 0.837 0.836 ## nfi pnfi ifi ## 0.870 0.691 0.870 ## rni logl unrestricted.logl ## 0.870 -371404.658 -366078.555 ## aic bic ntotal ## 742867.317 743087.743 14778.000 ## bic2 rmsea rmsea.ci.lower ## 742995.583 0.108 0.106 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.109 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.255 0.255 0.059 ## srmr_bentler srmr_bentler_nomean crmr ## 0.059 0.059 0.064 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.064 0.059 0.059 ## cn_05 cn_01 gfi ## 113.901 126.971 0.897 ## agfi pgfi mfi ## 0.849 0.611 0.699 ## ecvi ## 0.725 Click for explanation We don’t have to do anything special here. We can estimate and summarize the second order CFA exactly as we did the first order CFA. You should quickly notice something strange about the model fit statistics compared above. If you don’t see it, consider the following: fitMeasures(fit_3f) - fitMeasures(fit_2nd) ## npar fmin chisq ## 0 0 0 ## df pvalue baseline.chisq ## 0 0 0 ## baseline.df baseline.pvalue cfi ## 0 0 0 ## tli nnfi rfi ## 0 0 0 ## nfi pnfi ifi ## 0 0 0 ## rni logl unrestricted.logl ## 0 0 0 ## aic bic ntotal ## 0 0 0 ## bic2 rmsea rmsea.ci.lower ## 0 0 0 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0 0 0 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0 0 0 ## rmr rmr_nomean srmr ## 0 0 0 ## srmr_bentler srmr_bentler_nomean crmr ## 0 0 0 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0 0 0 ## cn_05 cn_01 gfi ## 0 0 0 ## agfi pgfi mfi ## 0 0 0 ## ecvi ## 0 The two models produce identical fit statistics! We also see that the degrees of freedom are identical between the two models. Hence, the two models have equal complexity. This result taps into a critical idea in statistical modeling, namely, model equivalency. It turns out the two models we’re comparing here are equivalent in the sense that they are statistically indistinguishable representations of the data. Since this is a very important idea, I want to spend some time discussing it in person. So, spend some time between now and the Week 6 lecture session thinking about the implications of this model equivalence. Specifically, consider the following questions: What do we mean when we say that these two models are equivalent? How is it possible for these two models to be equivalent when one contains an additional latent variable? Why are the degrees of freedom equal for these two models? Why are the fit statistics equal for these two models? We’ll take some time to discuss these ideas in the Week 6 lecture session. End of At-Home Exercises "],["in-class-exercises-4.html", "5.4 In-Class Exercises", " 5.4 In-Class Exercises This week, we will wrap up our re-analysis of the Kestilä (2006) results. During this practical, you will conduct a CFA of the Attitudes toward Immigration items and compare the results to those obtained from your previous EFA- and PCA-based replications of Kestilä (2006). 5.4.1 Load the ESS data. The relevant data are contained in the ess_round1.rds file. Click to show code ess <- readRDS("ess_round1.rds") We are going to conduct a CFA to evaluate the measurement model implied by the five-factor representation of the Attitudes toward Immigration items that you should have found via the EFA you conducted in the Week 4 In-Class Exercises. Caveat: Technically, the following CFA result have no confirmatory value because we’ll be estimating our CFA models from the same data that we used for our EFA. Practicing the techniques will still be useful, though. 5.4.2 Define the lavaan model syntax for the CFA implied by the five-factor solution from 4.4.9. Enforce a simple structure; do not allow any cross-loadings. Covary the five latent factors. Do not specify any mean structure. Save this model syntax as an object in your environment. Hints: You can algorithmically enforce a simple structure by assigning each item to the factor upon which it loads most strongly. You can download the fitted psych::efa() object for the five-factor solution here. The pattern matrix for the five-factor EFA solution in our Week 4 exercises is equivalent to the solution presented in Table 3 of Kestilä (2006). Click to show code mod_5f <- ' ## Immigration Policy: ip =~ imrcntr + eimrcnt + eimpcnt + imsmetn + impcntr + imdfetn ## Social Threat: st =~ imbgeco + imbleco + imwbcnt + imwbcrm + imtcjob + imueclt ## Refugee Policy: rp =~ gvrfgap + imrsprc + rfgbfml + rfggvfn + rfgawrk + rfgfrpc + shrrfg ## Cultural Threat: ct =~ qfimchr + qfimwht + pplstrd + vrtrlg ## Economic Threat: et =~ imwgdwn + imhecop ' Note: We don’t have to specify the latent covariances in the model syntax, we can tell lavaan to estimate all latent covariances when we fit the model. 5.4.3 Estimate the CFA model you defined above, and summarize the results. Use the lavaan::cfa() function to estimate the model. Use the default settings for the cfa() function. Request the model fit statistics with the summary by supplying the fit.measures = TRUE argument to summary(). Request the standardized parameter estimates with the summary by supplying the standardized = TRUE argument to summary(). Check the results, and answer the following questions: Does the model fit the data well? How are the latent variances and covariances specified when using the default settings? How is the model identified when using the default settings? Click to show code ## Load the lavaan package: library(lavaan) ## Estimate the CFA model: fit_5f <- cfa(mod_5f, data = ess) ## Summarize the fitted model: summary(fit_5f, fit.measures = TRUE, standardized = TRUE) ## lavaan 0.6.16 ended normally after 72 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 60 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 18631.556 ## Degrees of freedom 265 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 159619.058 ## Degrees of freedom 300 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.885 ## Tucker-Lewis Index (TLI) 0.869 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -520035.133 ## Loglikelihood unrestricted model (H1) -510719.354 ## ## Akaike (AIC) 1040190.265 ## Bayesian (BIC) 1040644.106 ## Sample-size adjusted Bayesian (SABIC) 1040453.432 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## 90 Percent confidence interval - lower 0.069 ## 90 Percent confidence interval - upper 0.071 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## ip =~ ## imrcntr 1.000 0.617 0.748 ## eimrcnt 0.942 0.011 84.943 0.000 0.582 0.696 ## eimpcnt 1.127 0.010 113.413 0.000 0.695 0.898 ## imsmetn 0.982 0.010 98.753 0.000 0.606 0.796 ## impcntr 1.150 0.010 113.623 0.000 0.710 0.900 ## imdfetn 1.132 0.010 111.802 0.000 0.698 0.887 ## st =~ ## imbgeco 1.000 1.608 0.728 ## imbleco 0.826 0.012 69.222 0.000 1.327 0.619 ## imwbcnt 1.046 0.012 88.056 0.000 1.682 0.792 ## imwbcrm 0.713 0.011 63.102 0.000 1.146 0.564 ## imtcjob 0.751 0.011 66.787 0.000 1.207 0.597 ## imueclt 1.008 0.013 78.043 0.000 1.621 0.698 ## rp =~ ## gvrfgap 1.000 0.659 0.610 ## imrsprc 0.855 0.016 51.881 0.000 0.563 0.535 ## rfgbfml 1.047 0.019 56.174 0.000 0.690 0.593 ## rfggvfn 0.849 0.016 51.714 0.000 0.559 0.533 ## rfgawrk 0.653 0.016 41.044 0.000 0.430 0.405 ## rfgfrpc -0.810 0.016 -51.095 0.000 -0.534 -0.525 ## shrrfg -0.999 0.017 -58.381 0.000 -0.658 -0.625 ## ct =~ ## qfimchr 1.000 1.836 0.629 ## qfimwht 0.941 0.017 54.250 0.000 1.728 0.659 ## pplstrd -0.366 0.007 -51.585 0.000 -0.673 -0.600 ## vrtrlg 0.252 0.006 41.294 0.000 0.462 0.443 ## et =~ ## imwgdwn 1.000 0.723 0.667 ## imhecop 1.151 0.023 49.736 0.000 0.832 0.771 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## ip ~~ ## st -0.605 0.012 -48.693 0.000 -0.610 -0.610 ## rp 0.264 0.006 45.566 0.000 0.648 0.648 ## ct 0.634 0.015 41.007 0.000 0.560 0.560 ## et -0.206 0.006 -35.411 0.000 -0.462 -0.462 ## st ~~ ## rp -0.838 0.017 -48.329 0.000 -0.792 -0.792 ## ct -1.622 0.041 -39.091 0.000 -0.550 -0.550 ## et 0.675 0.017 39.083 0.000 0.580 0.580 ## rp ~~ ## ct 0.626 0.018 34.950 0.000 0.518 0.518 ## et -0.233 0.007 -33.007 0.000 -0.490 -0.490 ## ct ~~ ## et -0.592 0.020 -30.127 0.000 -0.446 -0.446 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .imrcntr 0.299 0.004 77.941 0.000 0.299 0.440 ## .eimrcnt 0.359 0.005 79.638 0.000 0.359 0.515 ## .eimpcnt 0.116 0.002 62.821 0.000 0.116 0.193 ## .imsmetn 0.212 0.003 75.580 0.000 0.212 0.366 ## .impcntr 0.119 0.002 62.454 0.000 0.119 0.191 ## .imdfetn 0.132 0.002 65.344 0.000 0.132 0.213 ## .imbgeco 2.288 0.033 70.261 0.000 2.288 0.470 ## .imbleco 2.837 0.037 76.688 0.000 2.837 0.617 ## .imwbcnt 1.677 0.027 63.198 0.000 1.677 0.372 ## .imwbcrm 2.810 0.036 78.612 0.000 2.810 0.682 ## .imtcjob 2.630 0.034 77.524 0.000 2.630 0.643 ## .imueclt 2.761 0.038 72.515 0.000 2.761 0.512 ## .gvrfgap 0.733 0.010 73.584 0.000 0.733 0.628 ## .imrsprc 0.791 0.010 77.119 0.000 0.791 0.714 ## .rfgbfml 0.877 0.012 74.508 0.000 0.877 0.648 ## .rfggvfn 0.788 0.010 77.203 0.000 0.788 0.716 ## .rfgawrk 0.945 0.012 80.870 0.000 0.945 0.836 ## .rfgfrpc 0.749 0.010 77.501 0.000 0.749 0.724 ## .shrrfg 0.676 0.009 72.682 0.000 0.676 0.609 ## .qfimchr 5.142 0.080 64.113 0.000 5.142 0.604 ## .qfimwht 3.891 0.064 60.623 0.000 3.891 0.566 ## .pplstrd 0.804 0.012 67.054 0.000 0.804 0.640 ## .vrtrlg 0.872 0.011 76.990 0.000 0.872 0.804 ## .imwgdwn 0.652 0.012 53.300 0.000 0.652 0.555 ## .imhecop 0.472 0.014 34.353 0.000 0.472 0.405 ## ip 0.381 0.007 51.578 0.000 1.000 1.000 ## st 2.584 0.054 47.795 0.000 1.000 1.000 ## rp 0.434 0.012 36.748 0.000 1.000 1.000 ## ct 3.371 0.096 35.174 0.000 1.000 1.000 ## et 0.523 0.015 34.944 0.000 1.000 1.000 Click for explanation No, the model does not seem to fit the data well. The SRMR looks good, but one good looking fit statistic is not enough. The TLI and CFI are in the “unacceptable” range. RMSEA is in the “questionable” range. The \\(\\chi^2\\) is highly significant, but we don’t care. The cfa() function is just a wrapper for the lavaan() function with several options set at the defaults you would want for a standard CFA. By default: All latent variances and covariances are freely estimated (due to the argument auto.cov.lv.x = TRUE) The model is identified by fixing the first factor loading of each factor to 1 (due to the argument auto.fix.first = TRUE) To see a full list of the (many) options you can specify to tweak the behavior of lavaan estimation functions run ?lavOptions. Now, we will consider a couple of alternative factor structures for the Attitudes toward Immigration CFA. First, we will go extremely simple by estimating a one-factor model wherein all Attitude items are explained by a single latent variable. 5.4.4 Define the lavaan model syntax for a one-factor model of the Immigration items. Save this syntax as an object in your environment. Click to show code mod_1f <- ' ati =~ imrcntr + eimrcnt + eimpcnt + imsmetn + impcntr + imdfetn + imbgeco + imbleco + imwbcnt + imwbcrm + imtcjob + imueclt + gvrfgap + imrsprc + rfgbfml + rfggvfn + rfgawrk + rfgfrpc + shrrfg + qfimchr + qfimwht + pplstrd + vrtrlg + imwgdwn + imhecop ' 5.4.5 Estimate the one-factor model, and summarize the results. Compare the fit measures for the one-factor and five-factor models Which model better fits the data? Note: Remember, you can use the lavaan::fitMeasures() function to extract only the model fit information from a fitted lavaan object. Click to show code ## Estimate the one factor model: fit_1f <- cfa(mod_1f, data = ess) ## Summarize the results: summary(fit_1f) ## lavaan 0.6.16 ended normally after 47 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 50 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 49510.917 ## Degrees of freedom 275 ## P-value (Chi-square) 0.000 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## ati =~ ## imrcntr 1.000 ## eimrcnt 0.937 0.012 78.324 0.000 ## eimpcnt 1.114 0.011 101.263 0.000 ## imsmetn 0.987 0.011 90.990 0.000 ## impcntr 1.147 0.011 102.371 0.000 ## imdfetn 1.153 0.011 103.148 0.000 ## imbgeco -2.055 0.032 -64.749 0.000 ## imbleco -1.625 0.031 -52.533 0.000 ## imwbcnt -2.173 0.030 -71.324 0.000 ## imwbcrm -1.432 0.029 -48.849 0.000 ## imtcjob -1.532 0.029 -52.519 0.000 ## imueclt -2.198 0.033 -65.876 0.000 ## gvrfgap 0.807 0.016 51.746 0.000 ## imrsprc 0.757 0.015 49.790 0.000 ## rfgbfml 0.861 0.017 51.272 0.000 ## rfggvfn 0.722 0.015 47.671 0.000 ## rfgawrk 0.530 0.015 34.448 0.000 ## rfgfrpc -0.755 0.015 -51.462 0.000 ## shrrfg -0.931 0.015 -61.438 0.000 ## qfimchr 1.597 0.042 37.835 0.000 ## qfimwht 1.769 0.038 46.697 0.000 ## pplstrd -0.873 0.016 -53.994 0.000 ## vrtrlg 0.602 0.015 39.940 0.000 ## imwgdwn -0.682 0.016 -43.576 0.000 ## imhecop -0.773 0.016 -49.611 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr 0.327 0.004 79.021 0.000 ## .eimrcnt 0.388 0.005 80.422 0.000 ## .eimpcnt 0.161 0.002 70.832 0.000 ## .imsmetn 0.235 0.003 77.101 0.000 ## .impcntr 0.158 0.002 69.688 0.000 ## .imdfetn 0.150 0.002 68.791 0.000 ## .imbgeco 3.381 0.041 82.203 0.000 ## .imbleco 3.666 0.044 83.130 0.000 ## .imwbcnt 2.839 0.035 81.477 0.000 ## .imwbcrm 3.399 0.041 83.334 0.000 ## .imtcjob 3.260 0.039 83.130 0.000 ## .imueclt 3.683 0.045 82.092 0.000 ## .gvrfgap 0.938 0.011 83.176 0.000 ## .imrsprc 0.906 0.011 83.285 0.000 ## .rfgbfml 1.092 0.013 83.203 0.000 ## .rfggvfn 0.917 0.011 83.394 0.000 ## .rfgawrk 1.031 0.012 83.913 0.000 ## .rfgfrpc 0.832 0.010 83.192 0.000 ## .shrrfg 0.803 0.010 82.499 0.000 ## .qfimchr 7.613 0.091 83.803 0.000 ## .qfimwht 5.772 0.069 83.442 0.000 ## .pplstrd 0.988 0.012 83.040 0.000 ## .vrtrlg 0.958 0.011 83.728 0.000 ## .imwgdwn 1.010 0.012 83.583 0.000 ## .imhecop 0.954 0.011 83.294 0.000 ## ati 0.353 0.007 48.941 0.000 ## Compare fit statistics: fitMeasures(fit_5f, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 18631.556 ## Degrees of freedom 265 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.885 ## Tucker-Lewis Index (TLI) 0.869 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 fitMeasures(fit_1f, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 49510.917 ## Degrees of freedom 275 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.691 ## Tucker-Lewis Index (TLI) 0.663 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.112 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.087 Click for explanation The one-factor model definitely seems to fit worse than the five-factor model. 5.4.6 Given the CFA results from the five factor model, would a second-order CFA be appropriate for the Attitudes towards Immigration data? Why or why not? Click for explanation Yes, a second-order CFA model is a theoretically appropriate representation of the Attitudes towards Immigration items. The first order latent variables in the five-factor model are all significantly correlated. The first order latent variables in the five-factor model seem to tap different aspects of some single underlying construct. 5.4.7 Define the lavaan model syntax for a second-order CFA model of the Attitudes towards Immigration items, estimate it, and inspect the results. Use the five factors defined in 5.4.2 as the first order factors. Click to show code mod_2o <- paste(mod_5f, 'ati =~ ip + rp + st + ct + et', sep = '\\n') fit_2o <- cfa(mod_2o, data = ess) summary(fit_2o, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 94 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 55 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 19121.111 ## Degrees of freedom 270 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 159619.058 ## Degrees of freedom 300 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.882 ## Tucker-Lewis Index (TLI) 0.869 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -520279.910 ## Loglikelihood unrestricted model (H1) -510719.354 ## ## Akaike (AIC) 1040669.820 ## Bayesian (BIC) 1041085.841 ## Sample-size adjusted Bayesian (SABIC) 1040911.056 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## 90 Percent confidence interval - lower 0.069 ## 90 Percent confidence interval - upper 0.071 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## ip =~ ## imrcntr 1.000 ## eimrcnt 0.943 0.011 85.095 0.000 ## eimpcnt 1.126 0.010 113.523 0.000 ## imsmetn 0.982 0.010 98.910 0.000 ## impcntr 1.149 0.010 113.651 0.000 ## imdfetn 1.130 0.010 111.789 0.000 ## st =~ ## imbgeco 1.000 ## imbleco 0.822 0.012 68.916 0.000 ## imwbcnt 1.047 0.012 88.172 0.000 ## imwbcrm 0.709 0.011 62.846 0.000 ## imtcjob 0.747 0.011 66.424 0.000 ## imueclt 1.013 0.013 78.434 0.000 ## rp =~ ## gvrfgap 1.000 ## imrsprc 0.854 0.017 51.127 0.000 ## rfgbfml 1.048 0.019 55.377 0.000 ## rfggvfn 0.853 0.017 51.170 0.000 ## rfgawrk 0.657 0.016 40.785 0.000 ## rfgfrpc -0.828 0.016 -51.249 0.000 ## shrrfg -1.020 0.017 -58.369 0.000 ## ct =~ ## qfimchr 1.000 ## qfimwht 0.939 0.018 51.902 0.000 ## pplstrd -0.389 0.008 -51.072 0.000 ## vrtrlg 0.271 0.006 41.908 0.000 ## et =~ ## imwgdwn 1.000 ## imhecop 1.158 0.024 48.877 0.000 ## ati =~ ## ip 1.000 ## rp 1.264 0.024 53.732 0.000 ## st -3.123 0.051 -61.058 0.000 ## ct 2.638 0.058 45.467 0.000 ## et -1.000 0.024 -42.490 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr 0.299 0.004 77.900 0.000 ## .eimrcnt 0.359 0.005 79.597 0.000 ## .eimpcnt 0.116 0.002 62.698 0.000 ## .imsmetn 0.211 0.003 75.502 0.000 ## .impcntr 0.119 0.002 62.476 0.000 ## .imdfetn 0.133 0.002 65.406 0.000 ## .imbgeco 2.285 0.033 70.158 0.000 ## .imbleco 2.852 0.037 76.762 0.000 ## .imwbcnt 1.668 0.027 62.920 0.000 ## .imwbcrm 2.821 0.036 78.653 0.000 ## .imtcjob 2.646 0.034 77.607 0.000 ## .imueclt 2.734 0.038 72.213 0.000 ## .gvrfgap 0.740 0.010 73.738 0.000 ## .imrsprc 0.797 0.010 77.211 0.000 ## .rfgbfml 0.885 0.012 74.621 0.000 ## .rfggvfn 0.791 0.010 77.189 0.000 ## .rfgawrk 0.946 0.012 80.833 0.000 ## .rfgfrpc 0.741 0.010 77.149 0.000 ## .shrrfg 0.665 0.009 72.020 0.000 ## .qfimchr 5.347 0.081 65.623 0.000 ## .qfimwht 4.084 0.065 62.673 0.000 ## .pplstrd 0.778 0.012 64.838 0.000 ## .vrtrlg 0.854 0.011 75.931 0.000 ## .imwgdwn 0.655 0.012 52.977 0.000 ## .imhecop 0.468 0.014 33.353 0.000 ## .ip 0.177 0.004 44.418 0.000 ## .st 0.596 0.023 26.030 0.000 ## .rp 0.101 0.005 21.784 0.000 ## .ct 1.745 0.060 29.185 0.000 ## .et 0.316 0.010 31.813 0.000 ## ati 0.204 0.005 37.371 0.000 5.4.8 Compare the model fit of the first- and second-order five-factor models using the fitMeasures() function. Which model offers the better fit? Which model is more complex? Click to show code fitMeasures(fit_5f, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 18631.556 ## Degrees of freedom 265 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.885 ## Tucker-Lewis Index (TLI) 0.869 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 fitMeasures(fit_2o, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 19121.111 ## Degrees of freedom 270 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.882 ## Tucker-Lewis Index (TLI) 0.869 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 Click for explanation The CFI and TLI are both slightly better in the original five factor model, but the RMSEA and SRMR of both models don’t differ out to three decimal places. As usual, both models have a significant \\(\\chi^2\\), but that doesn’t tell us much. Qualitative comparisons of model fit are fine, but we’d like to have an actual statistical test for these fit differences. As it happens, we have just such a test: a nested model \\(\\Delta \\chi^2\\) test (AKA, chi-squared difference test, change in chi-squared test, likelihood ratio test). In the Week 7 lecture, we’ll cover nested models and tests thereof, but it will be useful to start thinking about these concepts now. Two models are said to be nested if you can define one model by placing constraints on the other model. By way of example, consider the following two CFA models. The second model is nested within the first model, because we can define the second model by fixing the latent covariance to zero in the first model. Notice that the data contain \\(6(6 + 1) / 2 = 21\\) unique pieces of information. The first model estimates 13 parameters, and the second model estimates 12 parameters. Hence the first model has 8 degrees of freedom, and the second model has 9 degrees of freedom. In general, the following must hold whenever Model B is nested within Model A. Model B will have fewer estimated parameters than Model A. Model B will have more degrees of freedom than Model A. Model A will be more complex than model B. Model A will fit the data better than model B. Saturated Model All models are nested within the saturated model, because the saturated model estimates all possible relations among the variables. Regardless of what model we may be considering, we can always convert that model to a saturated model by estimating all possible associations. Hence, all models are nested within the saturated model. Baseline Model Similarly, the baseline model (AKA, independence model) is nested within all other models. In the baseline model, we only estimate the variances of the observed items; all associations are constrained to zero. We can always convert our model to the baseline model by fixing all associations to zero. Hence, the baseline model is nested within all other models. When two models are nested, we can use a \\(\\Delta \\chi^2\\) test to check if the nested model fits significantly worse than its parent model. Whenever we place constraints on the model, the fit will deteriorate, but we want to know if the constraints we imposed to define the nested model have produced too much loss of fit. We can use the anova() function to easily conduct \\(\\Delta \\chi^2\\) tests comparing models that we’ve estimated with cfa() or sem(). 5.4.9 Use the anova() function to compare the five-factor model from 5.4.2 and one-factor model from 5.4.4. Explain what Df, Chisq, Chisq diff, Df diff, and Pr(>Chisq) mean. Which model is more complex? Which model fits better? What is the conclusion of the test? Click to show code anova(fit_1f, fit_5f) Click for explanation The Df column contains the degrees of freedom of each model. Higher df \\(\\Rightarrow\\) Less complex model The Chisq column shows the \\(\\chi^2\\) statistics (AKA, likelihood ratio statistics) for each model. \\(\\chi^2\\) = The ratio of the likelihoods for the estimated model and the saturated model). Larger \\(\\chi^2\\) \\(\\Rightarrow\\) Worse fit Chisq diff is the difference between the two \\(\\chi^2\\) values (i.e., \\(\\Delta \\chi^2\\)). How much better the more complex model fits the data Larger \\(\\Delta \\chi^2\\) values indicate greater losses of fit induced by the constraints needed to define the nested model. Df diff is the difference in the degrees of freedom between the models. Since both models must be estimated from the same pool of variables, this difference also represents the number of parameters that were constrained to define the nested model. Pr(>Chisq) is a p-value for the \\(\\Delta \\chi^2\\) test. \\(H_0: \\Delta \\chi^2 = 0\\) \\(H_1: \\Delta \\chi^2 > 0\\) The five-factor model is more complex than the one-factor model, but the extra complexity is justified The five-factor model fits significantly better than the one-factor model. 5.4.10 Use the anova() function to compare the first- and second-order five-factor models from 5.4.2 and 5.4.7. Which model is more complex? What is the conclusion of the test? Click to show code anova(fit_5f, fit_2o) Click for explanation The first-order model is more complex than the second-order model (df = 265 vs. df = 270), and the extra complexity is necessary. The first-order model fits significantly better than the second-order model. 5.4.11 Based on the results above, would you say that you have successfully confirmed the five-factor structure implied by the EFA? Click for explanation Nope, not so much. The first-order five-factor model may fit the data best out of the three models considered here, but it still fits terribly. None of these models is an adequate representation of the Attitudes toward Immigration items. This result is particularly embarrassing when you consider that we’ve stacked the deck in our favor by using the same data to conduct the EFA and the CFA. When we fail to support the hypothesized measurement model, the confirmatory phase of our analysis is over. At this point, we’ve essentially rejected our hypothesized measurement structure, and that’s the conclusion of our analysis. We don’t have to throw up our hands in despair, however. We can still contribute something useful by modifying the theoretical measurement model through an exploratory, data-driven, post-hoc analysis. We’ll give that a shot below. 5.4.12 Modify the five-factor CFA from 5.4.2 by freeing the following parameters. The residual covariance between imrcntr and eimrcnt These questions both ask about allowing immigration from wealthy countries. It makes sense that answers on these two items share some additional, unique variance above-and-beyond what they contribute to the common factors. The residual covariance between qfimchr and qfimwht These questions are both about imposing qualifications on immigration (specifically Christian religion and “white” race). Click to show code fit_5f_cov <- paste(mod_5f, 'imrcntr ~~ eimrcnt', 'qfimchr ~~ qfimwht', sep = '\\n') %>% cfa(data = ess) summary(fit_5f_cov, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 77 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 62 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 9740.512 ## Degrees of freedom 263 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 159619.058 ## Degrees of freedom 300 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.941 ## Tucker-Lewis Index (TLI) 0.932 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -515589.611 ## Loglikelihood unrestricted model (H1) -510719.354 ## ## Akaike (AIC) 1031303.221 ## Bayesian (BIC) 1031772.190 ## Sample-size adjusted Bayesian (SABIC) 1031575.160 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.050 ## 90 Percent confidence interval - lower 0.049 ## 90 Percent confidence interval - upper 0.051 ## P-value H_0: RMSEA <= 0.050 0.280 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.036 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## ip =~ ## imrcntr 1.000 ## eimrcnt 0.928 0.007 126.255 0.000 ## eimpcnt 1.184 0.011 106.508 0.000 ## imsmetn 1.012 0.011 92.436 0.000 ## impcntr 1.213 0.011 107.078 0.000 ## imdfetn 1.181 0.011 104.566 0.000 ## st =~ ## imbgeco 1.000 ## imbleco 0.826 0.012 69.006 0.000 ## imwbcnt 1.050 0.012 88.051 0.000 ## imwbcrm 0.715 0.011 63.128 0.000 ## imtcjob 0.751 0.011 66.542 0.000 ## imueclt 1.015 0.013 78.256 0.000 ## rp =~ ## gvrfgap 1.000 ## imrsprc 0.858 0.017 51.965 0.000 ## rfgbfml 1.046 0.019 56.104 0.000 ## rfggvfn 0.848 0.016 51.644 0.000 ## rfgawrk 0.652 0.016 40.998 0.000 ## rfgfrpc -0.813 0.016 -51.233 0.000 ## shrrfg -1.002 0.017 -58.499 0.000 ## ct =~ ## qfimchr 1.000 ## qfimwht 0.979 0.020 48.332 0.000 ## pplstrd -0.586 0.014 -40.685 0.000 ## vrtrlg 0.397 0.011 36.273 0.000 ## et =~ ## imwgdwn 1.000 ## imhecop 1.157 0.023 49.549 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr ~~ ## .eimrcnt 0.230 0.004 59.907 0.000 ## .qfimchr ~~ ## .qfimwht 2.558 0.064 40.233 0.000 ## ip ~~ ## st -0.580 0.012 -48.041 0.000 ## rp 0.255 0.006 45.185 0.000 ## ct 0.467 0.014 34.425 0.000 ## et -0.197 0.006 -35.077 0.000 ## st ~~ ## rp -0.835 0.017 -48.285 0.000 ## ct -1.394 0.040 -35.128 0.000 ## et 0.670 0.017 38.935 0.000 ## rp ~~ ## ct 0.538 0.017 32.407 0.000 ## et -0.232 0.007 -32.949 0.000 ## ct ~~ ## et -0.469 0.017 -27.959 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr 0.330 0.004 78.903 0.000 ## .eimrcnt 0.396 0.005 80.392 0.000 ## .eimpcnt 0.109 0.002 60.401 0.000 ## .imsmetn 0.220 0.003 75.979 0.000 ## .impcntr 0.107 0.002 58.874 0.000 ## .imdfetn 0.131 0.002 64.630 0.000 ## .imbgeco 2.301 0.033 70.568 0.000 ## .imbleco 2.845 0.037 76.832 0.000 ## .imwbcnt 1.669 0.026 63.272 0.000 ## .imwbcrm 2.808 0.036 78.659 0.000 ## .imtcjob 2.639 0.034 77.663 0.000 ## .imueclt 2.741 0.038 72.463 0.000 ## .gvrfgap 0.734 0.010 73.743 0.000 ## .imrsprc 0.790 0.010 77.164 0.000 ## .rfgbfml 0.880 0.012 74.676 0.000 ## .rfggvfn 0.790 0.010 77.322 0.000 ## .rfgawrk 0.946 0.012 80.924 0.000 ## .rfgfrpc 0.747 0.010 77.519 0.000 ## .shrrfg 0.674 0.009 72.713 0.000 ## .qfimchr 6.815 0.090 75.362 0.000 ## .qfimwht 5.250 0.072 73.378 0.000 ## .pplstrd 0.674 0.013 52.766 0.000 ## .vrtrlg 0.818 0.011 73.191 0.000 ## .imwgdwn 0.655 0.012 53.496 0.000 ## .imhecop 0.468 0.014 33.845 0.000 ## ip 0.350 0.007 48.646 0.000 ## st 2.571 0.054 47.662 0.000 ## rp 0.433 0.012 36.718 0.000 ## ct 1.698 0.073 23.296 0.000 ## et 0.520 0.015 34.814 0.000 5.4.13 Evaluate the model modifications. Did the model fit significantly improve? Is the fit of the modified model acceptable? Click to show code anova(fit_5f_cov, fit_5f) fitMeasures(fit_5f_cov) ## npar fmin chisq ## 62.000 0.342 9740.512 ## df pvalue baseline.chisq ## 263.000 0.000 159619.058 ## baseline.df baseline.pvalue cfi ## 300.000 0.000 0.941 ## tli nnfi rfi ## 0.932 0.932 0.930 ## nfi pnfi ifi ## 0.939 0.823 0.941 ## rni logl unrestricted.logl ## 0.941 -515589.611 -510719.354 ## aic bic ntotal ## 1031303.221 1031772.190 14243.000 ## bic2 rmsea rmsea.ci.lower ## 1031575.160 0.050 0.049 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.051 0.900 0.280 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 0.000 0.080 ## rmr rmr_nomean srmr ## 0.103 0.103 0.036 ## srmr_bentler srmr_bentler_nomean crmr ## 0.036 0.036 0.037 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.037 0.036 0.036 ## cn_05 cn_01 gfi ## 442.344 467.858 0.944 ## agfi pgfi mfi ## 0.931 0.764 0.717 ## ecvi ## 0.693 Click for explanation Yes, the model fit improved significantly. In this case, the original five-factor model is nested within the modified model. So, our \\(\\Delta \\chi^2\\) test is evaluating the improvement in fit contributed by freeing the two residual covariances. The \\(\\Delta \\chi^2\\) test is significant, so we can conclude that including the two new parameter estimates has significantly improved the model fit. I.e., Estimating these two residual covariances is “worth it” in the sense of balancing model fit and model complexity. Also, the fit of the modified model is now acceptable. Caveat If we had found this result when testing our original model, we would be well-situated to proceed with our analysis. In this case, however, we are no longer justified in generalizing these estimates to the population. We only arrived at this well-fitting model by modifying our original theoretical model to better fit the data using estimates derived from those same data to guide our model modifications. We’ve conducted this post-hoc analysis to help inform future research, and this result is useful as a starting point for future studies. Now, anyone analyzing these scales in the future could incorporate these residual covariances into their initial theoretical model. Basically, we conduct these types of post-hoc analyses to help future researchers learn from our mistakes. End of In-Class Exercises "],["full-sem.html", "6 Full SEM", " 6 Full SEM This week, we will focus on integrating all of the disparate methods we’ve covered so far into full-fledged structural equation models. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-5.html", "6.1 Lecture", " 6.1 Lecture This week, we will begin with our final theme and discuss structural equation modeling (SEM). This powerful technique joins the strengths of CFA and path analysis to produce a highly flexible and theoretically appealing modeling tool. Essentially, SEM allows us to build structural path models using the latent variables defined by a CFA. 6.1.1 Recording 6.1.2 Slides You can download the lectures slides here "],["reading-5.html", "6.2 Reading", " 6.2 Reading Reference Weston, R. & Gore, P. A. (2006). A brief guide to structural equation modeling. The Counseling Psychologist 34, 719–752. Notes: This article is quite general and provides an overview of things we have discussed so far in this course. This article also also adds an important new idea: combining factor analysis with path modeling to produce a full Structural Equation Model (SEM). Skip the part on GFI (p. 741). The GFI has been shown to be too dependent on sample size and is not recommended any longer. Skip the part on missing data. There is nothing wrong with this section, but missing data analysis is a broad and difficult topic that we cannot adequately cover in this course. If you would like to learn more about missing data and how to treat them, you can take two courses offered by our department: Conducting a Survey Missing Data Theory and Causal Effects Questions The authors state three similarities and two big differences between SEM and other multivariate statistical techniques (e.g., ANCOVA, regression). What are these similarities and differences? Do you agree with the relative strengths and weaknesses of SEM vs. other methods that the authors present? The authors miss at least one additional advantage of SEM over other multivariate methods. What is this missing advantage? Explain what the terms “measurement model” and “structural model” mean in the SEM context. What are the 6 steps of doing an SEM-based analysis given by the authors? The authors claim that testing an SEM using cross-validation is a good idea. When is cross-validation helpful in SEM? Hint: You may have to do some independent (internet, literature) research to learn how cross-validation can be implemented in SEM. "],["at-home-exercises-5.html", "6.3 At-Home Exercises", " 6.3 At-Home Exercises This week, we’ll take another look at the Kestilä (2006) results. During this practical, you will conduct an SEM to replicate the regression analysis of the Finnish data that you conducted in the Week 4 In-Class Exercises. 6.3.1 Load the Finnish subsample of ESS data. The relevant data are contained in the ess_finland.rds file. These are the processed Finnish subsample data from the Week 4 exercises. Note: Unless otherwise noted, all the following analyses use these data. Click to show code ess_fin <- readRDS("ess_finland.rds") We need to do a little data processing before we can fit the regression model. At the moment, lavaan will not automatically convert a factor variable into dummy codes. So, we need to create explicit dummy codes for the two factors we’ll use as predictors in our regression analysis: sex and political orientation. 6.3.2 Convert the sex and political interest factors into dummy codes. Click to show code library(dplyr) ## Create a dummy codes by broadcasting a logical test on the factor levels: ess_fin <- mutate(ess_fin, female = ifelse(sex == "Female", 1, 0), hi_pol_interest = ifelse(polintr_bin == "High Interest", 1, 0) ) ## Check the results: with(ess_fin, table(dummy = female, factor = sex)) ## factor ## dummy Male Female ## 0 960 0 ## 1 0 1040 with(ess_fin, table(dummy = hi_pol_interest, factor = polintr_bin)) ## factor ## dummy Low Interest High Interest ## 0 1070 0 ## 1 0 929 Click for explanation In R, we have several ways of converting a factor into an appropriate set of dummy codes. We could use the dplyr::recode() function as we did last week. We can use the model.matrix() function to define a design matrix based on the inherent contrast attribute of the factor. Missing data will cause problems here. We can us as.numeric() to revert the factor to its underlying numeric representation {Male = 1, Female = 2} and use arithmetic to convert {1, 2} \\(\\rightarrow\\) {0, 1}. When our factor only has two levels, though, the ifelse() function is the simplest way. We are now ready to estimate our latent regression model. Specifically, we want to combine the three OLS regression models that you ran in 4.4.16 into a single SEM that we will estimate in lavaan. The following path diagram shows the intended theoretical model. Although the variances are not included in this path diagram, all variables in the model (including the observed predictor variables) are random. 6.3.3 Define the lavaan model syntax for the SEM shown above. Use the definition of the institutions, satsifaction, and politicians factors from 5.3.2 to define the DVs. Covary the three latent factors. Covary the five predictors. Click to show code mod_sem <- ' ## Define the latent DVs: institutions =~ trstlgl + trstplc + trstun + trstep + trstprl satisfaction =~ stfhlth + stfedu + stfeco + stfgov + stfdem politicians =~ pltinvt + pltcare + trstplt ## Specify the structural relations: institutions + satisfaction + politicians ~ female + age + eduyrs + hi_pol_interest + lrscale ' Click for explanation We simply need to add a line defining the latent regression paths to our old CFA syntax. We don’t need to specify the covariances in the syntax. We can use options in the sem() function to request those estimates. 6.3.4 Estimate the SEM, and summarize the results. Fit the model to the processed Finnish subsample from above. Estimate the model using lavaan::sem(). Request the standardized parameter estimates with the summary. Request the \\(R^2\\) estimates with the summary. Click to show code library(lavaan) ## Fit the SEM: fit_sem <- sem(mod_sem, data = ess_fin, fixed.x = FALSE) ## Summarize the results: summary(fit_sem, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 82 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 59 ## ## Used Total ## Number of observations 1740 2000 ## ## Model Test User Model: ## ## Test statistic 1287.421 ## Degrees of freedom 112 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 10534.649 ## Degrees of freedom 143 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.887 ## Tucker-Lewis Index (TLI) 0.856 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -57914.779 ## Loglikelihood unrestricted model (H1) -57271.068 ## ## Akaike (AIC) 115947.557 ## Bayesian (BIC) 116269.794 ## Sample-size adjusted Bayesian (SABIC) 116082.357 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.078 ## 90 Percent confidence interval - lower 0.074 ## 90 Percent confidence interval - upper 0.082 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 0.160 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.045 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions =~ ## trstlgl 1.000 1.418 0.669 ## trstplc 0.609 0.031 19.403 0.000 0.863 0.508 ## trstun 0.887 0.038 23.484 0.000 1.257 0.626 ## trstep 1.134 0.041 27.652 0.000 1.607 0.755 ## trstprl 1.192 0.040 29.444 0.000 1.689 0.815 ## satisfaction =~ ## stfhlth 1.000 0.979 0.497 ## stfedu 0.602 0.043 13.872 0.000 0.589 0.416 ## stfeco 1.266 0.067 18.848 0.000 1.240 0.681 ## stfgov 1.639 0.079 20.638 0.000 1.605 0.846 ## stfdem 1.521 0.075 20.180 0.000 1.489 0.793 ## politicians =~ ## pltinvt 1.000 0.567 0.566 ## pltcare 0.953 0.048 19.653 0.000 0.540 0.590 ## trstplt 3.281 0.133 24.675 0.000 1.860 0.915 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions ~ ## female 0.019 0.073 0.259 0.796 0.013 0.007 ## age -0.008 0.002 -3.740 0.000 -0.006 -0.105 ## eduyrs 0.034 0.010 3.233 0.001 0.024 0.091 ## hi_pol_interst 0.358 0.076 4.730 0.000 0.253 0.126 ## lrscale 0.104 0.018 5.634 0.000 0.073 0.147 ## satisfaction ~ ## female -0.147 0.050 -2.910 0.004 -0.150 -0.075 ## age -0.007 0.002 -4.598 0.000 -0.007 -0.129 ## eduyrs 0.005 0.007 0.775 0.439 0.006 0.022 ## hi_pol_interst 0.164 0.052 3.162 0.002 0.167 0.084 ## lrscale 0.099 0.013 7.501 0.000 0.101 0.202 ## politicians ~ ## female 0.010 0.029 0.349 0.727 0.018 0.009 ## age -0.004 0.001 -4.490 0.000 -0.007 -0.124 ## eduyrs 0.007 0.004 1.697 0.090 0.012 0.047 ## hi_pol_interst 0.258 0.031 8.364 0.000 0.455 0.227 ## lrscale 0.039 0.007 5.370 0.000 0.068 0.138 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .institutions ~~ ## .satisfaction 1.030 0.069 14.933 0.000 0.796 0.796 ## .politicians 0.675 0.041 16.628 0.000 0.908 0.908 ## .satisfaction ~~ ## .politicians 0.365 0.027 13.544 0.000 0.713 0.713 ## female ~~ ## age 0.071 0.212 0.335 0.738 0.071 0.008 ## eduyrs 0.179 0.046 3.869 0.000 0.179 0.093 ## hi_pol_interst -0.017 0.006 -2.767 0.006 -0.017 -0.066 ## lrscale -0.032 0.024 -1.316 0.188 -0.032 -0.032 ## age ~~ ## eduyrs -22.750 1.722 -13.212 0.000 -22.750 -0.334 ## hi_pol_interst 1.377 0.215 6.413 0.000 1.377 0.156 ## lrscale 1.774 0.853 2.079 0.038 1.774 0.050 ## eduyrs ~~ ## hi_pol_interst 0.270 0.047 5.787 0.000 0.270 0.140 ## lrscale 0.735 0.186 3.946 0.000 0.735 0.095 ## hi_pol_interest ~~ ## lrscale 0.016 0.024 0.672 0.501 0.016 0.016 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trstlgl 2.477 0.093 26.743 0.000 2.477 0.552 ## .trstplc 2.140 0.076 28.334 0.000 2.140 0.742 ## .trstun 2.453 0.090 27.322 0.000 2.453 0.608 ## .trstep 1.950 0.078 24.906 0.000 1.950 0.430 ## .trstprl 1.443 0.064 22.437 0.000 1.443 0.336 ## .stfhlth 2.922 0.104 28.103 0.000 2.922 0.753 ## .stfedu 1.663 0.058 28.613 0.000 1.663 0.827 ## .stfeco 1.775 0.069 25.755 0.000 1.775 0.536 ## .stfgov 1.020 0.056 18.371 0.000 1.020 0.284 ## .stfdem 1.307 0.060 21.953 0.000 1.307 0.371 ## .pltinvt 0.682 0.024 27.818 0.000 0.682 0.680 ## .pltcare 0.547 0.020 27.582 0.000 0.547 0.652 ## .trstplt 0.672 0.069 9.676 0.000 0.672 0.163 ## .institutions 1.881 0.125 15.077 0.000 0.936 0.936 ## .satisfaction 0.892 0.086 10.386 0.000 0.930 0.930 ## .politicians 0.294 0.024 12.224 0.000 0.914 0.914 ## female 0.250 0.008 29.496 0.000 0.250 1.000 ## age 313.238 10.620 29.496 0.000 313.238 1.000 ## eduyrs 14.818 0.502 29.496 0.000 14.818 1.000 ## hi_pol_interst 0.250 0.008 29.496 0.000 0.250 1.000 ## lrscale 4.034 0.137 29.496 0.000 4.034 1.000 ## ## R-Square: ## Estimate ## trstlgl 0.448 ## trstplc 0.258 ## trstun 0.392 ## trstep 0.570 ## trstprl 0.664 ## stfhlth 0.247 ## stfedu 0.173 ## stfeco 0.464 ## stfgov 0.716 ## stfdem 0.629 ## pltinvt 0.320 ## pltcare 0.348 ## trstplt 0.837 ## institutions 0.064 ## satisfaction 0.070 ## politicians 0.086 Click for explanation The fixed.x = FALSE argument tells lavaan to model the predictors as random variables. By default, lavaan will covary any random predictor variables. So, we don’t need to make any other changes to the usual procedure. 6.3.5 Finally, we will rerun the latent regression model from above as a path model with the factor scores from 4.4.10 acting as the DVs. Rerun the above SEM as a path model wherein the EFA-derived Trust in Institutions, Satisfaction with Political Systems, and Trust in Politicians factor scores act as the DVs. Request the standardized parameter estimates with the summary. Request the \\(R^2\\) estimates with the summary. Click to show code ## Define the model syntax for the path analysis: mod_pa <- ' trust_inst + satisfy + trust_pol ~ female + age + eduyrs + hi_pol_interest + lrscale' ## Estimate the path model: fit_pa <- sem(mod_pa, data = ess_fin, fixed.x = FALSE) ## Summarize the results: summary(fit_pa, standardized = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 44 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 36 ## ## Used Total ## Number of observations 1740 2000 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## trust_inst ~ ## female 0.004 0.045 0.091 0.928 0.004 0.002 ## age -0.003 0.001 -2.229 0.026 -0.003 -0.057 ## eduyrs 0.023 0.006 3.642 0.000 0.023 0.094 ## hi_pol_interst 0.167 0.046 3.599 0.000 0.167 0.088 ## lrscale 0.059 0.011 5.258 0.000 0.059 0.125 ## satisfy ~ ## female -0.125 0.040 -3.115 0.002 -0.125 -0.073 ## age -0.005 0.001 -4.102 0.000 -0.005 -0.105 ## eduyrs -0.003 0.006 -0.534 0.594 -0.003 -0.014 ## hi_pol_interst 0.073 0.041 1.782 0.075 0.073 0.043 ## lrscale 0.085 0.010 8.510 0.000 0.085 0.200 ## trust_pol ~ ## female 0.016 0.046 0.338 0.735 0.016 0.008 ## age -0.009 0.001 -6.480 0.000 -0.009 -0.161 ## eduyrs 0.018 0.007 2.839 0.005 0.018 0.071 ## hi_pol_interst 0.464 0.047 9.801 0.000 0.464 0.232 ## lrscale 0.055 0.011 4.801 0.000 0.055 0.110 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trust_inst ~~ ## .satisfy 0.437 0.021 20.609 0.000 0.437 0.568 ## .trust_pol 0.498 0.024 20.480 0.000 0.498 0.564 ## .satisfy ~~ ## .trust_pol 0.367 0.021 17.664 0.000 0.367 0.467 ## female ~~ ## age 0.071 0.212 0.335 0.738 0.071 0.008 ## eduyrs 0.179 0.046 3.869 0.000 0.179 0.093 ## hi_pol_interst -0.017 0.006 -2.767 0.006 -0.017 -0.066 ## lrscale -0.032 0.024 -1.316 0.188 -0.032 -0.032 ## age ~~ ## eduyrs -22.750 1.722 -13.212 0.000 -22.750 -0.334 ## hi_pol_interst 1.377 0.215 6.413 0.000 1.377 0.156 ## lrscale 1.774 0.853 2.079 0.038 1.774 0.050 ## eduyrs ~~ ## hi_pol_interst 0.270 0.047 5.787 0.000 0.270 0.140 ## lrscale 0.735 0.186 3.946 0.000 0.735 0.095 ## hi_pol_interest ~~ ## lrscale 0.016 0.024 0.672 0.501 0.016 0.016 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trust_inst 0.866 0.029 29.496 0.000 0.866 0.958 ## .satisfy 0.684 0.023 29.496 0.000 0.684 0.945 ## .trust_pol 0.902 0.031 29.496 0.000 0.902 0.902 ## female 0.250 0.008 29.496 0.000 0.250 1.000 ## age 313.238 10.620 29.496 0.000 313.238 1.000 ## eduyrs 14.818 0.502 29.496 0.000 14.818 1.000 ## hi_pol_interst 0.250 0.008 29.496 0.000 0.250 1.000 ## lrscale 4.034 0.137 29.496 0.000 4.034 1.000 ## ## R-Square: ## Estimate ## trust_inst 0.042 ## satisfy 0.055 ## trust_pol 0.098 Click to show explanation We don’t so anything particularly special here. We simply rerun our latent regression as a path analysis with the EFA-derived factor scores as the DVs. 6.3.6 Compare the results from the path analysis to the SEM-based results. Does it matter whether we use a latent variable or a factor score to define the DV? Hint: When comparing parameter estimates, use the fully standardized estimates (i.e., the values in the column labeled Std.all). Click to show code Note: The “supportFunction.R” script that we source below isn’t a necessary part of the solution. This script defines a bunch of convenience functions. One of these functions, partSummary(), allows us to print selected pieces of the model summary. ## Source a script of convenience function definitions: source("supportFunctions.R") ## View the regression estimates from the SEM: partSummary(fit_sem, 8, standardized = TRUE) ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions ~ ## female 0.019 0.073 0.259 0.796 0.013 0.007 ## age -0.008 0.002 -3.740 0.000 -0.006 -0.105 ## eduyrs 0.034 0.010 3.233 0.001 0.024 0.091 ## hi_pol_interst 0.358 0.076 4.730 0.000 0.253 0.126 ## lrscale 0.104 0.018 5.634 0.000 0.073 0.147 ## satisfaction ~ ## female -0.147 0.050 -2.910 0.004 -0.150 -0.075 ## age -0.007 0.002 -4.598 0.000 -0.007 -0.129 ## eduyrs 0.005 0.007 0.775 0.439 0.006 0.022 ## hi_pol_interst 0.164 0.052 3.162 0.002 0.167 0.084 ## lrscale 0.099 0.013 7.501 0.000 0.101 0.202 ## politicians ~ ## female 0.010 0.029 0.349 0.727 0.018 0.009 ## age -0.004 0.001 -4.490 0.000 -0.007 -0.124 ## eduyrs 0.007 0.004 1.697 0.090 0.012 0.047 ## hi_pol_interst 0.258 0.031 8.364 0.000 0.455 0.227 ## lrscale 0.039 0.007 5.370 0.000 0.068 0.138 ## View the regression estimates from the path analysis: partSummary(fit_pa, 7, standardized = TRUE) ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## trust_inst ~ ## female 0.004 0.045 0.091 0.928 0.004 0.002 ## age -0.003 0.001 -2.229 0.026 -0.003 -0.057 ## eduyrs 0.023 0.006 3.642 0.000 0.023 0.094 ## hi_pol_interst 0.167 0.046 3.599 0.000 0.167 0.088 ## lrscale 0.059 0.011 5.258 0.000 0.059 0.125 ## satisfy ~ ## female -0.125 0.040 -3.115 0.002 -0.125 -0.073 ## age -0.005 0.001 -4.102 0.000 -0.005 -0.105 ## eduyrs -0.003 0.006 -0.534 0.594 -0.003 -0.014 ## hi_pol_interst 0.073 0.041 1.782 0.075 0.073 0.043 ## lrscale 0.085 0.010 8.510 0.000 0.085 0.200 ## trust_pol ~ ## female 0.016 0.046 0.338 0.735 0.016 0.008 ## age -0.009 0.001 -6.480 0.000 -0.009 -0.161 ## eduyrs 0.018 0.007 2.839 0.005 0.018 0.071 ## hi_pol_interst 0.464 0.047 9.801 0.000 0.464 0.232 ## lrscale 0.055 0.011 4.801 0.000 0.055 0.110 ## View the R-squared estimates from the SEM: partSummary(fit_sem, 11, rsquare = TRUE) ## R-Square: ## Estimate ## trstlgl 0.448 ## trstplc 0.258 ## trstun 0.392 ## trstep 0.570 ## trstprl 0.664 ## stfhlth 0.247 ## stfedu 0.173 ## stfeco 0.464 ## stfgov 0.716 ## stfdem 0.629 ## pltinvt 0.320 ## pltcare 0.348 ## trstplt 0.837 ## institutions 0.064 ## satisfaction 0.070 ## politicians 0.086 ## View the R-squared estimates from the SEM: partSummary(fit_pa, 10, rsquare = TRUE) ## R-Square: ## Estimate ## trust_inst 0.042 ## satisfy 0.055 ## trust_pol 0.098 Click for explanation It certainly looks like the way we define the DV has a meaningful impact. The patterns of significance differ between the two sets of regression slopes, and the \\(R^2\\) values are larger for the Institutions and Satisfaction factors in the SEM, and the \\(R^2\\) for the Politicians factor is higher in the path analysis. End of At-Home Exercises "],["in-class-exercises-5.html", "6.4 In-Class Exercises", " 6.4 In-Class Exercises In these exercises, you will use full structural equation modeling (SEM) to evaluate the Theory of Reasoned Action (TORA), which is a popular psychological theory of social behavior developed by Ajzen and Fishbein. The theory states that actual behavior is predicted by behavioral intention, which is in turn predicted by the attitude toward the behavior and subjective norms about the behavior. Later, a third determinant was added, perceived behavioral control. The extent to which people feel that they have control over their behavior also influences their behavior. The data we will use for this practical are available in the toradata.csv file. These data were synthesized according to the results of Reinecke (1998)’s investigation of condom use by young people between 16 and 24 years old. The data contain the following variables: respnr: Numeric participant ID behavior: The dependent variable condom use Measured on a 5-point frequency scale (How often do you…) intent: A single item assessing behavioral intention Measured on a similar 5-point scale (In general, do you intend to…). attit_1:attit_3: Three indicators of attitudes about condom use Measured on a 5-point rating scale (e.g., using a condom is awkward) norm_1:norm_3: Three indicators of social norms about condom use Measured on a 5-point rating scale (e.g., I think most of my friends would use…) control_1:control_3: Three indicators of perceived behavioral control Measured on a 5-point rating scale (e.g., I know well how to use a condom) sex: Binary factor indicating biological sex 6.4.1 Load the data contained in the toradata.csv file. Click to show code condom <- read.csv("toradata.csv", stringsAsFactors = TRUE) 6.4.2 The data contain multiple indicators of attitudes, norms, and control. Run a CFA for these three latent variables. Correlate the latent factors. Do the data support the measurement model for these latent factors? Are the three latent factors significantly correlated? Is it reasonable to proceed with our evaluation of the TORA theory? Click to show code library(lavaan) mod_cfa <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 ' fit <- cfa(mod_cfa, data = condom) summary(fit, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 29 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 35.611 ## Degrees of freedom 24 ## P-value (Chi-square) 0.060 ## ## Model Test Baseline Model: ## ## Test statistic 910.621 ## Degrees of freedom 36 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.987 ## Tucker-Lewis Index (TLI) 0.980 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -2998.290 ## Loglikelihood unrestricted model (H1) -2980.484 ## ## Akaike (AIC) 6038.580 ## Bayesian (BIC) 6112.530 ## Sample-size adjusted Bayesian (SABIC) 6045.959 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.044 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.073 ## P-value H_0: RMSEA <= 0.050 0.599 ## P-value H_0: RMSEA >= 0.080 0.017 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.037 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.036 0.068 15.308 0.000 ## attit_3 -1.002 0.067 -14.856 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 1.031 0.098 10.574 0.000 ## norm_3 0.932 0.093 10.013 0.000 ## control =~ ## control_1 1.000 ## control_2 0.862 0.129 6.699 0.000 ## control_3 0.968 0.133 7.290 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.340 0.069 4.957 0.000 ## control 0.475 0.073 6.468 0.000 ## norms ~~ ## control 0.338 0.064 5.254 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.418 0.052 8.047 0.000 ## .attit_2 0.310 0.047 6.633 0.000 ## .attit_3 0.369 0.049 7.577 0.000 ## .norm_1 0.504 0.071 7.130 0.000 ## .norm_2 0.469 0.071 6.591 0.000 ## .norm_3 0.635 0.075 8.465 0.000 ## .control_1 0.614 0.078 7.905 0.000 ## .control_2 0.865 0.091 9.520 0.000 ## .control_3 0.762 0.087 8.758 0.000 ## attitudes 0.885 0.116 7.620 0.000 ## norms 0.743 0.116 6.423 0.000 ## control 0.497 0.099 5.002 0.000 Click for explanation Yes, the model fits the data well, and the measurement parameters (e.g., factor loadings, residual variances) look reasonable. So, the data seem to support this measurement structure. Yes, all three latent variables are significantly, positively correlated. Yes. The measurement structure is supported, so we can use the latent variables to represent the respective constructs in our subsequent SEM. The TORA doesn’t actually say anything about the associations between these three factors, but it makes sense that they would be positively associated. So, we should find this result comforting. 6.4.3 Estimate the basic TORA model as an SEM. Predict intention from attitudes and norms. Predict condom use from intention. Use the latent versions of attitudes and norms. Covary the attitudes and norms factors. Does the model fit well? Do the estimates align with the TORA? How much variance in intention and condom use are explained by the model? Click to show code mod <- ' ## Define the latent variables: attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 ## Define the structural model: intent ~ attitudes + norms behavior ~ intent ' fit <- sem(mod, data = condom) summary(fit, fit.measures = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 24 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 18 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 27.890 ## Degrees of freedom 18 ## P-value (Chi-square) 0.064 ## ## Model Test Baseline Model: ## ## Test statistic 1089.407 ## Degrees of freedom 28 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.991 ## Tucker-Lewis Index (TLI) 0.986 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -2533.616 ## Loglikelihood unrestricted model (H1) -2519.671 ## ## Akaike (AIC) 5103.232 ## Bayesian (BIC) 5166.618 ## Sample-size adjusted Bayesian (SABIC) 5109.557 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.047 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.079 ## P-value H_0: RMSEA <= 0.050 0.523 ## P-value H_0: RMSEA >= 0.080 0.046 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.036 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.039 0.068 15.365 0.000 ## attit_3 -1.002 0.067 -14.850 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.983 0.087 11.333 0.000 ## norm_3 0.935 0.087 10.778 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.439 0.063 6.990 0.000 ## norms 0.693 0.077 8.977 0.000 ## behavior ~ ## intent 0.746 0.045 16.443 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.347 0.069 5.027 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.420 0.052 8.103 0.000 ## .attit_2 0.306 0.046 6.604 0.000 ## .attit_3 0.372 0.049 7.651 0.000 ## .norm_1 0.483 0.064 7.581 0.000 ## .norm_2 0.521 0.065 7.954 0.000 ## .norm_3 0.610 0.070 8.713 0.000 ## .intent 0.423 0.048 8.769 0.000 ## .behavior 0.603 0.054 11.180 0.000 ## attitudes 0.884 0.116 7.614 0.000 ## norms 0.765 0.113 6.767 0.000 ## ## R-Square: ## Estimate ## attit_1 0.678 ## attit_2 0.757 ## attit_3 0.705 ## norm_1 0.613 ## norm_2 0.587 ## norm_3 0.523 ## intent 0.639 ## behavior 0.520 Click for explanation Yes, the model still fits the data very well. Yes, the estimates all align with the TORA. Specifically, attitudes and norms both significantly predict intention, and intention significantly predicts condom use. The model explains 63.93% of the variance in intention and 51.96% of the variance in condom use. 6.4.4 Update your model to represent the extended TORA model that includes perceived behavioral control. Regress condom use onto perceived behavioral control. Use the latent variable representation of control. Covary all three exogenous latent factors. Does the model fit well? Do the estimates align with the updated TORA? How much variance in intention and condom use are explained by the model? Click to show code mod_tora <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms behavior ~ intent + control ' fit_tora <- sem(mod_tora, data = condom) summary(fit_tora, fit.measures = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 31 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 27 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 48.757 ## Degrees of freedom 39 ## P-value (Chi-square) 0.136 ## ## Model Test Baseline Model: ## ## Test statistic 1333.695 ## Degrees of freedom 55 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.992 ## Tucker-Lewis Index (TLI) 0.989 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -3551.160 ## Loglikelihood unrestricted model (H1) -3526.782 ## ## Akaike (AIC) 7156.320 ## Bayesian (BIC) 7251.400 ## Sample-size adjusted Bayesian (SABIC) 7165.807 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.032 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.057 ## P-value H_0: RMSEA <= 0.050 0.870 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.033 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.033 0.068 15.221 0.000 ## attit_3 -1.025 0.068 -15.097 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.984 0.087 11.256 0.000 ## norm_3 0.955 0.088 10.881 0.000 ## control =~ ## control_1 1.000 ## control_2 0.859 0.127 6.789 0.000 ## control_3 0.997 0.131 7.609 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.447 0.063 7.100 0.000 ## norms 0.706 0.078 9.078 0.000 ## behavior ~ ## intent 0.563 0.063 8.923 0.000 ## control 0.454 0.119 3.805 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.342 0.068 5.011 0.000 ## control 0.474 0.072 6.548 0.000 ## norms ~~ ## control 0.352 0.064 5.521 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.432 0.052 8.381 0.000 ## .attit_2 0.330 0.046 7.220 0.000 ## .attit_3 0.344 0.046 7.439 0.000 ## .norm_1 0.496 0.063 7.820 0.000 ## .norm_2 0.533 0.065 8.152 0.000 ## .norm_3 0.595 0.069 8.643 0.000 ## .control_1 0.625 0.075 8.372 0.000 ## .control_2 0.876 0.090 9.757 0.000 ## .control_3 0.746 0.084 8.874 0.000 ## .intent 0.409 0.047 8.769 0.000 ## .behavior 0.542 0.052 10.423 0.000 ## attitudes 0.872 0.115 7.566 0.000 ## norms 0.751 0.112 6.709 0.000 ## control 0.485 0.096 5.059 0.000 ## ## R-Square: ## Estimate ## attit_1 0.668 ## attit_2 0.738 ## attit_3 0.727 ## norm_1 0.602 ## norm_2 0.577 ## norm_3 0.535 ## control_1 0.437 ## control_2 0.290 ## control_3 0.392 ## intent 0.651 ## behavior 0.566 Click for explanation Yes, the model still fits the data very well. Yes, the estimates all align with the updated TORA. Specifically, attitudes and norms both significantly predict intention, while intention and control both significantly predict condom use. The model explains 65.11% of the variance in intention and 56.62% of the variance in condom use. The TORA model explicitly forbids direct paths from attitudes and norms to behaviors; these effects should be fully mediated by the behavioral intention. The theory does not specify how perceived behavioral control should affect behaviors. There may be a direct effect of control on behavior, or the effect may be (partially) mediated by intention. 6.4.5 Evaluate the hypothesized indirect effects of attitudes and norms. Include attitudes, norms, and control in your model as in 6.4.4. Does intention significantly mediate the effects of attitudes and norms on behavior? Don’t forget to follow all the steps we covered for testing mediation. Are both of the above effects completely mediated? Do these results comport with the TORA? Why or why not? Click for explanation mod <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ a1 * attitudes + a2 * norms behavior ~ b * intent + control + attitudes + norms ie_att := a1 * b ie_norm := a2 * b ' set.seed(235711) fit <- sem(mod, data = condom, se = "bootstrap", bootstrap = 1000) summary(fit, ci = TRUE) ## lavaan 0.6.16 ended normally after 36 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 29 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 48.629 ## Degrees of freedom 37 ## P-value (Chi-square) 0.096 ## ## Parameter Estimates: ## ## Standard errors Bootstrap ## Number of requested bootstrap draws 1000 ## Number of successful bootstrap draws 1000 ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes =~ ## attit_1 1.000 1.000 1.000 ## attit_2 1.033 0.060 17.261 0.000 0.925 1.165 ## attit_3 -1.025 0.064 -15.894 0.000 -1.163 -0.902 ## norms =~ ## norm_1 1.000 1.000 1.000 ## norm_2 0.984 0.071 13.794 0.000 0.843 1.127 ## norm_3 0.955 0.093 10.324 0.000 0.792 1.157 ## control =~ ## control_1 1.000 1.000 1.000 ## control_2 0.860 0.113 7.624 0.000 0.653 1.098 ## control_3 0.996 0.147 6.790 0.000 0.748 1.320 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## intent ~ ## attitudes (a1) 0.447 0.067 6.674 0.000 0.324 0.585 ## norms (a2) 0.706 0.078 9.094 0.000 0.569 0.878 ## behavior ~ ## intent (b) 0.545 0.075 7.282 0.000 0.389 0.686 ## control 0.428 0.232 1.847 0.065 0.046 0.934 ## attitudes 0.010 0.122 0.084 0.933 -0.249 0.226 ## norms 0.041 0.118 0.345 0.730 -0.194 0.266 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes ~~ ## norms 0.342 0.070 4.883 0.000 0.208 0.480 ## control 0.475 0.069 6.850 0.000 0.344 0.612 ## norms ~~ ## control 0.350 0.067 5.218 0.000 0.221 0.484 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## .attit_1 0.432 0.050 8.720 0.000 0.331 0.526 ## .attit_2 0.330 0.045 7.382 0.000 0.238 0.415 ## .attit_3 0.343 0.049 6.992 0.000 0.244 0.444 ## .norm_1 0.496 0.060 8.305 0.000 0.376 0.614 ## .norm_2 0.533 0.077 6.951 0.000 0.390 0.687 ## .norm_3 0.594 0.069 8.597 0.000 0.443 0.719 ## .control_1 0.624 0.076 8.216 0.000 0.477 0.763 ## .control_2 0.875 0.092 9.495 0.000 0.686 1.052 ## .control_3 0.745 0.079 9.398 0.000 0.574 0.889 ## .intent 0.409 0.050 8.169 0.000 0.309 0.507 ## .behavior 0.544 0.058 9.379 0.000 0.415 0.639 ## attitudes 0.872 0.104 8.387 0.000 0.675 1.077 ## norms 0.751 0.099 7.557 0.000 0.556 0.941 ## control 0.486 0.096 5.042 0.000 0.303 0.684 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## ie_att 0.244 0.050 4.860 0.000 0.150 0.352 ## ie_norm 0.385 0.066 5.835 0.000 0.268 0.527 Yes, both indirect effects are significant according to the 95% bootstrapped CIs. Yes, both effects are completely moderated by behavioral intention. We can infer as much because the direct effects of attitudes and norms on condom use are both nonsignificant. Yes, these results comport with the TORA. Both effects are fully mediated, as the theory stipulates. In addition to evaluating the significance of the indirect and direct effects, we can also take a model-comparison perspective. We can use model comparisons to test if removing the direct effects of attitudes and norms on condom use significantly decreases model fit. In other words, are those paths needed to accurately represent the data, or are they “dead weight”. 6.4.6 Use a \\(\\Delta \\chi^2\\) test to evaluate the necessity of including the direct effects of attitudes and norms on condom use in the model. What is your conclusion? Click for explanation We only need to compare the fit of the model with the direct effects included to the fit of the model without the direct effects. We’ve already estimated both models, so we can simply submit the fitted lavaan objects to the anova() function. anova(fit, fit_tora) The \\(\\Delta \\chi^2\\) test is not significant. So, we have not lost a significant amount of fit by fixing the direct effects to zero. In other words, the complete mediation model explains the data just as well as the partial mediation model. So, we should probably prefer the more parsimonious model. 6.4.7 Use some statistical means of evaluating the most plausible way to include perceived behavioral control into the model. Choose between the following three options: control predicts behavior via a direct, un-mediated effect. control predicts behavior via an indirect effect that is completely mediated by intention. control predicts behavior via both an indirect effect through intention and a residual direct effect. Hint: There is more than one way to approach this problem. Approach 1: Testing Effects Click to show code One way to tackle this problem is to test the indirect, direct, and total effects. ## Allow for partial mediation: mod1 <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms + a * control behavior ~ b * intent + c * control ie := a * b total := ie + c ' set.seed(235711) fit1 <- sem(mod1, data = condom, se = "bootstrap", bootstrap = 1000) summary(fit1, ci = TRUE) ## lavaan 0.6.16 ended normally after 33 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 28 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 47.389 ## Degrees of freedom 38 ## P-value (Chi-square) 0.141 ## ## Parameter Estimates: ## ## Standard errors Bootstrap ## Number of requested bootstrap draws 1000 ## Number of successful bootstrap draws 1000 ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes =~ ## attit_1 1.000 1.000 1.000 ## attit_2 1.034 0.060 17.222 0.000 0.925 1.167 ## attit_3 -1.021 0.064 -15.877 0.000 -1.158 -0.898 ## norms =~ ## norm_1 1.000 1.000 1.000 ## norm_2 0.985 0.071 13.803 0.000 0.848 1.133 ## norm_3 0.948 0.093 10.204 0.000 0.786 1.155 ## control =~ ## control_1 1.000 1.000 1.000 ## control_2 0.861 0.113 7.635 0.000 0.653 1.100 ## control_3 0.996 0.142 7.020 0.000 0.760 1.318 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## intent ~ ## attitudes 0.357 0.115 3.113 0.002 0.146 0.603 ## norms 0.646 0.095 6.794 0.000 0.473 0.859 ## control (a) 0.199 0.199 1.002 0.317 -0.188 0.633 ## behavior ~ ## intent (b) 0.551 0.074 7.487 0.000 0.391 0.683 ## control (c) 0.469 0.142 3.298 0.001 0.231 0.791 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes ~~ ## norms 0.344 0.070 4.905 0.000 0.210 0.481 ## control 0.471 0.069 6.838 0.000 0.342 0.608 ## norms ~~ ## control 0.345 0.066 5.240 0.000 0.215 0.481 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## .attit_1 0.429 0.050 8.628 0.000 0.329 0.524 ## .attit_2 0.325 0.045 7.230 0.000 0.233 0.408 ## .attit_3 0.347 0.049 7.011 0.000 0.248 0.455 ## .norm_1 0.490 0.060 8.172 0.000 0.373 0.612 ## .norm_2 0.525 0.076 6.869 0.000 0.385 0.684 ## .norm_3 0.599 0.070 8.529 0.000 0.447 0.729 ## .control_1 0.626 0.074 8.429 0.000 0.479 0.761 ## .control_2 0.875 0.092 9.522 0.000 0.689 1.049 ## .control_3 0.748 0.078 9.532 0.000 0.579 0.893 ## .intent 0.412 0.050 8.283 0.000 0.307 0.504 ## .behavior 0.541 0.055 9.873 0.000 0.423 0.639 ## attitudes 0.875 0.104 8.385 0.000 0.676 1.081 ## norms 0.757 0.099 7.616 0.000 0.560 0.949 ## control 0.484 0.095 5.092 0.000 0.306 0.683 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## ie 0.110 0.105 1.048 0.295 -0.105 0.309 ## total 0.578 0.186 3.108 0.002 0.235 0.971 Click for explanation From the above results, we can see that the direct and total effects are both significant, but the indirect effect is not. Hence, it probably makes the most sense to include control via a direct (non-mediated) effect on behavior. Approach 2.1: Nested Model Comparison Click to show code We can also approach this problem from a model-comparison perspective. We can fit models that encode each pattern of constraints and check which one best represents the data. ## Force complete mediation: mod2 <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms + control behavior ~ intent ' ## Force no mediation: mod3 <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms behavior ~ intent + control ' ## Estimate the two restricted models: fit2 <- sem(mod2, data = condom) fit3 <- sem(mod3, data = condom) ## Check the results: summary(fit2) ## lavaan 0.6.16 ended normally after 33 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 27 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 62.797 ## Degrees of freedom 39 ## P-value (Chi-square) 0.009 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.033 0.068 15.295 0.000 ## attit_3 -1.018 0.068 -15.087 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.985 0.087 11.305 0.000 ## norm_3 0.947 0.087 10.845 0.000 ## control =~ ## control_1 1.000 ## control_2 0.864 0.126 6.855 0.000 ## control_3 0.958 0.129 7.417 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.352 0.096 3.669 0.000 ## norms 0.644 0.088 7.347 0.000 ## control 0.207 0.163 1.268 0.205 ## behavior ~ ## intent 0.746 0.045 16.443 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.345 0.069 5.023 0.000 ## control 0.476 0.073 6.513 0.000 ## norms ~~ ## control 0.346 0.065 5.361 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.427 0.051 8.295 0.000 ## .attit_2 0.325 0.046 7.101 0.000 ## .attit_3 0.349 0.047 7.477 0.000 ## .norm_1 0.490 0.064 7.702 0.000 ## .norm_2 0.524 0.065 8.025 0.000 ## .norm_3 0.600 0.069 8.652 0.000 ## .control_1 0.610 0.076 8.015 0.000 ## .control_2 0.861 0.090 9.580 0.000 ## .control_3 0.769 0.086 8.938 0.000 ## .intent 0.412 0.046 8.890 0.000 ## .behavior 0.603 0.054 11.180 0.000 ## attitudes 0.877 0.115 7.596 0.000 ## norms 0.757 0.112 6.733 0.000 ## control 0.500 0.098 5.076 0.000 summary(fit3) ## lavaan 0.6.16 ended normally after 31 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 27 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 48.757 ## Degrees of freedom 39 ## P-value (Chi-square) 0.136 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.033 0.068 15.221 0.000 ## attit_3 -1.025 0.068 -15.097 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.984 0.087 11.256 0.000 ## norm_3 0.955 0.088 10.881 0.000 ## control =~ ## control_1 1.000 ## control_2 0.859 0.127 6.789 0.000 ## control_3 0.997 0.131 7.609 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.447 0.063 7.100 0.000 ## norms 0.706 0.078 9.078 0.000 ## behavior ~ ## intent 0.563 0.063 8.923 0.000 ## control 0.454 0.119 3.805 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.342 0.068 5.011 0.000 ## control 0.474 0.072 6.548 0.000 ## norms ~~ ## control 0.352 0.064 5.521 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.432 0.052 8.381 0.000 ## .attit_2 0.330 0.046 7.220 0.000 ## .attit_3 0.344 0.046 7.439 0.000 ## .norm_1 0.496 0.063 7.820 0.000 ## .norm_2 0.533 0.065 8.152 0.000 ## .norm_3 0.595 0.069 8.643 0.000 ## .control_1 0.625 0.075 8.372 0.000 ## .control_2 0.876 0.090 9.757 0.000 ## .control_3 0.746 0.084 8.874 0.000 ## .intent 0.409 0.047 8.769 0.000 ## .behavior 0.542 0.052 10.423 0.000 ## attitudes 0.872 0.115 7.566 0.000 ## norms 0.751 0.112 6.709 0.000 ## control 0.485 0.096 5.059 0.000 ## Do either of the restricted models fit worse than the partial mediation model? anova(fit1, fit2) anova(fit1, fit3) Click for explanation The above \\(\\Delta \\chi^2\\) tests tell us that the full mediation model fits significantly worse than the partial mediation model. Hence, forcing full mediation by fixing the direct effect to zero is an unreasonable restraint. The total effect model, on the other hand, does not fit significantly worse than the partial mediation model. So, we can conclude that removing the indirect effect and modeling the influence of control on behavior as an un-mediated direct association represents the data just as well as a model that allows for both indirect and direct effects. Hence, we should prefer the more parsimonious total effects model. Approach 2.2: Non-Nested Model Comparison Click to show code We can also use information criteria to compare our models. The two most popular information criteria are the Akaike’s Information Criterion (AIC) and the Bayesian Information Criterion (BIC). ## Which model is the most parsimonious representation of the data? AIC(fit1, fit2, fit3) BIC(fit1, fit2, fit3) Click for explanation While the effect tests and the nested model comparisons both lead us to prefer the non-mediated model, we cannot directly say that the complete mediation model fits significantly worse than the non-mediated model. We have not directly compared those two models, and we cannot do so with the \\(\\Delta \\chi^2\\). We cannot do such a test because these two models are not nested: we must both add and remove a path to get from one model specification to the other. Also, both models have the same degrees of freedom, so we cannot define a sampling distribution against which we would compare the \\(\\Delta \\chi^2\\), anyway. We can use information criteria to get around this problem, though. Information criteria can be used to compare both nested and non-nested models. These criteria are designed to rank models by balancing their fit to the data and their complexity. When comparing models based on information criteria, a lower value indicates a better model in the sense of a better balance of fit and parsimony. The above results show that both the AIC and the BIC agree that the no-mediation model is the best. Conclusion Click for explanation So, in the end, regardless of how we approach the question, all of our results suggest modeling perceived behavioral control as a direct, non-mediated predictor of condom use. End of In-Class Exercises "],["multiple-group-models.html", "7 Multiple Group Models", " 7 Multiple Group Models This week, you will cover multiple group modeling and measurement invariance testing in the SEM/CFA context. Homework before the lecture Watch the Lecture Recording for this week. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-6.html", "7.1 Lecture", " 7.1 Lecture In this lecture, we will explore how you can incorporate grouping factors into your CFA and SEM analyses. We’ll cover three general topics: The multiple group modeling framework Measurement invariance testing Using multiple group models to test for moderation 7.1.1 Recordings Multiple Group Models Measurement Invariance Measurement Invariance Examples Moderation by Group 7.1.2 Slides You can download the lecture slides here "],["reading-6.html", "7.2 Reading", " 7.2 Reading There is no official reading this week. Please contemplate the following image instead. \\[\\\\[12pt]\\] "],["at-home-exercises-6.html", "7.3 At-Home Exercises", " 7.3 At-Home Exercises Coming soon to a GitBook near you! "],["in-class-exercises-6.html", "7.4 In-Class Exercises", " 7.4 In-Class Exercises Coming soon to a GitBook near you! "],["wrap-up.html", "8 Wrap-Up", " 8 Wrap-Up There will be no new lecture or practical content this week. This is an open week that we’ll use to tie up any loose ends and wrap up the course content. "],["404.html", "Page not found", " Page not found The page you requested cannot be found (perhaps it was moved or renamed). You may want to try searching to find the page's new location, or use the table of contents to find the page you are looking for. "]] +[["index.html", "Theory Construction and Statistical Modeling Course Information", " Theory Construction and Statistical Modeling Kyle M. Lang Last updated: 2023-10-16 Course Information In order to test a theory, we must express the theory as a statistical model and then test this model on quantitative (numeric) data. In this course we will use datasets from different disciplines within the social sciences (educational sciences, psychology, and sociology) to explain and illustrate theories and practices that are used in all social science disciplines to statistically model social science theories. This course uses existing tutorial datasets to practice the process of translating verbal theories into testable statistical models. If you are interested in the methods of acquiring high quality data to test your own theory, we recommend following the course Conducting a Survey which is taught from November to January. Most information about the course is available in this GitBook. Course-related communication will be through https://uu.blackboard.com (Log in with your student ID and password). "],["acknowledgement.html", "Acknowledgement", " Acknowledgement This course was originally developed by dr. Caspar van Lissa. I (dr. Kyle M. Lang) have modified Caspar’s original materials and take full responsibility for any errors or inaccuracies introduced through these modifications. Credit for any particularly effective piece of pedagogy should probably go to Caspar. You can view the original version of this course here on Caspar’s GitHub page. "],["instructors.html", "Instructors", " Instructors Coordinator: dr. Kyle M. Lang Lectures: dr. Kyle M. Lang Practicals: Rianne Kraakman Daniëlle Remmerswaal Danielle McCool "],["course-overview.html", "Course overview", " Course overview This course comprises three parts: Path analysis: You will learn how to estimate complex path models of observed variables (e.g., linked linear regressions) as structural equation models. Factor analysis: You will learn different ways of defining and estimating latent (unobserved) constructs. Full structural equation modeling: You will combine the first two topics to estimate path models describing the associations among latent constructs. Each of these three themes will be evaluated with a separate assignment. The first two assignments will be graded on a pass/fail basis. Your course grade will be based on your third assignment grade. "],["schedule.html", "Schedule", " Schedule Course Week Calendar Week Lecture/Practical Topic Workgroup Activity Assignment Deadline 0 36 Pre-course preparation 1 37 Introduction to R 2 38 Statistical modeling, Path analysis 3 39 Mediation, Moderation 4 40 Exploratory factor analysis (EFA) A1 Peer-Review A1: 2023-10-04 @ 23:59 5 41 Confirmatory factor analysis (CFA) 6 42 Structural equation modeling (SEM) A2 Peer-Review A2: 2023-10-18 @ 23:59 7 43 Multiple group models 8 44 Wrap-up A3 Peer-Review 9 45 Exam week: No class meetings A3: 2023-11-10 @ 23:59 NOTE: The schedule (including topics covered and assignment deadlines) is subject to change at the instructors’ discretion. "],["learning-goals.html", "Learning goals", " Learning goals In this course you will learn how to translate a social scientific theory into a statistical model, how to analyze your data with these models, and how to interpret and report your results following APA standards. After completing the course, you will be able to: Translate a verbal theory into a conceptual model, and translate a conceptual model into a statistical model. Independently analyze data using the free, open-source statistical software R. Apply a latent variable model to a real-life problem wherein the observed variables are only indirect indicators of an unobserved construct. Use a path model to represent the hypothesized causal relations among several variables, including relationships such as mediation and moderation. Explain to a fellow student how structural equation modeling combines latent variable models with path models and the benefits of doing so. Reflect critically on the decisions involved in defining and estimating structural equation models. "],["resources.html", "Resources", " Resources Literature You do not need a separate book for this course! Most of the information is contained within this GitBook and the course readings (which you will be able to access via links in this GitBook). All literature is freely available online, as long as you are logging in from within the UU-domain (i.e., from the UU campus or through an appropriate VPN). All readings are linked in this GitBook via either direct download links or DOIs. If you run into any trouble accessing a given article, searching for the title using Google Scholar or the University Library will probably due the trick. Software You will do all of your statistical analyses with the statistical programming language/environment R and the add-on package lavaan. If you want to expand your learning, you can follow this optional lavaan tutorial. "],["reading-questions.html", "Reading questions", " Reading questions Along with every article, we will provide reading questions. You will not be graded on the reading questions, but it is important to prepare the reading questions before every lecture. The reading questions serve several important purposes: Provide relevant background knowledge for the lecture Help you recognize and understand the key terms and concepts Make you aware of important publications that shaped the field Help you extract the relevant insights from the literature "],["weekly-preparation.html", "Weekly preparation", " Weekly preparation Before every class meeting (both lectures and practicals) you need to do the assigned homework (delineated in the GitBook chapter for that week). This course follows a flipped classroom procedure, so you must complete the weekly homework to meaningfully participate in, and benefit from, the class meetings. Background knowledge We assume you have basic knowledge about multivariate statistics before entering this course. You do not need any prior experience working with R. If you wish to refresh your knowledge, we recommend the chapters on ANOVA, multiple regression, and exploratory factor analysis from Field’s Discovering Statistics using R. If you cannot access the Field book, many other introductory statistics textbooks cover these topics equally well. So, use whatever you have lying around from past statistics courses. You could also try one of the following open-access options: Applied Statistics with R Introduction to Modern Statistics Introduction to Statistical Learning "],["grading.html", "Grading", " Grading Your grade for the course is based on a “portfolio” composed of the three take-home assignments: Path modeling Deadline: Wednesday 2023-10-04 at 23:59 Group assignment Pass/Fail Confirmatory factor analysis Deadline: Wednesday 2023-10-18 at 23:59 Group assignment Pass/Fail Full structural equation modeling Deadline: Friday 2023-11-10 at 23:59 Individual assignment Comprises your entire numeric course grade The specifics of the assignments will be explicated in the Assignments chapter of this GitBook "],["attendance.html", "Attendance", " Attendance Attendance is not mandatory, but we strongly encourage you to attend all lectures and practicals. In our experience, students who actively participate tend to pass the course, whereas those who do not participate tend to drop out or fail. The lectures and practicals build on each other, so, in the unfortunate event that you have to miss a class meeting, please make sure you have caught up with the material before the next session. "],["assignments.html", "Assignments", " Assignments This chapter contains the details and binding information about the three assignments that comprise the portfolio upon which your course grade is based. Below, you can find a brief idea of what each assignment will cover. For each assignment, you will use R to analyze some real-world data, and you will write up your results in a concise report (not a full research paper). Guidelines for these analyses/reports are delineated in the following three sections. You will submit your reports via Blackboard. You will complete the first two assignments in your Assignment Groups. You will complete the third assignment individually. The first two assignments are graded as pass/fail. You must pass both of these assignments to pass the course. The third assignment constitutes your course grade. "],["assignment-1-path-analysis.html", "Assignment 1: Path Analysis", " Assignment 1: Path Analysis For the first assignment, you will work in groups to apply a path model that describes how several variables could be causally related. The components of the first assignment are described below. Choose a suitable dataset, and describe the data. You can use any of the 8 datasets linked below. State the research question; define and explicate the theoretical path model. This model must include, at least, three variables. Use a path diagram to show your theoretical model. Translate your theoretical path model into lavaan syntax, and estimate the model. Include the code used to define and estimate your model as an appendix. Explain your rationale for important modeling decisions. Discuss the conceptual fit between your theory and your model. Evaluate the model assumptions. Discuss other important decisions that could have influence your results. Report the results in APA style. Provide relevant output in a suitable format. Include measures of explained variance for the dependent variables. Discuss the results. Use your results to answer the research question. Consider the strengths and limitations of your analysis. Evaluation See the Grading section below for more information on how Assignment 1 will be evaluated. You can access an evaluation matrix for Assignment 1 here. This matrix gives an indication of what level of work constitutes insufficient, sufficient, and excellent responses to the six components described above. Submission Assignment 1 is due at 23:59 on Wednesday 4 October 2023. Submit your report via the Assignment 1 portal on Blackboard. "],["assignment-2-confirmatory-factor-analysis.html", "Assignment 2: Confirmatory Factor Analysis", " Assignment 2: Confirmatory Factor Analysis In the second assignment, you will work in groups to run a CFA wherein the observed variables are indirect indicators of the unobserved constructs you want to analyze. The components of the second assignment are described below. Choose a suitable dataset, and describe the data. Ideally, you will work with the same data that you analyzed in Assignment 1. If you want to switch, you can use any of the 8 datasets linked below. State the research question; define and explicate the theoretical CFA model. This model must include, at least, two latent constructs. Use a path diagram to represent your model. Translate your theoretical model into lavaan syntax, and estimate the model. Include the code used to define and estimate your model as an appendix. Explain your rationale for important modeling decisions. Discuss the conceptual fit between your theory and your model. Evaluate the model assumptions. Discuss other important decisions that could have influence your results. Report the results in APA style. Provide relevant output in a suitable format. Include measures of model fit. Discuss the results. Use your results to answer the research question. Consider the strengths and limitations of your analysis. Evaluation See the Grading section below for more information on how Assignment 2 will be evaluated. You can access an evaluation matrix for Assignment 2 here. This matrix gives an indication of what level of work constitutes insufficient, sufficient, and excellent responses to the six components described above. Submission Assignment 2 is due at 23:59 on Wednesday 18 October 2023. Submit your report via the Assignment 2 portal on Blackboard. "],["a3_components.html", "Assignment 3: Full Structural Equation Model", " Assignment 3: Full Structural Equation Model In the third assignment, you will work individually to apply a full SEM that describes how several (latent) variables could be causally related. The components of the third assignment are described below. Choose a suitable dataset, and describe the data. Ideally, you will work with the same data that you analyzed in Assignments 1 & 2. If you want to switch, you can use any of the 8 datasets linked below. State the research question; define and explicate the theoretical SEM. The structural component of this model must include, at least, three variables. The model must include, at least, two latent variables. Use a path diagram to represent your model. Translate your theoretical SEM into lavaan syntax, and estimate the model. Include the code used to define and estimate your model as an appendix. Explain your rationale for important modeling decisions. Discuss the conceptual fit between your theory and your model. Evaluate the model assumptions. Discuss other important decisions that could have influence your results. Report the results in APA style. Provide relevant output in a suitable format. Include measures of model fit. Include measures of explained variance for the dependent variables. Discuss the results. Use your results to answer the research question. Consider the strengths and limitations of your analysis. Evaluation See the Grading section below for more information on how the component scores represented in the rubric are combined into an overall assignment grade. You can access an evaluation matrix for Assignment 3 here. This matrix gives an indication of what level of work constitutes insufficient, sufficient, and excellent responses to the six components described above. Submission Assignment 3 is due at 23:59 on Friday 10 November 2023. Submit your report via the Assignment 3 portal on Blackboard. "],["elaboration-tips.html", "Elaboration & Tips", " Elaboration & Tips Theoretical Model & Research Question You need to provide some justification for your model and research question, but only enough to demonstrate that you’ve actually conceptualized and estimated a theoretically plausible statistical model (as opposed to randomly combining variables until lavaan returns a pretty picture). You have several ways to show that your model is plausible. Use common-sense arguments. Reference (a small number of) published papers. Replicate an existing model/research question. Don’t provide a rigorous literature-supported theoretical motivation. You don’t have the time to conduct a thorough literature review, and we don’t have the time to read such reviews when grading. Literature review is not one of the learning goals for this course, so you cannot get “bonus points” for an extensive literature review. You are free to test any plausible model that meets the size requirements. You can derive your own model/research question or you can replicate a published analysis. Model Specifications We will not cover methods for modeling categorical outcome variables. So, use only continuous variables as outcomes. DVs in path models and the structural parts of SEMs Observed indicators of latent factors in CFA/SEM NOTE: You may treat ordinal items as continuous, for the purposes of these assignments. We will not cover methods for latent variable interactions. Don’t specify a theoretical model that requires an interaction involving a latent construct. There is one exception to the above prohibition. If the moderator is an observed grouping variable, you can estimate the model as a multiple-group model. We’ll cover these methods in Week 7. Assumptions You need to show that you’re thinking about the assumptions and their impact on your results, but you don’t need to run thorough model diagnostics. Indeed, the task of checking assumptions isn’t nearly as straight forward in path analysis, CFA, and SEM as it is in linear regression modeling. You won’t be able to directly apply the methods you have learned for regression diagnostics, for example. Since all of our models are estimated with normal-theory maximum likelihood, the fundamental assumption of all the models we’ll consider in this course boils down to the following. All random variables in my model are i.i.d. multivariate normally distributed. So, you can get by with basic data screening and checking the observed random variables in your model (i.e., all variables other than fixed predictors) for normality. Since checking for multivariate normality is a bit tricky, we’ll only ask you to evaluate univariate normality. You should do these evaluations via graphical means. To summarize, we’re looking for the following. Data Consider whether the measurement level of your data matches the assumptions of your model. Check your variables for univariate outliers. If you find any outliers, either treat them in some way or explain why you are retaining them for the analysis. Check for missing data. For the purposes of the assignment, you can use complete case analysis to work around the missing data. If you’re up for more of a challenge, feel free to try multiple imputation or full information maximum likelihood. Model Evaluate the univariate normality of any random, observed variables in your model. E.g., DVs in path models, observed IVs modeled as random variables, indicators of latent factors If you fit a multiple-group model for Assignment 3, do this evaluation within groups. Use graphical tools to evaluate the normality assumption. Normal QQ-Plots Histograms Results What do we mean by reporting your results “in a suitable format”? Basically, put some effort into making your results readable, and don’t include a bunch of superfluous information. Part of demonstrating that you understand the analysis is showing that you know which pieces of output convey the important information. Tabulate your results; don’t directly copy the R output. Don’t include everything lavaan gives you. Include only the output needed to understand your results and support your conclusions. "],["data_options.html", "Data", " Data Below, you can find links to a few suitable datasets that you can use for the assignments. You must use one of the following datasets. You may not choose your own data from the wild. Coping with Covid Dataset Codebook Pre-Registration Feminist Perspectives Scale Dataset Article Hypersensitive Narcissism Scale & Dirty Dozen Dataset HSNS Article DD Article Kentucky Inventory of Mindfulness Skills Dataset Article Depression Anxiety Stress Scale Dataset DASS Information Nomophobia Dataset Recylced Water Acceptance Dataset Article "],["procedures.html", "Procedures", " Procedures Formatting You must submit your assignment reports in PDF format. Each report should include a title page. The title page should include the following information: The name of the assignment. The names of all assignment authors (i.e., all group members for Assignments 1 & 2, your name for Assignment 3). The Assignment Group number (only for Assignments 1 & 2). You must include the code used to define and run your model(s) as an appendix. Try to format the text in this appendix clearly. Use a monospace font. Length You may use as many words as necessary to adequately explain yourself; though, concision and parsimony are encouraged. Note that the assignments are not intended to be full-blown papers! The focus should be on the definition of your model, how this model relates to theory (introduction), and what you have learned from your estimated model (discussion). For each of the assignments, you should be able to get the job done in fewer than 10 pages of text (excluding title page, figures, appendices, and references). Submission You will submit your reports through Blackboard. Each assignment has a corresponding item in the “Assignments” section of the BB page through which you will submit your reports. For Assignments 1 & 2, you may only submit one report per group. Designate one group member to submit the report. The grade for this submission will apply to all group members. If something goes wrong with the submission, or you notice a mistake (before the deadline) that you want to correct, you may upload a new version of your report. We will grade the final submitted version. The submissions will be screened with Ouriginal. "],["grading-1.html", "Grading", " Grading Group Assignments Assignments 1 & 2 are simply graded as pass/fail. To pass, your submission must: Do a reasonable job of addressing the relevant components listed above Be submitted before the deadline Otherwise, you will fail the assignment. Individual Assignment Assignment 3 will be fully graded on the usual 10-point scale. Points will be allocated according to the extent to which your submission addresses the six components listed above. The evaluation matrix gives an indication of how these points will be apportioned. Further details over the grading procedures for Assignment 3 (e.g., exactly how your 10-point grade will be defined) will be provided at a later date. Assuming your group passes the first two assignments, your final course grade will simply be your Assignment 3 grade. Resits You must get a “pass” for Assignments 1 & 2 and score at least 5.5 on Assignment 3 to pass the course. If you fail any of the assignments, you will have the opportunity to resit the failed assignment(s). If you resit Assignment 3, your revised graded cannot be higher than 6. Further details on the resit procedure will be provided at a later date. Example Assignment You can find an example of a good submission (for an older version of Assignment 2) here. This example is not perfect (no paper ever is), and several points could be improved. That being said, this submission exemplifies what we’re looking for in your project reports. So, following the spirit of this example would earn you a high grade. "],["rules.html", "Rules", " Rules Resources For all three assignments, you may use any reference materials you like, including: All course materials The course GitBook Additional books and papers The internet Collaboration You will complete the first two assignments in groups. Although you will work in groups, your group may not work together with other groups. You will complete the final assignment individually. For this assignment, you may not work with anyone else. For all three assignments, you are obligated to submit original work (i.e., work conducted for this course by you or your group). Submitting an assignment that violates this condition constitutes fraud. Such cases of fraud will be addressed according to the University’s standard policy. Academic integrity Hopefully, you also feel a moral obligation to obey the rules. For this course, we have implemented an examination that allows you to showcase what you have learned in a more realistic way than a written exam would allow. This assessment format spares you the stress of long exams (the two exams for this course used to be 4 hours each) and the attendant studying/cramming. The assignments will also help you assess your ability to independently analyse data, which is important to know for your future courses and/or career. However, this format also assumes that you complete the assignments in good faith. So, I simply ask that you hold up your end of the bargain, and submit your original work to show us what you’ve learned. Strict stuff By submitting your assignments (both group and individual), you confirm the following: You have completed the assignment yourself (or with your group) You are submitting work that you have written yourself (or with your group) You are using your own UU credentials to submit the assignment You have not had outside help that violates the conditions delineated above while completing the assignment All assignments will be submitted via Ouriginal in Blackboard and, thereby, checked for plagiarism. If fraud or plagiarism is detected or suspected, we will inform the Board of Examiners in the usual manner. In the event of demonstrable fraud, the sanctions delineated in Article 5.15 of the Education and Examination Regulations (EER) will apply. "],["software-setup.html", "Software Setup", " Software Setup This chapter will help you prepare for the course by showing how to install R and RStudio on your computer. If you’re already using R, there may be nothing new for you here. That being said, you should look over this chapter to ensure that your current setup will be compatible with the course requirements. If you have never used R before, this chapter is essential! The information is this chapter will be crucial for getting your computer ready for the course. "],["typographic-conventions.html", "Typographic Conventions", " Typographic Conventions Throughout this GitBook, we (try to) use a consistent set of typographic conventions: Functions are typeset in a code font, and the name of the function is always followed by parentheses E.g., sum(), mean() Other R objects (e.g., data objects, function arguments) are in also typeset in a code font but without parentheses E.g., seTE, method.tau Sometimes, we’ll use the package name followed by two colons (::, the so-called *scope-resolution operator), like lavaan::sem(). This command is valid R code and will run if you copy it into your R console. The lavaan:: part of the command tells R that we want to use the sem() from the lavaan package. "],["installing-software.html", "Installing software", " Installing software Before we start the course, we have to install three things: R: A free program for statistical programming RStudio: An integrated development environment (IDE) which makes it easier to work with R. Several packages: Separate pieces of ‘add-on’ software for R with functions to do specific analyses. Packages also include documentation describing how to use their functions and sample data. Installing R The latest version of R is available here. Click the appropriate link for your operating system and follow the instructions for installing the latest stable release. Depending on which OS you select, you may be given an option to install different components (e.g., base, contrib, Rtools). For this course, you will only need the base package. Installing RStudio Download the Free Desktop version of RStudio from the download page of the RStudio website. Installing packages To participate in this course, you will need a few essential R packages. Here’s an overview of the packages and why we need them: Package Description lavaan A sophisticated and user-friendly package for structural equation modeling dplyr A powerful suite of data-processing tools ggplot2 A flexible and user-friendly package for making graphs tidySEM Plotting and tabulating the output of SEM-models semTools Comparing models, establishing measurement invariance across groups psych Descriptive statistics and EFA rockchalk Probing interactions foreign Loading data from SPSS ‘.sav’ files readxl Loading data from Excel ‘.xslx’ files To install these packages, we use the install.packages() function in R. Open RStudio Inside RStudio, find the window named Console on left side of the screen. Copy the following code into the console and hit Enter/Return to run the command. install.packages(c("lavaan", "dplyr", "ggplot2", "tidySEM", "semTools", "psych", "rockchalk", "foreign", "readxl"), dependencies = TRUE) "],["course-data.html", "Course Data", " Course Data All of the data files you will need for the course are available in this SurfDrive directory. Follow the link to download a ZIP archive containing the data you will need to complete the practical exercises and assignments. Extract these data files to a convenient location on your computer. "],["note-on-data-updates.html", "Note on Data Updates", " Note on Data Updates During the course, we may need to update some of these datasets and/or add some new datasets to the SurfDrive directory. If so, you will need to download the updated data. We will let you know if and when any datasets are modified. In such situations, you are responsible for updating your data. Working with outdated data will probably produce incorrect results. Your answer won’t match the solutions we expect. Your answer will be marked as incorrect, even if the code used to produce the answer is correct. Points lost on an assignment due to using outdated datasets will not be returned. "],["introduction-to-r.html", "1 Introduction to R", " 1 Introduction to R This week is all about getting up-and-running with R and RStudio. Homework before the lecture Complete the preparatory material: Read over the Course Information chapter Work through the Software Setup chapter Watch the Lecture Recording for this week. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture.html", "1.1 Lecture", " 1.1 Lecture This week, you will learn the basics of R and RStudio. Rather than re-inventing the proverbial wheel, we’re linked to existing resources developed by R-Ladies Sydney. 1.1.1 Recordings Tour of RStudio \\[\\\\[6pt]\\] R Packages \\[\\\\[6pt]\\] Data I/0 1.1.2 Slides You can access the accompanying resources on the R-Ladies Sydney website here. "],["reading.html", "1.2 Reading", " 1.2 Reading There is no official reading this week. If you’d like to deepen your dive into R, feel free to check out Hadley Wickham’s excellent book R for Data Science. Otherwise, you may want to get a jump-start on the At-Home Exercises for this week. \\[\\\\[12pt]\\] "],["at-home-exercises.html", "1.3 At-Home Exercises", " 1.3 At-Home Exercises This week is all about gaining familiarity with R and RStudio. We’ll be using the primers available on Posit Cloud to work through some basic elements of data visualization and statistical programming in R. Although you should already have R working, this week’s at-home and in-class exercises don’t require that you have R installed on your system. If following along within this GitBook doesn’t work for you, you can also find the tutorials online on the Posit Primers page. 1.3.1 Visualizations with R 1.3.2 Programming with R End of At-Home Exercises "],["in-class-exercises.html", "1.4 In-Class Exercises", " 1.4 In-Class Exercises In the practical this week, we’ll go a little further into what it’s possible with R. Don’t worry if you cannot remember everything in these primers—they’re only meant to familiarize you with what is possible and to get you some experience interacting with R and RStudio. The following primers come from Posit Cloud and were created with the learnr package. 1.4.1 Viewing Data This first primer introduces a special data format called a tibble, as well as some functions for viewing your data. 1.4.2 Dissecting Data In the next primer, we’ll explore tools to subset and rearrange you data: select(), filter(), and arrange(). 1.4.3 Grouping and Manipulating Data Advanced If you made it through the previous two sections with ease and want to challenge yourself, go ahead with this next section. If you’re running short on time, you can skip ahead to Exploratory Data Analysis. \\[\\\\[3pt]\\] 1.4.4 Exploratory Data Analysis 1.4.5 Visualizing Data Visualizing data is a great way to start understanding a data set. In this section, we’ll highlight a few examples of how you can use the ggplot2 libarary to visualize your data. Primers on many other visualizations are available on Posit Cloud. Bar Charts for Categorical Variables Scatterplots for Continuous Variables 1.4.6 Tidying Data This primer will provide an overview of what’s meant by “tidy data”. You only need to complete the Tidy Data section—the sections on Gathering and Spreading columns are useful, but we won’t ask you to apply those techniques in this course. Recap Hopefully, you now feel more comfortable using some of R’s basic functionality and packages to work with data. Here’s a brief description of the functions covered above: install.packages() for installing packages Remember to put the package names in quotes library() for loading packages View() for viewing your dataset select() for picking only certain columns filter() for picking only certain rows arrange() for changing the rows order %>% aka “the pipe” for chaining commands together In RStudio, you can hit ctrl+shift+m as a handy key combination ? for help files Logical tests and Boolean operators == equal to != not equal to < less than <= less than or equal to > greater than >= greater than or equal to is.na() is the value NA (not available) !is.na is the value not NA & and (true only if the left and right are both true) | or (true if either the left or right are true) ! not (invert true/false) %in% in (is left in the larger set of right values) any() any (true if any in the set are true) all() all (true if all in the set are true) xor() xor (true if one and only one of the set are true) ggplot2 ggplot() create the basic object from which to building a plot aes() contains the aesthetic mappings (like x and y) geom_bar() bar plots for distributions of categorical variables geom_point() scatterplots for plotting two continuous variables geom_label_repel() for plotting text facet_wrap() for creating sets of conditional plots End of In-Class Exercises "],["statistical-modeling-path-analysis.html", "2 Statistical Modeling & Path Analysis", " 2 Statistical Modeling & Path Analysis This week, we will cover statistical modeling and path analysis. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-1.html", "2.1 Lecture", " 2.1 Lecture In this lecture, we will begin by discussing the paradigm and contextualizing statistical modeling relative to other ways that we can conduct statistical analyses. We will conclude with an introduction to . 2.1.1 Recordings Statistical Reasoning Statistical Modeling Path Analysis 2.1.2 Slides You can download the lectures slides here "],["reading-1.html", "2.2 Reading", " 2.2 Reading Reference Smaldino, P. E. (2017). Models are stupid, and we need more of them. In R.R. Vallacher, S.J. Read, & A. Nowakt (Eds.), Computational Social Psychology (pp. 311–331). New York: Routledge. SKIP PAGES 322 - 327 Questions What are the differences between a “verbal model” and a “formal model”? As explained in the paragraph “A Brief Note on Statistical Models”, formal models are not the same as statistical models. Still, we can learn a lot from Smaldino’s approach. Write down three insights from this paper that you would like to apply to your statistical modeling during this course. Suggested Reading (Optional) The following paper is not required, but it’s definitely worth a read. Breiman provides a very interesting perspective on different ways to approach a modeling-based analysis. Breiman, L. (2001). Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statistical Science, 16(3) 199–231. https://doi.org/10.1214/ss/1009213726 "],["at-home-exercises-1.html", "2.3 At-Home Exercises", " 2.3 At-Home Exercises Load the LifeSat.sav data. library(dplyr) library(haven) LifeSat <- read_spss("LifeSat.sav") 2.3.1 Make a table of descriptive statistics for the variables: LifSat, educ, ChildSup, SpouSup, and age. What is the average age in the sample? What is the range (youngest and oldest child)? Hint: Use the tidySEM::descriptives() function.` Click for explanation The package tidySEM contains the descriptives() function for computing descriptive statistics. The describe() function in the psych package is a good alternative. library(tidySEM) descriptives(LifeSat[ , c("LifSat", "educ", "ChildSup", "SpouSup", "age")]) 2.3.2 Run a simple linear regression with LifSat as the dependent variable and educ as the independent variable. Hints: The lm() function (short for linear model) does linear regression. The summary() function provides relevant summary statistics for the model. It can be helpful to store the results of your analysis in an object. Click for explanation results <- lm(LifSat ~ educ, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ educ, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -43.781 -11.866 2.018 12.418 43.018 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 35.184 7.874 4.469 2.15e-05 *** ## educ 3.466 1.173 2.956 0.00392 ** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 17.64 on 96 degrees of freedom ## Multiple R-squared: 0.08344, Adjusted R-squared: 0.0739 ## F-statistic: 8.74 on 1 and 96 DF, p-value: 0.003918 2.3.3 Repeat the analysis from 2.3.2 with age as the independent variable. Click for explanation results <- lm(LifSat ~ age, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ age, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -35.321 -14.184 3.192 13.593 40.626 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 200.2302 52.1385 3.840 0.00022 *** ## age -2.0265 0.7417 -2.732 0.00749 ** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 17.75 on 96 degrees of freedom ## Multiple R-squared: 0.07215, Adjusted R-squared: 0.06249 ## F-statistic: 7.465 on 1 and 96 DF, p-value: 0.007487 2.3.4 Repeat the analysis from 2.3.2 and 2.3.3 with ChildSup as the independent variable. Click for explanation results <- lm(LifSat ~ ChildSup, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ ChildSup, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -37.32 -12.14 0.66 12.41 44.68 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 37.559 8.342 4.502 1.89e-05 *** ## ChildSup 2.960 1.188 2.492 0.0144 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 17.86 on 96 degrees of freedom ## Multiple R-squared: 0.06076, Adjusted R-squared: 0.05098 ## F-statistic: 6.211 on 1 and 96 DF, p-value: 0.01441 2.3.5 Run a multiple linear regression with LifSat as the dependent variable and educ, age, and ChildSup as the independent variables. Hint: You can use the + sign to add multiple variables to the RHS of your model formula. Click for explanation results <- lm(LifSat ~ educ + age + ChildSup, data = LifeSat) summary(results) ## ## Call: ## lm(formula = LifSat ~ educ + age + ChildSup, data = LifeSat) ## ## Residuals: ## Min 1Q Median 3Q Max ## -32.98 -12.56 2.68 11.03 41.91 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 134.9801 53.2798 2.533 0.0130 * ## educ 2.8171 1.1436 2.463 0.0156 * ## age -1.5952 0.7188 -2.219 0.0289 * ## ChildSup 2.4092 1.1361 2.121 0.0366 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 16.92 on 94 degrees of freedom ## Multiple R-squared: 0.1741, Adjusted R-squared: 0.1477 ## F-statistic: 6.603 on 3 and 94 DF, p-value: 0.0004254 2.3.6 Compare the results from 2.3.5 with those from 2.3.2, 2.3.3, and 2.3.4. What do you notice when you compare the estimated slopes for each of the three predictors in the multiple regression model with the corresponding estimates from the simple regression models? "],["in-class-exercises-1.html", "2.4 In-Class Exercises", " 2.4 In-Class Exercises During this practical, you will work through some exercises meant to expand your statistical reasoning skills and improve your understanding of linear models. For this exercise, having some familiarity with regression will be helpful. If you feel like you need to refresh your knowledge in this area, consider the resources listed in the Background knowledge section. Data: You will use the following dataset for these exercises. Sesam.sav 2.4.1 Data Exploration Open the file “Sesam.sav” # Load `dplyr` for data processing: library(dplyr) # Load the `haven` library for reading in SPSS files: library(haven) ## Load the 'Sesam.sav' data ## Use haven::zap_formats() to remove SPSS attributes sesam <- read_sav(file = "Sesam.sav") %>% zap_formats() This file is part of a larger dataset that evaluates the impact of the first year of the Sesame Street television series. Sesame Street is mainly concerned with teaching preschool related skills to children in the 3–5 year age range. The following variables will be used in this exercise: age: measured in months prelet: knowledge of letters before watching Sesame Street (range 0–58) prenumb: knowledge of numbers before watching Sesame Street (range 0–54) prerelat: knowledge of size/amount/position relationships before watching Sesame Street (range 0–17) peabody: vocabulary maturity before watching Sesame Street (range 20–120) postnumb: knowledge of numbers after a year of Sesame Street (range 0–54) Note: Unless stated otherwise, the following questions refer to the sesam data and the above variables. 2.4.1.1 What is the type of each variable? Hint: The output of the str() function should be helpful here. Click to show code ## Examine the data structure: str(sesam) ## tibble [240 × 8] (S3: tbl_df/tbl/data.frame) ## $ id : num [1:240] 1 2 3 4 5 6 7 8 9 10 ... ## $ age : num [1:240] 66 67 56 49 69 54 47 51 69 53 ... ## $ prelet : num [1:240] 23 26 14 11 47 26 12 48 44 38 ... ## $ prenumb : num [1:240] 40 39 9 14 51 33 13 52 42 31 ... ## $ prerelat: num [1:240] 14 16 9 9 17 14 11 15 15 10 ... ## $ peabody : num [1:240] 62 80 32 27 71 32 28 38 49 32 ... ## $ postnumb: num [1:240] 44 39 40 19 54 39 44 51 48 52 ... ## $ gain : num [1:240] 4 0 31 5 3 6 31 -1 6 21 ... ## ..- attr(*, "display_width")= int 10 Click for explanation All variables are numeric. str() uses the abbreviation “num” to indicate a numeric vector. 2.4.1.2 What is the average age in the sample? What is the age range (youngest and oldest child)? Hint: Use tidySEM::descriptives() Click to show code As in the take home exercises, you can use the descriptives() function from the tidySEM package to describe the data: library(tidySEM) descriptives(sesam) Click for explanation We can get the average age from the “mean” column in the table ( 51.5), and the age range from the columns “min” and “max”, (34 and 69 respectively.) 2.4.1.3 What is the average gain in knowledge of numbers? What is the standard deviation of this gain? Hints: You will need to compute the gain and save the change score as a new object. You can then use the base-R functions mean() and sd() to do the calculations. Click to show code Create a new variable that represents the difference between pre- and post-test scores on knowledge of numbers: sesam <- mutate(sesam, ndif = postnumb - prenumb) Compute the mean and SD of the change score: sesam %>% summarise(mean(ndif), sd(ndif)) 2.4.1.4 Create an appropriate visualization of the gain scores you computed in 2.4.1.3. Justify your choice of visualization. Hint: Some applicable visualizations are explained in the Visualizations with R section. Click to show code library(ggplot2) ## Create an empty baseline plot object: p <- ggplot(sesam, aes(x = ndif)) ## Add some appropriate geoms: p + geom_histogram() p + geom_density() p + geom_boxplot() Click for explanation Because the gain score is numeric, we should use something appropriate for showing the distribution of a continuous variable. In this case, we can use either a density plot, or a histogram (remember from the lecture, this is like a density plot, but binned). We can also use a box plot, which can be a concise way to display a lot of information about a variable in a little less space. 2.4.1.5 Create a visualization that provides information about the bivariate relationship between the pre- and post-test number knowledge. Justify your choice of visualization. Describe the relationship based on what you see in your visualization. Hint: Again, the Visualizations with R section may provide some useful insights. Click to show code ## Create a scatterplot of the pre- and post-test number knowledge ggplot(sesam, aes(x = prenumb, y = postnumb)) + geom_point() Click for explanation A scatterplot is a good tool for showing patterns in the way that two continuous variables relate to each other. From it, we can quickly gather information about whether a relationship exists, its direction, its strength, how much variation there is, and whether or not a relationship might be non-linear. Based on this scatterplot, we see a positive relationship between the prior knowledge of numbers and the knowledge of numbers at the end of the study. Children who started with a higher level of numeracy also ended with a higher level of numeracy. There is a considerable amount of variance in the relationship. Not every child increases their numeracy between pre-test and post-test. Children show differing amounts of increase. 2.4.2 Linear Modeling 2.4.2.1 Are there significant, bivariate associations between postnumb and the following variables? age prelet prenumb prerelat peabody Use Pearson correlations to answer this question. You do not need to check the assumptions here (though you would in real life). Hint: The base-R cor.test() function and the corr.test() function from the psych package will both conduct hypothesis tests for a correlation coefficients (the base-R cor() function only computes the coefficients). Click to show code library(psych) ## Test the correlations using psych::corr.test(): sesam %>% select(postnumb, age, prelet, prenumb, prerelat, peabody) %>% corr.test() ## Call:corr.test(x = .) ## Correlation matrix ## postnumb age prelet prenumb prerelat peabody ## postnumb 1.00 0.34 0.50 0.68 0.54 0.52 ## age 0.34 1.00 0.33 0.43 0.44 0.29 ## prelet 0.50 0.33 1.00 0.72 0.47 0.40 ## prenumb 0.68 0.43 0.72 1.00 0.72 0.61 ## prerelat 0.54 0.44 0.47 0.72 1.00 0.56 ## peabody 0.52 0.29 0.40 0.61 0.56 1.00 ## Sample Size ## [1] 240 ## Probability values (Entries above the diagonal are adjusted for multiple tests.) ## postnumb age prelet prenumb prerelat peabody ## postnumb 0 0 0 0 0 0 ## age 0 0 0 0 0 0 ## prelet 0 0 0 0 0 0 ## prenumb 0 0 0 0 0 0 ## prerelat 0 0 0 0 0 0 ## peabody 0 0 0 0 0 0 ## ## To see confidence intervals of the correlations, print with the short=FALSE option ## OR ## library(magrittr) ## Test the correlations using multiple cor.test() calls: sesam %$% cor.test(postnumb, age) ## ## Pearson's product-moment correlation ## ## data: postnumb and age ## t = 5.5972, df = 238, p-value = 5.979e-08 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.2241066 0.4483253 ## sample estimates: ## cor ## 0.3410578 sesam %$% cor.test(postnumb, prelet) ## ## Pearson's product-moment correlation ## ## data: postnumb and prelet ## t = 8.9986, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.4029239 0.5926632 ## sample estimates: ## cor ## 0.5038464 sesam %$% cor.test(postnumb, prenumb) ## ## Pearson's product-moment correlation ## ## data: postnumb and prenumb ## t = 14.133, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.6002172 0.7389277 ## sample estimates: ## cor ## 0.6755051 sesam %$% cor.test(postnumb, prerelat) ## ## Pearson's product-moment correlation ## ## data: postnumb and prerelat ## t = 9.9857, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.4475469 0.6268773 ## sample estimates: ## cor ## 0.5433818 sesam %$% cor.test(postnumb, peabody) ## ## Pearson's product-moment correlation ## ## data: postnumb and peabody ## t = 9.395, df = 238, p-value < 2.2e-16 ## alternative hypothesis: true correlation is not equal to 0 ## 95 percent confidence interval: ## 0.4212427 0.6067923 ## sample estimates: ## cor ## 0.520128 Click for explanation Yes, based on the p-values (remember that 0 here really means very small, making it less than .05), we would say that there are significant correlations between postnumb and all other variables in the data. (In fact, all variables in the data are significantly correlated with one another.) 2.4.2.2 Do age and prenumb explain a significant proportion of the variance in postnumb? What statistic did you use to justify your conclusion? Interpret the model fit. Use the lm() function to fit your model. Click to show code lmOut <- lm(postnumb ~ age + prenumb, data = sesam) summary(lmOut) ## ## Call: ## lm(formula = postnumb ~ age + prenumb, data = sesam) ## ## Residuals: ## Min 1Q Median 3Q Max ## -38.130 -6.456 -0.456 5.435 22.568 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 7.4242 5.1854 1.432 0.154 ## age 0.1225 0.1084 1.131 0.259 ## prenumb 0.7809 0.0637 12.259 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 9.486 on 237 degrees of freedom ## Multiple R-squared: 0.4592, Adjusted R-squared: 0.4547 ## F-statistic: 100.6 on 2 and 237 DF, p-value: < 2.2e-16 Click for explanation Yes, age and prenumb explain a significant amount of variability in postnumb (\\(R^2 = 0.459\\), \\(F[2, 237] = 100.629\\), \\(p < 0.001\\)). We use the F statistic for the overall test of model fit to support this conclusion. The variables age and prenumb together explain 45.9% of the variability in postnumb. 2.4.2.3 Write the null and alternative hypotheses tested for in 2.4.2.2. Click for explanation Since we are testing for explained variance, our hypotheses concern the \\(R^2\\). \\[ \\begin{align*} H_0: R^2 = 0\\\\ H_1: R^2 > 0 \\end{align*} \\] Note that this is a directional hypotheses because the \\(R^2\\) cannot be negative. 2.4.2.4 Define the model syntax to estimate the model from 2.4.2.2 as a path analysis using lavaan. Click to show code mod <- 'postnumb ~ 1 + age + prenumb' 2.4.2.5 Estimate the path analytic model you defined above. Use the lavaan::sem() function to estimate the model. Click to show code library(lavaan) lavOut1 <- sem(mod, data = sesam) 2.4.2.6 Summarize the fitted model you estimated above. Use the summary() function to summarize the model. Click to show code summary(lavOut1) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 4 ## ## Number of observations 240 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## postnumb ~ ## age 0.123 0.108 1.138 0.255 ## prenumb 0.781 0.063 12.336 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 7.424 5.153 1.441 0.150 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 88.864 8.112 10.954 0.000 In OLS regression, the predictor variables are usually treated as fixed and do not covary. We can easily relax this assumption in path analysis. 2.4.2.7 Re-estimate the path analytic model you defined in 2.4.2.4. Specify the predictors as random, correlated variables. Hint: You can make the predictors random in, at least, two ways: Modify the model syntax to specify the correlation between age and prenumb. Add fixed.x = FALSE to your sem() call. Click to show code lavOut2 <- sem(mod, data = sesam, fixed.x = FALSE) ## OR ## mod <- ' postnumb ~ 1 + age + prenumb age ~~ prenumb ' lavOut2 <- sem(mod, data = sesam) 2.4.2.8 Summarize the fitted model you estimated above. Compare the results to those from the OLS regression in 2.4.2.2 and the path model in 2.4.2.5. Click to show code summary(lavOut2) ## lavaan 0.6.16 ended normally after 26 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 9 ## ## Number of observations 240 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## postnumb ~ ## age 0.123 0.108 1.138 0.255 ## prenumb 0.781 0.063 12.336 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## age ~~ ## prenumb 28.930 4.701 6.154 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 7.424 5.153 1.441 0.150 ## age 51.525 0.405 127.344 0.000 ## prenumb 20.896 0.688 30.359 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 88.864 8.112 10.954 0.000 ## age 39.291 3.587 10.954 0.000 ## prenumb 113.702 10.379 10.954 0.000 summary(lavOut1) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 4 ## ## Number of observations 240 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## postnumb ~ ## age 0.123 0.108 1.138 0.255 ## prenumb 0.781 0.063 12.336 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 7.424 5.153 1.441 0.150 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .postnumb 88.864 8.112 10.954 0.000 summary(lmOut) ## ## Call: ## lm(formula = postnumb ~ age + prenumb, data = sesam) ## ## Residuals: ## Min 1Q Median 3Q Max ## -38.130 -6.456 -0.456 5.435 22.568 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 7.4242 5.1854 1.432 0.154 ## age 0.1225 0.1084 1.131 0.259 ## prenumb 0.7809 0.0637 12.259 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 9.486 on 237 degrees of freedom ## Multiple R-squared: 0.4592, Adjusted R-squared: 0.4547 ## F-statistic: 100.6 on 2 and 237 DF, p-value: < 2.2e-16 2.4.2.9 Consider the path model below. How many regression coefficients are estimated in this model? How many variances are estimated? How many covariances are estimated? Click for explanation Six regression coefficients (red) Four (residual) variances (blue) No covariances 2.4.2.10 Consider a multiple regression analysis with three continuous independent variables: scores on tests of language, history, and logic, and one continuous dependent variable: score on a math test. We want to know if scores on the language, history, and logic tests can predict the math test score. Sketch a path model that you could use to answer this question How many regression parameters are there? How many variances could you estimate? How many covariances could you estimate? 2.4.3 Categorical IVs Load the Drivers.sav data. # Read the data into a data frame named 'drivers': drivers <- read_sav("Drivers.sav") %>% as_factor() # This preserves the SPSS labels for nominal variables In this section, we will evaluate the following research question: Does talking on the phone interfere with people's driving skills? These data come from an experiment. The condition variable represents the three experimental conditions: Hand-held phone Hands-free phone Control (no phone) We will use condition as the IV in our models. The DV, RT, represents the participant’s reaction time (in milliseconds) during a driving simulation. 2.4.3.1 Use the package ggplot2 to create a density plot for the variable RT. What concept are we representing with this plot? Hint: Consider the lap times example from the statistical modeling section of Lecture 2. Click to show code ggplot(drivers, aes(x = RT)) + geom_density() Click for explanation This shows the distribution of all the combined reaction times from drivers in all three categories. 2.4.3.2 Modify this density plot by mapping the variable condition from your data to the fill aesthetic in ggplot. What is the difference between this plot and the previous plot? Do you think there is evidence for differences between the groups? How might we test this by fitting a model to our sample? Click to show code Hint: To modify the transparency of the densities, use the aesthetic alpha. ggplot(drivers, aes(x = RT, fill = condition)) + geom_density(alpha = .5) Click for explanation This figure models the conditional distribution of reaction time, where the type of cell phone usage is the grouping factor. Things you can look at to visually assess whether the three groups differ are the amount of overlap of the distributions, how much distance there is between the individual means, and whether the combined distribution is much different than the conditional distributions. If we are willing to assume that these conditional distributions are normally distributed and have equivalent variances, we could use a linear model with dummy-coded predictors. Aside: ANOVA vs. Linear Regression As you may know, the mathematical model underlying ANOVA is just a linear regression model with nominal IVs. So, in terms of the underlying statistical models, there is no difference between ANOVA and regression; the differences lie in the focus of the analysis. ANOVA is really a type of statistical test wherein we are testing hypotheses about the effects of some set of nominal grouping factors on some continuous outcome. When doing an ANOVA, we usually don’t interact directly with the parameter estimates from the underlying model. Regression is a type of statistical model (i.e., a way to represent a univariate distribution with a conditional mean and fixed variance). When we do a regression analysis, we primarily focus on the estimated parameters of the underling linear model. When doing ANOVA in R, we estimate the model exactly as we would for linear regression; we simply summarize the results differently. If you want to summarize your model in terms of the sums of squares table you usually see when running an ANOVA, you can supply your fitted lm object to the anova() function. This is a statistical modeling course, not a statistical testing course, so we will not consider ANOVA any further. 2.4.3.3 Estimate a linear model that will answer the research question stated in the beginning of this section. Use lm() to estimate the model. Summarize the fitted model and use the results to answer the research question. Click to show code library(magrittr) lmOut <- drivers %>% mutate(condition = relevel(condition, ref = "control")) %$% lm(RT ~ condition) summary(lmOut) ## ## Call: ## lm(formula = RT ~ condition) ## ## Residuals: ## Min 1Q Median 3Q Max ## -317.50 -71.25 2.98 89.55 243.45 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 553.75 29.08 19.042 <2e-16 *** ## conditionhand-held 100.75 41.13 2.450 0.0174 * ## conditionhands-free 63.80 41.13 1.551 0.1264 ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 130.1 on 57 degrees of freedom ## Multiple R-squared: 0.09729, Adjusted R-squared: 0.06562 ## F-statistic: 3.072 on 2 and 57 DF, p-value: 0.05408 anova(lmOut) Click for explanation The effect of condition on RT is nonsignificant (\\(F[2, 57] = 3.07\\), \\(p = 0.054\\)). Therefore, based on these results, we do not have evidence for an effect of mobile phone usage on driving performance. 2.4.3.4 Use lavaan to estimate the model from 2.4.3.3 as a path model. Hint: lavaan won’t let us use factors for our categorical predictors. So, you will need to create your own dummy codes. Click to show code mod <- 'RT ~ 1 + HH + HF' lavOut <- drivers %>% mutate(HH = ifelse(condition == "hand-held", 1, 0), # Create dummy code for "hand-held" condition HF = ifelse(condition == "hands-free", 1, 0) # Create dummy code for "hands-free" condition ) %>% sem(mod, data = .) # Estimate the model summary(lavOut) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 4 ## ## Number of observations 60 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## RT ~ ## HH 100.750 40.085 2.513 0.012 ## HF 63.800 40.085 1.592 0.111 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .RT 553.750 28.344 19.537 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .RT 16068.028 2933.607 5.477 0.000 At this point, we haven’t covered the tools you need to conduct the ANOVA-style tests with path models. So, you can’t yet answer the research question with the above model. When we discuss model comparisons, you’ll get the missing tools. End of In-Class Exercises 2 "],["mediation-moderation.html", "3 Mediation & Moderation", " 3 Mediation & Moderation In this lecture, we will discuss two particular types of processes that we can model using path analysis: mediation and moderation. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-2.html", "3.1 Lecture", " 3.1 Lecture Researchers often have theories about possible causal processes linking multiple variables. Mediation is a particularly important example of such a process in which in an input variable, X, influences the outcome, Y, through an intermediary variable, M (the mediator). For instance, psychotherapy (X), may affect thoughts (M), which in turn affects mood (Y). We can investigate mediation via a specific sequence of linear regression equations, but path modeling will make our lives much easier. We can use path models to simultaneously estimate multiple related regression equations. So, mediation analysis is an ideal application of path modeling. In this lecture, we consider both approaches and discuss their relative strengths and weaknesses. As with mediation, researchers often posit theories involving moderation. Moderation implies that the effect of X on Y depends on another variable, Z. For instance, the effect of feedback (X) on performance (Y) may depend on age (Z). Older children might process feedback more effectively than younger children. Hence, the feedback is more effective for older children than for younger children, and the effect of feedback on performance is stronger for older children than for younger children. In such a case, we would say that age moderates the effect of feedback on performance. 3.1.1 Recordings Note: In the following recordings, the slide numbers are a bit of a mess, because I made these videos by cutting together recordings that used different slide decks. My apologies to those who are particularly distracted by continuity errors. Mediation Basics Mediation Testing Bootstrapping Moderation Basics Moderation Probing 3.1.2 Slides You can download the lecture slides here "],["reading-2.html", "3.2 Reading", " 3.2 Reading Reference Baron, R. M. & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical Considerations. Journal of Personality and Individual Differences, 51(6), 1173–1182 Questions What is mediation? Give an example of mediation. According to the authors, we must satisfy four criteria to infer mediation. What are these criteria? What is “moderation”, and how is it different from “mediation”? Give an example of moderation. What are the four methods given by Baron and Kenny as suitable ways to to study interaction effects? The authors suggest that one of the most common ways to address unreliability is to use multiple indicators. Thinking back to what you’ve learned about factor analysis, briefly explain why multiple indicators can improve reliability. How can you determine whether a variable is a mediator or moderator? Reference Hayes, A. F. (2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76(4), 408–420. Questions What is an indirect or mediated effect? What is the difference between the total and direct effect? What is the main problem with the Barron & Kenny “Causal Steps Approach”? What is bootstrapping, and why is it a better way to test mediation than Sobel’s test? Explain how it is possible that “effects that don’t exist can be mediated”. "],["at-home-exercises-2.html", "3.3 At-Home Exercises", " 3.3 At-Home Exercises 3.3.1 Mediation In the first part of this practical, we will analyze the data contained in SelfEsteem.sav. These data comprise 143 observations of the following variables.1 case: Participant ID number ParAtt: Parental Attachment PeerAtt: Peer Attachment Emp: Empathy ProSoc: Prosocial behavior Aggr: Aggression SelfEst: Self-esteem 3.3.1.1 Load the SelfEsteem.sav data. Note: Unless otherwise specified, all analyses in Section 3.3.1 apply to these data. Click to show code library(haven) seData <- read_sav("SelfEsteem.sav") Suppose we are interested in the (indirect) effect of peer attachment on self-esteem, and whether empathy has a mediating effect on this relationship. We might generate the following hypotheses: Better peer relationships promote higher self esteem This effect is mediated by a student’s empathy levels, where better peer relationships increase empathy, and higher levels of empathy lead to higher self-esteem. To evaluate these hypotheses, we will use lavaan to estimate a path model. 3.3.1.2 Draw a path model (on paper) that can be used to test the above hypotheses. Label the input (X), outcome (Y), and mediator/intermediary (M). Label the paths a, b, and c’. Hint: Refer back to the Mediation Basics lecture if you need help here. Click for explanation 3.3.1.3 Specify the lavaan model syntax implied by the path diagram shown above. Save the resulting character string as an object in your environment. Hint: Refer back to the example in which opinions of systematic racism mediate the relationship between political affiliation and support for affirmative action policies from the Mediation Testing lecture this week. Click to show code mod <- ' ## Equation for outcome: SelfEst ~ Emp + PeerAtt ## Equation for the mediator: Emp ~ PeerAtt ' 3.3.1.4 Use the lavaan::sem() function to estimate the model defined in 3.3.1.3. Use the default settings in sem(). Click to show code library(lavaan) out <- sem(mod, data = seData) 3.3.1.5 Explore the summary of the fitted model. Which numbers correspond to the a, b, and c’ paths? Interpret these paths. Do the direction of the effects seem to align with our hypothesis? Click to show code summary(out) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 5 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## Emp 0.234 0.091 2.568 0.010 ## PeerAtt 0.174 0.088 1.968 0.049 ## Emp ~ ## PeerAtt 0.349 0.076 4.628 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.934 0.110 8.456 0.000 ## .Emp 0.785 0.093 8.456 0.000 Click for explanation The results show estimates of the a path (Emp ~ PeerAtt), the b path (SelfEst ~ Emp), and the c’ path (SelfEst ~ PeerAtt). All three of these effects are positive and significant, including the direct effect of PeerAtt on SelfEst (\\(\\beta = 0.174\\), \\(Z = 1.97\\), \\(p = 0.025\\)), and the parts of the indirect effect made up by the effect of PeerAtt on Emp (\\(\\beta = 0.349\\), \\(Z = 4.63\\), \\(p = 0\\)), and Emp on SelfEst (\\(\\beta = 0.234\\), \\(Z = 2.57\\), \\(p = 0.005\\)). We can see that the direction of the effects seems to support of our hypotheses, but without taking the next steps to investigate the indirect effect, we should be hesitant to say more. Remember that an indirect effect (IE) is the product of multiple regression slopes. Therefore, to estimate an IE, we must define this product in our model syntax. In lavaan, we define the new IE parameter in two steps. Label the relevant regression paths. Use the labels to define a new parameter that represent the desired IE. We can define new parameters in lavaan model syntax via the := operator. The lavaan website contains a tutorial on this procedure: http://lavaan.ugent.be/tutorial/mediation.html 3.3.1.6 Use the procedure described above to modify the model syntax from 3.3.1.3 by adding the definition of the hypothesized IE from PeerAtt to SelfEst. Click to show code mod <- ' ## Equation for outcome: SelfEst ~ b * Emp + PeerAtt ## Equation for mediator: Emp ~ a * PeerAtt ## Indirect effect: ie := a * b ' Click for explanation Notice that I only label the parameters that I will use to define the IE. You are free to label any parameter that you like, but I choose the to label only the minimally sufficient set to avoid cluttering the code/output. 3.3.1.7 Use lavaan::sem() to estimate the model with the IEs defined. Use the default settings for sem(). Is the hypothesized IE significant according to the default tests? Hint: Refer to the Mediation Testing lecture Click to show code out <- sem(mod, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 5 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## Emp (b) 0.234 0.091 2.568 0.010 ## PeerAtt 0.174 0.088 1.968 0.049 ## Emp ~ ## PeerAtt (a) 0.349 0.076 4.628 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.934 0.110 8.456 0.000 ## .Emp 0.785 0.093 8.456 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie 0.082 0.036 2.245 0.025 Click for explanation The IE of Peer Attachment on Self Esteem through Empathy is statistically significant (\\(\\hat{\\textit{IE}} = 0.082\\), \\(Z = 2.25\\), \\(p = 0.012\\)). Note: The p-value above doesn’t match the output because we’re testing a directional hypothesis, but lavaan conducts two-tailed tests for the model parameters. As we learned in the lecture, the above test of the indirect effect is equivalent to Sobel’s Z test (which we don’t really want). An appropriate, robust test of the indirect effect requires bootstrapping, which we will do later this week as part of the in-class exercises. For now, we’ll add another input variable to our model: parental attachment. We will use this model to evaluate the following research questions: Is there a direct effect of parental attachment on self-esteem, after controlling for peer attachment and empathy? Is there a direct effect of peer attachment on self-esteem, after controlling for parental attachment and empathy? Is the effect of parental attachment on self-esteem mediated by empathy, after controlling for peer attachment? Is the effect of peer attachment on self-esteem mediated by empathy, after controlling for parental attachment? 3.3.1.8 Run the path model needed to test the research questions listed above. Specify the lavaan model syntax implied by the research questions. Allow peer attachment and parental attachment to covary. Define two new parameters to represent the hypothesized indirect effects. Estimate the model using lavaan::sem(). Use the default settings in sem(). Investigate the model summary. Click to show code mod <- ' ## Equation for outcome: SelfEst ~ b * Emp + ParAtt + PeerAtt ## Equation for mediator: Emp ~ a1 * ParAtt + a2 * PeerAtt ## Covariance: ParAtt ~~ PeerAtt ie_ParAtt := a1 * b ie_PeerAtt := a2 * b ' out <- sem(mod, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 10 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## Emp (b) 0.206 0.088 2.357 0.018 ## ParAtt 0.287 0.078 3.650 0.000 ## PeerAtt 0.024 0.094 0.252 0.801 ## Emp ~ ## ParAtt (a1) 0.078 0.075 1.045 0.296 ## PeerAtt (a2) 0.306 0.086 3.557 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## ParAtt ~~ ## PeerAtt 0.537 0.103 5.215 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.854 0.101 8.456 0.000 ## .Emp 0.779 0.092 8.456 0.000 ## ParAtt 1.277 0.151 8.456 0.000 ## PeerAtt 0.963 0.114 8.456 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie_ParAtt 0.016 0.017 0.956 0.339 ## ie_PeerAtt 0.063 0.032 1.965 0.049 3.3.1.9 What can we say about the two indirect effects? Can we say that empathy mediates both paths? Click to show explanation According to the Sobel-style test, after controlling for parental attachment, the indirect effect of peer attachment on self-esteem was statistically significant (\\(\\hat{IE} = 0.063\\), \\(Z = 1.96\\), \\(p = 0.049\\)), as was the analogous direct effect (\\(\\hat{\\beta} = 0.306\\), \\(Z = 3.56\\), \\(p < 0.001\\)). After controlling for peer attachment, neither the indirect effect (\\(\\hat{IE} = 0.016\\), \\(Z = 0.96\\), \\(p = 0.339\\)) nor the direct effect (\\(\\hat{\\beta} = 0.078\\), \\(Z = 1.05\\), \\(p = 0.296\\)) of parental attachment on self-esteem was significant, though. 3.3.2 Moderation Remember that moderation attempts to describe when one variable influences another. For the home exercise, we’ll go back to the Sesame Street data we worked with for the in-class exercises last week. 3.3.2.1 Load the Sesam2.sav data.2 NOTE: Unless otherwise specified, all analyses in Section 3.3.2 use these data. Click to show code # Read the data into an object called 'sesam2': sesam2 <- read_sav("Sesam2.sav") VIEWCAT is a nominal grouping variable, but it is represented as a numeric variable in the sesam2 data. The levels represent the following frequencies of Sesame Street viewership of the children in the data: VIEWCAT = 1: Rarely/Never VIEWCAT = 2: 2–3 times a week VIEWCAT = 3: 4–5 times a week VIEWCAT = 4: > 5 times a week 3.3.2.2 Convert VIEWCAT into a factor. Make sure that VIEWCAT = 1 is the reference group. Hints: You can identify the reference group with the levels() or contrasts() functions. The reference group is the group labelled with the first level printed by levels(). When you run contrasts(), you will see a pattern matrix that defines a certain dummy coding scheme. The reference group is the group that has zeros in each column of this matrix. If you need to change the reference group, you can use the relevel() function. Click to show code library(forcats) ## Convert 'VIEWCAT' to a factor: sesam2 <- sesam2 %>% mutate(VIEWCAT = factor(VIEWCAT)) ## Optionally specify the labels # sesam2 <- # sesam2 %>% # mutate(VIEWCAT = factor(VIEWCAT, # levels = c(1, 2, 3, 4), # labels = c("Rarely/never", # "2-3 times per week", # "4-5 times per week", # "> 5 times per week"))) ## Check the reference group: levels(sesam2$VIEWCAT) ## [1] "1" "2" "3" "4" contrasts(sesam2$VIEWCAT) ## 2 3 4 ## 1 0 0 0 ## 2 1 0 0 ## 3 0 1 0 ## 4 0 0 1 ## If necessary, relevel # sesam <- # sesam2 %>% # mutate(VIEWCAT = relevel(VIEWCAT, 1)) 3.3.2.3 Use lm() to estimate a multiple regression model wherein VIEWCAT predicts POSTNUMB. Summarize the model. Interpret the estimates. Click to show code lmOut <- lm(POSTNUMB ~ VIEWCAT, data = sesam2) summary(lmOut) ## ## Call: ## lm(formula = POSTNUMB ~ VIEWCAT, data = sesam2) ## ## Residuals: ## Min 1Q Median 3Q Max ## -25.474 -7.942 0.240 8.526 25.240 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 18.760 2.316 8.102 8.95e-14 *** ## VIEWCAT2 9.331 2.900 3.218 0.00154 ** ## VIEWCAT3 14.714 2.777 5.298 3.49e-07 *** ## VIEWCAT4 18.032 2.809 6.419 1.24e-09 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 11.58 on 175 degrees of freedom ## Multiple R-squared: 0.2102, Adjusted R-squared: 0.1967 ## F-statistic: 15.53 on 3 and 175 DF, p-value: 5.337e-09 Click for explanation Viewing category explains a statistically significant proportion of the variance in the post-test score of numbers learned (\\(R^2 = 0.21\\), \\(F(3, 175) = 15.53\\), \\(p < 0.001\\)). Kids who never or rarely watched Sesame Street had an average score of 18.76 on the post-test. Kids with weekly viewing habits of 2–3, 4–5, or 5+ times per week all had significantly higher scores on the post-test than kids who never or rarely watched Sesame Street (2–3: \\(\\hat{\\beta} = 9.33\\), \\(t = 3.22\\), \\(p = 0.002\\); 4–5: \\(\\hat{\\beta} = 14.71\\), \\(t = 5.3\\), \\(p < 0.001\\); 5+: \\(\\hat{\\beta} = 18.03\\), \\(t = 6.42\\), \\(p < 0.001\\)). If we compare the box plot, kernel density plot, and model output below, the relationships between the regression coefficient estimates for the viewing categories and the group means should be evident. 3.3.2.4 Use ggplot() to make a scatterplot with AGE on the x-axis and POSTNUMB on the y-axis. Color the points according to the their VIEWCAT level. Save the plot object to a variable in your environment. Hint: You can map color to the levels of a variable on your dataset by assigning the variable names to the color argument of the aes() function in ggplot(). Click to show code library(ggplot2) ## Add aes(..., color = VIEWCAT) to get different colors for each group: p <- ggplot(sesam2, aes(x = AGE, y = POSTNUMB, color = VIEWCAT)) + geom_point() # Add points for scatterplot ## Print the plot stored as 'p': p We assigned the global color aesthetic to the VIEWCAT variable, so the points are colored based on their group. 3.3.2.5 Add linear regression lines for each group to the above scatterplot. Hints: You can add regression lines with ggplot2::geom_smooth() To get linear regression lines, set the argument method = \"lm\" To omit error envelopes, set the argument se = FALSE Click to show code ## Add OLS best-fit lines: p + geom_smooth(method = "lm", se = FALSE) The global color aesthetic assignment from above carries through to any additional plot elements that we add, including the regression lines. So, we also get a separate regression line for each VIEWCAT group. 3.3.2.6 How would you interpret the pattern of regression lines above? Click for explanation All the lines show a positive slope, so post-test number recognition appears to increase along with increasing age. The lines are not parallel, though. So VIEWCAT may be moderating the effect of AGE on POSTNUMB. Based on the figure we just created, we may want to test for moderation in our regression model. To do so, we need to add an interaction between AGE and VIEWCAT. The VIEWCAT factor is represented by 3 dummy codes in our model, though. So when we interact AGE and VIEWCAT, we will create 3 interaction terms. To test the overall moderating influence of VIEWCAT, we need to conduct a multiparameter hypothesis test of all 3 interaction terms. One way that we can go about implementing such a test is through a hierarchical regression analysis entailing three steps: Estimate the additive model wherein we regress POSTNUMB onto AGE and VIEWCAT without any interaction. Estimate the moderated model by adding the interaction between AGE and VIEWCAT into the additive model. Conduct a \\(\\Delta R^2\\) test to compare the fit of the two models. 3.3.2.7 Conduct the hierarchical regression analysis described above. Does VIEWCAT significantly moderate the effect of AGE on POSTNUMB? Provide statistical justification for your conclusion. Click to show code ## Estimate the additive model a view the results: results_add <- lm(POSTNUMB ~ VIEWCAT + AGE, data = sesam2) summary(results_add) ## ## Call: ## lm(formula = POSTNUMB ~ VIEWCAT + AGE, data = sesam2) ## ## Residuals: ## Min 1Q Median 3Q Max ## -23.680 -8.003 -0.070 8.464 22.635 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -10.1056 6.5091 -1.553 0.12235 ## VIEWCAT2 9.1453 2.7390 3.339 0.00103 ** ## VIEWCAT3 13.8602 2.6294 5.271 3.98e-07 *** ## VIEWCAT4 16.9215 2.6636 6.353 1.79e-09 *** ## AGE 0.5750 0.1221 4.708 5.08e-06 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 10.94 on 174 degrees of freedom ## Multiple R-squared: 0.2995, Adjusted R-squared: 0.2834 ## F-statistic: 18.6 on 4 and 174 DF, p-value: 9.642e-13 ## Estimate the moderated model and view the results: results_mod <- lm(POSTNUMB ~ VIEWCAT * AGE, data = sesam2) summary(results_mod) ## ## Call: ## lm(formula = POSTNUMB ~ VIEWCAT * AGE, data = sesam2) ## ## Residuals: ## Min 1Q Median 3Q Max ## -23.8371 -8.2387 0.6158 8.7988 22.5611 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -18.7211 15.5883 -1.201 0.2314 ## VIEWCAT2 9.9741 20.6227 0.484 0.6293 ## VIEWCAT3 23.5825 19.3591 1.218 0.2248 ## VIEWCAT4 34.3969 19.3600 1.777 0.0774 . ## AGE 0.7466 0.3074 2.429 0.0162 * ## VIEWCAT2:AGE -0.0175 0.4060 -0.043 0.9657 ## VIEWCAT3:AGE -0.1930 0.3782 -0.510 0.6104 ## VIEWCAT4:AGE -0.3416 0.3770 -0.906 0.3663 ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 10.99 on 171 degrees of freedom ## Multiple R-squared: 0.3046, Adjusted R-squared: 0.2762 ## F-statistic: 10.7 on 7 and 171 DF, p-value: 3.79e-11 ## Test for moderation: anova(results_add, results_mod) Click for explanation VIEWCAT does not significantly moderate the effect of AGE on POSTNUMB (\\(F[3, 171] = 0.422\\), \\(p = 0.738\\)). 3.3.2.8 Sketch the analytic path diagrams for the additive and moderated models you estimated in 3.3.2.7 (on paper). Click for explanation Additive Model Moderated Model End of At-Home Exercises 3 These data were simulated from the covariance matrix provided in Laible, D. J., Carlo, G., & Roesch, S. C. (2004). Pathways to self-esteem in late adolescence: The role of parent and peer attachment, empathy, and social behaviours. Journal of adolescence, 27(6), 703-716.↩︎ These data are from the very interesting study: Ball, S., & Bogatz, G. A. (1970). A Summary of the Major Findings in” The First Year of Sesame Street: An Evaluation”.↩︎ "],["in-class-exercises-2.html", "3.4 In-Class Exercises", " 3.4 In-Class Exercises 3.4.1 Mediation In this practical, we’ll go back to the data from the at-home exercises, SelfEsteem.sav. Recall that these data comprise 143 observations of the following variables. case: Participant ID number ParAtt: Parental Attachment PeerAtt: Peer Attachment Emp: Empathy ProSoc: Prosocial behavior Aggr: Aggression SelfEst: Self-esteem When we last worked with the data, we built a model with one mediator (Emp), creating indirect effects between our predictors ParAtt and PeerAtt, and our outcome variable SelfEst. Below, you will estimate a more complex, multiple-mediator model. 3.4.1.1 Load the data into the object seData using haven::read_sav() Click to show code library(haven) seData <- read_sav("SelfEsteem.sav") For this analysis, we are interested in the (indirect) effects of parental and peer attachment on self-esteem. Furthermore, we want to evaluate the mediating roles of empathy and social behavior (i.e., prosocial behavior and aggression). Specifically, we have the following hypotheses. Better peer relationships will promote higher self-esteem via a three-step indirect process. Better peer relationships will increase empathy levels. Higher empathy will increase prosocial behavior and decrease aggressive behavior. More prosocial behaviors and less aggressive behavior will both produce higher self-esteem. Better relationships with parents directly increase self-esteem. To evaluate these hypotheses, we will use lavaan to estimate the following multiple mediator model as a path model. 3.4.1.2 Specify the lavaan model syntax implied by the path diagram shown above. Save the resulting character string as an object in your environment. Click to show code mod0 <- ' ## Equation for outcome: SelfEst ~ ProSoc + Aggr + Emp + ParAtt + PeerAtt ## Equations for stage 2 mediators: ProSoc ~ PeerAtt + ParAtt + Emp Aggr ~ PeerAtt + ParAtt + Emp ## Equation for stage 1 mediator: Emp ~ ParAtt + PeerAtt ## Covariances: ProSoc ~~ Aggr ParAtt ~~ PeerAtt ' 3.4.1.3 Use the lavaan::sem() function to estimate the model defined in 3.4.1.2. Use the default settings in sem(). Summarize the fitted model. Click to show code library(lavaan) out <- sem(mod0, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## ProSoc 0.252 0.096 2.634 0.008 ## Aggr 0.185 0.085 2.172 0.030 ## Emp 0.143 0.098 1.460 0.144 ## ParAtt 0.244 0.078 3.133 0.002 ## PeerAtt 0.051 0.091 0.555 0.579 ## ProSoc ~ ## PeerAtt -0.037 0.080 -0.469 0.639 ## ParAtt 0.193 0.067 2.886 0.004 ## Emp 0.477 0.074 6.411 0.000 ## Aggr ~ ## PeerAtt -0.095 0.090 -1.055 0.291 ## ParAtt -0.034 0.075 -0.454 0.650 ## Emp -0.309 0.084 -3.697 0.000 ## Emp ~ ## ParAtt 0.078 0.075 1.045 0.296 ## PeerAtt 0.306 0.086 3.557 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .ProSoc ~~ ## .Aggr -0.086 0.058 -1.476 0.140 ## ParAtt ~~ ## PeerAtt 0.537 0.103 5.215 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.796 0.094 8.456 0.000 ## .ProSoc 0.618 0.073 8.456 0.000 ## .Aggr 0.777 0.092 8.456 0.000 ## .Emp 0.779 0.092 8.456 0.000 ## ParAtt 1.277 0.151 8.456 0.000 ## PeerAtt 0.963 0.114 8.456 0.000 3.4.1.4 Considering the parameter estimates from 3.4.1.3, what can you say about the hypotheses? Click for explanation Notice that all of the hypotheses stated above are explicitly directional. Hence, when evaluating the significance of the structural paths that speak to these hypotheses, we should use one-tailed tests. We cannot ask lavaan to return one-tailed p-values, but we have no need to do so. We can simply divide the two-tailed p-values in half. The significant direct effect of ParAtt on SelfEst (\\(\\beta = 0.244\\), \\(Z = 3.13\\), \\(p = 0.001\\)) and the lack of a significant direct effect of PeerAtt on SelfEst (\\(\\beta = 0.051\\), \\(Z = 0.555\\), \\(p = 0.29\\)) align with our hypotheses. The remaining patterns of individual estimates also seem to conform to the hypotheses (e.g., all of the individual paths comprising the indirect effects of PeerAtt on SelfEst are significant). We cannot make any firm conclusions until we actually estimate and test the indirect effects, though. 3.4.1.5 Modify the model syntax from 3.4.1.2 by adding definitions of the two hypothesized IEs from PeerAtt to SelfEst. Click to show code You can use any labeling scheme that makes sense to you, but I recommend adopting some kind of systematic rule. Here, I will label the individual estimates in terms of the short variable names used in the path diagram above. mod <- ' ## Equation for outcome: SelfEst ~ y_m21 * ProSoc + y_m22 * Aggr + Emp + ParAtt + PeerAtt ## Equations for stage 2 mediators: ProSoc ~ m21_x2 * PeerAtt + ParAtt + m21_m1 * Emp Aggr ~ m22_x2 * PeerAtt + ParAtt + m22_m1 * Emp ## Equation for stage 1 mediator: Emp ~ ParAtt + m1_x2 * PeerAtt ## Covariances: ProSoc ~~ Aggr ParAtt ~~ PeerAtt ## Indirect effects: ie_pro := m1_x2 * m21_m1 * y_m21 ie_agg := m1_x2 * m22_m1 * y_m22 ' 3.4.1.6 Use lavaan::sem() to estimate the model with the IEs defined. Use the default settings for sem(). Are the hypothesized IEs significant according to the default tests? Click to show code out <- sem(mod, data = seData) summary(out) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## ProSoc (y_21) 0.252 0.096 2.634 0.008 ## Aggr (y_22) 0.185 0.085 2.172 0.030 ## Emp 0.143 0.098 1.460 0.144 ## ParAtt 0.244 0.078 3.133 0.002 ## PerAtt 0.051 0.091 0.555 0.579 ## ProSoc ~ ## PerAtt (m21_2) -0.037 0.080 -0.469 0.639 ## ParAtt 0.193 0.067 2.886 0.004 ## Emp (m21_1) 0.477 0.074 6.411 0.000 ## Aggr ~ ## PerAtt (m22_2) -0.095 0.090 -1.055 0.291 ## ParAtt -0.034 0.075 -0.454 0.650 ## Emp (m22_1) -0.309 0.084 -3.697 0.000 ## Emp ~ ## ParAtt 0.078 0.075 1.045 0.296 ## PerAtt (m1_2) 0.306 0.086 3.557 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .ProSoc ~~ ## .Aggr -0.086 0.058 -1.476 0.140 ## ParAtt ~~ ## PeerAtt 0.537 0.103 5.215 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.796 0.094 8.456 0.000 ## .ProSoc 0.618 0.073 8.456 0.000 ## .Aggr 0.777 0.092 8.456 0.000 ## .Emp 0.779 0.092 8.456 0.000 ## ParAtt 1.277 0.151 8.456 0.000 ## PeerAtt 0.963 0.114 8.456 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie_pro 0.037 0.018 2.010 0.044 ## ie_agg -0.017 0.011 -1.657 0.098 Click for explanation The IE of Peer Attachment on Self Esteem through Empathy and Prosocial Behavior is significant (\\(\\hat{\\textit{IE}} = 0.037\\), \\(Z = 2.01\\), \\(p = 0.022\\)), as is the analogous IE through Aggressive Behavior (\\(\\hat{\\textit{IE}} = -0.017\\), \\(Z = -1.66\\), \\(p = 0.049\\)). Though, this latter effect is just barely significant at the \\(\\alpha = 0.05\\) level. The tests we used to evaluate the significance of the IEs in 3.4.1.6 are flawed because they assume normal sampling distributions for the IEs. However the IEs are defined as products of multiple, normally distributed, regression slopes. So the IEs themselves cannot be normally distributed (at least in finite samples), and the results of the normal-theory significance tests may be misleading. To get an accurate test of the IEs, we should use bootstrapping to generate an empirical sampling distribution for each IE. In lavaan, we implement bootstrapping by specifying the se = \"bootstrap\" option in the fitting function (i.e., the cfa() or sem() function) and specifying the number of bootstrap samples via the bootstrap option. Workflow Tip To draw reliable conclusions from bootstrapped results, we need many bootstrap samples (i.e., B > 1000), but we must estimate the full model for each of these samples, so the estimation can take a long time. To avoid too much frustration, you should first estimate the model without bootstrapping to make sure everything is specified correctly. Only after you are certain that your code is correct do you want to run the full bootstrapped version. 3.4.1.7 Re-estimate the model from 3.4.1.6 using 1000 bootstrap samples. Other than the se and bootstrap options, use the defaults. Are the hypothesized IEs significant according to the bootstrap-based test statistics? Click to show code ## Set a seed to get replicable bootstrap samples: set.seed(235711) ## Estimate the model with bootstrapping: out_boot <- sem(mod, data = seData, se = "bootstrap", bootstrap = 1000) ## Summarize the model: summary(out_boot) ## lavaan 0.6.16 ended normally after 16 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 143 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Bootstrap ## Number of requested bootstrap draws 1000 ## Number of successful bootstrap draws 1000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## SelfEst ~ ## ProSoc (y_21) 0.252 0.100 2.529 0.011 ## Aggr (y_22) 0.185 0.085 2.174 0.030 ## Emp 0.143 0.095 1.507 0.132 ## ParAtt 0.244 0.079 3.089 0.002 ## PerAtt 0.051 0.095 0.530 0.596 ## ProSoc ~ ## PerAtt (m21_2) -0.037 0.082 -0.456 0.648 ## ParAtt 0.193 0.068 2.831 0.005 ## Emp (m21_1) 0.477 0.078 6.092 0.000 ## Aggr ~ ## PerAtt (m22_2) -0.095 0.087 -1.093 0.275 ## ParAtt -0.034 0.076 -0.448 0.654 ## Emp (m22_1) -0.309 0.092 -3.356 0.001 ## Emp ~ ## ParAtt 0.078 0.072 1.092 0.275 ## PerAtt (m1_2) 0.306 0.079 3.896 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .ProSoc ~~ ## .Aggr -0.086 0.058 -1.493 0.135 ## ParAtt ~~ ## PeerAtt 0.537 0.128 4.195 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .SelfEst 0.796 0.082 9.698 0.000 ## .ProSoc 0.618 0.068 9.114 0.000 ## .Aggr 0.777 0.104 7.476 0.000 ## .Emp 0.779 0.090 8.651 0.000 ## ParAtt 1.277 0.197 6.473 0.000 ## PeerAtt 0.963 0.105 9.203 0.000 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ## ie_pro 0.037 0.019 1.891 0.059 ## ie_agg -0.017 0.011 -1.638 0.101 Click for explanation As with the normal-theory tests, the hypothesized IE of Peer Attachment on Self Esteem was significant (\\(\\hat{\\textit{IE}} = 0.037\\), \\(Z = 1.89\\), \\(p = 0.029\\)), but the IE of Aggressive Behavior has crossed into nonsignificant territory (\\(\\hat{\\textit{IE}} = -0.017\\), \\(Z = -1.64\\), \\(p = 0.051\\)). Note: Bootstrapping is a stochastic method, so each run can provide different results. Since the indirect effect of aggressive behavior is so close to the critical value, you may come to a different conclusions vis-á-vis statistical significance if you run this analysis with a different random number seed or a different number of bootstrap samples. When you use the summary() function to summarize the bootstrapped model from 3.4.1.7, the output will probably look pretty much the same as it did in 3.4.1.6, but it’s not. The standard errors and test statistics in the bootstrapped summary are derived from empirical sampling distributions, whereas these values are based on an assumed normal sampling distribution in 3.4.1.6. The standard method of testing IEs with bootstrapping is to compute confidence intervals (CIs) from the empirical sampling distribution of the IEs. In lavaan, we can compute basic (percentile, 95%) CIs by adding the ci = TRUE option to the summary() function. To evaluate our directional hypotheses at an \\(\\alpha = 0.05\\) level, however, we need to compute 90% CIs. We can get more control over the summary statistics (include the CIs) with the parameterEstimates() function. 3.4.1.8 Check the documentation for lavaan::parameterEstimates(). Click to show code ?parameterEstimates 3.4.1.9 Use the parameterEstimates() function to compute bootstrapped CIs for the hypothesized IEs. Compute percentile CIs. Are the IEs significant according to the bootstrapped CIs? Click to show code parameterEstimates(out_boot, ci = TRUE, level = 0.9) Click for explanation When evaluating a directional hypothesis with a CI, we only consider one of the interval’s boundaries. For a hypothesized positive effect, we check only if the lower boundary is greater than zero. For a hypothesized negative effect, we check if the upper boundary is less than zero. As with the previous tests, the IE of Peer Attachment on Self Esteem through Empathy and Prosocial Behavior is significant (\\(\\hat{\\textit{IE}} = 0.037\\), \\(95\\% ~ CI = [0.009; \\infty]\\)), but the analogous IE through Aggressive Behavior is not quite significant (\\(\\hat{\\textit{IE}} = -0.017\\), \\(95\\% ~ CI = [-\\infty; -0.003]\\)). 3.4.1.10 Based on the analyses you’ve conducted here, what do you conclude vis-à-vis the original hypotheses? Click for explanation When using normal-theory tests, both hypothesized indirect effects between Peer Attachment and Self Esteem were supported in that the IE through Empathy and Prosocial Behavior as well as the IE through Empathy and Aggressive Behavior were both significant. The hypothesized direct effect of Parent Attachment on Self Esteem was also born out via a significant direct effect in the model. When testing the indirect effects with bootstrapping, however, the effect through Aggressive Behavior was nonsignificant. Since bootstrapping gives a more accurate test of the indirect effect, we should probably trust these results more than the normal-theory results. We should not infer a significant indirect effect of Peer Attachment on Self Esteem transmitted through Empathy and Aggressive Behavior. These results may not tell the whole story, though. We have not tested for indirect effects between Parent Attachment and Self Esteem, and we have not evaluated simpler indirect effects between Peer Attachment and Self Esteem (e.g., PeerAtt \\(\\rightarrow\\) Emp \\(\\rightarrow\\) SelfEst). 3.4.2 Moderation We will first analyze a synthetic version of the Outlook on Life Survey data. The original data were collected in the United States in 2012 to measure, among other things, attitudes about racial issues, opinions of the Federal government, and beliefs about the future. We will work with a synthesized subset of the original data. You can access these synthetic data as outlook.rds. This dataset comprises 2288 observations of the following 13 variables. d1:d3: Three observed indicators of a construct measuring disillusionment with the US Federal government. Higher scores indicate more disillusionment s1:s4: Four observed indicators of a construct measuring the perceived achievability of material success. Higher scores indicate greater perceived achievability progress: A single item assessing perceived progress toward achieving the “American Dream” Higher scores indicate greater perceived progress merit: A single item assessing endorsement of the meritocratic ideal that hard work leads to success. Higher scores indicate stronger endorsement of the meritocratic ideal lib2Con: A single item assessing liberal-to-conservative orientation Lower scores are more liberal, higher scores are more conservative party: A four-level factor indicating self-reported political party affiliation disillusion: A scale score representing disillusionment with the US Federal government Created as the mean of d1:d3 success: A scale score representing the perceived achievability of material success Created as the mean of s1:s4 To satisfy the access and licensing conditions under which the original data are distributed, the data contained in outlook.rds were synthesized from the original variables using the methods described by Volker and Vink (2021). You can access the original data here, and you can access the code used to process the data here. 3.4.2.1 Read in the outlook.rds dataset. Hint: An RDS file is an R object that’s been saved to a file. To read in this type of file, we use readRDS() from base R. Click to show code outlook <- readRDS("outlook.rds") 3.4.2.2 Summarize the outlook data to get a sense of their characteristics. Click to show code head(outlook) summary(outlook) ## d1 d2 d3 s1 ## Min. :1.000 Min. :1.000 Min. :1.000 Min. :1.000 ## 1st Qu.:3.000 1st Qu.:2.000 1st Qu.:3.000 1st Qu.:2.000 ## Median :4.000 Median :3.000 Median :4.000 Median :2.000 ## Mean :3.642 Mean :3.218 Mean :3.629 Mean :2.288 ## 3rd Qu.:4.000 3rd Qu.:4.000 3rd Qu.:4.000 3rd Qu.:3.000 ## Max. :5.000 Max. :5.000 Max. :5.000 Max. :4.000 ## s2 s3 s4 progress ## Min. :1.000 Min. :1.000 Min. :1.000 Min. : 1.000 ## 1st Qu.:1.000 1st Qu.:1.000 1st Qu.:2.000 1st Qu.: 5.000 ## Median :2.000 Median :2.000 Median :2.000 Median : 7.000 ## Mean :1.922 Mean :2.012 Mean :2.469 Mean : 6.432 ## 3rd Qu.:2.000 3rd Qu.:3.000 3rd Qu.:3.000 3rd Qu.: 8.000 ## Max. :4.000 Max. :4.000 Max. :4.000 Max. :10.000 ## merit lib2Con party disillusion ## Min. :1.000 Min. :1.000 republican : 332 Min. :1.000 ## 1st Qu.:4.000 1st Qu.:3.000 democrat :1264 1st Qu.:3.000 ## Median :5.000 Median :4.000 independent: 576 Median :3.667 ## Mean :4.826 Mean :3.998 other : 116 Mean :3.497 ## 3rd Qu.:6.000 3rd Qu.:5.000 3rd Qu.:4.000 ## Max. :7.000 Max. :7.000 Max. :5.000 ## success ## Min. :1.000 ## 1st Qu.:1.750 ## Median :2.000 ## Mean :2.173 ## 3rd Qu.:2.500 ## Max. :4.000 str(outlook) ## 'data.frame': 2288 obs. of 13 variables: ## $ d1 : num 4 4 4 5 5 4 5 4 4 4 ... ## $ d2 : num 4 2 4 4 3 5 4 2 4 5 ... ## $ d3 : num 4 4 4 5 4 4 4 3 3 4 ... ## $ s1 : num 3 3 4 2 2 2 2 1 3 3 ... ## $ s2 : num 2 2 2 1 1 2 1 1 2 2 ... ## $ s3 : num 3 2 4 1 2 1 1 1 3 2 ... ## $ s4 : num 3 3 3 1 2 3 3 2 2 2 ... ## $ progress : num 8 4 6 1 6 5 7 6 9 7 ... ## $ merit : num 6 5 5 4 3 4 2 5 5 5 ... ## $ lib2Con : num 5 6 4 1 4 4 4 4 4 5 ... ## $ party : Factor w/ 4 levels "republican","democrat",..: 1 3 3 2 2 2 2 2 4 1 ... ## $ disillusion: num 4 3.33 4 4.67 4 ... ## $ success : num 2.75 2.5 3.25 1.25 1.75 2 1.75 1.25 2.5 2.25 ... We will first use OLS regression to estimate a model encoding the following relations: Belief in the achievability of success, success, predicts perceived progress toward the American Dream, progress, as the focal effect. Disillusionment with the US Federal government, disillusion moderates the success \\(\\rightarrow\\) progress effect. Placement on the liberal-to-conservative continuum, lib2Con is partialed out as a covariate. 3.4.2.3 Draw the conceptual path diagram for the model described above. Click for explanation 3.4.2.4 Write out the regression equation necessary to evaluate the moderation hypothesis described above. Click for explanation \\[ Y_{progress} = \\beta_0 + \\beta_1 W_{lib2Con} + \\beta_2 X_{success} + \\beta_3 Z_{disillusion} + \\beta_4 XZ + \\varepsilon \\] 3.4.2.5 Use lm() to estimate the moderated regression model via OLS regression. Click to show code olsFit <- lm(progress ~ lib2Con + success * disillusion, data = outlook) 3.4.2.6 Summarize the fitted model and interpret the results. Is the moderation hypothesis supported? How does disillusionment level affect the focal effect? Click to show code summary(olsFit) ## ## Call: ## lm(formula = progress ~ lib2Con + success * disillusion, data = outlook) ## ## Residuals: ## Min 1Q Median 3Q Max ## -7.4315 -1.2525 0.1307 1.4369 5.6717 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 6.81128 0.62073 10.973 < 2e-16 *** ## lib2Con 0.03052 0.03040 1.004 0.3155 ## success 0.42360 0.25853 1.638 0.1015 ## disillusion -0.78002 0.16864 -4.625 3.95e-06 *** ## success:disillusion 0.17429 0.07273 2.396 0.0166 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 2.041 on 2283 degrees of freedom ## Multiple R-squared: 0.1385, Adjusted R-squared: 0.137 ## F-statistic: 91.74 on 4 and 2283 DF, p-value: < 2.2e-16 Click for explanation Yes, disillusion significantly moderates the relation between success and progress (\\(\\beta = 0.174\\), \\(t[2283] = 2.396\\), \\(p = 0.017\\)) such that the effect of success on progress increases as levels of disillusion increase, after controlling for lib2Con. The rockchalk package contains some useful routines for probing interactions estimated via lm(). Specifically, the plotslopes() function will estimate and plot simple slopes, and the testSlopes() function tests the simple slopes estimated by plotSlopes(). 3.4.2.7 Probe the interaction. Use the plotSlopes() and testSlopes() functions from the rockchalk package to conduct a simple slopes analysis for the model from 3.4.2.5. Click to show code library(rockchalk) ## Estimate and plot simple slopes: psOut <- plotSlopes(olsFit, plotx = "success", modx = "disillusion", modxVals = "std.dev") ## Test the simple slopes: tsOut <- testSlopes(psOut) ## Values of disillusion OUTSIDE this interval: ## lo hi ## -28.9332857 0.2672244 ## cause the slope of (b1 + b2*disillusion)success to be statistically significant ## View the results: tsOut$hypotests Note: The message printed by testSlopes() gives the boundaries of the Johnson-Neyman Region of Significance (Johnson & Neyman, 1936). Johnson-Neyman analysis is an alternative method of probing interactions that we have not covered in this course. For more information, check out Preacher, et al. (2006). We will now use lavaan to estimate the moderated regression model from above as a path analysis. 3.4.2.8 Define the model syntax for the path analytic version of the model described above. Parameterize the model as in the OLS regression. Use only observed items and scale scores. Click to show code pathMod <- ' progress ~ 1 + lib2Con + success + disillusion + success:disillusion ' 3.4.2.9 Estimate the path model on the outlook data. Click to show code pathFit <- sem(pathMod, data = outlook) 3.4.2.10 Summarize the fitted path model and interpret the results. Do the results match the OLS regression results? What proportion of the variability in progress is explained by this model? Hint: the function lavInspect() can be used to extract information from models Click to show code summary(pathFit) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 6 ## ## Number of observations 2288 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## progress ~ ## lib2Con 0.031 0.030 1.005 0.315 ## success 0.424 0.258 1.640 0.101 ## disillusion -0.780 0.168 -4.630 0.000 ## success:dsllsn 0.174 0.073 2.399 0.016 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .progress 6.811 0.620 10.985 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .progress 4.157 0.123 33.823 0.000 lavInspect(pathFit, "r2") ## progress ## 0.138 Click for explanation Yes, the estimates and inferential conclusions are all the same as in the OLS regression model. The model explains 13.85% of the variability in progress. The semTools package contains some helpful routines for probing interactions estimated via the lavaan() function (or one of it’s wrappers). Specifically, the probe2WayMC() and plotProbe() functions will estimate/test simple slopes and plot the estimated simple slopes, respectively. 3.4.2.11 Probe the interaction from 3.4.2.9 using semTools utilities. Use probe2WayMC() to estimate and test the simple slopes. Use plotProbe() to visualize the simple slopes. Define the simple slopes with the same conditional values of disillusion that you used in 3.4.2.7. Which simple slopes are significant? Do these results match the results from 3.4.2.7? Click to show code library(semTools) ## Define the conditional values at which to calculate simple slopes: condVals <- summarise(outlook, "m-sd" = mean(disillusion) - sd(disillusion), mean = mean(disillusion), "m+sd" = mean(disillusion) + sd(disillusion) ) %>% unlist() ## Compute simple slopes and intercepts: ssOut <- probe2WayMC(pathFit, nameX = c("success", "disillusion", "success:disillusion"), nameY = "progress", modVar = "disillusion", valProbe = condVals) ## Check the results: ssOut ## $SimpleIntcept ## disillusion est se z pvalue ## m-sd 2.719 4.690 0.231 20.271 0 ## mean 3.497 4.084 0.190 21.508 0 ## m+sd 4.274 3.477 0.230 15.122 0 ## ## $SimpleSlope ## disillusion est se z pvalue ## m-sd 2.719 0.897 0.083 10.792 0 ## mean 3.497 1.033 0.065 15.994 0 ## m+sd 4.274 1.169 0.088 13.223 0 ## Visualize the simple slopes: plotProbe(ssOut, xlim = range(outlook$success), xlab = "Ease of Personal Success", ylab = "Progress toward American Dream", legendArgs = list(legend = names(condVals)) ) Click for explanation Each of the simple slopes is significant. As level of disillusionment increases, the effect of success on progress also increases, and this effect is significant for all levels of disillusion considered here. These results match the simple slopes from the OLS regression analysis. End of In-Class Exercises 3 "],["efa.html", "4 EFA", " 4 EFA This week will be a general introduction to latent variables and scaling procedures. We will discuss several different aspects of exploratory factor analysis (EFA). Most notably: The differences between Principal Component Analyses (PCA) and Factor Analysis Model estimation and factor extraction methods Factor rotations You will have to make decisions regarding each of these aspects when conducting a factor analysis. We will also discuss reliability and factor scores as means of evaluating the properties of a scale. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-3.html", "4.1 Lecture", " 4.1 Lecture How do you know if you have measured the putative hypothetical construct that you intend to measure? The methods introduced in this lecture (namely, latent variables, factor analysis, and reliability analysis) can shed empirical light on this issue. In the social and behavioral sciences we’re often forced to measure key concepts indirectly. For example, we have no way of directly quantifying a person’s current level of depression, or their innate motivation, or their risk-aversion, or any of the other myriad psychological features that comprise the human mental state. In truth, we cannot really measure these hypothetical constructs at all, we must estimate latent representations thereof (though, psychometricians still use the language of physical measurement to describe this process). Furthermore, we can rarely estimate an adequate representation with only a single observed variable (e.g., question on a survey, score on a test, reading from a sensor). We generally need several observed variables to reliably represent a single hypothetical construct. For example, we cannot accurately determine someone’s IQ or socio-economic status based on their response to a single question; we need several questions that each tap into slightly different aspects of IQ or SES. Given multiple items measuring the same construct, we can use the methods discussed in this lecture (i.e., factor analysis and reliability analysis) to evaluate the quality of our measurement (i.e., how well we have estimated the underlying hypothetical construct). If we do well enough in this estimation task, we will be able to combine these estimated latent variables with the path analysis methods discussed in previous two weeks to produce the full structural equation models that we will cover at the end of this course. 4.1.1 Recording Notes: This week (and next), we’ll be re-using Caspar van Lissa’s old slides and lecture recording. So, you’ll see Caspar in the following video, and the slides will have a notably different flavor than our usual materials. Don’t be confused by any mention of “model fit” in the lecture. We haven’t covered model fit yet, but we will do so next week. 4.1.2 Slides You can download the lecture slides here. "],["reading-3.html", "4.2 Reading", " 4.2 Reading This week, you will read two papers. Reference 1 Preacher, K. J., & MacCullum, R. C. (2003). Repairing Tom Swift’s electric factor analysis machine, Understanding Statistics 2(1) 13–43. Questions 1 What is a latent variable? Give an example of a latent variable. What is factor analysis, and what can you investigate using this method? In the introduction, Preacher and Maccallum describe a “little jiffy” method of doing factor analysis. Briefly describe this little jiffy—or bad practice—method. Briefly explain the key differences between Principal Component Analyses (PCA) and Exploratory Factor Analyses (EFA). What is the purpose of factor rotation? Reference 2 Kestilä, E. (2006). Is there demand for radical right populism in the Finnish electorate? Scandinavian Political Studies 29(3), 169–191. Questions 2 What is the research question that the author tries to answer? Briefly describe the characteristics of the Radical Right Parties (RRP) in Europe. What are the two main explanations of support for RRP upon which this paper focuses? Does the empirical part of the paper reflect the theoretical framework well? Why or why not? According to the author, is Finland very different from other European countries on the main dependent variables? What is the author’s conclusion (i.e., how does the author answer the research question)? "],["at-home-exercises-3.html", "4.3 At-Home Exercises", " 4.3 At-Home Exercises In these exercises, you will attempt to replicate some of the analyses from the second reading for this week: Kestilä, E. (2006). Is there demand for radical right populism in the Finnish electorate? Scandinavian Political Studies 29(3), 169–191. The data for this practical were collected during the first round of the European Social Survey (ESS). The ESS is a repeated cross-sectional survey administered in 32 European countries. The first wave was collected in 2002, and two new waves have been collected each year since. You can find more info and access the data at https://www.europeansocialsurvey.org. The data we will analyze for this practical are contained in the file named ESSround1-a.sav. This file contains data for all respondents, but only includes those variables that you will need to complete the following exercises. 4.3.1 Load the ESSround1-a.sav dataset into R. Inspect the data after loading to make sure everything went well. Click to show code ## Load the 'haven' package: library(haven) library(tidySEM) ## Read the 'ESSround1-a.sav' data into a data frame called 'ess': ess <- read_spss("ESSround1-a.sav") ## Inspect the result: dim(ess) head(ess) descriptives(ess) ## [1] 42359 50 Click here for a description of the variables. Variable Description name Title of dataset essround ESS round edition Edition proddate Production date cntry Country idno Respondent’s identification number trstlgl Trust in the legal system trstplc Trust in the police trstun Trust in the United Nations trstep Trust in the European Parliament trstprl Trust in country’s parliament stfhlth State of health services in country nowadays stfedu State of education in country nowadays stfeco How satisfied with present state of economy in country stfgov How satisfied with the national government stfdem How satisfied with the way democracy works in country pltinvt Politicians interested in votes rather than peoples opinions pltcare Politicians in general care what people like respondent think trstplt Trust in politicians imsmetn Allow many/few immigrants of same race/ethnic group as majority imdfetn Allow many/few immigrants of different race/ethnic group from majority eimrcnt Allow many/few immigrants from richer countries in Europe eimpcnt Allow many/few immigrants from poorer countries in Europe imrcntr Allow many/few immigrants from richer countries outside Europe impcntr Allow many/few immigrants from poorer countries outside Europe qfimchr Qualification for immigration: christian background qfimwht Qualification for immigration: be white imwgdwn Average wages/salaries generally brought down by immigrants imhecop Immigrants harm economic prospects of the poor more than the rich imtcjob Immigrants take jobs away in country or create new jobs imbleco Taxes and services: immigrants take out more than they put in or less imbgeco Immigration bad or good for country’s economy imueclt Country’s cultural life undermined or enriched by immigrants imwbcnt Immigrants make country worse or better place to live imwbcrm Immigrants make country’s crime problems worse or better imrsprc Richer countries should be responsible for accepting people from poorer countries pplstrd Better for a country if almost everyone share customs and traditions vrtrlg Better for a country if a variety of different religions shrrfg Country has more than its fair share of people applying refugee status rfgawrk People applying refugee status allowed to work while cases considered gvrfgap Government should be generous judging applications for refugee status rfgfrpc Most refugee applicants not in real fear of persecution own countries rfggvfn Financial support to refugee applicants while cases considered rfgbfml Granted refugees should be entitled to bring close family members gndr Gender yrbrn Year of birth edulvl Highest level of education eduyrs Years of full-time education completed polintr How interested in politics lrscale Placement on left right scale One thing you might notice when inspecting the ess data is that most of the variables are stored as labelled vectors. When loading SPSS data, haven will use these labelled vectors to preserve the metadata associated with SPSS scale variables (i.e., variable labels and value labels). While it’s good to have this metadata available, we want to analyze these items as numeric variables and factors, so the value labels are only going to make our lives harder. Thankfully, the labelled package contains many routines for manipulating labelled vectors. We’ll deal with the numeric variables in just a bit, but our first task will be to covert grouping variables to factors. 4.3.2 Convert the cntry, gndr, edulvl, and polintr variables into factors. Use the as_factor() function to do the conversion. Convert edulvl and polintr to ordered factors. Click to see code library(dplyr) ess <- mutate(ess, country = as_factor(cntry), sex = as_factor(gndr), edulvl = as_factor(edulvl, ordered = TRUE), polintr = as_factor(polintr, ordered = TRUE) ) The ess dataset contains much more information than Kestilä (2006) used. Kestilä only analyzed data from the following ten countries: Austria Belgium Denmark Finland France Germany Italy Netherlands Norway Sweden So, our next task is to subset the data to only the relevant population. When we apply logical subsetting, we can select rows from a dataset based on logical conditions. In this case, we want to select only rows from the 10 countries listed above. 4.3.3 Subset the data to include only the 10 countries analyzed by Kestilä (2006). Inspect the subsetted data to check that everything went well. Hints: Use the %in% operator to create a logical vector that indicates which elements of the cntry variable are in the set of target counties. Use the droplevels() levels function to clean up empty factor levels. Click to show code ## Create a character vector naming the target countries: targets <- c("Austria", "Belgium", "Denmark", "Finland", "France", "Germany", "Italy", "Netherlands", "Norway", "Sweden") ## Select only those rows that come from a target country: ess <- filter(ess, country %in% targets) %>% # Subset rows droplevels() # Drop empty factor levels ## Inspect the result: dim(ess) ## [1] 19690 52 table(ess$country) ## ## Austria Belgium Germany Denmark Finland France ## 2257 1899 2919 1506 2000 1503 ## Italy Netherlands Norway Sweden ## 1207 2364 2036 1999 In keeping with common practice, we will treat ordinal Likert-type rating scales with five or more levels as continuous. Since some R routines will treat labelled vectors as discrete variables, we can make things easier for ourselves by converting all the labelled vectors in our data to numeric vectors. We can use the labelled::remove_val_labels() function to strip the value labels and convert all of the labelled vectors to numeric vectors. 4.3.4 Convert the remaining labelled vectors to numeric vectors. Click to see code ## If necessary, install the labelled package: # install.packages("labelled", repos = "https://cloud.r-project.org") ## Load the labelled package: library(labelled) ## Strip the value labels: ess <- remove_val_labels(ess) ## Check the effects: str(ess) ## tibble [19,690 × 52] (S3: tbl_df/tbl/data.frame) ## $ name : chr [1:19690] "ESS1e06_1" "ESS1e06_1" "ESS1e06_1" "ESS1e06_1" ... ## ..- attr(*, "label")= chr "Title of dataset" ## ..- attr(*, "format.spss")= chr "A9" ## ..- attr(*, "display_width")= int 14 ## $ essround: num [1:19690] 1 1 1 1 1 1 1 1 1 1 ... ## ..- attr(*, "label")= chr "ESS round" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 10 ## $ edition : chr [1:19690] "6.1" "6.1" "6.1" "6.1" ... ## ..- attr(*, "label")= chr "Edition" ## ..- attr(*, "format.spss")= chr "A3" ## ..- attr(*, "display_width")= int 9 ## $ proddate: chr [1:19690] "03.10.2008" "03.10.2008" "03.10.2008" "03.10.2008" ... ## ..- attr(*, "label")= chr "Production date" ## ..- attr(*, "format.spss")= chr "A10" ## ..- attr(*, "display_width")= int 12 ## $ cntry : num [1:19690] 1 18 1 1 18 1 2 18 1 18 ... ## ..- attr(*, "label")= chr "Country" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 7 ## $ idno : num [1:19690] 1 1 2 3 3 4 4 4 6 6 ... ## ..- attr(*, "label")= chr "Respondent's identification number" ## ..- attr(*, "format.spss")= chr "F9.0" ## ..- attr(*, "display_width")= int 11 ## $ trstlgl : num [1:19690] 10 6 8 4 8 10 9 7 7 7 ... ## ..- attr(*, "label")= chr "Trust in the legal system" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ trstplc : num [1:19690] 10 8 5 8 8 9 8 9 4 9 ... ## ..- attr(*, "label")= chr "Trust in the police" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ trstun : num [1:19690] 9 8 6 NA 5 8 NA 7 5 7 ... ## ..- attr(*, "label")= chr "Trust in the United Nations" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ trstep : num [1:19690] NA 3 0 7 3 7 0 3 4 6 ... ## ..- attr(*, "label")= chr "Trust in the European Parliament" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ trstprl : num [1:19690] 9 7 0 6 8 8 10 2 6 8 ... ## ..- attr(*, "label")= chr "Trust in country's parliament" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ stfhlth : num [1:19690] 10 4 0 7 6 8 NA 6 3 5 ... ## ..- attr(*, "label")= chr "State of health services in country nowadays" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ stfedu : num [1:19690] 8 7 7 5 8 7 NA 7 6 7 ... ## ..- attr(*, "label")= chr "State of education in country nowadays" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ stfeco : num [1:19690] 7 6 0 7 8 6 NA 9 8 9 ... ## ..- attr(*, "label")= chr "How satisfied with present state of economy in country" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ stfgov : num [1:19690] 7 7 0 7 6 3 NA 5 5 7 ... ## ..- attr(*, "label")= chr "How satisfied with the national government" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ stfdem : num [1:19690] 8 5 5 5 7 7 NA 7 7 9 ... ## ..- attr(*, "label")= chr "How satisfied with the way democracy works in country" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ pltinvt : num [1:19690] 1 3 1 1 4 1 1 3 2 3 ... ## ..- attr(*, "label")= chr "Politicians interested in votes rather than peoples opinions" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ pltcare : num [1:19690] 1 4 1 1 4 3 2 5 2 3 ... ## ..- attr(*, "label")= chr "Politicians in general care what people like respondent think" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ trstplt : num [1:19690] 0 5 0 2 5 4 8 2 4 6 ... ## ..- attr(*, "label")= chr "Trust in politicians" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imsmetn : num [1:19690] 4 3 2 3 2 1 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants of same race/ethnic group as majority" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imdfetn : num [1:19690] 3 3 2 3 2 2 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants of different race/ethnic group from majority" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ eimrcnt : num [1:19690] 4 2 2 2 3 1 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from richer countries in Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ eimpcnt : num [1:19690] 3 2 2 2 2 2 NA 2 NA 1 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from poorer countries in Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imrcntr : num [1:19690] 3 3 2 2 2 1 NA 2 NA 2 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from richer countries outside Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ impcntr : num [1:19690] 3 2 2 3 2 1 NA 2 NA 2 ... ## ..- attr(*, "label")= chr "Allow many/few immigrants from poorer countries outside Europe" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ qfimchr : num [1:19690] 4 2 0 6 2 0 99 0 1 2 ... ## ..- attr(*, "label")= chr "Qualification for immigration: christian background" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ qfimwht : num [1:19690] 1 0 0 0 0 0 99 0 0 1 ... ## ..- attr(*, "label")= chr "Qualification for immigration: be white" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imwgdwn : num [1:19690] 3 4 2 2 3 3 NA 4 NA 4 ... ## ..- attr(*, "label")= chr "Average wages/salaries generally brought down by immigrants" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imhecop : num [1:19690] 2 2 1 4 3 2 NA 3 NA 2 ... ## ..- attr(*, "label")= chr "Immigrants harm economic prospects of the poor more than the rich" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ imtcjob : num [1:19690] 7 5 6 5 7 10 NA 8 NA 4 ... ## ..- attr(*, "label")= chr "Immigrants take jobs away in country or create new jobs" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imbleco : num [1:19690] 9 4 2 NA 3 10 NA 9 NA 6 ... ## ..- attr(*, "label")= chr "Taxes and services: immigrants take out more than they put in or less" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imbgeco : num [1:19690] 4 3 10 7 5 10 NA 8 NA 5 ... ## ..- attr(*, "label")= chr "Immigration bad or good for country's economy" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imueclt : num [1:19690] 9 4 10 5 4 10 NA 9 NA 3 ... ## ..- attr(*, "label")= chr "Country's cultural life undermined or enriched by immigrants" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imwbcnt : num [1:19690] 7 3 5 5 5 10 NA 8 NA 5 ... ## ..- attr(*, "label")= chr "Immigrants make country worse or better place to live" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imwbcrm : num [1:19690] 3 3 5 2 3 5 NA 5 NA 3 ... ## ..- attr(*, "label")= chr "Immigrants make country's crime problems worse or better" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ imrsprc : num [1:19690] 2 2 1 4 1 2 NA 1 1 3 ... ## ..- attr(*, "label")= chr "Richer countries should be responsible for accepting people from poorer countries" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ pplstrd : num [1:19690] 2 4 2 2 3 4 NA 4 4 2 ... ## ..- attr(*, "label")= chr "Better for a country if almost everyone share customs and traditions" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ vrtrlg : num [1:19690] 3 5 3 2 4 1 NA 4 2 3 ... ## ..- attr(*, "label")= chr "Better for a country if a variety of different religions" ## ..- attr(*, "format.spss")= chr "F1.0" ## $ shrrfg : num [1:19690] 3 2 1 1 3 3 NA 3 4 3 ... ## ..- attr(*, "label")= chr "Country has more than its fair share of people applying refugee status" ## ..- attr(*, "format.spss")= chr "F1.0" ## $ rfgawrk : num [1:19690] 2 2 1 2 2 2 NA 2 1 2 ... ## ..- attr(*, "label")= chr "People applying refugee status allowed to work while cases considered" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ gvrfgap : num [1:19690] 4 3 2 4 2 2 NA 3 2 4 ... ## ..- attr(*, "label")= chr "Government should be generous judging applications for refugee status" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ rfgfrpc : num [1:19690] 4 3 2 4 4 4 NA 4 3 4 ... ## ..- attr(*, "label")= chr "Most refugee applicants not in real fear of persecution own countries" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ rfggvfn : num [1:19690] 2 3 2 4 3 2 NA 2 2 2 ... ## ..- attr(*, "label")= chr "Financial support to refugee applicants while cases considered" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ rfgbfml : num [1:19690] 2 3 1 2 2 1 NA 4 2 3 ... ## ..- attr(*, "label")= chr "Granted refugees should be entitled to bring close family members" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 9 ## $ gndr : num [1:19690] 1 2 1 2 2 1 NA 2 2 1 ... ## ..- attr(*, "label")= chr "Gender" ## ..- attr(*, "format.spss")= chr "F1.0" ## ..- attr(*, "display_width")= int 6 ## $ yrbrn : num [1:19690] 1949 1978 1953 1940 1964 ... ## ..- attr(*, "label")= chr "Year of birth" ## ..- attr(*, "format.spss")= chr "F4.0" ## ..- attr(*, "display_width")= int 7 ## $ edulvl : Ord.factor w/ 7 levels "Not completed primary education"<..: NA 4 NA NA 4 NA NA 7 NA 6 ... ## $ eduyrs : num [1:19690] 11 16 14 9 12 18 NA 17 15 17 ... ## ..- attr(*, "label")= chr "Years of full-time education completed" ## ..- attr(*, "format.spss")= chr "F2.0" ## $ polintr : Ord.factor w/ 4 levels "Very interested"<..: 3 3 1 2 3 2 1 4 3 3 ... ## $ lrscale : num [1:19690] 6 7 6 5 8 5 NA 8 5 7 ... ## ..- attr(*, "label")= chr "Placement on left right scale" ## ..- attr(*, "format.spss")= chr "F2.0" ## ..- attr(*, "display_width")= int 9 ## $ country : Factor w/ 10 levels "Austria","Belgium",..: 1 9 1 1 9 1 2 9 1 9 ... ## $ sex : Factor w/ 2 levels "Male","Female": 1 2 1 2 2 1 NA 2 2 1 ... descriptives(ess) Click for explanation Note that the numeric variables are now simple numeric vectors, but the variable labels have been retained as column attributes (which is probably useful). If we want to completely nuke the labelling information, we can use the labelled::remove_labels() function to do so. In addition to screening with summary statistics, we can also visualize the variables’ distributions. You have already created a few such visualizations for single variables. Now, we will use a few tricks to efficiently plot each of our target variables. The first step in this process will be to convert the interesting part of our data from “wide format” (one column per variable) into “long format” (one column of variable names, one column of data values). The pivot_longer() function from the tidyr package provides a convenient way to execute this conversion. 4.3.5 Use tidyr::pivot_longer() to create a long-formatted data frame from the target variables in ess. The target variables are all columns from trstlgl to rfgbfml. Click to show code ## Load the tidyr package: library(tidyr) ## Convert the target variables into a long-formatted data frame: ess_plot <- pivot_longer(ess, cols = trstlgl:rfgbfml, # Which columns to convert names_to = "variable", # Name for the new grouping variable values_to = "value") # Name for the column of stacked values The next step in the process will be to plot the variables using ggplot(). In the above code, I’ve named the new grouping variable variable and the new stacked data variable value. So, to create one plot for each (original, wide-format) variable, we will use the facet_wrap() function to facet the plots of value on the variable column (i.e., create a separate conditional plot of value for each unique value in variable). 4.3.6 Use ggplot() with an appropriate geom (e.g., geom_histogram(), geom_density(), geom_boxplot()) and facet_wrap() to visualize each of the target variables. Hint: To implement the faceting, simply add facet_wrap(~ variable, scales = \"free_x\") to the end of your ggplot() call (obviously, replacing “variable” with whatever you named the grouping variable in your pivot_longer() call). Click to show code library(ggplot2) ggplot(ess_plot, aes(x = value)) + geom_histogram() + # Create a histogram facet_wrap(~ variable, scales = "free_x") # Facet on 'variable' Click for explanation Notice that the variables are actually discrete (i.e., each variable takes only a few integer values). However, most variables look relatively normal despite being categorical. So, we’ll bend the rules a bit and analyze these variables as continuous. It also looks like there’s something weird going on with qfimchr and qfimwht. More on that below. 4.3.7 Check the descriptives for the target variables again. Do you see any remaining issues? Click to show code select(ess, trstlgl:rfgbfml) %>% descriptives() Click for explanation The variables qfimchr and qfimwht both contain values that fall outside the expected range for our survey responses: 77, 88, and 99. In SPSS, these were labeled as “Refusal” “Don’t know” and “No answer” respectively, and would not have contributed to the analysis. 4.3.8 Correct any remaining issues you found above. Click to show code ess <- ess %>% mutate(across(c(qfimchr, qfimwht), na_if, 77)) %>% mutate(across(c(qfimchr, qfimwht), na_if, 88)) %>% mutate(across(c(qfimchr, qfimwht), na_if, 99)) ## Check the results: select(ess, trstlgl:rfgbfml) %>% descriptives() Click to show explanation Here, we need to tell R that these values should be considered missing, or NA. Otherwise they will contribute the numeric value to the analysis, as though someone had provided an answer of 77 on a 10-point scale. We’ve done quite a bit of data processing, and we’ll continue to use these data for several future practicals, so it would be a good idea to save the processed dataset for later use. When saving data that you plan to analyze in R, you will usually want to use the R Data Set (RDS) format. Datasets saved in RDS format retain all of their attributes and formatting (e.g., factor are still factors, missing values are coded as NA, etc.). So, you don’t have to redo any data processing before future analyses. 4.3.9 Use the saveRDS() function to save the processed dataset. Click to show code ## Save the processed data: saveRDS(ess, "ess_round1.rds") Now, we’re ready to run the analyses and see if we can replicate the Kestilä (2006) results. 4.3.10 Run two principal component analyses (PCA): one for trust in politics, one for attitudes towards immigration. Use the principal() function from the psych package. Use exactly the same specifications as Kestilä (2006) concerning the estimation method, rotation, number of components extracted, etc. Hints: Remember that you can view the help file for psych::principal() by running ?psych::principal or, if the psych package already loaded, simply running ?principal. When you print the output from psych::principal(), you can use the cut option to hide any factor loadings smaller than a given threshold. You could consider hiding any loadings smaller than those reported by Kestilä (2006) to make the output easier to interpret. Click to show code Trust in politics Kestilä extracted three components with VARIMAX rotation. ## Load the psych package: library(psych) ## Run the PCA: pca_trust <- select(ess, trstlgl:trstplt) %>% principal(nfactors = 3, rotate = "varimax") ## Print the results: print(pca_trust, cut = 0.3, digits = 3) ## Principal Components Analysis ## Call: principal(r = ., nfactors = 3, rotate = "varimax") ## Standardized loadings (pattern matrix) based upon correlation matrix ## RC3 RC2 RC1 h2 u2 com ## trstlgl 0.779 0.669 0.331 1.21 ## trstplc 0.761 0.633 0.367 1.18 ## trstun 0.675 0.556 0.444 1.44 ## trstep 0.651 0.332 0.549 0.451 1.57 ## trstprl 0.569 0.489 0.650 0.350 2.49 ## stfhlth 0.745 0.567 0.433 1.04 ## stfedu 0.750 0.603 0.397 1.14 ## stfeco 0.711 0.300 0.616 0.384 1.44 ## stfgov 0.634 0.377 0.587 0.413 1.88 ## stfdem 0.369 0.568 0.325 0.564 0.436 2.38 ## pltinvt 0.817 0.695 0.305 1.08 ## pltcare 0.811 0.695 0.305 1.11 ## trstplt 0.510 0.611 0.716 0.284 2.40 ## ## RC3 RC2 RC1 ## SS loadings 2.942 2.668 2.490 ## Proportion Var 0.226 0.205 0.192 ## Cumulative Var 0.226 0.432 0.623 ## Proportion Explained 0.363 0.329 0.307 ## Cumulative Proportion 0.363 0.693 1.000 ## ## Mean item complexity = 1.6 ## Test of the hypothesis that 3 components are sufficient. ## ## The root mean square of the residuals (RMSR) is 0.07 ## with the empirical chi square 15240.94 with prob < 0 ## ## Fit based upon off diagonal values = 0.967 Attitudes toward immigration Kestilä extracted five components with VARIMAX rotation. pca_att <- select(ess, imsmetn:rfgbfml) %>% principal(nfactors = 5, rotate = "varimax") print(pca_att, cut = 0.3, digits = 3) ## Principal Components Analysis ## Call: principal(r = ., nfactors = 5, rotate = "varimax") ## Standardized loadings (pattern matrix) based upon correlation matrix ## RC2 RC1 RC5 RC3 RC4 h2 u2 com ## imsmetn 0.797 0.725 0.275 1.30 ## imdfetn 0.775 0.794 0.206 1.70 ## eimrcnt 0.827 0.715 0.285 1.09 ## eimpcnt 0.800 0.789 0.211 1.49 ## imrcntr 0.835 0.747 0.253 1.15 ## impcntr 0.777 0.782 0.218 1.63 ## qfimchr 0.813 0.688 0.312 1.08 ## qfimwht 0.752 0.637 0.363 1.26 ## imwgdwn 0.807 0.712 0.288 1.19 ## imhecop 0.747 0.669 0.331 1.42 ## imtcjob 0.569 0.334 0.484 0.516 1.99 ## imbleco 0.703 0.554 0.446 1.25 ## imbgeco 0.698 0.605 0.395 1.52 ## imueclt 0.568 -0.340 0.545 0.455 2.43 ## imwbcnt 0.673 0.633 0.367 1.87 ## imwbcrm 0.655 0.478 0.522 1.23 ## imrsprc 0.614 0.440 0.560 1.34 ## pplstrd 0.324 -0.551 0.468 0.532 2.11 ## vrtrlg -0.345 0.471 0.419 0.581 2.67 ## shrrfg 0.365 -0.352 0.418 0.582 4.16 ## rfgawrk 0.614 0.396 0.604 1.10 ## gvrfgap 0.691 0.559 0.441 1.35 ## rfgfrpc -0.387 0.327 0.673 3.34 ## rfggvfn 0.585 0.417 0.583 1.46 ## rfgbfml 0.596 0.460 0.540 1.61 ## ## RC2 RC1 RC5 RC3 RC4 ## SS loadings 4.374 3.393 2.774 2.199 1.723 ## Proportion Var 0.175 0.136 0.111 0.088 0.069 ## Cumulative Var 0.175 0.311 0.422 0.510 0.579 ## Proportion Explained 0.302 0.235 0.192 0.152 0.119 ## Cumulative Proportion 0.302 0.537 0.729 0.881 1.000 ## ## Mean item complexity = 1.7 ## Test of the hypothesis that 5 components are sufficient. ## ## The root mean square of the residuals (RMSR) is 0.05 ## with the empirical chi square 29496.06 with prob < 0 ## ## Fit based upon off diagonal values = 0.976 Feature engineering (i.e., creating new variables by combining and/or transforming existing variables) is one of the most common applications of PCA. PCA is a dimension reduction technique that distills the most salient information from a set of variables into a (smaller) set of component scores. Hence, PCA can be a good way of creating aggregate items (analogous to weighted scale scores) when the data are not collected with validated scales. Principal component scores are automatically generated when we run the PCA. If we want to use these scores in subsequent analyses (e.g., as predictors in a regression model), we usually add them to our dataset as additional columns. 4.3.11 Add the component scores produced by the analyses you ran above to the ess data frame. Give each component score an informative name, based on your interpretation of the factor loading matrix I.e., What hypothetical construct do you think each component represents given the items that load onto it? Hints: You can use the data.frame() function to join multiple objects into a single data frame. You can use the colnames() function to assign column names to a matrix or data frame. 1. Extract the component scores Click to show code ## Save the component scores in stand-alone matrices: trust_scores <- pca_trust$scores att_scores <- pca_att$scores ## Inspect the result: head(trust_scores) ## RC3 RC2 RC1 ## [1,] NA NA NA ## [2,] 0.09755193 -0.01552183 0.994954 ## [3,] 0.23069626 -1.53162604 -2.022642 ## [4,] NA NA NA ## [5,] -0.21112678 0.84370377 1.200007 ## [6,] 1.86596955 0.31083233 -1.062603 summary(trust_scores) ## RC3 RC2 RC1 ## Min. :-4.035 Min. :-3.706 Min. :-3.139 ## 1st Qu.:-0.527 1st Qu.:-0.652 1st Qu.:-0.649 ## Median : 0.155 Median : 0.094 Median : 0.092 ## Mean : 0.055 Mean : 0.015 Mean : 0.049 ## 3rd Qu.: 0.727 3rd Qu.: 0.742 3rd Qu.: 0.742 ## Max. : 3.302 Max. : 3.452 Max. : 3.539 ## NA's :4912 NA's :4912 NA's :4912 head(att_scores) ## RC2 RC1 RC5 RC3 RC4 ## [1,] 1.9873715 1.3233586 -0.8382499 -0.02172765 -0.0908143 ## [2,] 0.1692841 -1.2178436 -0.5016936 -0.21749066 0.6758844 ## [3,] -0.3630480 0.3260383 -1.5133423 -0.51405480 -2.2071787 ## [4,] NA NA NA NA NA ## [5,] -0.1137484 -0.7891232 -1.4732563 -0.05843873 0.4110692 ## [6,] -0.9195530 2.8231404 -0.3480398 -0.75699796 -1.3230602 summary(att_scores) ## RC2 RC1 RC5 RC3 ## Min. :-3.660 Min. :-3.929 Min. :-3.824 Min. :-2.764 ## 1st Qu.:-0.616 1st Qu.:-0.585 1st Qu.:-0.656 1st Qu.:-0.748 ## Median :-0.085 Median : 0.062 Median :-0.008 Median :-0.121 ## Mean :-0.013 Mean : 0.012 Mean : 0.021 Mean : 0.014 ## 3rd Qu.: 0.680 3rd Qu.: 0.654 3rd Qu.: 0.652 3rd Qu.: 0.698 ## Max. : 3.743 Max. : 4.584 Max. : 4.108 Max. : 4.084 ## NA's :5447 NA's :5447 NA's :5447 NA's :5447 ## RC4 ## Min. :-3.784 ## 1st Qu.:-0.683 ## Median : 0.046 ## Mean : 0.003 ## 3rd Qu.: 0.717 ## Max. : 3.254 ## NA's :5447 Click for explanation The object produced by psych::principal() is simply list, and the component scores are already stored therein. So, to extract the component scores, we simply use the $ operator to extract them. 2. Name the component scores Click to show code ## Check names (note the order): colnames(trust_scores) ## [1] "RC3" "RC2" "RC1" colnames(att_scores) ## [1] "RC2" "RC1" "RC5" "RC3" "RC4" ## Give informative names: colnames(trust_scores) <- c("Trust_Institutions", "Satisfaction", "Trust_Politicians") colnames(att_scores) <- c("Quantity", "Effects", "Refugees", "Diversity", "Economic") 3. Add the component scores to the dataset Click to show code # Add the component scores to the 'ess' data: ess <- data.frame(ess, trust_scores, att_scores) 4.3.12 Were you able to replicate the results of Kestilä (2006)? Click for explanation Yes, more-or-less. Although the exact estimates differ somewhat, the general pattern of factor loadings in Kestilä (2006) matches what we found here. End of At-Home Exercises "],["in-class-exercises-3.html", "4.4 In-Class Exercises", " 4.4 In-Class Exercises In these exercises, we will continue with our re-analysis/replication of the Kestilä (2006) results. Rather than attempting a direct replication, we will now redo the analysis using exploratory factor analysis (EFA). 4.4.1 Load the ess_round1.rds dataset. These are the data that we saved after the data processing in the At-Home Exercises. Click to show code ess <- readRDS("ess_round1.rds") 4.4.2 Kestilä (2006) claimed that running a PCA is a good way to test if the questions in the ESS measure attitudes towards immigration and trust in politics. Based on what you’ve learned from the readings and lectures, do you agree with this position? Click for explanation Hopefully not. PCA is not a method for estimating latent measurement structure; PCA is a dimension reduction technique that tries to summarize a set of data with a smaller set of component scores. If we really want to estimate the factor structure underlying a set of observed variables, we should use EFA. 4.4.3 Suppose you had to construct the trust in politics and attitude towards immigration scales described by Kestilä (2006) based on the theory and background information presented in that article. What type of analysis would you choose? What key factors would influence your decision? Click for explanation We are trying to estimate meaningful latent factors, so EFA would be an appropriate method. The theory presented by Kestilä (2006) did not hypothesize a particular number of factors, so we would need to use appropriate techniques to estimate the best number. In particular, combining information from: Scree plots Parallel analysis Substantive interpretability of the (rotated) factor loadings Since the factors are almost certainly correlated, we should apply an oblique rotation. We will now rerun the two PCAs that you conducted for the At-Home Exercises using EFA. We will estimate the EFA models using the psych::fa() function, but we need to know how many factors to extract. We could simply estimate a range of solutions and compare the results. We can restrict the range of plausible solutions and save some time by first checking/plotting the eigenvalues and running parallel analysis. 4.4.4 Estimate the number of latent factors underlying the Trust items based on the eigenvalues, the scree plot, and parallel analysis. How many factors are suggested by each method? 1. Eigenvalue estimation Click to show code ## Load the psych package: library(psych) ## Run a trivial EFA on the 'trust' items efa_trust0 <- select(ess, trstlgl:trstplt) %>% fa(nfactors = 1, rotate = "none") Click for explanation (EFA) First, we run a trivial EFA using the psych::fa() function to estimate the eigenvalues. We don’t care about the factors yet, so we can extract a single factor. We also don’t care about interpretable solutions, so we don’t need rotation. ## View the estimated eigenvalues: round(efa_trust0$values, digits = 3) ## [1] 4.980 0.716 0.482 0.165 0.069 0.014 -0.066 -0.092 -0.182 -0.207 ## [11] -0.284 -0.296 -0.319 Click for explanation (eigenvalue extraction) We can check the eigenvalues to see what proportion of the observed variance is accounted for by each additional factor we may extract. Since only one eigenvalue is greater than one, the so-called “Kaiser Criterion” would suggest extracting a single factor. The Kaiser Criterion is not a valid way to select the number of factors in EFA. So, we don’t want to rely on this information alone. We can still use the eigenvalues to help us with factor enumeration, though. One way to do so is by plotting the eigenvalues in a scree plot. 2. Scree plot Click to show code Given a vector of estimated eigenvalues, we can create a scree plot using ggplot() and the geom_line() or geom_path() geometry. library(ggplot2) library(magrittr) efa_trust0 %$% data.frame(y = values, x = 1:length(values)) %>% ggplot(aes(x, y)) + geom_line() + xlab("No. of Factors") + ylab("Eigenvalues") We can also use the psych::scree() function to create a scree plot directly from the data. select(ess, trstlgl:trstplt) %>% scree(pc = FALSE) Click for explanation (scree plot) Although the scree plot provides useful information, we need to interpret that information subjectively, and the conclusions are sometimes ambiguous, in this case. In this case, the plot seems to suggest either one or three components, depending on where we consider the “elbow” to lie. As recommended in the lecture, we can also use “parallel analysis” (Horn, 1965) to provide more objective information about the number of factors. We’ll use the psych::fa.parallel() function to implement parallel analysis. Parallel analysis relies on randomly simulated/permuted data, so we should set a seed to make sure our results are reproducible. We can set the fa = \"fa\" option to get only the results for EFA. 3. Parallel Analysis Click to show code ## Set the random number seed: set.seed(235711) ## Run the parallel analysis: pa_trust <- select(ess, trstlgl:trstplt) %>% fa.parallel(fa = "fa") ## Parallel analysis suggests that the number of factors = 6 and the number of components = NA Click for explanation The results of the parallel analysis suggest 6 factors. If you’ve been paying close attention, you may have noticed that we need to compute the eigenvalues from the original data to run parallel analysis. Hence, we don’t actually need to run a separate EFA to estimate the eigenvalues. ## View the eigenvalues estimated during the parallel analysis: pa_trust$fa.values ## [1] 4.97995262 0.71644127 0.48201040 0.16517645 0.06885820 0.01422241 ## [7] -0.06606777 -0.09225113 -0.18231333 -0.20740917 -0.28415857 -0.29573407 ## [13] -0.31877470 ## Compare to the version from the EFA: pa_trust$fa.values - efa_trust0$values ## [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 ## Recreate the scree plot from above: pa_trust %$% data.frame(y = fa.values, x = 1:length(fa.values)) %>% ggplot(aes(x, y)) + geom_line() + xlab("No. of Factors") + ylab("Eigenvalues") Of course, we also see the same scree plot printed as part of the parallel analysis. So, there’s really no reason to create a separate scree plot, at all, if we’re doing parallel analysis. 4. Conclusion Click for explanation The different criteria disagree on how many factors we should extract, but we have narrowed the range. Based on the scree plot and parallel analysis, we should consider solutions for 3 to 6 factors. We need to examine the factor loadings to see which solution makes the most substantive sense. 4.4.5 Do the same analysis for the attitudes toward immigration items. Click to show code This time, we’ll start by running the parallel analysis and get the eigenvalues and scree plot from psych::fa.parallel(). ## Set the seed: set.seed(235711) ## Run parallel analysis on the 'attitudes' items: pa_att <- select(ess, imsmetn:rfgbfml) %>% fa.parallel(fa = "fa") ## Parallel analysis suggests that the number of factors = 7 and the number of components = NA ## Check the eigenvalues: round(pa_att$fa.values, digits = 3) ## [1] 7.895 1.449 0.734 0.533 0.313 0.156 0.121 0.019 -0.001 -0.064 ## [11] -0.083 -0.103 -0.119 -0.131 -0.150 -0.175 -0.185 -0.200 -0.212 -0.233 ## [21] -0.239 -0.247 -0.334 -0.422 -0.427 Click for explanation For the attitudes toward immigration analysis, the results are even more ambiguous than they were for the trust items. The Kaiser Criterion suggests 2 factors. The scree plot is hopelessly ambiguous. At least 3 factors? No more than 9 factors? Parallel analysis suggests 7 factors Based on the scree plot and parallel analysis, it seems reasonable to consider solutions for 3 to 7 factors. Again, we need to check the substantive interpretation to choose the most reasonable solution. To evaluate the substantive interpretability of the different solutions, we need to estimate the full EFA models for each candidate number of factors. We then compare the factor loadings across solutions to see which set of loadings define the most reasonable set of latent variables. 4.4.6 For the trust items, estimate the EFA models for each plausible number of components that you identified above. Use the psych::fa() function to estimate the models. You will need to specify a few key options. The data (including only the variables you want to analyze) The number of factors that you want to extract The rotation method The estimation method The method of estimating factor scores Hint: You can save yourself a lot of typing/copy-pasting (and the attendant chances of errors) by using a for() loop to iterate through numbers of factors. Click to show code ## Define an empty list to hold all of our fitted EFA objects: efa_trust <- list() ## Loop through the interesting numbers of factors and estimate an EFA for each: for(i in 3:6) efa_trust[[as.character(i)]] <- ess %>% select(trstlgl:trstplt) %>% fa(nfactors = i, # Number of factors = Loop index rotate = "promax", # Oblique rotation scores = "Bartlett") # Estimate factor scores with WLS 4.4.7 Repeat the above analysis for the attitudes items. Click to show code efa_att <- list() for(i in 3:7) efa_att[[as.character(i)]] <- ess %>% select(imsmetn:rfgbfml) %>% fa(nfactors = i, rotate = "promax", scores = "Bartlett") 4.4.8 Compare the factor loading matrices from the models estimated from the Trust items, and select the best solution. Hints: The factor loadings are stored in the loadings slot of the object returned by psych::fa(). Looping can also be useful here. Click to show code for(x in efa_trust) print(x$loadings) ## ## Loadings: ## MR3 MR2 MR1 ## trstlgl 0.839 -0.115 ## trstplc 0.763 -0.218 ## trstun 0.579 0.161 ## trstep 0.554 0.198 ## trstprl 0.444 0.342 ## stfhlth 0.656 -0.125 ## stfedu 0.695 -0.157 ## stfeco -0.102 0.704 0.146 ## stfgov 0.593 0.226 ## stfdem 0.183 0.476 0.150 ## pltinvt 0.813 ## pltcare 0.808 ## trstplt 0.330 0.526 ## ## MR3 MR2 MR1 ## SS loadings 2.299 2.016 1.970 ## Proportion Var 0.177 0.155 0.152 ## Cumulative Var 0.177 0.332 0.483 ## ## Loadings: ## MR2 MR1 MR4 MR3 ## trstlgl 0.797 ## trstplc 0.725 ## trstun 0.656 0.113 ## trstep 1.003 -0.175 ## trstprl 0.121 0.455 0.200 0.112 ## stfhlth 0.663 -0.106 ## stfedu 0.704 -0.110 0.100 ## stfeco 0.729 ## stfgov 0.631 0.175 -0.149 ## stfdem 0.501 0.107 0.115 ## pltinvt 0.855 ## pltcare -0.103 0.863 ## trstplt 0.479 0.340 ## ## MR2 MR1 MR4 MR3 ## SS loadings 2.161 1.952 1.722 1.239 ## Proportion Var 0.166 0.150 0.132 0.095 ## Cumulative Var 0.166 0.316 0.449 0.544 ## ## Loadings: ## MR1 MR4 MR5 MR3 MR2 ## trstlgl 0.935 ## trstplc 0.810 ## trstun 0.505 0.168 ## trstep -0.138 1.128 -0.108 -0.154 ## trstprl 0.359 0.250 0.140 0.201 -0.104 ## stfhlth 0.557 ## stfedu 0.752 ## stfeco 0.710 -0.118 0.172 ## stfgov 0.973 -0.132 ## stfdem 0.556 0.153 ## pltinvt 0.882 ## pltcare 0.855 ## trstplt 0.288 0.308 0.313 ## ## MR1 MR4 MR5 MR3 MR2 ## SS loadings 2.019 1.716 1.655 1.674 0.936 ## Proportion Var 0.155 0.132 0.127 0.129 0.072 ## Cumulative Var 0.155 0.287 0.415 0.543 0.615 ## ## Loadings: ## MR5 MR1 MR4 MR3 MR2 MR6 ## trstlgl 0.980 ## trstplc 0.655 ## trstun 0.911 ## trstep -0.116 0.739 0.163 ## trstprl 0.197 0.577 0.138 ## stfhlth 0.614 ## stfedu 0.771 ## stfeco 0.689 -0.123 0.144 ## stfgov 0.891 ## stfdem 0.513 0.144 ## pltinvt 0.816 ## pltcare 0.778 ## trstplt 0.706 0.193 ## ## MR5 MR1 MR4 MR3 MR2 MR6 ## SS loadings 1.606 1.417 1.442 1.327 1.014 0.879 ## Proportion Var 0.124 0.109 0.111 0.102 0.078 0.068 ## Cumulative Var 0.124 0.233 0.343 0.446 0.524 0.591 Click for explanation Note: Any factor loadings with magnitude lower than 0.1 are suppressed in above output. The factor loadings matrix indicates how strongly each latent factor (columns) associates with the observed items (rows). We can interpret these factor loadings in the same way that we would interpret regression coefficients (indeed, a factor analytic model can be viewed as a multivariate regression model wherein the latent factors are the predictors and the observed items are the outcomes). A higher factor loading indicates a stronger association between the item and factor linked by that loading. Items with high factor loadings are “good” indicators of the respective factors. Items with only very low loadings do not provide much information about any factor. You may want to exclude such items from your analysis. Note that the size of the factor loadings depends on the number of factors. So, you should only consider excluding an observed item after you have chosen the number of latent factors. When we print the loading matrix, we see additional information printed below the factor loadings. Proportion Var: What proportion of the items’ variance is explained by each of the factors. Cumulative Var: How much variance the factors explain, in total. If you estimated as many factors as items, then the Cumulative Var for the final factor would be 1.00 (i.e., 100%). 4.4.9 Compare the factor loading matrices from the models estimated from the Attitudes items, and select the best solution. Click to show code for(x in efa_att) print(x$loadings) ## ## Loadings: ## MR1 MR2 MR3 ## imsmetn 0.802 ## imdfetn 0.754 0.106 ## eimrcnt 0.843 ## eimpcnt 0.814 ## imrcntr 0.857 ## impcntr 0.769 ## qfimchr 0.235 0.858 ## qfimwht 0.132 0.719 ## imwgdwn 0.293 -0.181 ## imhecop 0.371 -0.162 ## imtcjob 0.619 ## imbleco 0.702 ## imbgeco 0.687 ## imueclt 0.561 -0.207 ## imwbcnt 0.732 ## imwbcrm 0.637 ## imrsprc -0.494 -0.125 ## pplstrd 0.249 -0.413 ## vrtrlg -0.275 0.240 ## shrrfg 0.514 -0.111 ## rfgawrk -0.386 ## gvrfgap -0.601 -0.148 ## rfgfrpc 0.432 ## rfggvfn -0.489 ## rfgbfml -0.545 ## ## MR1 MR2 MR3 ## SS loadings 4.819 3.950 1.683 ## Proportion Var 0.193 0.158 0.067 ## Cumulative Var 0.193 0.351 0.418 ## ## Loadings: ## MR2 MR4 MR1 MR3 ## imsmetn 0.788 ## imdfetn 0.731 0.153 0.110 ## eimrcnt 0.855 -0.143 ## eimpcnt 0.790 0.165 ## imrcntr 0.860 ## impcntr 0.743 0.182 ## qfimchr -0.122 0.853 ## qfimwht 0.723 ## imwgdwn 0.638 0.264 ## imhecop 0.680 0.217 ## imtcjob 0.633 0.136 ## imbleco 0.563 -0.212 0.153 ## imbgeco 0.604 -0.168 ## imueclt 0.392 -0.236 -0.168 ## imwbcnt 0.526 -0.282 ## imwbcrm 0.397 -0.292 ## imrsprc 0.616 ## pplstrd 0.231 -0.378 ## vrtrlg 0.279 0.264 ## shrrfg 0.299 -0.271 ## rfgawrk 0.452 ## gvrfgap 0.123 0.774 ## rfgfrpc 0.193 -0.281 ## rfggvfn 0.467 ## rfgbfml 0.619 ## ## MR2 MR4 MR1 MR3 ## SS loadings 3.828 2.778 2.570 1.602 ## Proportion Var 0.153 0.111 0.103 0.064 ## Cumulative Var 0.153 0.264 0.367 0.431 ## ## Loadings: ## MR2 MR1 MR5 MR3 MR4 ## imsmetn 0.792 ## imdfetn 0.728 0.169 0.113 ## eimrcnt 0.910 -0.150 -0.237 ## eimpcnt 0.779 0.126 0.213 ## imrcntr 0.910 -0.128 -0.187 ## impcntr 0.731 0.131 0.236 ## qfimchr 0.109 -0.156 0.882 ## qfimwht 0.139 0.736 ## imwgdwn 0.740 ## imhecop 0.700 ## imtcjob 0.543 0.124 0.182 ## imbleco 0.682 0.135 ## imbgeco 0.799 ## imueclt 0.572 -0.202 ## imwbcnt 0.712 ## imwbcrm 0.545 -0.124 ## imrsprc 0.620 ## pplstrd 0.207 -0.396 ## vrtrlg -0.198 0.151 0.285 0.116 ## shrrfg 0.208 -0.263 0.139 ## rfgawrk 0.457 ## gvrfgap 0.783 ## rfgfrpc -0.338 0.156 ## rfggvfn 0.477 ## rfgbfml -0.125 0.538 ## ## MR2 MR1 MR5 MR3 MR4 ## SS loadings 3.970 2.790 2.215 1.693 1.166 ## Proportion Var 0.159 0.112 0.089 0.068 0.047 ## Cumulative Var 0.159 0.270 0.359 0.427 0.473 ## ## Loadings: ## MR2 MR1 MR6 MR3 MR5 MR4 ## imsmetn 0.705 0.166 ## imdfetn 0.833 ## eimrcnt 0.249 0.859 ## eimpcnt 0.946 ## imrcntr 0.456 0.517 ## impcntr 0.951 ## qfimchr 0.134 -0.122 0.875 ## qfimwht 0.151 0.725 ## imwgdwn 0.748 ## imhecop 0.678 ## imtcjob 0.566 0.123 0.175 ## imbleco 0.753 0.144 ## imbgeco 0.822 ## imueclt 0.580 -0.201 ## imwbcnt 0.751 ## imwbcrm 0.597 ## imrsprc 0.146 0.527 ## pplstrd 0.204 -0.392 ## vrtrlg -0.204 0.143 0.281 0.115 ## shrrfg 0.198 -0.275 0.141 ## rfgawrk 0.517 ## gvrfgap 0.784 ## rfgfrpc -0.294 0.144 ## rfggvfn 0.512 ## rfgbfml 0.596 ## ## MR2 MR1 MR6 MR3 MR5 MR4 ## SS loadings 3.304 3.013 1.994 1.649 1.065 1.133 ## Proportion Var 0.132 0.121 0.080 0.066 0.043 0.045 ## Cumulative Var 0.132 0.253 0.332 0.398 0.441 0.486 ## ## Loadings: ## MR2 MR1 MR6 MR3 MR5 MR7 MR4 ## imsmetn 0.700 0.162 ## imdfetn 0.821 ## eimrcnt 0.245 0.879 ## eimpcnt 0.935 ## imrcntr 0.452 0.523 ## impcntr 0.938 ## qfimchr 0.751 ## qfimwht 0.720 ## imwgdwn 0.700 ## imhecop 0.172 0.624 ## imtcjob 0.574 -0.120 0.174 ## imbleco 0.679 0.108 ## imbgeco 0.832 -0.145 ## imueclt 0.531 -0.191 ## imwbcnt 0.649 0.138 ## imwbcrm 0.464 0.131 0.290 ## imrsprc 0.146 0.440 -0.100 ## pplstrd -0.274 0.392 ## vrtrlg -0.121 0.190 -0.297 0.115 ## shrrfg -0.124 0.437 0.131 ## rfgawrk 0.538 ## gvrfgap 0.616 -0.237 ## rfgfrpc -0.131 0.437 0.135 ## rfggvfn 0.504 ## rfgbfml 0.526 ## ## MR2 MR1 MR6 MR3 MR5 MR7 MR4 ## SS loadings 3.224 2.467 1.456 1.305 1.105 0.901 0.984 ## Proportion Var 0.129 0.099 0.058 0.052 0.044 0.036 0.039 ## Cumulative Var 0.129 0.228 0.286 0.338 0.382 0.418 0.458 It is very possible that you selected a different numbers of factors than Kestilä (2006). We need to keep these exercises consistent, though. So, the remaining questions will all assume you have extract three factors from the Trust items and five factors from the Attitudes items, to parallel the Kestilä (2006) results. ## Select the three-factor solution for 'trust': efa_trust <- efa_trust[["3"]] ## Select the five-factor solution for 'attitudes': efa_att <- efa_att[["5"]] 4.4.10 Give the factor scores meaningful names, and add the scores to the ess dataset as new columns. Hint: If you’re not sure of what do to, check 4.3.11. Click to show code ## Rename the factor scores: colnames(efa_trust$scores) <- c("trust_inst", "satisfy", "trust_pol") colnames(efa_att$scores) <- c("effects", "allowance", "refugees", "ethnic", "europe") ## Add factor scores to the dataset as new columns: ess <- data.frame(ess, efa_trust$scores, efa_att$scores) Kestilä (2006) used the component scores to descriptively evaluate country-level differences in Attitudes toward Immigration and Political Trust. So, now it’s time to replicate those analyses. 4.4.11 Repeat the Kestilä (2006) between-country comparison using the factor scores you created in 4.4.10 and an appropriate statistical test. Click to show code Here, we’ll only demonstrate a possible approach to analyzing one of the Trust dimensions. We can use a linear model to test whether the countries differ in average levels of Trust in Institutions (as quantified by the relevant factor score). ## Estimate the model: out <- lm(trust_inst ~ country, data = ess) ## View the regression-style summary: summary(out) ## ## Call: ## lm(formula = trust_inst ~ country, data = ess) ## ## Residuals: ## Min 1Q Median 3Q Max ## -4.2295 -0.6226 0.1171 0.7194 3.3061 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -0.09028 0.02445 -3.692 0.000224 *** ## countryBelgium -0.28923 0.03642 -7.942 2.12e-15 *** ## countryGermany -0.05966 0.03211 -1.858 0.063205 . ## countryDenmark 0.75509 0.03882 19.452 < 2e-16 *** ## countryFinland 0.59235 0.03439 17.224 < 2e-16 *** ## countryItaly 0.10991 0.04071 2.700 0.006939 ** ## countryNetherlands -0.05357 0.03379 -1.585 0.112893 ## countryNorway 0.36922 0.03493 10.570 < 2e-16 *** ## countrySweden 0.28560 0.03613 7.904 2.89e-15 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 1.029 on 14769 degrees of freedom ## (4912 observations deleted due to missingness) ## Multiple R-squared: 0.082, Adjusted R-squared: 0.0815 ## F-statistic: 164.9 on 8 and 14769 DF, p-value: < 2.2e-16 ## View the results as an ANOVA table: anova(out) ## Post-hoc tests out %>% aov() %>% TukeyHSD() ## Tukey multiple comparisons of means ## 95% family-wise confidence level ## ## Fit: aov(formula = .) ## ## $country ## diff lwr upr p adj ## Belgium-Austria -0.289225482 -0.40219224 -0.17625873 0.0000000 ## Germany-Austria -0.059655996 -0.15926604 0.03995405 0.6429963 ## Denmark-Austria 0.755089552 0.63466911 0.87551000 0.0000000 ## Finland-Austria 0.592348290 0.48565882 0.69903776 0.0000000 ## Italy-Austria 0.109910185 -0.01636587 0.23618624 0.1476635 ## Netherlands-Austria -0.053567808 -0.15838407 0.05124846 0.8131104 ## Norway-Austria 0.369224250 0.26085692 0.47759158 0.0000000 ## Sweden-Austria 0.285601197 0.17350905 0.39769334 0.0000000 ## Germany-Belgium 0.229569486 0.12386351 0.33527546 0.0000000 ## Denmark-Belgium 1.044315033 0.91880537 1.16982470 0.0000000 ## Finland-Belgium 0.881573772 0.76917165 0.99397589 0.0000000 ## Italy-Belgium 0.399135667 0.26799745 0.53027389 0.0000000 ## Netherlands-Belgium 0.235657673 0.12503199 0.34628336 0.0000000 ## Norway-Belgium 0.658449732 0.54445381 0.77244566 0.0000000 ## Sweden-Belgium 0.574826679 0.45728417 0.69236918 0.0000000 ## Denmark-Germany 0.814745547 0.70110863 0.92838247 0.0000000 ## Finland-Germany 0.652004286 0.55303505 0.75097352 0.0000000 ## Italy-Germany 0.169566181 0.04974170 0.28939066 0.0003895 ## Netherlands-Germany 0.006088188 -0.09085878 0.10303516 0.9999999 ## Norway-Germany 0.428880246 0.32810453 0.52965596 0.0000000 ## Sweden-Germany 0.345257193 0.24048642 0.45002796 0.0000000 ## Finland-Denmark -0.162741262 -0.28263218 -0.04285034 0.0008579 ## Italy-Denmark -0.645179366 -0.78279052 -0.50756821 0.0000000 ## Netherlands-Denmark -0.808657360 -0.92688442 -0.69043030 0.0000000 ## Norway-Denmark -0.385865301 -0.50725174 -0.26447886 0.0000000 ## Sweden-Denmark -0.469488354 -0.59421139 -0.34476531 0.0000000 ## Italy-Finland -0.482438105 -0.60820928 -0.35666693 0.0000000 ## Netherlands-Finland -0.645916098 -0.75012357 -0.54170862 0.0000000 ## Norway-Finland -0.223124040 -0.33090264 -0.11534544 0.0000000 ## Sweden-Finland -0.306747093 -0.41827017 -0.19522402 0.0000000 ## Netherlands-Italy -0.163477993 -0.28766412 -0.03929186 0.0014719 ## Norway-Italy 0.259314065 0.13211649 0.38651164 0.0000000 ## Sweden-Italy 0.175691012 0.04530545 0.30607657 0.0009794 ## Norway-Netherlands 0.422792059 0.31686740 0.52871671 0.0000000 ## Sweden-Netherlands 0.339169005 0.22943659 0.44890142 0.0000000 ## Sweden-Norway -0.083623053 -0.19675232 0.02950622 0.3462227 Click for explanation According to the omnibus F-test, average levels of Trust in Institutions significantly differ between countries, but this test cannot tell us between which countries the differences lie. Similarly, the t statistics associated with each dummy code in the regression-style summary only tell us if that country differs significantly from the reference country (i.e., Austria), but we cannot see, for example, if there is a significant difference in average trust levels between Belgium and the Netherlands. One way to test for differences between the individual countries would be a post hoc test of all pairwise comparisons. Since we’ll be doing 45 tests, we need to apply a correction for repeated testing. Above, we use the TukeyHSD() function to conduct all pairwise comparisons while applying Tukey’s HSD correction. The TukeyHSD() function only accepts models estimated with the aov() function, so we first pass our fitted lm object through aov(). The second part of the Kestilä (2006) analysis was to evaluate how socio-demographic characteristics affected attitudes towards immigrants and trust in politics among the Finnish electorate. Before we can replicate this part of the analysis, we need to subset the data to only the Finnish cases. 4.4.12 Create a new data frame that contains only the Finnish cases from ess. Hint: You can use logical indexing based on the country variable. Click to show code ess_finland <- filter(ess, country == "Finland") We still have one more step before we can estimate any models. We must prepare our variables for analysis. Our dependent variables will be the factor scores generated above. So, we do not need to apply any further processing. We have not yet used any of the independent variables, though. So, we should inspect those variables to see if they require any processing. In our processed ess data, the relevant variables have the following names: sex yrbrn eduyrs polintr lrscale 4.4.13 Inspect the independent variables listed above. Click to show code library(tidySEM) select(ess_finland, sex, yrbrn, eduyrs, polintr, lrscale) %>% descriptives() Click for explanation It looks like we still need some recoding. 4.4.14 Apply any necessary recoding/transformations. 1. Age Click to show code ess_finland <- mutate(ess_finland, age = 2002 - yrbrn) Click for explanation The data contain the participants’ years of birth instead of their age, but Kestilä analyzed age. Fortunately, we know that the data were collected in 2002, so we can simply subtract each participant’s value of yrbrn from the 2002 to compute their age. 2. Political Interest Click to show code First, we’ll transform polintr. ## Recode the four factor levels into two factor levels: ess_finland <- mutate(ess_finland, polintr_bin = recode_factor(polintr, "Not at all interested" = "Low Interest", "Hardly interested" = "Low Interest", "Quite interested" = "High Interest", "Very interested" = "High Interest") ) ## Check the conversion: with(ess_finland, table(old = polintr, new = polintr_bin, useNA = "always")) ## new ## old Low Interest High Interest <NA> ## Very interested 0 144 0 ## Quite interested 0 785 0 ## Hardly interested 842 0 0 ## Not at all interested 228 0 0 ## <NA> 0 0 1 Click for explanation Kestilä (2006) dichotomized polintr by combining the lowest two and highest two categories. So, we don’t actually want to convert the polint variable into a numeric, Likert-type variable. We want polint to be a binary factor. The recode_factor() function from dplyr() will automatically convert our result into a factor. As with the ess_round1.rds data, we will be coming back to this Finnish subsample data in future practical exercises. So, we should save our work by writing the processed dataset to disk. 4.4.15 Use the saveRDS() function to save the processed Finnish subsample data. Click to see code ## Save the processed Finnish data: saveRDS(ess_finland, "ess_finland.rds") Now, we’re finally ready to replicate the regression analysis from Kestilä (2006). Creating a single aggregate score by summing the individual component scores is a pretty silly thing to do, though. So, we won’t reproduce that aspect of the analysis. 4.4.16 Run a series of multiple linear regression analyses with the factor scores you created in 4.4.10 as the dependent variables and the same predictors used by Kestilä (2006). Do your results agree with those reported by Kestilä (2006)? Click to show code ## Predicting 'Trust in Institutions': out_trust_inst <- lm(trust_inst ~ sex + age + eduyrs + polintr_bin + lrscale, data = ess_finland) summary(out_trust_inst) ## ## Call: ## lm(formula = trust_inst ~ sex + age + eduyrs + polintr_bin + ## lrscale, data = ess_finland) ## ## Residuals: ## Min 1Q Median 3Q Max ## -3.9499 -0.5102 0.1337 0.6638 2.5919 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -0.057518 0.124294 -0.463 0.643595 ## sexFemale 0.004091 0.045170 0.091 0.927849 ## age -0.003071 0.001380 -2.225 0.026219 * ## eduyrs 0.023223 0.006388 3.635 0.000286 *** ## polintr_binHigh Interest 0.166860 0.046448 3.592 0.000337 *** ## lrscale 0.058951 0.011232 5.249 1.72e-07 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 0.9321 on 1734 degrees of freedom ## (260 observations deleted due to missingness) ## Multiple R-squared: 0.04155, Adjusted R-squared: 0.03879 ## F-statistic: 15.03 on 5 and 1734 DF, p-value: 1.78e-14 ## Predicting 'Trust in Politicians': out_trust_pol <- lm(trust_pol ~ sex + age + eduyrs + polintr_bin + lrscale, data = ess_finland) summary(out_trust_pol) ## ## Call: ## lm(formula = trust_pol ~ sex + age + eduyrs + polintr_bin + lrscale, ## data = ess_finland) ## ## Residuals: ## Min 1Q Median 3Q Max ## -3.03673 -0.67306 0.05346 0.69666 2.38771 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -0.165989 0.126840 -1.309 0.19083 ## sexFemale 0.015572 0.046095 0.338 0.73554 ## age -0.009112 0.001409 -6.469 1.28e-10 *** ## eduyrs 0.018476 0.006519 2.834 0.00465 ** ## polintr_binHigh Interest 0.463763 0.047399 9.784 < 2e-16 *** ## lrscale 0.054932 0.011462 4.793 1.79e-06 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 0.9512 on 1734 degrees of freedom ## (260 observations deleted due to missingness) ## Multiple R-squared: 0.09806, Adjusted R-squared: 0.09546 ## F-statistic: 37.71 on 5 and 1734 DF, p-value: < 2.2e-16 ## Predicting 'Attitudes toward Refugees': out_refugees <- lm(refugees ~ sex + age + eduyrs + polintr_bin + lrscale, data = ess_finland) summary(out_refugees) ## ## Call: ## lm(formula = refugees ~ sex + age + eduyrs + polintr_bin + lrscale, ## data = ess_finland) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.9118 -0.6860 -0.0594 0.6904 4.1044 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) -1.690e-01 1.438e-01 -1.175 0.240080 ## sexFemale -4.828e-01 5.181e-02 -9.318 < 2e-16 *** ## age 2.903e-05 1.604e-03 0.018 0.985561 ## eduyrs -2.537e-02 7.459e-03 -3.401 0.000688 *** ## polintr_binHigh Interest -2.131e-01 5.345e-02 -3.986 6.99e-05 *** ## lrscale 9.359e-02 1.296e-02 7.223 7.65e-13 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 1.06 on 1699 degrees of freedom ## (295 observations deleted due to missingness) ## Multiple R-squared: 0.09535, Adjusted R-squared: 0.09269 ## F-statistic: 35.81 on 5 and 1699 DF, p-value: < 2.2e-16 That does it for our replication of the Kesilä (2006) analyses, but we still have one more topic to consider in this practical. One of the most common applications of EFA is scale development. Given a pool of items without a known factor structure, we try to estimate the underlying latent factors that define the (sub)scales represented by our items. In such applications, we use the factor loading matrix for our optimal solution to make “bright-line” assignments of items to putative factors according to the simple structure represented by the estimated factor loading matrix. In other words, we disregard small factor loadings and assign observed items to only the single latent factor upon which they load most strongly. We then hypothesize that those items are true indicators of that latent factor. We can use confirmatory factor analysis (which you will learn about next week) to test rigorously this hypothesis, but we can already get started by estimating the internal consistency (a type of reliability) of the hypothesized subscales. 4.4.17 Estimate the internal consistency of the three Trust subscales and five Attitudes subscales implied by your EFA solutions from above. Use Cronbach’s Alpha to quantify internal consistency. Use the alpha() function from the psych package to conduct the analysis. Run your analysis on the full ess dataset, not the Finnish subset. Are the subscales implied by your EFA reliable, in the sense of good internal consistency? Note that \\(\\alpha > 0.7\\) is generally considered acceptable, and \\(\\alpha > 0.8\\) is usually considered good. Click to show code ## Run the reliability analysis on the subscale data: ( out <- select(ess, starts_with("stf")) %>% psych::alpha() ) ## ## Reliability analysis ## Call: psych::alpha(x = .) ## ## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r ## 0.79 0.79 0.77 0.44 3.9 0.0023 5.4 1.7 0.41 ## ## 95% confidence boundaries ## lower alpha upper ## Feldt 0.79 0.79 0.8 ## Duhachek 0.79 0.79 0.8 ## ## Reliability if an item is dropped: ## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r ## stfhlth 0.78 0.78 0.73 0.47 3.5 0.0026 0.0064 0.46 ## stfedu 0.76 0.76 0.72 0.45 3.2 0.0028 0.0109 0.44 ## stfeco 0.74 0.74 0.70 0.41 2.8 0.0031 0.0069 0.39 ## stfgov 0.74 0.74 0.69 0.42 2.9 0.0030 0.0035 0.41 ## stfdem 0.75 0.75 0.71 0.43 3.0 0.0029 0.0074 0.40 ## ## Item statistics ## n raw.r std.r r.cor r.drop mean sd ## stfhlth 19481 0.69 0.69 0.56 0.50 5.8 2.3 ## stfedu 18844 0.73 0.73 0.62 0.55 5.9 2.3 ## stfeco 19211 0.78 0.78 0.70 0.63 5.0 2.4 ## stfgov 19106 0.77 0.76 0.69 0.61 4.5 2.3 ## stfdem 19106 0.75 0.75 0.67 0.59 5.7 2.3 Click for explanation Here, we estimate the reliability of the Satisfaction subscale from the Trust analysis. According to our EFA, the Satisfaction subscale should be indicated by the following five variables: stfeco stfgov stfdem stfedu stfhlth We select these variables using the tidy-select function starts_with() to extract all variables beginning with the three characters “stf”. To estimate the internal consistency of this subscale, we simply provide a data frame containing only the subscale data to the alpha() function. The raw_alpha value is the estimate of Cronbach’s Alpha. In this case \\(\\alpha = 0.794\\), so the subscale is pretty reliable. The table labeled “Reliability if an item is dropped” shows what Cronbach’s Alpha would be if each item were excluded from the scale. If this value is notably higher than the raw_alpha value, it could indicate a bad item. Note that reliability is only one aspect of scale quality, though. So, you shouldn’t throw out items just because they perform poorly in reliability analysis. End of In-Class Exercises "],["cfa.html", "5 CFA", " 5 CFA This week, we will introduce confirmatory factor analysis (CFA) and discuss how it differs from EFA. Furthermore, we will revisit the idea of model fit and introduce into the R-package lavaan. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-4.html", "5.1 Lecture", " 5.1 Lecture Often, we work with scales that have a validated or hypothesized factor structure. In the former case, the scale structure has been validated through previous psychometric studies. In the latter case, we may have conducted an EFA to estimate the factor structure on prior data, or theory/intuition may suggest a plausible structure. Regardless of how we come to expect a given factor structure, such situations represent confirmatory modeling problems, because we are attempting to empirically confirm an a priori expectation. Hence, exploratory methods like EFA are not appropriate, and we should employ confirmatory modeling techniques. This week we consider one such technique: confirmatory factor analysis (CFA). As the name suggests, CFA is related to the EFA methods we discussed last week in that both methods are flavors of factor analysis. However, the two methods address fundamentally different research questions. Rather than attempting to estimate an unknown factor structure (as in EFA), we now want to compare a hypothesized measurement model (i.e., factor structure) to observed data in order to evaluate the model’s plausibility. 5.1.1 Recording Note: When Caspar discusses the complexity of the second-order CFA model, it’s easy to misunderstand his statements. We need to be careful not to over-generalize. In general, a second-order CFA is not more complex than a first-order CFA. Actually, in most practical applications, the opposite is true. A second-order CFA is more complex than a first-order CFA, when the factors in the first-order CFA are uncorrelated. This is the situation Caspar references in the recording when claiming that the second-order model is more complex. We hardly ever want to fit such first-order CFA, though. The default CFA fully saturates the latent covariance structure. If the factors in the first-order CFA are fully correlated (according to standard practice), and we include a single second-order factor, the following statements hold. If the first-order CFA has more than three factors, the first-order model is more complex than the second-order model. If the first-order model has three or fewer factors, the first- and second-order models are equivalent (due to scaling constraints we need to impose to identify the second-order model). The second-order model cannot be more complex than the first-order model (assuming both models are correctly identified and no extra constraints are imposed). The above statements may not hold in more complex situations (e.g., more than one second-order factor, partially saturated first-order correlation structure, etc.). You can always identify the more complex model by calculating the degrees of freedom for both models. The model with fewer degrees of freedom is more complex. 5.1.2 Slides You can download the lecture slides here "],["reading-4.html", "5.2 Reading", " 5.2 Reading Reference Byrne, B. (2005). Factor analytic models: Viewing the structure of an assessment instrument from three perspectives, Journal of Personality Assessment, 85(1), 17–32. Questions What are the main differences between exploratory factor analysis (EFA) and confirmatory factor analysis (CFA)? In which circumstances should a researcher use EFA, and in which should they use CFA? What are the five main limitations of EFA that CFA overcomes? In which circumstances can a second order CFA model be useful? Consider the following four techniques: PCA, EFA, CFA, second order CFA. For each of the following three research situations, which of the above techniques would you use and why? A researcher has developed a new questionnaire that should measure personality and wants to know how many factors underlie the items in their new measure. A researcher is modeling data collected with a seven-item scale that has been used since the 1960s to measure authoritarianism. A researcher has recorded highest completed level of education, years of education, and highest level of education attempted for all respondents in a survey. The researcher wants to include some operationalization of the concept of ‘education’ in their model but is unsure of which observed variable to use. "],["at-home-exercises-4.html", "5.3 At-Home Exercises", " 5.3 At-Home Exercises This week, we will wrap up our re-analysis of the Kestilä (2006) results. During this practical, you will conduct a CFA of the Trust in Politics items and compare the results to those obtained from your previous EFA- and PCA-based replications of Kestilä (2006). 5.3.1 Load the ESS data. The relevant data are contained in the ess_round1.rds file. This file is in R Data Set (RDS) format. The dataset is already stored as a data frame with the processing and cleaning that you should have done for previous practicals completed. Click to show code ess <- readRDS("ess_round1.rds") Although you may have settled on any number of EFA solutions during the Week 4 In-Class Exercises, we are going to base the following CFA on a three-factor model of Trust in Politics similar to the original PCA results from Kestilä (2006). Note: Unless otherwise specified, all following questions refer to the Trust in Politics items. We will not consider the Attitudes toward Immigration items in these exercises. 5.3.2 Define the lavaan model syntax for the CFA implied by the three-factor EFA solution you found in the Week 4 In-Class Exercises. Covary the three latent factors. Do not specify any mean structure. Save this model syntax as an object in your environment. Click to show code mod_3f <- ' institutions =~ trstlgl + trstplc + trstun + trstep + trstprl satisfaction =~ stfhlth + stfedu + stfeco + stfgov + stfdem politicians =~ pltinvt + pltcare + trstplt ' Click for explanation We don’t have to specify the latent covariances in the model syntax, we can tell lavaan to estimate all latent covariances when we fit the model. 5.3.3 Estimate the CFA model you defined above, and summarize the results. Use the lavaan::cfa() function to estimate the model. Use the default settings for the cfa() function. Request the model fit statistics with the summary by supplying the fit.measures = TRUE argument to summary(). Request the standardized parameter estimates with the summary by supplying the standardized = TRUE argument to summary(). Check the results, and answer the following questions: Does the model fit the data well? How are the latent variances and covariances specified when using the default settings? How is the model identified when using the default settings? Click the code ## Load the lavaan package: library(lavaan) ## Estimate the CFA model: fit_3f <- cfa(mod_3f, data = ess) ## Summarize the fitted model: summary(fit_3f, fit.measures = TRUE, standardized = TRUE) ## lavaan 0.6.16 ended normally after 46 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 29 ## ## Used Total ## Number of observations 14778 19690 ## ## Model Test User Model: ## ## Test statistic 10652.207 ## Degrees of freedom 62 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 81699.096 ## Degrees of freedom 78 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.870 ## Tucker-Lewis Index (TLI) 0.837 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -371404.658 ## Loglikelihood unrestricted model (H1) -366078.555 ## ## Akaike (AIC) 742867.317 ## Bayesian (BIC) 743087.743 ## Sample-size adjusted Bayesian (SABIC) 742995.583 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.108 ## 90 Percent confidence interval - lower 0.106 ## 90 Percent confidence interval - upper 0.109 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 1.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.059 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions =~ ## trstlgl 1.000 1.613 0.677 ## trstplc 0.770 0.012 61.866 0.000 1.241 0.567 ## trstun 0.929 0.013 69.227 0.000 1.498 0.642 ## trstep 0.908 0.013 70.929 0.000 1.464 0.660 ## trstprl 1.139 0.014 84.084 0.000 1.837 0.809 ## satisfaction =~ ## stfhlth 1.000 1.173 0.521 ## stfedu 1.106 0.022 50.840 0.000 1.297 0.577 ## stfeco 1.415 0.025 57.214 0.000 1.659 0.713 ## stfgov 1.480 0.025 58.764 0.000 1.736 0.756 ## stfdem 1.384 0.024 57.904 0.000 1.623 0.731 ## politicians =~ ## pltinvt 1.000 0.646 0.613 ## pltcare 1.021 0.016 62.862 0.000 0.660 0.628 ## trstplt 3.012 0.039 76.838 0.000 1.946 0.891 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions ~~ ## satisfaction 1.391 0.032 43.206 0.000 0.736 0.736 ## politicians 0.909 0.018 49.934 0.000 0.872 0.872 ## satisfaction ~~ ## politicians 0.539 0.013 41.053 0.000 0.711 0.711 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trstlgl 3.068 0.041 75.262 0.000 3.068 0.541 ## .trstplc 3.248 0.041 80.037 0.000 3.248 0.678 ## .trstun 3.197 0.041 77.141 0.000 3.197 0.588 ## .trstep 2.776 0.036 76.243 0.000 2.776 0.564 ## .trstprl 1.776 0.029 61.361 0.000 1.776 0.345 ## .stfhlth 3.695 0.046 79.989 0.000 3.695 0.729 ## .stfedu 3.368 0.043 77.916 0.000 3.368 0.667 ## .stfeco 2.656 0.038 69.070 0.000 2.656 0.491 ## .stfgov 2.264 0.035 64.201 0.000 2.264 0.429 ## .stfdem 2.289 0.034 67.172 0.000 2.289 0.465 ## .pltinvt 0.694 0.009 78.255 0.000 0.694 0.624 ## .pltcare 0.668 0.009 77.562 0.000 0.668 0.605 ## .trstplt 0.978 0.028 34.461 0.000 0.978 0.205 ## institutions 2.601 0.059 44.198 0.000 1.000 1.000 ## satisfaction 1.375 0.044 31.407 0.000 1.000 1.000 ## politicians 0.417 0.011 38.843 0.000 1.000 1.000 Click for explanation No, the model does not seem to fit the data well. The SRMR looks good, but one good looking fit statistic is not enough. The RMSEA, TLI, and CFI are all in the “unacceptable” range. The \\(\\chi^2\\) is highly significant, but we don’t care. The cfa() function is just a wrapper for the lavaan() function with several options set at the defaults you would want for a standard CFA. By default: All latent variances and covariances are freely estimated (due to the argument auto.cov.lv.x = TRUE) The model is identified by fixing the first factor loading of each factor to 1 (due to the argument auto.fix.first = TRUE) To see a full list of the (many) options you can specify to tweak the behavior of lavaan estimation functions run ?lavOptions. Now, we will consider a couple of alternative factor structures for the Trust in Politics CFA. First, we will go extremely simple by estimating a one-factor model wherein all Trust items are explained by a single latent variable. 5.3.4 Define the lavaan model syntax for a one-factor model of the Trust items. Save this syntax as an object in your environment. Click to show code mod_1f <- ' political_trust =~ trstlgl + trstplc + trstun + trstep + trstprl + stfhlth + stfedu + stfeco + stfgov + stfdem + pltinvt + pltcare + trstplt ' 5.3.5 Estimate the one-factor model, and summarize the results. Does this model appear to fit better or worse than the three-factor model? Note: You can use the lavaan::fitMeasures() function to extract only the model fit information from a fitted lavaan object. Click to show code ## Estimate the one factor model: fit_1f <- cfa(mod_1f, data = ess) ## Summarize the results: summary(fit_1f, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 33 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 26 ## ## Used Total ## Number of observations 14778 19690 ## ## Model Test User Model: ## ## Test statistic 17667.304 ## Degrees of freedom 65 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 81699.096 ## Degrees of freedom 78 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.784 ## Tucker-Lewis Index (TLI) 0.741 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -374912.206 ## Loglikelihood unrestricted model (H1) -366078.555 ## ## Akaike (AIC) 749876.413 ## Bayesian (BIC) 750074.036 ## Sample-size adjusted Bayesian (SABIC) 749991.410 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.135 ## 90 Percent confidence interval - lower 0.134 ## 90 Percent confidence interval - upper 0.137 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 1.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.080 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## political_trust =~ ## trstlgl 1.000 ## trstplc 0.774 0.013 57.949 0.000 ## trstun 0.930 0.014 64.200 0.000 ## trstep 0.909 0.014 65.679 0.000 ## trstprl 1.182 0.015 79.401 0.000 ## stfhlth 0.615 0.013 45.947 0.000 ## stfedu 0.695 0.014 51.424 0.000 ## stfeco 0.895 0.014 62.316 0.000 ## stfgov 0.985 0.014 68.200 0.000 ## stfdem 0.998 0.014 70.899 0.000 ## pltinvt 0.382 0.006 59.215 0.000 ## pltcare 0.396 0.006 61.195 0.000 ## trstplt 1.183 0.014 81.716 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .trstlgl 3.370 0.042 79.787 0.000 ## .trstplc 3.410 0.041 82.311 0.000 ## .trstun 3.451 0.043 80.749 0.000 ## .trstep 3.019 0.038 80.272 0.000 ## .trstprl 1.938 0.027 70.878 0.000 ## .stfhlth 4.201 0.050 84.093 0.000 ## .stfedu 3.941 0.047 83.419 0.000 ## .stfeco 3.565 0.044 81.289 0.000 ## .stfgov 3.044 0.038 79.326 0.000 ## .stfdem 2.631 0.034 78.072 0.000 ## .pltinvt 0.775 0.009 82.043 0.000 ## .pltcare 0.743 0.009 81.579 0.000 ## .trstplt 1.548 0.023 67.052 0.000 ## political_trst 2.299 0.055 41.569 0.000 ## Compare fit statistics: fitMeasures(fit_3f) ## npar fmin chisq ## 29.000 0.360 10652.207 ## df pvalue baseline.chisq ## 62.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.870 ## tli nnfi rfi ## 0.837 0.837 0.836 ## nfi pnfi ifi ## 0.870 0.691 0.870 ## rni logl unrestricted.logl ## 0.870 -371404.658 -366078.555 ## aic bic ntotal ## 742867.317 743087.743 14778.000 ## bic2 rmsea rmsea.ci.lower ## 742995.583 0.108 0.106 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.109 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.255 0.255 0.059 ## srmr_bentler srmr_bentler_nomean crmr ## 0.059 0.059 0.064 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.064 0.059 0.059 ## cn_05 cn_01 gfi ## 113.901 126.971 0.897 ## agfi pgfi mfi ## 0.849 0.611 0.699 ## ecvi ## 0.725 fitMeasures(fit_1f) ## npar fmin chisq ## 26.000 0.598 17667.304 ## df pvalue baseline.chisq ## 65.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.784 ## tli nnfi rfi ## 0.741 0.741 0.741 ## nfi pnfi ifi ## 0.784 0.653 0.784 ## rni logl unrestricted.logl ## 0.784 -374912.206 -366078.555 ## aic bic ntotal ## 749876.413 750074.036 14778.000 ## bic2 rmsea rmsea.ci.lower ## 749991.410 0.135 0.134 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.137 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.364 0.364 0.080 ## srmr_bentler srmr_bentler_nomean crmr ## 0.080 0.080 0.087 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.087 0.080 0.080 ## cn_05 cn_01 gfi ## 71.949 79.980 0.825 ## agfi pgfi mfi ## 0.756 0.590 0.551 ## ecvi ## 1.199 Click for explanation The one-factor model definitely seems to fit worse than the three-factor model. A second order CFA model is another way of representing the latent structure underlying a set of items. As you read in Byrne (2005), however, the second order CFA is only appropriate in certain circumstances. 5.3.6 Given the CFA results above, would a second order CFA be appropriate for the Trust data? Why or why not? Click for explanation Yes, a second order CFA model is a theoretically appropriate representation of the Trust items. The first order latent variables in the three-factor model are all significantly correlated. The first order latent variables in the three-factor model seem to tap different aspects of some single underlying construct. 5.3.7 Define the lavaan model syntax for a second-order CFA model of the Trust items. Use the three factors defined in 5.3.2 as the first order factors. Click to show code mod_2nd <- ' institutions =~ trstlgl + trstplc + trstun + trstep + trstprl satisfaction =~ stfhlth + stfedu + stfeco + stfgov + stfdem politicians =~ pltinvt + pltcare + trstplt trust =~ politicians + satisfaction + institutions ' Click for explanation To define the second order factor, we use the same syntactic conventions that we employ to define a first order factor. The only differences is that the “indicators” of the second order factor (i.e., the variables listed on the RHS of the =~ operator) are previously defined first order latent variables. 5.3.8 Estimate the second order CFA model, and summarize the results. Does this model fit better or worse than the three-factor model? Is this model more or less complex than the three-factor model? What information can you use to quantify this difference in complexity? Click to show code fit_2nd <- cfa(mod_2nd, data = ess) summary(fit_2nd, fit.measures = TRUE, standardized = TRUE) ## lavaan 0.6.16 ended normally after 44 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 29 ## ## Used Total ## Number of observations 14778 19690 ## ## Model Test User Model: ## ## Test statistic 10652.207 ## Degrees of freedom 62 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 81699.096 ## Degrees of freedom 78 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.870 ## Tucker-Lewis Index (TLI) 0.837 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -371404.658 ## Loglikelihood unrestricted model (H1) -366078.555 ## ## Akaike (AIC) 742867.317 ## Bayesian (BIC) 743087.743 ## Sample-size adjusted Bayesian (SABIC) 742995.583 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.108 ## 90 Percent confidence interval - lower 0.106 ## 90 Percent confidence interval - upper 0.109 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 1.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.059 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions =~ ## trstlgl 1.000 1.613 0.677 ## trstplc 0.770 0.012 61.866 0.000 1.241 0.567 ## trstun 0.929 0.013 69.227 0.000 1.498 0.642 ## trstep 0.908 0.013 70.929 0.000 1.464 0.660 ## trstprl 1.139 0.014 84.084 0.000 1.837 0.809 ## satisfaction =~ ## stfhlth 1.000 1.173 0.521 ## stfedu 1.106 0.022 50.840 0.000 1.297 0.577 ## stfeco 1.415 0.025 57.214 0.000 1.659 0.713 ## stfgov 1.480 0.025 58.764 0.000 1.736 0.756 ## stfdem 1.384 0.024 57.904 0.000 1.623 0.731 ## politicians =~ ## pltinvt 1.000 0.646 0.613 ## pltcare 1.021 0.016 62.862 0.000 0.660 0.628 ## trstplt 3.012 0.039 76.838 0.000 1.946 0.891 ## trust =~ ## politicians 1.000 0.918 0.918 ## satisfaction 1.531 0.033 46.494 0.000 0.774 0.774 ## institutions 2.583 0.045 56.796 0.000 0.950 0.950 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trstlgl 3.068 0.041 75.262 0.000 3.068 0.541 ## .trstplc 3.248 0.041 80.037 0.000 3.248 0.678 ## .trstun 3.197 0.041 77.141 0.000 3.197 0.588 ## .trstep 2.776 0.036 76.243 0.000 2.776 0.564 ## .trstprl 1.776 0.029 61.361 0.000 1.776 0.345 ## .stfhlth 3.695 0.046 79.989 0.000 3.695 0.729 ## .stfedu 3.368 0.043 77.916 0.000 3.368 0.667 ## .stfeco 2.656 0.038 69.070 0.000 2.656 0.491 ## .stfgov 2.264 0.035 64.201 0.000 2.264 0.429 ## .stfdem 2.289 0.034 67.172 0.000 2.289 0.465 ## .pltinvt 0.694 0.009 78.255 0.000 0.694 0.624 ## .pltcare 0.668 0.009 77.562 0.000 0.668 0.605 ## .trstplt 0.978 0.028 34.461 0.000 0.978 0.205 ## .institutions 0.255 0.022 11.691 0.000 0.098 0.098 ## .satisfaction 0.551 0.020 27.846 0.000 0.400 0.400 ## .politicians 0.065 0.004 17.091 0.000 0.157 0.157 ## trust 0.352 0.010 35.005 0.000 1.000 1.000 ## Compare fit between the first and second order models: fitMeasures(fit_3f) ## npar fmin chisq ## 29.000 0.360 10652.207 ## df pvalue baseline.chisq ## 62.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.870 ## tli nnfi rfi ## 0.837 0.837 0.836 ## nfi pnfi ifi ## 0.870 0.691 0.870 ## rni logl unrestricted.logl ## 0.870 -371404.658 -366078.555 ## aic bic ntotal ## 742867.317 743087.743 14778.000 ## bic2 rmsea rmsea.ci.lower ## 742995.583 0.108 0.106 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.109 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.255 0.255 0.059 ## srmr_bentler srmr_bentler_nomean crmr ## 0.059 0.059 0.064 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.064 0.059 0.059 ## cn_05 cn_01 gfi ## 113.901 126.971 0.897 ## agfi pgfi mfi ## 0.849 0.611 0.699 ## ecvi ## 0.725 fitMeasures(fit_2nd) ## npar fmin chisq ## 29.000 0.360 10652.207 ## df pvalue baseline.chisq ## 62.000 0.000 81699.096 ## baseline.df baseline.pvalue cfi ## 78.000 0.000 0.870 ## tli nnfi rfi ## 0.837 0.837 0.836 ## nfi pnfi ifi ## 0.870 0.691 0.870 ## rni logl unrestricted.logl ## 0.870 -371404.658 -366078.555 ## aic bic ntotal ## 742867.317 743087.743 14778.000 ## bic2 rmsea rmsea.ci.lower ## 742995.583 0.108 0.106 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.109 0.900 0.000 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 1.000 0.080 ## rmr rmr_nomean srmr ## 0.255 0.255 0.059 ## srmr_bentler srmr_bentler_nomean crmr ## 0.059 0.059 0.064 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.064 0.059 0.059 ## cn_05 cn_01 gfi ## 113.901 126.971 0.897 ## agfi pgfi mfi ## 0.849 0.611 0.699 ## ecvi ## 0.725 Click for explanation We don’t have to do anything special here. We can estimate and summarize the second order CFA exactly as we did the first order CFA. You should quickly notice something strange about the model fit statistics compared above. If you don’t see it, consider the following: fitMeasures(fit_3f) - fitMeasures(fit_2nd) ## npar fmin chisq ## 0 0 0 ## df pvalue baseline.chisq ## 0 0 0 ## baseline.df baseline.pvalue cfi ## 0 0 0 ## tli nnfi rfi ## 0 0 0 ## nfi pnfi ifi ## 0 0 0 ## rni logl unrestricted.logl ## 0 0 0 ## aic bic ntotal ## 0 0 0 ## bic2 rmsea rmsea.ci.lower ## 0 0 0 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0 0 0 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0 0 0 ## rmr rmr_nomean srmr ## 0 0 0 ## srmr_bentler srmr_bentler_nomean crmr ## 0 0 0 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0 0 0 ## cn_05 cn_01 gfi ## 0 0 0 ## agfi pgfi mfi ## 0 0 0 ## ecvi ## 0 The two models produce identical fit statistics! We also see that the degrees of freedom are identical between the two models. Hence, the two models have equal complexity. This result taps into a critical idea in statistical modeling, namely, model equivalency. It turns out the two models we’re comparing here are equivalent in the sense that they are statistically indistinguishable representations of the data. Since this is a very important idea, I want to spend some time discussing it in person. So, spend some time between now and the Week 6 lecture session thinking about the implications of this model equivalence. Specifically, consider the following questions: What do we mean when we say that these two models are equivalent? How is it possible for these two models to be equivalent when one contains an additional latent variable? Why are the degrees of freedom equal for these two models? Why are the fit statistics equal for these two models? We’ll take some time to discuss these ideas in the Week 6 lecture session. End of At-Home Exercises "],["in-class-exercises-4.html", "5.4 In-Class Exercises", " 5.4 In-Class Exercises This week, we will wrap up our re-analysis of the Kestilä (2006) results. During this practical, you will conduct a CFA of the Attitudes toward Immigration items and compare the results to those obtained from your previous EFA- and PCA-based replications of Kestilä (2006). 5.4.1 Load the ESS data. The relevant data are contained in the ess_round1.rds file. Click to show code ess <- readRDS("ess_round1.rds") We are going to conduct a CFA to evaluate the measurement model implied by the five-factor representation of the Attitudes toward Immigration items that you should have found via the EFA you conducted in the Week 4 In-Class Exercises. Caveat: Technically, the following CFA result have no confirmatory value because we’ll be estimating our CFA models from the same data that we used for our EFA. Practicing the techniques will still be useful, though. 5.4.2 Define the lavaan model syntax for the CFA implied by the five-factor solution from 4.4.9. Enforce a simple structure; do not allow any cross-loadings. Covary the five latent factors. Do not specify any mean structure. Save this model syntax as an object in your environment. Hints: You can algorithmically enforce a simple structure by assigning each item to the factor upon which it loads most strongly. You can download the fitted psych::efa() object for the five-factor solution here. The pattern matrix for the five-factor EFA solution in our Week 4 exercises is equivalent to the solution presented in Table 3 of Kestilä (2006). Click to show code mod_5f <- ' ## Immigration Policy: ip =~ imrcntr + eimrcnt + eimpcnt + imsmetn + impcntr + imdfetn ## Social Threat: st =~ imbgeco + imbleco + imwbcnt + imwbcrm + imtcjob + imueclt ## Refugee Policy: rp =~ gvrfgap + imrsprc + rfgbfml + rfggvfn + rfgawrk + rfgfrpc + shrrfg ## Cultural Threat: ct =~ qfimchr + qfimwht + pplstrd + vrtrlg ## Economic Threat: et =~ imwgdwn + imhecop ' Note: We don’t have to specify the latent covariances in the model syntax, we can tell lavaan to estimate all latent covariances when we fit the model. 5.4.3 Estimate the CFA model you defined above, and summarize the results. Use the lavaan::cfa() function to estimate the model. Use the default settings for the cfa() function. Request the model fit statistics with the summary by supplying the fit.measures = TRUE argument to summary(). Request the standardized parameter estimates with the summary by supplying the standardized = TRUE argument to summary(). Check the results, and answer the following questions: Does the model fit the data well? How are the latent variances and covariances specified when using the default settings? How is the model identified when using the default settings? Click to show code ## Load the lavaan package: library(lavaan) ## Estimate the CFA model: fit_5f <- cfa(mod_5f, data = ess) ## Summarize the fitted model: summary(fit_5f, fit.measures = TRUE, standardized = TRUE) ## lavaan 0.6.16 ended normally after 72 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 60 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 18631.556 ## Degrees of freedom 265 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 159619.058 ## Degrees of freedom 300 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.885 ## Tucker-Lewis Index (TLI) 0.869 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -520035.133 ## Loglikelihood unrestricted model (H1) -510719.354 ## ## Akaike (AIC) 1040190.265 ## Bayesian (BIC) 1040644.106 ## Sample-size adjusted Bayesian (SABIC) 1040453.432 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## 90 Percent confidence interval - lower 0.069 ## 90 Percent confidence interval - upper 0.071 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## ip =~ ## imrcntr 1.000 0.617 0.748 ## eimrcnt 0.942 0.011 84.943 0.000 0.582 0.696 ## eimpcnt 1.127 0.010 113.413 0.000 0.695 0.898 ## imsmetn 0.982 0.010 98.753 0.000 0.606 0.796 ## impcntr 1.150 0.010 113.623 0.000 0.710 0.900 ## imdfetn 1.132 0.010 111.802 0.000 0.698 0.887 ## st =~ ## imbgeco 1.000 1.608 0.728 ## imbleco 0.826 0.012 69.222 0.000 1.327 0.619 ## imwbcnt 1.046 0.012 88.056 0.000 1.682 0.792 ## imwbcrm 0.713 0.011 63.102 0.000 1.146 0.564 ## imtcjob 0.751 0.011 66.787 0.000 1.207 0.597 ## imueclt 1.008 0.013 78.043 0.000 1.621 0.698 ## rp =~ ## gvrfgap 1.000 0.659 0.610 ## imrsprc 0.855 0.016 51.881 0.000 0.563 0.535 ## rfgbfml 1.047 0.019 56.174 0.000 0.690 0.593 ## rfggvfn 0.849 0.016 51.714 0.000 0.559 0.533 ## rfgawrk 0.653 0.016 41.044 0.000 0.430 0.405 ## rfgfrpc -0.810 0.016 -51.095 0.000 -0.534 -0.525 ## shrrfg -0.999 0.017 -58.381 0.000 -0.658 -0.625 ## ct =~ ## qfimchr 1.000 1.836 0.629 ## qfimwht 0.941 0.017 54.250 0.000 1.728 0.659 ## pplstrd -0.366 0.007 -51.585 0.000 -0.673 -0.600 ## vrtrlg 0.252 0.006 41.294 0.000 0.462 0.443 ## et =~ ## imwgdwn 1.000 0.723 0.667 ## imhecop 1.151 0.023 49.736 0.000 0.832 0.771 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## ip ~~ ## st -0.605 0.012 -48.693 0.000 -0.610 -0.610 ## rp 0.264 0.006 45.566 0.000 0.648 0.648 ## ct 0.634 0.015 41.007 0.000 0.560 0.560 ## et -0.206 0.006 -35.411 0.000 -0.462 -0.462 ## st ~~ ## rp -0.838 0.017 -48.329 0.000 -0.792 -0.792 ## ct -1.622 0.041 -39.091 0.000 -0.550 -0.550 ## et 0.675 0.017 39.083 0.000 0.580 0.580 ## rp ~~ ## ct 0.626 0.018 34.950 0.000 0.518 0.518 ## et -0.233 0.007 -33.007 0.000 -0.490 -0.490 ## ct ~~ ## et -0.592 0.020 -30.127 0.000 -0.446 -0.446 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .imrcntr 0.299 0.004 77.941 0.000 0.299 0.440 ## .eimrcnt 0.359 0.005 79.638 0.000 0.359 0.515 ## .eimpcnt 0.116 0.002 62.821 0.000 0.116 0.193 ## .imsmetn 0.212 0.003 75.580 0.000 0.212 0.366 ## .impcntr 0.119 0.002 62.454 0.000 0.119 0.191 ## .imdfetn 0.132 0.002 65.344 0.000 0.132 0.213 ## .imbgeco 2.288 0.033 70.261 0.000 2.288 0.470 ## .imbleco 2.837 0.037 76.688 0.000 2.837 0.617 ## .imwbcnt 1.677 0.027 63.198 0.000 1.677 0.372 ## .imwbcrm 2.810 0.036 78.612 0.000 2.810 0.682 ## .imtcjob 2.630 0.034 77.524 0.000 2.630 0.643 ## .imueclt 2.761 0.038 72.515 0.000 2.761 0.512 ## .gvrfgap 0.733 0.010 73.584 0.000 0.733 0.628 ## .imrsprc 0.791 0.010 77.119 0.000 0.791 0.714 ## .rfgbfml 0.877 0.012 74.508 0.000 0.877 0.648 ## .rfggvfn 0.788 0.010 77.203 0.000 0.788 0.716 ## .rfgawrk 0.945 0.012 80.870 0.000 0.945 0.836 ## .rfgfrpc 0.749 0.010 77.501 0.000 0.749 0.724 ## .shrrfg 0.676 0.009 72.682 0.000 0.676 0.609 ## .qfimchr 5.142 0.080 64.113 0.000 5.142 0.604 ## .qfimwht 3.891 0.064 60.623 0.000 3.891 0.566 ## .pplstrd 0.804 0.012 67.054 0.000 0.804 0.640 ## .vrtrlg 0.872 0.011 76.990 0.000 0.872 0.804 ## .imwgdwn 0.652 0.012 53.300 0.000 0.652 0.555 ## .imhecop 0.472 0.014 34.353 0.000 0.472 0.405 ## ip 0.381 0.007 51.578 0.000 1.000 1.000 ## st 2.584 0.054 47.795 0.000 1.000 1.000 ## rp 0.434 0.012 36.748 0.000 1.000 1.000 ## ct 3.371 0.096 35.174 0.000 1.000 1.000 ## et 0.523 0.015 34.944 0.000 1.000 1.000 Click for explanation No, the model does not seem to fit the data well. The SRMR looks good, but one good looking fit statistic is not enough. The TLI and CFI are in the “unacceptable” range. RMSEA is in the “questionable” range. The \\(\\chi^2\\) is highly significant, but we don’t care. The cfa() function is just a wrapper for the lavaan() function with several options set at the defaults you would want for a standard CFA. By default: All latent variances and covariances are freely estimated (due to the argument auto.cov.lv.x = TRUE) The model is identified by fixing the first factor loading of each factor to 1 (due to the argument auto.fix.first = TRUE) To see a full list of the (many) options you can specify to tweak the behavior of lavaan estimation functions run ?lavOptions. Now, we will consider a couple of alternative factor structures for the Attitudes toward Immigration CFA. First, we will go extremely simple by estimating a one-factor model wherein all Attitude items are explained by a single latent variable. 5.4.4 Define the lavaan model syntax for a one-factor model of the Immigration items. Save this syntax as an object in your environment. Click to show code mod_1f <- ' ati =~ imrcntr + eimrcnt + eimpcnt + imsmetn + impcntr + imdfetn + imbgeco + imbleco + imwbcnt + imwbcrm + imtcjob + imueclt + gvrfgap + imrsprc + rfgbfml + rfggvfn + rfgawrk + rfgfrpc + shrrfg + qfimchr + qfimwht + pplstrd + vrtrlg + imwgdwn + imhecop ' 5.4.5 Estimate the one-factor model, and summarize the results. Compare the fit measures for the one-factor and five-factor models Which model better fits the data? Note: Remember, you can use the lavaan::fitMeasures() function to extract only the model fit information from a fitted lavaan object. Click to show code ## Estimate the one factor model: fit_1f <- cfa(mod_1f, data = ess) ## Summarize the results: summary(fit_1f) ## lavaan 0.6.16 ended normally after 47 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 50 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 49510.917 ## Degrees of freedom 275 ## P-value (Chi-square) 0.000 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## ati =~ ## imrcntr 1.000 ## eimrcnt 0.937 0.012 78.324 0.000 ## eimpcnt 1.114 0.011 101.263 0.000 ## imsmetn 0.987 0.011 90.990 0.000 ## impcntr 1.147 0.011 102.371 0.000 ## imdfetn 1.153 0.011 103.148 0.000 ## imbgeco -2.055 0.032 -64.749 0.000 ## imbleco -1.625 0.031 -52.533 0.000 ## imwbcnt -2.173 0.030 -71.324 0.000 ## imwbcrm -1.432 0.029 -48.849 0.000 ## imtcjob -1.532 0.029 -52.519 0.000 ## imueclt -2.198 0.033 -65.876 0.000 ## gvrfgap 0.807 0.016 51.746 0.000 ## imrsprc 0.757 0.015 49.790 0.000 ## rfgbfml 0.861 0.017 51.272 0.000 ## rfggvfn 0.722 0.015 47.671 0.000 ## rfgawrk 0.530 0.015 34.448 0.000 ## rfgfrpc -0.755 0.015 -51.462 0.000 ## shrrfg -0.931 0.015 -61.438 0.000 ## qfimchr 1.597 0.042 37.835 0.000 ## qfimwht 1.769 0.038 46.697 0.000 ## pplstrd -0.873 0.016 -53.994 0.000 ## vrtrlg 0.602 0.015 39.940 0.000 ## imwgdwn -0.682 0.016 -43.576 0.000 ## imhecop -0.773 0.016 -49.611 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr 0.327 0.004 79.021 0.000 ## .eimrcnt 0.388 0.005 80.422 0.000 ## .eimpcnt 0.161 0.002 70.832 0.000 ## .imsmetn 0.235 0.003 77.101 0.000 ## .impcntr 0.158 0.002 69.688 0.000 ## .imdfetn 0.150 0.002 68.791 0.000 ## .imbgeco 3.381 0.041 82.203 0.000 ## .imbleco 3.666 0.044 83.130 0.000 ## .imwbcnt 2.839 0.035 81.477 0.000 ## .imwbcrm 3.399 0.041 83.334 0.000 ## .imtcjob 3.260 0.039 83.130 0.000 ## .imueclt 3.683 0.045 82.092 0.000 ## .gvrfgap 0.938 0.011 83.176 0.000 ## .imrsprc 0.906 0.011 83.285 0.000 ## .rfgbfml 1.092 0.013 83.203 0.000 ## .rfggvfn 0.917 0.011 83.394 0.000 ## .rfgawrk 1.031 0.012 83.913 0.000 ## .rfgfrpc 0.832 0.010 83.192 0.000 ## .shrrfg 0.803 0.010 82.499 0.000 ## .qfimchr 7.613 0.091 83.803 0.000 ## .qfimwht 5.772 0.069 83.442 0.000 ## .pplstrd 0.988 0.012 83.040 0.000 ## .vrtrlg 0.958 0.011 83.728 0.000 ## .imwgdwn 1.010 0.012 83.583 0.000 ## .imhecop 0.954 0.011 83.294 0.000 ## ati 0.353 0.007 48.941 0.000 ## Compare fit statistics: fitMeasures(fit_5f, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 18631.556 ## Degrees of freedom 265 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.885 ## Tucker-Lewis Index (TLI) 0.869 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 fitMeasures(fit_1f, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 49510.917 ## Degrees of freedom 275 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.691 ## Tucker-Lewis Index (TLI) 0.663 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.112 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.087 Click for explanation The one-factor model definitely seems to fit worse than the five-factor model. 5.4.6 Given the CFA results from the five factor model, would a second-order CFA be appropriate for the Attitudes towards Immigration data? Why or why not? Click for explanation Yes, a second-order CFA model is a theoretically appropriate representation of the Attitudes towards Immigration items. The first order latent variables in the five-factor model are all significantly correlated. The first order latent variables in the five-factor model seem to tap different aspects of some single underlying construct. 5.4.7 Define the lavaan model syntax for a second-order CFA model of the Attitudes towards Immigration items, estimate it, and inspect the results. Use the five factors defined in 5.4.2 as the first order factors. Click to show code mod_2o <- paste(mod_5f, 'ati =~ ip + rp + st + ct + et', sep = '\\n') fit_2o <- cfa(mod_2o, data = ess) summary(fit_2o, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 94 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 55 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 19121.111 ## Degrees of freedom 270 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 159619.058 ## Degrees of freedom 300 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.882 ## Tucker-Lewis Index (TLI) 0.869 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -520279.910 ## Loglikelihood unrestricted model (H1) -510719.354 ## ## Akaike (AIC) 1040669.820 ## Bayesian (BIC) 1041085.841 ## Sample-size adjusted Bayesian (SABIC) 1040911.056 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## 90 Percent confidence interval - lower 0.069 ## 90 Percent confidence interval - upper 0.071 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## ip =~ ## imrcntr 1.000 ## eimrcnt 0.943 0.011 85.095 0.000 ## eimpcnt 1.126 0.010 113.523 0.000 ## imsmetn 0.982 0.010 98.910 0.000 ## impcntr 1.149 0.010 113.651 0.000 ## imdfetn 1.130 0.010 111.789 0.000 ## st =~ ## imbgeco 1.000 ## imbleco 0.822 0.012 68.916 0.000 ## imwbcnt 1.047 0.012 88.172 0.000 ## imwbcrm 0.709 0.011 62.846 0.000 ## imtcjob 0.747 0.011 66.424 0.000 ## imueclt 1.013 0.013 78.434 0.000 ## rp =~ ## gvrfgap 1.000 ## imrsprc 0.854 0.017 51.127 0.000 ## rfgbfml 1.048 0.019 55.377 0.000 ## rfggvfn 0.853 0.017 51.170 0.000 ## rfgawrk 0.657 0.016 40.785 0.000 ## rfgfrpc -0.828 0.016 -51.249 0.000 ## shrrfg -1.020 0.017 -58.369 0.000 ## ct =~ ## qfimchr 1.000 ## qfimwht 0.939 0.018 51.902 0.000 ## pplstrd -0.389 0.008 -51.072 0.000 ## vrtrlg 0.271 0.006 41.908 0.000 ## et =~ ## imwgdwn 1.000 ## imhecop 1.158 0.024 48.877 0.000 ## ati =~ ## ip 1.000 ## rp 1.264 0.024 53.732 0.000 ## st -3.123 0.051 -61.058 0.000 ## ct 2.638 0.058 45.467 0.000 ## et -1.000 0.024 -42.490 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr 0.299 0.004 77.900 0.000 ## .eimrcnt 0.359 0.005 79.597 0.000 ## .eimpcnt 0.116 0.002 62.698 0.000 ## .imsmetn 0.211 0.003 75.502 0.000 ## .impcntr 0.119 0.002 62.476 0.000 ## .imdfetn 0.133 0.002 65.406 0.000 ## .imbgeco 2.285 0.033 70.158 0.000 ## .imbleco 2.852 0.037 76.762 0.000 ## .imwbcnt 1.668 0.027 62.920 0.000 ## .imwbcrm 2.821 0.036 78.653 0.000 ## .imtcjob 2.646 0.034 77.607 0.000 ## .imueclt 2.734 0.038 72.213 0.000 ## .gvrfgap 0.740 0.010 73.738 0.000 ## .imrsprc 0.797 0.010 77.211 0.000 ## .rfgbfml 0.885 0.012 74.621 0.000 ## .rfggvfn 0.791 0.010 77.189 0.000 ## .rfgawrk 0.946 0.012 80.833 0.000 ## .rfgfrpc 0.741 0.010 77.149 0.000 ## .shrrfg 0.665 0.009 72.020 0.000 ## .qfimchr 5.347 0.081 65.623 0.000 ## .qfimwht 4.084 0.065 62.673 0.000 ## .pplstrd 0.778 0.012 64.838 0.000 ## .vrtrlg 0.854 0.011 75.931 0.000 ## .imwgdwn 0.655 0.012 52.977 0.000 ## .imhecop 0.468 0.014 33.353 0.000 ## .ip 0.177 0.004 44.418 0.000 ## .st 0.596 0.023 26.030 0.000 ## .rp 0.101 0.005 21.784 0.000 ## .ct 1.745 0.060 29.185 0.000 ## .et 0.316 0.010 31.813 0.000 ## ati 0.204 0.005 37.371 0.000 5.4.8 Compare the model fit of the first- and second-order five-factor models using the fitMeasures() function. Which model offers the better fit? Which model is more complex? Click to show code fitMeasures(fit_5f, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 18631.556 ## Degrees of freedom 265 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.885 ## Tucker-Lewis Index (TLI) 0.869 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 fitMeasures(fit_2o, fit.measures = c("npar", # Estimated parameters "chisq", "df", "pvalue", # Model fit vs. saturated "cfi", "tli", # Model fit vs. baseline "rmsea", "srmr"), # Model fit vs. saturated output = "text") ## ## Model Test User Model: ## ## Test statistic 19121.111 ## Degrees of freedom 270 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.882 ## Tucker-Lewis Index (TLI) 0.869 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.070 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.048 Click for explanation The CFI and TLI are both slightly better in the original five factor model, but the RMSEA and SRMR of both models don’t differ out to three decimal places. As usual, both models have a significant \\(\\chi^2\\), but that doesn’t tell us much. Qualitative comparisons of model fit are fine, but we’d like to have an actual statistical test for these fit differences. As it happens, we have just such a test: a nested model \\(\\Delta \\chi^2\\) test (AKA, chi-squared difference test, change in chi-squared test, likelihood ratio test). In the Week 7 lecture, we’ll cover nested models and tests thereof, but it will be useful to start thinking about these concepts now. Two models are said to be nested if you can define one model by placing constraints on the other model. By way of example, consider the following two CFA models. The second model is nested within the first model, because we can define the second model by fixing the latent covariance to zero in the first model. Notice that the data contain \\(6(6 + 1) / 2 = 21\\) unique pieces of information. The first model estimates 13 parameters, and the second model estimates 12 parameters. Hence the first model has 8 degrees of freedom, and the second model has 9 degrees of freedom. In general, the following must hold whenever Model B is nested within Model A. Model B will have fewer estimated parameters than Model A. Model B will have more degrees of freedom than Model A. Model A will be more complex than model B. Model A will fit the data better than model B. Saturated Model All models are nested within the saturated model, because the saturated model estimates all possible relations among the variables. Regardless of what model we may be considering, we can always convert that model to a saturated model by estimating all possible associations. Hence, all models are nested within the saturated model. Baseline Model Similarly, the baseline model (AKA, independence model) is nested within all other models. In the baseline model, we only estimate the variances of the observed items; all associations are constrained to zero. We can always convert our model to the baseline model by fixing all associations to zero. Hence, the baseline model is nested within all other models. When two models are nested, we can use a \\(\\Delta \\chi^2\\) test to check if the nested model fits significantly worse than its parent model. Whenever we place constraints on the model, the fit will deteriorate, but we want to know if the constraints we imposed to define the nested model have produced too much loss of fit. We can use the anova() function to easily conduct \\(\\Delta \\chi^2\\) tests comparing models that we’ve estimated with cfa() or sem(). 5.4.9 Use the anova() function to compare the five-factor model from 5.4.2 and one-factor model from 5.4.4. Explain what Df, Chisq, Chisq diff, Df diff, and Pr(>Chisq) mean. Which model is more complex? Which model fits better? What is the conclusion of the test? Click to show code anova(fit_1f, fit_5f) Click for explanation The Df column contains the degrees of freedom of each model. Higher df \\(\\Rightarrow\\) Less complex model The Chisq column shows the \\(\\chi^2\\) statistics (AKA, likelihood ratio statistics) for each model. \\(\\chi^2\\) = The ratio of the likelihoods for the estimated model and the saturated model). Larger \\(\\chi^2\\) \\(\\Rightarrow\\) Worse fit Chisq diff is the difference between the two \\(\\chi^2\\) values (i.e., \\(\\Delta \\chi^2\\)). How much better the more complex model fits the data Larger \\(\\Delta \\chi^2\\) values indicate greater losses of fit induced by the constraints needed to define the nested model. Df diff is the difference in the degrees of freedom between the models. Since both models must be estimated from the same pool of variables, this difference also represents the number of parameters that were constrained to define the nested model. Pr(>Chisq) is a p-value for the \\(\\Delta \\chi^2\\) test. \\(H_0: \\Delta \\chi^2 = 0\\) \\(H_1: \\Delta \\chi^2 > 0\\) The five-factor model is more complex than the one-factor model, but the extra complexity is justified The five-factor model fits significantly better than the one-factor model. 5.4.10 Use the anova() function to compare the first- and second-order five-factor models from 5.4.2 and 5.4.7. Which model is more complex? What is the conclusion of the test? Click to show code anova(fit_5f, fit_2o) Click for explanation The first-order model is more complex than the second-order model (df = 265 vs. df = 270), and the extra complexity is necessary. The first-order model fits significantly better than the second-order model. 5.4.11 Based on the results above, would you say that you have successfully confirmed the five-factor structure implied by the EFA? Click for explanation Nope, not so much. The first-order five-factor model may fit the data best out of the three models considered here, but it still fits terribly. None of these models is an adequate representation of the Attitudes toward Immigration items. This result is particularly embarrassing when you consider that we’ve stacked the deck in our favor by using the same data to conduct the EFA and the CFA. When we fail to support the hypothesized measurement model, the confirmatory phase of our analysis is over. At this point, we’ve essentially rejected our hypothesized measurement structure, and that’s the conclusion of our analysis. We don’t have to throw up our hands in despair, however. We can still contribute something useful by modifying the theoretical measurement model through an exploratory, data-driven, post-hoc analysis. We’ll give that a shot below. 5.4.12 Modify the five-factor CFA from 5.4.2 by freeing the following parameters. The residual covariance between imrcntr and eimrcnt These questions both ask about allowing immigration from wealthy countries. It makes sense that answers on these two items share some additional, unique variance above-and-beyond what they contribute to the common factors. The residual covariance between qfimchr and qfimwht These questions are both about imposing qualifications on immigration (specifically Christian religion and “white” race). Click to show code fit_5f_cov <- paste(mod_5f, 'imrcntr ~~ eimrcnt', 'qfimchr ~~ qfimwht', sep = '\\n') %>% cfa(data = ess) summary(fit_5f_cov, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 77 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 62 ## ## Used Total ## Number of observations 14243 19690 ## ## Model Test User Model: ## ## Test statistic 9740.512 ## Degrees of freedom 263 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 159619.058 ## Degrees of freedom 300 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.941 ## Tucker-Lewis Index (TLI) 0.932 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -515589.611 ## Loglikelihood unrestricted model (H1) -510719.354 ## ## Akaike (AIC) 1031303.221 ## Bayesian (BIC) 1031772.190 ## Sample-size adjusted Bayesian (SABIC) 1031575.160 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.050 ## 90 Percent confidence interval - lower 0.049 ## 90 Percent confidence interval - upper 0.051 ## P-value H_0: RMSEA <= 0.050 0.280 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.036 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## ip =~ ## imrcntr 1.000 ## eimrcnt 0.928 0.007 126.255 0.000 ## eimpcnt 1.184 0.011 106.508 0.000 ## imsmetn 1.012 0.011 92.436 0.000 ## impcntr 1.213 0.011 107.078 0.000 ## imdfetn 1.181 0.011 104.566 0.000 ## st =~ ## imbgeco 1.000 ## imbleco 0.826 0.012 69.006 0.000 ## imwbcnt 1.050 0.012 88.051 0.000 ## imwbcrm 0.715 0.011 63.128 0.000 ## imtcjob 0.751 0.011 66.542 0.000 ## imueclt 1.015 0.013 78.256 0.000 ## rp =~ ## gvrfgap 1.000 ## imrsprc 0.858 0.017 51.965 0.000 ## rfgbfml 1.046 0.019 56.104 0.000 ## rfggvfn 0.848 0.016 51.644 0.000 ## rfgawrk 0.652 0.016 40.998 0.000 ## rfgfrpc -0.813 0.016 -51.233 0.000 ## shrrfg -1.002 0.017 -58.499 0.000 ## ct =~ ## qfimchr 1.000 ## qfimwht 0.979 0.020 48.332 0.000 ## pplstrd -0.586 0.014 -40.685 0.000 ## vrtrlg 0.397 0.011 36.273 0.000 ## et =~ ## imwgdwn 1.000 ## imhecop 1.157 0.023 49.549 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr ~~ ## .eimrcnt 0.230 0.004 59.907 0.000 ## .qfimchr ~~ ## .qfimwht 2.558 0.064 40.233 0.000 ## ip ~~ ## st -0.580 0.012 -48.041 0.000 ## rp 0.255 0.006 45.185 0.000 ## ct 0.467 0.014 34.425 0.000 ## et -0.197 0.006 -35.077 0.000 ## st ~~ ## rp -0.835 0.017 -48.285 0.000 ## ct -1.394 0.040 -35.128 0.000 ## et 0.670 0.017 38.935 0.000 ## rp ~~ ## ct 0.538 0.017 32.407 0.000 ## et -0.232 0.007 -32.949 0.000 ## ct ~~ ## et -0.469 0.017 -27.959 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .imrcntr 0.330 0.004 78.903 0.000 ## .eimrcnt 0.396 0.005 80.392 0.000 ## .eimpcnt 0.109 0.002 60.401 0.000 ## .imsmetn 0.220 0.003 75.979 0.000 ## .impcntr 0.107 0.002 58.874 0.000 ## .imdfetn 0.131 0.002 64.630 0.000 ## .imbgeco 2.301 0.033 70.568 0.000 ## .imbleco 2.845 0.037 76.832 0.000 ## .imwbcnt 1.669 0.026 63.272 0.000 ## .imwbcrm 2.808 0.036 78.659 0.000 ## .imtcjob 2.639 0.034 77.663 0.000 ## .imueclt 2.741 0.038 72.463 0.000 ## .gvrfgap 0.734 0.010 73.743 0.000 ## .imrsprc 0.790 0.010 77.164 0.000 ## .rfgbfml 0.880 0.012 74.676 0.000 ## .rfggvfn 0.790 0.010 77.322 0.000 ## .rfgawrk 0.946 0.012 80.924 0.000 ## .rfgfrpc 0.747 0.010 77.519 0.000 ## .shrrfg 0.674 0.009 72.713 0.000 ## .qfimchr 6.815 0.090 75.362 0.000 ## .qfimwht 5.250 0.072 73.378 0.000 ## .pplstrd 0.674 0.013 52.766 0.000 ## .vrtrlg 0.818 0.011 73.191 0.000 ## .imwgdwn 0.655 0.012 53.496 0.000 ## .imhecop 0.468 0.014 33.845 0.000 ## ip 0.350 0.007 48.646 0.000 ## st 2.571 0.054 47.662 0.000 ## rp 0.433 0.012 36.718 0.000 ## ct 1.698 0.073 23.296 0.000 ## et 0.520 0.015 34.814 0.000 5.4.13 Evaluate the model modifications. Did the model fit significantly improve? Is the fit of the modified model acceptable? Click to show code anova(fit_5f_cov, fit_5f) fitMeasures(fit_5f_cov) ## npar fmin chisq ## 62.000 0.342 9740.512 ## df pvalue baseline.chisq ## 263.000 0.000 159619.058 ## baseline.df baseline.pvalue cfi ## 300.000 0.000 0.941 ## tli nnfi rfi ## 0.932 0.932 0.930 ## nfi pnfi ifi ## 0.939 0.823 0.941 ## rni logl unrestricted.logl ## 0.941 -515589.611 -510719.354 ## aic bic ntotal ## 1031303.221 1031772.190 14243.000 ## bic2 rmsea rmsea.ci.lower ## 1031575.160 0.050 0.049 ## rmsea.ci.upper rmsea.ci.level rmsea.pvalue ## 0.051 0.900 0.280 ## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0 ## 0.050 0.000 0.080 ## rmr rmr_nomean srmr ## 0.103 0.103 0.036 ## srmr_bentler srmr_bentler_nomean crmr ## 0.036 0.036 0.037 ## crmr_nomean srmr_mplus srmr_mplus_nomean ## 0.037 0.036 0.036 ## cn_05 cn_01 gfi ## 442.344 467.858 0.944 ## agfi pgfi mfi ## 0.931 0.764 0.717 ## ecvi ## 0.693 Click for explanation Yes, the model fit improved significantly. In this case, the original five-factor model is nested within the modified model. So, our \\(\\Delta \\chi^2\\) test is evaluating the improvement in fit contributed by freeing the two residual covariances. The \\(\\Delta \\chi^2\\) test is significant, so we can conclude that including the two new parameter estimates has significantly improved the model fit. I.e., Estimating these two residual covariances is “worth it” in the sense of balancing model fit and model complexity. Also, the fit of the modified model is now acceptable. Caveat If we had found this result when testing our original model, we would be well-situated to proceed with our analysis. In this case, however, we are no longer justified in generalizing these estimates to the population. We only arrived at this well-fitting model by modifying our original theoretical model to better fit the data using estimates derived from those same data to guide our model modifications. We’ve conducted this post-hoc analysis to help inform future research, and this result is useful as a starting point for future studies. Now, anyone analyzing these scales in the future could incorporate these residual covariances into their initial theoretical model. Basically, we conduct these types of post-hoc analyses to help future researchers learn from our mistakes. End of In-Class Exercises "],["full-sem.html", "6 Full SEM", " 6 Full SEM This week, we will focus on integrating all of the disparate methods we’ve covered so far into full-fledged structural equation models. Homework before the lecture Watch the Lecture Recording for this week. Complete the Reading for this week, and answer the associated reading questions. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-5.html", "6.1 Lecture", " 6.1 Lecture This week, we will begin with our final theme and discuss structural equation modeling (SEM). This powerful technique joins the strengths of CFA and path analysis to produce a highly flexible and theoretically appealing modeling tool. Essentially, SEM allows us to build structural path models using the latent variables defined by a CFA. 6.1.1 Recording 6.1.2 Slides You can download the lectures slides here "],["reading-5.html", "6.2 Reading", " 6.2 Reading Reference Weston, R. & Gore, P. A. (2006). A brief guide to structural equation modeling. The Counseling Psychologist 34, 719–752. Notes: This article is quite general and provides an overview of things we have discussed so far in this course. This article also also adds an important new idea: combining factor analysis with path modeling to produce a full Structural Equation Model (SEM). Skip the part on GFI (p. 741). The GFI has been shown to be too dependent on sample size and is not recommended any longer. Skip the part on missing data. There is nothing wrong with this section, but missing data analysis is a broad and difficult topic that we cannot adequately cover in this course. If you would like to learn more about missing data and how to treat them, you can take two courses offered by our department: Conducting a Survey Missing Data Theory and Causal Effects Questions The authors state three similarities and two big differences between SEM and other multivariate statistical techniques (e.g., ANCOVA, regression). What are these similarities and differences? Do you agree with the relative strengths and weaknesses of SEM vs. other methods that the authors present? The authors miss at least one additional advantage of SEM over other multivariate methods. What is this missing advantage? Explain what the terms “measurement model” and “structural model” mean in the SEM context. What are the 6 steps of doing an SEM-based analysis given by the authors? The authors claim that testing an SEM using cross-validation is a good idea. When is cross-validation helpful in SEM? Hint: You may have to do some independent (internet, literature) research to learn how cross-validation can be implemented in SEM. "],["at-home-exercises-5.html", "6.3 At-Home Exercises", " 6.3 At-Home Exercises This week, we’ll take another look at the Kestilä (2006) results. During this practical, you will conduct an SEM to replicate the regression analysis of the Finnish data that you conducted in the Week 4 In-Class Exercises. 6.3.1 Load the Finnish subsample of ESS data. The relevant data are contained in the ess_finland.rds file. These are the processed Finnish subsample data from the Week 4 exercises. Note: Unless otherwise noted, all the following analyses use these data. Click to show code ess_fin <- readRDS("ess_finland.rds") We need to do a little data processing before we can fit the regression model. At the moment, lavaan will not automatically convert a factor variable into dummy codes. So, we need to create explicit dummy codes for the two factors we’ll use as predictors in our regression analysis: sex and political orientation. 6.3.2 Convert the sex and political interest factors into dummy codes. Click to show code library(dplyr) ## Create a dummy codes by broadcasting a logical test on the factor levels: ess_fin <- mutate(ess_fin, female = ifelse(sex == "Female", 1, 0), hi_pol_interest = ifelse(polintr_bin == "High Interest", 1, 0) ) ## Check the results: with(ess_fin, table(dummy = female, factor = sex)) ## factor ## dummy Male Female ## 0 960 0 ## 1 0 1040 with(ess_fin, table(dummy = hi_pol_interest, factor = polintr_bin)) ## factor ## dummy Low Interest High Interest ## 0 1070 0 ## 1 0 929 Click for explanation In R, we have several ways of converting a factor into an appropriate set of dummy codes. We could use the dplyr::recode() function as we did last week. We can use the model.matrix() function to define a design matrix based on the inherent contrast attribute of the factor. Missing data will cause problems here. We can us as.numeric() to revert the factor to its underlying numeric representation {Male = 1, Female = 2} and use arithmetic to convert {1, 2} \\(\\rightarrow\\) {0, 1}. When our factor only has two levels, though, the ifelse() function is the simplest way. We are now ready to estimate our latent regression model. Specifically, we want to combine the three OLS regression models that you ran in 4.4.16 into a single SEM that we will estimate in lavaan. The following path diagram shows the intended theoretical model. Although the variances are not included in this path diagram, all variables in the model (including the observed predictor variables) are random. 6.3.3 Define the lavaan model syntax for the SEM shown above. Use the definition of the institutions, satsifaction, and politicians factors from 5.3.2 to define the DVs. Covary the three latent factors. Covary the five predictors. Click to show code mod_sem <- ' ## Define the latent DVs: institutions =~ trstlgl + trstplc + trstun + trstep + trstprl satisfaction =~ stfhlth + stfedu + stfeco + stfgov + stfdem politicians =~ pltinvt + pltcare + trstplt ## Specify the structural relations: institutions + satisfaction + politicians ~ female + age + eduyrs + hi_pol_interest + lrscale ' Click for explanation We simply need to add a line defining the latent regression paths to our old CFA syntax. We don’t need to specify the covariances in the syntax. We can use options in the sem() function to request those estimates. 6.3.4 Estimate the SEM, and summarize the results. Fit the model to the processed Finnish subsample from above. Estimate the model using lavaan::sem(). Request the standardized parameter estimates with the summary. Request the \\(R^2\\) estimates with the summary. Click to show code library(lavaan) ## Fit the SEM: fit_sem <- sem(mod_sem, data = ess_fin, fixed.x = FALSE) ## Summarize the results: summary(fit_sem, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 82 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 59 ## ## Used Total ## Number of observations 1740 2000 ## ## Model Test User Model: ## ## Test statistic 1287.421 ## Degrees of freedom 112 ## P-value (Chi-square) 0.000 ## ## Model Test Baseline Model: ## ## Test statistic 10534.649 ## Degrees of freedom 143 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.887 ## Tucker-Lewis Index (TLI) 0.856 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -57914.779 ## Loglikelihood unrestricted model (H1) -57271.068 ## ## Akaike (AIC) 115947.557 ## Bayesian (BIC) 116269.794 ## Sample-size adjusted Bayesian (SABIC) 116082.357 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.078 ## 90 Percent confidence interval - lower 0.074 ## 90 Percent confidence interval - upper 0.082 ## P-value H_0: RMSEA <= 0.050 0.000 ## P-value H_0: RMSEA >= 0.080 0.160 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.045 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions =~ ## trstlgl 1.000 1.418 0.669 ## trstplc 0.609 0.031 19.403 0.000 0.863 0.508 ## trstun 0.887 0.038 23.484 0.000 1.257 0.626 ## trstep 1.134 0.041 27.652 0.000 1.607 0.755 ## trstprl 1.192 0.040 29.444 0.000 1.689 0.815 ## satisfaction =~ ## stfhlth 1.000 0.979 0.497 ## stfedu 0.602 0.043 13.872 0.000 0.589 0.416 ## stfeco 1.266 0.067 18.848 0.000 1.240 0.681 ## stfgov 1.639 0.079 20.638 0.000 1.605 0.846 ## stfdem 1.521 0.075 20.180 0.000 1.489 0.793 ## politicians =~ ## pltinvt 1.000 0.567 0.566 ## pltcare 0.953 0.048 19.653 0.000 0.540 0.590 ## trstplt 3.281 0.133 24.675 0.000 1.860 0.915 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions ~ ## female 0.019 0.073 0.259 0.796 0.013 0.007 ## age -0.008 0.002 -3.740 0.000 -0.006 -0.105 ## eduyrs 0.034 0.010 3.233 0.001 0.024 0.091 ## hi_pol_interst 0.358 0.076 4.730 0.000 0.253 0.126 ## lrscale 0.104 0.018 5.634 0.000 0.073 0.147 ## satisfaction ~ ## female -0.147 0.050 -2.910 0.004 -0.150 -0.075 ## age -0.007 0.002 -4.598 0.000 -0.007 -0.129 ## eduyrs 0.005 0.007 0.775 0.439 0.006 0.022 ## hi_pol_interst 0.164 0.052 3.162 0.002 0.167 0.084 ## lrscale 0.099 0.013 7.501 0.000 0.101 0.202 ## politicians ~ ## female 0.010 0.029 0.349 0.727 0.018 0.009 ## age -0.004 0.001 -4.490 0.000 -0.007 -0.124 ## eduyrs 0.007 0.004 1.697 0.090 0.012 0.047 ## hi_pol_interst 0.258 0.031 8.364 0.000 0.455 0.227 ## lrscale 0.039 0.007 5.370 0.000 0.068 0.138 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .institutions ~~ ## .satisfaction 1.030 0.069 14.933 0.000 0.796 0.796 ## .politicians 0.675 0.041 16.628 0.000 0.908 0.908 ## .satisfaction ~~ ## .politicians 0.365 0.027 13.544 0.000 0.713 0.713 ## female ~~ ## age 0.071 0.212 0.335 0.738 0.071 0.008 ## eduyrs 0.179 0.046 3.869 0.000 0.179 0.093 ## hi_pol_interst -0.017 0.006 -2.767 0.006 -0.017 -0.066 ## lrscale -0.032 0.024 -1.316 0.188 -0.032 -0.032 ## age ~~ ## eduyrs -22.750 1.722 -13.212 0.000 -22.750 -0.334 ## hi_pol_interst 1.377 0.215 6.413 0.000 1.377 0.156 ## lrscale 1.774 0.853 2.079 0.038 1.774 0.050 ## eduyrs ~~ ## hi_pol_interst 0.270 0.047 5.787 0.000 0.270 0.140 ## lrscale 0.735 0.186 3.946 0.000 0.735 0.095 ## hi_pol_interest ~~ ## lrscale 0.016 0.024 0.672 0.501 0.016 0.016 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trstlgl 2.477 0.093 26.743 0.000 2.477 0.552 ## .trstplc 2.140 0.076 28.334 0.000 2.140 0.742 ## .trstun 2.453 0.090 27.322 0.000 2.453 0.608 ## .trstep 1.950 0.078 24.906 0.000 1.950 0.430 ## .trstprl 1.443 0.064 22.437 0.000 1.443 0.336 ## .stfhlth 2.922 0.104 28.103 0.000 2.922 0.753 ## .stfedu 1.663 0.058 28.613 0.000 1.663 0.827 ## .stfeco 1.775 0.069 25.755 0.000 1.775 0.536 ## .stfgov 1.020 0.056 18.371 0.000 1.020 0.284 ## .stfdem 1.307 0.060 21.953 0.000 1.307 0.371 ## .pltinvt 0.682 0.024 27.818 0.000 0.682 0.680 ## .pltcare 0.547 0.020 27.582 0.000 0.547 0.652 ## .trstplt 0.672 0.069 9.676 0.000 0.672 0.163 ## .institutions 1.881 0.125 15.077 0.000 0.936 0.936 ## .satisfaction 0.892 0.086 10.386 0.000 0.930 0.930 ## .politicians 0.294 0.024 12.224 0.000 0.914 0.914 ## female 0.250 0.008 29.496 0.000 0.250 1.000 ## age 313.238 10.620 29.496 0.000 313.238 1.000 ## eduyrs 14.818 0.502 29.496 0.000 14.818 1.000 ## hi_pol_interst 0.250 0.008 29.496 0.000 0.250 1.000 ## lrscale 4.034 0.137 29.496 0.000 4.034 1.000 ## ## R-Square: ## Estimate ## trstlgl 0.448 ## trstplc 0.258 ## trstun 0.392 ## trstep 0.570 ## trstprl 0.664 ## stfhlth 0.247 ## stfedu 0.173 ## stfeco 0.464 ## stfgov 0.716 ## stfdem 0.629 ## pltinvt 0.320 ## pltcare 0.348 ## trstplt 0.837 ## institutions 0.064 ## satisfaction 0.070 ## politicians 0.086 Click for explanation The fixed.x = FALSE argument tells lavaan to model the predictors as random variables. By default, lavaan will covary any random predictor variables. So, we don’t need to make any other changes to the usual procedure. 6.3.5 Finally, we will rerun the latent regression model from above as a path model with the factor scores from 4.4.10 acting as the DVs. Rerun the above SEM as a path model wherein the EFA-derived Trust in Institutions, Satisfaction with Political Systems, and Trust in Politicians factor scores act as the DVs. Request the standardized parameter estimates with the summary. Request the \\(R^2\\) estimates with the summary. Click to show code ## Define the model syntax for the path analysis: mod_pa <- ' trust_inst + satisfy + trust_pol ~ female + age + eduyrs + hi_pol_interest + lrscale' ## Estimate the path model: fit_pa <- sem(mod_pa, data = ess_fin, fixed.x = FALSE) ## Summarize the results: summary(fit_pa, standardized = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 44 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 36 ## ## Used Total ## Number of observations 1740 2000 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## trust_inst ~ ## female 0.004 0.045 0.091 0.928 0.004 0.002 ## age -0.003 0.001 -2.229 0.026 -0.003 -0.057 ## eduyrs 0.023 0.006 3.642 0.000 0.023 0.094 ## hi_pol_interst 0.167 0.046 3.599 0.000 0.167 0.088 ## lrscale 0.059 0.011 5.258 0.000 0.059 0.125 ## satisfy ~ ## female -0.125 0.040 -3.115 0.002 -0.125 -0.073 ## age -0.005 0.001 -4.102 0.000 -0.005 -0.105 ## eduyrs -0.003 0.006 -0.534 0.594 -0.003 -0.014 ## hi_pol_interst 0.073 0.041 1.782 0.075 0.073 0.043 ## lrscale 0.085 0.010 8.510 0.000 0.085 0.200 ## trust_pol ~ ## female 0.016 0.046 0.338 0.735 0.016 0.008 ## age -0.009 0.001 -6.480 0.000 -0.009 -0.161 ## eduyrs 0.018 0.007 2.839 0.005 0.018 0.071 ## hi_pol_interst 0.464 0.047 9.801 0.000 0.464 0.232 ## lrscale 0.055 0.011 4.801 0.000 0.055 0.110 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trust_inst ~~ ## .satisfy 0.437 0.021 20.609 0.000 0.437 0.568 ## .trust_pol 0.498 0.024 20.480 0.000 0.498 0.564 ## .satisfy ~~ ## .trust_pol 0.367 0.021 17.664 0.000 0.367 0.467 ## female ~~ ## age 0.071 0.212 0.335 0.738 0.071 0.008 ## eduyrs 0.179 0.046 3.869 0.000 0.179 0.093 ## hi_pol_interst -0.017 0.006 -2.767 0.006 -0.017 -0.066 ## lrscale -0.032 0.024 -1.316 0.188 -0.032 -0.032 ## age ~~ ## eduyrs -22.750 1.722 -13.212 0.000 -22.750 -0.334 ## hi_pol_interst 1.377 0.215 6.413 0.000 1.377 0.156 ## lrscale 1.774 0.853 2.079 0.038 1.774 0.050 ## eduyrs ~~ ## hi_pol_interst 0.270 0.047 5.787 0.000 0.270 0.140 ## lrscale 0.735 0.186 3.946 0.000 0.735 0.095 ## hi_pol_interest ~~ ## lrscale 0.016 0.024 0.672 0.501 0.016 0.016 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .trust_inst 0.866 0.029 29.496 0.000 0.866 0.958 ## .satisfy 0.684 0.023 29.496 0.000 0.684 0.945 ## .trust_pol 0.902 0.031 29.496 0.000 0.902 0.902 ## female 0.250 0.008 29.496 0.000 0.250 1.000 ## age 313.238 10.620 29.496 0.000 313.238 1.000 ## eduyrs 14.818 0.502 29.496 0.000 14.818 1.000 ## hi_pol_interst 0.250 0.008 29.496 0.000 0.250 1.000 ## lrscale 4.034 0.137 29.496 0.000 4.034 1.000 ## ## R-Square: ## Estimate ## trust_inst 0.042 ## satisfy 0.055 ## trust_pol 0.098 Click to show explanation We don’t so anything particularly special here. We simply rerun our latent regression as a path analysis with the EFA-derived factor scores as the DVs. 6.3.6 Compare the results from the path analysis to the SEM-based results. Does it matter whether we use a latent variable or a factor score to define the DV? Hint: When comparing parameter estimates, use the fully standardized estimates (i.e., the values in the column labeled Std.all). Click to show code Note: The “supportFunction.R” script that we source below isn’t a necessary part of the solution. This script defines a bunch of convenience functions. One of these functions, partSummary(), allows us to print selected pieces of the model summary. ## Source a script of convenience function definitions: source("supportFunctions.R") ## View the regression estimates from the SEM: partSummary(fit_sem, 8, standardized = TRUE) ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## institutions ~ ## female 0.019 0.073 0.259 0.796 0.013 0.007 ## age -0.008 0.002 -3.740 0.000 -0.006 -0.105 ## eduyrs 0.034 0.010 3.233 0.001 0.024 0.091 ## hi_pol_interst 0.358 0.076 4.730 0.000 0.253 0.126 ## lrscale 0.104 0.018 5.634 0.000 0.073 0.147 ## satisfaction ~ ## female -0.147 0.050 -2.910 0.004 -0.150 -0.075 ## age -0.007 0.002 -4.598 0.000 -0.007 -0.129 ## eduyrs 0.005 0.007 0.775 0.439 0.006 0.022 ## hi_pol_interst 0.164 0.052 3.162 0.002 0.167 0.084 ## lrscale 0.099 0.013 7.501 0.000 0.101 0.202 ## politicians ~ ## female 0.010 0.029 0.349 0.727 0.018 0.009 ## age -0.004 0.001 -4.490 0.000 -0.007 -0.124 ## eduyrs 0.007 0.004 1.697 0.090 0.012 0.047 ## hi_pol_interst 0.258 0.031 8.364 0.000 0.455 0.227 ## lrscale 0.039 0.007 5.370 0.000 0.068 0.138 ## View the regression estimates from the path analysis: partSummary(fit_pa, 7, standardized = TRUE) ## Regressions: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## trust_inst ~ ## female 0.004 0.045 0.091 0.928 0.004 0.002 ## age -0.003 0.001 -2.229 0.026 -0.003 -0.057 ## eduyrs 0.023 0.006 3.642 0.000 0.023 0.094 ## hi_pol_interst 0.167 0.046 3.599 0.000 0.167 0.088 ## lrscale 0.059 0.011 5.258 0.000 0.059 0.125 ## satisfy ~ ## female -0.125 0.040 -3.115 0.002 -0.125 -0.073 ## age -0.005 0.001 -4.102 0.000 -0.005 -0.105 ## eduyrs -0.003 0.006 -0.534 0.594 -0.003 -0.014 ## hi_pol_interst 0.073 0.041 1.782 0.075 0.073 0.043 ## lrscale 0.085 0.010 8.510 0.000 0.085 0.200 ## trust_pol ~ ## female 0.016 0.046 0.338 0.735 0.016 0.008 ## age -0.009 0.001 -6.480 0.000 -0.009 -0.161 ## eduyrs 0.018 0.007 2.839 0.005 0.018 0.071 ## hi_pol_interst 0.464 0.047 9.801 0.000 0.464 0.232 ## lrscale 0.055 0.011 4.801 0.000 0.055 0.110 ## View the R-squared estimates from the SEM: partSummary(fit_sem, 11, rsquare = TRUE) ## R-Square: ## Estimate ## trstlgl 0.448 ## trstplc 0.258 ## trstun 0.392 ## trstep 0.570 ## trstprl 0.664 ## stfhlth 0.247 ## stfedu 0.173 ## stfeco 0.464 ## stfgov 0.716 ## stfdem 0.629 ## pltinvt 0.320 ## pltcare 0.348 ## trstplt 0.837 ## institutions 0.064 ## satisfaction 0.070 ## politicians 0.086 ## View the R-squared estimates from the SEM: partSummary(fit_pa, 10, rsquare = TRUE) ## R-Square: ## Estimate ## trust_inst 0.042 ## satisfy 0.055 ## trust_pol 0.098 Click for explanation It certainly looks like the way we define the DV has a meaningful impact. The patterns of significance differ between the two sets of regression slopes, and the \\(R^2\\) values are larger for the Institutions and Satisfaction factors in the SEM, and the \\(R^2\\) for the Politicians factor is higher in the path analysis. End of At-Home Exercises "],["in-class-exercises-5.html", "6.4 In-Class Exercises", " 6.4 In-Class Exercises In these exercises, you will use full structural equation modeling (SEM) to evaluate the Theory of Reasoned Action (TORA), which is a popular psychological theory of social behavior developed by Ajzen and Fishbein. The theory states that actual behavior is predicted by behavioral intention, which is in turn predicted by the attitude toward the behavior and subjective norms about the behavior. Later, a third determinant was added, perceived behavioral control. The extent to which people feel that they have control over their behavior also influences their behavior. The data we will use for this practical are available in the toradata.csv file. These data were synthesized according to the results of Reinecke (1998)’s investigation of condom use by young people between 16 and 24 years old. The data contain the following variables: respnr: Numeric participant ID behavior: The dependent variable condom use Measured on a 5-point frequency scale (How often do you…) intent: A single item assessing behavioral intention Measured on a similar 5-point scale (In general, do you intend to…). attit_1:attit_3: Three indicators of attitudes about condom use Measured on a 5-point rating scale (e.g., using a condom is awkward) norm_1:norm_3: Three indicators of social norms about condom use Measured on a 5-point rating scale (e.g., I think most of my friends would use…) control_1:control_3: Three indicators of perceived behavioral control Measured on a 5-point rating scale (e.g., I know well how to use a condom) sex: Binary factor indicating biological sex 6.4.1 Load the data contained in the toradata.csv file. Click to show code condom <- read.csv("toradata.csv", stringsAsFactors = TRUE) 6.4.2 The data contain multiple indicators of attitudes, norms, and control. Run a CFA for these three latent variables. Correlate the latent factors. Do the data support the measurement model for these latent factors? Are the three latent factors significantly correlated? Is it reasonable to proceed with our evaluation of the TORA theory? Click to show code library(lavaan) mod_cfa <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 ' fit <- cfa(mod_cfa, data = condom) summary(fit, fit.measures = TRUE) ## lavaan 0.6.16 ended normally after 29 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 21 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 35.611 ## Degrees of freedom 24 ## P-value (Chi-square) 0.060 ## ## Model Test Baseline Model: ## ## Test statistic 910.621 ## Degrees of freedom 36 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.987 ## Tucker-Lewis Index (TLI) 0.980 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -2998.290 ## Loglikelihood unrestricted model (H1) -2980.484 ## ## Akaike (AIC) 6038.580 ## Bayesian (BIC) 6112.530 ## Sample-size adjusted Bayesian (SABIC) 6045.959 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.044 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.073 ## P-value H_0: RMSEA <= 0.050 0.599 ## P-value H_0: RMSEA >= 0.080 0.017 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.037 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.036 0.068 15.308 0.000 ## attit_3 -1.002 0.067 -14.856 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 1.031 0.098 10.574 0.000 ## norm_3 0.932 0.093 10.013 0.000 ## control =~ ## control_1 1.000 ## control_2 0.862 0.129 6.699 0.000 ## control_3 0.968 0.133 7.290 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.340 0.069 4.957 0.000 ## control 0.475 0.073 6.468 0.000 ## norms ~~ ## control 0.338 0.064 5.254 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.418 0.052 8.047 0.000 ## .attit_2 0.310 0.047 6.633 0.000 ## .attit_3 0.369 0.049 7.577 0.000 ## .norm_1 0.504 0.071 7.130 0.000 ## .norm_2 0.469 0.071 6.591 0.000 ## .norm_3 0.635 0.075 8.465 0.000 ## .control_1 0.614 0.078 7.905 0.000 ## .control_2 0.865 0.091 9.520 0.000 ## .control_3 0.762 0.087 8.758 0.000 ## attitudes 0.885 0.116 7.620 0.000 ## norms 0.743 0.116 6.423 0.000 ## control 0.497 0.099 5.002 0.000 Click for explanation Yes, the model fits the data well, and the measurement parameters (e.g., factor loadings, residual variances) look reasonable. So, the data seem to support this measurement structure. Yes, all three latent variables are significantly, positively correlated. Yes. The measurement structure is supported, so we can use the latent variables to represent the respective constructs in our subsequent SEM. The TORA doesn’t actually say anything about the associations between these three factors, but it makes sense that they would be positively associated. So, we should find this result comforting. 6.4.3 Estimate the basic TORA model as an SEM. Predict intention from attitudes and norms. Predict condom use from intention. Use the latent versions of attitudes and norms. Covary the attitudes and norms factors. Does the model fit well? Do the estimates align with the TORA? How much variance in intention and condom use are explained by the model? Click to show code mod <- ' ## Define the latent variables: attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 ## Define the structural model: intent ~ attitudes + norms behavior ~ intent ' fit <- sem(mod, data = condom) summary(fit, fit.measures = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 24 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 18 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 27.890 ## Degrees of freedom 18 ## P-value (Chi-square) 0.064 ## ## Model Test Baseline Model: ## ## Test statistic 1089.407 ## Degrees of freedom 28 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.991 ## Tucker-Lewis Index (TLI) 0.986 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -2533.616 ## Loglikelihood unrestricted model (H1) -2519.671 ## ## Akaike (AIC) 5103.232 ## Bayesian (BIC) 5166.618 ## Sample-size adjusted Bayesian (SABIC) 5109.557 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.047 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.079 ## P-value H_0: RMSEA <= 0.050 0.523 ## P-value H_0: RMSEA >= 0.080 0.046 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.036 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.039 0.068 15.365 0.000 ## attit_3 -1.002 0.067 -14.850 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.983 0.087 11.333 0.000 ## norm_3 0.935 0.087 10.778 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.439 0.063 6.990 0.000 ## norms 0.693 0.077 8.977 0.000 ## behavior ~ ## intent 0.746 0.045 16.443 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.347 0.069 5.027 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.420 0.052 8.103 0.000 ## .attit_2 0.306 0.046 6.604 0.000 ## .attit_3 0.372 0.049 7.651 0.000 ## .norm_1 0.483 0.064 7.581 0.000 ## .norm_2 0.521 0.065 7.954 0.000 ## .norm_3 0.610 0.070 8.713 0.000 ## .intent 0.423 0.048 8.769 0.000 ## .behavior 0.603 0.054 11.180 0.000 ## attitudes 0.884 0.116 7.614 0.000 ## norms 0.765 0.113 6.767 0.000 ## ## R-Square: ## Estimate ## attit_1 0.678 ## attit_2 0.757 ## attit_3 0.705 ## norm_1 0.613 ## norm_2 0.587 ## norm_3 0.523 ## intent 0.639 ## behavior 0.520 Click for explanation Yes, the model still fits the data very well. Yes, the estimates all align with the TORA. Specifically, attitudes and norms both significantly predict intention, and intention significantly predicts condom use. The model explains 63.93% of the variance in intention and 51.96% of the variance in condom use. 6.4.4 Update your model to represent the extended TORA model that includes perceived behavioral control. Regress condom use onto perceived behavioral control. Use the latent variable representation of control. Covary all three exogenous latent factors. Does the model fit well? Do the estimates align with the updated TORA? How much variance in intention and condom use are explained by the model? Click to show code mod_tora <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms behavior ~ intent + control ' fit_tora <- sem(mod_tora, data = condom) summary(fit_tora, fit.measures = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 31 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 27 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 48.757 ## Degrees of freedom 39 ## P-value (Chi-square) 0.136 ## ## Model Test Baseline Model: ## ## Test statistic 1333.695 ## Degrees of freedom 55 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.992 ## Tucker-Lewis Index (TLI) 0.989 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -3551.160 ## Loglikelihood unrestricted model (H1) -3526.782 ## ## Akaike (AIC) 7156.320 ## Bayesian (BIC) 7251.400 ## Sample-size adjusted Bayesian (SABIC) 7165.807 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.032 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.057 ## P-value H_0: RMSEA <= 0.050 0.870 ## P-value H_0: RMSEA >= 0.080 0.000 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.033 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.033 0.068 15.221 0.000 ## attit_3 -1.025 0.068 -15.097 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.984 0.087 11.256 0.000 ## norm_3 0.955 0.088 10.881 0.000 ## control =~ ## control_1 1.000 ## control_2 0.859 0.127 6.789 0.000 ## control_3 0.997 0.131 7.609 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.447 0.063 7.100 0.000 ## norms 0.706 0.078 9.078 0.000 ## behavior ~ ## intent 0.563 0.063 8.923 0.000 ## control 0.454 0.119 3.805 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.342 0.068 5.011 0.000 ## control 0.474 0.072 6.548 0.000 ## norms ~~ ## control 0.352 0.064 5.521 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.432 0.052 8.381 0.000 ## .attit_2 0.330 0.046 7.220 0.000 ## .attit_3 0.344 0.046 7.439 0.000 ## .norm_1 0.496 0.063 7.820 0.000 ## .norm_2 0.533 0.065 8.152 0.000 ## .norm_3 0.595 0.069 8.643 0.000 ## .control_1 0.625 0.075 8.372 0.000 ## .control_2 0.876 0.090 9.757 0.000 ## .control_3 0.746 0.084 8.874 0.000 ## .intent 0.409 0.047 8.769 0.000 ## .behavior 0.542 0.052 10.423 0.000 ## attitudes 0.872 0.115 7.566 0.000 ## norms 0.751 0.112 6.709 0.000 ## control 0.485 0.096 5.059 0.000 ## ## R-Square: ## Estimate ## attit_1 0.668 ## attit_2 0.738 ## attit_3 0.727 ## norm_1 0.602 ## norm_2 0.577 ## norm_3 0.535 ## control_1 0.437 ## control_2 0.290 ## control_3 0.392 ## intent 0.651 ## behavior 0.566 Click for explanation Yes, the model still fits the data very well. Yes, the estimates all align with the updated TORA. Specifically, attitudes and norms both significantly predict intention, while intention and control both significantly predict condom use. The model explains 65.11% of the variance in intention and 56.62% of the variance in condom use. The TORA model explicitly forbids direct paths from attitudes and norms to behaviors; these effects should be fully mediated by the behavioral intention. The theory does not specify how perceived behavioral control should affect behaviors. There may be a direct effect of control on behavior, or the effect may be (partially) mediated by intention. 6.4.5 Evaluate the hypothesized indirect effects of attitudes and norms. Include attitudes, norms, and control in your model as in 6.4.4. Does intention significantly mediate the effects of attitudes and norms on behavior? Don’t forget to follow all the steps we covered for testing mediation. Are both of the above effects completely mediated? Do these results comport with the TORA? Why or why not? Click for explanation mod <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ a1 * attitudes + a2 * norms behavior ~ b * intent + control + attitudes + norms ie_att := a1 * b ie_norm := a2 * b ' set.seed(235711) fit <- sem(mod, data = condom, se = "bootstrap", bootstrap = 1000) summary(fit, ci = TRUE) ## lavaan 0.6.16 ended normally after 36 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 29 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 48.629 ## Degrees of freedom 37 ## P-value (Chi-square) 0.096 ## ## Parameter Estimates: ## ## Standard errors Bootstrap ## Number of requested bootstrap draws 1000 ## Number of successful bootstrap draws 1000 ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes =~ ## attit_1 1.000 1.000 1.000 ## attit_2 1.033 0.060 17.261 0.000 0.925 1.165 ## attit_3 -1.025 0.064 -15.894 0.000 -1.163 -0.902 ## norms =~ ## norm_1 1.000 1.000 1.000 ## norm_2 0.984 0.071 13.794 0.000 0.843 1.127 ## norm_3 0.955 0.093 10.324 0.000 0.792 1.157 ## control =~ ## control_1 1.000 1.000 1.000 ## control_2 0.860 0.113 7.624 0.000 0.653 1.098 ## control_3 0.996 0.147 6.790 0.000 0.748 1.320 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## intent ~ ## attitudes (a1) 0.447 0.067 6.674 0.000 0.324 0.585 ## norms (a2) 0.706 0.078 9.094 0.000 0.569 0.878 ## behavior ~ ## intent (b) 0.545 0.075 7.282 0.000 0.389 0.686 ## control 0.428 0.232 1.847 0.065 0.046 0.934 ## attitudes 0.010 0.122 0.084 0.933 -0.249 0.226 ## norms 0.041 0.118 0.345 0.730 -0.194 0.266 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes ~~ ## norms 0.342 0.070 4.883 0.000 0.208 0.480 ## control 0.475 0.069 6.850 0.000 0.344 0.612 ## norms ~~ ## control 0.350 0.067 5.218 0.000 0.221 0.484 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## .attit_1 0.432 0.050 8.720 0.000 0.331 0.526 ## .attit_2 0.330 0.045 7.382 0.000 0.238 0.415 ## .attit_3 0.343 0.049 6.992 0.000 0.244 0.444 ## .norm_1 0.496 0.060 8.305 0.000 0.376 0.614 ## .norm_2 0.533 0.077 6.951 0.000 0.390 0.687 ## .norm_3 0.594 0.069 8.597 0.000 0.443 0.719 ## .control_1 0.624 0.076 8.216 0.000 0.477 0.763 ## .control_2 0.875 0.092 9.495 0.000 0.686 1.052 ## .control_3 0.745 0.079 9.398 0.000 0.574 0.889 ## .intent 0.409 0.050 8.169 0.000 0.309 0.507 ## .behavior 0.544 0.058 9.379 0.000 0.415 0.639 ## attitudes 0.872 0.104 8.387 0.000 0.675 1.077 ## norms 0.751 0.099 7.557 0.000 0.556 0.941 ## control 0.486 0.096 5.042 0.000 0.303 0.684 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## ie_att 0.244 0.050 4.860 0.000 0.150 0.352 ## ie_norm 0.385 0.066 5.835 0.000 0.268 0.527 Yes, both indirect effects are significant according to the 95% bootstrapped CIs. Yes, both effects are completely moderated by behavioral intention. We can infer as much because the direct effects of attitudes and norms on condom use are both nonsignificant. Yes, these results comport with the TORA. Both effects are fully mediated, as the theory stipulates. In addition to evaluating the significance of the indirect and direct effects, we can also take a model-comparison perspective. We can use model comparisons to test if removing the direct effects of attitudes and norms on condom use significantly decreases model fit. In other words, are those paths needed to accurately represent the data, or are they “dead weight”. 6.4.6 Use a \\(\\Delta \\chi^2\\) test to evaluate the necessity of including the direct effects of attitudes and norms on condom use in the model. What is your conclusion? Click for explanation We only need to compare the fit of the model with the direct effects included to the fit of the model without the direct effects. We’ve already estimated both models, so we can simply submit the fitted lavaan objects to the anova() function. anova(fit, fit_tora) The \\(\\Delta \\chi^2\\) test is not significant. So, we have not lost a significant amount of fit by fixing the direct effects to zero. In other words, the complete mediation model explains the data just as well as the partial mediation model. So, we should probably prefer the more parsimonious model. 6.4.7 Use some statistical means of evaluating the most plausible way to include perceived behavioral control into the model. Choose between the following three options: control predicts behavior via a direct, un-mediated effect. control predicts behavior via an indirect effect that is completely mediated by intention. control predicts behavior via both an indirect effect through intention and a residual direct effect. Hint: There is more than one way to approach this problem. Approach 1: Testing Effects Click to show code One way to tackle this problem is to test the indirect, direct, and total effects. ## Allow for partial mediation: mod1 <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms + a * control behavior ~ b * intent + c * control ie := a * b total := ie + c ' set.seed(235711) fit1 <- sem(mod1, data = condom, se = "bootstrap", bootstrap = 1000) summary(fit1, ci = TRUE) ## lavaan 0.6.16 ended normally after 33 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 28 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 47.389 ## Degrees of freedom 38 ## P-value (Chi-square) 0.141 ## ## Parameter Estimates: ## ## Standard errors Bootstrap ## Number of requested bootstrap draws 1000 ## Number of successful bootstrap draws 1000 ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes =~ ## attit_1 1.000 1.000 1.000 ## attit_2 1.034 0.060 17.222 0.000 0.925 1.167 ## attit_3 -1.021 0.064 -15.877 0.000 -1.158 -0.898 ## norms =~ ## norm_1 1.000 1.000 1.000 ## norm_2 0.985 0.071 13.803 0.000 0.848 1.133 ## norm_3 0.948 0.093 10.204 0.000 0.786 1.155 ## control =~ ## control_1 1.000 1.000 1.000 ## control_2 0.861 0.113 7.635 0.000 0.653 1.100 ## control_3 0.996 0.142 7.020 0.000 0.760 1.318 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## intent ~ ## attitudes 0.357 0.115 3.113 0.002 0.146 0.603 ## norms 0.646 0.095 6.794 0.000 0.473 0.859 ## control (a) 0.199 0.199 1.002 0.317 -0.188 0.633 ## behavior ~ ## intent (b) 0.551 0.074 7.487 0.000 0.391 0.683 ## control (c) 0.469 0.142 3.298 0.001 0.231 0.791 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## attitudes ~~ ## norms 0.344 0.070 4.905 0.000 0.210 0.481 ## control 0.471 0.069 6.838 0.000 0.342 0.608 ## norms ~~ ## control 0.345 0.066 5.240 0.000 0.215 0.481 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## .attit_1 0.429 0.050 8.628 0.000 0.329 0.524 ## .attit_2 0.325 0.045 7.230 0.000 0.233 0.408 ## .attit_3 0.347 0.049 7.011 0.000 0.248 0.455 ## .norm_1 0.490 0.060 8.172 0.000 0.373 0.612 ## .norm_2 0.525 0.076 6.869 0.000 0.385 0.684 ## .norm_3 0.599 0.070 8.529 0.000 0.447 0.729 ## .control_1 0.626 0.074 8.429 0.000 0.479 0.761 ## .control_2 0.875 0.092 9.522 0.000 0.689 1.049 ## .control_3 0.748 0.078 9.532 0.000 0.579 0.893 ## .intent 0.412 0.050 8.283 0.000 0.307 0.504 ## .behavior 0.541 0.055 9.873 0.000 0.423 0.639 ## attitudes 0.875 0.104 8.385 0.000 0.676 1.081 ## norms 0.757 0.099 7.616 0.000 0.560 0.949 ## control 0.484 0.095 5.092 0.000 0.306 0.683 ## ## Defined Parameters: ## Estimate Std.Err z-value P(>|z|) ci.lower ci.upper ## ie 0.110 0.105 1.048 0.295 -0.105 0.309 ## total 0.578 0.186 3.108 0.002 0.235 0.971 Click for explanation From the above results, we can see that the direct and total effects are both significant, but the indirect effect is not. Hence, it probably makes the most sense to include control via a direct (non-mediated) effect on behavior. Approach 2.1: Nested Model Comparison Click to show code We can also approach this problem from a model-comparison perspective. We can fit models that encode each pattern of constraints and check which one best represents the data. ## Force complete mediation: mod2 <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms + control behavior ~ intent ' ## Force no mediation: mod3 <- ' attitudes =~ attit_1 + attit_2 + attit_3 norms =~ norm_1 + norm_2 + norm_3 control =~ control_1 + control_2 + control_3 intent ~ attitudes + norms behavior ~ intent + control ' ## Estimate the two restricted models: fit2 <- sem(mod2, data = condom) fit3 <- sem(mod3, data = condom) ## Check the results: summary(fit2) ## lavaan 0.6.16 ended normally after 33 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 27 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 62.797 ## Degrees of freedom 39 ## P-value (Chi-square) 0.009 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.033 0.068 15.295 0.000 ## attit_3 -1.018 0.068 -15.087 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.985 0.087 11.305 0.000 ## norm_3 0.947 0.087 10.845 0.000 ## control =~ ## control_1 1.000 ## control_2 0.864 0.126 6.855 0.000 ## control_3 0.958 0.129 7.417 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.352 0.096 3.669 0.000 ## norms 0.644 0.088 7.347 0.000 ## control 0.207 0.163 1.268 0.205 ## behavior ~ ## intent 0.746 0.045 16.443 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.345 0.069 5.023 0.000 ## control 0.476 0.073 6.513 0.000 ## norms ~~ ## control 0.346 0.065 5.361 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.427 0.051 8.295 0.000 ## .attit_2 0.325 0.046 7.101 0.000 ## .attit_3 0.349 0.047 7.477 0.000 ## .norm_1 0.490 0.064 7.702 0.000 ## .norm_2 0.524 0.065 8.025 0.000 ## .norm_3 0.600 0.069 8.652 0.000 ## .control_1 0.610 0.076 8.015 0.000 ## .control_2 0.861 0.090 9.580 0.000 ## .control_3 0.769 0.086 8.938 0.000 ## .intent 0.412 0.046 8.890 0.000 ## .behavior 0.603 0.054 11.180 0.000 ## attitudes 0.877 0.115 7.596 0.000 ## norms 0.757 0.112 6.733 0.000 ## control 0.500 0.098 5.076 0.000 summary(fit3) ## lavaan 0.6.16 ended normally after 31 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 27 ## ## Number of observations 250 ## ## Model Test User Model: ## ## Test statistic 48.757 ## Degrees of freedom 39 ## P-value (Chi-square) 0.136 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) ## attitudes =~ ## attit_1 1.000 ## attit_2 1.033 0.068 15.221 0.000 ## attit_3 -1.025 0.068 -15.097 0.000 ## norms =~ ## norm_1 1.000 ## norm_2 0.984 0.087 11.256 0.000 ## norm_3 0.955 0.088 10.881 0.000 ## control =~ ## control_1 1.000 ## control_2 0.859 0.127 6.789 0.000 ## control_3 0.997 0.131 7.609 0.000 ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## intent ~ ## attitudes 0.447 0.063 7.100 0.000 ## norms 0.706 0.078 9.078 0.000 ## behavior ~ ## intent 0.563 0.063 8.923 0.000 ## control 0.454 0.119 3.805 0.000 ## ## Covariances: ## Estimate Std.Err z-value P(>|z|) ## attitudes ~~ ## norms 0.342 0.068 5.011 0.000 ## control 0.474 0.072 6.548 0.000 ## norms ~~ ## control 0.352 0.064 5.521 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .attit_1 0.432 0.052 8.381 0.000 ## .attit_2 0.330 0.046 7.220 0.000 ## .attit_3 0.344 0.046 7.439 0.000 ## .norm_1 0.496 0.063 7.820 0.000 ## .norm_2 0.533 0.065 8.152 0.000 ## .norm_3 0.595 0.069 8.643 0.000 ## .control_1 0.625 0.075 8.372 0.000 ## .control_2 0.876 0.090 9.757 0.000 ## .control_3 0.746 0.084 8.874 0.000 ## .intent 0.409 0.047 8.769 0.000 ## .behavior 0.542 0.052 10.423 0.000 ## attitudes 0.872 0.115 7.566 0.000 ## norms 0.751 0.112 6.709 0.000 ## control 0.485 0.096 5.059 0.000 ## Do either of the restricted models fit worse than the partial mediation model? anova(fit1, fit2) anova(fit1, fit3) Click for explanation The above \\(\\Delta \\chi^2\\) tests tell us that the full mediation model fits significantly worse than the partial mediation model. Hence, forcing full mediation by fixing the direct effect to zero is an unreasonable restraint. The total effect model, on the other hand, does not fit significantly worse than the partial mediation model. So, we can conclude that removing the indirect effect and modeling the influence of control on behavior as an un-mediated direct association represents the data just as well as a model that allows for both indirect and direct effects. Hence, we should prefer the more parsimonious total effects model. Approach 2.2: Non-Nested Model Comparison Click to show code We can also use information criteria to compare our models. The two most popular information criteria are the Akaike’s Information Criterion (AIC) and the Bayesian Information Criterion (BIC). ## Which model is the most parsimonious representation of the data? AIC(fit1, fit2, fit3) BIC(fit1, fit2, fit3) Click for explanation While the effect tests and the nested model comparisons both lead us to prefer the non-mediated model, we cannot directly say that the complete mediation model fits significantly worse than the non-mediated model. We have not directly compared those two models, and we cannot do so with the \\(\\Delta \\chi^2\\). We cannot do such a test because these two models are not nested: we must both add and remove a path to get from one model specification to the other. Also, both models have the same degrees of freedom, so we cannot define a sampling distribution against which we would compare the \\(\\Delta \\chi^2\\), anyway. We can use information criteria to get around this problem, though. Information criteria can be used to compare both nested and non-nested models. These criteria are designed to rank models by balancing their fit to the data and their complexity. When comparing models based on information criteria, a lower value indicates a better model in the sense of a better balance of fit and parsimony. The above results show that both the AIC and the BIC agree that the no-mediation model is the best. Conclusion Click for explanation So, in the end, regardless of how we approach the question, all of our results suggest modeling perceived behavioral control as a direct, non-mediated predictor of condom use. End of In-Class Exercises "],["multiple-group-models.html", "7 Multiple Group Models", " 7 Multiple Group Models This week, you will cover multiple group modeling and measurement invariance testing in the SEM/CFA context. Homework before the lecture Watch the Lecture Recording for this week. Homework before the practical Complete the At-Home Exercises. Practical content During the practical you will work on the In-Class Exercises. "],["lecture-6.html", "7.1 Lecture", " 7.1 Lecture In this lecture, we will explore how you can incorporate grouping factors into your CFA and SEM analyses. We’ll cover three general topics: The multiple group modeling framework Measurement invariance testing Using multiple group models to test for moderation 7.1.1 Recordings Multiple Group Models Measurement Invariance Measurement Invariance Examples Moderation by Group 7.1.2 Slides You can download the lecture slides here "],["reading-6.html", "7.2 Reading", " 7.2 Reading There is no official reading this week. Please contemplate the following image instead. \\[\\\\[12pt]\\] "],["at-home-exercises-6.html", "7.3 At-Home Exercises", " 7.3 At-Home Exercises 7.3.1 Multiple-Group Path Analysis To fix ideas, we’ll start these practical exercises by re-running part of the moderation analysis from the Week 3 At-Home Exercises as a multiple group model. 7.3.1.1 Load the Sesam2.sav data. NOTE: Unless otherwise specified, all analyses in Section 7.3.1 use these data. Click to show code library(haven) # Read the data into an object called 'sesam2': sesam2 <- read_sav("Sesam2.sav") VIEWCAT is a nominal grouping variable, but it is represented as a numeric variable in the sesam2 data. The levels represent the following frequencies of Sesame Street viewership of the children in the data: VIEWCAT = 1: Rarely/Never VIEWCAT = 2: 2–3 times a week VIEWCAT = 3: 4–5 times a week VIEWCAT = 4: > 5 times a week We will use VIEWCAT as the grouping variable in our path model. To do so, we don’t really need to convert VIEWCAT into a factor, but, if we do, lavaan will give our groups meaningful labels in the output. That added clarity can be pretty helpful. 7.3.1.2 Convert VIEWCAT into a factor. Make sure that VIEWCAT = 1 is the reference group. Assign the factor labels denoted above. Click to show code library(dplyr) ## Store the old version for checking: tmp <- sesam2$VIEWCAT ## Convert 'VIEWCAT' to a factor: sesam2 <- mutate(sesam2, VIEWCAT = factor(VIEWCAT, labels = c("Rarely/never", "2-3 times per week", "4-5 times per week", "> 5 times per week") ) ) ## Check the conversion: table(old = tmp, new = sesam2$VIEWCAT, useNA = "always") ## new ## old Rarely/never 2-3 times per week 4-5 times per week > 5 times per week ## 1 25 0 0 0 ## 2 0 44 0 0 ## 3 0 0 57 0 ## 4 0 0 0 53 ## <NA> 0 0 0 0 ## new ## old <NA> ## 1 0 ## 2 0 ## 3 0 ## 4 0 ## <NA> 0 7.3.1.3 Create a conditional slopes plot to visualize the effect of AGE on POSTNUMB within each of the VIEWCAT groups. Based on this visualization, do you think it is reasonable to expect that VIEWCAT moderates the effect of AGE on POSTNUMB? Click to show code library(ggplot2) ggplot(sesam2, aes(AGE, POSTNUMB, color = VIEWCAT)) + geom_point() + geom_smooth(method = "lm", se = FALSE) Click for explanation The regression lines representing the conditional focal effects are not parallel, so there appears to be some level of moderation. That being said, the differences are pretty small, so the moderation may not be significant (i.e., the non-parallel regression lines may simply be reflecting sampling variability). We will use path analysis to test if VIEWCAT moderates the effect of AGE on POSTNUMB. This analysis will entail three steps: Estimate the unrestricted multiple-group model wherein we regress POSTNUMB onto AGE and specify VIEWCAT as the grouping factor. Estimate the restricted model wherein we constrain the AGE \\(rightarrow\\) POSTNUMB effect to be equal in all VIEWCAT groups. Conduct a \\(\\Delta \\chi^2\\) test to compare the fit of the two models. 7.3.1.4 Estimate the unrestricted path model described above. Include the intercept term in your model. Judging from the focal effects estimate in each group, do you think moderation is plausible? Click to show code library(lavaan) ## Estimate the additive model a view the results: out_full <- sem('POSTNUMB ~ 1 + AGE', data = sesam2, group = "VIEWCAT") summary(out_full) ## lavaan 0.6.16 ended normally after 1 iteration ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 12 ## ## Number of observations per group: ## Rarely/never 25 ## 4-5 times per week 57 ## > 5 times per week 53 ## 2-3 times per week 44 ## ## Model Test User Model: ## ## Test statistic 0.000 ## Degrees of freedom 0 ## Test statistic for each group: ## Rarely/never 0.000 ## 4-5 times per week 0.000 ## > 5 times per week 0.000 ## 2-3 times per week 0.000 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## ## Group 1 [Rarely/never]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE 0.747 0.239 3.118 0.002 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB -18.721 12.142 -1.542 0.123 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 73.285 20.728 3.536 0.000 ## ## ## Group 2 [4-5 times per week]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE 0.554 0.234 2.369 0.018 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 4.861 12.178 0.399 0.690 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 135.923 25.461 5.339 0.000 ## ## ## Group 3 [> 5 times per week]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE 0.405 0.214 1.894 0.058 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 15.676 11.249 1.394 0.163 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 115.942 22.523 5.148 0.000 ## ## ## Group 4 [2-3 times per week]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE 0.729 0.255 2.855 0.004 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB -8.747 13.003 -0.673 0.501 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 112.019 23.882 4.690 0.000 Click for explanation There are some notable differences in the AGE \\(\\rightarrow\\) POSTNUMB focal effect between VIEWCAT groups. It looks like VIEWCAT could moderate the focal effect. 7.3.1.5 Estimate the restricted model described above. Equate the focal effect across all VIEWCAT groups. Click to show code ## Estimate the restricted model and view the results: out_res <- sem('POSTNUMB ~ 1 + c("b1", "b1", "b1", "b1") * AGE', data = sesam2, group = "VIEWCAT") summary(out_res) ## lavaan 0.6.16 ended normally after 38 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 12 ## Number of equality constraints 3 ## ## Number of observations per group: ## Rarely/never 25 ## 4-5 times per week 57 ## > 5 times per week 53 ## 2-3 times per week 44 ## ## Model Test User Model: ## ## Test statistic 1.486 ## Degrees of freedom 3 ## P-value (Chi-square) 0.685 ## Test statistic for each group: ## Rarely/never 0.413 ## 4-5 times per week 0.027 ## > 5 times per week 0.760 ## 2-3 times per week 0.287 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## ## Group 1 [Rarely/never]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE (b1) 0.592 0.118 5.032 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB -10.966 6.154 -1.782 0.075 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 74.505 21.073 3.536 0.000 ## ## ## Group 2 [4-5 times per week]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE (b1) 0.592 0.118 5.032 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 2.869 6.275 0.457 0.647 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 135.988 25.473 5.339 0.000 ## ## ## Group 3 [> 5 times per week]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE (b1) 0.592 0.118 5.032 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 5.923 6.313 0.938 0.348 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 117.616 22.848 5.148 0.000 ## ## ## Group 4 [2-3 times per week]: ## ## Regressions: ## Estimate Std.Err z-value P(>|z|) ## POSTNUMB ~ ## AGE (b1) 0.592 0.118 5.032 0.000 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB -1.826 6.157 -0.297 0.767 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) ## .POSTNUMB 112.751 24.039 4.690 0.000 7.3.1.6 Test for moderation by comparing the full and restricted models from 7.3.1.4 and 7.3.1.5, respectively: Does VIEWCAT significantly moderate the effect of AGE on POSTNUMB? Click to show code ## Test for moderation: anova(out_full, out_res) Click for explanation No, VIEWCAT does not significantly moderate the effect of AGE on POSTNUMB (\\(\\Delta \\chi^2[3] = 1.486\\), \\(p = 0.685\\)). 7.3.2 Multiple-Group CFA In the next part of these exercises, we will estimate a multiple-group CFA to evaluate the measurement structure of a scale assessing Prolonged Grief Disorder. The relevant data are contained in the PGDdata2.txt file. This dataset consists of a grouping variable, Kin2 (with two levels: “partner” and “else”) and 5 items taken from the Inventory of Complicated Grief: Yearning Part of self died Difficulty accepting the loss Avoiding reminders of deceased Bitterness about the loss You can find more information about this scale in Boelen et al. (2010). 7.3.2.1 Load the PGDdata2.txt data. Use the read.table() function to load the data. Convert the missing values to NA via the na.strings argument. Retain the column labels via the header argument. Specify the field delimiter as the tab character (i.e., \"\\t\"). Exclude any cases with missing values on Kin2. NOTE: Unless otherwise specified, all analyses in Section 7.3.2 use these data. Click to show code ## Load the data: pgd <- read.table("PGDdata2.txt", na.strings = "-999", header = TRUE, sep = "\\t") %>% filter(!is.na(Kin2)) ## Check the results: head(pgd) summary(pdg) str(pgd) ## Kin2 b1pss1 b2pss2 b3pss3 ## Min. :0.0000 Min. :0.000 Min. :0.0000 Min. :0.0000 ## 1st Qu.:0.0000 1st Qu.:0.000 1st Qu.:0.0000 1st Qu.:0.0000 ## Median :1.0000 Median :1.000 Median :0.0000 Median :1.0000 ## Mean :0.6661 Mean :1.236 Mean :0.4622 Mean :0.9771 ## 3rd Qu.:1.0000 3rd Qu.:2.000 3rd Qu.:1.0000 3rd Qu.:1.0000 ## Max. :1.0000 Max. :5.000 Max. :3.0000 Max. :5.0000 ## NA's :1 ## b4pss4 b5pss5 ## Min. :0.000 Min. :0.0000 ## 1st Qu.:0.000 1st Qu.:0.0000 ## Median :1.000 Median :0.0000 ## Mean :1.009 Mean :0.6761 ## 3rd Qu.:2.000 3rd Qu.:1.0000 ## Max. :3.000 Max. :3.0000 ## NA's :1 ## 'data.frame': 569 obs. of 6 variables: ## $ Kin2 : int 0 0 1 1 0 1 1 1 1 1 ... ## $ b1pss1: int 1 1 1 1 1 2 1 3 1 1 ... ## $ b2pss2: int 1 0 1 0 1 2 1 2 0 0 ... ## $ b3pss3: int 1 0 1 1 2 2 1 2 1 1 ... ## $ b4pss4: int 1 1 1 1 0 2 2 3 0 1 ... ## $ b5pss5: int 1 0 0 0 0 1 2 3 0 0 ... 7.3.2.2 Run a single-group CFA wherein the five scale variables described above indicate a single latent factor. Do not include any grouping variable. Use the default settings in the cfa() function. Click to show code ## Define the model syntax: cfaMod <- 'grief =~ b1pss1 + b2pss2 + b3pss3 + b4pss4 + b5pss5' ## Estimate the model: out0 <- cfa(cfaMod, data = pgd) 7.3.2.3 Summarize the evaluate the fitted CFA Does the model fit well? Are the items homogeneously associated with the latent factor? Which item is most weakly associated with the latent factor? Click to show code ## Summarize the fitted model: summary(out0, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 19 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 10 ## ## Used Total ## Number of observations 567 569 ## ## Model Test User Model: ## ## Test statistic 8.110 ## Degrees of freedom 5 ## P-value (Chi-square) 0.150 ## ## Model Test Baseline Model: ## ## Test statistic 775.364 ## Degrees of freedom 10 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.996 ## Tucker-Lewis Index (TLI) 0.992 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -3219.918 ## Loglikelihood unrestricted model (H1) -3215.863 ## ## Akaike (AIC) 6459.836 ## Bayesian (BIC) 6503.240 ## Sample-size adjusted Bayesian (SABIC) 6471.495 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.033 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.073 ## P-value H_0: RMSEA <= 0.050 0.710 ## P-value H_0: RMSEA >= 0.080 0.023 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.018 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## grief =~ ## b1pss1 1.000 0.752 0.759 ## b2pss2 0.454 0.043 10.570 0.000 0.341 0.495 ## b3pss3 0.831 0.058 14.445 0.000 0.625 0.691 ## b4pss4 0.770 0.055 14.010 0.000 0.579 0.667 ## b5pss5 0.817 0.057 14.410 0.000 0.614 0.689 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .b1pss1 0.416 0.037 11.300 0.000 0.416 0.424 ## .b2pss2 0.358 0.023 15.549 0.000 0.358 0.755 ## .b3pss3 0.427 0.033 13.117 0.000 0.427 0.522 ## .b4pss4 0.419 0.031 13.599 0.000 0.419 0.555 ## .b5pss5 0.417 0.032 13.160 0.000 0.417 0.525 ## grief 0.565 0.059 9.514 0.000 1.000 1.000 ## ## R-Square: ## Estimate ## b1pss1 0.576 ## b2pss2 0.245 ## b3pss3 0.478 ## b4pss4 0.445 ## b5pss5 0.475 Click for explanation The model fits the data quite well (\\(\\chi^2[5] = 8.11\\), \\(p = 0.15\\), \\(\\textit{RMSEA} = 0.033\\), \\(\\textit{CFI} = 0.996\\), \\(\\textit{SRMR} = 0.018\\)). All of the indicators appear to be more-or-less equally good indicators of the latent factor except for b2pss2 which has a standardized factor loading of \\(\\lambda = 0.495\\) and \\(R^2 = 0.245\\). 7.3.2.4 Rerun the CFA from 7.3.2.2 as a multiple-group model. Use the Kin2 variable as the grouping factor. Do not place any equality constraints across groups. Click to show code out1 <- cfa(cfaMod, data = pgd, group = "Kin2") 7.3.2.5 Summarize the fitted multiple-group CFA from 7.3.2.4. Does the two-group model fit the data well? Do you notice any salient differences between the two sets of within-group estimates? Click to show code summary(out1, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE) ## lavaan 0.6.16 ended normally after 27 iterations ## ## Estimator ML ## Optimization method NLMINB ## Number of model parameters 30 ## ## Number of observations per group: Used Total ## 0 188 190 ## 1 379 379 ## ## Model Test User Model: ## ## Test statistic 11.317 ## Degrees of freedom 10 ## P-value (Chi-square) 0.333 ## Test statistic for each group: ## 0 8.976 ## 1 2.340 ## ## Model Test Baseline Model: ## ## Test statistic 781.358 ## Degrees of freedom 20 ## P-value 0.000 ## ## User Model versus Baseline Model: ## ## Comparative Fit Index (CFI) 0.998 ## Tucker-Lewis Index (TLI) 0.997 ## ## Loglikelihood and Information Criteria: ## ## Loglikelihood user model (H0) -3206.363 ## Loglikelihood unrestricted model (H1) -3200.705 ## ## Akaike (AIC) 6472.727 ## Bayesian (BIC) 6602.937 ## Sample-size adjusted Bayesian (SABIC) 6507.701 ## ## Root Mean Square Error of Approximation: ## ## RMSEA 0.022 ## 90 Percent confidence interval - lower 0.000 ## 90 Percent confidence interval - upper 0.070 ## P-value H_0: RMSEA <= 0.050 0.789 ## P-value H_0: RMSEA >= 0.080 0.018 ## ## Standardized Root Mean Square Residual: ## ## SRMR 0.017 ## ## Parameter Estimates: ## ## Standard errors Standard ## Information Expected ## Information saturated (h1) model Structured ## ## ## Group 1 [0]: ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## grief =~ ## b1pss1 1.000 0.702 0.712 ## b2pss2 0.372 0.076 4.922 0.000 0.261 0.410 ## b3pss3 0.938 0.118 7.986 0.000 0.659 0.709 ## b4pss4 0.909 0.116 7.848 0.000 0.638 0.691 ## b5pss5 0.951 0.122 7.774 0.000 0.667 0.683 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .b1pss1 1.346 0.072 18.727 0.000 1.346 1.366 ## .b2pss2 0.441 0.046 9.499 0.000 0.441 0.693 ## .b3pss3 1.059 0.068 15.618 0.000 1.059 1.139 ## .b4pss4 1.122 0.067 16.671 0.000 1.122 1.216 ## .b5pss5 0.745 0.071 10.442 0.000 0.745 0.762 ## grief 0.000 0.000 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .b1pss1 0.478 0.067 7.118 0.000 0.478 0.493 ## .b2pss2 0.338 0.037 9.205 0.000 0.338 0.832 ## .b3pss3 0.430 0.060 7.170 0.000 0.430 0.498 ## .b4pss4 0.445 0.060 7.408 0.000 0.445 0.522 ## .b5pss5 0.511 0.068 7.519 0.000 0.511 0.534 ## grief 0.493 0.098 5.007 0.000 1.000 1.000 ## ## R-Square: ## Estimate ## b1pss1 0.507 ## b2pss2 0.168 ## b3pss3 0.502 ## b4pss4 0.478 ## b5pss5 0.466 ## ## ## Group 2 [1]: ## ## Latent Variables: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## grief =~ ## b1pss1 1.000 0.769 0.778 ## b2pss2 0.502 0.052 9.597 0.000 0.386 0.542 ## b3pss3 0.785 0.066 11.945 0.000 0.604 0.680 ## b4pss4 0.708 0.062 11.497 0.000 0.544 0.652 ## b5pss5 0.762 0.062 12.185 0.000 0.586 0.696 ## ## Intercepts: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .b1pss1 1.182 0.051 23.277 0.000 1.182 1.196 ## .b2pss2 0.475 0.037 12.973 0.000 0.475 0.666 ## .b3pss3 0.934 0.046 20.460 0.000 0.934 1.051 ## .b4pss4 0.955 0.043 22.270 0.000 0.955 1.144 ## .b5pss5 0.644 0.043 14.879 0.000 0.644 0.764 ## grief 0.000 0.000 0.000 ## ## Variances: ## Estimate Std.Err z-value P(>|z|) Std.lv Std.all ## .b1pss1 0.385 0.043 8.862 0.000 0.385 0.394 ## .b2pss2 0.359 0.029 12.468 0.000 0.359 0.706 ## .b3pss3 0.425 0.039 11.025 0.000 0.425 0.538 ## .b4pss4 0.401 0.035 11.420 0.000 0.401 0.575 ## .b5pss5 0.366 0.034 10.767 0.000 0.366 0.516 ## grief 0.592 0.073 8.081 0.000 1.000 1.000 ## ## R-Square: ## Estimate ## b1pss1 0.606 ## b2pss2 0.294 ## b3pss3 0.462 ## b4pss4 0.425 ## b5pss5 0.484 Click for explanation The two-group model also fits the data very well (\\(\\chi^2[10] = 11.32\\), \\(p = 0.333\\), \\(\\textit{RMSEA} = 0.022\\), \\(\\textit{CFI} = 0.998\\), \\(\\textit{SRMR} = 0.017\\)). No, there are no striking differences between the two sets of estimates. Although, there is certainly some variability between groups, the two sets of estimates don’t look systematically different. 7.3.2.6 Based on the above results, what can you conclude about configural, weak, and strong measurement invariance across the Kin2 groups? Click for explanation Configural invariance holds. The unrestricted multiple-group CFA fits the data adequately (very well, actually), and the measurement model parameters are reasonable in both groups. We cannot yet draw any conclusions about weak or strong invariance. We need to do the appropriate model comparison tests first. End of At-Home Exercises 7 "],["in-class-exercises-6.html", "7.4 In-Class Exercises", " 7.4 In-Class Exercises Coming soon to a GitBook near you! "],["wrap-up.html", "8 Wrap-Up", " 8 Wrap-Up There will be no new lecture or practical content this week. This is an open week that we’ll use to tie up any loose ends and wrap up the course content. "],["404.html", "Page not found", " Page not found The page you requested cannot be found (perhaps it was moved or renamed). You may want to try searching to find the page's new location, or use the table of contents to find the page you are looking for. "]] diff --git a/docs/software-setup.html b/docs/software-setup.html index 0e69a768..b2e3fa94 100644 --- a/docs/software-setup.html +++ b/docs/software-setup.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/statistical-modeling-path-analysis.html b/docs/statistical-modeling-path-analysis.html index 392f8ace..d02534d3 100644 --- a/docs/statistical-modeling-path-analysis.html +++ b/docs/statistical-modeling-path-analysis.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/typographic-conventions.html b/docs/typographic-conventions.html index c2303c89..32d65ea6 100644 --- a/docs/typographic-conventions.html +++ b/docs/typographic-conventions.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/weekly-preparation.html b/docs/weekly-preparation.html index 8365484d..6356ae13 100644 --- a/docs/weekly-preparation.html +++ b/docs/weekly-preparation.html @@ -440,7 +440,11 @@
  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/docs/willy_wallaby_from_wasatch_files/figure-html/unnamed-chunk-132-1.png b/docs/willy_wallaby_from_wasatch_files/figure-html/unnamed-chunk-132-1.png index 67835cd4651c3e7d939c10fcd9d7369fb98fb85d..2d23613f4f64e1a0cb7ec73ab19dce2f911c3c82 100644 GIT binary patch literal 70188 zcmeFZg;$hq)HRO!s3;Oj2?&TvmrA#yNO!k@(jg5)j6q0CODiGW-AYL#F?2{bL-)XU z@cq4i!*8u`eJmfJS;IJU-`91{KKtymukllm6UV(teiI7|3s+L&r4km_)n`~(SAJq& zhG*RVdl?M>yYWs!(;f?p()Z%OOCj@HZ?UlMV@bYzuHq8EHsS0}ggQRlDqg-}_Fov* zheuv=wiF*w);EVo zoBtjSW8HE2_rMbChTy*k-0%OR`uBk1*=78T2k&w3f4F|};ODh3Kd)Roczs3U@1=_e z1(*Kc+x%a>HoPhjFt2w8%=ggs=X&N`*RNlvrcSDITt&FKsf(%%Mt%A6Zgr5gQTqCI zUpC#Uh|G_rR~X8Zlas^g<(Y13mzYj3E}Dey=XQ;X*L}<~alxk&l#r5Y@9gaC={eZG z__>LVT~ItLEKa7e8CQVtl)(DN58?LSUrJsc-J9YY6RFCi+7ZQi9iQrSP}}C{e)7dh_AYVp0$e@@SqkK~DYX*H0-WC8bxv_sGb}t6u;5 zxU<;9T;xpi(%O1sYxZ}#d=k~&yS#7PBWh}DnEZCoS+VYf!FQh|2X%hDij&232kSAR z^WRI~%;)CkYd0Hk%uA0W)%7x0G&V)Z$jGXl)}OM;%gG7b%}Mkm3Ned3(imSKD|g#n zx=l@OBlk8}t=hCV`O0;?d;AU=PV1wC!Rtvs%r?T*yI=b_$IC|a ze=hFlHaU;A;On1Qu9C*a#tsg<%l+v%H*e0(i;0QtEOcqtxx3C^`+SFZ45?RSJKOTM zBZ`loU(Hw$DPlER5-Kvewz8ttcBwN-xQC+K?Q9!j10iN;t94-J;aJpd(*$&?V`D%0Kvv)f)?Ua+dn2veJpBJZSJ zRMSMYa|oq?THomI?ykqlz7S@&Ka#9(O1|8B0)kY{`_k#*Y{+(WaB%SIV6NN#hzT(& zEF5ApvBseBn(MSbF|)+quHXOrNh*o&-@h-l7~-~>Je{m{^J$oyn_F61^1-Dbsg_h! z?1fKLR#F-tA2&y3p}sx8LbyIvX`|;xfxTwe)XPPFsv|U8)bXrnUQh0cHP5vG{Aw$3UL;ks)ot=^rT%mF} zoosx3d?$XVQu;zm5Jg*C+jdw%!DC21L+10d6SR5h=w^{z0)JO57jxMUwd|mP0BKoS zt3ThKC-6I}Fh%;^ZGZAYBW9F`m7815_5||@U-)tNq&I=%%7A9Eu`J|Z=tJ>a-X`7g zmCoqI`4Sd! zO6zAYf<_-)4n^eJaH0Qk{rOpSa(7!e{dHV&m2zv{X89NR_#x>@o(Omme?oV~{E(iT ztRA1BpbnD=WWeuW%CrU!lWvVS`ua#rX-yI2z9#cs*dq6>X43W1((oZ?dwaLjt=|tH zJSfq!D%6gdsyosuHK(=N3F4V_Sxj)AqutopI669_(_+{sGUbfDibK*`tncP_@NZ%N6Q)e#flT_LKc}3SsvOL&OfC$S!*7 z95SfcTB**W%so0Vjho%?YWy5bFn+&hYg0ytgngp!;^ZC9o;&2&B9fq-@%xz-_-sMu&}6? z4_ngjKYTa~Pp~gflmajePhr&fxnT3z*x3BB13VrAT1Khc+t~p|^W=1Oby13V)+|qB z-J_)Z9Yhfw6JzG-LSi<>IZIgOeq=|u$MN(j$tengBrqE?2;D}X#CbbcDh)C1O&S&! zMnn;{7mHH=8&=r5KnF$OPiMAvO0P^Dch|_sNYj{Nrb>^!#bw-I0DYsR-;(@D>h z^)R}+qP4nqnx{{<_0Nx^U!{bSj*g8{$d@LBhxg~}*HiFVYj9)n)#}w-Pw>08!5ySk zfAd#y8p_j!wKnEX6Psn70IyJw#>>twK6vW~89Z0ln$j$+*lT z3|yD{(zbs$$O^dauhA#D8+y77T2J-&D?j=841qvIv*}VHPq%2LqvD0!%2Fl&lTJpN ziwBXv70mG{WYepGsz^jgETdmcT=OOiMNBW_Nf>L?mn!Lx^V`c}C=ZufczkmbcJSWq zi=Xd@ZX3(#Nm1?kF4I?Ed@mK*#8Ti^#o%AY8m@Ud;rvuC>hk5wsjB=5a!*#2pezmL zXxblb&%?Uokg`GX(eFJ!%1loTlwauE5E6cESZ$ej9+*K1nRfjnFC1K z^(yV=DQs#C#%G;R54U_say-l-0Etot+t7K^L`dp^48<3o%8E}SbQKi-EOf>7_xE#= z%NiQa0P4qbnmx$>yRpGGRWm@S{%c=)Z$4U|JR$kq*1-Xa2!#-0x5jk`YI|gCta()_ zF|!(jR)&84Szns8e3GyS1PtsHS;(pLTQ z^?_X2{Sb;+DkB!XZ0sBN$E0jJ8K#T9;?$Aq7c|N?WAN+(rGec)6C*pfq!e^{x1mq_ z=utectzmlvV~bIU{MX0N%@+X%x9tEfzka>jTxZ<*l_tMGN0Z}d)UfzB6i%DEqlLq* z*}}p?^?bdqwzk67I;O8bnW@CoGF4h8z6L2jbX_nS`OmY#>&Eur$4{T;Eg&9Q^v57D z!$zM7UxoUqD#+!r&`IXz=H{BPF;Pu1sMe4f!)u#d^D#R++we^tM1BWzAeGQ)0TU~$ z?AA3NtI^{hwM(rT3Sb_KNE9 zF)&Ej)&XII-KjsE3GgX}C4=~5@@$T5SDNxda0oXFx$Uc1*12pMiY4e)JE36nY+vKx z;LJNp$^K=YzH$TWZ3Cwr)@xVoGD}-$XP)#Q2?-N(ZQ*J20B!pl6JrHTj~+dO&r#IR zMz!lN%+JSI*8?>X55A{(>R;Md*Zu}-_I5PpXmoo zy1qU!nj^F(B_g`V`|kDMpC5egi1o}tnJ13|1SZN|2TB9aZqsLf`uI`!XpYczVRm}j zz}RxoS|8iM#N?oZy&j4gkNrYtsJxL{*y>O|5-Ka?#k|)ILJ~F|2K3px`SwUi&NgQG zn5Sa4b*HDN7q}B5eB7#FczpCZEbkv(uxpM)#}%kZN=o|TQHDj|0yc1w*ya$faZlg2 zFD4BeBY1IV`yW zdA5^e7^f6}8S>jO8`fP~udBJaxl@7F&KwxnjEFZdRx&kOI=Xh>XZfnxQSM&^fe5*y z2Zn|Y;MR4jovaFHry0LqL{=QThzJsH;o+-5NWr?t@>6vl^Q^7s*KdpR^74*XJNLMV zL-5Ge<&&mqki=o*kJqFPQWsvT~>H?m9G!VE3oXN2@dJ z;km!{ynpHEy>V!Km^j=IH+zIna)wnAwPyhNzeA|&5qr?Ur~|JL4GU8m+!qoQBvKaA zw$XnJkVQL_O5W^u^K$iX=K{bz9SaQSyNVeT_VLR1#tJtFq zKeJqER$ihloXjNSFb+KJo&#RKhp3Gxm)=RHP{W1X5q69E^r=xBGc)so zn*$0mcwytrH$y)f!+MS|O1)`@Lh1!FzC>o*th>lLL;@vb{HsGQl^2}XDMWrxq zZKC?1{v3mrX0Lrzt1|8h8DD33dRg-I>%Tlx?nZJFw7UPQz!-7*>zTL$4gcRV<^vhM z(9aoKK&bD8ic^J(2)`E8PBn`>`tTo?{RL7HZ|?73TvfVE%!7Iqqok~Cg0ROiv=_?C z%FGp{&EaD$EEV}r+ge){0*fgL35~japkv|VvCJJgwuXm)n3lGff{TVUV-gvOdDueGZ&hOle-@P?4O(R!2h$n)|k&l4zPWtJlk z&IMjzZvhv&d*={>;V26b^rnwuAD8wo=^q6J1;;wJc64<&WGj&8b*E{|e5azKqH{ND zwm8<)vcm8mVMTa>OvBPMG{nQnIZ!bU{ReaxP%f6H>g&_f(`~&0#{$VXS`O)rYY}@? z9=p9j3uUrVvB>OpmlFHc^=z5I#TjmWEv?wM{eFPG!1!CGNmw)rUjMq>uf6P01&QB$ zn(ow*dPhzch%Udw(k5gsxfE;*2^CddujxK|%7!5a_Hrw5_t&pqp`oEk9{VGsr541Q z5%F5NQO0w}ODQ47J&9JHfH70|fP7Iwule`aC*(Qi02sql5s|3^CShS=4vzk@F>UP7 zxH#siEa$)!W^}Ax z@xky~sXues!$LlY8k3;^>iYg+DAA46@k)F7L_uJ@W(@b97``kle+6+^3C%Wi4EF?_ zEP*fyIB%#p&_%~_n8-nAk_0?($EaBE1RBpU=lpk0PS$|se#bz16S=Ko4UN%|e$utX zMAW1Hg8x}-!L7}PYw)S-9M#>BPHSyTNlB4ask2`UK23{}&u7p$xq^?i@DSi&WoGL9 ztX$9v2NxGMG-oDQM@J{VYrR7wSF04-yc&~j;0NuToF!?G=;@h#in*ttMxw_n3Gdu7 z9QneuJ`i~K$;^cF6v~vduY*}VZ@kW9tm$y4Wo>;u$z{fmKGDSxs?U=rPk=td&JBHk z;Y-9L;Y|RYfFs{FnMKL@uj|y7qa|&deQj-w4<9xH-EX4vzjWJ+z`RsTOF?jpih@F0 z>7lo`_jdFTx8HZO|I()Pnr5{To7@I!5?L=JA<^2=!MuEAe?){YirrZMdj9n~z981u z#=Ko7toKs38{>RO9VSU0gNN#MhP!le!_iMbOoCq5)o{GyRmYcX!=bETUIm6$3_r+`SoYV!IT1heSK%>^E3I-k=P1d&9-%52r6eq z85yP^p#0!@B0TC-h(B6x!_F=*D=Xx<5@Cx2@8ldBxX3dwtUFX>M z5FU~qN_f=5?x1iHjzz`&Bb|3ZFU??^>t_#`p*si$a*?>1bBxZ`kQdzYI-Lhf=bw9l z2sBSQj5`Y~@A{wZKzs|yC<)G4LWDh)|5v5i)pNBRw`Rl|rDvyCy$E3C?IRU;SxH4J zs;YwT@!0{7+!hrOut(ZxRAZne)>3@xdFn!FlzYKeFIaaNR0OgrU5C{{*c>Nl*FEK5a;VPS9nRM#$Vh(G=ing?#P{+5$MIM=#B93>L{Tl!%znYF5q6ICG z&`)+|nI7|F9iBY)+;ki+Fqk(@hlX(b)rAT`O#D{EZN8njS%_0{RB&^8dU|k7{p#X} zkibBdxyMS*)~*Q|PFtgHaFTrH?8qnl4zS=XTsxo)@7dw6(21@iX=k9KvJ#i<=>-m!YXjo>Dk%Q zfLkXA=$`KGD4V?o9QLndVWng0nJS={LXj@7c`P7svU-sitNE%-k)Sq#WMOAlv}H>o zdU$ez-0cCH3aY@ia)HBg9~6Iwjq#C*2@D{h|KxyyfdR0c#~fOfpaP+${fobW zUX!m?Ds>k994b59@xu_oJ_!0a9vgCfMmagTmQRh$XWvHZJWd+CaVS6qA|l!yHVV11 z`agkoO#L5LfooJ1+Tn0>T1jA#pO3FiKT9R6vs2~;&caj|NM=kf*Y)y;($mtoO#l3J zxOEMfj9*X?tMW;jQfDRQ-MbnqbXvKCJw5VusjOO*z!Ay7E{TenL7PRb|nr`??}Z~a1( zX8qR0gy%w4ZOhdz?~dm)`t_6PXkd2*)#0LJ9VxYUy3<3dJvBM$2|WzFg1Z>==g*(6 zIG(t3iGtFxXwXO2A>o<(cNYKzgQ)U4?59K8{4-r(?fy=yd`Lfh#D$pX(VdDjy^O89}iUJh`ZUhOV*^gqX z_h^Gg5{T1r3x!QY>?NQ}+i*poCKVaFT)q-1#Ra-0G(V9M3!qyBQSfX)TMarmi+G0J zg_^MGU+j}8$OqKyH)x?2Ria0{#6*i%sO;yqSZiFsG;dPHViD!;hB_O61Zn^gQ>(cQj3qTw1Ws zG)<7^@V3SbAWY8}z3s>;$%o5oKlxByT@4DFu3quLO}oB_74&lP5^{2E3=F0Kv=F|~ zM2-NF4%-Nc%=9pVr8@pZe|F>!0s0fgY-$RPX80Xf$V>_!0O% z5YXX^lH69oT)ih3)BxcPXhF|@v{8*FN--yuPVZ0PwVeT}5x9!9v~)*Tm*cy45bw8c z+*tnez47OVD*>|Eo%s3g>(QzP2Gfw&J&-SeE-;%}TeB1w7iVXOhNoVf+M@(8x&;{> z@iKoXNd)N$+#f`H30Yax>K-r#Ygo_oGfV)b!n26g*vN?4u)@C!%|(BC3^YX=K@NxU z#Ft6ASFe10eSMRX^a0)h+?DG+F-p#)l$4l$d69`qdwNF3wyrK#US1baOIpr{RJyO< z!IA+2g!PwNHXBt{QDMqSzMPhlVg$`QP@gPrCYVbFvh8AFou)t}bbYDCp*h}Pm)^(2 zdbalx2A4lYqV@8Vo*4Pmyn(k=CBB7)^*HF?>t}(%bxuwSv$0u2H|vW}%^l%}K+w_A z4Q42&+%$p-C#`M93KkaDv$t^~$TJ`nfgM2c{=Idn7%2h^>%JEO5K(I6sjaFw%t56t zJ`d|95FNmv)qzZ9)CVlA_jh`|sj%K7`yK8hUq~Ah^K#EfPsuH zNZzW-N>#`Ek-q<(*_e(Ly@e+1?Y|()iiQqU)e#%(wIsl%KIchQ z%l0O?a_yE2Oh2;uNnq5!2P8}Q{B#HUaJfXm$}FQxSRKPazFvag~1qkMz5 z*$>$n`$*D+B>I=QM6uzYt*sPcZk+J!j{jX;QQw*vIi40xqq(p;JUg1GwMdG1GgWA$ zn90ROjtYI)ty@J@RkgxQQ_xPjcH75%l+wRYKl<=5B>-}X5*2=E_{MWDJ48) z8p*^pTR}PK*L(Pj*8b7&s;GK`n8jBV$#7iB$jMm56E0uO6w1TPQ@!)(?8E=W-2B0y zHkU76Zmqd0Bc&h89F?6v=)NVEM9e(SO#hgo=!tPk$>ZET&P>9S^|i6+K6VVo$$V8s zlvs$ z?uxDQ#N1?h!cXxrC*CglTl6f#)PRGuD4b{!~6P^k;A_)xl zx=;sHO9!=^s17O zau%a1%Cxp>+GDe2*^cHkO&rgAc*g|rZ&H=awcnv>8m!1oD#0}Az15#^LI|AfcUKG8 zyqRoeXwO(zs!nt6NT(2FMOvQJE1pl>E`M0epp2;8ZSTV^=fS3l5~&*>YF5YIYA;kT z3MkS(Dv9lN^07S`;>bt3FBs7@RV>UVW}QdadP^@y2S1_=z5LGkUA}sX-ir{QPoJ*2 z@iAT}Ey!{z9mBQabuij}6(qNVEI+f^ei+_CYM$wvEt5NK6Y|d`MEun5O&x*ZiVC{Y zUdWqZ`<`C=zT_UA3`1d*0XTipIZ_UxS=Y0J#i<(T2;^Y^_zw*`gb&v`PR4CbzB zi+&ch`mVBVvXZpyy^QD7Z1R9_<(}Ejr4Lt#vhVB=(kI2cKbZI0QY-H1@vNO+MGI~R zYoez_UvZOHLJSfZA{#u#F57r!q*3KPE%m7v`saTFm6A_z#4Xd%-soHgqm14AC&i$8 zbUn4<`SVW;`myX@!$h@hQl#1v)y$%p;t zTqDbnYZ4=FU)=sodb+XW)m9v}bs_Q2>|f;LI_j)q(>P5`l&)^D(Y&nCKddmGJ?2`X zw;HasKCta-39{hso+uQZCH1tQr(pEMJ-`upvf0#oN&F!~e9hDRAkBomYyI(%+*^7t z4a(B02h|MB*9Clk=*fB$m?s>|?I&z|YpqOY`Pkf3{VZ_dK07yUYuoqsPE+R)U+ng# zcK;$|lejounYcPwDJCx9;<7}-+P=F#*xaJEplQv{URz$?{XkYzN7S>S;7X&6N`d-_ z%~I)ey$6mz)o!5v{Qd6J;6sHA!Cn>N$*7)C&F(n`DW$>PZMVi*%wR7$cZbF+DPXF* zPQZeHh%*N-~BFa0=m$O#Y_j}~G!iMSMpdtEUjIvQNK?^wwH}_}EYS;+y4gcal!}Y1y11aV zRwz}gH(qZOopk=#J6ltEaXFKxRx*auIww!DF$`Dh%cu75)d=nz$yNjLkYIO`rVv9X?vQ zle}3s|2ZI8e!2`)UBfMD?qKMLNaeErMuIW%+Z!}Q zx*N<>%LDxZ4@r|G_-JGeZT^}rYK*1rW3C@Ci=-~ao0@$O)~0l=eETBzBJREa2Q*dJ zzV6U1`1*clcFXKeP4w#s!g0XQX-JStI1g|TBMt$0lU$gO($(cYXwWem8EdPKl+Pxa zWc?`kk=&IVF|gQsHJUW1rmpBtL%6itm#D8v3|^bgd(l2K<*zbOn?Wfz!61Q!^-WZ! z^R76aeh$D*otK9wwp%0yS$#f)nk*eQFJ|wo`{MMReOKGCT;^tiinqX??Y<wkQ{)FPTWI8saUzRZsM4cdJ)7K!ie`neybd0sg#@q+W^UA zYfbNXb>)^?Qo@Pl#JZ#CXAI^bZlqr?o!yh^(z2_Yw^iX-s`tpc38?@}{q7)>a%i3@ zDoC|&dxxfnqMen(oC{J|zVj1W<9O+@&_?<3GXyWhm`e2+U&qIy}+3(UZ z(bo}Xy{4`3%AjQ~JX~Eomqc_pb8SI{drEGvJ}2wrcP2dbQp9Wq-jXPQ#`|sU8s5gv z*c9=_5UKFU6;)+|Vnp)=JM? zRO{*OR>Z!&LG8J4Mn=B=2$ueaf;7KQPY_<%HABLAN5 zS;k+DYEkQ(GoV0Nxj^mNzG@blkQb~aG`kjPh~7mAxaO}_6%`{*UPvl=D*eu-oR5^p z<)?KlVE;oTSc_h!h~e>{I7$5@Ii+`^meSv8CbF3JgPp9MR8Q{^HJ|T4F?aN3(G9iA zbymLR6IT49TdJ|)U=+2ASlD}g@RzVB{xK$xw!GNJd91f9K9(w&ies$%UazUwR+gQY z`OLdWbkI3r4Q77~%r-FWg2~}P{AVPG!}km8H&fgEZimu?_35Qv=5OkmdqR{#xR}0< ziGWEZ0guY%4&c}m^xr~?Ym2}C`8{G{Ia^qddn%gHSH=Tq6Rc(OIgWHFdd*N5xUKg&Pj$6y^PJD(^#!XyN&EG9I zzWrh2Lw>2t`tWXP%{bDs*BmtoCST(MCVt4c1;H1Q^Y9wKBA zG2k#i_i=7%&6LI~x>#Kr9ko!B^d*`gn!?2RB(k{wvh1&knrheB8d#y&ZZ6r9@RV-9 zHb2Hb;-l*0>jT&dyBmWz9)@~`4%WN5IokZSc>rm~lU-f}$nldG`9r=gK8MxE6y-75 z-9g=cWy&L5?`-}qFEQ#n(l5#(3=rSkz9j`Dsn|VPH>{(ZI(>2Q?Bwq$W(T@OyZvp) ztv$yruJp_zA0{aQowru&Hs~bR>xm|RA-djg+ZWld%uTg*wgWATDD|~}7WDo-`K7lZ z9sUB2j_<5iPy_K!g7<_P+#95{vhOgNiTjIRX}~ndmMm{9=_$q0P|?7j?Xba8tvJH; z4*|6SM<&H8Z*QtbI90TdlAlHW%WU|d|3A$A?_a3?J5VrOa9DHryG+>F69esO;wU03 zc{4puME1l!wq$L6&ck5os_gu-=$BeSQ9EVW2+2 zqk1gZ`p4>jmDl%d{vbsIZq%{WMSZ@F!UH!pED96JZAt@;ZkBez2$xnVfn%EiBxLq* zG--WcyMLGoTJ&?%X%jjRIppj-MXMgQw9C(YRUsikPH4hI{9r?%3hg(gj4F&!wzOW` zw&>VY zp7TukOG>xv!9X=%Y`l70trj%=?FL7__u)!|sM!Z|ZU>MCNQP(eLydjkKEFJ7q|>_Q_nMsKl2a7c| znx>rMVy%;sjis?dyOd9kpsSMDZ>MTht6|d^cM~0;)g3tjqqrNs{0y<+&J&-Yx@Zxn z^QuWa2i%Q1TOR0!5*Rwn_U3aVRYPl~j|b5wV!rHN1c>kF3++7Hs*V#YgZYY6JRTgTJ^6S?pSSM0ZOaIAT z_tqVf@m@gIzJ~Bi^067*oGweg=^`wB#0gSDX}xCm)~FV6Nc-4`OmB`;PBgkV+=|S9 zqtnLLp0})1T<=Yd+; z`%e4Zo`0TUhXT0Cgb;~({X?y|^GPm*td6zLD+8bEPj0p3dx6#5O$#k_3$%J2HFUK# zkshmW+9hdT3QRf4Ikn4OXS~YTcGT9r-~D{!IiType_u2s$dX9&hgPqc|98lH@|D{4 zsMDNhRb#ubV!UE(q&(yi<+5hJNg;~GO+(5Gm*n~7^FJ^!C@L=YCZObZC@5Awqt5JK zj$+prHfjk36O5}XOd?_SJ_^&>cAS!`<7Il=;}UFa0`0QT-3iU@ku#-3FINXuDM8_U zwr6~|Zlm(cq8EY3<>Lze3KG+sp7jTAQqQGg`c(Rl0=*O$ z~BF*sjQDi5vMdDjxe6?_3+8SE$NnE2?} zx;Nd`XQ1bGK4$4?JxwMrUaln- zXaUn2Aat`RvIi&hND~OHGRPJPx${OUem(z$d+xw+Pe!nU*KVuwWHGfDS49*xS&nhb zopc*Nx3!;PrzUlahFlv;HxVHgJ+)gTo&9g}qi^%+;OpBC-;qmuL;Cpte%Kip6mtAS zLePC_9dR~wnXPav0L=JqFqxS|o_myP>1u|miC0$6q=a}8fc=DIeJo%|pHwRsEc97; zaFpyD0#rh!zSTF2oD8O42pYP6?+B8NSt z=Mfl>h+iM2aT0|#*FJb%*=|xAJqcr$K1MGm>3m^{tQf9#;YU>QRY0;aRVYbKUr>GoVRG^bCbgum)e{O$_Bd*8mcAgVaEA5T^amF|E z_p>UGn&v5{U5M~7u;d;8UWsAkReb3ftxsa_uGcx_*KD`LxQB+-*W0@k208|fK4AR< z!&ACkLZ5jl4hc(ZYpX_SdljjHBezMggbdO2wBat0j>BE$t(~5kO@l()6qo&xEIbg= zzg6*nefK!!G1$M}{0!HV(^H!M`}dd9(;84h4;N-xOY==WCTdcaYGK>5t?Ps&2u+$E zy{bRPm{$+vxpe#v3tslFi;~V|43W8UfY07Xvi^B&E6L)#ccnCaBVnbg)&gO0{E&;< zm3cS!PwaQeBT4CeHlbJt73{*z?S?|x7Ol+uy{);~fCr>EXzr=ty`aGJNZufX0th?^ zG}t5?%)M}maGfuwzn1Ym*UnN1W*P^nJ%^7QA4-b9qNZZ?I!taP-Bz68M08~|rdXRy zk%&Sa4KnA<66+#?U2WZMrM5Zysu|(>V7{sK#l!Nw5##(K9?Q~|<1F|E1FQ2s`t;=l zJ-HiKU97F+!Nf!;S!i*0*VCQz(Csbzz3f>n`pWx!Y4>R>?^gy}PzF#(6_s5Jxc`A2 z5Pjl2<~?H3&rsw<1ToXQBI)JZW~jD&W%_eD_ zD3C_o@~Z0Njct36!#8fsNor7R_E*Y9KLcV=IlAR+)M+8itG*b1LIp3P<1h7$LDq@WK`Se$pr|aJ{%%UdMVIdY@g<{5xqf?@t?p zs8iQGpu0voAC}W>ZA^VTXYSG?b9Nt3%0%i(RBvK}(74EoH`&Fc`kE|W1js-p0v2 zh<9?k&%yigPhO;{;Rsa6XDSC!d6dLPTdWSWX%##}D^Mbv=;ml8ntz!*<8z}iP{>!PCiU0I-Y8)`0i%PN^Gj=L>`l>5u(q(t?NVGEN4}dHY94wiK;})R zuX`qi-P>6`QWLyG^r;->lv$hFAEsB0n77!Tzj0cwULSd^b_~n^LVA< zn#S$B8fLdIVZDF)E-=smUZ=&jHrVy$vHF-@ZHm!8ebSNzs%V%dgMZ8jT)$PgH^`Ik zUba&BbD#0U)V*p~^VLj}8aFv-zf>J+(efV9IZhE7^^=EAraV>cil8_0)Aj z{DlmAFM*hgmI0AsYo&?29-Zn538a zst~%k0>+b*;K<*h9Dyo49#t9Y@|F93^i9P*{y`MYzo`R~Jd4clL$Jo*ymspk9Cc}L z>#eRJDz(zabd-MhP_+_Uds2BQYr`?|@~Rk>^^ypCkO=!wshm3K^pMj41wE;W%YVmy z6@4!QFlbLlL8nfhu%OhE7S2d7KPlk);K*y&x1lR1&L|`>q1U)2yGx%dor|5G-C5jO z?EAFY=Bvb@KpsZ3PA7{FhR%u@kDX@?A6TDgcaG1EW9<7rrX(0ihDh9{>#UA(mt$$`XM`G+9gGPh&@4QF$F$QOpqawBXBUVEOk`YutoTQT>7hTjiG7l(;_xfu7#8=VJ%X#=<0O*%78S8(QhvoWH8geh6Tz zsOGWpMLTUMG<;vx97?fIp0%4z-MIEP?D)Q_*4O>T5+!EohwJGgF1*f>26IRw&pgiy zrALHL&F=h{huN>OHtoW!8Q*3^?yr)Yx^m!769)ezIAMHmxgK^lMl!4W;88L;3XqDH znD!cixnsUVH2V`Ak>Tg%g>!*$1|ueegqoV2_3%X7F0AoF^l>l$GgfK=hoJbt7lU)n zh~?pn&Ui*I0vr--(&(90PD_R#GPms){oBHum-`!nHF1Fbdl3N7A9<9vq*c3L9x){| zFiusD+1XDL$jC{&RxMD4Z^65^d`(`&ZF3|5g$F{55v?|civ@Enl zseJ!fg0NCiiy&y|?S9gen7{k6=MgC2U*P9W5~@bK7l`O6kl@ z)_YE`zj8|H5a!dRP=r`3XUjs{>{GYF(bL6Q%-Fc=dHt9=L%%Uc{{-X{3Fje*9~l*C zW*H6+AG-_Uw>$L0v`e8e<%C3#g~SHen91a_2ps!jbc+u7{JFwqE14i!I+`uzRS>w| z|G@cMI8#u6y4{f`9j(UM*Zvfq1cNyoGY1D89r*7sYsEcI=t;`kw+rEP3%XfpgzrFs0mJ_yi4WpA0Ka88<}Y)Z63%p+&!xcsZyKmc+_&rCrU{(+*A0ir;-* z$#*elZL$MZWIlvFzm?(r2QI2fKbcnP#~dHZl}E0+{x{KC3>s7G>2Dbt8vXza z3s~rAyrj}%`5nzsFLj6^Y0OLEa9N4DDq_CSNGzJ|0j*ePF^Wn9)+4Hf(Kx*1F@vds z~o#eHnJQdv705eMRu9VA~2FRh0bc8#8R=i1y=LsV-DLkc$~< z_}5r_kr4XJvVb(yX~{HxKCPpoZB!nmvN)f@exZ$4e(E86#b!D)@)A2#Gs28_=q5JFqmB)201&wYu98T0a zfg{~GavSUZ5&Pzk7U+o*K-bf1oJPJPU%UvN;70GfFv}fbM#7!it1_20|D0S zW}GKLGXw1p0ddNktvQM*Nqv3gX+aJmqusDOo`2!z+i95v%6hK5)(skHL!DWUzS??! zUi$2D<0Q{S4?ma8ywwQz${~AAjY2Ge9vh>`@>up%EZH+n2(P2Q-e?Cz0>R8Y9Xr%Z zGBwT30*xB3ubG0nOK`>^%^LuN>?SxnW8o;d*Agl_`uk%R78$Jyn9aPE=Rs}s5M6Dk7^xPb$$MLSb2&SayVcr8Vv6JVdJT6U^tmj=W+rGVf%4_;5viQiJi|DF2jZBVbQ&*zJUH zKV3*dz!5z_D3Iimk8-l+*$J@(60*b}!m$p!>)|#O8#d7^z&s^zcG$c0Cp=1hXB$rfVD{~sK~18o z_vS965TD2h7&8hh1?OR8-C4%rOLbQ(m6_blWi1}h{*_nY7oS1()d8TK39e2*2pXCv z-^$EB@yUf*+nQ?r(66RXww1P+{r!_4{XFX)W7%F(=|c$;d|QKPoB0g;H5DhNSSvDAJ?EdJRHk13^y*>P=G-i5zZLOyEoQDRu0Y1Ih+CXOb z_dg>en&izBkH1&RZth+DMAqM?d5J(&!N%1*5)`ZfzcYLNY0)}fu|8h9yhu%e=UELe zvaU1EW|C!l$2%^rc^1C(rU2Jxdew)U)A2<2DX&XG#7~%v+M<669Zd`rWv!~gL8y8` z=f9OengnGhbKnSlLfrhu(#G;q=3r+3&HlNzx!JRj9yD+L_z6=vXZ}{+O%t zV&;G{rMm!h{V$DQPPSL2ksjYa_*U)e+$D*){vV!(lcT1J+5q^f1gw{VP8OFJJ6p=F z?d7aOkpZjJE(pCt6S|_Gd639rd5mzlD7`B$;_eS^?R%AnuLf~T<%CkY0!P_OsdwBg zVS-^{o!h&_MIC?k8uV1z+E_4nKo^+iIt$AzEx>NIHljh*7+A*9otALH*W%oqH_v`S z!x$t;G0|Pe>d0ogHo`m3?!KN|I?pQwo1$Ch7#f|iw^DN5%;(aMHvzn-^-|?8)Grjm zH_bgaauy?+CwQZVbG=e79lpz9Z^!MB*6wiAfCikU5^#p=7T4^|3@3R|!S1QKYQyi} z;))mNM40?i%^g=B)nU}$m_*y}yb5ZayaeV~^*nxOn^QeASBZkQN_QY4f?BJ+&klZ8 zjPvH|RdYG5W_Ri3wob;mafxj2h8{&Ps?Xv(Q(U0tIcC3;)jf@5zMUs+tSu)aUp1J&0dEe@-IylUP|sny|Eph1H(S z%FZ)9O4%$%+x)HBd>HCY93Mv%x2$YRR7^~4EXB8v$sa53-%l>Gx!L-m5gjsy_QSel zgGa?zlZe#SS*${zGTJ3`M$Z?)7^nn z6EXsVZ=7Wp$L+V@+-78BI|GO1d`S3;*c*^KrCzk1KEAZD@YE`{FaYXZg^@;__QAm# z`-92-NnAYPP-Jqn8~g38!hbcdK^!4{eN*vV8;v6oE~Egoi5X*dYhVYj#F z>9xj@w2`CvBe^I!%6t5W!zcgoUVZ8Ea)4~Yt$cZ+J5Tq$^0U3}$K{XhgDS&SDNk<2 zSae2?Da}@vG<1#19+m8OuB3lVMEqI&b)I&L*6FQ`n2E3#TXayZa;w>Jtvy&>UmF#3 zd8wkNiWqTK6-o8g##d(^a&lVFa=ylY^=$huyEdC8U#uHv8gH|V(ZcFqfpIsN^D>?v zBWH_s=Q#q^Y-ohubzeL@61)e7Z;eY0mVA(jXN#F1mFKIr%gU_wJ}5toU3`(0MDTbz zI#3_agu0utqp-Z-z%J$q zsK>1fDqCt&Jbs@gVg&d(o$9T$akS@upxQ>-M;cpFLd25{CL4FPr%=YjW63j?rVoO) z9~`h|W@KRF;lUY){mspb@7#dHEbs*Y<8Y2PpcQIO>XZd5E2|Y)@Zk%21_uY>==>j% zY;k+w#~}d$Zg7Mq(gT?OGv?^LyfIRo4t%WaT8oGM};s*=MI` zo8LcT(#&_x&U2~k-MjXxLT=-lX5aGyv0IbZg}m+aC;trm|ck03JU0|%DMxe`6g(5L|Mj9uD$E=kHieFKHTAMzHPZ5;R+7Ocjpx!=j@=WRAlSthsE#t7m)&QwIQ z7ovf~;O7UadMA`A@SsGYBK7=+jT^U5#9y}0T3Sl|TC#_z5t0>>U6Fskm3|)+56d0~ zlM!L!9ST{a95$e5UV;BpwJ=_4~ZjddVcZi&R$h|*`==zXn#m^VZ@2oGEQ1pQIJwCp4Tut*7eR$7g>F|#Jqg*L&}GbkL2fUPE4$_4#rq}C0c#^$#&evY4BU&kxGHwt4<~$ zLA>1IyL4d5yd{0IVv6B(e%gn$&u$xa-!9kGe7C$kQAc-!e}sD^_*$6f1vO(eYqd1b zY0L5Y&+k3XRy}UsN!N)6d;^2tN}3NcT>F?@4;M1rwkX}!{7Q)ThhekHa#vE=*2*I3)) zh9$Y?4mp0g<06SeV!i&o*#_Cvs?;nW>WwAjm+WOqo?AR}9yT6snyz4@VU-s-aamG) zV3nLa|F3{xVLt@A&`(UHK0s5RK7A_nwW^8-N)OOlfn*1a#|cTFThaEruyn8YOUM*- zFs)oJz9+(OOY@5M$px85&uJ5M{_)wx!4P!3P(=Ndq{JTWei`adPl`@I3OJx`d2H^r z=dq`{wz`Xp_d`5_qxbBWGwQEuVQ$q4OO=fQuk~cO>3l<=o3TwZ{{y49^D~dlC{0t- zlLTdbvQ&qRD*XKY(X1wV@Zfc%Zc0i@vW~RS3h>2sIEH4SqelnO1I&H=_({d+TMpX@ z@+Igq41OhCVmeBORYz#qn5#ux7rBy8qz;TXtqw2dn_D#KNTvyoW=xx97VO;rKw98^ z!u!M-*U|p1uI75R(vnrwAzCyA-y3b-CtulOAFqb~umLn7(d^rIR17Lhh|~v(PV|1S z?fcOioN83(z?2fYulR8Kr#M$|5g&q)j| zM%5JX8iXawT4q=>^-IVbgJDY_yK}$X)qsT6PjkBTKRwHuQ%zFKQ=Vy4(M$F2TYx^2 z1>gymn-&&?@K}JZYg5JW+#DCK@zsQDe<5~402oNgnO*0*h@Q9Cjz4Rr$G8iYSdx=# zlEX+x%$NE2QdGUMR1Z~$Ok8C=^NiDU(Z7^ZJ&!0Pa+?=b;X0LKhJM#b8SDG$*PZT+ znH{Q<6QML@%2l>3F8NZ@@HsP4rlV}uXXcpW)i~~zL!*y+KJ=u;8q?5I=*o0l@9&gZ zW=j9jDqwe@S|q=J@@?(r$f=4c#$yMXp*GahXzVwB)F?J+c$;TgY;xk;a*6uVudg@TGKLpiZ=k&9YEcPM@1>*FC-* zajTF=EM=#r&9RR z;RvMxb4q4f$D0fXfOf}2WPPg?7xW0F#o*OmoYN8F(z4D6nhnAC} z*=okJ#HYnDX`QPf+sXF|g`584!$>EBN_vT#Nc2^e*hD9{gaNb|Td`Q?O@4Zk`rh^S z#~)CnQV2X<0O+B4P&*M4KVBGDeAtaHnpK$JOXHo&b+_u3 zwzf}P!7l@@_~kk+wH>c^)iCNcqNuL&40>O0EM_JuW4)iZ&MC^WZl-+rxOBn%=WmoN zww)Hm&A+RvLSz=bo=-ee{5|&|mex?Z`ifd-;*CE)ix;NH%*Giz_q`KcNQjD2_fNWQ zIQ?C^T^Wms0QIMc%y4Z^-UFA{Ca?Ka4s!C}{+;l+fippHa)`6FzK@5CpNr~ZnZnVA z<$U`lZWj^CdC>{!d4`u8j&MjkpJiCazgzrU&~>vH(O^_)P~>l*oNDxx;`7&+56eI| z!D-f%>LQ{JfvD#NLZ5p$$XCj!^cvuauCoR!pXAup`zPnOQisgd1GEQb%9Z7?@cZHZrB|PLrut9ae@nr>>5}4x39JLMk9pi4 zxPk0PeY3XX@oqsb(LFVj^Svy;4M#H^JBM;w+T*8n>vdcNXX{tru8yZaG2v2(ThcE) zvJe|5t8No9{lrt!UOJcgkE_fhhH))~!-+2obnNxtC&*^bes4xdAEWKuTrWTsNF}Ui zm>Ec8NW-*sr8J_XX){0+pl26j4LRnpLwfsGu118(D-0v=nEjGBxskWY^(%6@Ibk#B6 zGcD6(^~NgYqQfkI?Q1wKU7i&_en&&)8_LVHW-Ju^PqFC=eW7(jY5h;Dw&t`pKI?py zAF{356aR<>%sJbPaZ%CLng7@u@;1uV|D$t5)Y;4K)z#IsR9@1x@G}cCwrEsG+jwMc zZ*OmO^lt9Z#6)}h3$_{Ik#bvSJvNbb%Bv{dNHq!gm|*6C{_=zlJ6-?yQAV+qjEbY* z_NQ8165IG)u&+aUrHdn$PfFu(Q0Wp=t(~pqNL^d};-B2^tLaxWbz4_F#}7Y@SJ8@S zd#zcTx{N@gf-OKVMehS&nB>=rU&AeB7L44CvEB}1voqx}vQ}Py7d{WqsBDq^+T7}B zJJ-Y{#6 z@UwX>CM{lCEbYkCZHTm^dOY!LUZpV9&nVYuM|Iuq36@F4M~}oA<^x@0a^Yj)Qkq-R z-FhduSSG8I?Sv>ht}RBfgG~~+Xs6rE&4R>OYrJE~ygQ*_JEfg-W5egUu2IUX1Drzz z=6fBQ9cA-9Jy+OljVCizJZ@C~Du=m;HXw_#s;UhbQfO3nl!kVpd}QroPY8E?TDZQ! zamIJFg-z%cLg7Nj(y_&HZNpY;uF2o0t)8sT4~7V>%yPvajbHg>VOGBJMqXI9Of_vN zt!lp^xSQ_U^~ykGgbqQp|YM)f#{I6_>jmK6m)E%&gHEtM(#Kjq!MV=qHtDKqeEiN>#+jG=Fgqg9!0Qi^b%|Z8gP+D6y=EGv7rmJJ z#J*QH$2)$gReAM1);Ff6yhiwZ8|dW?1A~3PE*sl7JIZf+t=L`rc1+BlOm~smg^IV~ zb@jMl!R*s97!#Mfvrq9+;2rCw++1*&jdh7T*XFV)@JFS$R-?^mu-0j&;}#tI@`fFi zk`9$;SL}^~=11$UH>Rdr-rkVA@)?>CEyI)(5F$yuyG`@`!~_GmYu(Yc$Y^C?A#d*0 zrPuE`coMq#8G>sbF?}j3?Qo@`e`-5BIz)GUwp8k)U+MT{>EEd*GS18zmev>juZ-rW ziQp2(&WyVLPG9~#%VaLLq8qS!HP33*e_6IWe|E0k!B1MCYpg4hLOx))yxXeV|58hR zdhFz&NNM}sBy0PuqN4iM7w>h)@;^ED$j!raWPkDWbR~GCWwoKeJZeCk+a%5 z|0IS%VLranRrf;yM~A%GLoqEbq+K0O<9St8RR|S9rrD1))SF3c=m2u*$5(Y6q$J;j zgrJYSF`VzGZ}M|wrgm|%3f1@RI?S@WEhI?McD#=VR}81?g~hXyxYuTJEk}*xKgmB2 zsiF-{jEb56^Tpe3_d+W9a9U>4s-+@h?EOK>fV

    Ik|>%raGvNpXzwI)R(u)7A(E? z90&+nT>Mtx<~lSbtl*m5^P#5fSmhP^jV-$mjEOlrmCW9syx+fD^klWA&c8;Ip+YWS z#J(V%;;IO5yOf*K-z5wNecw3*LF9!%POxmt`I^t5QxZ z(7rbrxvdsiQ?BgdG$R(f>ND-Fx;+-wA3htQimm?0KU7W>i=RigNH8v0>v{*ZTxBd5 zpPqZ^Ch@}IU5WV}+%z-O<{rOZfh8yU3tJb1^M?k92Vw>MBKpjwi(PVJ@?w^vbnD`U z0Mytq%+Uwe(5cDZeNr=#G8NNq(P+_M^*2FF*xxC&I!Mq2^?Pu9kjmJPrZ*6(tGgM6 zpInu8EGOgrA!JWKmjZYcAPxeG8>DhSDUuiw5prz1TO21g& z@FPxSU(c&v9nNf47BjOJ&Wdraor|NbN*y+Gk$i<(8X9KpxekyU1H&Q9c2pC3WLw2?Jx7DbzPcW%0$?rut$Qf(Y6z@wj^uvL8Xhc$}iT&z}rF@9K1Pv8Mi_ zhKFKITcB`smc=a1EF5B!K~r>b^hpM*yvc=L-wLnO!NL+c^Ge27LYZ%1)8!Y5O()lp zG&M7WxX=t#gm^$sK$RYXg5~$8=)KsXx5vk4iM42NZ-2-|&8<$P4yyxUB|N~PIx`wA zZ&V<W&nw)wEH4HMu-|*ejaB!pgltf}AI7WpU*O5wLn8-nXEUqy*Z?Sy^*S zfYR1p(SdeZvio#>R_;(GrzhcFyaft6INHA0`#L~Mzq(gU*1hA6t)5&4o6D=Z z>XDyl_g@vQ>nhp!E%kvBVys+7|K-q_7|w|I$09y@ zPCvh5=;0PLAMNTgIg}!s%0BY;gw1N5TgXa)D_i#8NBt5@6-9?LypA3_9_QjIzAzM~ zX{(V@v&|+}Pl0QBawPLIe#y8>L1LR*#B}*5t6|BnHQy)mZEcPpPgJz)*pkU`jKyEo zA3w9nYLp9LU9ga47yQdh#Wyqbg)pJJb5|t_jD!zCK_Dc=Yvs_dIiX z6WRiQ#OPnE@T_`9LuFz$yw#a0%1ECfSZHJOZpCH>6`}U$zi&scY{?3UZ{Yu2?Ri1@ zM$OBd(6~RA_7|nTrERR0<&y37X6WXiLMkgRGNUm%l_-)UZ`|5@H=t(Qot=vG3;iwi zPaRG#^r~!*^kWVl|7*1ikr_YF$pl9QL*qWbJQY6);Y+)NM$B|Ox-Z(yjSliCaGCU3 zTxd%YHny2L4>;W5PsMJ-U%3)neB8uVi=${8-fD2M7}BRwFRQ&acBOoHdC8X3+HHQZ zm~mb^^A?ge0t8;|^q>UDn-ATtEnDXXYmb~fX@}i|Jaf_$lg|S4e;?=M=o`vPKRm$? zfc14Qj%sYUtmMNCW|T10S36}~nA@BT5E&_7{ ztI8{pqGGlw)-7P91DN`?JJLcD@z5!|T z373UVfo2P94-XmZ1H9uCqVe*<8bJ;xhTo-ZXf^v$l@LWzQ`*clOs`H5eakx4JR@;OR};$7LwZq#u#WX~{}t&ebWX zwLh6@e*9tHeEZ|MgsVEbcb$u$baTZ}!Upi9xb)O5t4Qf>hQsWUb^ zsW2!JQYMl-HX^+2O#UD&qWjb5;c2&#s7Fs0*;y2}N`zeV%Ux-8q-wG==*Z_s^AH zQL8arO+%E#IP@RWZtvK!LlWdf)_5H(AH0hk#3DVfsHq8!kKYqvN`+?48Lqo5<6i^6 z@U#3zrEQS0snh<5+>u`GTiJQ312ZaffP!g?xnow8qD%(fKAK&EBdrhf%PtXZWSbAm zky!Y%!fZ0`ClpcP$3163SL5m#5g3t=I?(gojI^_w_IG+dlSR4@b?agUkPmGV-mE6! z2B@J!+I3Oqs}rTQ{lOHMqRcAAYw_A{`)7-u{h&j@F(87JtaR(M)zPkwY5rA)@Zb4Z zjkNKabdO&TGq)$M^khGB@}BW%P#6}D-(K+RvW@k?ylI2coOo`gt>e|GdE4fubSG52 zY|I_)WLz4!L!=9ArNsF7`SWsvrmwzOU70I@+@GXMm3ztmtOFIS=0A@(^{h8%Tz^E} zVc<|xN-&e36@Q_iU0VI(?5zz&JVU)IaGgb+lyn_VSs$+@dX1_ z12}=D;lEXCRr$N~PcFJ3gOKBt9a>E>pULNtLi`e5Z`{(7*|1s;*t;z2)eWN?S6BJw z9S!^qWUQXIkEj@#7*R4%E;HWx0bqtjIgVQ%|JUFTbaL2iTw9NCDgJG|D3HeKq zM!>TYocw3+J6DG65{jKWoqK$Qg?65a{UCZz!t!e)G1aD37}f|0<4MaTc!&5Z@>!Gu z+`Q7HYdk;xv5`$z?H%gU8}HT|w~`aIZ?+c_e|TD5*xG7XtzTy4`3nB2cRHs%tp4rV zjgP#s{dRWgwr1Z$kmxOnEXZw)r4dMqc=KCrJlHc!Gw0Xa*p8p?^{|j$5|6#IE4h~$ zqB@A*flFYY`iHaXVKY5-=jiBYe`Rpw>RqT~L8Xs!#|}rx^Du~JKu!xRm!ymg-0n-~ zBISSgu?GgCl$B;-F>;$IBfX0-zT$3o=MFKmU|J#{YCOd2ckPOA-(IJnQHs`PQYMY3 zW?rgfqDBP{Qa%KU<8$miV?D+-RVl^Gl_C?>qEp}bt3~xDl&?>&!Tx0LG+)vr6JnoW zcZGfUure7?u#-{vN{58QWDjsesL(^CuKs88d$xl=ZYL4B7TMSV;D~vxqXtxiW53!O z)5*L(h!>872nH@7G*VfV^?d$*`SJy9TT`f?p>nwl))Sm&v$yDXhFplD={**fB*hlH*Oqafxel? zmR+(JFB;$UKe3D6YkGv^%dxeZVwMf(#w9?oOQygaLF7?*UG!H2#a2w?{H#Y~h{Vnv+ zJ(nhmC{gIWr5t#II!D+iuo6eM4P z5HJgPl+gJaW~Rdg4RdoI76h%FJez96;`HnWeXr%4j$lL-wpdDOZ>O25sbn}JGf3I4 z_`p5+`}-3wzjAHc$453J#M6j(6+d|Jj4>p=->vu%g`FZu9~M8j?)zG5t*Xe0FNVr( za%$>cA5LpS#GMK>9#KA52+sA+?l%00FO40f$QRNIR9ESUMS)OSS z4&h;AV-pe4f}UJ<_U28Sj%rns+pw&kircr{uf7fp#D5;h4A?j%<4$b5yHSh`lKAaker;IPqIN2i?_Q~_1ctgI}#hljTlqz;Qc#SP!RwWe*b*1c~KoX#+4U2;!yCRYvVS1A5jh$~rnLFe}8|6-e8G z;3{~h{?_07_rEr2@aXt)UHw{7(O3_dyar;$D4j!C@T$7{tr1N!vLQrc82UpT^b|@wcv9+dRN znYa}6;DHqM0F{({l0mojaKL;Y%t~3M&w0{vxp%L9#LogZHy0xS2U4tu4t33`XsM$Z zYT8rg0|o|$MEM44q*`b^dc`Js&2{neO}*`?iwcNfaL?7NSM{!vlWoy6FfbU|^X$y! zC_qy*RQ&w>fc(GU{z(#~O_v@hd}5LVG;1M~_Okq6j^Bc7SLT)|uGzP57bUHQwPzZl z{K!IIzgBk%_wew@%*^yWfMF>ds$W>C*38DcKcGh7QgL_{J)#^I$;-=wQLwbK0^vB8 zlIQqwTB`Q!b7mnTPg7F?Y^%LmKXD>&gUBTAbe@5-Bb5X2kS9-{TD&Rye(cmKbxDk2 zU|23g)DahVjOx$a+{rV^C%9od2M)39p*13zpU&sN+(bk~6yBUtlEnU=JAb|dy?+sI zQ0HbzfqpS$D4$%w5G$w|`g`NAyOen@?Ia>Tm@c0y2|HJ-M8@am=RGetIzo1_#lNcm zty9&PFDw!gZcspnEycCQ)!*X-`YWb3?T@+ndJhp~6BnVJY`nLZnvqe=c0`?lfx*QH z;-ADImo%JQ8{rVo`M-aObocV{ee91()CsR}LuUI2nVECY2#u{B#QMyH5cgvDZbAVb z($2%9qXhXH(&a`*M(6!@?5)dAJtYiW$Tx(WiK!%E3-QTg*M~Odet~o7_Wo>wQ-I>Q z>2KqovabU#R=-WL2!*v-o?{?cn=Fs$H|Hq#?^yrIy!5*#R&fro-LL{PVivi79!iJN zi8)$Z(LDThfBjqjyyIT|?HiR3ME70=1e|meyJbUixS)&uk1pu-6kwpZ?VNK@!QuG?SmOL$=5MQ^4Qw}Jy=oJEiR`zb1dq7R5>pl3GZf-x45OKH+K&lre(~c@Uunh30{i9EbtgV zO&`3cCb{4<9`p>Iu@t*^GYQ_l0wK_yI$l)BH)3L9q<5=HtEsB0f;cmN5)WsC_%nBs zPMx}NJEo`SLa@+(8+qaOh9!6m5c7>Dl%H_{3&I)5w7ziZ(z6G}%;udCf8DZ+UcW3w zbE^Sk&c>}h14?v4-wB(90Dv*NOKQZu#a&~X_0euMXAimYQqndwM({jxubh2sUb$`Xwuh-84>47ccv_Ub|~| z(pHf8Id}RG6*dr!3GmgJ@$nLYBOtT}w}rt-NcwVu2r_$h5*)8p-aXMHG_(V&`U)e5AcKzxc4nl>s<+%9 zlPX5cc#5DHjh}b*ZYKuQ;N8Gd{VaD|t*gs&;@y9Isdu~M&gq`CvYH|^^hNDY96cHp z9j(gDe>b_k?g_Tf!!1oM6{Cpluv_n)S|YM z=aXZG98UFL(CG0-FN1?caMHlffnQmA#`aV>gvKi(zRS7aA#}}W*;B^;_G=a* z=Q(073($7BT=<9*o>GCC!NW>QaNa&XYak@7%?VsvgaU3$-OQJX+pQo+IYESXA5pf4 z%y{p1XAB^GrtK5mMRFTibQwN529UDHqa@*r0Lejg8p8BoB#Q%>qrvucIGF5j$>Z;( zJa79u4e@noY)gk`yK^k;ne%u38VBqf}D2&g1eR zEE;XiPR%~PsrC#h+7TACaIdbYSj72&&&DD)#L?dWc5p31IulZtcrG+B@Gm+QAP#I~ zXxMIg30yW~<7m$b8ZmkfR>_eO_0lzw(A&y%ow$%9rj1F^K9|-m`f?L;-$OkA{*u-x z^@n`Tk)oIBSOMeD>@a`fnazu(zK_SJ!i9iMK&PL;d|nSim{KG>lu$&O~5_o(!pQ zxH(v+hsWqXJ}2F}^#8A51#P0w27b`0;Y_`)HJ9a$NhQunuHC}vWs4G+P z^7E@O6>#}+OyhDVid(|i0@fB26N9O^Xz)QWIqAj!<2aTo?(GbD^F~rKvX+{LW=O>U z|1GNckbZ+kjQF6sckG3BLvn(67`dISdrX}a8TaLD%DM@=1_NEJ^> zQ2ofrZsW~SU%O>&%)yeFH>-K~?jNtc2!^+AeOiWoo-pP>3?uaSXJc7eS=n-wCVtER zd|60H2t=)qvLH2i|NcFq+x!{{sF71?*z_uqZniS!~M;HtItG#Y6UMNNm?yLZo`U^%0v zs=E4W#&bj$i$Gz6bdG8=82%)3!80~AG2YV{BQx7#VI!<5BL60Q>qcd zF$@#LP?Sz=ny2#5PozBRo$w5~&JBd{U0q6fM*ky{ASVSlS0XKUi2Z0icfPnpy6Ua} zmvd{CT+}SxKZ@U@r3#S`fP=wkxZvQ}`rB=7ZRgLQcbXZBFcsVyrDx31_UT(!k>$wi zo%H%7^IO;$w6Bwklk-=C_F`YK1w_r0OJopWA%=CvV1T%|xFE~zX@;=1^GW~1)^1lB z(7SmPdhfy@|AHzc2{~ZHsy9t&VB5HLx4?VRwA<^p))5)f5jM6qjAVJRI%i7^SioR0 z_-wp!)mt$JTG~tm0PxLqb#;j4BxPjmMw{#J2!^GT?1KB-ax&-)S91+wD$x>#1dYMT zd67fwm=KM(9#F*GOc={m%pnj;S56bZapR`mVWX-GH{K`F9>N~DtI1dHTj=T0XM3?Y zKX2OhDcu&-MVy?~=~@MhuXx_k-d=ML>V8` zNcmb@J7rh|KQM(svGu1mWwDX$1lfZiGPl7Rwg_G*sV~S3ZM}w}hs6)bNV|0uH}d}d z4G~=!Q@VO#nGCj#ns&oFR)T9Wqm@(o^=q93pRll|F>>DMtbhKYVTTk*-?M#iGHZF- z2~I(B*RP&bvU>1{$cRodsA*{YZFd@f?3tK2E;iR1mLQ&HsiwAvhFus?iXkZmFO0%R zBS>&Uk((t2_KHL0;Kqp9j{LYK5bC-Rjn)SW@^$0=4^{nQVWLK=bl6X4 zga*3E!R9*LBrstX-g#lY9Ga|Gim^#ALqm_Vz;hr4kU%8k#SiAV&A^ZMkL$w(*Y_@p zVge>h>yx#ys;d8_9^y4jL>;2b1&-6ai~$wchRi3iVdLf-auwYcFtb4=?*p!)0r?IjF{aMbdB}k`bmioE+=!F?D@8p+4%&#&EKEp zqR9owfQ`)n{+e~S+-un5}XCu!@Rso z#Sh$v;^Nr^u5fbo&fi_X>zN_A$A2dcsD9oOfQg>*-nnB3-^r7-KB!RO%HCGg@%Rr5 zCMz^Ue%sR6=y?Iz1h#GYaB4~l*vC?mVLzj?{!!ex{`u+ne0O)Uh`Wd`7G$KQq&Rj- zNJtRxw!Xi_6Oeo1>O3xs(@32b`BqP)b_6vrIVFXjPv3KB54qT{WWt^NxQ&1Uh4}KS zJ7|fW1LGPU<-h>EcyX6OO}Jv~9*SX5VPSf;CYw!7Sy}%jC3tDlG()?!Fn0@AcsGDP zTz;g8=rKNY?AZ5)hKA;5VL?Gb4vs-INtBmiHW;Sa7Twt#mQFz#FmM0Z%YS9rYdyY@ z{{IWHE%At6TbVhor7k-4WJBZm6 zt^R(Z#~?H~wq`#Qa>uT&6kG1KSmewwFli#BN3_lb{PTM!rrza&tsj%P`AX^DHC!hC zMLEYbCbbdsJr383Tvr!!^nCDum0i|2dL342k*eG~k3kK98MKVi!%Iw~0L6gb>*HGh zW#{UyY^^K)rS{3zfA5i=`MZM+)$1vx*?aavc!_;9G*8Nqu&5>KXuDZ&4LWb1_2$%} z^^`g% zXAc^Z2%0@sAFgn(2d2 z0OdRaHVcq4)px2&sa~j|)&XvXuEqp>TybeBC4)VQmOr5bkQAfVu(f;xzwhl9*>WQA56Nvc>8)*<%K;>r?YtbI~9B)<1Ep%D~V2 zq$DbhX_I~0^*qPN+1YPIIf#U94U-MhD2n~xx|S?Dzz1I?Pdg|tO!JZw>>qDn@iCUx z8TVW2E3m-r6AN!ktnS=7SQvKt-$!ycKl~Sjy*?$lSeR)E9TGrB6>IbEKP`^sAjtR9 zR*I!z;u&t3BBf`w4R_oTbS*$lK=SLC-zA-a{6^v!Wu9qGcjU+s>SDTL#9 za6KG;vB|Q-P$xi1a<9+VuU{kIIt~HPTKJ3!?I?!yJc3@+eb=*liwNcy9P8tU8*@R# zjEyy!VM2f-BU5U~_3gEgY2yT7+@8L^AtwS1LU?D|n#BtU57`>p-Ret+tv?a0B3uWe zK^?P8?zRj+m>zKetUugP+}ymrTd7L?5Ast|l#O2p2Q#GYKiraLTnhl`_MAoAr1dx`?%FEOE zU_2%!-x(vm#|$TwiaG!}!KiDiso7#q4wBJhy7|`%Xxu%sa&m~N^{^ByleiId@e995 zJzwtx75N{yBIW|o1(^5GpFdG$*^>C}C=g#ZmW+%HAcYC-1gQ_=6TQ5PfCWto1UoxB zPavSfRS7w5#m>c5;r_G+$vAFxl*kW7!L)7b2mIdu=$}LMtB-~ofd1(zDSY-CnZy)8 zsc>A#^oD+l)q<wPiEhioNNgh)Cg_Jp1Cw;}zJ9+X|K>WW zcKGp9ulNMMTSiinJuE+A1`ra$5AM&1Ye@_bmy{g3rd@!n7o&M4q8iC|; zfbuV4GW?T`NIw8dW028K_3t5(zw4bvoMhj}rHeKL*^O2mSDmf>a)9<{g!U;&0nv$4S9#80XB_Bv4U zj(l;&>yR_VJ=s`<3E#hePrIqz+JzXCqvO&=HDdPOtl70)~ zHk56u-<;+b5pe=~4%85{pM!CWJDzmnkNkC%IeEbD>f0`Ny|ptbc{yxQ>C7!y5xGZuqp!B3opg%I0e@eK?_wbLQ> z24?ZuqdOCwKyyT99vs|JvOy_x;?Ca%6NEL9p+Y1q?#IsmZ3;XFe{Ioc7+k_ZC^WaI zZKwA*vX(M$D~cr?%`ucuD|g=-_UnmaZ&@pzg$T6UjMkY|4Vc5%bT z-UXYB7fz6Ng?M7FtI6E!*RK&e%gAtfk=(gCHK}Ow`u|5(-1&->eGt!G0o?r-!@fO7 z0I*{0_(EG}EN~Af#SG!#qNy2!FmD@v>^oN&Pb%7;Q01Jvr<-c%B^~w`o!j}i6hw6u zFU1X-o|~if!31gC9F!B;B_wnu!IkbFA9ur*pf^fNO&vywac}I_+XHQjb2cN#xc}R` z=dcj4c7)cCsw#&BIvV5DlRv!uj6^H8jYukiO2%dhD^eagvVGf&R3kQCCCBL#r0C%K zsoRDb0sj6NrdxN~is^(fDMC4M@%Gz7N`&^HtX|Gd;_F~Kk%d#<$ADA5I=A{?Zir82 zhTo;5GoDZ;^XOa~M?m)S>+gvZf*5WnefZCx5B2oxki&ruqL;i=Ml_Z_6wg24Jb@EqoY}9`{d2o~c>w`u{f93*elu5bF zbz!{v^7=d7B>tWBPV5iKC~Mc>>E{2YD7Ho5;&?j=<0}joIM9iKgOWYy;|77XQkU}j z{)B{@=$C$e7nFTTp*Pw2q+53+VH~8w)6%RQwpUGKS^BBar2=_m|3|s8w>3OTifQST; zsl>)nDWcw+zX;T!H#%~CeSd_u zqGA$WiAm20Ed;b~oKeaC^dvMiJiKhMb`Sk}x#gX4-N;7t#fu`sA=Gkm=?jxcT#rJ1 zfTbvU5#bex-aB=6T~}Y<6?oGEx_C7$H8sP+y&EpCZxCM)edpBVq>jr>J%ErFWc}sk zkm;(a5p!(uU13rrrKG57X}Olc zp8!yykzuN#sQ3eKsimW1{!aOxR$m``emZ3O_6<01W^w{CL5(F_Tn+Efl~C{ofl;skzi32%yT`N$Nd?Wl-1BdRhj7 z1XU%a8!s&HX-Sdp1^v1Sclz;9bMDk=4Fza+1KX=P)oaMs8Ek2LM~+@iEDz-WFWorv|K7qzfV1HIoY=N* zEXZM3(P8d+MO8IK%qe5V2l+$opfv%0#r!_!esIc@6FETh>gzvWrd$(gD0H}FYu z)AFS&t5lR9U|ci~k_*b|)nWM%5&f8g^XN7fmZ-wz<-6nV^yZH!0(z%b!Wz1q3i{t{ zJ>pbpRXz7cCGK@N%EiZl{>s;)CFPW5&xMP{PpC0OAh65xjF83-XZ6=LwWEw{Dc3VT zc7UEk3hmHtDfV;+C%T5W0)p#TBBHUd(PNYG2rs*r%6RbxsL@ICz zToE*B7FjF9?7!KwBbOCiBcxSZwr+Jo-=HkULEruT6`8M)S@S+BsvE}Rpryl|te~GJ zduex0x6m4f@Hgc1hT|AjY4-g!FRT^AsXN!NUw_wYbJzD0y9$^|L}N;rjQ4YflU0EB zrIpCLkFJFxlOi5+1N?rRs<+{Od@-dO(JbpX*c#*nTYi$YILzO;*qo^_IB0odC7qhE z9F1O;G0)(8F}61}lzEiVL9xVYIwee*E~6aQJki+BIFEI-E3jUYIs! zVBKtbuVsx`)1$J5VhMQ#D%OrIL|Gac0h^#%vo)dsgu#CM_U?ryrf;R6d%WWM3#4~A zU%S?dD1$Q^4c6$a9)xm>!4i_`7d_+S1xULablqIQd;lxyr2u|>R@SK&gDuZe_W{GS zCt(!MB^T~dQc&cepdF4Q7>?u>Oc>wqhErfruo^tOKre9XML6OTl+c@w@x9o^CKq#w zPCGxYyLA1AF#YSZ!3`QR6(wL&hVVjz^*zq{MMn?|__r`S4jihGE^(Q3Z%cl%wok75 zcedxTLITFZV*J@>M7kP6L7+DA4ggB->0#yI*CL;dW-=#I@y$!ogFkVwM?Fv zfV8{a(iIOQrZzf!E~1nJdc_yK{u{=EeKuf1wu{^&W`GO`@4+-}x9|3vpSQmVO6sMO zG^~pkqllXC)2DKner~pJe_4Vro9d@Dj8`Wukf6jVW5l*a@b*SV(s|GC);B%*^4oORBW8j(Ap!UW=ZZ@;W*VX8mMvS_MI+BW z*g+&xKmtWYMNO%wljwF3j3#&I((Cs!?SXf_#s3_W5O$i4OH8z|-Mzc#K!w%Fx$_1> z75XW5Yb3@j$}IsuI`-l=Yn*YH^;bT3pM_=pri^);?Z|C1eeUfIDy}}-6fW=H)+3;{ zR;Vn~P0{Ks0Cp&=(5!#`4Yu0y}yoKiuDM)G3)g0)Z3bj{LmdF|v!milTM$ zKJ`*~BF$bK`bxu{VsfTYJ&Km zF2e7&7Pzg3d|U<}4=I=bm4s{A&#FR=;?5+yE;5&%mpU<(nnP_Q?Z z!DFk8%j7w;zCErtCd4KV9XjNTKf#hsLb)XbabuqX40hzDz#1IVv^0T>2pdiYUTUfi z3Ugguw!`ud62$YxNFp9;N{tN1iXFR_bt1FqQ0clIqm?hQMU}M;$YjZL#$aDxKJZge zLd^d3Nft1l3N``mmY+vu4i*BCpio7y-tz;eS)y_FBV_%{Ex#b{2DHyeOFM^D)pn@v z+vH+j6tT4lOA=9VPAbZvgvaltK66r{l&9LQViS(rJ4Jr=PBt(pPMS4Ok7jnPphV7! zUB4iE>C*QBFKW&!uTR~*3684S9)ZZ>;^K=twr$I*-2|mo9F+o4|FP@vc?_p5_2p%n zQ6ab`#Is8`+p_rKjU6#52Gsw&8@Pq3@=vHoLr5gY@Amfs>R=jy@8HLdYu1dM}wp9h7uf@ARyyY3ImfwKk)%C!qoxq z*D^nEfADzTbPX6+4v`U?vMzgU(aceM|9;)&i$Lu4k}u zWDbEo)SW<&&?m5}yr z1ysLO?Y3ftnkXR{g@b7pnVaNg*?T1=yehO|ur@~VbGtF|8J)XfK0w>Y`H=X*gQ_Mp z<9P2RwD#cF7}tJ8As<S17MxYUjXgnfCqe(TzedUh=B zRS>-!5Sge8gM{JI&?7)#{$d&>0$`Xa=kh>j9CN%Gu?|4qVHW-(H|O7XT;k8nOuF5s{gNb)HP>6W$I;Dg z6(q$W5M2EQun^!ra6FRVq_6vTl2hqiamOd)YkH4R zyN1&ZMQ8yYvQ2jKy9*&7Ib zX3(n!-`1jFhr;E~1I$Sj>-PCLzj7yFe>}{XHYX+Spzaw@e7#>vI_BDGr^vN;*XkYF z<9N``_{W)>)C6tJUG(t=O8wuYWWeE9Uce86^ZRsJ4+z)xNxKGd5ch4v_cgj4q%l zAQn=KQ&*}%F2V@PArli5c$%PwTtoojExr)NeR$*P@5-HXW9fElWX!C`J{y>snBKm6 z#cm9}SO93gxw*OU$E+OP##5-nf4-lII7o?8Asq$^9@Z;zn8ghhGB-W>0^e-0I7eTM7 z=;0T`>lsBZuABv;Odl30R=@_ryu3dh4{m(~X$e`)T>HBv--&LptUG6wap2AVp(*3` z=c;J9`t#?LCn==x>4kn=AuX5MNHMLhALgGV;Mgey3sv&9rx?gxM8twh>C6Q^~+Dd|3`3h8wn^bME#!354r+cehsBt*+r= ztq+oqQQDMq>=j=1&W#&3fPdN+AxC@lYQwY7 zF{-ihuTOkVVPs~uaIm=s1C2QD?ee=bJYl5tPvy|?gi4HK}v2-3tM2UWz z?}7Y?GS%=$u<8o{=-}0d$E~E zI*}lPV1b_-EMUwB{ndZ}-PBUDVRFN|?G}NXRF056eOjZT9r0q0+v*BJGRMJ)0NgqR z$Y%7ENk>T8SZ>jUd*fJGkQ8ut(C;(Y*?ACHlLHt?0BB+mkb+zcTBm8q-lUEHy{2=| z0#*aVh1_sHiI6K&0R^sPgC&GAI2VK+#2*kahx+;H=R}<`kJVIdZx?n_`}vb(N8bHF zoX78n92IvIt0Ih~2nhp&z^#MdOoa%BB<}d`ZQJbs_6ERcnm@&R@CRNg$p31!*6G(n z-o!mYhXm*p{4;{sWkAtH?Lsy7RX7{~LG~lnyFbLsZ|Y%RUzfwCYMuNr8+vdKp!!2+ z78&(Z143I*NP_AG#X9)3?MMp~n%@inT%gTo8QF(J=7Ak#q#rN(MQF1)2Zu58VE_%< zs7nGC<11l2a6l6{8v-;$aZNpvXLw@OE*C!;{7xq&!*#HbWC!=%Y~q}NE(I{01uz7N zHUnTA!4kKTgn=^+o(}z)!LzQ04$s@FCAf}VLL{_)GI){o@$ifRT$#rUPnx&dw^WXz z^+6G#Fj(Mf(aLNLY@0@o9lap28uvqS?W>YG0X>ueuuoJlWn$FgoAK@N^sQUAG_%vb zFuKFAmXGHi)RG&3L8wKjg1-`=V#bb5J%a1e$vvM9u0=q?)J zXhY5QvFt*KM4~SnDVjcW!yvp6ZH2)?)#N=Ii0a3pBS+Ta>@zodua6^_EaY?~4!W(( zy{%&;zgReSbfPd1S#o>cY!%^V-~EHzH>VaKl$*PcE}DXtl<%LpA%&Mzem*-hOh6~= zhbZX-p89}qDb|n*cNeA3dyC3>pG3uM7Se9sTz|Xwh-JnoNTI=X zUHPV_(`;ft=ibom-%r47BX1~@a^4*@Yx(#?G2Lw4+3sNO{=bNO3#cmBC~Ob| zQ4}x;rIl`JX$<1f-Jx^|(hUM4N;lF-cXwkD0wOI9(j2;**+=g`Yvy17S~IiO%o^6! zb=SRz?|kPw@4MfUoi;6fCrG}k=O@Q{P)qrinFs8X#m}2>DeYz-?v&+f=-S=

    %ImC9Qu5#94~0Vo(q4dvUL!!=FDoHYYEMbN zQOe&YqoDTbU$la|!iD6d3eXYN9EEux5dlY-fVP}&{Y%dHd4F5ve*13qixo00sy2J;aM)tbXa02LYzV z!oNJ#CMxW*6n=F|ok{K+aHWV2<7g{Tv=7*f2xh=ri92gcNjH8K;_dej_Oproe}qSQ)%*NQCt@DZZPL-vLGTM#D$)B62mm*zqc&`z z9H;_}0OF}+9KW*-U=mzDmz^>HJuha+u;HwFaPJ-kY7`8lVvcdp6ToUY{saL6cuTRT>(Bir>sE zmlCwB`bRe1FhGo1r~Xs7&n*~B0w)<)htK~=)VbkvhwyKIFw{rEA7UYmon3$$XmY{l zM!`&`aA-o2*R}8z5Cx;JmpvgZ2G;Yz@7~fEj|2oJR1l$5|4xx{hz5|(R)J6i8nPBa z$pqi1mGi*q)c*_#q0G5ank~2@@F1&+-_5{X9gaATdKfZb{QCYiPu050@d^9WYG&>5 zs;d8D!4m8`RFVbXj4b#8K+O(nEZEF0J6Rxb0h#*^Z0yR&830Y7#v^*yjevaz;r#^m zIsh5O4<69vz|IHY!8OHa{Or9q2DHY9g7o(aADBxqa&c9@_;{D=mQRzd-N17k(;iY^ z-{q~g|4xCY#>LH5=EIi2`M?vFMj##lFbU9+mbNw(peZ?k-$B_201F#x@(tjYrRtN! zL-5p5P9botMnE|~k}Xfc_udp36SxPg)oQpofDJ*`6)*PXwYPViRA!rCjM$SCyKB7UHrL0k=%3oU)@&I zI*sQP0Jxl=pI;9&R>7u%%mDTtXM8s(;QoIFj<@h zo)aQCIQTD$lNBP}H+Khqr^NB<98QIKXScNH+4ujpX7wum;pOF;#U$ts&5MbNY49O0 zvbE`ZjE^hS@{a-F{U7$7awjhp;xo_rdL^DmSOuRpdy2U;JM3N!|< z&&>hm3NHHV>z30mgeT!pg%A*mO8x+#;eede%K^F>p#5P1MexJ;swfy1+l=t4hq@JB zN$xn;clQ|B0^pCaVCIf!5Fy~*KNBM^*Q7YShFUZd_a7-H_X}$0DNuMoXhlgL)BIy- zIY13r>TA-x^kBLJ@`T3RweK*tlr9vJ3h2HCHgnwp#2DWpd_u4|ld1^~D=vVDNI znl65aVTzN;;_EQ4%)Dv$453@0X3#!+(GT268-gBVJx! zY8o1tsNo3B$brGZ;ACI|z!D0TDx11eoUD+snh2Lf6Mpz|vv#HloTNbi2^9;ZR(p^j zI#%x;r*#;Z0jdeJrpyO(D_*vEgN#!gI;4@H81lS0(FH^fo(NzszsUtrmqJ6ul_n@( zHaFiO6f>o^_V*k6%K=i8PUKbYgLD^!X!;MJKrw{TR+MLuxQcH7CF100;d6l!GCBy( z(J6N$(4PY52~sFPli04on^XM*z5kj$2+u$~prNML-~#H)eccA)NZ=a038L0|_Pv+l z6=yAla1<$wy0Y+7cx?=e?qq2Ri~*|a83&vpV{Ynn{QzKP@m$E+QNJ)U*|5E%7()7A zQbi7_AkaXH@aPeMPzQhfSkqq)?UpF%HfTBf6=eksAMituGHA?bb6_RFr*MuM$-i{P zZY>UbzI|2Bn!_3H=GZRSAF`8Wu&Jp*|Li#WJ5DKUCH(K##dktL zn;E1ca9V+wU=mI$V4ZRTLw-avp{9U>%`2!ejmi(`m2-O`MlFc8@a`wW+MvL*2mLVTxn2GT<-0Ct!K$CJv|#f7lYsyQTVN`4O+i&L7=n2 zjs!FX_!5gG>~^1PI75SZF3VJ$=0+s3g4$1?9!6|gZZT|nsscU$ zw1VNtq?fnZiG%w^!}!GBj9;wJ-pcZPOjI1bB|hpwy=16l+!THlkyQB9$AHUxPCSam zvr~-Bjoiu#z$Z@gRa_KWBHUuLLA6W7WueyO(Lf1JdLWTKSO_~AhQk^{DpA1oIdxsP z&wy1iWVQHMU9?Z>ZZs7+PWCo?xojgQb}7YBH}?3?B)SKrez3d3Ae#`8^y;}^;|Y?} zjOMhrHE!xHE#bRcZ8nZQdS5iHOxLwYcv98N)Fl6gusSuc9l#~dUozo*ohy@Ank1M0 zAycM1;qQEp5$g?xO_1+7LE%dY#V`%@C>%zCLuyLDnhpUD?tRa*j9B*h=@ePPunxO6 z(;oh`tmL853QX*w)sdP{s5cPyvfa46F^l@K-T9D(`Y{d`xxDQK8{@3@_&`Z`L+FI-jk5{hXdO{Dc~ur8HqhdO zfRL-!=;jQ)D4??h$bSM(PtA8%w?iMW+s%F8=T}O77tL5R)j;YhB^hn9Bqwl^ucd8u zVK>R|eo~-OE;m+bHXtrDa^{M9Y$<4LI=FiJ`npatzA%>V^ehc}86njGc)kt@bEp*@ zdbaU2h0i%&#{pdk&z96_DuSlMB38#~W`8T$bBLvBDMpvWquw8{KSDNFnyA7wO(M$T z_I4idx zGa3Nd@Ps4+J_j+MgT>}W($f6w>^<<^L9xJ)C_~NzyZb(u#ixS1SN1{O>w;t=BI_Sf zhqYjw6qdeZQlMKjm{(yM5llMRn-<6VTt+5WRklC_M#Suec(!|G$)aMiZ%7zvC{Sb& z=Mcc7j02zv0_O|CEI9IDdt^guk`CA(bdDOb)5?ioy_#(-dp32k`~`h&A`cNUkQ*gM z?fc$r+kn#wi2cEgfQ6rfvF!UlQTiKl0ys#(%QX;2XrmxS7^TA}y|cM_pW8Aa9YQ#S zbHd>3xH7@Pn3#Q=Go!^%t*zm&E7!91M83D0b==+CC;~1e!~ghTI9FK$QR1itS4HFL z688d#vDktGn6`CxGS^ks&eq;EF|35^Z$*0HbeBkWOT9;9cUryBCBOJ{dt2efyM>?1 zC;Qub%E+NOe)Z3v>tCZ7V%krE<5&I{nK!UKz*!nolDuuo(NaG<&e0Xh$fW1j%2P4j zGxPN|ofUmOPK%(DTq$YWT`hCjS3f#lFsn4|#3jg5)#(_F`1@Qz-LbK;!2*kd3>eDC zk2jgwDJUqw?uF{;=;%Qw#8l2(7DxnM-^G0^J$~u}-%y3YM{v4^(zX%gQqF@#MRzFdX1=(SJf;?%ZIkTF zO4dk@;I_|ANZ37ENWr|Vey)C#Y0vDs^RqZ^ zP_f5xtwnEb%~m*-a9Iuc*$XI~U=yND@VFzX4`%bWGA$ zt}S_T_p2QpsufM*kmj2Y+fdTLFUb9d0_A?W!AFmC_49L=i!(++_cMEe-OxvmAkU~t zLS2;h0|1_Z;)vejSK$z@>l?~GO)r^ft6sEV`1}7Dw1AfgdjU}P@|zgh@*H9?4>o4a zYywTP(#gJr6aX_dpZ;DZ1ug?4smmZJglEdeq|Htk$t2Pf5V-FId7LmgrlJ-SIzBpS z^A#f2X9xs5PA1JseWNIP!pFkP%Y;wljBG6pP{$i4JtqHKj560MeCEi!yl2nW%>nGp zQ=(fN&G;T*`{V()+_S?$2cDi*=E@r1r2tqC?1cE0Hm6p)9C#;g7 z344^9gn67PhetE@k5(Misk+osCi0}^D3*CF62q(Bx&nIWgyx5%$&F3&V`!Z_UO-bU zcqs$OBLjZaStU1(daJk7FYQ-N8d5EVU%tfmfu8e2Yw6@dUi*Q(7#%nH3<*OT3e;;K z|Gh058cvg!v-;qFqh3}H?|z~E+|TUW^@`VdIt>Gj4O-f--I(FkUw*g{G$=huqt#&tJ*dZa7oAj?bY>}Ar2y0rypA%q3)!ZMRFozy zmtoOqR6yo)k7ktwwYG~H;MvuGAJaTLHa&5Gn8B(K6ODt3dw-YYYAAQdLEvjmM+vv& zmMMybjsX(K^4Ialdl7YY_NFi|ccsraV-H?JTMTPwn+gMi(>#;M)^0x^{;!08G^0Fn zh+bERLUidjtE(7F7-E6CYtNnSQsx37dx;3@90|Nc9(`?%i_NdcbN5id*&}$>%bS3_ zp}(lHAr#aS3(<%fa&@8SQ1ZE!BlW<3IaN6?GR|z{1H4X{3qAra?E^_Y8eF;=2-TFg44Ol=<)A9Xx=*gF3$H6R94t(XX6y|j;(7uWJLhLs z1!|u!U9otBB7UYf;2~fJq>#IBfBczL!CW9K*}pQl()S%MadB=za+($b&?WfF7F3wO zzC!@NQH=-R(#)R~Gi?pemB>t4)rneve+YWHPp=FX7IZ||&wRN;wzH}(oy3Sp^`%e6DbBF^4?vo@ZisvA{Rg-z1;;SM}+626r-GDEJZjnAOWRsa=J z-xuFp`R9H(pk+)5gst`V89)c|wJyr$LrbgbfDVufM^GQ1+KPynZs`f^1S%*(8VC;) zI@opVHYN*vZ_cmehTi7kGyj2$Amkg`$LWB5F8ol;%TL<<@bmeRY!p-8_BI(MUu$>TFePRsn(puuc;{CjWtExud@IjM4^3d?=63Z%eb(yg%OMGi%Q-a3140+E-^IWXfJ2GhI9Ciuxro`*v%)+J zM`$2Zy4Hbd7~op+N6-2*I}8A-V2ic97-$olZAJ9``s_PTcl+cZ&&e7}#0FcqEUp$V zd}No}U}AZj?X#a3d#Y!rWz^CNe%KI^Eb>)~0HFiO=eBmmi*pWx4+;tcWm$xrKU|(Q zdE{!U(#U`b$GEf+PPE^Vc0+Da(@-$UU&KI8SbqAI;ys#NBWY)+Y*8!{9-ED$!a|o- zRxGl@`A$5%jC(W?DX(EGl^K~$p6++#7N&gd@<&|tNI;x&$a`ksLdl}Wvd*?VRd?cfYk}5jQvVBKukh~0R-%AG_uhF z0cHcgZn7+G&-1P8l%&==4(~3HR<1?+xoGC=7A3)`NJU6$U#1-I_7dnWO+U@Zm~Y>> zJpNQ8?iyA&s?1;RbBsSaoOL;O)~-nnquAhLqC?!oH|ZukJFmNY*Wv-EBPEUIMWak| zAtoNkR;(IJY$@S(E`Rj<-IA$MQOdP4H>a8OV{J9RoP$IlH%)BsCm$weX>dVaM|XzVBByUy%sr7d1sw%F1$@rpB@>SC|E#=?SC4rPV@4YUdQ{&Wk%{fv7C}$-y5DUE$2YK-&p@6T6v@tv{SY6jrHRbuXh-J ztB8M)Nm4pHcA3Qjc#l9A-fQKz{CEEmJ~Tb_A|VZX7f&uEZ@HQaDs8%zS6U@S<+ z*C&o)IUMIfT&{D7#d4c@265kPaG<^h)S$de>z3L@ZnN*dd?|OW+F$K0)aJE*h8#9u zl)>b;dFoc*8W@&Nr#QFbvT+neZ@T$X`V)Xd(9&$vZZ@JA!_4cUMI%ekrT|R`tnz-? z%fCJ|8XX_ySd1{PxW}^DSa(da7!Aa-A1_fL;smvWgI6ZC%Z$y(xZpA!5o`B4bnsbJ zEb=Ga`X_l%9|DBizapW*EKs|66yfA6pn4eC4X z`cKXet)ON&ugg_YP7`(6l}Ap4lEXs_FguS;>sb!xQg82p^p9~I@@b;<%0Kp-2ry%( zUw&`Z7;c@)3ZvNT=ElSt{NlUPOA&AbkKSZ|wTmnw0#@~h>p(o93aVNsc>)%Ce*OTT z!^3PPx+vy?A5qFRHVY0Q@BGnm(T78nnNDfH26|yi*rhbdOEZ3)?zK7|>Y%Ke&e;Lw zJ8U$|ak&aJv#}u*DJXnN#LG{chj%4RY`2}FDuhKlH`$}FM1Om)!SI1)vY>xpWcpny zT$0z?HoB$hdV1xC0lCVGH8!!xO%meBg#6cIa1z^pD^+Q-)`!F-j?;Eu_4!jcU<5C; z!uMChiZ-sSeu|_-P>AKa9=Hyha&w#CxTrfk9MfztoSj2_Z(|S?L?ZVE1*?>f1J^(& zwRDKq^nL1YjyAuLiHKmsAmWfK(m6gh$0Euk^JpL1+6e>9Cx|F~cYh2MM>XuxJ~jkd z%tx5~+o1X?^+?=3J<@1|13OnDsw>r!3rTh%r{|^ zfG&~OkcpUpcIdVs6X!L1k{sr4?oEhU4ZsK0f1vtyMw6?Jba$RE1<^G4N_+ z$s)TRK659RjmcMi{ZN5*8DWv+`r}z0nf!SH z*a99yxuy2!YT^d7D34I>wIc0aDKSvLme^csVj7z zohCLv8^Pl-?@!1f_srYPeQzNic>XTxjoAnh*NZ@ zbVeHfxYJ-`(`)l0#r{lHf3{uBcO0Hl*e)xN(KmOlMbIX8WO4UagVKu2!t&%O*)q?p8*{Fv`PKj*YtTJr%6 zWkf-w2V8EQ2Zp%yd4{Uu85lcJ*_*E#ja33|%=p^t%`xoqr%~&#K{*okgo7dPd2Q9|<`n!k;~x zn+f~=u4-zi-kH};&2|BIbFmikn_yDq0u7fnPCUBQtfW3!A=9q=-3g`}2JkeSKQ~W( z1@mA%Jw2Gk34R)&so)n72mnAO;U>e8cUs!swegz-b8Cq_mRt;+%8NfiUE%DkezV@s z4-U)FhZ0gsda2+|+jN%MBN&0&1b0()8dkUX(0E{t4Cdx&a8jIjY~Zlf zD|_~Ax5yK!UeHjyWD(P?PQek1OwUm)xQUf1GngqI$1Zkul4q%Xk_jg|Iu^R%O%Ttb zL?6K5fBE_qr8NJ#{W3Kk+6+KeQE4`~qI+=+p>l8S?p>Ue7auh}c6Z8j-taps;0Q$W zklxSErbUVzR=yrg^|fy}JK8xu)IU4Q0%pA}IM%AY&P_cj>A=ZRK<+l2UVuva_+-j9 zjZMA9CK;$K7%J%hT{2|#b$!kU$8IIEzJ6|vi7dIx)5xN_ZtOIsI0#OvXWpX~0OqTU zF~y{_=Oa@p3rsj=DKS8=QV@&y+^ z^Dj)SQ87?m0&OT}xm`xi&dU|U`P}!C1@fGCcE_wFBqQ?j;<&m`g=$DiC8VPwBK+|b z$0~;mt7BL+lWSv=m~MF>C*9~ zNs8s!4(kJ6SKt?7nCS!scak3vsTQrAI)fKn)5C^(%c5h)+RBZ!#G2fNr|>(wc38wt zC&+boHx7sVNvG$}-!H`)TVBMglE-$tY>7Z&ke%(CUkQ2PNDJPf6iQ*W-N`})un0K* zm4P@th>+8!>Fbwnum=G0YaaT^M}gLa*3T*+I#4oLqiMbwOgH9_A;Z26wr5^rmj_M5Lwh3vgN`d3#fuPV!@s=I5!R8i&H`>WE1oO5xz} z@a5T=!e1|L1d*7#C;#@xLuIRy&%z#IUN81womr;i=8j@!5)(T*TuOAD+h9TPInYwR zfToQ^-3)*G#)yd1Q>ENUxY~RMsX)FVa`Tn@Q3k+3rlvLyQ0K;4sUGZ<8Ex(=<*5v9 zEtxRs7S6UsMN`AqoEurQU1xxZVxTEgCne2Q@8@-z_vd=8V3F^S8-t8-K!irff_PH}2kLo1$A>OVnNeE-tC)bB)(YE`r+ha1$s2 zL7l%q(AU^a?-d3HySpkhU`T&)O18G$XJi6I| zOTHq|nqXrCsBj3%84}zbpeY(fZb9_3p<>*=Mn)zbAAK~thlH}r^?;sAL`J68L7JJ_ zWdHH<6-KfN3g91lz5(G>j-3IY6wZl zs@jwm>c#{}F5GO^fXs@f+2P?eYWxY`D(KGMT}6h71mud} zL2@4L+CfS{BIMksG0mb4JtYfxC!I0up4J!`XGg|Hfu0wdkk<5WG#US_dGin{^i?8i zps9(HNj)`+fxQifw9BQ@$H)F00Lq=7vLxFwmZpJD7RkQDt%E)EA76ZjE;^vadQvy# z39s3a4Tv`rg-rWMvS49KN5#Z=9JU;jehUrd8`AA`Bn}RS@S&=^+$%-}i?ZR&VS#{_ zlyry+&(e}!;T!K%zaqPl+C`LFqZZ(XT@MgI_rA9c@}%SKk0B9$I))>^$)r$!iM zJz|TOHqGJYckiimCsw7YAu*hP9bb zM~X0K;5WFbZ<*@Q(WHj|DQe4fNw&)m?>$n~XQ}y?fRL%bV_#!pV6zo@KP!!ONQ70V z*3HaQ<6u#1-Dc6v-sam>6JSP|Ci$u`ihsymxPKn^(<@t=o*cjdrBShRX+h|$Jjvtu(!#-NPLhEAUT&ebV%VdqqKCo3 zK%m-K%g3?}LB1+~aSjDK6;0=edW9Ke=6gs935YIG6Gbtw#R6kvf9^+?t1kv6sDgq4 z7xE)b{uBu%^^Xq3ojVeUu(C2R4oZL_YoLT5F6!(+$nw8p3eCxpkQrL;hyNcCz-wRn zlZ;*T{Cu>2g!f{rjok`SZ6gBtDUUjw1M}CyCtW0CSjhbSI00L z8>wBp6e3jaSY$aK>f>WJ{FDEDFwg&^yhfZ||IVFw4$!gre~@#jm7CovN_^K$)wJhO$E9Uz` zc2jx}=azD}O1QzYeP@iZ`zcG~FNPxVzHIq8Rth8!_sIeg68TUxcp#FF+w;#uXK_P2 zWO9$=l=?+f{Ntwp^vpl(*AgHu(jL51Rv)nBM+O}=;k)xOP?BobJc7D!#{ocHoan7}`yL?C6S9bI&*G(2R|>8yQtqYwFAw zq>vdaOqv`v>F39iMrFh#%1s^%0ql%Z=TCTZP2|xr^4y#eP|>xKG4N{{mHFe!H@Lv2 ztgeG)Kpnv^uy>3c8YF8AHXUO0_0zQK$Was%6dY;_x3sjejTp(-`#$j5XYXcYHZ4>S zLg#WCapMzERxS_(H#UMCLnE%f0T0(eooBnF`dzCr*J)Z;&UvCjoBQgWlFxHV2NQ%9 zZ=!%lhpZW^aBs+f4>vKB;Qb9>Y?8TBl$gBi z4_J&)j`lruK|H%j6%@L+URGOU>$<%B!OpUb3{OPdkI`np}3>UGOLaUQKvEvJ?QScu{H9K)`ze73>YdaQb z*Ksv@Ukbe_MGAu&%Wt}Wo7mC%^~w*bXAu7fawChtM6x`(Yi)eA-_QPCjtJswSq`o z^s6SvcFH-A5P2#n2Y()o^hO_gr-uhlmlvyll$JCKf3?jf^|c5JOS?w9m6H7#^^Hk^f1t@O@* zyHSvPsx?(UuF`O@`D-ybPmu;e$g5=482g$X!4-e`3Y=66AEG|q1N44p-^&Y9s_x9< z$4uI0!D_KWx&S1RmOP?+@_UuHgF|n=tcSy;Tqcq23auuWJHC|ohx!J{i^8cmUwNxj z8m?&oS_FtnH#HgAV4=HU$CR2iIpR_no*VDDzI=%uA@9KHy~!FyKw5{HotXH`7VG6>_Pg1s zX;IOc&CR=XB9|`Ft4QVK7>P->-eK606HG-y_uVt?vEYdE)&$gn{Xq(nYZfVR{d~Wy^6B2RqHz@zaQ^y zY|uu6``+=NCex59G9lN>E}~9R<9D-Mhvr2(SDHr!=>G*BwLO3=-oAZ7SoT{rLNP@dmcS&Q|QFS z%V^J{jw;ZZ@?4}YyV6eUYxUSgG!UDh4L7Mb4evqDlqa`Xe!0mjYOiY)uXgqZ&ko{U zb)23e5TnH{319llO^;a&Y)XG7DQm>Jxj_k?{_wdS|JuieFwa5B2{WlX8p(Z2Zr-vD zW~%f`p@ll2U|87ejOJwN%M4N}U^CZoa>|y#L%+821)cr;w;F5{+kjj1oeUft?&Kd? z=iBaIqKxIlVb<;&DKqjpbXufPVb-cPdeOQ*VRX64G=gm(#?0!$SS-MWD9;v!e#mX8 z_~FEalG?kuO1Y?pE2X_3dC2BiC@4i%lDc|40NE<-PE>4|>zQ`5VrPc_QLv4|vqeKBjloybFhgX8R+<94(wB*4P5 zL%_!U5;JW2kvdh|MKh#IS=#3Q*gVpF=!5mQd${RYNo*IvGRZ?U$tIonwlAc< zzF(%$L*3O%v#|Zr%SP;u4nS<7*Jnr#645hR3P!2k-dYGEX~m?=zZuf zB*Bdn1KmjytG*~SN^G;Y1NaL@V|Qx z@BSyj4A}WcN2Z9rP;^+K7@cY9&P&vah|;R^0k}q0)a4k79ak|^J3ST z?DJ>L8#S30{yzFO;%Q^;q1OJP6NQI|yJx}gM)Z0=-h-{>UZ+0&HPpIRPIRm){>Evp z@@O@yyQc%E`e->Tp9wz8(6DaSMvqz#RT?2@7k1gJ)u#B?Bdb~{9`5JYc=!y5pS(kh`kQrI`w}_()ob+B zqAEb~LuSV&s0PAsLLLRNkT^vTdevxFR}OIAG+hvno_@)sWpZ9WR5(~|y%0&p!ZP7e zr+zW0=Dqmq;n5*PxrU?6T$QMF0ap;C4HZTW(gXix`MMJ4dgz2Dei7D9K+G0imVdg> zq?JwP@mMTn+`Z~ZWqN~!g=KB>^^n?w$IYjB(&E1yr(n)tglR$B-I z(^?#z$hEX9RI>p%1|bi3GojD9PYuHoMEkaj3tYI5eL%7W!tT|vs*Tq7Q&X6tf9=x= zblrLfOVEU8UX|sjv%j9qFVW|;)Y9pJ*di-bz-Cx(HMQ40+W6)vG+2I=w<~AXdd$t{ z5zj6p1SZ^UeW!%@4&7B5l2fa>VO;TXT!Ya}agU!C)!TG`hWShLF~`GAQFq@>mTKN7+~J`qk}-CBxp1^+QyJ`LX*HNRR2!^f@j6J+ z$k1}BKwNrl`}|F@f?{_Z-39UmJcFU4;bc!J`vnDa6=@oLTKg4M3uzxdy>w~icdt|z zgmPF+mE8%M1I4e7uM?y-zmrd|G*@Nc3J4zh;yYU2u`15SYZ7?p(d6X9^TQMKX<>G+ z@ft;|#+=tHT#Wcf$16YZ(9wbTE4u~4T|#ad44gO$6JiotQPR!~VA z1Xcq9!=Spb$2jY=m7@qUB;G!cGpie4R_XI7sp(N13Imr4D=Wz0vSa|d`2GzRgz--( zOn{{P>X(>onSXj8#d^g$ZbJw{3R#eD3p#VxeP;fVl*F+w(u z1P)=e*g?%$^0btP5k-0_VeQa?^(Mg`9 zJooFOmp#iq$RI(1G%tT%}&fYQjeUeDCLbvFgBSLe9c+bogoKDJ433 ztiTg>Dk>l+Z0+&dQYI+G`m!#5uMOLE^O}A8)|MrU4v+2iMBJp&*K0I5lsT@B>|VOX z!kztoR;Cvoi)-cskNFs(!y+zecr~-kkns`nzWiN+n5<)#YSVEq*(wmyL)Wj?`IN3W zC|;py8ko!=0Pw)d?d0fa#G(>3gHw%uxwm4h|AdQcJThtl5MiT|_x!m^bO&^I zA8?x5xOb9L2!%H5&ce_guwxkmbBVeR?`DrN-3AoyK4G;9P% zwr$X3GBWla?$%Tnm$;~cAm<@b_OuOhT3FnM*)tnb|3n+AI@UU-p5ss4PfLKg1;W9P z^hcHPK~;rK#zJ?*FO?;by!_Jwq*qO4x{#rbYfx^E(1SS*O1rz*umVbW--(q$Ah8!6 zli0U5fvPa;eWsWR{!dcoiy6pqb>uL^!xk;tZZ+u+HN&p#%Smy$@2;vB7~#?Aw;XoH z7*@n}hevdO|9`iV_w{oT5*SEub5@N*!G4ke?iG9UnAt+GVdd9K;!p zVkYOQ_TPMfKeFO}qJY^^*O6J1WNnY44m zC_r(w!-XFZ$JtqRoqhf&hPlItJ;ooNRU|GIRjhsDnnT&AOpR6??ct%W7motA0fZ?SJHs*47Kf&Uu{fL38z+ z2da9hpvly%98dj)`Ez~3WkbN(*>s_6oT%z`o*!x7Vlr|*rcC?jr0YR<+`NJ56+qgU z6O#sGnDW}&cdaf2^|!@>K-26peaU{67IAjGTu`<9CqSdDbubJMH*b(Q`YwLvO#*PK z;mk9+BHT!UYPhT6v-w6G<1zOkn7C%W|6pBNL16(eZ2V^c3xF3_Fc$3Pn@#({RUcGi z@N6=r6FENk_Z{zzmOQ7CgFut>GiMo{r2r!*-a~2WJ%sObTLg%?~ z8{=ZqqD4_HM(YoRbRfTOBMXq&6GTan#<-xhrl?6Jf*LYK_Y>X=b+|9`zxQQ6Ag)E3C&k&I7~Gg zczp2A*AznKfmT@dB+c@+VGs!e0&+Q*Jy29;N_8w7W1(?}Tm_ZsT*q2f9q;8BSQxZ6hpNEM_4cW&H#^=jB)??J?A zdpoj|N9gE<`7k~IxSKlh@m~Mw(P)h#yRq3^JF+^3k5EXbqy%&BQnivxN={*{v znG~xm3$d|}DfmJdjG=AaXc7NVMNxGd5f2~B; zSZ}B6>D$kEGi&`y5|EL(bzQE;#y?OgOU7mEyVhlK8Lo&O5QgUe=JW$j(l!$)__+>n zptQ(4z`KlrW45jZIpTs;*#^ZEwR_r69YB}X424zja4eg zFl&@NOB1bW3vkc}DV>D>@vi)A>oQ-mhsMsL`HqovVy^Y97QW`CIvWxFwv|cO!P<(v z=a>A^?mB@^?MqL8KXLTNz`I;M1WbTF}Hgm7^wtv1G!^?&yDb!E)92HTgeoIrxI)mCAb17K^Z zA@_-zr)MxW$a^_W!!xCg(fSnT^uK5B49QGnpd+yEfU2RLX?O; z+-9SY{+phDqMgwU{@-UQyl%mTMMznv{q_C4;1hmYalpQGFF@G4(nmGsZ|QR_*0Br= z$}=8yl{{wMXfS;+tOCWIY7x|L2t?@!;2V_SvKtQ#+Y*T0!8~x>BMb68D$Q5fku%G^ zd$$hQYI>X?IlDG~sf z`(Jggv)RRfB6LPTr5(i0@##KP^j3K=%b z@lHpE-@oU$yH0p|%5=ZS8HP%;QW~aA#v0rB&qc^WyU>U0zJ3gBeTfU9e7ts`HSooB z*;58IzNoRUY6>_#RSZ>fw#Lga+-*6_cDax~a(}JAE4Flu)8NB;QU3HN8_I9e%ZzmdFBk9~d)OEO&{qj2NfL|w&8uljlpTcdTH ztz+cMkTj6My<+Svm5aq7fW1|16q20$qr zS>5s7GuOhudteG~hn^82f463#^fUfROD%fi2FN4ZLX2J2b#X^1Gf-pO>$1-L0u`y>YDKaX-K8=po0wjXVpufiC@glX@i=_?Js{px zQ>QB-hgzD$wKBX>X1L$p-UZdRD)?6}6*zI{Iub~^8KuI) z{QaS0;ma2`s~0ckzLuYUya&OSZvz6*G9QRy>~o>(3ssJu4gK_aQv**zHARyf<_tUL zx;nb=hQZ^{O0MOoz8n^p`|vrX>8hH*VM?uie;b+kx9cEt%cEdeWpt>U7-S_+#)8RM zV9`$no`}&CZ<)pk!Fmjuc1zV5!2+W&7@hfL*@y*KJhYZ(m_(;AZRe}xQT9C3D#RQq z8(UM!8~pm!=iuVY56hTA`Z{g=fC#o(hdSv}b;cxx1$~D)=V(>I2~Jx#8*VN(kM8O) zhFNukQ!|UXd7)1eh2Z?Ty=~Jc$RzDh$Ip*xyVodH^mc0z>}`+AITD{dtkVt)Ti^(K zw%SLPD8yoA6~)95AQHStif5=&N{a?EF@(7IAq;nc#&C6EoFndLlEY`&(3c(#6Wu+} zqYGQSS05zL%5!8qxagBt8nHaQMm3&1Xf}V=^o#xOwgtA~$+=Y(1LvNKubn=d;W;<8 z--Q#`SWw9D8;W4nW}`9^u(&Fj`edbmo1O_HnauZJh!L2e;Rj^s-65*zAvyfA3{0Zi zds?R^m{k@bD+sm_X)s~6f3Tz(9U8SNp_3VEAAa!PL_Nn%u;FZNYeNkefD{C?tjrxpGf*I$4>E9}4%7tP2F zq8GGtT>BfwpxNRIDo2H%>Px6|7JkOj{!OVsvwVxngW!i-_^%fO+FPQ(nE>>UVEx^y z|F2x;C{wLOFSv--8_dGKu?XJ*BNL6_1EW6NH4|<~IQSg#V1o3$d}7 zAH!aO0j~A%5TR$uV6a|B_rZ67jRC2VH908^I&cYW_z7bYQtaTAMJtWw|tUA=C!-JW!AJ$gRRbt&f0x2%c4!<}Ar!p-PYD^iB5r>cp0I8)# zavjrq&}Dr77Vfkm^-=EkAs6_ z1)3uGSX6+Lp~hN+Q3@FETibtXhlJ$yF+egqeWf4>E^0cj-D*mah@uCB8P4}}&}+U5 zqOr5(tYpjADb<`{G8-CW1eRe=;PnHZ7$}3|DeE>8u(Ukb+6uD+=n3!=mDF|$$02>%EUePE}(Z2wz2U3s2SF>_XPKVI+t4&0MDFrwS$wuM07t1a& z*%w%gV6@1rZw%(&AjmXZ1#A6);9!T>q17Vb;cR{ynWvn$595XT_N~CxxN^*r9UErM z*)j*-J^}@zCyQ=l{Z>Tx5%wv-gMmiig%<4P1oJt9dw`~m0edg{(CjcF7{$N=Bdgmk z4v5#AQC=*;s81#?CB?B1Lk>cKNWEXIU!n%;5JkC?wA+uoFf8TZy8y3ADraYBaJYVC z?gP^ah&HCdGl%YQQ*IMAd?s)rjdIg>42+Y5LiFJKX>dET$xI6laRGs;9m1(P16)dc zyuAY_#{f+-=5>L=Trd|J0Tw-QnlgX9ee?&6NWln(5X>lGakDixEQU5~77U^6`?ohY ze^*=UXQqKIPPw)3Fo31_ZNYBE5ZOTe4BMjnzD88c)kc7-MB~A}%20q;zxCFv33vfn z8BWvAiBEeG%`NGnT#OUBo_L&A_mV$)l=@cxc?&#UX8F{2>0ng);c6r8(pXi|mXE`z zr{ne<0+tT~77Wwj?>Bj8VBp;LFX0RfBvorZ( zvI$RK)w6@C$D+!$;M3k(d-JR&v=ORJ@KtW5`~l+ew(PDz3ocC65kX%-e-(%6ES*A6ve2OSrG zTT@(I97OQMQ*hkE669M{40ODPK+Pu`N`pA@8Wh10!@#r&Wx)!2c2U2)-4S%zKh-+) z;Pqgb_T%#zoGPeI3`ThO1vq2X7^5L>fs=)JA4Cx|7>6?TFiji|MN5=#N?;J`7KU5J zM`0jjS#+w%EAU##2gjF5y%I)^@(*}={@&1VlO*V_p#Ug$nPhJe9T#mRc(0gStd7_8 zn_g)2m5v)a`d_!`m^eB24!35(@(coV5eV!1U0G;3Q2vZS ztwR6-Lw4idR1r84emL2IO+o9E75MOAh(17d=Zi8dTB`r8yEFgGIc?u~^2Cf3W*BWZ zX2`^}Xi-|NWvB_+%9eyIX%XFR;$ci?w9v#v3(15@nM#U8mbr_Hnn)$xW^O}iv80H~ z^S;dU{r(5v<@NYwyk6tpKG%7j=W!nAbsfhzJw3gGO9||E4QRMXDA1D*F|R;9Y>e5@qLcMN!tbPCWbpK0%t@rq9 z11B6JqnzwZO--teN@_Ihuk*V?I$PFz$7q_$a<3F+R2mO_l*7!Hdajw;O!Nsu0t!!o zm0tZPLM#)X<1vWT?zdWKKidU^t`}&ICt7VCZ^lhi%t#LEcRkzwUN{K~S>C$K+qbg- zZz5Oh`AsH6x}dfx1w-ayZ`Yxc=*zEl9OMYaYHLTk?9qEg=}L1>syk+{n%XWc+4s3~ z`Vr;e>yn^Ra@UM%%8`_l*#wbzPPld~FW&H6Z#B4Fp1#cTYyrE0|ctkLSB|!OFG)5ZP#b|tewdiyZ zLqy7z5!RUq!Ep6FsQnb!jHLZYasz3XZj@uVWu+OC%B<9U%>G7cp#Nbm+RNQ>P+s zv43=^1@60-K>uV9q?nRvJJ5;6RAkwZu|5aTc%6*>hUlS8vZdYIc{fAn-V;_jRwQ z*6m8}yp+KYr+?{}L*2p9vjWlL(V<3*GwZRgdd^UFf8p>|lpB-JLXX&n2BNOJOrkzu z$&@KOvR)MR9W*J!d>Oik%xh+&*;t-GaM*wdwDz=Lqok6QU>s^8_8faWwk7Z#kW$so zCVw`@NdBLqtKD9kYl$aH);xUl*ejXOZ&^RP^UK20k_PmP=_3)%v&)t~Qf({hYoTjs zm`ROscsgS6;0ZXOiDLx*QuW<;$&7|Ks;UC`4DpzQN;c$bX(J-zU6mGj ztACOgZCtwCMM&^%oKHSylPq!Z_V(_xd-u*&rxo~=_8`n(wSJcnD@9b1eT;PwItwXR z*9`H(>@pNMF%SSB- zhyBBcCdf|#@!N*|;OJ1VvalhmbC+yK>9-|%1*fK6RhMYWoR0aObjwN0+J9||+swt$ zia^@|5vUxecDth26;M*P@y<#7G%XX#KLdxSOxvwIaB!yhT*gjmKmW|2vvj@xe9vLV zvxpnRnu@to`l)HHUi~ccP_e-@wmj-x`guUHM9S>yL|U4a-CnO;&ti4YRJu=%;kkt1?mjbA|W zwFezUy@oa+F}|qr;VQn0slzw#Q2=-TG$_^c98iWmi|SybkE&z z;KT{0oC#s}_npOcg+5~Sre7(xXw*9*J)BjU z8Zm7pqM{D>HX(EferU|#{@EhY+du=N zdt*bc-HYp=;)r!CsF9LLw3vKXF(=9DKuh6sO}nuC{qX{yy0y2X(DwT2-O|eJ9iF>3 zayso2?Ne+Z0YCzajG!Y$txCl+!98+X6PwdE==ss(Z2TV!71bUe+4}9`M{M%$2J#TT zWhpZ*^hT(X6418GxtYfH@w9E<$6 z9xvIBrshaoZYb8sgw~M$d7z^6I7ty?lcR+CD~4Ei?kt!^FowUXoVi@0KcEiJol8_T zb|Y3hHI27a)zlu)Otbr=4o$R0gBA>uer-l?K$wXQy0kov0H69Z?vsDH|EFxiw6P`?Fw8mxuH zCg?fIU+3-sQz)l7+;&g24A4>Cl{&bjba2QuUX7@#MtuBF@>BmGew5*P z&1xm3A>qO!puf^XEzu+X72=w)XmLb~G79$6i^^vTJC|g(BD22MtO5Mi=UeIWwu|k~ zdw+8wL&BsfO)(UVpPx1ghE4Lz@3}CyTv9So7Blj_g;QxCCB3kV%Ow2P?ZWc#X6bS_ zUa53NLwHXiOt%1aXJ`hM&ZFniKo!o#q73gd!T{2Ku%HWXbv%;`1=7sutG3;wd#>Vj zqHXH?;gdaNA|Cw|bUMvVP2+_9IpnSS#qqhgu+<63)p@+^DW;GQ$Nb5Pan8 zHdoy9z7W=x{lG_hBlXGN=8*C}O7@?L5A1r3EGTZCbEsX{u3n|>;Z7h&`O-iBbSxWt zY(x%CNXJ!GBVfr_v?5?HxCCvxus5{t5;X{d)p34u^rw$=aZpQchdJCK!>xc}dh5HS zG=2Rtx%~3K{oz2v8**y1?b>kYNXjzlrXbgy+oX*nFxj&?q?)=!SwDPS{Az4n%HDlX zw>44uO%HQx^L*;2!Do)ATz)obTZXlL!(q|*ro12^K@Q+DrO9)vaZ1E09#l`lCz zBb;aGb*b)-ae9@cM#vQE`s2YaqzPd{VD7Do5~%I@8h!jd?MnqXDvO`SDafk$2ge#KOI*5G_(Ps>8lCHw_I9 zvOKOemvwX{qiemc9-+a42fg(E+QV9yF9?3|bSo{8f3c@!;H^WZ#Wh{jL(6SUuJXV#E`RlJKblC{PN_H?i{7jTh$+Re*`0kA2f0|yir=(Orw>~Pna@g z3XVol_5t({bevF{bb~p(wdQRtloui7bb~`ec&e|cxf6VH;0cAj9qj|`9D8vKmOp%V zU0ZO15JCLm2QC*MIL4OuYHr zcFFO4;qO?i6GUhbX3+n;KdbVkFUEfFF~WdA@8K(gJh<|DFLbwd1$V>ggrF9ivWiEf zb8(PnSr}R*CL&hZG;>5Sw?iI7ak3g0X)D=Qe1mpWh8>r{-4a#V)%P8poH(7;e~`p# zsD_3>3io{qT8sYK7dRJ#xa>FIpoX&=D&>7|k7;r?yMxe8!|V(Z-wo6u7}@PS2*LdV zN?41_?YFE2PG*m+RqK7slbmOg{MEQIV;&{=Ub+#q4px)tlS~C+Q^`BY>mN*2gbkLU)G*)#(3c0T<68lq z&6VTL*1O22M;BaS-FY6hP-!EMoUtvH1kq;lKJ>L3)?x;7swX4AuhA(Uo2L5_3cBW* z1dprVevI3x{}%omonx}1B50wSnwHrl44)Kp*={U`w+?Z6+_7*^DXV7>l$`tj2f&M#Sk+cV4&X|N>g1W;Yz z{d2+`n|EEk>~J_P5VE1WAp9%~SHd}P(r_t)g~iE7j~)eBqL=$17B=s?R_*iMjF@Z>SB-|yc{{Gh~_({6O;JvA( zty#*E9cdfhc?TC@uEP6!*>XWOnT`tBKT!be_Ee$+{>gEGnwrFH-Q4Pa04N9Pl(4Xi z#IjnK$+duVya%3uiifhy&Y)m$?D|IYoSMy%&frDvDNWY}MzaH(=bX z3s|{UsZwdiOqw*QiW+Ic=>X4nG!9!@wl`H(i2IWRQD2yAah^_ z;$z}mZ17g+g0@Zs&o^g8`GO_S)(YQaHY4WSaVVqShTAfIlGwGD$<1TGJL}J1N<4G$ z+U;XJZP5g}xcm;)OZgAwG@0ibyO~`*I+K#l?TcZn9BmXs#Ze=LJUQjp?MZq>L(KHSNuhYJ z+=`jf7X7CO4|uK$b8>RxozHCi?()2*Q9pKX6A0oT0zp_bgCGi?B!|-Fk3sOIBRRBf zJ%W+M3#nIY$Pj-v&*_gZdai=+c;frAQc`s;r{KO(6TV8aJw1`Sa-GeE5n+Y2w2KM0 zH*elx0dLb;^*Ib=#`ZEc4sDuyAXEUwY=3f{qW4!Zn^TLXkU`;(iX4VEtjJ#>P1y%#a07;xI2{I%88vktllwB;)$}bXtwa z#{ESG@8C?5Gps*8EhI)>&u}^S$Jki-_IHR$NG6A0q(>F8?(E$A=U$@*EUCCaq@fxw zr_skBx{PEdWrQf$Z6-|6fcx|EUW>6THVlfJj8H#PRSmG&mo;Xow|5IUm*I-$kRj-_ zS&OmHnTs!P@<^+G|Gghhra)Q?RIF`m!Z-*GXP&;l5c)J2`dOv>CRLL1IQhZ3O$k-0 zl<||egE1wTlWZdj3r?nNaRtl;d{MhF5*RmO!i0i8PrGj6+P5}Glr{i`&)>6+PMR8W zx#y5c5QiXeuf&mmC={lIzcA2h?PTZ^PJlCh@dekB;ui5WF#4Xwrd^S}de)WegOv0> zY2h^9Ev4oeg<{bZ5KBTWNo3E)!tnN$-dH(Rq>6yaNOaN(3srklgCl{!5cf2sD zm8bU)#TyUiJK$}LaL@*Mop^EY)O;_X!=XBP8RbN2I_^mMZVWxd$N`O}Ed<%(lXI`o z(lDi?#Pm?)1gQi>FGro7kmCqIN=gdj!Oe*b?I&W-AZE-271e_O$8z&CM!35rf&oa2h#)B~t%!6;OAAV;bYlP_DIg#qAky8LS>ixD_< zet$mgpwn>Q-w6o7&CJA&2|E9tF#itIvx^dOy3c0sGJik!pid}J+BVtmu{$Fo-0Hag zHg>r(-ioiyCL(HmezJKbmfO@@s3!8`cp~>aQ>k_y+yQ(H#w(~_`OiyPsq;ixj#b* ziP+iM>HPhh=|8`=#(C9!t!jP;FGDe*+J08Hq}0|tx^9a^Ew7>G&2)=DA!R6)fc@$S zZn5~Z8rNNwR+`Itt(;FH&99MDFSTLmoI8m}WgZlN}UXu-e zgp_<^rRJ*YA3qpjHX4ci{lrdoA|+F@FfU)ywY9Zz)NY7NOaCr5?qJuePK`m})7d*X z_#CfR440T*xp=YjDZF&Z*5Q(5COZG=&C zVB<$3S`qhB=hczu&{bvCGM}rq<8@w#TgE84eko#{$Lh&fXmGOoX4)cCe*DN5#KSP~ z@RVj|X8zUFDYwd+dHM1x5smQeM|R&o4cBy+St9+fQMWS@;7@YU{paQ$)=P#^5R!9A z=tsoHl9Q9W?X3*Ie}CT1OJGMD4MRS2EslP}xzNj~1&UHk?0 zSa*iYmZ$?cTpW|LJT-?*SBNKJ(YPJm+S;o0?UAyr?_Mai(1EQb?{!K_kFA-uh`scT z3|e8=jjW8s?9R_vH}|t0)hu8pcsbn?vK0e26whz#y*F&So6W$$ppYc$1-G%)P*8Ui zcoFT0VYrA>LjUf{+xF@C`Tdjqaem7oRtAP=nqR+sxkXMs{Qb!XBccb)%z+^x3W|!( zZ{B37=S0`n*N5+|k5xr}{`}ozacvD~?rUyNo0>27&CShiYHEV7 zZ{NPHc3Rfh8ulZGdl|AQtBw&B6}@)t+E&pJyup|FlkyZdA|m3(&70IzR22L+D-#nh z-Cxcfnc0ljC4BjEkHdVpI4vedvncBHV45gRCCft#A{~~7R7W*WqqV6iH9h_4_!ya` zDwX{F_NQbj7T>5WujwBtDZAC<;~wiGjVpS%%J3r!@q7%7jI?6jx|*(7%ppAEG(KMT z;8JR8>X$EHe*E~MU1o9V@?{G0GUZ<$vm^)o4e?+q zfj;RIG6PTL0xYc$CRuZSXR*K0pD+h93fhMdH8z&$zLb@xQ(?hF(HMQYTj0~i;2@p#57g0%VQ@9LK9S@AFvw{nNcxfW?uLYK6tG+ianP zl-68sb#z6uZZCveH`DVJdm_0+spx4p*BKkZ3~^~Ukg4q65s%DNezh_2%d{tvZSV6I zs${mtxYNm1rt=`CP~OGG(Q#+gwjTB489rj@{Q2|xJ}0|DWE>PUTWyRcA79dOy*)&Q5toMn(i~kQY5ApVeng2Rl1yB_-r)S65fUqc=Y`7jafr zR~hBMevXYbcXu^0AAi(H3%*3=|{ zxX|I=W5HtA@paaY9@gSdQ;klVN>?0jM{jTM?@?u@TRmc@o*Ej_n4iV>H=>Wl&p*}* z|9Ro@v#8zry#92B{+xp4epTiu5sy6{te9MaMxHha2}zp{pZsM=_(rd`D=tYnp7It< zRCPxD^f^7*->S=vsI01@hNMZg8q;>XUe96sr>940Y5W=(=iuwtkVEyuWO!n~fpd(+1+IYin1C zXb32HG6(kWnxkvobalV2{pDiVAU!#I;G$qE9JO*5&HRO^1;Hkhp2QP~mDs=Ckmc%6 z_H^h;)R+fL&HH7w587+znLid579#aM6w*}M@U5+_31ddO+uF!^%>KX~c2U0lrjdiH zbt|cT4PmLmk*UvI@_cXtBRC zoT4d}rvU-|T%XIN^EPTPj@NP~%#YFqN~D0p+*g{!v{4rq7i9dCJ0c>G?$_Wnz9_R8 z^us5U)s`r{XHn)3wa~wRglxGziiO*x^P$SG0}-8HI_cV|Hw_E03i1hp>gz3>Q*&MM zDUu<5i2cdNk01TdK9){|_Xkv!WMGp0d=e^o}aQC@W~VZ?4wr(2M~XFc+pMLtOs^-xXDhwj{WdBFbP ze-NCG3VI%FjMWE)g^4*W4Fr>O6U@^j@t8*pbnh-BXJ^aH%axUt-RI+kT=Hub;Rw!p z1a;AzU@#g5`e$+ZOiF6J@(n#BV=L=?_RI4H`%Hl19#XYfeWhsQrro~35qV!BqJs3+ zt$X96nacEsXn+n2w@{nTtAyMfd0YdnMV6XU${cXDaTLRNv2GJdf?LvFOFsOaQ) zAGwMLg`I$4Ag4ehXO;snb8y9Y^3#vgtzpY297E#s%#l!p+YGMJ2)`)sc#g|y*x;Kc z7lW>HkkZ`h;HckOLyZO-N`H}^h1AE&;f6h&-_>W=t$e<}^(Xo1{`xpSm8)(pTw*aV zco2^1b$ZF)JxsJYx>sqtD&I`c;M+iop%V3UUHFq+eFG7G22kJI2+`9&EhvLRNk}cI zhJEey^5w)g0f(8`u=x1+PeoeQUUa@8AzeyhC#3+N;K;VeaI&;P(2#QIV-_=#6ytcz z54UEwCW#>4>)p1~YquaWc0@U7X_L(R(@*BbPJ5$n`-uKX{kNP^V{c}Ul5bX*WaXVLs=CZ`y)R8JDCBj6@6|1fJnb?r5uEP0pYr`;uW1m5Y%x=)a`>^l9%?k-#T24o6 z^{!sMy7k}>AO?n{dq8<8_7dDZJnDfSjQgB=6>U#0a?`b?OzgiOg1QZ6km zUFmta1sj*ITV=o4m$tF7VMHWgH}UfnyI;RK-73ccpcG(L0l&|?dw1@<1iYT^tJe3g z7F46&xCdhxSfkOe?!zjWA@lW$crAx=5C6UjYon!osLR$@*^UuE|MtOIy25jHclX}- zbp@))XBQ_~6;)NhR|tps?ty^;yu@egZPrgsgf(Mdzka>YRYFZdV%+`?2&#a9z#MgA zTx4X4$G(kQTcKei!Id;3sPl<=3Bqn%bacu32=^9BevD^G%<_w)6mlWwT;X?0ii)BR zbMjD4e19Fd9b&d4;0Y7Od_>{<|3(++0R*6b8}ygu%=j{+JqBW8+A2S71JZlb@xf3^mL!7$&Jl-aksWEaQyJpm+)qJ#&X6)_!X6tG|_1oz;fn{xtScBwBdNH$H-ap7}*q4&v=KTF@0 zOPBoo{JeI0rQpyY^qCnl)R@yqs<7Khr>rjufA0K+FnM5s%ol+I1Eu{GIKGnbri8b5 zosFihuC9p*5V;KEyZro`-fCx=+Z7$Lc6NT;BkzCN-LM~jGmDio`-Uc5o}4sHwTJUQ z!{K0W-|k+$wBdIVhZ>ZshxKTL$Ff48N#MYM8lp7MZF_FLcDoxm+|oc6eUv9KOp{NA zTs_9W?eud%0-!iGs0>BpS&(S1JT_7V&0_e*RnIgp9$2$}xd}2hZO5aXMZms#lMB5m zL8}ybS|xLVBrK2toB=kUft=wIDcgrtV-0@zLQYFNJN-)WH1V?|VF3Z3i3;o-9j%6P zW5bUC2*>kUMx)!&6*i~{NtOjKbCxaQ$~Ppn#sp$b?N!t5`!R)T?gDPFU#Et*@2{aE z>v=X8NkpQt4Hw=no&|dK!AzvmjJXa74Un97+-Ho8jB?j>{VfKw)1EWU&_~9wYN@1W zC3i7AcpzgdXxX0th#1!5*RNl5gjs!}ae(N1z6q$NsmO3Z0{7lu)1%++p)4*dTLm!% zC~-8q9t8ZlV3&{HlhF4(PLsvrDx z?}^2;)MP2$m1?D=q+$l74GohqX`8*A{>-i&z_BdsTVtQW!_RU|PftT3ly%4I)-`<~ z7T;%iA@Je|p9m+t@Bso3=evq-t*bJ2YuGf27h_O|UQy6^!~PU>B{rS%FEO{*pO$=s zS8A^vH(}*FZwIILiBbRuMclH1wc5H^Clh&}p^MOjiHT`p9C8eZ2yxNTdmxo88J9~| zv#?a10LlApKuSWgI$l41KcMSebTz=ZwDxL_wLDtB)q>aaJMl3x+KwOmkNXFnUbmNvQL zBpv_@EYz_O!bT?kNJyxDqX8cf*ICmQr09+4XB*U>810WT^ps z)G4t0`SzToHXlZyr_VqdhpB{FyY3neR)B!klUl(3wZO>YKvraJI4A;sA3rYdfgGk+ zK&0@c{F)~uK~|T1y!;Eaq@?LFjna)(vrgXFL({|aCgK4JLN0MMkF$1KhVagE+bhgx zmvJx6_0NTc58W=^6g!Xj_Pp+zCnOX>5Wf70G?&Nfyhb-ZpW&EgR}T+}PQ3y}cX#*J zQ0m#i%LKPy8o83#jz0nj2Pz!GWod0q)az&mSWZKOxLz5ES(orh4~~wO)Ds09(rceI zhWFXlZu)EI4V9HRE~+5#pnB%5+5KOoIKSRk`(|Aut>?x&bQJ&_*e?Y=znPiWjfZ5A zIdhSrXu*v$29on~k>eiD7m^+*%1*u6>NzE*JxluOX=x&Re_uiB@~x=Y2aI&Ks%mPf zF49#WeKhB)0!eP%*jg#+IpZy|(HsWe0K@>N@!6t<+1OTrurn%t`%K%H{PfDUsD{NO zFBBxh5Lzy)zYh`YLlEb`k*{)~#+kt(OLUuyg|th)W-(reK$|(VIrNAy`uGd;@|HnO zKRMdPzj}3ZYpcDZBPbx?k>kR1XXkQEW{kdjThnJqw&KYzYMia12F&^m4-bQI0_WK$ z|2dPVrzaIPHOK@61tr7{z<$I|55sC_;A;=CiW%Bpz%R;eP$PLd{2-cu5U~`J4$xs1 zlAuDcMn0%QPLX z)oSD@$jNQ*@AK1qP=vIcl9JLf;GMNIGMj-AO3;Hz;4(=XoPW$I|C5eqUk&dBI zv(pbsjjK${Na~E4U0t)7ORnhW3?>S@so2)OeFI#J+iC{nA=dIOpgC;nncpBbzJ^LU z;(V`mg__H#lxUSI4oL+x2nLQ`i!yvN_T0I7=1(y(NWfD&#S2DdR>NBry^2&*nB^YQUL^(P3Ak&ux1Cgy_+3RpN##kF&3#m{llxh)h*h$y!q&@|~)&-*JSrF+Wq2!;N#KarT zWCpsGcE4Q9+HDQE-b^(C@RX0`N*f?3gl|gn+8aI-uhJR+fI0@v(0pe4rJBS#z=vsUJ+cY$FfOsM2{Qdg^C=c-H5GnzcqtjB!6aedA zJQM8bnt-n9!f~T(0)zlqFpb3)dKUo8*&Oq&N6Xt<-_=xCub~k-`XvK+_I>n~AP&jO z$}%%Cnf>|x1k#OmsabEm&uPR_Yx_W5GMrRVkfHbc+S(S820pxKGR6l?Ot-mS2Y|{E zHBew+uz$Pi6YVcRksA5BSssg^7(e=gvZoD6gacb{JFc&PRtc86vh`R)Bpm?%K=J}< ziJOg$jh`Rc-QA5siGWTEhwtgfYixXZTq3nwWd<66-~bTRxo%DY*yrNrhPKb@>Z*va zFeHsL09o`(sxe7U_Q#+FhhDYg#LssC8#B_=Ey_mVQ7bHwHUM**6`Ag|JN3x6zv=k8 zxnUH$$ziBf-?+HA2;GWkw4$V>Kd{cQFeS+FR|sxDulGJtQ6eKJKSa5VpLLQrIeVL% z8DL$`^rCvCS~uj2V36o>Cc@J-auU9MQ)7wH4YHB>h#l=G?c(Wh!;ov;KQImZQ`kiX!5GL58Pt8{OxMr)q{M)`pwW@3nxUwE1zN$f|G7H>sME~M%s0dj4h}9} zx+H=T)p7e|Z67 z44?~>eeL4e&a+Y!1=OOKg}u42Ug!07@q)}I{;;iSd^Y%_0(80Y6VMG)}d zR6_Yvce)ex@gkO1I!g_Y&Yyt150sbePzXsifS$EsV%P&vD*@~mEWN<_DGUjU!Iz1d zIUBA|0@v37)oa{+IqyH$M*vOrftseMsHmDQpY`u28lYv{(2y+{*xqu_`Rp$Lmze{c z)v3=O>SePEZK@*Ht9Wr&itn~S^`sf1p~F0eOjrAk^_A*N?CZ5-R+~||vWnVmCKv(w z`~2zoa;QO+M!bp`otWVgt*YF}^5Vkc^h|%9^D*O+*rd(6P2IzDs%Lo!2WM6@lU+E$ zeq`q;p;m~T)TXXR3;BETOTX&V^vl(k;~48Aw02S-TzcS>j(Ol~rjwIbpHp8I$Cksg zsA*%2#LtW_1(v5B2yCMt90jeFBsXs@+BUciOMhtxP+8-OGscarC^BQ;$FXt5KvM@emDQ2^XFXhbHWi=;0My=Fn_7`Bc6)je8 ziC@iA=kr(QR3^@Y@3b6QftO4y zZdF-AH~U0ZUCLML(phbQJ5G_D%^93Rxrm5i{-hK>=QNhh*S`dJUf?q*?E0sz@#%IW zk-0)#X7;^q3rm+yn>5J>zOI5B=2AMXhEEzb^b8n$|B{zDDAov+^H4d zCAbNtFr}2Mu97CQC2sl1V*l=)Hk|HTb{51wi|y;+MIx;bHR`xU`K^MsjJ9-5IgAEf z?CmzQgBGw3{;T8dwY$F#iF}D~#woZKV4&%zj!_U0QLRmW z7pr>vtgHa#E%D(xY+M#kHZ8sPH|*b?EkEt0S!PwsgVwXcjPuwJTo84{XAeroW=|&m z+rjmL2XJk8z!EnH?d73%=AhzgybyvYKFLdAH^h0i&4!v%WLz~|WuLTX&8Q-lcCtko zyEJEe5D#5CP;AID;qy8 z!XazE55IraDzHbw{)<50+^EFEXIQp&6^)C`R+o;KZE$mnv}uq1Ic~JFn7nm5!p4*UX>>q4tQ? z?r&d?oFDNheTB~5cO2s~_=A2NR9=&qlkL8a5yi3(sQ1%bQ#ZD54ruQljV@1bPuDLU zMu{^e=AOPjq1au2jXD&a@}O(te$rE#H7JQYxj3N}Ug+>e;3*M-0<-iD@xM$~?@{j& z3ANj5Zps46!|f}c>e%9(j#?^&z759yS+~l4YD*W#!Z%EPxAb}{^-~|%!)vL`H2r*w zgBE2!*UM+H!31Q3!zAxG^H=nwZ1x56*}Rm|yMV(N!Nf#{O|^xO4iX?;T=! zFA-k*rr*?EW#RFeTw2aayH_5h6r=i7DdD#L-BO>0I>voHuf3%op+Q!+_{0MUCn=fj z1B;7GiY%vkC2*4mnAVNQSq%Q?t{2Xt$2FCKVVIOOTEFWSs+y#3Uw~(qKcv(m)EiuU#M2(e6$VQ zMGO&%JXFIin%wixz$9_mmI|7x_TbU2Sgj20>#ol(q_I>~6!}zbcC_9Ir4AIk6;2YK z)0sEHamjdUdWD|L7c?#6n&GE;iMCYgerFE3iC(9Px6qz;kJ2w#xl6z5ODrJ3H)lQG z{LEv|%DyXZknO7H!KMnf=ZAMed= z_bY#__dM3kim>Km5Qj8U$_V{DXP#0@o$lX~J6^h+{Z#x*Rh5ZHu1XHc{`%<5yqNr? z#n3=yjb#X}%KZ?E2bKd5O+T2t&M3^$Ny!h#%MpMdsw*9bMGo+42zpvh8 z$tlfF=oRuKhD>v@F45+{dnd>uF%;r!Ra^~1@%#g4XI1?_^;5aITfp|Um+OLmam$sI!tPKnQVJl4bXyntk zjs_bScIZeJSBZV~iMF`+dr)(RHRA%uZzO9p@`o}@oUQL$JkDYxDgnPaY6lYCZh$1?{(Ql5h%)Rc78r?s$=of>mWkh5nXeGVP0is6q}_4q!aY*ZP9DqcU6dBQp3|Ay z{{=LD-1Q3!Gg%%g{rq{V#HFqB&7oZMYr#im_;}20r54Ph9N3;2F+bwa@=&EApFZg> zs4O+d&N1Z`QZ@w*k)c)(i({p@>6*G15_HyVdWBxM4KnnlOoSo5#tNTANv3J`{VaP& ze3y?OjwPpCm39txp-<|K+Z+E79!E$C8gUcl+6ZO>X)7l*60o{g`^0+#1(;6ZMU6mC%9=DN*yoeY*qK{LlGYA zquh4sZUM4>APwn%>6%ciqY7>jy?g5wYq8Iafl+QMlzv^cd)tz<)Oo!YCW*0bJey|g zf(XiLe)&!%f)Ii;$bmaKy}}KkO^?1>sw|9wW?8SILM0^^sxP`lAGf!;qT}#*-jDcH zaJK-E>(RjS^y}qY(bvoAJR4)ba5wu8lqf1u`h`g5XxX_vDd+Ndd~z=FT4Jw2FOn-i zucnGU?N0NrmM3wqY+=6uE<2Vwwu)e<8jH_?Mb@7*H>jMtC+nFemMRP)K>)m0ak zZH>>Cf3h-^$bFCzfFV6jMmoO4+3eYDk91L@>`Eu|?R_m~?4#C0#PH5sg%lLk(AIx5 z9h~>+fT;0ITEY1qp(|H^pP>(a=mc{Vr9F;9>+!QA#!?o#occ85YKzy~ye?oaKN=BS z!b1RNvD%qhf9exs3Svw(yOWsJesW}9q~US5bNA@6ZPI{ML_Pk$5~hto)&B&fw*mE60Mz6y>owogtl{N|pZ})Ew1})^xa_ zI;vvyztT)lOia(vN9oGcL=?31pY#Fxs&qV6k?|0r5~)q71;D3Zo}HT<*vGD^rE}cs z!Q(OE_7BQ2iCv-mm|v%-kgMBTrKlzK=y-5rJ7-dJuaebO#Z+h>*ps|@YZp2%O|v6n zkJ(pTFdUt7PS3OYwt4>`x-69sASry#GUQp7@i8b;GwLw*i*>0)Jy25~nPs-LJdiT_ zcdo(Y{r|$#|37?tYHLbxeZ|WC1dcb_`Xpa}(#54N-pMAqbaipU&8^AUgVLkK@4%(1 zeayC@e)xJBy&!+}bG=NKf_W8Nv!8#I38i?q%}iX?K2lTP*uJTQv_QB_=;L3~usPY= ztBG=B`OUj9(4Ubl7|r#$!DeoF<@W@A`apfWR|T>F5sbX*NJzCJ=BlzSgH}Y(1zaDz zCUQ3W@R$TYgkaFxl+>6;;gLo1%@YocPlAgxcBegKQfqPx?6l#ZlQw4aIIgTwRxTYe zhiJS?#>p-z$ujjg9jVzbKz#UxuP5GbHjbYQ1S3^ zs^L;EqNZ>%gAPzoz~-B+Sd_DK)*vuSB+IAnkUG0lXt2`V(oQB3w+b5(EyO-9kvbU6 zE0{C*#$nBGXZi51nZ#Sm$84?ISwt$pKK5GH5V3BmkQb_o?;##~>DRcK5Pgs9(43We zErm-5^Vr+G&0*82yozkWTL389XV)gMx*z7--%OPMw(e^Q3NaY-bSY6u5x(Wa zBo#YtvTo9{VdU_N%s`*jy2p}#y)3<8mQIPZcd^G7t7cxt4Z@J zd4SJD_lb@P@EbvgConLuy{Bh?Zx7trCvVSP0DUc6FyG=y%Ju&At9cr(t^v8Mo4>A1 zDyIO}^>-~KQYbds&)wR{t6xGVPMt>8Xn_o}T%#KP7P4p+C_!k9+i-zI>FZW*> zm>Tqt*U07swuQ7Yb=s9VAnqr#=Im)7yK z?&oh`l4i`7Z!b_RSj}6>;y@(@fC25+0{^6#l;-}AhpY2R@`B`%ln(sIYnx5eUd!OT zigV+T%gX!3Dm_6v0!}=6`CwwBmSFO|y(0(j`}gldLxq=zN3CG2tIHbvq#!yLxy>XU zq6?qsR^C}cl6mX}C?+)Vj|Y6Z`RwC0b!t*Lpu6eub!$}<^S;DqWoJ}?EE5kaeMnz* zr%K*D`wh_mQ7U1XhIMKjBIG1>E)tk|YMk?F5{JA1F9S2btS#vgQMZpB->#KUJ=U^1 z&uzX_sOFXPva`U5peW_e3TAIxZ$%ex(66I_?UZinW9JWfb6d6y?e%hZ7x_j`_Y3Fr z1vX{f4c&VLMJjgKOsgwM*3v+KOc@ zd{jbpZLX-JdXGwrynJiwB&cuYGG6k+1Zr|s!*+jT)le2Q3&3bcyB{f9yOB&hmEj>q zet*NHmNmWNa*69ezhc6fVHM^Z&Rozzfq^^D34%`9V36@2t|7k5$Y=^3B2bkTWM#9z zJj}`ZM#X6g5{u_imf4`JeC%Bw3Qn);KNdNHTxantpQbJ&Q}7|A5ooG7E}2jacJ~OM zjz>FW=Uzz_kDcWm>s&;gMocH!^2yM8!BVw_I#L}VZF*_$0F!~hW}vXgzXR%1dpun^ zJDNC~?$d!L-Ckg1D@3w)y1@gW>9>=C|jbPGTGJ%C#HR(DD{TU%bUlrTps#|y^#&nd}K0{359<(f8rplbOL-+e+Z zJShpL5Ds#4S?PR(f;ub<5JND}ahZ_f-o1OietzIi6or{3aHqmN4lM8Z>IzNo^5JId zT~1}kRc6UBQt<#Hv0GwTCdGssVCs|%$zGa(gq58`zbOUi+m~4BKxA@qf{x@G5d}L# zBPeYOR(!p>^X|a1K|%2)ChKPxt()I`z?M#sEtUzoUtonN*Ji4QnWeyB+~(h4_BuUK zZR=YwN8|CQ>Xw71D&0F{C~m~VlcKmV1o@VUbl2x5OcO8v&^KveI^P%G)`yU^OaHuB z9*^+Wxn3@gYnV|;+51u=soD!b2}n>-5YRSk_Vnl^zFSUw^?wzIw9?OHCy*l=bv1ouqC5wr7_>4& z!T6Co=+}MD{NpxI0nBH57bIIkyo!z+xEi3g*-ov~&rVOTr~>R!d7@(f z#2&s<&kLH=vl?3ynrsH#|CW9XlUVe3@A?vh&j~a6krs1uGV1|Op2I#l4lwzgcE)nU zpn~g(6L)U^OQ$t7Nsh)|$uZhyk$rshDIdWc|L1m47D%CJst{y-M*%gFsH!_X_NCk%QXh`JRg+DJ<*j z$kXN)j%$DU@7(8~lW!-_`@`(p;Ho-~#%i>qr7ndXrA{+eH+dAEPH4v02~JjA1tz7Z zScB$Vx!70;yhSIic)dc8!hz$2FQ=O4 zZCg5w_J&^K`TwCz8MLTBUK65|0B>G})$otAQMXzIg5P>%1}qr9FDh*GNPJ%qc7CP9 z!*4#={PC)_9*QG6@6oRb_e z5~B05Aq|oh{DO?t!s|^JLF|R>gw4l7hM6*1AIeeWppyvYn#X=PZW)Ejc(5{JDjtxY zs1W0*{R3&OKbp0dou8BXWNv&6E2)WX4&g{GB019?i(NiJwVBrwC9)@E-q+y{uR+w5 zhO2)`W8py>7fDz?yd<*GBJu25T52jToxa!ZAXr05bTXhBcb1$PlRVGtve0`&=3Dz4 zO5X^Pyh%c$9(p0|+-cPG@9RaDF?J3&@kudxV=a=N*s6tLRaSjgeQ&Lc7uB>SfT0nr zG&Xz-rHLxX_t<5&=ueM}v0Pm(neXf!$~G>Sd+y{cwH`g>=KSYtXXpTFzDj;=u({I3ybiQ#t%f4!#dyj%Jz??8^WoK zLy3*78jJi@3-HgHLDh%OLR3DF*g4n%zx?5Sj!Tz!-Ylmuq1WDjl8z5AzK!jEz001+ z%~wpcS4Kbfl3L(7`OQ_cMw=0)-+9M>pN&mn#0)&(;TCK(40 zM;~v1Wwsv1zUJ`KUob~~?uP#S4;#G`1ga(|oW8k7ENHl-d4F7ohZvePikF`HXYQ{k z&&-%5dOy;c^E9bQuugriPm@n6#?d9lfdsXIL#kk>VdwAI-|eh< zA$ONsCWGxc>MR8s(8SB;$<86yRANjLO|U;w>wB@HCFCmU~AI1olJ9$ zCVwyA-LE?17w8~7!l|hOqik#`52qFP5*jx@x=5@(-25E0PkT+Ku%d`#@%mNNdpDQj zwTso4fup9*aJ8NheKT5N!Xc`v&p)4~?%r%AYt#_M&ezj!@QFO~I&uWV z24GdNa-~0ivDtskX-U{qqY1PM=`wgvmQpsYFWA%;1~}~O>;$d_hlO#lvhsXW9A6pA zyNS&IT*JH?B z_-1p78#D}$dz-@uPJ43WD?B|0oJ$xt1fBVuL9Vvc%t3u2Rke?*I-cI+s3~XVbCbB^1th4WyUyyE;urb6w>6VT7h$^N5F){SZ{@I)n<&Yx z5)%+O`CjPm2 z+vw?^V9YeAb?t=)c<`^=?Z1h)=ke*9Op~s*-Z1&Ivroot8W<)<+Mf~RFz-vK$_|Ac zm(KSuO_`o-@1XYXdRskZ7BpY?Td76MRbL1bI+wXn55JsI#VUc0ED2k7VWsP-QyH{bF{eVjx3K>&GWY*6gqaSOK>}WiFu+r(#uvg4?JtCgrJ;{ zNQ(CM;*kdKGrgGV-m2)h^N;h?KbrdC)2&#qd_bMbwstycvG!8PD|d@s1kWW*qQpc? z?r(b>c!GnQ0^IHFfRn~xDhvj7!5T$Wm}vUFthlTLtY_em0}KAy3^h1q$b{k^=M2hT zK!F)9r{-4JdjSUt?YMcrYx3Z*ylg~p zSdc_#9vU*j>iUv~Eugjm=1~ssKOx?1T-h%1Bc`o5JxFWeAkSK~n#XuA?;AWidQUuM zGS?OOZD3`l$38|yRqxT(#_!BACOU~%VJ)vHL#%ajtQ~shs*BdP*}J)m9-y|oKVG5D zhX%@nj{|S=>38#6XIgG&MQeODF(R0?o5WsP%=q{@j;>m2zspst6HOtn)_u_%?0_nJ z>|_)a?z8PAbP`}!Yz-!N*jeZW*H&73y7$py23X#*%u=J8k;sb8Q_H?o8E{P3!Egvz zs=UD_0oJ0yoZMbBwA*|FEa6o;$>7_DgFX0P+CnI&Iyn6D@(wWDDi(r5LC=DkD&H6l zv2L!EG$6iQv4dE_9PrmH zx-+9FqoQZ!$Ryaye2HPe04B8|p`KDxPzbv3n9%_h^NdVPw5j{b#v&$m3eFN3%Lm_? z_iU7=WGGdZN|rA%t@pv?WTQ{K@f%1o-~vAePpmI-etv$5DP?@g4DDQAg|93F_QZDc zbbFti+a!3Q_vt;xpynVpT5<@TvJ4$6n->#>5ydX%1*LBuuD!ezi2%(jRclt9{;4WW z>g@uIPM>X`Bz_MmiGX>POi3N3V5EIZ6KQ^$K_nk^%G9ikV!tMt8L8@d)<{zdrMN_^ z%}3oV*clyNucGf+G+{SeLBGHa*ZQK>C;nDv-I|y4t$k&LGXoEo8m^(~EB^G}+j>>x z96`f6A3n85{xqm|d;qfvxJCA}?O^DvuBy69SE{`VPBUNP>Kk-TFwX2p4CC}Lgylah zULLF@dN>orQTG+*onWl*Y_^v%!{{>CB3sWM4Dx|lSrUGWX@y~o=r5A}n zRzXPt57G-ji$VP*mgboLga-5&7a}`lD z)_gpT0DReGmZ*T{PcaO^|Qx5d?wPNtYon-qDHxSO)mnx5Z09cw!O>x3oJ?SzMi_qD(f zhNB;hF#YcAyd}P~j8xLh)G=6vaRZnWW%BtNjfBCH?gXJ_eXtT%R#pa_y1+l-e?e)T znzMlOHw+$8Kt=|iercpNh;McaiR1QufA^xgl;Fxx7X6JIO*QL*2Hr)_&x9b235mX| z=I$FWD<-1#TO_%eeT_gT+^s<3)0Iov@OuRw8zE3zt4}vS_R*v!$O!bCDkW9uAVluH zlopBksP~*A?>--gv(8L8a4%5o#RE*=Cl-yZ>@%2al^Dz0$iK^eIhVw(>8O|N5qBgp z^~pRVWAUg$Zudo3YidZEW{k}O;#U{e@brgD3Wg6eN`ll_Fp*ViSK4y!Gq96Qv}f@( z0t7{WEeu-u2-lTnKWehjYcy;Mc?APZ^6I_NJx-Pla((mp1g_Y2UGm; zQ4gab41;bk{Q>UWvx!WhnrvOll!yc&#xw3UHTCSxrco@>c$MnRJU$s&Nh{dRUQ#>u zBSXwy<{s;D37b`Z`I#gu8nt!j89H8D!%1%mJFuQHe@zUO@x+Ywc%L^|OwhL8svc?; z6-8mJZ5SsUyD~ZY$~>cZXYgxVhk`zutU@VeaOD%q;r_0?)OsjzenWD@>sr^flL;LO zgS&!w85XH;JbVpZQh4|bRYjMP-YjUKC`t7!d|S}bssMg%@ptexNW8Wy`gBc{KDC_R zv2U~JR8cD^$Bw)_0v$J@<#LsjZ==pO))^xSUV`<17n&8W$3s~L9YV+6SmtKfeT%#! z#UW~Q)FbF<#o1U=RPjf&KCD2E1(}m^<8g!5`H_}l4XDE(#3Zzt`zGE?u7v=J1kcB@ zy>8k@?#5b8b%)#+ZR3#-JSg8o@c_tV<$I_lkkG0+RO*J>H@)dJ6Fa#dK75Xw+-nAt zY+jx;*y{q$D_@E4K6>;*G(!0PeM$GXV@M>D;FktBJ}GPF5^?rpWd2nBDM@VXtZ?m{ zSH&)8-Sl?#Ihp{WJJw=1cjU8nM+Yo!PEZg>Ql<=LgOuutSX?F%@Zb(JX-yKW+ga-C z+UJvVfPR3(Zeo7Hs+x!WcH7N3#mtD^QRq7w{4xCXUJidt{cxXlY@F;E`QZZe)+jw} zSLzGuA6%`C%|@hyiP53$J;OA_Q({(Q4Z^V0ak zCf?WHVTT{QayWhyI$zZ%L#+8-;Aom}36d@mg`{OnUnm*;f44%y0uKu2!04(;`<30Z`t_EVZk3fKrHzAh$JkCXA%zxFg$ zG4jjk*G0{KO7+Zu0M`*W_C>^N{ia%!rZa83=oh`X1cStNqt(9dtc0W#gU@o$zi6uE z^&(7j60NC5E>>M6zvjHTV3bwlSd?X)L?;)LwpFB7!;!a9xcgi`ud4W{?K9)YL&9Ux z*<*)i^D2R=-6RS9>WgceRi;mRF2N-v+@7gxs2@E#xSiOCN~x&{@u2+3=l|gs`>*oz3Q`VX>9;aon6Y9KFdvVut&SeGxKEyr?I)GVSZ?RfmpNF>zn;B!myeWh`EtQA zGS(-7{gGZii>>vzHti$cA8ve@;WtmFy}J%}pYJ7qE2ffSX?;BZ({6v`%81B!9^Bll zNSzFYr~Io`#q$+%^?J3=$~pbzPG&*_whKS`F0k=NuTqch@2-uavyzXkM|Fl-Xtncn zt#uJAweD`a6jMSH^}BisEhjJ3YAGpXURAETT~Jq3^Y!oBx`CxM^`|uVA&;@KR7-X`#+Su z1yoh*8a9feqGBPXf$J z+&K;(MtT!{%9?0)CnKA#+cIBKA=6t{n>p*~%XYzqL=@&-CrmEpJb8nQiwjc_i*a}Y zco`~CcL9r&ICAXRA5_6VrLMo3tnPxG7w96=GjEHYJ=@SUv;JN$&G;STh!uHDmovSu zgq}?)Dr#$De|P1^O_>Y-B>1ks=hiDb)_IJ1@M!0qM2kNL!h{@$Pg=>H+#4+=>(3Gx zU@*9T<7M3O0~c69BLtQMTn=)Sm8H%#P&3RlrQJ_`I9e*S0HrK4GLu!@DrT(k$9~ux0bdbTr|aEa0U`Et=U$L?)=EfL^i}Pr zm-1S_sly8S_gLc9Zp3(B)(%S;TEE<^sJ=bRZ5XWm9_B0?*VNP$pSLfB-`y)MalhZE zx_aV0;%DNHmT9fO7>j9M9266qmv;rGL7Ts>Cx76NAE}5Yf z>n`6I-ejKSV-MvcI*baUgAAk1X{f$(R{2!<&<1R;_I8vXc{~{r|MkTtOkAm|s&*t( ziD*WB`NFtqmDlr>0|y@AC;iwAv)@f`I!se@QtKt(o*%hN?{S)``QID^mw9_bI^(hn zhTTK`TDOYczUIs2&Xq}&I8w;%I{GS~ot9lmsPp}=&3yC?bPb|=L?0aP4eHG|%BNMK zWqVz3DkePBo%7hTbI)Q!Ye(V4b0HNQv%!)Cwn^R2x$Wa}aT%LLd3nb$o3a8tK|VrQ zLtC4h4Kq^)1_mIh!8FM0=g*&?@{zc8i4{{J`ZqU+k$ZUdJJBAqs&3L%*FDZQ65gVh z=km8iLu|5n0omA%UYaEvixgg!e$G9;Oqy=?6B%uWY?*u>1MZe%ln%QsU(4uoGV2Ig zF(Yl(4^j@V5X(Mc)V>z6qL(7?nlnCZSSpg!dxvLMb@nL7{#=Q*GpeN zPd({el}YAKUUZpTSu9CwJ^z-LDn-7K-Mw;zm;C4}D!Dw@`6Lm&mEJpgn$6k!1K=rw zKk@QA=$IT2BG&8)1OH&`b6pHGDM$*7h`3^;Ki68%dW@0s z^9hW!NDnmxhH{yjzBz$_vCMvFC1tL$1w9qL`SYcfQO}QSR7T4WpRrz12~6@EdQo<& zVj^K#=?y3Oo7(1-skM%!QidAuFCj7f8b%s#Qn&AZZMS1ZXsjs9sFIG#*4{o0v#!Xy z`hQP-o}$IvBbRD2VLE|g4XoNEH3XmZx0VEn0S z9BshE71o^Wx9&%}0_LbY=mS`Jm*-`w)wubCk0vBO)jHPuSM-tSh8=3zYPotXC1SVM zf&#f5RUKF5SBr~Xq;maIqX%>lH++7}`qVe@XX+ab)8yswm5VwTS$Ew2D0Ay)o2~HJ zd}?Lkcbc%Tywy&bc4k9OId^Y83|Kv^W0^QdwJ$*Jm#$#AXTfu|G{N>iVc*0SOn=24 zo4R^lOVG8|^HC(}o9ccf-E!KBPFK;GEJ{XZjqD4cH_6ZezWZ#WN4|${@wgOOCwfe>GWLt(5*D- zL{3g##XUtMW&;MZIu%a7Pfk?uRPr+|Ul!eIv-9>;ssP2?g9=m6f_89;ou9cDkQ3iB zKS^ELIXu%Bm2)@IPmeQy+U?sEzOCLZsa%SF?eH)wuYH0C{rHWJuXmd%m{XVq-dKjw zosnC}^6ohHD6)utrsKov-ZlH~I1P%MwC28R(}M$78}_I({$hB|%5j`mfg$IQUbWtV zTpROGkr4;U;tgjw>)@E%yxY1i?bj_?&MPjWv(B;XcRE6{%olS;m)^Cr@{Jb^ zKBi>IJz*5?ZBf~z%2Ot=+$vohV%9Xbs$G+hV0fVqo6&vEoQemFs8wrNzY8Wa zNn}4soUIk32CDyUN@izvZsyo%vq&Aka=veb3;#@7>Do-|RHv#?@5;To=O1~m)w*gu z%yBPTvY#7HwYV1S!M`$4eDxSTJL7lW20H(FhkJqj+CUHbK3{1ZeB0sJ@u77$>#&fz;E+r2%pb!@&#@lYwWc}$>z1}sjZdOC zOI~u8;+R-^Ur%}Fy76)5`6LZ{`vB46rSps1^0#p0mT)C)PMIs}P;3{~=AkD1YwgSA@5)O(&wWd|6SS`zI~O`#U!D1t))~GgFQM6* zyBZm_cFK(-<1gFt30k&aX1~v;eJ(Dx0qsZ$ zJ++zhMr^+fhB6Y@5(~9DQy%Np>$@eGCB?r%>U#Cp3upKN~-bqWjLuo93s-zIURJK+98MPvlp!!`E+wj?(_)m1Uq1) zplB%RmfoW;UT4W!hQ?oXt1wLIL^JSj5)~8Eli*{J!@%fJL^DQEDJdz#<>7YY-kj(2 z9BK7^{``h<<$cQ9lH!CUSn_&h<))FQR|%G_QJj>%Lv!tFM%JEE)ukh+Wll?4v+8`_ zBJ0|@|t}Xhn`(H=TNJ_X3yO;Q_HZ-?1+kF2gZ)=cBP%uL!FNH^f-s!*Ca)NWR3aTbE zr$U@=N2tcRuk;V;?O9fI9#EJ))cbw*aq5}r0Ue33j;1D=;Iw3-PyM!;M(FmD0c5*i&)QC&)ScMhn)7b z4tlu(MI*)RdJ+hX=5MwFEzxp?Q0MUah-OX zaHbk)El7)uj_j88pBH;=ad__H;9kkS<9K8Bh-d1kV6@O<-8OUE>}ij4f5tn$6|sWXI`>yMIy5HD7&6kH zH1i(Yw^wWL2Z^}2w@;D+6KgAKH7~Yj8C0t>4h+_)2#)8>vC#9Swj_xNyB^P>8CyYHip)LJ6QfE~+V=~pM}8}@a;1GAKC2YjvD-5m zUVUaaO^Em8i@VDUPfdGrmHccwQ|>EKcg+Y8gfN;;UY!LF=GjU=HeZtL*be52mqcnZ zZrnIloO`3~oSdyKPg-+xkm!6Dv*_AvR;_xrt%cI(Exuv`FBPpyDe zt0-nq)qYWYQN?gz8cxx)xqT&ba$zxLv2Wdp+>?*GJ@lrQYx2! zwV&E)OXIok6-n02u9aU~23|StJ-B-4V&`aw0`pKTWhLc|QC&q%#i=sEsFtWn@oevV zg%2AoI&_^Pr(E4eYjV>BZoJBv*ZN#qC3sFsV$Y36+AbA0D;x-!TU*o^)jrf8e6ZRu zVJ*kp#g%B))}5O@AAQNwup~U>@sbr!SO}RX(8Yp+0x+z72Nf!^QH+46IkY^YKBnuj zU_@o>;DCM(&*%z?Tb~H`)|SCpTuj;FQMmBEbcGaZ&ux=WjZd6EH6lfoQKz1hX&D>C ze~~#%D2!KB@0!X44%jNgq}t(z0h8YOaxcw-e+DF;+wvSf%GFmm)k$mWtNX~c{8O!D zr^gkE>6E@z=gG*(h}E|Z0;TG9*95#Mgy#60^d0fX8w(&4ZMJmDa{gkBl%`7c>`^W|i7Sy$vZm zNE_LzGNRHe<*t+)GMq9jG9sL0DPye7QO;3u$N#Q>cb;u&xX_KDGGxyo~*a&!2YzzBmsoczUtf3>JLEh9F3df^!J7twL8Pu3Kfdyn)SQ|{+XRncA&T=`Qq5E}PDF1*n& z!ebR*=Qh<8T9FazrzPNT72bXxsXImfmYMPXiP=ZFYR6K~_VH>Ta}Q2UhPZX_&n-m}FYZywk4IjyFhqe?Y@*O3>`m&fgwFPIXT0Z3(-^K@kEH3~g|8 zP4-rTZcO#swE$2C2L@h0_xatsZV+aKoiW}6s=+bCb+Cd*A|pq=S+h;iaACo#dW6S( zEV6XI=@(E11pOCV9+-Cjv@*VLU}f-V>m#;6R}nSVE)h%Xw~DS&?c}dgU*#@FAX{G1 z0!K(esrNJ0XDr_ZmvNLzMAj@3wnQtj&RDm5iid7uvrFuH7TB+ytP?z|yQsfJVV(0} zc~)UIHYup~uEH+SmzoLlZP+J(<1#BVm3)<~&;CIWSjqmJanfpV*}nT&Z<&qzJz}{> zh~?gPn<^tWS1(=v9ayVDG*o~xESO?ryNe{FckJHn_vA^$8RI8uX=xLHbKZ;@3fRRY z$inC6@toCZvM-)2@SZkrw^nHVTOf26`9sC)Ay!4tysAyQo)mdqi>ylz9T2H9g?$rp z&ic#c_>zLEX)s7U8mwMSaxXqAq6R1s%pkimz;@l*$kQL`$$Z<-pILu=DRgN9be=nL zZ1ON{@#ATxo72=br|{^FF0{lDk)oql(n9Sj*HX~k(THrPRmJ$FHrOWIFS5T_Jvp_I z47FR_b1zb5oMcO7KLia`hzU)-v=csR^Vp*;uk(BOtd4+Fy^~Cyie<}N=P_fNlhgn8 z0u;~f-@hLOKVUk;3-!hXr8p3NNV_~_@qb57<`sNXXV7nE%HJ`*_+8$wca8#9mygmO z4gV(QEX27arlO)!ai>;Bj%tK4C3)N0eBI~8)t(2LGFhKSm;{{R+C`>H7v=44?X7a} zZ+!T``XB%c^(h+n>3XT`;Y^F(zuKaMMN+0mnyo|MCYdrS3e+)`D!LerQKZZXF?(&T-_lQdRm9cMDpCU*6O4?gezyGy^_<1WF$ z!4d1P;NWC$afdCS(f{%KhIXZQ)ni&JfZ-I&Eb)dnxe8TRH0P9_cwS!UcBE0e+fCgi zIxDJlA0?O1|&G?&xs#qpbRB%PQYECcot*T&N<aeUZoX3_{&qQm19l92<#> z#HF@~!RcqUXFPQV3sfE@?Ydf{G61}zJyK(3>XZ-sYxQU0(!OB^s*E4%++$g77pA06 zTEMZtTKd_^J6bkYprMp?Z#8ml`H$;Zu>ie+7se@X$KwYVG2jq5#JtrS+=rV z@I!2o=7XS;IZ)U zV_6aHb?qI^@XfBAKJjzNof<~&Y4*>zA$LRRXmk`)xMq1rg0w4<-nj)a{l%zu$EdQ zd;h&s2x)#)y(!?T?40zHpCvB(SM;7)tr9l*0`C?W0{Y zhJ3~1*4)^}KDNYUFV1Y9%{o4j#wt?3#C9-_A}?V_^XwaMpyN@xN^)MkhCGo{u7 zCHbb<`e>ig$5W0UY&`ue96g)OEzSnj_mL#1GOEpuY$@Er_4mpjN-$}68&e66f7yiE z(z>x_nEoSzJnCjtRFt>`$cSnOpM`{g1xKTpJ zE(gk$uF<_71#593La}m#e5OBFLbqiz->@`DVXgCi1Dtw+YHH5EiM40{jdN?|sl2Hr z#t!MS>D@~x>?~PM=Z;AfPl*>&yl!n+t2yXfn6l)5(IbpSv^YqIYb7cxd#Wiaqf~6V z(4*J0SlP{rp@thZvMMirKv=uSgq`MwSFs{-ju9>1dP8qogt9#grhXQ(w&ELZfrDutRVkqzpGl^p)+TGevZ55xpiU8b8QLeC4V3D4;yqcfL_bFCqU|~ zMSLvn!-Qn5W6@HT=?0n3S%epgx(`JNFtSaKkDmj#ZGyRJYK23uZL=-mPC7qiNwq&s zd!{SRAwxx_rqy^dH~m48=cnvlM$E1kJ4*McD8>rAS{oVE|J&pY0B7=JW?}eJl}nsn zk>ja2?$7H%c-hIY#%mWHHD+VYpNFM{w+d!wWyBlH4kWcCA$Fo;@CB9*U%X52yq=rr zn2RpJfeuFjhQ4Dx$7q7+HMBJ6%<7X`miiOzD>U7CU3otw$A2k}NE?1hKg=*}^e1-L zXhD(r^7je0%(rv3JKj-C#AGI(9x>BgopD1n?|E!3bfsh>e*&PogQrIBVm`+uUC#U% z;tb~0uT{{e>VC-!?$+;BaAI^T&aB9c7L>A{O-xEjdLI6Kl{&tW$6bg`F_}jm|7i@$ z&9NQU#7@Itte7Ds5cJqN7sp&t|>m~{dTMGnwo+5A9cAi2;M2B*)SyA3L31=;@wV+Wu zsk0-&9glFgC$*4lq1K{xloiklZ*T0ls=lP#0(Es!WMlpP=ZwEIt@wiu6uFBvI6C!K zC!f1iTQ;V)qZ!D;(z3pBd)B^vf2C+js|ITq9rwtcy^0nm+>61c4)A-W~CfiHg z_iv)KRTdr%0GhtATXjD?@0Pe74{N)KCLSeUSWnHR(-Bvp4zMb%uExVK%Po4B>E21LCGz_g{MA9)K; ze5A(F0!`X!HwQ5L;gX&nyGj>;$_Ksf?N<}`-wUVv5Cj-xb=Ro7loS;Sf+IGNo1wI8 zsjshJCCI|UlBAjv0<{9~<@w+K$QewOU!J5VJzxbXIj_zvV}HAosNxkE6YQ9R=Ntx~ z`B1)FKE7#n+(4<3xFoVFKP0S>SD~sg;X@VEp?cSj9T{)m(geW8(9-|2F!1AVVc{cZ zluqcJUawk_{YXZ24NRu%_{h_zN^{?mlao}Kfc>iUZf)KGO}N7R&bbvu$ZvLrr!`g9X{ zbE53MmxtC_hP-x~Z7#Zd_ipAHZ8Y_PY8WWP`EYqP%kSX4g(gT4a~H_x28V~+<~0pi z`VH4ho1E$iLY5$iBY2F#s}4qULaPQ<6NKO>AcGe0&6_tE)gl_#UN)ny1d)NN+gqcX zDR4aYJSN!(XL)Sw--CHf_i0gnP38*DM z(eB84U31x=CDZWX$MUZi_r5G(ARdRw(K_x|YUlz>*3JhXvjM0JfiXScZEk4+ZTjuC z)fET=`GG+$+V@`mK)vj}&jcY6S(j>j5qTOt=r_^bJ7iygO$B9x6jcxgQ_;}GSnyNS z`4ABo8JSHI17jC0FbDK$EROVg6>w}?{c-5OMSi%*KE0do=C>ChwFKp&z|8$0j}Od} z_}Ta*er}t`z#rO+dr|a-H!!WGeC2#?v1teCL`J=<<9${A?VH3!KJMYfS9V>)6>%;+#v|z#{?yp&Qoqu{z&3GF17z zTDhT$z|H-wPKC_N7SG~E!4_)lqBEyYgY1r%4ZaNAeIOF&;^Z7eAD)@nAR5ELaw`#2 zROAMG0>V&d&}D-X*iV~3q%`U;+gnd%pZ81}nf_d_YwI_(wYB|o_ilB#kea%>$)F|~ z*&xtB=t2br1+itC)PTxafXdd^_8?mPz?oSL@^bdF)~2Rp-Qq>m*a;GB^b4#lHkLBv zTJbqC_(@27`5tc9<1IWNMc}w)$r>6O>_?A+xc6njKC+V`P-$2!U0X>@NXlzh?zp42 zXK{8Ui4`5|q28FpTApZ7RYm_T^Qr3}LG%te5`BZ39<7E~&W|2H3m0NP@dU0uHof`g};L-<){2rS2XMg9TMadb)yDDHjA-Bzo`#56%!Ael;eLYyd$ZgUZow zR7A=Jqx%833B4J%*FOh#u|_LyUPe;3%lN#Y;KJgf&liwrGAglu+fP(l$;cQGl8*g& zb=S_O|Hq+~C?u}o;X2sOW@d@qJ(H7mRNXJR>S&m_t*l@^)ly%O+7j}9<4<6QAvY(7 zE&zZ>c3$56Cq@l*w5|AjLGx{NNv%OB#L zLLZ3f2{gyE1x7@aK)c9j{H1Y3MrI}kUs+G;dp&&k5ahSmkPKkk|>-9jNjtBuC6Xzw36WW>gvV;d$#>x_=E}IFHv~Gz8QRpFc|-axOdT( zl#~!`z4CFDRaNzSzk>suk8k?JUa>1OGLthiem*`S0@e{pNlB+;MMH~NRESLU%!(C%Wm{SqxmC)Cr$ zjr>lUGx*t$ohWpwtcjj1BY%^M!74JcQRNxlS5QTm!66y!@4wBMXnMi}J{k=Q*Fi}A z=ETP<8y<47KFYEODjLX5=SDkA#SS07D6vlYUKtE?zY4_!>n;i-vP@7?(rv7Qq z4wL2tBbDGo4SHpXp7nO9_#H&?5Ye91mU%IV4-sk^&KO9F1iLd_c^f{HRy0`wZ;~X^ z@6k#)4j5)NcbC8azC-0}YQ`+&la+-z=;r2My+(s(#A0`Szvz`_1?38u2v2A#p((6T z|J^?X#dA_x$<7@+@>hb~AbExuN!9k3*y@CTOw17o=tV!TvwtS)=sNo);IQ4=%70r& zc+0%zmMvR`ak*au*|E5`EBBGkCJ^@bq&OUhzl%o0CHc=AmpENJ4wQyf^F8Pdu&3knUzM(aoHN=Hv0T5#eFXvxnB$!dn& zFy3h5%A4BP#dxFVO8~50DfAUC5smlWzt7rIr>Flr+I~sm2@5|d9<%wAQ2MBy&^bXb zh@@^GMn_prU6(lA`SC5-ZKt5Wf{gFJpeQ zO6G|_RV})d9PoT%YRY^RH8wtmD$sC3J}nr~0I_r&sR!t8{lP znG9dL2t^qcku;+E;l0s)e){yfy2&g~+1KN}ui5o!3CS5L$n% zaDbWx_6gqKBE|ILe)1VoQI7N4H|K}pLq{L zd;ia1LUVttzm7@}lZe?MWMgrppS(I+R*1m|Z}~VbZtl9hoSRnb5|kdC2@eZ%b9cYg zgHbHMg#A2znLhtBA932nzo`TvW5qit3{BsKdx5*fO+wb-!Be=XOED0ma`!WF-pC>n zksM))!rIc3hYcgAX12Byegc{4P|c$aAdH*X8Hl5Bamh+-CMWk(;MUwl+N{daxU_TGN5FMVEj*%PapV=}Y7J@v{_p+7MC=mOg=lHFk;t9>|qzG1=AGwG(p}_b5 zFPPxD6B}zkBuO&n^LH!A#(w{%W@KDK>ekxaJV1f;h7bkqJ%?zx`TU8${_286g}K`+ z6I1m|>&NavvU23W&=7<;I|uCx+S;^Nx{xfv9YKTV!@4!>gmwT35b_HO7^0Qma+5a9 z$r#ZxJbrF??Ojrs<3cRyEm^`NzU~jmE^|C@vA5sYC!b!fIRV|7}<1b&n>=j#1#~?`kQAnuX08Iom61X`zUEp2_ znbS|IKEcPhuMCf{3u^4#`LU{sWFowz(XbR57M8rlPE4#66PJxw#@K2xFb5x%kIQAt zM4SahM}bU*x8ka*wJD*>Dqm}&*E<5q$gZG;Y&{=OzdZplw6Rb*NHf(lS{Gd#q3H_V zFn2U&o0pM?KD&$2&Uz6gv2GN`kW-UPo;Xeugs_fN5UM!Ubysa19aU`)d68~!C;ukh z((ENX&B2I+jEqT4_7IvZzwb3}laUJp*|WB4wob&%^DKl+%x&JLS0A-iyaYm3H8d8! z3ypbumzJ)H%(X?QioUhM)n_N+>N5=gyLuTpFO13;uE*Z!U48*)=OJ+WB9bXMWQc`^ zN=7eWBlB?AX>Z@BT0_LHUf%H?6do`}41RrZ;fu>t)dG-&Fs+S&8q1b#+k7s|$Pg+d zkXZ|ljO1m*lZ42aFoKtC^oH9ILP>{{q+~bt={5Q;qI7oSYhYp`XYc2K{`m(&c#Wr= zF<#F98Cgxo=D25ikt6_Q*5F z@`rholXEEsjVyU|3M*b+v>Sc_fewWMnIWUGNd~Impez@!4>r@Hx0Yj%* zZg+wKnCz`K;&faJbPrpJEFx;l?UlA8~!a_24;*@G@%=4&{%e^@H zw?uNX)s+YYlS)@b9Auv(ExmS)DT>tB{o{Q-YghaFYqwMwLO=w8$w#8-NG5@YNw&X6 z*nWV`(J}DZkxQ5UTjf{RHmxFJf{w^L*k)iRR_8<^Re+oWf}*d1UfA|-5>VGdM*8~a z|FSv$cK7Nmc6Cxv_yIMKIUp5(`uZ43l-1SqaHR+cB_$-7&^BLlX){~$Ya$B$kqICS z@)AtwO@m0%WVD~3Uw0DlGnvnYP)OTEz6}rpU#5)scEF=#WSVD#Hm&CWiQwYkpb9t^ zahpw0&>3Qn57MCfCk%ooBz$a&s;aZ^DS7!y6i6o`iXGYF|1oOjb@};;>*id>)jLx_ zy^mE9$4SHl!?U}PoTwAIg|MdL9jubF_V$xE3^yPyp6CBx(AnlhgLQ}RMJJ11hpa5> z09YqvxCO>Hjf~JZ>;u_@Yqp1?7}jm!`n&#L@iH&7jfcRE+yS5%`n1_jboq8(zx{qz>kaoko2lpoYNIK@XW7UEkqaS!E&%+92zO#` zE;l<{4zDX$aAY5!CMK=Zif2P6c1p_9ALNjFf^mZBvH!m6tm45gh%P|OKo~(aYKNxT zEf5#RE956glctb5!)cvOz1+1oARyC)pM*-7*!~}bW?NRpNsg80X@$_Z3*tpW00bUP zS{kS@^oaqM5)(}DT{R9vN6(%Od~sYY&FAUU_3k<#OR*X2c{YKnb?WXhAsk$^6uYSI z;>;P6gR?1&*SoOy#ZD3?_Ey}lfLu>i)e*6qW^1DgH62|8j`=5T${)UupH+(||__sb%i?)MP)i7%nk_a>l!?CrmTWc#1Ck!aH`Ik zjo7|0KA$`?GI9-5`<$PVbHiW70B?a-_wQ#&_xS5KWoKj;*L)srIz@8bCLAzXcq0uR zozIsT>8FI6gp)l*$OYIg5zhZ%yY7duPeWKvWlx985Kpbpm48V2%v(Ned)U=WF=$bM z!^hx@$v)El43s*S3KMx#X0Ab*wdND>fREdWjw+IJnG{jw0fB z0a0*rv5R(Wq#2g4j+POTO8>_-zyp7gNlW~#jEI7bK>E$w$nHNPIYDvX*GJ1h>HtwM zQOI8*V1z1AYl(YdP37i6QWE>)b$qkriK9nv0yc$$j_aRrr(eH+!?ynF?XAZtgLFOxPEArT%^fw}Jr|1&Ubz_JN;vBO&L`hirhh z*Uf@|kUMP_652uev_eMEqm6Vc;$Uv>(!-dQkJdB>pBaknHI&Lq!<( z@@-p>yTvQt?RVib+_1b9n^>yU=SHtv^2Eub46qSlA650FU&J%FY%freXlrjr;%;Ew6*hTVXs1^cYvZUIveY?WuB9EAE*qH6( zCOZQH?3VLBmXi|o7lQmtWw|zR5%}gxFuhGovI0PhKz}iS zz;}>pW3OCI$@ZDX1fPUi_zfex=OgFfQ!j!dZ@xLr0A6qo{?~jR*U}F03p&aCw z?Z-G*5IM%(qUj8;qc0zd^7d5|2-!*(2a^!>$hhpKq%8as;R;3(D3lXHAD$X=Fl6MP z%F8(f25p6!o25P0Ue>@&&#vA(~kV8l1>biu4^57HT z*+tW}vK5xIzsUlq@r9v$4hR zgpjc^Ff0NJK^AZuAOz5n?2L?4E*iO=hz2BLuuJ%trY>=Ax_0E`$@GR>80mX<7yjem zsWL(l1Tbe1=$zALf9mg~sN@Xs{HI`zJOfq$rh=_@GBBAsy zZ~Mx`bc9b?KH?emu^6ZSusXSm$g`mU?>3)~6e!R3HvuC;6c7WJ4$wbCz>mZ#r#09d0&m(-2l-Qx9;GisEg9nG;R&p?*iE*NuDkw|)`QP~u%5Gt!m~t* zl=`yc(h*)>Ydl2&i6}?v9j+i}NR;wtbRXWONliooUJDA&6i|Sgz$h)4i?%bdY;MDhl#ZrHkXrJ+gY&Z2}Q23G%v!#D&h`t<43 zAo0;Ze{z8-5ZO6FJ%q(YUZ5+1H3!qdf+XF}I^T_*b_&iF8f+EsewF*evsj-^!A^s6 zp=7p_*Beq^rvC}y2~Ii(1)O@4>l=^vh&EKTx971_QC&CXICgctn~BWppdxh_<<6bY z$dNr27!MwUd}LDIJ{wsey_zXu!ltAMwXCTQ*;2zOQPgt1Xo2;tJOyFsD8=LLq<0L2AU#_ zivcw&A%d!+61%LfppXs);`2v=F2C)YN4Wr+gbBX}#hJsxVhp7O-8JPSifjee%X8oE zh6IhE(+R_q*{NX^SDugx2Hyg4u>ZH5_wk9-iY1Fa>i|Mo8!}>mDtMvk;^fYaO)Sgz z@FDQUeUy}fwii2Dcy-W}*TN~0_ zgOD#wN1cDsYzs<9H<1j=X&m@*d{NY~w|66%ef(;bEb{5o!)z#7aUa|9YQe@X@ZWu; zAmk*g>dbeO|I^}Do(f{AzQ-TSr6uld#A}jCVr6?7_`WnZ=cA0dUf6nib^13Lx1RX; zm~|P^LEc>d-R2FQvVcWLerZR2K z8G(fk`VE8x@c=~GEkqOh)9`T1XZttRyUZ>}`>zuM@R<*iyJ=`Znz<5J3t@I9YHB5I?c~SLpFJb= z-hu5Pk%N75bau`UklAYW(${y(R%wlwVdsrV79h$N-PI|oLS}V@MrZ<>{tiw|PY>g` zLH!FhC@27;(*V^8c~cbUK5@|HIglU-eHJt-ICVH1qtM6p3k^@`$cLyWilI#B?M;wa zpdSaddfwvcKw$aa7HxSGi01@uel1J+i*)q&0tQx1(#gE>$h)bfWwa^75M4Z?qM{F| zfwTs5Yh;=?p1^=SmGf^u0T_|TiYIK6NNEU8DB1s!b%?iZ z+$TBibM?~_!kD3mgh++4Vwb1^5+eCx6aS`Fr0xz54zvOA5#^shcUt`4=3Gh#_urB9 zc}(Lc`GlFY>|Qso{5SlOLll>Z-Yqbe0oHv$4Ib)_+rRw)0HNgOO_)nd3ky8CsDDcQ znTu(Nnj@c{>Y#k+!di3TRC?mzNB`fEm7KoGue~YFQcYy8KYfyag(E0dBl-(|b*KfRK%`~D{kN&i9NN96v)z*|a@>xVpLV2Hr_`f5ot9ed8zRlQm>vpZU^u7FJ zGddSGW$@ZLqFSVLN(2=IxCFGY45=N-!GZ4XLmZgC$EPtIgzls%gi~@O zK~wzx?la&HN{UFLP-#E1Fh76p{CUko4EiEfp_Tdy+F_HE zJ<4^ZF2q%)Z>k(C)B6DPzfO}BIlD)w=XdmZhi5^tD@&Gy4kSeUu4ixF+a!2%9o;=% ze|-En@9}jELxz8+a0TT~0txj}^xNz|`D@T&9o&qndsP-)G(Mv;(o$eAO|hqm^hnwS z@qVg}?n&xm{xg(rs;UhD`somndaijy z*O<&qk<(4hsHC8XB~7unW#bRLVxyhM+TBE;PSjMoBTz$wdNBsh?&4p2cI+_4dB%hH zW!j(4uyG-=+}z_h^r-xJPJTUR{O!dx^%_(d3B_XM;k0{NOFcbdEUrcfo8FJ{;#XL& zfIm~Dqnn3BEssvYbO=F}=jZGDv<_6t*b-flYs+x}GGh>T)0|@0=b>3|eT;p^z7;~K zy)f}ll@k2^k4rY-;o9=&NC|KRnZT_t9d3_F{ro9`_#*aJf0`FHO zAfG?S+5AD+euWWg^_*=3gM))TJx4e|YI6D1rlD8smhreBH^(^M8;J4@;-8&P9;p8Z z;>=Lf&?x6kp`-@N#=u0}dtHoI(8hC=#Py&7q4I6U%_@eUlIZe7a5VErdL|GM!;O}c z4jyGJZ%q!}{kza9>j~jKylOuNZbqJIHaL!=6mrJPSFX5$`9?`ei5(@^1e+!znfGHn z_O-q~8>zYK&RsM?k&$K%%91E*XKp+~%V3(@$BxBeE6Ac-+yzq%geT(BJi>wYD>Sb1 z5zRBZ8Tc+=zWnbr@MYyx?PiykcV3&-T)0k0)N{QQ%L6y_CZL*xz`DrAzp!u#R!dH{ zLAnLI0MRWwzeUq{NPt(>#CoZ~OY!s9%n!S^eE_)*fwKdU#WY|k)*P-z5uLX6hG;-z zb)T-mu0h(Oewb6Ae`Di3xxcle=MDwS?XEi%sY}-`T=*w4Qtz7l9vW7qJN|wu9wki7 zmf`C~cB{MhmHMq8&F*pS8%yibZ&;5m6hhSF${4}mQdbAUOjOe}d+&+c8j^0KHa33i zD#|p2V2ee08D9+4SI@bZR@&6mlow=A_}_KEd*T^Uq&3#z<{d}-DgZoeQtcAA1?1~# ztH+Vzz~&(E7_N=2MEvT5117q0qRZcJMyF{nnoCd^ubDO^npi{l)57ZBz0*H=Su@|x z2%Vb!i%!{AVk+QBt}oE7u|ZK63xTgHInn!s*Q-Dj8SO)9b|U2f``c0_YV_ zJtdGf{k2Sxa@NtI^31 zo7NstKqD~n+S*K=O3zMgESNX@)>Q;!Z~SmHJKb&|J(pvAC{xYXiw8vT6hY+FUu0Gt zC?5hCTQo1FqE+MQLr{GmTko#0M;^`|Onpg;5I7@K@KIOnLQckhaMuG)mBco;7|e^~ z|HlBC*RpxF|0W*;j9t|p=JVQ94j+&j7TOQ&r=l|8W3UX`UH=@Tcc07s$QyajbWJG;yplj*rRYk-2cZ_neRt!f{R$kivRu%q37 z4rwc*kw#_+Ykx+u?iW~xH8A@~FoxnM*ZrmuF}|du%2QgK|19?F*RLzQlw~~k?+?^* zlz)u(aMzL|qC77~F2y^~d;0sEPzS(5YOwq5&NizzL%@Vb0`nD#oZRn0RhZzd4-a<3 zZ6mWK7?6=&`|<~&M6y*%5W$X!^g%khH@%g#)X}=d7Nju7>T#S$724a(3MN>93{p{D zo8Ps@Z`P1}keQi>0|C6Su<)TnuOyUSA4n&88pS)Ct^3fdK}&1Y5^>(a47(8iQV4x_ zs9EctnIz_SNAv$28EMWscH{`rK8p3^9mq>(QM%KaExUDV4xw>OMuzNxW)blKHhz1} zA>imaYf_nOGija(oUewSm8%6nL(i@L7l*KLkxUS0<74e~{X1w%g=qtlYEp2>zcxvR zRwe1py)DdwuU%#=HjI5);N~^pJCc%;9W{s01Pzy!-KEL?RsSTAo^1R-^vRB5mss)n z>IUAu`(bIU)DtclLl(X0nqxn>zx)vQl^r{L9+CWay51$7kEz2u_7X2KA`aAx=obBZxLZJJ`beB!Pb`JYkYce`pFdaS1%6C44V!eCvMn=9ETYGs_g5) z0}9 zcG1hri)cYX{ug+&I-=OBz2JJmJ_Y+X4#Tyz)1O)YTN#C-w16`y{U=P(Em-byD9W4U z49dRKTHnDAEFmnEkoyRe!h2GvI;=fGcl++UuF=sG35deiKk`3+Gmi^Hm$ytJ_HKsm z_T#UU8rMBAnUM11t*dD2Kv(Q(OQkM@KZ(lM6#lKPweRsxi6$<(Y({;OA)n|J1k_Zt z@B-bG!2efaqXQAnJEBQRBXH00zYPq|yKA00m8jL&aBclwdEx1R6FhqSxM*eK+O=zF z$EMTXeq7qXKr>(Q)G3!ij*W9wdc4E9Yp5<^3fYF}T-&X37uqsTbrJw0lGU?knJcZR z7S$N7J(Rn4;gIKcEqlqmXvpw=6!5=1+G_l{P3tdJ|IJOR`ca9hD%W2>VX4<~YCgZ- z{JX3%A^(a(hMp8arfbpM?%KH?2apvy3~@|jWK=uj`XwsfpYRrNRrC`_#BHP3Lksk# zoft`E1T}8_V!|LV=@`ZsXonO;!!Dm-gs1;_`T)6T0RbE`! zyS#%GKBQkQ*X@dQ8BBnt0d`+P9`6?QO9poY7&h{uRK+49kT*i0L`aJxuJM_<;FYbM z^rlQa_RZkD?ZzWdYFj)w&nW0RJN)?I84nm`Sd9*oZwOCOz&J8NBO(*fSCxI!@A`G+ z!kmrkSW{Q;0%cDdEGBA7Z4LZkVdz5RDcq+p`TqU4ttDCl8(+SbVY71urw}P*efTr zq9q`GKdA~{c)4?JMGrwOSh9%T@1d>Rd~{|8ckWV_@v#~^NN|Oc9p6FMaYw-Ntyz7d zjWI5m@FBT2{+KMXw|^Uhz#KhKG#p12GD8HOZyd@9A`S`+JR%4}L#xuz7jJ%`z&AY@ zKuoqA*o9IX(&gidSE6o^LT)lwL+LwnlaqnUfYO75Rr7u-cOfNmnz?-SYBm0>^dgWQ z-@yIH4c=@ny%m>oi@2<^HT)6twXu;VbmnMvf5-b&;uX5l_egZ)y4c9eKZyZ7xMl>^ zaogne8hBX;K0%mDV`F1!L^;w|uL?wddH0H-!zQTJ-xGtDf=;7hF`#o(Kg6w|xUmk! z2sm9`U5PfiK)~uXs5PPJ`eK&A*=wJ1D0wVTNJ)dmo|sY?h(=1Ig+;s$@cP%O^@P&# z12PZ}PR>Lub0Df7=(5spA>e3?Z-DMS$Gj0k3Tfc@hJgqm?C!t+?lWiKCbMm05xJ1U zmX(zeZEUdRw4BPv+t9$=j;!`&4zek^v?oa_Neu`}eOYLZ8<0X<$MUbg4&%qO@O`v2 zG+|<%B{qMwtc&YWL?KdN7%V(6b1-)Troh}(3Ki|~YyW*TaC78~7wU-jrEXZ>(*3U& zKuSs)y)7^*YMw+q52K)Uj9FiZ-z?tF2*4EeF&ZX|AwFVlDnw-8{HP8UNR#wqfHOd1 zURqDSX*SyV)}zpMKw|jr{c56JQ2ym<_9I7PP=-JpgmQ3}6?YlSNnZjc_gYDhDN{?) ztpEB|cy1N@o8_F1J}wzrn9YbF86m&QC@FEAGwhKV+zbLTFj zfVv0*c+`#*5E!8ii%G<#u0?PYBPl~lXQh*Wm8P6f3$UJOotvMZM>PVL(@q!l`BC@cwo{MU%tE$^Ynm6 z#S}!c!=b1nr$|pEJYDk+Zym@}aWBIdpF&#|xb5GBqZv)Lbf3cJ4gYrUsl(Oh;o-%I z%0SVWqMq@OQ{}5&Cu!%yg)GB~#y|0Vz zFc3-bP6qGZ?cA=Qh&`(k{UGHhv9!lO`zUw)tgQthL_XSZ5oH(|8|$rH1Mq?mF4QSq z-MVcX9nT?sL$oUHW~8Ae#o_w0FhsCZ1Xc_<8TVGu*FB4Vr$R6kAVEzARbtTo&1dQ8 zZk`y|DI~!}UUuIQIHB4mB`K*}9EZ>V?Y4Dry{h2O% z$|Blt`Jc5?zOrlU`gW{3*L0ptVdskos&3M$J}YsD!pd*s5}3BKXdOD7cC4TAT3BJsYoLsDIg#r2qGmV4T5xc zqllD%Ac%BHH%O-fQqtYsUD9yZ*7N)08{@lUT*oaP#!aV5MgR+~9G!(jWro&$n6T1=ySNnnZ>+rX+a`7YN*dRAv^a41p!t3*4C%~}9hwjRLyicHq~1ca;u%24TUG1%z$hrSoOA^js1k}g!b0iu zcOh?msgbaR>=N1^wg+>DVM~*dkPJgu0D&#|a{$p6C4%~n0A+yYgQr25BdGsv6WEqk zBZc=kIQG^AA!vqQynOI%=^!jD6#Hk1dGy){U6USHud|+CX0p|HsQC)ouI#$l%f3^Gf zbe-0QB11z72;jl~i;Ea||KDLJHZ&iEkEHtxOA{FAy{&gYv4tWUOl&YdeF(%8Fi)zo zYcsm)rIw5Bkr2hDHl}Xq{e!~@-srsz4GbIuLd7>wrvCc%D{ywe;t_eSt*?^-yBydh zQUc)21>MS&i?99h`3Jnef2BB6Qc?nzSxgExFF=51;CZh2)1d|hT1VJ8IFW=D6!B16 zYD)6f`VTJE1KR}~o}QLg3M$RTets>%AAn)-5qc98fg%kw)CVCqM(+K$JU?N1z)f>o ze7BrDUAhmLlG)fJ`;||qB-b{+{7CAN2bAZ3YlR0_URW@YLi0(6e3F-H73fL;*RFz^ zHewqX{r=0wl@E`5@Q4nYv%`Ch9|JcW;_0smaMi2+cyN%|hT#!&x3+?V*aSFOg{~6zP zcyi(CGSP^-I@mM_a)YUaFS9Zds!scXi3%|XWY96~J3xKIM)M*1Nt^+Kq>$^cgLzp6 zf9URB^9`aeFF@p-?89S)<*hS-lQot8f-JSyIRm{8TnQK|*y<(vx6dO@D5|ME<;`;` zU7G&oqY(}eXMuSNg_<1nHUlmRxHbHOfa}wDfF(h)1hPHCp)L{~_aZ>wR5~<%Lp)!H zV7-Bpic*46-4I7~f6!Y4MjPOIKi)zBD*No2H3Z78G@ADx$OxY&{m?dBqzWzS8jXV-kQj+(VWyF9ErJQL`0p|i#*8aT&+!~aM!Vyi+YT+b^pYyhDgUSsVnc0{zQN4&HkY-kk`Ona0>jgUel7;h18ULusFOacN22+N7bY`PUkh z0{>o^8_X{>AeoGf)OnnBN{4w8D_w~(sPNK1l_!)U( z_!$@!0INj{U@R9S!2H@?7YC0G9GgkRx1NT!5kl4FVMdEA|Q=BKQ6*)DRK8AONs{S@5jy(>*b0#|Jt- zg6atQwy=P}`0()2YDQvNI~&qSz|r*Z51+lAPv87+7#fc9--K&>7dM9=H9QBU4}Ej< z4FJ5nK>7p>5-?CQ5sG%-klo!vXWzo(0Vsp#MIL(l$KF(j*>{v4N})p8(VT|t#(M_wk4@Iq5La#o z`cMrt=9fd~RjTA)wj_WNkeaZZXM%n8I6VNHbOM6rnB!+!PLrj;k%wdnArX+7h3r=B zR=(67RZ@~J7xir@Z!j;mxR}fKTi$2HLuU|!&8*W3MGG!5ZAGx&AVC1wZ$`$ljwBC) zv-+9tX$`rq-v?|3&pI?bmD%=>{TXJ5rXH7-Xt+wTJUlpv7%M5a@ErGC$HmokC{tm+ zYLkK?hnQ^$PnhBE6RUa;L!)3)*Lj>f!K57GfaGHR{R!-%H*e9~V{;$ikkLtP*W9N{ z6!zfKe{VW8kgs*Ra4S|Ekq5~SO}JuSfX9G@{BGOuuoKi0RJSWVpk@Xx;QIq=-STwV z!QFXG9ObLaVb6u7bYPrql#mK($_W8QKx5(WMLtYaA<9u60Dp#q6*gqr@pZFI%*dce zr8JZgeU~m124`(?#W@~XPB@Rx?z=-yN(v#7^3PSc1EdIUnPl__g!sw7We!)89XIoZ z8+3beU)fssRBx~C%!dE_{=X{^DULxOU~|+cF%seh%Eut^1|l@GA*}Pw*YzV<1RPg_ z*p!mrrKDQ%d*{zMKh`cn#?RIC_MOw-s%$Ov^AEV2}z7`Qq1NL9zFXEL&CX?2pOT zgnlrhAeHo>`jGH6_6)*IaG*g52po8c;Sh43cJfYxI0+7w5W)x6!@uO>6BOS6Ovm_8 zAt50!w7wZ-M>R)B#}ChJL+x;lR=D0wIQZj0jf-i&#Oy34o@y}ZP%!r2k0`8Z0H64m zw9IRHzAg!cZNSUH_Q?W%0Y%g2z+a@MxxUfy(>FXKEE)g)*73;xGBkj@9@RN|y-6py zRq9+s%=YEmr<=Zh>&botWqHTth#lq~;QJRS26I0NG3aVbL&Fmik|;`l^ zL;4Ou%aAno$k6N1)uNVMuQ7fHE?sXlYn zaMbLdot@i&X?_EE0P=e^a(ssJ^NYdS>ZPv1fLr{w;&#i~ZcfU%nI30^6nP;Oi^~}y z6hstxUvFy|A!sl1zyI$6w|pyCi|4LBpi{wQ+~!|_#vYmWB9_a&Lj5~Hjjl&S;k42( zN*z=X-@LgY-01(OdW(=eg0c^b6#5Jslajn2?plvXIDA7qVdH+fJoU8xj8fw;Viq8w ziHdc;9xMG^V%KIE`}mbnm+>#`p>f3e<#vNR1u?ERV=hV|_eV4|3v~DYb&%7XoSkWW z!E1P-l%Kmb3EvqLlaS0z`*bTK5fb;Wx1UOO^uOF3&#^NSJoxm^Cy4iVnFT4IqC~3t z_nwk&;_Xtny0>j>IO74(5erOS10{u*s5t8VIc#ENrbSiVw7;+avk@+_|G( zDvg4kt->vay|inUL~uCJc(~lk zu+d(`rq8$J?U5lqh#l}bVB)#}#TP-eftf)--axRzOd#!pelv)MtZEOxtWIu26t52| zDga1$qXr%x?rqnkHi?&-rX>gt7N;e6s*jEa`ZFX3gA43gEj#8jh+gd*ZrZ8 zD}+PD0ERDEBJ~?c&me@6pcJZtf!Uy*?^z42iZMc3c!D&%<;(?91xiw(!ucrZmh0w{ zA;v=w_#PQA$?VVV_xA4X^n}x%UY^aJBYL*IZGY_nbp@O~L~sbk$t}6Z3Xq5cR~o9w zXz1t&Q4lzm{2;JXpk13wMEJzUxKCWn&lI$`gbNPW4}XzWomm$HMI(a-z>@S@vmJRIqc|4NgxhcI~}A3 z&ucEqx)O&`@K$zPtRB&EDP5c~^QwyZebMDWeALI329%1|U~8uy^YNQ5{nq&CzD#j) zY}`qjr=>yN25<2$4!VXn6M_Sx8k%69x9ZM_@1tCUTc?56Nn2a#yNHli{tdt+6VX=n zgQtgkhl3+U<=mI0!vjfgFOS8j+~t4&y%0w*TnKkzfq_~y{8y-Insm8?|Gyxq^;x{2 z#>dgWpVrwngp3qHXScMnqbEfwsQ&ub#Ap!-8+2FzA-h_8sZ-vQv#+Nh>B&;2Kr*Fcnx>8z;5bbhGtPzzO#qXoncQ?<$&A&~_L z;~Js;Kp(x~7n90B5S!_`3Wacf+h_R45Q_c^o2qrzQsZQo{fGq1b$@uTHw6Ieg(Tc& zrX%bWLb-{aecs4>2TC#@A(UHeU+WcPR%K%A9>PZi4CBcSjoG$#hiBUewj6GRJ}z$6 zm1k#sj@|Ni=OVBwiyi94;?Vb}!h%DYJMC5OAPzs(K+LdHm>qCyFgMzCuy3XJbRJJ5 zLu7SnDSd0Q)FxFn0qjGzhAIYz>$V^ap(bYGGW!z#*@6O5$6^p6bXFVm!OPYbjOR}a zxTRcQH&)vJ4z>OFcZUZNmV-I58AIC>S;#00i)mkPH=M`_d zl={=9Wd_`~R}q{Mt22%Qk5+|pWAn$=eg$S=MoRXox#5RE1v z2KdW3h%i&N=mJwJWRLVSxGo0}h(UjDuVJFk#EE!}kLaj8k_r<+eD))u6g zH5boaU?a@89JVtvF%)`8*2ZMbkmZEMO@hv)Jpj&hK%}KLaDS$+C96=flbl;OPyJuoy7~$cQ#h}((TVmC$Dm8 zHZFNjq2B4~`Z)i2tkritpYwCBHy^RlAWlX{@B9i#XCNOhJj3=on2)8K>aX(NqQI$k zcQ;_y9KG6YIX$X(m}$uCo|#GVqZ*3R0R+AFsxF3CRXWh+b?c`&tPtBYsar4NvyYwg zwDVe@Ijq^Qi`m+8nSZ5u@%65kmlTBH9;XcSlKA+|u?RxfVsL++AJWs|6kql46^=%( zRi{V1!WI)5p~Md=@?M^tnDpF*CVy~-=S5#wm3=XXaDr^RdKKdNCso{{xIC(GsZ^f% zqhVJRb23SHd@(t(&C@CHqhv})SY-yM3KYEYEH#DaQQ1av$_1=e+stPr#auQiLT54o z0lWe=uQ)s7pf=3qvR6vAxVZZJ&N@2U<;KRVeVyr#cf8d1lckA*Z{L#=^JC&2=0SKD z`9H8#U&;w(vDL}D*SzVhDz@8y6HJ+Bl&D;48Y4U%-VbeANq1qxhYG7md?@{SZTL2i z3<=v>06ih4=cc5XAK1#x@r+74i?H7kb92Weu*ONZ6=w9jC|#d%8fCwvSI*B3Br4bt zc2x=sQ!iZ`vRW}6luk$(t5Uu=%`#O!+{{VzteWTbLFO$ri-I6IC@3YhU_UD;2$2;( zd>s*|`Loyj^@S^x+947I@dDGQwEJu| zNloG`YDr}lkXFP>CbLzL7}%tgSt`qRMaEN7!ubNUR?Lsf zG$LDdc(xG^*43$M8@lA^{;1Jd)nR+d7Vq9xwXNe2Y&=0XX{P94_v7S0+fmikG!Ri1 zmsn8TKqR>-4XFp2icjZT%>KGhpimckjj+Qq@K_8K&N^sw=#=MAxk$Eznr*7%+)i3f zWc$&?fWS~pBWPYEJYXw`4d%*|O-%F(a-yK4F&Mvd8b$`sV>HT>E(+ zi4Neg;GS3eE#ff{pF78G%Wxrd`i#memqlY?lf^Hm?sJTQ>gecci?IFCq^7&TV>v?FSI=VT}SNBGPCaRIu8 z`oBqejm@uQHar>3%~~5&uC03K(@Dl{#>6fU*t{neGvq0ZjF2p8Rke1@S&V}ngTZU2 zL*E@B6!R|jh3C)4bs!fs602F08KuL+rg^kcqW*p5gJ1n+6^})DP7b8O({-*Z_xSjU zU#myh?(RYejpp;F>%9NyDev5RPSI=dbfo-Bc)|}5ae@?G(Qy)$yS)>KNPx~lvy8-`e$oUlB%suiqlBC@8W2^jlR@88hjMEl)Irg z!DJaFM%-?HEH3n5H>!DG3*~64CK8VE6~KS66z+`4>QfKlKBUuyS7o`d4%z zv9aNh+HZCNMM^L`Rnp<~JHf32b8=zJ%Wj>@s%&*`pid>;9n7-Zlz8ZJ;jp-9(t4BI z(tK)vt6IHUCDAh*8O8qUqS-;qZNCvO@MFX`w{7L=Iv!jI8{3(D!ko+^`9Nk2@Q zKOsmm8b(FJ%qm}(ACh5%uVAauo$@;UPR!ERLbX04vY#-N$88a#UiSTk>#m{ii#cp{c(^t>_wX`Sqe=+`*t+HXa4%Nx zMD(U|+ob&bd9YrvIOk|D@)j6L%}vh~BboS}<)@~~x~?NVc?G<-ZbW?udc+Z;P8RWL z3)bkAi!vFqV6;m;Yk_H%)faiOx+*O<{7uT%uD{q&y+&=kY*qLwxnbTE85!vPjEuyT zQ-#TKf#+UT>5K;=BA8g!e^Q@E!P2`>UbcM>3?0sT2`_D*+gsU5F6;NeSZnE|e#lk@ zdFhY1}NmIV`2|<3o|oG zf2JSj2Z){_2pVFH;^Kf$;JTV-cGMl4;WQoj4U_1*UB;WvzAr3*nk=CFOh83jE1x!} znEN#`IhYwn%3ak3InUR`{429sVK43FZf4dkx%p{lvHtYQIotm9)sctXmhOhs9fML} z4r<+)sYSJ_iC5|BFRHCYp!=b3AS?T&Vr<+GFd?L+Q}2%)<;A60lIe*gggKq;uI8|{P80hfJVr%q1s{w(m1}D zMCKGbr_N|oG2n*Hb7@e~*F%LtTam{yf<{YnN=knbnOLCX#VtHw{U>*Ac(^$TaveNDgLQYvS+pLb;FSPd z>I5jw7cg4@(sC=VU%kCWfMv-&-W3OuD@^EA3*VJb5jqhEWYdpoVVPc15{~fGpzpeG z0cY44jSaa+$s(qd#Q_4?zJJ`{!A{K&Ez3&2Tf?yZI^ zDSiDpAe%=`EQIvwfaEpE7eG^K{juUC<5mMn^?=gJ*_BDH6$wdaY=)4Bc@zf2J-~iS z``GHi5!CK?v7nJmg-jKSfJ(F1@3iM2=dC_wp*7jRKPK&!n!UZQvC%zMpqBzzw;9dD0I=FJ;u_XN^^2* z+DYnhQCep4@-rQDf2|KO^1U)KiD4H(^MTBvynt1+2Vj|nwd_W}$XI3)lEGX?Ufz>4 zvx!G1$DrC^znd~KF_06*!O3lvfQf0qWqRegxX{`r6-=6?Nb7>gfnaY>@*5gtilPc0 zC&8>5^XR>O+SL^Wt-1$PRD%U<^z<%n>p##JS29u1Bcd5eNVqL?&)qG?(`z0}wRgOf z4T+0at}H&`g_9M|!4VP5Oiv%Z6bXI$y$bpH?Hy&2NPI;%PPn1y378(tu4qz!H8p#*w4OGd|7a@ox=;43rl{PVac5B(|BmLs#Css zc04gkOA%eT_e|=Oa&>m*()uCHl$dsdkCYWMaG*a6}M6h`N& z>QdLq38L9RqQeH1_SR43Fwn5_hm|?LG$uonMsNXMshmt?uhr zPDWHmK7WS7k7fSeUPO2bG*3K&St>-#?c*ADORbYy0AKu;0sq=`gO8tyL-FLKy{&h) zk%{5ckE9rG<&ty&CIq%&`Gctnav$&stS?5=vjo&Y~CGMie8%K!^HB`RjUDjSlinvecTms~%a}d4> z2cGKz^kh^|{R;jHHRX?>b^|3!sQtYF_H1BaN<$Rvp74W_*QXnt@j@)<=z{wnVeRwd z`}!llq;J=~RQY0Sdl2fCsYnX}%S(E#x?HKx5nt$hL@rL-y5-)!g;W3P)#;_1p<$bD zPwi2AqZko$wwbD-ky!&$T6+5rk;y7}-|A|6*HGH?u{y0NHfCWR95SZ6Pbeu($5kpT zG4X~^_MsX!eZ@*eb#Y$83X)!5polRYSfH&XBxGv`;#07};`zb%rKV;pg=J)ra>=X5 zuxcqve5jOib#85bnxAj$pe!3=YZv)Bb-^GlgPBdv#AK;I=5}!VZ>2ZseSHye59sMP zM_&tdZ%x`dL3MZdlFeo1lgsNXPd)*&6WjgJ7qKz*+!eo~IZqE&rPcyeagwP?D9SSxL>!jh7qf~bs)u~JTwOFW@ZB_)t( zcDJ%WyuGl+N~!JsDQhnY(KZw4|yI^ zqWPGXk)`u|nMKOgDsIfY*N^HHW~l?n`{8P#!@I<&bDGw14Hy&$OR)fcoqGi?N@~Vk zUOFAFrCpu>ZBH;HMV&GSS|Oredcs4^=ooxJbZd z?;9$*cAfjE`WYj2k(;wlC(j zO4o?z3rq&e&t=!VEgz|&`BXWn z%7)N@@%YZg_b5|aknhn~9b;BTItfNbTWk6CA;5Fa{}>CLQowC@7_3FYkq_Mdt) z3+h!AVH(Oo(=FH_f=3K|uTD-Nv(QydAcmDET4oYJkT7(aT}bmrWQC% zIEAty;MN_>;7|SOdxw!38CCWtUVpX(^0QB`P>$DeXNq}m;L6{$HtQmxCW*A}5KV8*{A; z%YCdjQym$dJ$EU(3?`G$K?dTSyUp5=udHACi?FXZqOnU&e+&^jtod&W?e6xN-V1B^ zfvm_#PBFWNiS*}>d{)w+D3h-yE_2RfjGAQRrz=kRO4J;R=6aP5&&fyinyhI!2y81W zmacjK3ceThdb$qjE|NFpGgPDugI!M3%hfJZwcr*sy`u&UH=Kpw9hZb4Gw;0R=1o#= zT|?UZMBKY9I736n2QTNHvA5OS+(t%pTW{L%QePVy^5sr)KcXPSH0e$V@xDI;b>qUa zUvZFd?$$f0B%#O9xw(vM%=H=*Ay_@D$8p}U_K?K6>)uA_t^ zBV|kAJ~~I=*yyPh;Na9V^fm0b8>TzJ(7Nd2nmaQ3qaD-I@_1KrWGqET_*)u0iy&g; zhW8jf#!0<1yRV-7#!A@4%CXueia+)<3%KkA?sea8khDMhdXMX(kfeQ8%`Hq zKJSp_o2cG6XpZT@f4wkL`O=(yZ4k?Q+S|9Ep3N-K zIPS2o&4R4?WgQLeJD%aohLC2G3e@g|T%s{+KI6HuDn()B)ne0PDg=NI$n;W`D zs%nO*`F$To^WQ@Q|DC+P;i}RK1$n$lA&c@t-83!Y**#mWFJr^mMw6*OON7mep@b4lvT3S6lrHjqxJW7Z^_V| z<-Kfr0ED%ZpEe!_55M90Q4hKkiY4!)DAB^X><`Q!yl)ghOr)veHN` z-QaxJXLS`v)zr+*(y>{&qZ-#Pj;_!;xLy&|=G#v2Ftv??8WyueM0@C z%0b_!b=)clh4p=5cl<4lccfh2L0E*C_S?8fRGtif81C>BDrSv%Mnngc7=?X7L%Mg* z+6Lb(Y31CL*4W#-6Lhq}Q}m)8{;Cnl2Q8(Pq#qkBxlzR(?f z>lq)F;@~JQE#{=p)i#Ou9Z+J3{Pe7el8PzPCKP5{Ae}}G*qtR-RSRZWt`ym>5}2FY zo@U&>9U?9vO8tYa-oHAS^v4*f*QJT(^p4Z);qbjkeV>t;Yv|}|uk^-w=1)I3={A1& z)Moo@=39msEAKh~0P*F~?c;-TDJ1W(eoeGT>U_8!lDSm)$dwb~D8WCRrhl|EEgw0p zDZvG{Ca$e-PvKNP&hYE_9WBq$^d{*1o8tKiJq{69(D=c%~CJvlkMQeoA~YvNaun3M_<`J-JTSBo40QS|9@ezNG1 znB`VMjk(z>zCl8yI^b&1dAGAX|qLO?4aTpR3tR0%;_~dgQIzEVy zW+42^gOi`HR&-6BMTsenA2k>a6E%j{FN!2Cj@^=c!1Q-KGtWxR5oKI_kp-DOzk%Uw zE51$sXSx^iX_yleJ4>-WrbT)_BZVB+cQ-bR%t*{Fl6z-*?ycCvTKA{xhkpIDncP3y zLKPN=jfZEyh3zXE_xh=^?x1ONve?j&wWEaYU8K0j0SESQT3zf~7x~2AZ%wrI&&+MI zcxq$du=4O^YQAzG&KiVg<|+1}QXTvK5HWSiUI<3bGrG^nJxg^ihczUOj4xui_&`S7 z?!m2_R8$2@Ok(f1r^S_!F_uok7G0LUs~jGzJr8GI`=>p zd-qF)#y}Nzs>I@GJYR|bsYLmNY93R{B`6_}axavl@;S42Z(WfZ1>2j2GM7KpyOtX@ z)%EgI!H;Lxa@Boz=kW`PhkyN~W5l>~hcBSz(>>8&LdTuI=e-Qulck`aqqK|3BOC=u zuBo!Uf8z1C7^(}VlV|VH4+=KI7PoM{R(+&T)r=%-?fPRdt2J?0m%HyXVsHQ%A6gYsaZQpT-aY!HMoQVwu8?OE?KJ zxs1W!c%0s4dEuT;8ay(SqD0Nal<}i^h}h9_XZLMs=}6;KR>{q=)^>t9KNYMr#kk+1 zcYmYIt@yV`Hlxvzkl5Iwrm|r$GKvl-7eqB_fb58bMP!yU-y=IZIm0|+ zr!D_+)%=DZ_l%zP`eMOtSk>RDAH<~~;N^Gv5d1MYTH3 z38D{6SQn3MIcsJOIiAt*v3xBmDlF!B_^oxvU-#iz3yi67Y2OdmzFH)4Qad*CF^=?$Ep4cP)pPe|{H7Ehs zbvy$;Vm}f{g&}}x$E)tgw{49Eg_$t6?zFPD$cA0@>tvrn^{rV z-E`Wg175Og@gu(EH!qMX9I#-P3dI~Y4r@RFgg^5Blb!JA?!MQ){@T}XIV%9+6j>9R z{D1hM559MM*<5BdErdi6u<%Om!2}Rwuj<;`!U72>DeIsi3|d|yblJtlug{!q+pOH2 zn6Rz7BC-5^+kC>GS+!6fl}@f}yinhe_IpO|r0>g&nt(-3*qFoSE-$?ZAp;ZhVePMWvM8+uni2@d{Z`#h~~xr_HsM z$!dRkPFsV7$-+Vk0rqpDni}hM%$Wz2mi*VFaf%F3(6qj)mRshZesC|oDa`yu2yeW zVs;6-QQsHK{St{O+HDM0?9w<7TMkN^CXp2G4#yA*48IVgd7}q!y3Bf$Q1`l znZGH<@!jWP_-=|CiTUHloT2aa2b{OwzAY^K;Dh3N_)ar~%JU|f{plZHMGOG}o2Jge z95|=f32qix8T!&bZ@TyFz9h9E1ta6LXWr_;)bLatTl9#H`slIA&HE>x3lculvqEI{8Y^JWxv#467W_z`6&e!)Dbvc zoG10`P4ds54(A6zeF!}9##QhxKAORzmGwgz)Afbx0oOt3sah1BJQY{J^5XbDO?Vvf zuhss0VT41ChTBt*6Y&z=uVsC2aFBeDMp8*m!GBkPjN^{b6*&@gLG7PnZEVhNg=kgZ zjhs6iZlbGkH-b_tDRv!=f3N)RE9|#icQeOTywF z;5y)+GTgBgnXbIwm8MuP^j+P4gMo*3qAqwqX~lkL|JK>?4Lknx!>Ez@`9am>-Xuqh zOeHXdtKlOdJ(>9C=662__(<|t5xeDLQ)1^4Pfb00@zj-{Rfm|&m6Yq|vNgVTRi)kT z0tOc;HWjNQ4gLqhcmeb@0$0IzKC>IxZfV+ug^Sy>o-FT=qgKfn z*<<`E8Y3PzAYkq`YclPk;}q@-(T=bu&w4}2@tx%2o;a2Tphb+R*}%8C7q)QFuy^FA z9z2*64AqzExChEP*D61>8XAdZaC*9-9ACQG%(qY_t1N%~5fJhLeDdbcZysl#qNDpo zPj_3a?YS-;IIV4eUS71ei?)Q!Y0{Vt|7>H-kPO&qjC?jTi|pp+?5sP^W6X7%9cgv7 zu%yu_5wgPlr2s)8%c=LW=wz{RXt>EMjd)=RxFjU+eMMMX)*xC-Qkma)wHuY@>2Z7t z{z%LZ&Nhh-IZ3p^OA@``MI}T2kE4-+XBtp9?&y;0M zOuh-5^Zfh zr#+u{O{|-~8WlGc6pROMY}c7-a(M~WT~VUr>UJljWc>VY{Fz=XOT6Aw{|`$T?0YVY zK*J7pdxm%KepBc?eUtrzR<1EN&mzI$DsV$Q+AT9o2^?$F=DV2bvGVQ`v#&lScYjWN zLYtj^I>386_G;l%N@wi;2I-I*qkpp4+lI_WS}m`^4qu1q~FQXG&Hx57?5vLvDIxT zD9nAm-^e7qx=Oe7q}rn-f!PxC+)#CFa_HW$8F zyYrz>y9LE*MfS+WayxR~zkiR=%j)DdQ-UxAMlj_lWq}%KraXf^f4p>>G-%2}uMIQ| zCR}a{go#rll#HMv@|&>YVKUYQbP^(Fa&D>l`?rU3o3UHQt=d#HLu=lgfLvCe>F)8v zxj9Ukz5f0eiOx$h3FEaVg6<5SR3JAHH^^hFQAd6Y+AfiX^P>|!C6hn0xt6CclH72( zzH`*gCuGl1cTQ02syED9(kvmq7|0%Q|89Qqz0UzbXAFe;s$uWn$KB&2#gUCCAQOL2 zNqy&zK8jzhV3nh5HX{?yy<+0|7SP&Nt` zIyDB?Hh(;BL;cm|nboD?mp#zUf=;7XRH>B(0sfg2E>|9phkw(&x&JxfV!_vNc(0?C z{bPBz)SmC*7$&a?L4-6+*x`3WXLo$SWA$q=N39eSAD`{u!DXyHKOFu7P+oQ^c0$4; zp2!V zm|n=t>{~lHK;y(FG;)HTMiGoyg#WU9m-5|VyemSj9J=^{gZv5uiyK-QUJ(iiTt^EG zawnSJoj?AgF)~J6#vaFhTyJOeJK9IYM+Wbb1V&tkv?(*_zD9uvio8ht8VL!Kwc`gR z@YzLfxrM*p#Z%0BMswfCL z!_=s`IRZjLkXSL?Vh?~l7c<)ha*8|q%(-teWSuV@Hb#A-Ig4ywza3Omyh%dem7+}g zanF7BPvqxY1{@E&%S(Ml@5%mo@cBn4&ef6E&JM#yjETt{(!$dq8997Gy=$->hwEzf zl&@R99NV3CkABCH487J+YJiDGZj%k z(-CR8oY-zmhqBc*WGbNIW?PLDx3$f^JAlwfe0X)WXZ2I~HKew-p7pSp7^8HHNNLRj zt(>Q(Oh`8QLdh}gno4N|Mv(-8;o*U2fs^-(20Ixh)PEJ9T5d3mSNZ~4HUj+%*9W(J;6rdnTLlaj^m zoIZ;z`AzSl*Sn)e$1^D8g6cEj4s^>{Y>c7_1-;->Tm{2FBj2S(<&XcmjKv&eWEfy4 zYh(uveSwoz%_$RX$r7We{n?Zfqx<)e^EB?W-_$6(OP%tw zb1Gx_bSFoMUY#2?yKV^GIC*%<#PK_BK@h9 zJ2wwGX(m!H@rCoF;$j45IF#!j^E@g=Pb^m}DD1vMLBc9O6OpzXPF7Z?<{jm1K&_N> zegYohV9ON!^?7$^tXheLc^52$@RcUM(e`+en5OOee)mKD>uW>YPEW*@@*lpXr>0Ix zc`6y?1cKP_tIs&&GL^11L@+w% zC6|_DR?}z+WB`?Rbkduf3VEdN?zVkydganY&Td!q+P`fA@FUIsEr_u+?KedvNCOoG_g6`AxMHrvyG|y)tH1X}ZuoBE z+nJL%K2Zt4t<0I*A&0_`Mid1F|BvG44^mQhSMVtXMeV;=_fY3h0!Vpm;C@xOex7TZT3M#Xt$k%+EdVg)G`7KJbV2+!U z`vo^Obx$dPjWSBs4lpYty<0i!CL$7smujbGoZ0D2hNh;wbiWP)Fi|11oJY~o8OXe6 zdE@HzBvOVmP(3){7V^w|s6tk_bD=2Q)6A^cgICuAt~0ANG@@|6x%&Y77G2VFl73+` z>J*Ze>j`kg0-L+?KU!{H^0KI3M=~*SzBn`zI{IUAe$I)8?Oax3ZlREh1`VJJ5%IxI zO>yo*PT^D(!QZ(iYT=Gv#2H)QI34}{Tb;Zy&fK{=QzGj7__zI!rn~KG=6+A1-=)5G z=g#d%H!a4H8qT&~_tROx+ChR-dVKTYL%wQJ1@qk14p-+SRW|c6rKVJR8r>~TB(=&u@s_}}=ecZ4r9u}@ zf?E#_11sl~0^j~M4)`L}ZP6k5H}&*-Oz|m0_#H5F=hp&Xm9Tq0{@I8XS1ZTv$_0{E z8?y!+MKA6-Sz3V~s*a;JGJ2LaCmtXgri5u^G*4}2HZs0U$j2u_=Mz84DfcMq%ZF%A zWZgD`k#VvpwClPPj~={wSIAB-h#9aj%4%uDs@+7QgY-lKEnVivpUoxQ%->oy{tBr{ zgVc|1*Sf_DkY*!&ZMX%tOQFd9O7QXoJt5)i!&Lv6?OMNJGMhzAm0Uo%P*v84@d5$> z8F1xCdhCG-TNDaCick|#&vTYPE|0iTSu6f-Mt5UeRh$I`U;i9Cj_`>Z*KBGj$$uR@Sel2t0f+~77>AeoBuVtF^U<{dMD=l zV?mn>ewExze~MKgrJ>=F1F()3&zxF_1Mj{2V&kQ0c}zMJ zL%osgH~dfbce@0L%#hzT`t43SAvL8}&0#ktlQjJ`IsSn+Z~6FQA;T#?zi_hYUK_%x zKZj`oFyTC1wdO3eu!wQ!*|86Lkx{6#Pp!2gqe||R^&z#2o5{&8Z?d~BhFNpf^!O>+ z*p81IIcY3Os;DHQurSg1e&c8xP{xN^<)zK^iMl;^-H-20IX7~Q-PQio z^NqIV8pj>i!=GD|UwK}{5hhW?nIJ|(%FlN`CG1ZZ^m<7_MD^dw|7?$xXF?{7uU^XN zpY4|eg??H}rRYB6CQZX%AN42oiO|}-&RB%b3mdW~MaA@9`>Yt#(A?!A4yBskef`)K zP%ClC`}|c6K^huC(KWN_wdK5z1??0)x2ekEk?nPe zO8e!W^CFc51^h^P`XpZ!P}GfazdVhHu3RzPoL2UR24F)gyt9>yoY-P`n!m8z=FU!e z?8N!cl!`xH8G{vhQsC9qfA)u^*Z8+yCn=a*Z={b@IukFyJou3yI8uZHBY-1SWt0j_ zxhKaTbKa6mm|1<<^lPp+H4dlIQj?rgsHxG!4R7-kiimW3U0eyx60HogEfZxbpy=trn(661a?Cwv zd-5qJFjN@LyIV-C))hje2kJ@+3DquA2i+G^GFQsbvknqmb};2Q|3+=p1@h zE9!flE145Odmlv2;~MQ<{Vj)et5?s5{O(cs%PhD^?HS?=b;H-F|&JKs#^bCu7{@} ze3kWo+B^5K8q>b-FJm7eBuPUkTBTB?>>8=ihRTvDZ8V}JB(x*pu1pb#N^kA-^v=dPrNZv@7v=h>WF7_BD*PRW83!+K7%3}s<8qg) z=v@l_84O{o?*MdLxtm*`TY6+w6?y7LBMHbuQh9Kj9wsC z#Rvk6YAD6KckZ0~VhT9)U@YMQME_M%Qs$P;S!rHuy8uRdz$#60v_8D9XUd5adw!nj zFea*|s}@9WC!m{~%A;d)I3`|N>m!1ssPCsKd3(>Z6<{k3@9wSNG!~yaAN19C!`!*4 z8mg+Q-0rT0F5(z~p1LY3SF5_&mTrEZ^FXv4gVbhEbar~=qGvm*IyyQ!*&&|Y7p(;< zM98t_Z0u6GPcQ+^*zo@S$6!4OJyit(=degWcPQQQH-*5YH_#QIveBLG5=o8FN%rFk_@}gPJiNEz*L5!}@C+*zYWwmarCG2ORnJ^76@kha~$K z;2Xq_vzPavmVv5S7+Tm9W)dpdkh~fodR*Om7U#5{{*t~C;>AgW`=GV3yf;X-?R$mI z@DQ!Ig;^Ia^xC&?pN`kVEMP1#YmZABmq6MIc$K272PUxiq(;+~*v(b zH)Zv~=0%VuUke_~e?iZ3D%={B=z;pPa`!&-zv2Hd;A$Oi{R+Dm;uM{DnBC2ZW1Y>+ z!d^_D!z9FvuWX#BSW#VFom3ZA*c4i|y^EROU-(u@5^?A&af~Ox0^^;^;U+#vg4q!X zWhz9=+w;>UdAYgW_T8vOaVm331UOkVz6}D@)?6zv4MQ`3!xRiF zd$ZKA%s}jhrds$*PD#XUD*V@-z|HR;Pf5N@9swyLk_k1=usfn^ES7`ZC-BlHud?NS zZU`<=*neepj)Rerk;pc^1u4cXRRAt|hvF-Hz^6g-(|}R43b4t>3S7qNbVT$OzxR(9 zr|Ch%=}0Ao4>AAOjveND-r_feyP4zX4EYMs+tSXz`T2J1t52Tf?_Qpq4bW;VnO+2$ z$Mq){x!s&iYny$0`uBOk~2n2h*YNn@hs;hKows+JZrFx&(?tYK#dEs_Jfq&jQX&%XdUmocvhRneDd zEl#&JeR#>IHlkD}>{ZVdZEkYxtKRtX&%VWtXmUPpKB_qt_}e^W&sj{V4A?Df0(G|G zN;7GD*WRV)R$4VaC3?%x+U5FBq2f>06NYM!9KG1uQvCbTN+#0W-NUh#U0j-a0%LJx8S3M$`()GUT04e6}W&$h*zjYM^mMe>&%vkD09FT z2s4O;n3%BTOgV|CRJK!OWpQ}((mEsmoRg)FOi2MFpr=t18?(* z1#%_0iWC&vRivCe*-yzV`rsS2GcNn`br%nUK%|80de%(L!aw$!Gb~cJ*hICVGu^~M`JFF5OZQbz#r?D>%vEe6{)}xR z;H9NGo~n~O7sMM>)HIwlx%TJuW)oFP@3jv(f!$J%6GyjY?(P>|i+dG&HcM`eQ&q+6 zQ|i zxst!TJ#UVXc}aZREGh%9DLM&0$*Y}0{2NVZ7-#NuOso8FamN}-{%1i#r(zI83 z+K|5G_dR&S?*Zd>ce1=@&X-eNYjnmN8t3i7%YBG$CP zmI>!R`*`sUihgT?S$M`-nwMLgn7z{7eY{rzb6?F}i6Lvclh|F9hXUn{&`Zbmv_w53~4d^L0lGn$RQ7oqX%RPc62% zUj4qIVdbrODg&#PO_DMGy^q#~V&jp5AB^$aweIc$`m^s4wc(AoYyvjnIwI1I6D|;( z;!UTN`<|hZT<3Y9=nt!NH=M*;P_g&?)VafJ2OOt9?RlRN`zYT}dmB4wb!B#yMP%){ z7!z|}X`;${S{+D5=K^T4zzGu%ceVjn- zFI3nlR=jKFNdc)Clch9b#I)efjvWW(#{~}Hu|DczW>&H~Q>y0H)S&0m4KsUA1D-s7 z_rQUnt`j|K-s;V-odZNV%VyqO`#FpDyzreFGZ6G=!1`Zgoz|!(w{YJn$ zTeogqF?@8~BBC{b?!*d@v(8daAD>08@{>dioYs>&X}d#0L;F6O{_0tIPXAFce(d!p&>X7k?a!fQ*Ss9myu_J4~UGzD@h}+rOdFDKB zcA8O0+0hTi;=Ku{sm6)D6NW}$Cyck;P#?c<%-y0e409FD=B}h_nw$8b#U=RL3f=7r zBYfp$Ot~934D`j_fr`R!SKY>uy>AdM&Gb?R#q^=wpX%4&WLQ0xE z>C6i5`~G?3axJ<5y%A5l)jc}*1}7m4BRBDU+7fcvDCUbnbn#273D(}I(Gue0i4JRr zdznhI+hTUq8D==;+xGiOd%|6KZ)#v}$VBrHtEsg7lp@&;)~ z90=ciha#1y>j!VecvlgJvNZdWKcWKQP-PH`M3Bw1Iks9wnDQZ%6zM-3UVE2Fk0rJ2 zGM@DE>hzYZelsgJz+KvwwH^yQ`Vx8DMTyqb*Qu5}#w6+9r9{9yg)a6VY85c1BMp7r zAWo?cE~U%h9>yL%EOVSiPp)#3Uy9@d)z@SqA4vP#T!SZBC)D$4*R8TRw#VPyU z+gAE>omj-byvH;TXm44y>aRInat+2V>)qVlKi5~j z0R*gP|Cl}yniZb+(%s2P*-;4z%Ix%8`*d2dS7`41ar?Wz-g-9hPmyTo(fKB`C52r} zG23-@^%nPzs0?++gKIk@CljqaVijE^v#I31r>0GmQyrU}oI0Z9xF?ehEq=0zdYHMl zE1MK=Vr%T?6ux)eS7!Wzz@wb4mf2BcMpgUrLsl;3eZ2@VLQ#f-eoXd99 za(B;-m~*nS@6>8IqdYtwjoxV(%3izn=bxf9hn!Owc@cbdQ*G0kow4!p4UAh;lXlD1 z#1G>sNO7GIZ3zn6l^c<7o4z|D!raQLwY8OqajGKT*UesV4XY^~A`)E1a+~i6#vuMjtABW1n{p(l3;G_KK^Yb_gPO&@P zr)Hl0<)Qq7{GjjixsrFSL>t;L*HsY-;gf*!q~QPY;WL&i1ftOdZe7fo%~Chfi3JJm z&#!jhuyc-yiR9>yweT59$;rE13_0_RGAjbFCs?9+&X&ilV(gSDi!&b_sjaa1Q&0Mu zUvd5ZE8 zzMe%}Kl{ZQJ_E?2l%h@x74Oz9eSQ7T&Q9q5Z0Jd})-yExk_@(Y(aAlu>iPMbN%G{) zQ+B8O^;MrZaSbl-Sl;?js(5?%>DyNkemWnY+4agI(XuB zJ^?%S2i{~a(2l#TAKGY9M-?5rPPaX*>kN7q+fmOQFTB8I6n3_sE**lWihAskL;Fig zN>&0IHco&+8yzbvD|<629p_qyrAv>Tc|@CCSLczuv-+I-Uh6=T9XouO8h!Jr?X^C9 zQBzZsiJ1;%>cP8#%G1V`FM9(498<%>PV|lbdsi4-!kpv9^9sy%XL0B&L4>zv%`E@t zSGpAsyk1;9wkx{z)2GXbTKHJPJpC9;t6`g5QFD125D}5UPB1g;)uTs^(Y3O&w}2m} zD83*$zJK*<`G+Z#b9;90*1Y;49+^1$A&*qdHaN9i&BX&nH9i5K8=s=@Q2YM2e7S%) zudAy=R79U~hO7MSnbBu^xbN}P{j@}Jmeii-z#;Uz_y~4*R7_0k)!w6U?G+{qT`1|S zhYq{5sDs2v((jBQI_Csu;p)!89V7iaG%A5OG>t* z?M4DwO4>9}jo7<480RS9utNq9cAy!+84xE`$_Z4A+ai$vWbwR?kMGrE%BamyZU3sS zzRacx(2s?TGg%7z#U#7a+zdQqC$B6&RCO|*;)$|_`zk^~5H^*O%R;sZOb7PhTsUy^)O?$SYweCN`muLgAR#Dsjk&6cU$CY^V!ftZ7%BEHyx`@-aD zH%dx+ZHtbsM7KPe??I1fn=ZMu42y~vmtHvkVI$f*%E;Hp=cm!prI?Y&=TG1?7*(UO zh}m|-$?cL-115lj26>}|{5H!a=<@XDKq&@bsaPu&U1d5d-q(%~2y7owWK`nsVJBh_admn8pqks=?uD@Re?P2A7r?~hP@)^yK-bkVd((kBgevSP{qjb=;ge6h9 z#LwJ3?1}*FwmN)Z^T-g|Xc(bXV7;Fsmr%{!?D|#p!{H%vHm1eG+Dn~jjz`5yf+@Na zzDpZBZE?_Y9s9Xu6)bVe`rg*)Co}-HMyoMF{_&bLX;P?Bp518*e+>~Z<*cl%4tc!V zLtbw?e|~b}vBD@q*d`da&2!a-cB21CgF#mSmUBrhFBLQ$DG90GA_w&9*|V=w@y>x$ z#o2%Sk?t&=wynI}GFliSD}U8LnTkjhHmjCC5`<6l(krUDBJv114kQ z3^$y}lF|os$eX{NGJZ^wOz4k0y86k5jD1b=DQrwE3Zow6RXPo!UU!^NuCmcq$4jf8 zp+;5th07kEy3V<+=1nh@W~+q@d*7h~?&zCw+^?w`^D4z{49N)DnqHvoieJvKUN6wN zDbE=e9s#V@mwo;8-r{Z97oijFnvFrdYcYr3=A_T>X*K5|(dY1#)j~0hLjcOx;P7#j zpKNm#p=BmoPp~|S1IJj`IhSnKrSx#wFlNjtgYT$OX%dO!y_2v1BLfIJf#ZkE@i>!14n?m59cf_E($WY z<6J^4)ihG>)oct3b;;OwjBG3HIXw~;B{yWqBFvR&++A19DsK52s$=Mt`0v*fvrWi( zCMGqbG8-|ZW6F1&%QgyXmps(5?D$l5_wL=0$Q^&{4T%^xIQ;T9`TE_%9D=cNlob>= zx(Hu_CXJMqbHSw^+RlMU2mN>D)w&NKj%dA<4|X5%Lc4t9vJ|)PLooXL5J=OQDwo^e zo+joB!AtOBfc@SZ;jULN`dOU7YZgA2?fwd13Fqbt+2RZR@}3NQPM$E{`>krxWyPTZ z1Hagl%+AtX;JHi|7VI++jgB=h)?LE{XIHc2(wPcQYk4?}!glTI=OcZMO3(6# z-HRpq0XGK+3t4x}_qE*?C|N}z4MFJNJlR_tW6EW0S?FhgXfd?MQ9{kLCW0HHYIHoxFLi#$Q$AE*khr5`rL>z2_XvHj zOC0Jd>MpF<>QsMaWg)Dw#jL+ZoRn&ko1P*ps9^#uG|eB-w}v@CavT2HxgzbI$d`mj?Dh` zV|jITCkLM&Qyq4sGXLcBzQx9o@cMrIq7*l6 z)2TtFWMtXip)20viZ0k8MlU&{yP}L{lPCGx%gT|1OcHeG4TZV+`F?eGuGYzY)LuTz{nkiV2{Ha0 zKAUm*b>w>R?L$A@WtE`4y?xf3k6V1fQx~;0QxxDL^E%TS__phEqA`IYcbDK-$YsUO z-j|Dk9-3RV&S|4PLGTe_LDNO~p%hWFM_Fn3@IKF3k&DxD-Jzo{{UyVWgw}6Y zi6D($TEfoCmX`*-zQ=5kQqwICYRn3h>VZ8?*bVHIp zVz^-?3LGLKS-zY9=FPWJvuTn?M7~DQeaJgg))D(}A;i{Pms8oCX_H)n>++4Z7}9GS zzB`!SEXx+0h$xkTW(Bu3yK=UTbv<=6cJBW9x<}DRysN5G85#`_mGi<<#Fn-@b)HBca^{GwEoU^n6gQf=y`1Wp; z#&2dlM0yla`Z;4*92sfVIS=VKxV?US^zBBLWj6Wv=qd&$Oxq-ksie!)nZz;X`xc8p z8YBCiezkzLym%?uIz_1`S(euO$Ng9m$L}UPK-gE#fHa zL*^_F`GRqOdry5sicC5v&GQO8 z_%TDnFMFNb$vLDk;(1%P2xTl*;8geJ%KhZzDuM_6za!#wJ=?0h4>+c%PkUFM^7hMv jdiD63|NCG4&z}y=$p89jg|C2^70owYU~<-Y*^d7LvA&w% diff --git a/docs/willy_wallaby_from_wasatch_files/figure-html/unnamed-chunk-134-1.png b/docs/willy_wallaby_from_wasatch_files/figure-html/unnamed-chunk-134-1.png index 4c36db7948717ef4c426af83539f596ad8c7e648..4679b4443476f66466fc949f3051139382ee470d 100644 GIT binary patch literal 71554 zcmeFZhdY-4`#-L{l2IsRg(wjr**ka19@$&S7TKE$k!Z=tEPKnIx4mcf-h18lZSyd4h$7U4wfW zzT@%Fv(NAa?~Rm}0~Qud0OtRtFALiySXg(kq@Rhay2h+en0b<_oosCvL=&#iAvl|2^h^ z_1dgs4Cap;Rmfs)^*Kv^6cULnDRBv*;t>`RX{XP~zeP>$u{OvyC+p#H(iJO2o{7-9 zs$_I@bi||>r%K9eHQ3SJ-OaKQQIs#bldLLGU*U7%?dj?1?(VLj@HqhUA(^dR)NhG$ zAoV4y#$#Oy_y>yx8;puMyT8AmDc-ZUu8k!u!)dI4Yy^9e%!x^M))&tQY8hKOdKJaa8>8?P{W5!r z%z-wIvhwm`n~8GUsVWZSo^odt4=W3T+NjaCPG+C}(pwq->hMdybSbZW$kfP%nU#q9 z;)tE?sYnF)UOqkD?N0HpK0nUX&*ivysr39N35jF~6&WdMp?Ob2^~Jf)HzWalet!OE zIx$h>rr@`4-=?dinwptuNW~Ck%IwJV+t0TQKK^uzEh9||s|GRt?&{m1Z?Umwv!6ID zxuY};yowTxYsGqddtt;T+HqgMzL{_NMl0g}!$;~*IZRUQXg-valQUd(aK%wJ{Gq^` z>6+h912WVy)d%xLELqUekGQ$_W*a{>MT8&R!Fn51N8^f>CPl)e5F8kow#>ZsMLwE$ zVQGoH)Uy7=^@ZEl+Um|u4o>z~j1AE?jtgyJ@361-5+nw1$>U<(nk5v*(lmhIyLye{ z4u`a&A`uahn8Tu+jZIFau$Y)n?fWaN8aWTAuwJBHKIxX_wVkXqYQ(>KHDD)rX{N8- z$?9mXlbxM?;dV`(KLIRuHna6WhFZL+*J^*dv6&e;2?+@mRk?99VWWXSszU5Bdd$|q z=WO5p+F zDX6iMT%U{c1o&jmL1aPz#++xekG=(+gx5APJ3}zQc|)V zyI?RK@A`FlDXFg!|6IN5;o(89c?8bbYtG#iF_5WY3%34_$4O96RaMpA-ab~)DMKSS zM>)NtLFqkZUC=<4+qZAug2pyCt1qf7(CmI``p9ENv9lkMk*@PCH;iY>9G1R+|K6d< zoYuE;^X5&s-r3#t^7OonjorC7%Dlj%(E=MGEwT3cXy-uZ?`22dK;!{4plJm0@+wsyDxkJ&#r%lwhEvzo|;8CiSi0 zkGJih*@p5>WZj}|tBw|hHY@LP*f}|MwzhKiA>O4v$sAzUE|iKp8yy+pu^D%5{m!VJ zJt$|TF2*zEeUg1x0#S#obnWoawNXJMS3f%q-W_W<*TiG~w`TG@{^XK^f`WV$x7%8F zNkT#bzV*lsiAFxUlPfmOvpY`2bTyjL^Ud+zN*&84xNJkDHW)?|P31M=KwxpWc5sI6y;5d4#Ii1P{{f(SGuzW-Yhcd#3J6pZsEXJV%H7!Gw!^ z3CkyISdeFBWu>dDD+rgs^RCrUPMC&+qobU>JWoz%TN{;_cjeIx*6rK3 zn?k4~;^J)FT-7b6g!@R!?H6RY_WOE!Z=H?B3c1K-GBPrXobC0QCa9&Wa#S4}d3shF z)z9h|d{EYXtUfY+jn$h(+ttacXmq77RY*YKr<)K<*y_U;F31z#f3uY8)B>TE0L&JZyc7M=zW^7IAHk`}@&72`!0 zrzU=h`;vzISV^aZ+~S!nll-`o^83L+yZ=879-rrYqDuz5`VJaJwNF9CUlvGo&jhzS_UqdkdAVI-c zo2e?Vo$oz~lGG6q90u$@563Ham%0IP_U&x7m3z6kxHRHkPESvV?MC$`x5I99#fupX z-+S-hv*MU2`GuEeu<8l;XcPphpZJTpBg*=Z!&ZYxOsnzOI4yebHToMj!SE!5DGWU~ zAze>GC{B@&CiUsw-<&dF@!4)95AioS0|S^%U-lu!s#zpks|vNqSA{LK2-DBi2rlo3 zTjITsSK7i@U%Yrh&Y`bc`sR1S#V%L3@LgM7J3d7_Iz@(wPS&$38K0|FVE8-u=jZ2` zeD3J;n32(BveGRmE)N^$8ngOIqmkg@DP->?wWaQOjjZFD06KgtNjv;|;QYB&$E&@` zvJe~27Nc#W(1$eO2YLSaqC5bAf?ll6y9_Y~RTE{XXR}nNr%|-AKYFxvFy*5tFF#;e zWM^l$Hk51IpZ2ZteP(8+=PrYy?t?Lo-lcdS8~dR1ndxcb+qZ>HJ0h5ij7d2<`?C;iEa)Z25M#sgteR^m=mDE15F4Z&~~D=6|N2+E3S9 z>ci$SY6uLW=95vuQi$Si-FwNdU#Z#CW&B0te(ocaHqt(&_G65dAt2Qzn;aK7ZB5r| zU0|TyY{N&Gd(>wUQNy}_(qjP%wY6`ouQM<(ke)>GScO+Agx;5*w<74EwHnCSAJ8y> z6%c;>cn6}!&!0clM>HOgxHcxXu!OZ0QWscQ-s|?vimp+Nd?Y@pjqx=rYF=w*12Foj zx=~J%D=|XBf?94j_$xXM2$l-0`Is4WVF%f(-IZ3UiH)R#M~VIZpQ1QcFdc{V4cs#xc-nveM$=(b>UN zh0CT*E-R6JvX8MWo#{{z-03E^Y78VvkgU*|iBJka#(1CaF-a!)jVJ2#0 zV{@xXeQ<+-zTa{mV8tYEq0M;d_J~=$*ZI*B$$dG^O4q!~lX`$0kipn< zg`HMa8;<}hL<_s^_*|SA7=Jukx{F_7D(lv&+Crw0Z(GOE*xXDf=qRQC*!=G=>CpQR zGWIvdTmbihbEfCr&BV!X|aKI{`jij3ts z?F#4DZZv%rZj99Oqu;&4b_hMe73QtT*>yEv-kWO*3DvL~$xmLmq-ir&{5ohZCDMHr zVq*~g#mUqKzO{@U{^{8nK^A(unXWw(tV@R?Lro#1_NCHk>LZ(R{<)0OQvM_2Wk7X1 z;yc__AuC7~1B0mueRm5m(<`3?QeTQTcME>JO$7Quj8qZ~jb!<15}amh9FOk_o?dKVGDLa`ODAS=>mKS4Cyf z{QNu};OGE4Z(;;s(1TTs5$J;{@*zWXlk#m0x*8i7AWKKM>ohbr>Q%aKLs*2JJ3iU0 zuCJ@ZAAbc9WyP@VN;t;9GYWn(i>xsl-Ad#G^r`{6~aBB*t zgzcAAP+(usCsnkG&Y!IcVq{~J=j?KR_RNowdU0t9GNE>fho!0M;%6G6c}8EH>yI~n zh?@Vs<5uhpnXa_76yjQx?fBkl6bB#6nhKX`2Zh_^ctFbl<=__l(DU7$?IlK-o`?%d z7e3yx8I;*W(ZmXsnBAK<6x`;2b!?Gp?oIrR#2Y3dt zBkUrU-Y-(XA%*iDWQ4F>O!@Ypo_RfrZGkZ~B0|G0hhKne%FlEh>7jAndwSa&vSI|& zR!Vmvc{)P$E0z)kl*xZY!A^huy3=Rx;&MR6V<`-kA>eKY7Z<+lKW5!=dB)9A=M6A~ zMTqujcrnDJc3ZWdpPlucPgrjCm6ftZeE)7V;b_zz&Q?}dw)0&A%2MsZmQ!_i>U(7QE{Pp7Iod5QPCatqnd< zejpI?{BCE<{?x^}OIM6QcryeUodoS7OUUW8!meAGB5$uxacO>@LZh*vAt^Q0!`&Sp zQ#H@)xVgE7*;oI^CP;nnle+BQG_Yelzt62`SL^QT8kxR6H+Rp-q_bbl#iW?wkw==xsC{f{ZUe}bkEC%s7lu0oqHIFgIA9J!3J$J&L< z!+FLShb(hQP2egUA1`wda$Me8FX+~$)fp-B0Y1d-0H28|70fp(eF_pNfLT^kbd}xQ zhkhOL?cZ#jPp22dcD{3pd|Ys73|o)>BNY@QEG#@Ow*a82#X$RTF9@OKGkvMX9(Xn- ztquTCam81zT!|I)(YKRiXJKhuM4PlVH%lffw|d;?(ERgD^XXF^Sp?#}5x-S?EC2&w zEWlh!Y@`;PEuVkBE%17jJ6oLFJklACElDP0WCNNM6yaG_1#A#ku>S zL}hMiDOG9)+?8H&nwrl>cX;?^%IaYDN&Lm>(igFFDQW2=7(D6E?4M&{3*+999u_!u zcXtogM~uBm`VwE?KKTU`g^WHw5BMOu_z$oTW|S5-mkF@Lf9!lu7uJYTb3~16o%gI zcLVA*B!5cfZj+G;SX%yH%WcJj)W6Cw~2d3g)- zaX0}zgcK;xc)U+t-F|DoUfB%Ew}HHW9`qd`e>A1l8EkEP1V=|kcc(meLrzUj&>Nq4Orw*2M949g&1NM)K@-HfUC=EWDt1iYW6ljBO$jQyi7~nM z6rUxS(oL+O?Dr|T*lz;@R(x||wOPf5zzn|f&Cbn{+`46|>#@J8)+oaLqF;E6M#%YL zg;$t8?I92q5`=f1mcoaQHcF=ENh0y+VG^cpSQ5&6*<^?gs0dE)Zz2!kj6&g8-L4QQ zzGGne(T#0DnA*fFBs8`+W10B(uTr1<2SOT_+_|+i%v&U4f%bzq)1J8x1FYMUnW}Pg zM^*uN2y$6Pp+rsEegH#KSt)E>6c`u?hJJW*!pfo;%_~3o%R+IbWYS$WwU2U43j_G; zT&BFt#>aQ8ieV*Ia!VPs)}sHutTj7?q@lW=36=a%t^uFAl$;zljZdCIwRey+>2A#z z3Qoj}-&v53m2`!%VWTA=GD3M{?D_Ni_ud4FPk_8ehlUOSdIS1s>F(CBR-Q#oEtKV6 z1`ibhhAEo&_2ck|`jxI1=VyQ=&A|#}WZIx2`~u9#R1%oppjTGE<6$|Jr_ocd{{9jV z(#ZlO{I$DT-F?B$GD9`}@vD}v0FqiA)gYgdAv+oC1BfETp2o$*K#e_KVAc&G-_~|> zeSICae`Bgz?C#yCRRf}_gOeU&ilC|@UBy#nU^Zt~Slt7k%J9wunr=Oml0)hdS zNA61jaoyC^MfJs5Zsk0D-UO>!NPT&z&jD@e>7lTBT>X9b;B|G~i# zg`AhLj#*P{vNNK~Si{0r6E1?Dhi|^oSkchXEO4l?sJXkB=Ur*vTnzIzNUp@TeMF(G ztc>9ZMR5SiP)R(0ek=2g;lbiKAucYnJxQzrP-je28HV*AcJPy{F;?~51`_NQl+pHV zK|PZ!>-p}zLPxcT#gV}i&MpfLKL1ESlw%vybds!;adIeK#W?SkvGt5__b z0H@hQkz~pOY=O;4zRAu_-`Yn&hsjjs)9O@f@6pa1Z1d+ySuOX}k%kPOHY zb*@uBPHM>p%>DKf_Bh}-l1g8JnPQ(^nHtN$Znv1hOD`dz3W&b2!fkbE z-}mpmFV0-r>G0Us(2NkgE}-xX;z7DWjDxH?jP!8nQv+bn?KL?1qoU%Z+-VI!fokxR zi)`?LMtwW=p(3jx5f8NHW44&Y#5uq^q!(dQ`rbz-P;zmNg?D4nRr3J>mMVjR=XPVq zPcE_p4S(X#pFhLP#K4%9l)f4bK)C|6E$dMgejgMU@V?63E+PNUjDbIf^%$mYeWW1e z3qyo5-f_1q&ULCJ=DcGiXRvIrSGlF2HE;!>S@Fk`Ogp2XPY}ar8<)WK9sytw0EgH4 zsoU;ATeVj!tQg=#G|ceU&r86Hoq~Or+Ajndxn8~-Cc^`(46Z~LzL-9+3cfs1>EZkx8xUE4AJn&@cmDnZGau3a0~Dv>eHh@5I)o`{d=0_y|DZ4A1s11 zD`h3DCwNK8$q#S@R{^HM3}nY&YHAM0iFg19oRFA!c6uu4xcta^_|eKip?)qMr;(p$ z+#C7^+v+p*Y;7L;hVP8B#kNy64*@tF1(C6~nHxm&*@horyEr*bgwx#PPdx@_cSy&YoseD18t9C ziO#B1JOSjsq{Xx+zi@-QudinfSSCjh5)uN{36&ZdH5nGxBMEYMEG$|JOe8nmLUZ1y)d5yPVt5L3`Ck}D%L=p$5?c@`T%}Kz<^wQtvD}Q*H;`U!H zK#}DDuvP%ElM}EXh~3*F`VFPMClE0IT3MQIoa^kzAi_&nSu69cq4H7Oz!f}`xT@?e zHBE;AG>*&pvgR$&#Hx-Dv9Ny4nwXjS9IvEeg5YBl8Hwi0Sc7?9C(iWGyuH2Qt{k{a z8xV@?QmhM$AgC9VwowrKfSnQ)5~@C0j3%QYz|xe|)YO!cl9G^oF-g&Y@d+%f|5XwJ z%)VeIWv#h$HviE_p|JA&k9+-dUW#<)%1C>=t|^D4g(oF7jbAv5wLSeazs80aq^G7l zH<1`fQ=ZFIFwjyKV`sAIH3-GmSzDQ3Sy~?|)*ef>65Bvc+aMoZlEp+KEG))|)OMkt zj=ej_G@e3{w{ohHlfx~oNui_@|$29O@>A~ zJ4fr#Lh86;4^<%o7jBy!4LZv^;X@BEFU@KkPmCPTAj;@5l8`5@5|U$$&0|i>UkzT# zXRABS2dXS}xv{f>NJV1(P&Y4 zEoJEsxtLQ3T~qdi*GITKw?aH5bn5FA|3NvR3Vn9uDM<( zG04h2uam`aV@KX;{c|LC-c>(h6d}sX@_>?*Uy5bwX4L}y*h+oUPqTu}a<9~zEc6Yd z(N>`-CN;-ruBv|_!N7L&89n=~z%=R6y_@(&cxOk|w(4hc+OM9pN!NO-DN!>bqO)$X zRBoHB?63T+*)%VNxZOjDXGdzxV%KAE251V`S0B?i6sNuB8tW)j8Br^ao$M+_a=DwC zyPIo!Yx68wc8WwFnOS*fP@7W&>EKu9pk|MI!^LvLzG;p_9^E9}2 zxU}6%Vg^jZEl&7)Z9QF4Pw5*h3?X^4E=~Oa^NB^OpW+$@S7=f!O+1=|Go8Ef6)a50@<~=bYTY1POb8~Gy zI=l08OK{8ISM%l$xAgSqH|Ecj6C-2_B8)5yvXP0SoCfheJR6o(qEjz6G~X+I64njd z3Q4@YeGQ}E%6i$K7Sk=JsFB%N>4sueFa3_xnLP-cO1`q9P}*?#>=+d}ao}{o+NywY zw6;lXJ?sPYgonj)@>Kl!+4^(=wTDd>N*R<1tnNcO%X)~w&)zi6Jd{Hv3I-PSD4y~! zR*!9TL+x@1lc}n=GBXXNrL85aKQrOP@e58br|3sJixr&gI7BQJ#iy#L`x9H$4~CnP zH=5)Y+phO$;{NWjLcJP)pp_;;nT0FrR&*3{hFHH+4{1Z;SnkvW)}JA#&r+gEv>aCt@p7hOQ{?LTKE079MEGUW~mXO z5q4_i7wnl8*|eoDzle=qFOQx(9qaIECvvr#`4OWnPX_cfJcQh++iCgEpJ15}XG{chr0@UzO%6 zfBkXTXyTu$S3-K!#k@~LfhPWI{a)B3qHx=SWS-0stUyXTP@EzuZzS7BYJy&I$bEFI zRVS{2N;+|w(vMyvLMP76TJ7X&D2G(Dq|t;D>k_r3#J{22)$t0>>ihTHT3YHJS-mfb zT{bfu>}iwU=8x7<{B6ezw`Hkt{SbJEb?m*o`A9 zw6rOJW5|Eog`bV1liQ3v@B2nSzvpNR&RqhV;-SU- zZ1D;<`)J}i1rv?V^eFC(7+vsASU2 ztdD4laujIv-Z`|3p#w6C2-nnuolQJ?Qax1r)X#Y? z*{9)gQo{7|Mz-_6oYNL{H&AC43v|k^rlo~SNEs)$GMKpEe@z4{S#3IU*K{!;Adl7K zTNh*er?w#cH`mi@Vr614cK45xzxFX&-c4;?_>%2U1nYxtE zwc{jDLgQ9;RbKh+x=`7&_?rGa5-oS$!3J^2~c z;gxsdN9@c}v034>X|$yLvPgh@Nms7SocKfbHyx%c@)bNk&XiI6apx zlTPs;>dlpw&Z_V|c~3}ie08rxMaxXzw|DqP(Npc^>9JZ(^Z(e^|Gci1Okhe#pfX)7 z;k>190gqnr#0B-DK+fZs#HTNouo-*f;cFCnC4yY)%X=}cIq5!}BKPcsq&O=P z`e%;ndo!i=JNB{S2dz`_4X>Ibd3$TJ-xY3Y)ENY+X+>pOq^9U8Rx_{~D7k?cN~c5N8}4)ON8PC0jHU6gUtd@yCGN=ogp zA2>mh-KhJ(y4rld6@)ArS^oQe!&GJa^@DV6B#k@c>DtM06VB+Cr+4>p&I&17^#+F_$@Vs)$se4b7|W}NaA-D%4=7Mb_45@Qc~i?d_>vVyFl&+Vv)GGD~cbq-lZtS zmcMHK4kD)6SeME4Wliw6#tM&voZQ^pl9GckRt*-x@Bk$_=Z?8CT5<%pqpf+X@Rs6+ zr;K}thx`yvQ_hM%47+fATscbE+PFJJ@3i8_L#fPl$JfK7T1#eRuBk(PJq}Jo1F@$< zl(Mr9vqIKucXB=5|5~K`dgbtzTvVO^Uip0m<)!>LDL2nF7EO~#R0T>L9gH=CRPok{2HkG$S59xi&{5^^|LB7Y%ER>7N~bK ztW^k=QlBJ8l#(NhoR4MZGyq3bxMWW%I4d+EY?(I6sj9Y?YQz>sDu@%Fhf&cL9kq;f zjTCy3dql7yi{;D(Ya7oEuo=?SRzG&$uNqoKUQUdv)F4YHEhgiX;v5Q`NHaz#voDxCJQQ;T zD0oqI;$mne5+m#x>*W>e#b%LF`a39rP{U5gV_781WmV6QcPJz#lUNS0NOkB4JwKv`>@=q zx#!ch-;T#u^3C>FLaYdD)rCGTy=fO|w-`qyl9m78LkzBw zTh~spPYyOgHyZe_=oG^M z&`;Rf+R`_i9__NS9JbZGBvwhmMe3`jGbk%JtdEe<0I!FIC9DhF2|rNw#C2U=7~&s| zzHpQS)oa(@3ejS{>)HNLrdnpo@ZYg9omFkRn>QJpQhm73c0L`ham;LT5Ym#O(X(s0 zSG@KxPpb)9MqJbYdFW3J)M4g}#1BSlAh%2Q2MYFb2~T))L&M?Wp(bn6(e4ru#7m6B zk62k{aqSj6D6J%vlg4#9jXw}0I^sD5ok>aV2a>Ti{CF@sU29M9?`aKM=pqwAf1QAS zd}2ag!_C!|`4(wqC2cVBXLPhx<5wWJL7^#A>87gsgUpw6v!MH#>JRh2=aq*E#0byh z^x;3^w*~JN+md&S>e#&acf9q%?0ZDquSZpJ?RFRC`wWjjvP;i4Ug=g)^E!mg(~B)u zx$Y*bdX^O1Hz3EgL@2lt5*ue55y>|VydU&i()(@g+$=D~qi^U>XW+>7Ou#(!ZNP23 zQ*6*vlOtZ4ngR{2o@WUQ+gk_ZDZ#ip%fZeLD4XM)5_ec`K(k`)gCR%*8JHx4Y1zu1 z&r-LIYR&Y^Hg%g zZ87iDG23c2>r%^s4CwbjnMOpHqb_FOyxV}6XA%0w?#AxoS?2hp{zEMWKVpZ)z{5j) zyz7`})3SWgt<$0aO#q0gRdK(Ch3$`8<-(Z)Ae1Mdv(`-KLyC}#;omM>iX%Y;LszZN z4=;d-?(@fw0-lF|_LOmFR#pVex?+O72o@A`$-&5c#XYy);o>UgY>cP~pJcUkkkWZ; z-l)awVNJci`1(1g_vzvC@-i5~+P-_XdURwYa~QNV+@RqFf=lRjfany3k&$F;Cl5P= zFnAi~a951icg5l7XHR^i)!b5xGOGaTc~c2@=p;xi~+lhGlPT zY{U#2dK$S%bX0Y<&y_1W-$c(f@7)XRif!DQS|gMbW&pDs8rKcCuKZPu*hVpk`VfWE;brg=$mzPo^)aJfHv7x>@9|_~LM`lP)zr zmYvN*&|Uwwdyn|W%Oc_nHrdeq*TW?`cZ;8!N-VDI1TXMp_TQ#>^yZoG4S;x2HUlY~}s`y*=4--T&*>xtO@%K;uV6fgm*F#1|&vpTZ(vf4MizdF{gVvKI0V+haj z`npCc=!Hl;W9iB+71Q0gx5Z2w@?0+Jo*%Ke_-p`OP@dt&D1)FjSlRHO-@yo=CI6@a z7^95n+wCK2q*+y|P}EvqKKZjruCE^%7jIL}!p?+>rk?VoMsE~OTiEjW$Y#6GAFswq zi|+cQUMS*np5?7{_j{)#)Ia#xlGOsJE2pB!#yNWAE_HArP%^cKZjoPW5?vMIZw=)V z&^M&3%=V`n*`BZE&Q@&V(-c9hu5msRIO#MO0lKnNIQoGMLjgngln95P5?+`%g&(mF zL-7xZ%hjt*8DBU&AF>TOWbzfF{$dSb(J{r(pR__jSCQZXh&E@yid#m~`> zq3B9|AUWF%-JW5FOzWfjQ1gzmifpM{f$6JFDC}qEQOly0aU&~k_;~`J1%+FPMDoEY z{oEcabTg{E|5dp_sTODDa@KM;yZ3#zZX2$XqWifIP*!`>ljWAt zq%vXU$d@zG;}OvH%c0Pr&`64z5_q9HQH$1tP%*)ap5m6*h`=x1KI3f_YFDQS7pQ64 zZs7?BHP?z)rh1wafW0xA$xQ=4Lmp33E#~YK$8M{!m$S4cX4{E;X5BTkL_pufeS6ov z@4amYI(=fP039zc!ZUE^nt|S#OkB-SnsR+m?XTcN!4vO^BP+hECBRWp`pr!AZPKpz z%-`^&nx%jKZ!xSIx$Dv``APzpe)O==VtVrBn~B;J;{yWvVn5qFo6A0o`$2_Uj@hZ5 zs?^#p%U8`rTZWe66+{7yZVV9Z_gLzq)LS=-d6(U)U7Z{{lC#^?HEXo{2JD^$LY%ZbV`j)xl6yj2dsBq~Y z^ZcYwr$+=HqxR4APT71mwS9S6SXqFH7?>84y{|O=Qw)K7_T`Y zKf%UqT=b?-l4K|C74-s?;e&i0%`2ETAC{6~aQ)ErTrpkTo(gqaKzO>ZQ3ldJjEnbg zvC8voR-vj2e{n5$s5&SK$(i%gvok4VSX^A}`tvDPQhp=*P2qoD??4A$)R(EAyAT_G zF|~3rcB81iw5_#krL4T$3r4_#R@-rO!vOiC!S1HoUC$V=NvosDWt>DZz5oM(V;M_* z%b`_0zCGrb<8#z2d>?3T-}j3gvq80o7!=Rz#mfj5?HsoO5o%vl)`Xxli|4%FR5^ZV zGB*R9lJkDZKofp@F1vxZQyKB>-mgAb?r?mJeIkaks*$3?T;rNt!$PjW_yHYEeX*j% zPvP>$qNSI>g+nTlI?XR5*0#b$r)sRR`V-@JL~uduu9dw0$?~ZrBG6#-)H13@XZw%l zBvsb?Nm5v^Cyjrp3YB};(wuJtgO2;6u}UU43h77e?yK+qeMEb;Y#{WOx~`s~+#CLKI!#Ks|yyA9qUa$6*+ zuGZ2uMaO9rdru6sITD(?pW9|Vdh&z;UiTXgnBNt8oTZ;9u_^GICTo1B|Z2QQ)*--3VAhtoH^0eh!= zU}Dc{sEUvW&O%|ac9}eUgYbjetahkcMbzffKhR#!P1YNuK?>bto1Y_~(>c75YCSAF zRC7s>`x^Z#;8vT?fBVB9LZGr*W=z z7as|$Xk=ofsQrFJ4Owx#zT@Dw^~wV{C>+DXl|pd} zRfI`Z`X-_}H2(m4G99*JQ<1I*nSGlB_j5lBiK47`Mj#|sUq2f>43PbB-*{yp8G&0Q za|T8~X?>oeI$5E<$roKMu(4MNJu$M9tn8dB53d!J3!+O5uab=D(}0dI?nM{<-lNwH$UEn&|TKM)G<|M*Z_N#w0%nb60izeYO5& zgu@R*PqtDiQnk|ze1_z`Sh6+sZ=S83TJ^G$ID8TPx2>3g=;BZ9Wbd3w2Ut6N zdIqk$?ac4KbObr(JM1`YhcboVgk%FvpX`bvc9l@KgL6FtGXt)Tq=jTJy9EOJXHe4; z2tHK0HmTlOHk!{AzCS*|B^iBJak{z+UDRJRC`!h<8z9ACFSS~=H9Pv*nj^oaz;~n%9Ig;zljsMjGILp0K ze+7w(*hpiKo!&3D&Bk&|wNJjEWxrvOWJ=7qsM-{N-u&RssIXUD-e2v2XOSxBChGMHmcY9&G~U z`Yz|)$f$LCyVqN*w?De0-OC62Hb3RdPZw}&^}gR$_cWAwquG|t!>oDH*4o)A;$urW-^Of&Y_rX4~AA8mozkRfaz9b_cS_=9^tn+ z3J+vcSDX%6A3(qM9EBdruIVlxyv4%^{jfZrS&&uR_HXx*hXSP2WW32vt|{?f3HDB{ z5SIgth$c#1?h11Wv#`pn+9*8)c$VzHVwOs-rA1{Sw(p&>*UP_C{XmRdOtL6(?~62L zna4#=;q=OKx~AH3y|Qz5ZKblF0w|_VK)efM7kt&?Yet)+j|3^%-#>pIt(ag`>Yzh} zZfOLs92o61eLkbnrymJx*UH66F1S@3B>O)rj(J(;vg~pX57asV5q!3`=YyJd05Zqo z`e>1%6}h9P(%=c(UC06|3JNc7>p-^0vAIrr;33Q{;%KlgDm%)lP`0!b_UY`mKwTN<=hpROvKgI2}Hvvx{sV1X`Yhu3E2*bh{DPI2l1@? z@qX`m|LT0VJ^2fj2)^~2M;1RyA?5?R?Hf&X;_5bSszMcTbQJ%2Ft_J@^_V72uc(+Y zTGb_;eM0X`ThL<_Hb#!dItrQT(}~?)10qP4O_B%Mu9{hsR1a0_{_%#he#&U}5)j?M zi4JI4xS)jH_m)92{d%%eKtSLKqrt55bhEhYzrpYD{F&q@v5R|{-U@R~Rx53l#~j)Y{8h z<4@gV49CXC$yv|O=T5}JQT2Mm^q5y@uW72#)gg2jFZ2DNlE$a^^zw4qoIC?f zycwLIyes(`)E@Bn6M!ff{6rb1*BOPty+N+gV+S^;Dg$Mf6af(VWZ_6zt7LyK&J&rR z>pW=lhaoH_4?IgQ1aohDB`@W~)e{l&eb)Wkg8W9zAR;#5nnQ-^pcrB?zIm|F3(^1t zu1C??@`uEbh5yb>`tk*wlQc6i!J`jn*AoQIB_AIjyT=3?5~b_Om!E*&VarKv^(sx?uS z(4(e!+XmC-Tk5-DW4SrrjBEh061aufE?ID&{Pp;qB5<1gLw4+LiO%rnf-~RLn>$Wd zf^WJo?zjCeeR_*`pvZj)Vx&JYG_AzK-*gn#!Awq!4ko0kZ_Jau>M#z$KIEv%*Nh_8 zXBTBUU$ZjMp?G390o%He$gy_aoxXuJM1R`>&6SfsYnbP5^+!qi=Hb@X8;jp?UZtrH z#F5Du#?BqRZt?IMAz}C})OeZ74uzprDW-F-6q$%8nLJawRxvm}XNXG`+;k05WmAo#n?Gc|r_dRRNt z^sSe~{-zvXMCv%p@2CmWVqyk^_=~z3^1%shCXwJfMa8_<8-*%;e#BkQUB=X(d29x1 zOP+Q|iTD%0(0bu@fEJ1NRL9r1{Y$6FKk1@4B`2Tqvx}UO#F@kTJo(5Az4uK}i+jg> z&pXqnQ`M}oUio-X=`o`lj{Yi9u6m}3$i>gE)AOxYLm5Q38%UZEOiWDhhY_;XskZ(+*5%3QKP+0xttYys*eMH`*vE5b z4j6{@)#+8#bWV?z@{V$nNAY*|bG=*6nwqm-VZ*W^2BS0uI<|i|gD2etj~qRw&@eUV z8mR$o7!cYA_aGW}>F=dON zoc}xkj^poeVqVv;3b)M^Zrd2>+D%%xJw5oaLt$F<(}{iop{=hCX?*(3cjERqRViO- ztesAq6LmS?`tp$7M79$nzl`BQouKXJLyjH_?gs}d!Ptug^i&?)lf%Y|pOCU6G~wTi z)ZZTvlzl&4BcY&R_WQfG4?5qbIyiwqK)(t)Ta~s+KVrl)2?>iL=oe*|Af}evEnA2L=b?VqX9{K%)K%l)glXiVx$SFqq-Baar4BuReA`eqZ%|JQM=?-x$ z^$TEO^`}c2?6rp({Fv+cok+btRbL_bNA^#ks+z{YE}`>XiBie+DxU-6nJ<(FrX0QG z$>B++;U-incGLm990#69@du{`&(Re;IE6HS(heAyVnYL8g_)eiobI|vPc<$zYseM_ z#5RlJoF#rs93R-`%-0I4N%Cq+| zz6d=8wZlbgEE8*eWUEB0DH#g`!zAJ4r;~?qLk^R1g(3l5L}uZpi|X^`BjrU77Uc7{ zNf&PocBjU3PDYm_J2%c1&JAr0Q$qBR<)h}-L%%pPeG(y35f799YXe7CJiEUC~g}TUeSmbC-6>Ma4PC z!IM)$VA-n2584;oqraHZDL$0zw=AS*r?1BjII#72UA0w>IvOfJbP`w)*w7s#I}ZPBS0|m?^Y6Y<*9tG_D)Y4pVD3&6 zD>@0yfAaEmtcaj3ZIzqt=p;?LP!6d<5B1(9g-i9mABfO&)`@v|&V;DVF@qy)d1L<7 z+hIi63MglkD4{6Q%bkeew(0nB4sW~nLlzVcmqkV<^~wE@FHB5Kp8bEEy$3kf{U0_= zOOsTRl_JV68IjRIAxcQuCD}WatZogF++>th*;}@36^iVYEh9=o+4DVr*X4fR=RJ=1 zectDNuj4+B|F|#we&6vK=jZb|PZ=2u?fLrF(d#)kz-{9{Cb3`ieynawHlPeq$VF7L zW8cvOpj^Q~E=HP*i+>(o>U%Wxvt<-hh%}N@|7Hldt7n<8x3~L8@H>xDP+ZlyawW*u z*Ecx04Woz2I%wgonEKH~^z)*4=8UrKj#*2t=Iy>eZ|9?w;C&vmutl>yOgneJj-JK% zG75S;E*MN`1v5-Qz#iJW{Z9^`A5YnliLQ5qbQx;3R<7vSM}5kyqknL4YGrYJ^mg0# z&R<~PDJ;~M0Xcx_l#JOacTzWd_2s|Rg`UPjA)-BZ*u}-2Fib`KWN+~@3Vd`73??05 zDjCG@(8D-GW@aXbf~?({|9+IAkM8WF?F@o9&xglh8mO?a@XI@~5aD$M21ju(}O@u1L>+d$h#d+-UuhG*Wa(e>!WAY0v#v2o#j#h^d1-9!(yO(&HpplOiaR zvlR1ysxp{2_44-S)1lGa9E)Ck6b1|vwoD-YQ|GL!tfTn(N%$8caF(lvXnJNd=EmUhdiZdU@90{@J*#zrTOe_I+>d zhLx*ZAM7n{z0#58m&pD3(_05F;y2c_#b0~4f-!i^SWW)@%Y=fI^y+-w$cQm3>nmJX zDk`dzCr&iq{fxpo0#oC^3bMy{?>xZ3;2+{-aT?PIcwn*S4Ra`)VGOi1i{m))`CI?y zY+J!035hU@-j2D&YO4}?)5NKthGSr+Ltps3s@TK*#ydy95*zBiF@fR!E6iVD*2ijO z7bNx>Yp}USC~Tjlq;Uz7FD=VyBUjbOet6`823B0V{XNu!fBvuW!IGb~>bc*qmGPHZ&E6|GwydgnIgaIpLs91bnf=Ao#YG)cjC+N?)U?*v z8uss1+1+=)LZ4`=$=WxhK6H%Xn9q1`W!%NF3OA0W>O0S0JbhuJ(sV}q46kah8%&5# znq9-BY@Or1mGp_qEXJOsqGN|`l|K%ZKg@s7lVLP;Wl$u;&BV~eV|CLC?JA8{%2wfM1>`p}<#mwrAq!>*Ztl;qe z2(8Q41hcDmT;M-tcReY4+V0PBi-h>ZPyE%X$*Fs2i%zeWwbSIpm}^PjEJ}+?oA*5# z>ge!uC4ns?V^v|@QjLM7d)?>HchSF(aXTA=Z0P+NS{qwiucuF+K6~c+?IAtL)_7G8 zg>Vpe#`x|TowIH(eHwK1mKWP2^!xe^=&*gnhfaZ z==O5c)m-SzxnaQ0z;OAR3F)LJTbym&G}IUKTY&5M@b0aeDec$lR4HvR&knUi~pFLTw>4c&6+j$Kjx>1^hxmRTOD$qbQLQ z9Bbc)hPe39#6XlwdrL~v#AQbBT$GysEq*bm7X*&L8#vow{unj9aa* zK2JB_shZ>M8Jf~rbb3~5&!2xAS1kH-Te&}X*SeTLPOqA5@5*fDUf-<*UOT$U%WrH-h{)1TKw_O;D4p0Tq4&;j$WHe zy`(p{95arU<)_nWG`W2+^&stUTAGJa(M{T-j$vQK-Yx$8{rB#$9gnTfZGOsH$}ycz z->nQomKt^}l(cu}N2-Ty>!xRZNX9LOHNA_IezG`t`?rZkcv4!F)w$efzK4(yQ&+AVwwYB=>CeXl+kL&y;0_3+0t5E!zHF88}3bfPyABt|zhllbi$)=42U6kY5<>n1{igZiv zOx~bP&&}<9XJ_8Y*wRE+9Vc7W4PZo$A0BoBTdtdST47{#}-cf9td2%BDL>N zAI`=tmN0chkRRIZSksuYQlsvjGf>o(jmXfyXL`# z+V_%N!*ehHm8H>FZF<^z1OBtMg}HMdZ*UY*o!B@9PMPk4f(+Sfwkm&A7b6sooIg6I z7cam(UwxjV!9=*jI(Gv0!s>6nQhQiobjHp3&qJ%8tLMT4D~BsD`o2sR9dTiCK9|LG zOvpOkR{gH}yj^--ibT<$w8d4;c!6JoJtntDMqG=R1Lj=bjr@7~m7!|uk}$7Cas7*) z_2KJ3biEIFGY(IuZgi!_GrZxGuUh%A!<)YMBV$oMKf13CceP|D39268l)Ux6c1w9+ zxsP-HR>!P&AC%1wvl-desqI@SxYNGF<)quzg(kBi-OznfD)Tn@i)~ItGn#2_62BzW zx^_&f9_Mz&VC+yI`W%u5+joWSf%RLl`N*XWC_rIaHnl0`=5fZNHpFj2SGVNr>tfr*(bhyR1 zT8H}{$?*4DraNos>I_-jS5XN(XXsZ92bgq|Js$DHM(O#V0Uqt z+s)!P9dBo5vwa<%3mqnLAf9}hivMvuzoy>Ap3Pq+tchu=GQ*dHjx*cQLT}(v)u!(alnsiD{L^+*yZzR=lsz_**0#SD z=;<6jH~sZ(I#QdqRlmf!E}5HU=^Nh-y<^+ew`W3kb#`H4ro}YN$zvs$r{s%gIBhVi z#_HLc=%%0I?;OR$?*SOrL!_wJr%C47Mae>&fi&H#)*#TNv|vndq`_2<1!{cHLn{H`S7Ag_G5Xgp~DHpu`{78!_jOA)*U>hvnD3R4*f(jtm|&WO`@yyg2)gyFd!B%S>tJ^ejV@>zBwLZ`0^m z_G{;6;8Xp!eA|6NHNIgWe7~OK_3rt?zl)7Cf$D)=c2t52X(`t&ld@A9dasx*4ot9C z?-&wpHDF>9ce>hl6f4{$V-(HHqQTb^qaI`bDBpEsTQJ{rv~f4PSh9vMpBFp%H|J%= zl;r}G&sGLQ?Ni@4RdntCm39@p<$6;1>WpPdSO|p`^POH0wk}h;f*nVg|8!F1)6(O_ zyfkOh((eMr3g5Opds}U4%ibX83%#?dbuq4C{^Vj?)9J-Z+1P+@j`IUccgsBv((sEH z$7?1!#{910tDDX$X<^sZN;4n(-LkxDzv33_gN5Ltn%&T)I;_svF!krRz0|n#!#Djh zGg~;C943E7hPjO9=4zYk>^Giu|G_Hww7NSq(Wv;gt88)lQ*p7;u(LX6&4joG)V$9J z7^#hwcaG`}FY%WYJgsgEZ8Xht`!N+P_V~N!r?M{#>k7TS;1dsQd0H_R$dq`D_1P<3}7n#e3cqXd==FnYy3ng6;x_IFYj?R;$Khrf@#f9@TCDY`G@1$qs+af1$`#op0t@TH;-Ad(a@hcmTdmfBA!NJM?ebifzeq@`j)IvtMPq@we zG1_DN!n41BI7*An%7=@VT(*@JJ1?N5LA@!5<%ERwTx&qeUAns$$K9NsjfKHCIpzN7 zu8z}46DbNXY)p-Nf64FCzt-t}H*9VG1orbx&QG4pcov`@*kbpj$j3!;F{wrDi&hD> zB}=L!cbBlXSbX|sy&c?AxnX~p4e#eur?RBx2I?Elim%LEzgAJR`R5YTV=t??{Xs=q zqtTqDo$c*4phZCCBqk;Xvo?n&WdeG&v*rk=w;>Cd1C%H9cpw&jzeF zOn5h{GWm|~hj#$0k9U5{_jtQ0hl$j5u5d$+y`0=suF9^lV?oOfA+?U(T?t*GF5kbr zBKPS!6HdF+OjWnVmqFoKcWo5$Pj!pdaa;Hy|UH*kCuKJ>;Fz8kB~F3fwoYE5a)SRu}zY^*@?ZfW??Z@-_-T^lxkod}EV6FoC+L6;X*3#5VT zc8y3{Waq5!q-Or=@Y*xrO3?n#;u$~w#7@(I>W#@^xP!+DPIMwK<)j z6fDX`75kT+c-d_7%gTOiXqW^yVz{7T6~Xjj(YcqNK92W9T7xkFFYNU2mzTiHjo)2v z;GJY7lI?I2y&i2{b$+(>9n|}F9UXM)-aj@GFZg4jGuq0^#$EIo#c<%N_?U1JHfp=Ku7Hh zX2%4Wolna+=yj)Tm}BMF>GZcy8JD$BElqR2o%`k6mk(9c*4YY0WWcLm2;*4NN2lYWw09eTDqkv6Xt`` zg6c?RFQ@B09YRkT8t%_&>W z&H490$E+TGiLzMUhhgfW17CzMh#h@q6}tEDnP1z6|I~6x_Y_VJGwLEBOaEmM%l9MZ z$Me|>S5`)jCJaQM8eVvGxW&ct;%(P6*N(fLZG6;XlcXeYOpPfh?5gd7qVK~7v6lnZZ$q@gxa;Vl zF1{szMtmAM%wJB#(<&Lw*4oV5aJwcHdKFqdUzs?3u;}zFogf#s;g6}!Iu$1LXn?6`y^+s-k7j)TU#nLoeIx5=#lr69%|shp zQH|G8xmk9Z6ozW#4(ChF&LrekCshT9Yv=qL6l|(BIQi6v+A|`<%8NkxZhn5kaqb+e zunv27zwXG)d*9E}5mjQ@{I20P#qRH0TJ5|0>?@=ln$MOGbFy8OkwBC&G5@em_9luV zMJD%r;)Eh!w%O?dZOEs1bvZ2Z*Pc6)sX3|JEcK4^n``?x*!L>uY31!|b}L*O@C|m` zE$8bMtK|MKKK;MSUD$$B-?qM6xo z4>u!?@b*kfF}tmQ+he)gygR`0%}{G?q^NSYZ?!kEu|E<}y#cNEirJu^QdEFX1?PMA z{^gUsy|>3>J5I(Z{g$|k`q|?R+tXV*7@344w>WHx=(St9???5%>-~krlKQcS!j36h z9Jh%Uxw-YOj7GJUbdQgr9He^t)Cbn^m^&T!T%~_4{!nS^Jdt%wB)=p7**2!x(e)*a zMk}J{%ZFoWYxH{UCB^IPs;a6#U`hJ(2lQ(J*=8oGu?#^d$~V!2BaRDdI)OtRyI@$@o?K#+XYJgec%1Th1ofvqFcl9 z;#c9*GC@<<1dlmK5_Mf2?)(6)h*8Vo{r$Uai+X2eW}d*%E|erQEq4X5p<*z-$;I8y zeNv-X`?grjiE{a2wN=46IYV{B%FJha4>le>-Ufwiz*ju(K-PduuY1*~y| z`Kp27l21>H>l&e&6eyRU>^>~=$A9msWf5B+`u+P?`aJB-_qn&eR$@xe4h&=F;@SOJ zd}TkN|N8A;O~w{-$9g~2x;}Fi6{k8;FZOmtxRFtwJ*1+Ii!-UJh5eJ*5ulcjOegFo zES$I7rQ84b-jY(Q%A$Spw)L2P$F8Akl?vatrpRg4jsG?oZ+~&ILe8M6EZalqt?B9Y zT2)+@Q+B&)FW#%`u$%VK`3L_nx$UN6dd##SdZ`4sgv`g5)@i3@^{yS+!rY~(;MBc5 zUO6{V44T%l&Nfh8tf|kg=f#<~G%E4s?fxkQe#Fx4Y)$u096RApG-P{ksICI=7o?PSrIT#BQ}+cUhI)xmK)4NcpR+id>-{(BU2ogY8S98Z$Ttk?r`F_4VCs-jp0y=zcF zfafRh=mz0Sc=4u86f+esUL;D4o&kY@k~lWs>Ak)yZkZtRLwxMww@iSr8D4g;A1LZj z;3aZ|?8Q5G@22P$IT025KVbYugIRmBU_;n3a&7WEQc=i-Br&5R7q=1tw#O^f5PC@J zKW)rVSX5qi+SV>6`eR27@fS{I#&DME6*;NP2r(+5n!GG|(&3k}fdM-K4RY^)5Og}q zFULamg)G5rKymBR?miY~W|`>xd~qJ0q?HJ;L#nH*vqd%-t4VC;+q72Vy#Orh>eUyv zZ{1c`7VcGi{CI>4Y;X8&dp+z)WM!rJ#u7#;K_)446WLHj_@HU^=g%*}2=Do|y&b8u zjQta1%6-JzQ&1d7K^1KIRkhY=BmqBW9g54CS@n?ux25;9XIC8DUxC6I9QgCuk@J6M z>93O}<|)3sR&{I#Wg|$-i|`;AP4i(m9t+Ap{Ig@r7NGVQ6%gba%XcdkJ(_I2yT5)22nbukgfSVcG?qsGV!1YXA8C+fvuhcxE%v zvQMGUDXcm|2SRkrqKVsmCuUtGB_u9fZ13!(VG*AKi5-Sh198~eWh2|!NvuxpFV&uv zGTHz7I*CzLp?%x2*rTy0>pTvnFe6yg z!3K7OpMS3(dRhrZ9Vk8I=r|1bwf3b?>=pV=;}s7%6a5LzyQfq|e_Pq%I{e)Iwq1n$|O z+I#T$F-qzOrKNj2ve5v9I|3#}6?JtEwr%wMQxg+AX(1j1B!U;W33gAxJr~2d;}8_dc6|YhO+gX=nzI!0 z3A$n@FjSg;?C8<@va*Bl09;(3v+QJLA9_yI(meb}&#wmy4IT z(1ATg1mX!Y5VGJgsC)3>!PBSPX@Ate@!04kdz@UXp?DnKA?(zc?4hzWb zgMryB<+$pa8Wthre}jXA)6xj`S=pQDcY#z^SdopJjnS`6#!yt9p&e;B#(QqTRhE{*utw^;AU>g)HRKY7Qdvkne9 z={DftrKP21b)Gnlg1ElEeu?WcjKVPP|C7rNzt#+IVgk*?)|4xw0|N#ff<%Ry{anah zt*rBFK2K?AkJG7*IesKp*}_OY;kMY>3$u$-?o&R(_{c>uz#-P~Zs)tC|9bP$_-!_=Khdo+`Vqy;L=@rB{~`!QehfiTj2wx>!*JYVhr}-!-wb`!f39W zxp^;mWbvAe^Mry(;lj@(ZZnx}H?~js`L*(cv66x!B=AvOeXd3441BeXgG2Ggme=J5 z2T309Qk+WXC&+R9hI)_`6eD10J*rPr0vpAUOep>hws%0B%u2eEiWI+;XU1 zIemF?>@hWqAax;7&?N{@5M+wDAW4ad$!fNLKakxLPAAj5pr1HgAlk03u1<3KiTzpy zr$L|rJ$*24f`+;}Olm5+osyG(mDB6-3JAC%4YIN*s-asuRQelMLeQrG1-v6aiqJ4j z=wUs6{Mh#AI65)>dq5N1nx?0B^(rL3{2(7ePe)gT)E@_gEd%chQ=*x^a_eKS-UA}t zr=YO**gVl@{`KuQxI8vNq!x%AiAb@cLQZU&U>NrF@c018Yd@~x00%UHI3e1LRXX7KajSA+tc%Qq-0?Z5Zy z7;QTUi&az_QWA3+)*R~A9VP2a(2WnhiLx)15_5eXXk8cj%PHE@o@pc-jj9@`Ut`BK zR*$zJtm=*H+Nl)}|o_CzqoyUjkhqL4i}@ zoJ@V|*U3bozpPyUU(Hvd#KOI{Og`gA7yy|;I83ObYyG-IhtFMBC11Gut2?OfKxgjUxk{}|UiPrx;GQ=H zo%AaxTbY;;8cHz10jgF-)2#v}OSa&8AtWsDH;?P|r^d&R;A~cxgZ*T%&TYkRjA1Tb z^^|*<6kkF@V^IN-jH1Kr)<325dZ4?9@E)xnST5kOT)XmYh;71g7iO#h7=^HuAm#vn zZ!)H1I5}@2z5&g>@6)GCIDj&TczMn7r(0KcgzRWdC-vzu**EpxCw3C1g7{H5`kr4# zFB7m9>FMd<3>UvMp?~dKJG=n8mi40Q5U#v}+ySExOjswVDl#+wrX*&y8yrSMF$G1Q zim2lTsl$hd@p{F$B>1hCKRtNM>*~hgG>O$f-y%~}?}rZqHKM%|w?{5nTTj%arf{gp z^{r-}WhH*c>+3(y_5D8tVM0;4BiWi`0}v>{ybr{iZXrs+uk=RuXI-5DP$2;UgsZo0 zZMSaUj$4v9npyy6d{ZMMqcQ(bR>KVuPO!AHvXX{LudXy z++4LQ3b$haJ3wcKHW%8~gHFg`z#_MslMQ8+V>u(Y^U*@)`3X!dJ2rV-GmI?1&O+*l z%sYq#1_s~LI9te#Ra6eL!BPJ-lTnl*`6_p&NX}BIG1a%WUXjtgI}`|KVKprG)H8NH z)8xt|7^^HT%O(Xr-JX!Ri-~CsgHVcCph&AX%_ht|A8bFfm~HgST*xjoJ@NI8j#G46 z$ZjBE5T_*7l;2Q&spKU^EXERc=lWz3Iz##R`^P7{5s)Gc>+C$q&7FwsqUwrW>GXyT zpC$*$!JP4)%yrI&hC_fO2q7B0NAn4+hp^kq5~1$^p~J9quQ#qmL&g%wtq6L37ROkgfMBk*$yPzUC$iPeBamGyI{8G9$JqBS;5kdS~3f? zG@y9O%Ib#r2AuwC8XCxdeD)n<6A{S)${Ccz!1sXMmjJ6-afzYt)!pw597WJu8iho+y`HhulZEg&8hlw(m|At!iTHZ6Cpn|Tnw34BUb_aV)>Ea}6dt)rt3 zU?J4UXKul(?0|{Mf03DlFlA3(_9AZN=r{utQxfy-u@3cekNW=f=gh?U74i?EkiPM8 z^N>DD$w&v=Mz5%=Ps4?IBl-NK%Kof*O4#SzJWgtH0 zhJiU@vsNPc{RH+JsQg&N2{Afa+uEA8rG~$3iF)@Jcm=sRTG&O{%glY~3sn!Du4OX`OuMagvF%#^zr8G)!jPfpQLOEmcI5ja0% zJQTn+wRU3vMejMZGFVtFCNI}-P4^+M(=P81A3~g6WaFME^KuFNw+~sgeb<#VQ|9L7 zp-&fct$-k5pfK)Wj5t#QX-8yXUY>|=5wa+0Fle@QzQ}9;d4eCqI)g4g(>Hb<#LJfKRFMc9&>@RW{t8htlSK$pLD}ftjdCwEJ^!sbTYZjfa4Rmc_8}*Q zaLo)hl_(&<7R1IusZP|YM*_GGdKdPK-~RNAt-*ioa353KGzaXr7lJVW+^c-ym12z` zaT`cCj<^*JQc%k^z7=R(){zD3sn4JN8ZWQqUP>?izd(aQ3Q@Gu(zJeALjU8(Lnxpv zmUnN~l9mBFfY9r}WbmZ|Vt4~V(P5>KLu_cSUWVLb4fk$7Msn`*8eREIXs!(OfURrp zwF9$KRT;u#T(y)9CGfT9VEV-xU25O$@sjpC8< zF>_=>K#7#Qn9=(izzheWh6qmpE> zjGSCz0(HtJ^3nSl$5{&9o1@awdNQ0BBM8Z{g{YJD`xM+S%BQdSN3{e@yC1Z~jrNkT&}MlhcL5%r z-F{HlEYh|5;Z*NLYPtYUa8}8Eye;jOXFzZ;o}!-kIX1GJlf)ZPu{s~+=2no2aQ^eT zzyF3}@PjK} zW%uE7im6=)75wD(mDxM(!AAcWz6*HA#TK|R&PCrCZ8MEsr3OAul#S2 z%OkJlafRW;@IkeOXZSlOKmTR5L-X_fzb(g1iIsf)_@NAPEJjQ0d+7@9N57sO1Pe_r| z(bH>TcFVN3w*Nb)p`m}M+1jOgf&n6+G>I1?e0_YfZ%Z1F?R!AFq2h>LfDsBnSHSZ^ zs7Oj==(2BSXXC- zg8?x*X~cbyQt!yMVr6Fj)z^1vn1cr69T@kj;h{@e_uo%)mO=yQF8KOl`2Zo{gXhcd z;Q*)VhC4!!{;xK8U!dy9{ZN@L6XcNl)D9dEJjqG;HKzQJNJ9`rZAC0fs(VqmGa?Pp zB7MAr*KHmrhw1i5AMRK3K31xsp|utnC^SDUZPP!x5B7}UbtT=Z@v$)$I=U91h;l#P z{1X**$4S95^`?^|KHc?B<9HJWkK@AgnKjZ3gC*LB}#r7qH z_sAYPeAo!0oru|bO592qhTaJG3zLHTc%`Au&f3~tz!^ecy-Hi!2W?&8J)q2h9|0F8 z&_N3bzM_uB#bpBE7#TkZNYk}zjK8e`lBNKU|38G!JJy0S5n8OW!>9vX7(h$aoD4aq;%=L^$y?NKY7GWu*{HPuLvdyqZv5Y*(``=U9Zw zMa_>y`DSSf)0{~vCnsF-r-!5_kljI0kWdN(+?0b7Cp-_pg{ZYo$T?6_q+HA76cyEw zxm6d@jid$Qw{3YI&!4NJgYrxAem-@eMdBPli=rN{CHt|bmE83*Zhr1B_r@{=vI0yF z;?ufv5t^$(&6YSn6RA1{Y7#6dl;l+Rk_YBMuUxsM?YTdX0SV<=G4BjQ5%9(0K!0(5 zE7-d~{`s!H|1&`&7OH{t*(1VeGY59ipH>>eH}itu!DEv$+TG#uc9b z8Qw5CP#VRRksT;v5uK zW6)Zo^CQkwJGJ)#l3{R9c(mRb!^581sHsFo_J40%ul+%UJP4Zjjce+Eqe8K1Q;6zF z%?zkemBLOO6BD~Oe@sxY@$=^&PT_S>F}-?qB-mu#{bXz{Cjy`|RZJ_{_i1V$4KZ3x^AD z-Bsw1{4Ni{2zq0fijGQ2>B=Z5Dq5XU$%RgfloV#{e*9?6TAd?>y18dQP!_{6zCxI= z`=W%3wzl)Otr=`6Te>Ze>LGAhg=v6N-TcY%!oLt#jDY8?RTwBCZUPRgnqEYBxJB8G zWVQpR8XFG@2d+VPS`P`_6IsUW{qJv zpO{Fm+)rqu{N~5f`ZLK< z=04%Vtu7XDjzCp^qDWGnWFihSCRETIA-VUA>IE`*EQ!w^Gfyfrn+Ud6$`z^SFf zp70Z=cE9E<^<_CZ14J?OxY0BGwCV4>foCX8T#S&oCgTc5V_a#=uW!pbWV6}Gj?LXK zgu+eIsi-(P`O80|=36l`NpzKyBe>?^LKWZDBxgU zbx@@=YirabV}+&Xhp2~9BAFU#h%E>JLkV`Xr?pk&;K73s?36)DYE%=FjT;XW-C9vm zAvmoWC>Qk-z5^+FAnhFf%hiI&y{Q&j4L=SYSLNo0gYRe|*3?}+VfzGO*!_FL|EFQn zd4{@Mw{E3gP47ori2y(U3N*WWFiZ+eyGL+Zgc4hbPitzVdcA3R8$2_n*8lNH(IHV9 zUkwc5g36#PP6c{itxDkOQ1E*h>qv`MGGZzc$T7NPWe0skd*YLmv(CRWwpeQ{*(yxw ziZ%^EKcoT^P<-t)E&Lz!e42-d+Ax^7HFVfCkCT%jiigOs385N#mcqMX=-wieKz)EVAvyWNg$rFo&6MRNziwfCv?JuFk!j9D zvR!|U1?;2J_h$F^+I6qmEX{Y9)iT}6VGxP?a+kK9!-mUNU54uMHa{MgWwtX^^v{Ku zo*ZG5S9p@jd`2wh(;J>R#e;EQkD7}d&-AhA?Pcw?>I}=7|5e~J>C$DjIFLCzyF6d0 zcRW9JczAiGP+EEGR{!?o=ZDA0Y24SqG75puA3hl13Q$oqLmd4Qz+t{L@8>rKUjpF} zMY@do13^xKO%vg!qxMoKl_U>*P#pOSt$D&iT*j3KyulGDpx1-Lv?+$tpX}rhBn(nY z`C_3-8(K0Sq6p~a>4x9V+!BU@Xyh&4?K}4MXAqxD15aj zn9%o*59R(`iv1p>V?rOQcZ{9g06=%8FKY=3IZ(M&;(T87*ga|ua(%LH37K>z)N5N) zv>eAeS{nZ>j+aQIjD@R`>r7~a61STe^5on7jPJzN8(T<;Z{@9{rRXx59c?2hwEZDX z%IkwAK}Tqn;Wm^i5bbO@I*v3tYHtS8gC3pRP7s}2$d7Nt7``z`v-~loGY#4e z>6xzv{y-xP3=9;naxkxDQcpL`p-ckw`r_png+SbD;glI1d3ij^!>lw`U~$Vz?CN3 z*jMJkVR-A-@Ak~fTO1!ICm2a(xTCxJ&_;K$x{R>Og%v1qgh}4J(&#eV9o+UcUWL87 z4E!WMZJQr!q(@3f1cW619scjAJt^au@<)SIRj|OTw!FK4Z&jbZdGn@j6o5yx`9yEv zyr?2*PJ$jw%kXq4U-+XE0wLSKjBaz^sN{k4Cf~UBn^zu9W!1H{L3{S?yV|Jm?dw+( zxZ>_{d{>`YOQNCJ)$|(=hYs-ZSzsCRomGmW@q<^Xd0(;~ed;4m?d~qfHZ(p;gBTIEA*7)$iK6`DnSBw@K!JH2RFmV5Ozi1 z8wfAPvUs&x4Ap397ZWmb=x*@$y0}p`DA61kS4)A%R(Q{3KfR zTGI4N^k3E2*FWCxbVNj?6^#meW8S|>e?T;QU@7+X@u75rb5vmFdK<|u1aE@|`%j%F z-yZ7PzFiA!em&Q<*8X>@%<+l4*n8Y!RaI3qIsL;|;6o`gG<;YQDG?IsLJQXp+TMSqQZBuD4~xsj&dxzdNPuTA zQ$EGon*7qwzMOZj`|@R}&x7U=`f@C~^3u`QpfQV%*UzwgC;%QpTnb%=;J26D$+qZ( zAUmZiehc>9zrvSwd}zqN$Wm8V7n;hk(YTOEftmPe-|BB7?E*UvBn@hiV#b)i`6h{2 zMz^{Qlz2O~Z`gD5<$ZF$gY^Zubf) z0|!p|x0@5aKHntaOJP$v<`);4_H90LZ43}&r!2`pvV#qj`jGhr_s0)Lx!%e4_zg-rIuc2 zaHerKH&2cRAO^@1-A{m6=$3?V#Q;X&rNMFHC*<(!>hsTPe>5q{HvI9>jbB-C);{FM z8=xj&Cqw%R|sey_MPCXjke^iH?@+Yem-PYX;ytL)iV`J#=)?TTO)1 zzFJ)KmdqcIPwy6pa!#-d)hhdQ(p%ei_o&5Twc| ztf7Kh3(4lp_Fv=Uy%500NtF2e<#>?E-O?+($0ycqK)ej4u775x9nK9byw!74_xEJw zK8nYN&RZlet@WTX1LMlwwf2UFH@CRlxY3VRWvh_@fYMYOHtgHC?Z^><($JdzmIuTG zF?;z@td>!+f{v}bo(nZ#LOc2B-b??+#i45>BysTO{9Gv2>o%$-L>Aq%?|;B3lI2LEEV}dv z-T_{VKr$z!a8v;}yg+CJ6JXk!EN1^x?d%Yx#tl|dDgIePR!}{j+Zj!1M8Fb_D*!{| z7$dZkc?SvqayQ`aHtIX99A+$bHP8Bb0`-PUT4vt0wW(Cir(8icH}UuH^V?Br;pj4Ew->q|oNsczofwDOQ%_YPLd``6w*dj`kHt@;*DeY-=h zhf&=;REmY%@YaE0=7St5Mq`e7F-2%?T3;g}-Q%lz9W zTOR+wWks8k!X4UDa_Og(l8&Vhu41L(9BZKU3)C`p^Vf zoW8d2SG*$bzrkICF$q%8!gW19B6KZp~M z1q-y??=xOA$Vv@$rFwN}Ozy!^LjD|vdbpDdI`BgL{0=^e8@aAkR^}h@ul~3=aqeFy z@7q3s4aA_rGI+MUg&%n<0$_c*V}tf{@PUzCr_Da?GpjJ+T8#xzW=ERKtro9uVucQU zG}WUUs;ID#mk+@R(fx!@3bW_7l-m$w;@4XX$*o&LYY{?NCVAA82^n=l*95H#4<0_0 z55diVi@@kbU|3iefE{^+Yly+#8h$R-qm1AnZ{%Y?iV}EJq&M=2;raB63lHh(^`eJO z`*M%vU&I-q!e&Bs$VCEOs*CsSu z96BV^NW8qi+7@k}wP0uu9D$q_8jtmF07*A#2!5*KiB zejYAeu@T4wlaTQQk~_@~Od0TSiHOXgufzW~niY*A?pFL!@z}J({4)Q)kJlV89#Z!* zy073#Iq=G9n%Py2bD)aK&&xw_fiNl-b!LQHy1AC&(I_IlMF(h3;=Z?iWOGDauJ2B> zb!$h4p(3miWS0IiVmp`7ub}k|>R7yK&$eyb7AkSq_Vc!lsbprnzDwHMUoq_3m7!nu z59(?-CWFG%Od`Az5=+S4QFA|qP6yoI-a7+3q@|^`0=8;DBBiaRZ}x4OKr0=d26gHn zgavzc?z}Y71Yd}I^jV~ZvWsPSSI3r7;J(_w`SWq zug~q*0(p2uHR6=p$)$iJE}LH6zj*$3qW*VmG=>zNd<;@}QpH#!h*qKg(GF0T*HcGmnUcaG9ba@fpnojx`sc>Flg#*kLB z;`F02rcjGI#%Il?eGs9Pl9Y5o#EbY!GyO(6N`8Mk!o$Kka24U#iNDRco^qplDla`f z2L~6AqFc%gabes$RiCVLHZ}c;+^-n34gkscj~~zMJBQ8r1(S!l4aB$#R}Nb1C?h`y zb|rrI@|!m^ef?mTkA`t388rvJ=% zF2Xz4pUnw~I5ujb7p5D}hwr27xL3rUvi!r^O_q5QRR8$FgWG_D(1hjKZ6BPnb)Dtr zMH2I%^-j5Ug(W4cc6E|%089{4gkInbZQHO%&-(4hwRA}T5UF|YyFEq4QL?2z{dG=q zB9}6=eF?ZL|Fw)ka2>*+0NxpEjHw1qqV^{)fSPk)E$2Ybs@4uhp(9BO2g!w&6WiA< z8TJnhpsu@!uMTFL1CK@eikDvenUse3KEqR2@sGS#=6J$qOR@3VHFnjXg2MbDuI{E7d}+tlajf zSt~ER%0H~{^76if2v-ZzNW*XwXzCj&ZB}1Web{<~`3<9Nt6nL2+-MBcl`N>DDIMJc z(9Sz_^5i`r!=a*9{2m0Degf?@>kn6ym&*fE&#;#jRh@Nx&V=>l|oz_m6OrXr| zh#!fjNwg^80fH=E=4}G@M9#LldXTUZB><2~6yccJh@V5x^?u2YiK~XMfkGm%?s&}kOEX%u@hStLF#Mi+q^wBZ2uM5s_K)TSRYNT6+2OHhfbh*b(j?kJxd|di z*fjLbR6TLz6K(j3(ImhAD@6?+T|Fpq3$0Olao)-$m;3&*74voZy!=`M%T9bq-1w12c_s!vLDEmWG7JV0bS!dwGi|6zL)n|p^l2_6|0dw zhY$bW^ZNF~hbH5?{|9An6;RdsMUR3gsDOadf`CX1NQu%RBHi5}oze{kARwS1-Q6V( z5=u!+cXxN!o!j%fk2n52oQLCHYwz`~@0&5^m}91Ac8rhz&*0Djh6iXTKnPtD$l?%0 z*>6N4l7oNH==Xb+$d4-Ta9HKFfU?<5#M}B#Jbu%;WYit(@W6l~bn)qScXk^*;#Fc9Tpu(l;>um|8gP(68a zuszMc-G3L49wHJ-R#y2waGzY3W9+d5cX3URESj4}YW@Y-w6_2^0g+`08_A*R&H_RX zFuJwXGGWwGKt3bI!J(j|rl#iQt^MS4Lu9N(ncDa(JA<&890SlH;tVHi-0{%;jyVb_O+hCTscu%V&hx7*p-8N|SE zxZcfuf~)k~|NPyV4Q02A%aAU6^yJA%1uf!>im3iy5u}Ve59}8{KYlz=Uq4pXPZ>hX zC}sUqZ~F)0(gsHeCLu`EK#-3WoqHgBQoje=Sznht=|wP^ibdWNsf4G-#$wQFU}Y7< ztx|uXKi&85N+Y-uyd>nA5&^-%Q*h;BHKb1N5NU=&J{u8viX8ipPjhv3ZAc0VX&Poj z12ezcy9Q@|vBqwy|D(^2eExOZ05Z?w?VX(^Pe@Z!^Op?FL#39E8-bAae<>M9 zv?Rdkm19d?XLIN-&YGuZ$DC8LHbsXDh@Vo42`MIo-^9GGr3erU*4)q6H<*lHyAQ&H zT*d6UmepS`zwqk70qfmUFQK&Ua3-Y1K!?Ibcp}A z$YBny7cWis-bPCf@vb4l@tKGv`QQ6cWdz&`Y^qnRA_{KeqSW@cZ)eN=h2u1=^1Nf6)J9DvBcm&Xtxd z93Gxdlc*SoFLHo!rx&raoQrI2+mqN4ccH}tE0hV)8VwD>`Wh1x6o>!|h5w?W-Y77g zj2C}9m^f*&c=s_V58g2lCMEH1s zk`RrO9f>IS<_GB3p@r37`oEVP_zg)1N=2~cVE2og0kRBnI~dEkh41xb36ShS2AZ0Z z0wIvq>gvWu+_!JVkZuASWD_D#Abq=S6u17Yt+hO|YtZ{bl08)$9&5gH@K-ig`Cm!P zuo-at%mi0@N=ix|9(M?S77Ja|M&7bQqPYrnWoYz{E1#cJa!2? zWIN9h<}~DI_(Q}+-ueJ^J8g{Uy)*$I4nQ}^^o6y3kB^7gdhKV+ztjw2o{|wuj`R=k z2FPHs!D4i*hEO_@FWzZmd6@>1EeTZLf5f#07Vv^#^ead##!9ny5+I^2|8((T+w&SQ zeyYI&t?`Dxh+84Opa$#$xJ=ME845b|2yM4oD=2kABixTC5EnyercN9}#t*K^Rt#P< zhdi?``nz`lfhA5ji0f@Azua^N)hq}C#Sy%22w#?gGtumN0-6_)-2qnM3V?7ZD%K+s zZ0xTQ5zzadI0^X>pxJoziYLqb*QQ})NY9c)(7|EGbN|nW;>P|OMs(*|!E!-w+cl6w z5PVkn7HD%uB>2=}bbNdW6bGjLL?l27sR0x{s7#^n4bfK5eg(a$@G(bKm#!pxXUeYcg(f*Fu5B#8{XO3 zb#V7KHyc7c7Lk7FUE3Lx?sXwdBa^oDclSX>h5vkQTeF*245+K-7amLcDT7{qAwO4Ncwp zcevy*n38+u0VP$VU|xkLFgP%kmCnw8Mim3o=@V6OS4tHXA|hJ_o!|324&gf(HCqdXXZ zm!m2t5gB0o58IyT4A2&;hwTTEuviY1f?!U(c2W8Xyy?Gn9&QFyH_~$iQSw!j&$s$J zqwiu2#9EKlkR$d*Wkd?f1w^>uV`w2F=)6C(uO84dJ{}A8<)KL@^Ita{R#S=%l!{YO zvC5qU5trEDK|~gM@K^Y3;X$+{F_$7b{>XpGPNjG++YYc+b0aPWcZ3N@F&8PX?{u7PyDwgu*jO~Kehum1~ z&6}dOTgYxK(d_0C)s9N?pL&?Lcn60!I-v5@9~LDxeDhc?-$|cmZdx z9*D>qinqUW7#n+BRc%#??(Vwo2&Siv{I_hhV%{M6u?7i0(3OCMV+&jI1DT<*aSvci z5U@c^4V{}p{((2!WBmN;E|sNXRw||sWkdFEaoz9Yq@tpsfsHV3xcQyz9Tv+(!5RIUtOE)f&v@D5>MF z_aAKj?>ExXiG{Ykg@)9pKQo~x9#KYz&QC(Z+m$ZROgCBknNLj2bq2XBT7mlCnY4KZ zp{{^{H&nyiff@GO4ajLgP?6X>KIVLeZDa0o=>W6I-=jEeMAiQ54YnBv4+-|y>ZNL_ ziQFtN6mQL6n)IdlL$Imwr#t@a#6S%88|CZ&ZT1S%|If1Gj3bB1S6kbGR6ib`gJy!U z(zgFLd}S@+|4EtgeeBSOTW6~@N(FhV{Tvoioko}>SreLWk+PD)h+5#&ZBMdix&jboS*o(wl(hZ%n_DQ}r(vD#?oxY} zlcO>mOnSC{=RRUj6d`{E2pXal2wjQUO>|G6v#}|F9)vm!h#9qrz=hib{us?>JeDRq zlfj|)A(%w&)u*%LNHEp4m*r1De?neiQ-c_8tJQ)W*4C>;N z*^X{>7ld(odc!N&q>MT`jA|*{^~~#Ia3iD%&)YlRf{E@aI-YWG{x~d;pr9O%nTN8! z{mi6A-|;UOnDB*<0QZpir91#jP&S{mf?gwO(#_FSEdHk@|9vI_@fqvd^iMZMUgzv^ zGJa<;o=o)|prxV;cU7u!H2q5+&!@G9f_PwBx0}v0zNCD4bP7;pX$v!C#p|t-AmH8c zmX~KYl3Y~|kI$c~J)CkQ<|?&ELnKbR1c)mziBj^O!Xx4{1`K-jJ*lcRIkVrK(h5-O zib|gL3`^_bA8rXsfAP7l0){fHK+tkl*P5D=O+ zlyCUld_^|8E1d%$u@?U>YWcD3t{#)4_^nNGqVRY+GR(JtVIX&x*ys1 zc%UTrraCNrfsD&(I@{IZaxakYvh*hZ6S#|aEEXi;Ebrg9^c?P19Yq}a`*S(VhSB6| zao<8+YojSM`PSDT=mzQxhqHdRjzbG=dj|s=)?$61ak!kiowE_Y#OgUbaLZF2rjpT_ z;VQd;FF#v2po=y`ikL-cb{1-$KV4ks?hNK;#>MG}AMAAi+}XpM`8w!Hwv%<4S-L_d z?gN7kGDH^U?E&EIxt*>2+cc1vo~k{Y$7*iLaWEHo^(UL~S|IXi@9}i5(5-d0olr^Y zb?Q^_O%3n+@({7101@CCoRQpHe~a|stTJk;A8@ZRX_xYa@f&@iS8BJm9+IqeTb~oD zJu{<=Kr|L5J;j-MuQ9UFHde25xVBogTyR1AT)D39tjDX=Ed0M+scR08rtEN5I_0Sm zN-Y(dyB?m1ifYz$Tg>&;@FVv)V&_(7W5az0%?s%`K&>O!pw-;Tc@b|gJC)as-CTK8 z84+kE;UYfM2WNcYK{!L^od*TunVK(y$e6__Mb|o`Ss4(tOMz_I2Nh|8AHT=u{`%<9 zO6ud_b|6<~UAXWzh*U#4{NHvYg&#Du55}xWFq~oia5!)a1i5YB^}^_-ifQrMqVKrz zJZfvHa66=6AU96vv$7t*5ji*--`i)X_WFoQ{7hhUl$fW|9=*m!uq%d+N0UbS`<4KF z-rE3{umb3rb6rMhYALN7^bU^e7bYCFO#03zRirdeGwx8U~Ds} zaXhAzWm_w9gV}Z{G)6|^eGG*w$$ks=r$O8$YO&9=1yRD8t$!AsI5)SnzYWdJUFzx& zWX=&mpZ^y$*uV@Y8wN3PdF6;FR5T1PNB^u6Ft<-${%Eu@ERm;;wIU+`n8X_#cj!#YGF>AemHq$1Q86Br8JiJZ>W_1EF6T78pF z>+jJ@je;T=`+pePI;pefE zM+OVMHY`(En^B;(xM|rTk9cXKE!2)kPw2=6VB0&a0|u5MJ?L=^@)Y{c_}gzX9TLBeXNEH()Ln)K(*;mSKTw~h#?f<{LZ^G!(q+dvhhfE#|i zN80DM4e1Nlw5QMR{NiT5NMNaFEvXq|KU?ub?H~0&`?a3A)K$1^`sVZNFJIH%qlCw6 z;UALl=2SaySZS(#LHNJ#yl_8C*dY*_?DlF{YgP0x6w*GGtiLW8E$K)}Q_=3dx+G?M z?RFf|Yy+PcK*+mP>OoKbsi7Dc>0iHwHUW`wRh+$ICz!Ck-+ck;Zx9z`{vr60M&<3cGFNEUJjciV)nnkKE^!n`ZM^YEjQUv zVsUA0G1BmPjyA zzB`zoD;3M`Rp)w0ABaehXI?;M)~Z32Ol|rOr(wXierT6~nJ#K|3j8KMXq{1ILl3_eTBd%Czl$WtqmXwJ^{a}pz&nxq-^WZS>^*j|RE*|Pa zYEHM6>G}1-Q~il0Uos+ z`O5#&+O8wgd{O-LjGn%IOg}ondQkiU^-6XoF21t5u+FB=HE>?emL4Li>mT8Jk~)k0 zNxd~7V;LeS4|c!ykkPcf+VxgE$XP<{`Oh zL~NvBZ&>mnJMdbr65zGyH{?z;6O~Jh0z!tlw{hS78rD&%umV3I>}zsh;&Eo$%khNT zY(rHnhINrxVn|ph&);=NcU}V(cREds%SHh6VUB@ldb--QsY--&yu5ws6H@+y-T4kz zd-yIisFB#L6TOr-?paIrI|%W@9qMWmQ^@%97DyC&%+d8Wg0SMVLeK&`<)H$Fj5?3Q z^X=xGWRG+6ja+LMi>TkfOYJiGt}3$JMxCY)Za{ssyjcG6$jGE0qsSpQ#os*j6v z&3pIPfZyCVA7VZ$RC`s;cfK*^!NYsFUN}~xLPsYqp`aeq9;s66Dl+iQtz1*{V5??$ zbAC2CiV)KNZ-rzIBqkToI84)Ww(d-tLT_^5<-UR zi#4pG6W;vyR&(omto6Y`|K7pVb;K!aM@7D>J_5KDvH?W@J|JBnEC3ZJpg#Z&8roI? zLCc0>A=F}cPOtq{-pHtk2Xx2v_NES>*f$3-&`5fnim(}Fxw|Okd`~<@xq(Yat26zB zj;Mm9Bjw=NN2Lm@;OExCFrD*{*2q zVFb1G&-d#D)8&>ioHC2;bkr?ltgfX?Jr!G1pt+Ncq5gF=?_U5yA*`4gtgm0{T#U~& zaCT@mUOphooaJNJ0kmxpIR66vmjoI4N*qK)p-nIv)D@ruhGY_hT~ZY4}ZI~9j< zcNsIhN~u`Inj^!Ywl;`$uC9jSwzqWx_|dJz(FxXC$-rlMT`Bls5gg|w@N&47``#Oj zV6>&*4HJ$ardJ+#XpK7wbK-$!0L*n1RL$ey#%u-kI`Ggr8Y)V;Qvy^*C1BdE3ViP= z#d8h4?%47`)LOKA8?hqK>w0`2CK31pXd?=Y5a|18JgCeu5%=uw0Pdy9KHRspX7l- zA}lMd8`NPU*nw@}Mj)RN^}Yvob~U1rALb7rD7A2g0P}beX0+1a5MQL;hKY0Xr4jS~|C zdC@m+up0&l3FU1H`}#l#sh3M`ZPDv!^YNW-F1EJ~tJKx)9ej_~ z@T?ilD={wIU7p`!!U{3Jeo3%6<>s)PQmnAJnjOP|nGkFI`zGnj6ly6jfz7;&`4y!$ z$zYk%A5*ALl35}&4!tRNL)ti!*jJNb+v9h^SlU`iBomlHlI}Q z?OhvJ;q=t_QTen6<+)`JO=73Dy%UrGs~!6Pd|MmJ$0fV!dGIhdRTK{=Ih;YMFhM!* zZvfu#q-GI&Q!4}YYw9pC=Bt^mvpyHsS>L1e8cPlQJg>F1s#k)!+gtbqPRo#8!8~lmX2UMEOEN`;{MfUpdXHZHVT}t7+)ZgwhdJ zE%ZslOj^1_p}=I|tO7ID3uZ>*YuwnBVxkZvB7$Tioc=V3fXSqb*wC=8StOo&8Eu$S z^kBD@%S*ev8-m}21mMf+r+{W*q)xjYA~8DbYUa!a2>+Hp(|1@lf$*g zo(&<~f1oNVx-6uDdwb`kGxeD+-RbnNzO4u2KuM@>?C&RzO1R%|bofwLF(H zR<^iCkA+oL!@!|LBfYX!16F@nyF;ci(Zgtxwx>%9llV_F6{WC_C*80FK~6Q*m3FpN zfJB4sH4Srg-u!b3jp<{>FKrgxwtJuL>gPX%RV=33E04CEZ|MA{M6Yv&5d*48vWSOxoLiq=uzaPYVk$JU6*U^UD#|qnnXlrtC<&G4_uzs zUr-1Klnv{Ok?GM+1>MIigsDS;;FVWfqVJ zw%d1fOU2sv@cy{iK5fJ@AN}#;;n-s;5vK__x7-&S_@kp|b%tDgqJb1V{QRX@?0?l78TWrbXUm33h;eJFo%A#H!z zVKua`uhgUFXD#?%tI37E;l3zgzk_aWz;hPcfuM5H1VBl-l~Oo8U_@Uj_KwQS3)Ryr zrj=j&YHB%5dz{MfZ;xkAdn%|#@$lk3jIHk&I#?y<*M9jDQWoB0T5gT1s!;pAw4-e} zIDm6cRGq$e8F>b|)P}{&m)S;L7n>N;@qO6}FhwIXQ@NnCo&uC59hhyy#K2CypV(87 zpjVORzf7yyE-$Yz&m{At-KfxIg<{u5ZSU4k1#Tz&A)Ux+fov6-^CP@ouU&oKz0Zf0 zj+Ufciq6LcXGX<&oSOHtm8I}@HLscrBg5Se^wv4y9iX5EWEmftO?uTIR?1Oc9%<;s zcZBz!9R_?Ub&qsVVW_Z-y}PbcoHA6P^y<@S9&}n9ximQ}%CF{34>t!^JIg8MWC`73 zRQ<|XTa&TGW_fkd)F#$lAPx2w8F^^JaVzfr1GXED&o{8#yIqFbr*qWEuE==R0ro6W zD=5g(;Jo(jsImi7@#+)3QeWk}9L2UVK9A}t7g3`U@C(i-hMbzf$eg~)2*87m)ml{N z?!-c?$cZk_s?wSoNH z2Q=)~39kI#Qwr|su7=X9$MM}VIwywPH^|qk$cxE*_ z0I!gxDlYUf9jD6iWTsXP_==e8@Qz(OJ_eN>gVx`k`_DN|zD~LIxPoN+T7RabXe7I3 z%+>bwlhwt@&5}Spp$||{D%2igMM5(4M%w<1Fwwvx2Wc`GAzn!^VVF;d&#Vm{wkl#k z8E%fQV9o$;!&5f)AVj?yy3G-jbz7v-+CNY=YW$WI6b+$&sWvrrO4t3EPOQ6oZ>Q42 zt7yXlAOtq!g{^|3qK>f;l5h3(KWhO9^E_Yek9nzpK&iC4Vxcp_2OYepYxO-6S(BA; zPFcroW&6?}KNglU+_s8K6JMrw&KvVwk9@2b!C!Aw;Vb; z$e7>k(cb;sP$zUD#=U)hW-E`LgNOdSvYsIJIu-EZ-TCl%Mgjs3t2i&=g#**g>B1y( zrK}_wzBaFb*+CU(-h?CA-Mw4DA(L6Hl$S-!k%oW)>G=6wT-U~}_sMow`dcqf30RlK zsly7LJjr|7yEY~@IbEd{E6*3`p5fWm5mqM>HRigs9%xWBwfd@@S0*7&p>%fX*WbUxe zrGmWuK^8G*d4Wxv#+|aa{w*PqPn?`!Zo&07G$F+E<=1SiFUvTO*NlUgK3U!O>*Ej}mrCB-w(g-SyBsBnaE9e+ z%hE6Bi;?YZa&O*T*~z$M-n^;l(g^y#^_LkFnpHI&U6y|sq!ZoemzOfSV)2F=ayKIP zmY1NkdC~P0xd8`zhEyGj`v{^q(4s6Rs}ElzU>*}t0!srH?L<6hq^dp;j?RGJftff3 zRe$LD9aL^cssuJgiF!puP0i6J!`$i77YrXi1}15{r7WHLbw{^t-u3vq%bf>tUjhrd ztJsXV7I~xfYGsWw+&vhb$ZYKfQ+&#;*?JxoEoN&IEG}iVhXZ5N0n^E6^BsIeBQb0EAfXXzwU#`oKRXmYQS`pL=wol*{T*mztzc$T{# zsdYuCPH09;-o4kY^DZnR<2&xlmu=%3>%+Nnl;-CKA*=k?-`+;hN%3uIcQ3X>1CO{w ziDRX{k3_EWnBqzR27s4e>wl*UqTn4q$L}aSBD?U2K0KZp-k5uTncZFLr3u(ue_p-kHk05sJjylp=tPx5GCUe)w{b~?tFaw zIB%NoBnjN!^}vs~SAETa?d_oz74#}Yed(vnxhvB;AoDCHjc_Ueh7kCtWzf@!>FL2S zX(d~}q?tL#PgTD<=EY*pwABWmqT=Z}RBtOL#^tVBW=8iyHH>@(SQcDQjq93Qwjo*(km z#Tklv>+!$^1s5|T$YJ`JpTEpH@p}8R=}uQndK`&TL7W$X{B_TihaZQ-r5BQyALfk@ zD4MiPO?LuNRIb zxD;{IWSsTbLim(_8_o|GcB`h`ws>hMs@j$%8^(x39wjT%9o~(suzt62yRUzCBQE>5 zNKB(hOs+`TZxMPkSJxH>nO7Mh$nM{UpvDf|6X4fP4vja2HUkx?wzd`-8ap}^4mD_; zK;aA+DG49>0k%a0dKEMq9YEoANlEYX3ct|Z1ZJ(@d1_J5rM%9%VtecPHMMYXlKFxp`bawTGuC(0Ofyd0eo>#pAf_lJCoQi4{q%2FnD@MPB@TKkHG^3FaobRaKKbFTWshJXe+z zG_VfYuJ4KRG~9dJ9EU5^Fv&ZAc{X=TUthMm)YYfM8+qiXS`6D_w%TaLcBi;Tz5DS! z<;Ts^N2o##y*QcDhD~v#+;e{^w7euFB;vTjocZb}N|5VFSRTT~MIt-)W8 z8!O+PT)8!Wg^4z}+Zt|eO|6uVZ!C5+4&tp384hqf)&8>Bfrm%i6=V3bCy}>tvSw#T zpe;0tn=|#E{$;nxYcbJ$~rupahozI_po(QrmWV-Bs2?$?!`%1L2S;XUwoc`kvAN34A&Na({ z3e(A6!$le<4RQr)b?#L`K~nDSKjVvsHJ=N>7!6t3zkrN`LjL*(CUl%dmd209{@Bjj zym5R%Vqwp$wIAWmEK0y`jv~B?mMV(H>#v|7SHbkVvDlujbn>gd{s4h=MQPJuE|FF} zhdgRe{Z#-cu6-qchAk*)GDVUpO=K=zcIQEn2~N!O=4iGxwS(=DuGrl+nvvp0S~ZEs zlyWuCPy{UNI`^!}f4lCaESHJ&! z3lKa(9Ri`bM6VWaWLgE(*;&=Hy9yae%8Z(|F*rnqp>Y^?cB&f2ib+TT@7R@b%ra0yROf&Nt2_*<<<`Rv?uTF2Gcz7D-77Qu)yp>~zL~ z=L^E!-7F>a;NX+1^DnW@S#qis_k;wfKL&OLs^H(bQ&i-w;XnK%(jT5RIC!Lt!JsE~y<|p~%}WoiR6mIlBXyPZZ&9)H2d0a(`3vo3n_KJa#kL!&EonuloyDQ1_Xtqv zWq)YQhSGUBb{PEmN;~+7*VQdgRdwcI^~>F-h9P*-7>n(OO*`#kKJoGIE}UQA8f|zV zaF;_uVxTZ_SI+eeS5A&r?S8tr-uclN`oWiWMHWOzBV)gNnCHh2n$b0Dd}BD|Qf@Bw z^h6y2kN2r|UKCrp5`(HnW8eLwBj+QG>#IO|CE05Bx(n7+J5iIY6u-caD4;1mf9CIh z8(l$?Ydl}0WqD9t8!hvZSzxm?G{c zf!(<^yXUp;*{bPTHhM@pX$X;l#zsm;#(dy7LRXY&THI@mOc136ktdL@Qoh{_9rDoa zLR=H)tBWL~s7gisyAh=)L2pz2@>dBY11lpqLolC#vj1)1%dp7?=W9>7{eT+uRfiwC}&{-rWA-X+XQ&qH=jF zcA3kmrOtzphY+mB7&E50j7)2~1QBLHOOZ26ONeM!OrETc=*~e?u1fUK0ek4Q zmuQ8RjbqE^cmP?h?*MFZl|phU4O*#55FR%hR?cI_!PY&PpsI2FHBCk(t~>tRe&aSe ztbIhG#GOanzRxXnt+8lmc-%ibtp6x(H%f3m!+rE9p!=~x201qy4hpl*?;+*=9FhvF z*-Z^NAg>I25>dS4NjElhtqU=}PSuGB_oz6e9&zL(62EBUN*%ozWN60rS1*&zQ_V<5 z?(J2rRcC1pSi6x)N9VkE^KiZK7cwc=+lk4mQ&E=e{230DRPk_0 zJ9!?D094GGjV~gBtO;XH{;DCsJ%?vUq<_E~97nv%&4*BSLS>(ap z*NT}n|zG-WcNJq|yA{eFRnT;?1q#ELz@G zC5BfRtM5bemEM1K`6HA1T9%Z?TrZh$GbOL8L-W^nmh2**Tkh_-{#{+)zIvMv>zW7l zr59W4yf)7DLB|u;wXoQk^~;hKWcjqbrKMTv&1xu{ER=B|qaz~liHU*g-N$u3ZT<(QK0#M)B6?mBZUwVe zb!ig1&?^^VJlM6a{!Ci0LLQ}fd9m1Mjnt@k5%FEAB0)b-DRmTw1k&#g ziL~_9di8rGd{2>PXUHG%582osm>_>v&i@ixwy>%o_p=hI={?Gt9}NCNN^!LO7Gu_Q zBQ5Ro+Tbm62xv>arumR>i;C_p_~Y0`N+!Tf<|!9!n-k)>Z;^|C$A0(jb&poo&)c~w z=oC#2?{8phRWFDRg_g3*6(%4x{dCbEe~x^cfB+_HUEvAbhNrxNjY)hvT5@FMbB)tJ zhtux0we#w?!}+gR34D0c(jXG9S)+_pg-&5Lfh{eqzwn2rc4dG0F@@l(*E#&`9;FF< z&gbVR{d9i8{=$AuvyIV@{O{@&Z&%?ka^u`jJDBKm8J#Rh&a z+*Q*ujnnuY#O=49f{Ir>ci^1|O-=rfpUFa9`0VfjDc9#JyXN-r$5d35pxgtiFuO{( zHh=^FzuXcpBty!_&+lXooy9TF`IMFNa-r?_s2VNpyIvc>u$ft{rDptKi4 z@>2e9b58?OLRQgOwbFMw%D8yDJ@_q8)xVG5cq5k^6tuJ3)*^gn#L3xHLKdl24f}~C zizZr=$=&@XIlOO`A6hj7UJ=f2>s5Q-D62mhx5aV>&|EcQxfVEXC(LEe59L38p-RF- zJ5}3nCvKLLr67u=ew}gWPKqKO^5<#Ke!FavhdioCDGD^0ep5ej!Wqy7+^=hkZ3%=r zV`-4&awB{r8G6>guvfdOjEumVeERfuY`P#GscEIRrAv#|sbEgz>zX+9VwBONVJ(Sn z34K<|y-A* zj$MP{0v_7N-qf>?#z3e0BuTxo@k;Ug5c}%tTqjYnTgB~r`25#ThYIk93)icSV2w3Y zko14u;H^SKV<0w|pLd`N^E}SwJ5VmWdFzgVVm|#;-C&H-p&9b$_Hd5tI0ZZ+x_e~! z1k3?2-Nz(LP^vYO8QsoK&yec(Z+1ikR*Bim!t%RU&OFU^J))?z(^2KyM(CAIqDMyE z^_hd7-Z!$Ysa=bU8_x|G?CyyQG9Yfk;RdKXZKuS zk{cpUOS_GQh5UK3W1rb6c`t^W8(m;yD!vd-t>-oG{M2BcT887gJCk==S*0CKy;$}Y0Q<5icO>AvTW^s3&|P5(wXWGmd6jvA^tIjxk)(jV#}9aJGv^( zjcMi&ADa`s=wo~^{7=y^u)SBtl*~UA*5iAdOiifHHWeG(z&<(>n%AuX)eZ8oat`q! zarxn0t4Y?)f#*LO2ww-|dYepg^-Gg#vF%MjO$cdrjsE@vJhwgL6JO}e5TbrCG&FZ) zN%0A6G!od4rWV-m_xbseKNlN->4|m~Ucg#Bx18dUE6+n}Dy(19fzwc7EmCH70~?QF zt%^oA88#KW1VUcx!UaSyKuZwBNnNj07>ypFI6tP(v)w^I{Dq=tO!JoY-J0SfTyhFQ z9G>k_1KG2`vU*lbPoM5(TnXtr+hlS7Q3U!_sc9tU z!zodfe=PV06X+dpUEHR$p2CfY_&uQsvASt}a2w9v(E4yd)aG&`n^+HZ&3zYA4 z8=m^|x@HZNuWa6^H43_#z2l2v_ow*4zBk3EK(L;a>$ZUVq0*Fbej25`R;4Ubijma3 z<9s5k2>Cw7hx-Ot)3RfwVn`{H(O-_dl54da3BQxIka;O?e0xoZ#^%2Ie{%s=^GiAy z%obizfbA(&5$-|>^rt1?lbR9p$~8z@Wct27Tiy22iP>7$TK5xc~JBZ zRwVcF*F9;Cv!?3#v9v#JNPo?+OerHdKhi+HZEgM0XgDNZVh?p+TNFb0~-qGQo{DOKT?Jdd@)he zIp5+pZm_!h^DWAIap9M`9ZOeM8M-D$#jfA+{I1o?X*CeOkz0td#<+J<4A9-dO0x68 z7Ng_VfL#DZNg0tFdu4q+(yQyMm=NL@?v$KA3dpm*W0?!wrg+3RaG}R;6W%qUaBuE5 zR!=DqW^R$Y9I5ER#h=6WNQ0JQ}HbaCtTxsbd?eRuUUPQnfY>++u`@sB_ZogXf)>Ns&V=pBqDzI{)2BZca4n|3k%j``ZM%^@WuRl ze>qE=ba(gU(#d``g=2JUa%GCQKfxDsw}-#_E@c1Ik^+d72QMN*J)-3IUqi=bNn^f7ma3dx%xq(^BjZ>(*p+IgaJJ*E zWM#&*w3!95F0wTa^PR=WEKMR#>)$QZ+vwic*UyeN{5Qu3v7}-?-9rCcO4l9tUef6M zgpYS3nRNW`CjTgY{{Cdq5ZP2z?fRD8`JH_>@vUm~zsV?9XXuoo`ptI@77OTkNrN8F z_fa9)IEcKuNkY0$sS^sl;$9GDAruRS${NsFPCgV}%N)AVcV>`zcz2gDjk<*TGkg;qwjcT4^O5u|F`iwI-JhV6 zmnFvb<}k+)_gcQUHXvw5-<>Gh=v&oADqvqaC-N7cD@VQRe(LLFalDD2*@xi-H$+3` zPWQiDpPeL`Ddi8lA7MOt^hOc_b+A|}TkY+^9gz@W54}wl%|%nsRi5o&F3X7Qn4SNa ztHR>;9L6rhJf%Pe{Xz=l^P}0+5PPkBZC;Vus31%WOSw88>BQH&bLuk4Gut6rH9Mom zC#S~wk(PgvAcL>6+wI~+yo2TLWEZ=Uk(nhasIHJni3#j6dFg_=5OiRiW7?WXI<4 zYD7)U?tyY%mX|oXu&?!>#oy!H{pU@iYCIk}>pxJLwsduWbVQ1r?xnoC)7;wBD%u&J zyRSy{Tt%~-`cu!=cWYPCA8+Hen5|bC8$RRQz_uJCq;gX7_Xmg=`SXnKN9EQeo>ry< z#WzK_5? zkLe~?!O{pzsVNE?8XylAqneBNX4>)CE&e6?s90E+tBL%6erR)ZJLA?>9=lY`+6|2; z#l>)fudZD0XQuxehptjWF8BGahd({gH8iy1r^2Py?Ur=+-trP`Op(?&&F}XVn}!Xs z)5xS)YNTZtZMban@{(FUw`wBLNpg3AZ&hXKGVSZ{Eu~}9Y$!E_OmTRy?vz>~usEHn z+_HbaHHHPH#3j7@K2yOOozqbnV}J z!={Gt%gbH+E>0+~E(yXJ9x74-M01a%;rhzY8;RXC23gQz56DMnfIZ_&!#=+87bFu94EBJ8IbG|@ZT zO`4~T316dAko7K#pRC4l_IUc;3O1MhGw}>!AHP%=I}Aea=D>Ys1tkg!)e_YGrDXTh z4|D=@jrb=*pBZ;xDSbd`pep8OZzqPBSW!vGDW2noKu-eF%zA9Ec8-cVaWEO0_vt>% zJN<4uG;Dld*TWwyACneRLw1%E3Au31hnMJ`vg~P}rCsWe4gv^|})L zh0^ z8J}P<-%81WS6)uYPW#ZfxfP}1?C@kQcS73qxgGV~+~1t=88@V@zWw2HVZm|e$S(uXa-H6e@0_pbrNliAZuuVIpBYHc}DmGoB#6DyZN0~ zGWD=BGau#ry*ea({M&bh#>=C49|E7Al;3vsK0ebc|N96aVCP_98V99ewh^8du->C_ z$#WExVm%bAsiv{A>LvY7;qzmL>V>pSgHFP@7kYW=Xr*d(Bo)>u4c7IuJ9-eZ7FH2R zVay;zY_8TVEC8Gr#MfPtEVT|fI3P)ws(fdKj*WwiWIeTz#w5Tb`u1(Xjg#|ZJPE8% zpORkrTg085aUi+xV?s&FGR#7kTweZ}O;R$m(dk~)Gh{X+-OebUC~WNjET4unu`I>5 zB!#8UU%{j!6Bi zjS^Y61>-Q8NLd+b9g{T%W9$c8L7`NKk!+k(wT&aiNIR2Gs?h=XPxCT#(NQSi90zc6L$u zYcu+(pTA-`NBJh(>6abILQ_SHjj@oPJkcF_CfpMmKujEf`S97>HwAZ#^iY_zKgE;& ztc7Gg*%*S4a!dFrCgb=K5mofCA;2+NSrKw}|5b=!YEBj&DZSI^mnuhzd8Z}lV+H5d z#IMPk08yBLrBS3(@XGl`j#B?b%T)fuv5Hn5FR&u1g(CI(w|+OyuKN2wm|Y7<7G|*` z;d1EX;1`;TVZG(=zcN9@?U*D@s#zQ0u>Q>5nQu}1EsOVmJGc5i)sNwW_=+4AQB9K2 zFgxe!R0FG6s^d)si#K}i_oRCak(dwFWu#w6GL7;b4>4{`opJdNk=YQjXagthcS5nkP~?c9uVoN91}SVDpSxW~g3L zfXQ;)H;iU>E@g#?;?CU<6`acXzh{1BX^mIjqHQN^cz_tV(Cei>UVcmKhHfX}pT(0i z^UL1sk`h7l-t+VM89~MD4>?MB*JtOsGG8)lo|s7-FlvACI4^Qw@Wm4Ci16jj-EiAS z4WmYGheWGFxdB=*nW+E$=&?8lLY5m2 z>yKIQL%WNVfmQIrTh01lUN>yqPla{APj=4=e|psT4q5xNwX; za4RR88*;pvKR6rH)_D_R)SqZqSY;of!>HT4cj?uBynQ#w#qxOG4W@jXndQ73#!CM1 z6~M%V1S=~mrjBI}8-FBn5~EQ$ z6^RxdX|)y7Mo5k=?F-q;RJJs-MD=~Vo6jG=zrWY@y?)pA{r%1#V`kJj@AF#j<$2%t z^QkuDXVSf<+?bk>+T?To%QD*`n6x~4_Us$4XLmOT-&<$#ZkhXqt~p!b$$I75L+Xo! zv@eVWBYl6qM8&IceT>S+Pp|I4B8XhzW=f^KuWMM=+GcrdgBpMxm@m%y6= ze~nLv#UDXAzNmT@Q$_mu_*q$*Ku7jH%2j4fZ?Bh@Y9q`3QkLp9W|e2{-p-UYwN!Z> zWMO1p(2NYEC4AmEEiHM!%a^A<_pwyg(9pQ?(Lr6g=&Ngjg(T53!J=@z+CXiKKLviW zfPjKuCU+VLyql)XzZReWYbqk&`1kcIz;5SH{$KskW>c0hB_+jW^X6XHl`RLM(;};C zpXD)s@czRp4S{#Bj+7t7%E`R?>N@vW&r!Xu_tCdBgi<%#r>zPaKq?#y%`GgL)*1&X z&!#O~>{+JRdYH+Wz(5sgLj7Jd^V=-bJT`sf|FI02g`IbDzwMJfyrcq1t)o2y4tTG&+B(^^@-kV-ITIZZH-=UzsxF zlQ8fPOAQkf2$ZxB$77-_4e&i)o2xu;eX+7bA4s^bIaSVg%3N@Se%Z$#Cn5 z%Oz*>8W6-~6oyOtj$8KN;X`<+%;Eju(`V}FG-!|4(rSU@Ss*%saT8)3SU4-`(19fB zZ<~t=h1aNYF<69Nk<#G7f4_P2rlFw{%0l|$Hkn3-gurHM@N6(Vo}b`}J7QK8D;!il>T`XS)zxWjpZg68 z-9Q^N^&_WseRBXGvhRh2gt!}g)bAF7mJL@yQvdAc%@|lkAkJc_}1H5Q+@xVj2l71V-6{q>SC_zu|L?-(sf%l0{KxCtXB_bJQo=n84aDh`UWH*N9@kB6eHrV9c<(DnmecGUKPJ1>4Ud|)7 z_L{qq>MlohBj~SCZoj0VV`Z zq{_YsuTMn3QerHR_|W&iIbQ#?RR%RQFv78UvxXnsg7!#GsHnVx8f(gwDR6I{w?03@ z4-#K5eq66d>1FYZ8*~vnXUcFEXLUp^iBSm%aMQ26Jq?yl!Dg?*_dv^{sOyu(F>4Gj zhk+^mluG!0$c%rv@!-KbNQ#^cAVXEP`yIi@MH6SqcjMF18Q4Ix=fj5&LtS;3E?qi2 z9^D(0dpkeYoN}#;%QMgk@<3r@NfeH#^ci2rdkG%~O zRNxACO>gMKij)#qYI@j~&x(NIk;0xD=#q_PKeWd6s-l8|f{xZrSXCj*d5cS4mjdnK z&9MEwE^@U6VM`!E$SgIR#5DgbT)0pMwLyUJ zZ|xcWPmN*5U}(k{Z(osWo5AL8Ts)aY!7Rw)Xv|JcG2gvWkp8MtV{TZ`+9uW>EpVwX zQrD61%bJBu=grJhR%xZ5L77*&dmReSXFLtB%Y?z=0AqT+L?aPvY~J^ifqGpRPM(9j zziG@*&cBHgRq;ojX?h2GF*|^Uvfu%$i!5HgK6xU)=dBXMuZvEYj=f{8uhRq}daqHU zG}q_$iQ6rIJ9f{@+gtL!puo`aw^5^Bvg;x`0>|udTYe^M{D-Ra`$O)W)q72#QRII` zt&V3@cTs)rGq9msfrI~|J1grCsU8%|z;oZ-kCG}kf*X>EaGq3WNq}1d^x$0NW$jS#|4hpBG;!FIOC7@DlV4KeMFmGT(GxEKL$0_2z!-~>E`Cv44DzO z5+3Pc!-of2TdJFi_m#x`@m3L5F4HQrM-AQ(&VK&vzA5*ImSfTFygZc&6BZ9WJ-~5v zc9#hn8Ze4a8(Yx&rAtsmgnEnLGM}qsM~{9=lMChmNuRQ^viu3|!8a8=RHUOC3=%^_ zt@QNt5D4H=IAQS5m@&}_zj~@eZccP>!-XY8Kr$88)#2lQ zU8U?Zi`75G!5ymEPqpFBf4+wkbU848EqtS6cq=bYU{xD7EX$|Ov*D_&dAp&B$+0(I z%Rc2V8{zr$+qSv#e;z(kTf45odk6g$&{rpp2)SPeF{}@)fe4H1U~^OVo6fw}`KmX( z04Z|9RhSx&HlO}dd+%4r1z+Ww;KSjOnHS@`KFm~E7E^?r&u|0iBcZe$=Yp`4Rfa?5 zjAa=|k}TNNty`PuC=J(Gsfl!0)#%s%uw6mrdnTndO!YCR%(!an)~$tM`87566Q-Eko45NchDiGIWor0Aj9~=^K7<>t zFG>w?{{P#-R9#WADe}aTg8tlV*n6YW=0Nsa=Tjq68yP2&$RVA>F%_}vg$c{X)x}_w z#us19$lx#1SYp1R^?Cbe_}LXJDsZO<`wSJxSaCnmtBoC02{vORJ2%%*wA$G@69I#x zb6X8}Zvv#p`nI>XV}Sei?D-vtM(ljJ|Im}o;}ZUQvlvxc$KPNL)0OHkJf!J|cWY#E zLNCwo{rtp0POub6Fub+AreG0J7>$>|R%69l;3D=M<63TpM=6!)(DZq6w2 z-!Lai_Pizly_}jo4Iwi>KN0wpW1?KVzQ49xZ2%mt@FK?5d&3e1EwaVp8%oMUm#$f} zX7%dm@bK_K+181c4O|E(XP3|Ks|C-GmNbz^+u1U?+xht$(0#B-Sq~)?^M}Wy4oq72 z?a7$fNH)LetMDo>+lxks!__TzBIE%4&%anKHjyf}aWOZUoQYg$YcOe9Y$OT7VRzS( z=?7mSRWsgY*sPtV2I7Tr>N96nJ%64{Xj4>j5)wExAPeC~E-BNhl?ytu4hpNYO)iR*XR@2m2+sD@um;2^h$9;xs6RAbm}pq}8;kwNfiYt{0T z>C){fe0@kmRCVh&e(-ajrWb!)aWsOx*EtYVNkzi(D2=9bNRS-wlW-rj7OZ))Z5s{Wx=EgZzL2pSU>2Qx*Ct zP7^#aQRRa=t+i{`#2q-e`|EuB$$DCV;zlMY7ULWG`QgmQ{XTZM&(BW=jx#HW>+P>? zoxe6}w-)nL!OM^8qV414(5Je@6Yn)q0;6o1v zFGgjbgf04h-|Xwmf8vzbBWX`v-CQ2Eq@;us=4`t4UBwi~z(VNn8n;YQmf8@OhitG0 zj$B)&+z4$#aB#2)e#pm7hnmAq(V28~DK=hTpK?|AX=hIm_D<~LnE@4FK&d&tEFoSkoy=9A_@FuG| zQvP>b^Sjcd!Sws3!b5D^Qbn)K;p~OgkzNI7V3_81;NBePcm5R>n zPYZU05lcU=8el*4?I_sxVX{l``rgd2=ryh=)R}Y|c3%yxHMz#b0t<_mZZw7<946wX za|u7Dk0Av}MZ%cBU#D4le9Bs=US1$X0!iK4{63B%FZ?}rDAKp?Y+hbjnV_fuKE%ow z9iP|Z37gGbwlSU!B`WU<;?dO9ga_vNtc|}QbK%}tqShf@Om(T^BFSdg!X)cSbp<Vg?t1llWO>VEC@zC-gd0NY7*c}!mX^NBxjcN z!)<=vLZagb={cKs_x5d(D*71K4&IQzxtuWFNsNoCNyCqFju7s2dWGIm3Hi-68Q@i#0BDk>^OD0!CzkA*++ zu4rxV-hXmt=05LbIyRB7sRaHCYv1{D}~i;Z*?Z_v6!(Z z_GdZtx#O)Jp^OGRldHneCQ^wJrKY4Ny zB(40sgZ0kK@#-%pcRXP_DMWu zie2WFMjv?F$ukkU+tym~bnReI6=@d^_#W~Dl->62+lQ6R&i0-k9?%O)=QzTME~?h4 zk$)szL%w1r=`5);{%-sB?Zr= z&rj_!V#p<<5e-=Dic$WCD&Jwa$^{ZWN!) zL1x0@6~@g=)&2?*tL4~ZR^-P*bGCk5_f9+~Fon&(*8H9~kB>kcgfVQFuleYG5-@yC7)|NUi(br!nX2VqEyL9Ull%s;m%7$+% zfmK>-k1b8}u3lwh(}7l%;?hW_{tq1RS945GNdZlqDB1n#)q(u`eTPh**J1&k@`X#6 zijEiEy*qB^%vBTK5(yX?kp22}wC!Rft|1q(xxyb_AZT&7A=#GKV?k{rZlcmT!yRRPSi23W>951dsjM1v9Ds_86FYU_d|*Ik^eeskUWj z2-kW0W!cf^&rcE|EW;@W?%6O09FAz=yP^p_D!wTEEPw5658BIL&-UW11ZW>Z@WOn= z5bQepF0*3A3a@W35At6ZE??fmO5@IlyCd{DlN)7rI$$w+7YRh;jp?=VnH|~ zZo?N`k0dTmpeVn2^LLI*-@bhj?i4`_K#43Jzt;x9Br+DO#$>?&ZbIy?>axev^It^^ zTQXPxO7@(bm?ro-a%O;tOA75Ar6`})1lx&%`{G-I4j+FO{}BZTjyC52O5>wooHS=w zX6yCsgJj#3DHr)P0`p|J%y}Px^&hxK0ZVAU2jDsTP+*|Tlh`|mao5+^dyow>Zq!Aa zm=wjEpdC5coRwXSpKEGf0{f^X;1`q|OL!-L#yX{0s-f2WcUJZ|eeX6EzDmHQ7E&~~*px@$Y%%Hqd% z=pHKI?5Ru95+U ze^X!IRUO6fzy|enOusDR4MUP1F z>?UMSS@lB39aN;HMZkbQUpGR8DfIN|)Y&r;r>a|-I9P0Nb?EBVU6z*OW;45 zw?1T1l}S-?vE*F!g9n>m<{bc&kS>_ykTZBBI^p zHYH5_az+Ny!&D4V5bs%lCJ$j;m}1RfjTQZsluovMW^Wn27Zj8|rxx8LYy^)CzhZD? zNH8(x$)usN;DG>#m#$v5{@_T}gMW{E&-nsUSiHlr6`@ zHI>qxo*o{CN)&UJ&syi=Vwd5zUW8~()h-Hm#s;Z$T0)oO~*lF1kOLq*Oje;TW?yqjC% zqtkQ7YgO%PC|Z~vx*~x<^!hkm>+7JfxIj^wQ>nW1yzvcH>2HCOM6Ad;x!P_o-oJM- zT;#ukO@^xu`^d(Wb7x@_t1At?oJewCc8WSzk$-;h-_kRRjwk&uu+zAbTx=}x?Ee0m zCp6U58i0sJfJM|Xr=7L}lW<(%bk6$N!Sgx;2FMTZ3@xYi?$M*3U%av;gHuV&%AO1L zEP44IfIe{V2CBgoILXB_(NZnE&T{&bx97H)b)uAIXPlP3ty{!x#gwA1rz;|(&Om$k z!*QJ%;KMtHm1^5XeMoBfW`nbBA<@F3ffovL1Y}fkq2I;DAKylAtI!*f^G09a(!H}Z zEbW5PQ=GBIw(JwSDifJ&d^B_X4+>clxw_k~I$>h*XqQbkk4jztW0FDGnwl7-0+O?2VI<<-*<0%S2;D| zZcd}v)!4H-=rX+4KIh1eunQo;gcM3zPU6dNiMz5U9PbMXD}Zm?pCouT*B(lHSzG%{ zXn7XN{L~E(3cc3{mOuEXJTf5Quxw{|lcBZY!4Nf12f!RcgoNuMC#J8Lii4+<`@~NO zoR*QjZ{Gs^p1ZAaWmD>F3t$MRLxIwV4_%NZ2RI5^{zO1)AElmoJ+w|I!*+zcycHaD zfd;>+sEF`shI7Uu*gURf6EVW4=4RvSgxX;|?q5L_`%6ruDy$wejs}yV?ld=oh2yxuF^>FVPg>q_e>; z--}?h=-fMuJNh(wErr};luLfQBf*EL6WH{P+W2KR=IiyX=h=G(D=J24&3|@%ha~Vo z@4;vdCXA>8l_t(L`lZ$=zPa0 z`JxDoF{Pw~m@i)#(|KBmP=rE~>>J9tLx^($>v$e~MmZX^BS(%jF}Y9LFBIjkH-mNn zorMlHGP#7}ylK?jXY|&TmZqaZCbG4&i|5bZ^Y#PEftCuTMIsQa0-@$y3OS(E2^Q-C zV!r>B9#NVxQCs^PCtL9-M2n>K_?-@{C($@ySPHs*JSe(XKA?IcgiMRe=FfRd5J4*M zkmYd^(@G5tmFz7Z9^Wc2ZZ zR`O$k8kY#T8NegYeL6jTM_I@q3d*Q)DMmfxTMZvRTq`UWb!d*Bxk=PX!9oStby|u0 zQfVWRo`CHjHjhH1;7f5zW_e%x1K zrjt|T!mruc5h%1xS)0HA;6dM(QfsH7i%-winLN3WqBFjzvT~)%))ukavMo=q4Hz;6 z>R3C(x1F8eq7UQ#wv0t^A;y?=F1P~I!R}^VIRkD?az;zoX5crRzT}kViDW<|(nzuA zu-blLUfgVa{}^TE8@F!Vdi3ZK0M7mU&OnWHHF2kkJ(gp_QI^{HFrt(Q7uQ@>QBi_b z6o4iqirU-0Iwa)y1q7&`FVJsa(r*DNYurP$8$UkF{QKoga|(b0>jd3(eLGWWp}FbT z8=aZ0(7*HlR!`6WAiN>;Mg8}CrP>^R?3gDIb}QF+yxsXrs~-Q_?Uxb%{eQ~lMQpq@ UHMrfJw-6aET%w<<{9 literal 71546 zcmeFZgpAL*4hTk%Sz&3r?`%Vg@ym>Z9Oj1~9gmoH#W*Nq`#yyyz`}Z$Eu%Ah>3?2+`&xVD;?*0R zms9_~RB+|q=D)uhxW=^&kBBAFN&sRitR!U{-fi7nR$^0Z)MT(!0186%to z$G_+aaW^@$vC2 zj}md@V29_*`H+>BDz;d7_hKO6u94h-@Zzn_cvVDFp2{;z%hf`| zrjXIN#6*$VotFDF(B59fCLM~3il6*ny?XWa>(||tK~BsUSnSZy z(CqB2d4D#nzCTZE@bxjD_1MdoFCU8T?Ce;MmgQ~fySpESQu1k*nych#@~m{J(5z3? z#6?HD?yrweO-)^VeRJ5Kh?3Xx;q4oZfsh`Zot>D%fr~FB)yDbYef#zeG9)1(fs)U9 zb#>L!?M<1xMcwb;;orW^GUjQPVs>}gi!ii4C&}G!KM`6Cq9>^R0@zi&QpnMvK7)z=H3#;HWJp-}$r`ue0 zuTK|}d~_AgA&lkw1Rz(x&`H3arX83qPET{$PSwH&F7LYcM(TO2@wm-(CJ0H-Nq(kU z>dOjOcHLXmh>KtO$cg%*wzs#(VRbbXc1cd`w8ZOp4|CL?t+E22oGugRD5&#QQey%& zr&(^zZsXAN^Oa4_Ze(@+__zosXV+`I=cucidCuJ_&oQfmzkVgZ3coKM$<}RNYNAS8K=?e}H4hhfKtdS1IHhui6a?MY&K(Yp}GaWpD4v@fw9 zutTU8sG9kdbtZ~vDfnzm)HK$>W5N#boc~qC-C9fyyU)nTtjY1{&H)t7*_deII|cLF zWl~cv>tkl96@=a2#`J5S=GL^`GpO0+`Gt6gH}%D2uiZJZ_)TwZvO9zB9UQXMN^ z`I9X16@A>cl-JVU?&##C@=Se}M5Ej~p*ps>*umYs%Faesw$tw#NxK}*)a)#`_sIdo zNZaq?SI6@)sAN{Pd~PK%1_r}!v2m2wXo;qjRBO1LsM&qZ2&vsVJkERf@3RoFt?^hC z%zP_snKl{PjQdlUqm|R&8O`;(p`oF@S0PKeOJsj+9O*KL6UqCY6HRyz>pK|~1@)W~ zF|CZ(F)5igtG&>n;5+!v+j=q~{{8*^F_ALi?|pFyv-{80ZR-zu$66hR@4eSUd)HKR z=J3-0re&{z4b> zqIEaVZEx6g&O?PI!c5id;q4cnshAOaJH2w0E@q{gZEB-@QCjY*W7%f^IC z$<)e<$4vdTHm!%GBqY5B`V!J9R?xY;y-_DaiaN#ED;#%L2>H4KAhWc`p+KxmqJ4t9 zp7%fvt*GGtTbL#_)wX-(8dh4vBYQ0QZVuh*%m-Y~K^^e|+Xq{J$;up-deUrAhmEhV z=BWx9w|<`?Llv!6w%%o9Ta8fDgJ7a#9$(9+OYk7CV2PYb}OoMmK? z{QTYhDQVVfz7DdesEEySZ*{oXV}G4T&lJiw(q$tu>ide;W9u>gLkZts1r7_cn+ zA(lnmTB5lOKaE1Ch?*p`o~U$K5IvZ>()OEmV^2gxg!D1)&x{NgfCF0P)@_mOLN4pt zb{}!cBr|$L?>ugF)I6rwsdlv+DKRr)iAI~`RU2jZkutS=lV}=XW?Mq=nm$1f2&EF} zIT(XOsxpHei{oWJOQhtt{h@pYL310g?HD>*n$qz3T|g(SD(G;tBIOKOYO^R^gj#!5 zU#>|v3fp9ZlWiqA5k?#B?zoJ_rtp_FZUaeeM>GcG zTjjX4+@HfF6KfmS8xh25SwC zg}qN)9UXW2bJQT>tn4{hNn9K9npq-R9d8$#ncwU3w2Z5ji|1FE(5!Lao%NvPF~2u? zfxu=)pytGKo267(96=Bg5D;JlMLhPn*Yn<$&jW6WQnPh%P-<`IbC`ea<`4az!X+&& zjX7~Dl3nZ8nj}1rJ=3A9t{B8v475r&9D?7T+t-g6@y+0>vW)t8e%kt_-X^#8uatL|Z zj~`ETGwT0CT^9XqDjLu$0Ju+Zyhhil#Z&yS}sUk0V1 zWHZ9W+B-UUCcPn!KT`>$9FS_sF0e#EZ0erbLLUfOe@a&mnG7N7`nM6^um!?nq}WuB zGJvl8EjRiE^Rariav@PkI9|vle{1^e^mt>e(ok@-;^m~zd5wW_E;J-UT2XE97l-O| z*Ig|dLn)c`Y=y+c&THRaf#Sy-`QXLZ9oaCdyz|&Z-8B-h*2OtAR$yPf z+OiV8Hy>feYbM=kY)t4G$7?m~ago|a_n;+5sacFymCjvKw}#3UGAL8H(+59zi}dVx z@|x5Ba*aU_Ym{T%jWu-ylxgZuzhF5{I~u#QMxeRyVsoYWPf`*T<%*& z_uJRUvZtq~dY)Sifc(e>XcIpiZsyud+Pk^AiJk8BW-6pY4*D4%?XED{JUtT&BC6}| z=3-=Q*m>67-EGrceK;T2xO>j_ueRl)XQYgN+lX(|exvn1cxrb5)iUG;vxtb!7GYg` z%;Pyysnz$0W=KHw4>W*4xgnKL4*rIH95a4+8}{7wIy4(c53zOIa=#BCQHnUCCz;nKJ3N8ab&s{c2$)=1%_vP|Q@9^@- z*w_nTaxq?s2dU<1?k@ELf)Tw)(6tvF3U1|`3-j}PV-8)fQrU@#i8-_@Y`10_K9X~u zLF(STdGk3<5wyee(+wZanw@Vu^4+I2w{HE6Mn55@qB?;hs!6P9H<%j{^;!ivbF1Bx zgzS}F+^gN?{?SqkH6s~X+9b&GRvX%fI_y2wAtz5Rzhi_6@dvT1hX)k-W2jK!E>2R4 zN=l}k@s(DiWqp0htX(M;4x``kyHr^2+_@9tR+gHQ!X{8qQ1DPlXteCN5>zMXV;mQO z8MO3chfX4(xwzD@=`+MZU%t+$=il@V#G!pE#@hlaFss~DczC#6f?$88gQ@-_Z(Y&z zY9r{prd^3LFJC?=nu9*5hw^l5`Njy?SR&wt8z4d{A#7yzD5oNHLFi^8Z)&lYrW1r* zyw6Yf0f<`zNX4KZ`w`K4ZocPwFDEA#wUG@NBtrj|ONXXg%dK0tloMEGJKOn^?*L@` zD#nct(DTK{pS@PSB(lbs{T;&PRv^$amR*yRV%*$)C1bt4#c(tLsSn;=x*{whBEt%W z)D#XlU5pE!;ItV=sH4SH!GumsK1=U#XWNE$nn4v=`CLy=k7b;!nGp)PoR(4Zavn zVYmxXmXb~Fe#vTox*&EQ^VszBTw#REaU^6;BXk!iheu7cKEj8y;gC{|I(Bw;F)=X{ z6BFV=@$poL3ZkN-s)^+l7nS)1mzlU3A?G%!+C1dYE%Mx`sHm7$Z*Ol*;PCKpnuw0d z0R+U&iy=|BD^Y|rJVk?^hezdW8IR4x$>jN&%lX+U3`xupxjM*!`!`i*D__TSTJ)D$ zX>FtHLT|BVY9N~5*GCQkICHzVNxC=iiIPwDYi(B?-v(4Hc&f%ah13_M!#vA>{}vmy ze1pBA7ILeEu3I+vREFzTB0&*l~e28Ir0l-F_DNwU8S;=%WQ z92zV(N#NU58ej&PGt+web`dJaEfITwG|ny@O0Wei5mKU$+r60HqN~Aloxg&D z0`h{tu^5EIxkdd`Jctu@kaFYjwOh&gI&#B$xTxXa&+xDS14Am*6B7c3*Fksv{=DVx z37Gl3zqq(KL&5yu!51-ZF~P>~urk%0JbJZwnZ9fl>9dD$n0$QY6~|q3(!#iOpUyu- zJazK{hKG3hTMiZ$Oa0;Y!W?0e+x(qgy~B{7cb z;`F{uY%?v89qYrzO(%5eh%&o%Ujk~u<~AiIB}i)Hkviz2I>tXsud&@wH<)8oa?0nA zTZGPklKkt<1MAnV)4v1ZP|fCt9a7cf!H`kR-gq4`>or$B13NZo%O{l+AJceLCFs*Sd;BGTFF zZ^1|4IUc;o@PWxBJ=F7bcMuZ@;mOt?Z$-??8N*`VF#HUH^Oo8SRDTxqu0%@rTa(WC ztOu*3<%pO_ne$Fq5w-o}M1)Y}f+0=J+anjS*RqR904-w^7 z=n>q6w1>W?ZE2oPN5J9 zAWmjU**x)X2_$<`9{^7I1YDSmO+N2+LP9Z^6zw{Ua_j~qX9iT#-e_{1nB;zmkGB*& zpMj!zYwf|nds<-HQpDYS?q2#)QUN)b9Rhq?lBn0n!9#ZTJma=Vz%D=0=yL!DT`-ob zt7r26Z2ygB#b`gjj*ss(A7jdEHDbi3QSG_~aLog{7>pAoB_%+7!ci%Ezl7Rn>7kXi zwBrP2kjDpRONm=Lxd1D|Jw_cwWTdB;0Z{_Y zG&wmLg+f(UR`S_QjG=veR8=ESGRbvaf7N|yMS4X@l^QgJh^u4@bh1epNuoG(fE|VPA#TfzXRG8u z1=P5m7k|;AD`qU;0%Z#&_3ODY5j1xSijgAYHdr*Yn%_H8cFxYc2~@gUI318xe^r zLt_FK$wCx66>zPjq-0d~DKYcF7z_s-TSN)6Fiuw#O!B-%q(cVcE0+vySlr#HYwef za_FGQMH%G^S3e{D_vxRhh0?HcQbWeW`i>GQj*_zkcxwA43e^D%QfM zYAdh231GI!gCB1a|cS^$tF9eI&@y&icoZtm{+dT10>n>5^=q+EdI z`ys`Z!1QiFCH!VQ9iNaen5&U`fOv-Kk>(WO=Z_XBGl2Q#jQ~my>&Nv&=>B5xR?p9~ ziRy-j-5NjLkqo23$H!M$)vU19x3{+^6;rdL;IsCG^edlm!;1qylxo(M?q#vN57 z)(HKRYNHGGgg8#lSnaD#sIRZ@lBOL1M<85e?E(R79YdFe$xP|S3wc}ug0tlu1kin^ zD9wg)&X|Y#budZ7Y^`j7Jp`x(D5{eSzG~^(CuNrYu}TNv7*w=lGgNbcX!ij=7ufwM zI%NT8G89E_gL>Sh{SsZoMV;_Wu%)Z# zM4u?8zQCue2O=I2GM}Lvs1>TY>I$+8z$6rVQdJ<4$B+wiJkq&$nZ`c=Ar$=c=g*Uq z6Br-(%=7zJhza=G!z5R2yM%RYz+V$&? zCSF)iYZU;=1p^M~EHh{thf#&k%%QSAY zUm+V}&vX>2e1C01yswQT`Cu5~;pu<#5a2~B@Do?CZ#=LUzod|Ep{f`l-53x^-U75% z+Ubpqj0{?S+b8aV0K|cI1Hu~`6to{3d3pKZjE|o_om>EnNmN2~v^rEi7`HD-qMh^D zkV61=fXT41g!>!?6LNBLRC6L|1O)^b@83TFGY%HwkCL(t%EE>V4&(sdKwn4$c8o)z9e-E8Y zOia9X?HVi|Q9p!*^~8>fRR{!5`ED|pW(=DMdND8?C$3Usy<_=Ew*%jRO9z`X;d6f4 z`Wa;2TOThZ(jaVniq&ysxk*QC1T`)e)&sJCPq%m>L=+Ur-G{Hn!ZN)XcZ94gRVA|98p%|0HfW z31_xl-Q^WD+S={(=C0%?bABlWSGu*jc=Y_sQ92sBkF+|L#Rnm3%uIQ2de7p?!by~J zH1jx-o@TF3{i^+1d|q={Bg#Lzyt8<6OtF@9!@Z}srPo>iawPpft3+%56E)jMb3f;H zg$`XFv5(nW_!fQ*XJ%pwlqIkuER^wVU>yukpUpUl!DJAnrPeJF1V4f(QMH^n~~PMNEFmnrFoJo3~0| zck(ye?MM4xT@|W=;2c-um8_2BJW&SNDv%MkRV7Jsjifx zLM<@UQ)=-(E-v}<t;bm%S$C4X3G(lpv<(v6L^g|;2_nBVc77ExqxG`-;*eeS4 z)aSII&W>;SW9wJM+BrGQ&fCzV;8Ab7FeHi`tR`mcRR~n6NPTg|T!)aab;PVO~ zL=>9HT)Ps?L(!juII2EK*~nH|Cnbrd9;PbQ7>VvS3E&PoRUOEaUW%fr4j&PUkoQ0G z>on)H*jGYYR16S_kF{4-PR)F1{ftL=^PkJ*`*<{$n|RQ0aZUT|CPk=u-G515$J$<= zJEzABYHu8!r>v*PuWaG?5hcndo_;vOzjlW{!X|hO^DbfMY0nt)kWU1qa4mw!U_8`` zd0&x5h+dfOwm5iX-W5>_Jd(>G*{J?|IvuEc^=kI3aQj=N+!Xy5h{M&ZwzHMLJB_ee+?)wdq6|ea^)ADChWskhFF&-begM2Lf zSUB}zYJf#3*#tZG0QCd3)v?db^U<5d48;$X+$&C6F`?zW*XdN9wU#!PIsIG@huSp! zHS)Z)y3G07N={c|1}DXiQA7rw#anV$MT|A46_3&wf+c@qjZu$Vs3CMFQ_ucc0<+(j zg-yjhlxbK_RFwDcfD&FS?g%ntl~QK=@Di?~UoYvNJw}+wRL4FW3w_xlC-BuXMY4lN zW#OmWNVUd-$*JQClkb5CnQw=|b~WC%t~Ca8yf)#YEO4DJ#Xe=0q;j&o%I)z=I#J3K zwYw4VuSCLizpQ6+2$Hl67)b{bUZ?pv6!9hM;qGi7Nek2Ku-NUv_;ZUbADdFowv+`M zhE&SW6I7KNxP@ZgX!`f%!hOVg`HH0hI5fZ=VPWN}7!k832kFFwvP^&{F9SuRI( z>#|T0KQfw5^A`_d|Ekr>@Sbz@6`teORKieEkA>Ww0yqe48vGgizr+|~N9^(C&HgR7 zt(+=^;FD~cNH?=6j*`SX9o^;PoKBo^~ znBo9=v|#R#T9jJEtpU67Jc{hF>Hsx#@Z6$Pk%ijA(NR^1d>Id6q|Mw%E|`8^v9Z`pvm)5*I>9*NrwJ4|tZ<%p{htMcv+bIZ0i9oVcz_ zDzLLHi{pmsItZ;e4~w&J&1HX(qlyRurW@;Bpr+zCiKkCn`q{eEq<oJyudvtsFQ3*!~*TvI$SOeW z=U41EIezA@HPelCM>M-y$7QJR-prEW#hliYy>xP;yq%BYxr-k5PiObSt&f2u z1RG(KKHKXZR5G6<&ToB}dZ+cSnMfS>ugoehVX3*Ooif=)Lc9wcoT$m?M6YVu*Yte+ zPEnz0J+**Sh@!%-Xy2;o=Ue;O&`UekHGTII#oP4kV7bB*H_9)DL8isw%EpV}2&-*BlW zuZ)a?$5Q;?$HL-$BWR&BUR6au80kwiV_P@9^TCnMuk#0`i&1<`tX4|d5X*bc?N=LQ z6Qtq~0>;#;d$Tg40@y{qdWvh-Pc*+;Q~q7rnBd{UBKos=^`%*=Nog#`el$SyY7##S z-v22zv+>&|1A&2d4?Kc9(-T8mg0DFf=GTWy`JUtL`VTb8(;w1r+q359N4w}JmaMI< zj*pZ(Ic&@W48a$n6rDM4PHFq7s;8%OX}TqCTw1zWnn?V=s;g2;TTzmkDv*-I|JyJ%kF#Voru!;`L<5`h@3CXPpKM}3XsohzeK(v3drl$0uTb9epGdEe~zhP zzASpF&jo~C03a#+yDD< zKUSr*Hxj5O`aVZfU#+@K_iV5>zs7u@C4ZzDn}PGE4w1Me#I@IDn~qnIqp~4#EPqJ- zOau6~txeqfnij++022`zoWDCI`&=?e8KNuoq-s)kx_oi+XQ%-6{4D`?^iCg(Vlv8B z=77TNxDvbTPMZ~_`9%%X=8dzv1x1iBIp%TA-m2a1R`)qZ`sOgwXlJonPOL*Yt3D&i zeSJ*Bv1voX*rD$N$2WSmr!3# zQxi+wDgOUQ`{$~CfUFZ&dGgNtZ)3@yKPkm7Efscdcw|O`LX?fzpuBk|y$E~wTv{o{2q*ko5}mmbQ!^Y($&$=97g1GUfhO^=d#5yraV zG-QK?HWGKP-cgd1g&HG^o2;5eE?Om@cO;Zk^hmj(O(iXnKl}2(iS);6Bu7U%EQ<~H7 ztSEGiP7ufWqQer+l#(E1t|efetubj+YyW;!H}E`HD8fFw`AgN{YQAdLF7|0Wo&z*Z zFW)0~ArDDYNxjaBYRpLM_mf+urN!J$6OE&kOx$r(GgAh0HAgZ2F$^>7?b6%Trm z7|N!Ba~7N+_p;m#v4m>m=dR(`h|L{{(~|xeXti2XsP(42=&D(Gc>YIqzLl|DdJp%S zgc>y$y>Z5ZkA?TRI178rWHGH70rB6khZZchan>nsC*IMW9fssRTOLt^h_w!^9YvgX z3Vx;shBvJ`7fbR`8CT`pNRD-@Y~8D;#a2pxePJ*J!=RXTd?n9K+J`ri)q37?Gs(uQ zgZBm4yM?YK+BO@6rM5l6f82z2j!0)1-ig!$3(dJd%<@1saS z2sC)EmVj-NT+U9is^oU_budcB!Eg4b24m6AmVXNAi}#EC&FAN`1?8wzE{SZO6VcF! zf^!socGRXmGc{FUKi3LAJm9%N7hDJChFc^XL;x^0S%GAD^@X`C<6bQ}S%`Ghvp@n; zPfc*t6NqfiWy|Zia_q(v)pc~Q-9$^atI4*?g|vs<5_he1E#0)%e|5Z7xy2>GMOLQA z#ndr456x-b++n41aNBN+t5>${z7UHLyvOZDGW861>V`#Y--q{R??0P=Hc{=NWb?-b ze|4k}nME|y^<4}fTjEL=_IvC1=~nY^?dgjd`dOTG_N!oE!RVA1+hmac#6jZVRA9ek zB=6;zH``|OSz2FOKXqC_%Wx*j5rj*&-NJ<4Vhq>MG%X;y3R_iguT!)wpideWqI zsvKLuz%qWJVS{o2&RbkMPcN?vSp$M%b8{0Y{EW1;Yjj{1bOxhB$mfdN5z)2iqipr! z7rENZx*mBSx^7@$g*?AZIQ47W_WtH;gZ*-5rF4LxAcQ3PoE^eXf{Ds(xX2hB-$16? zKYjY1=t7u0*lMD+7^wC!d0!p?FpiH$Q89bO{di%F1q^SF3}uSF-OD#txu4i_f2bSgNbUN??QqhoG2pk zAUV;S!NC#46c=wUEKgJASIiILljMmBCZcCzS^&!>XiErS%6MAxs|klcSA6dIZ!=7Z8jfSM0?4iw!WNI+wYPfYX%ju|ApTu|pPJc5SgZ0hp* z`|)bj7K0(hrqAWSxE`;iu(D~;@2!!n33`RX&R!5!D}p~6Y^Cgee^z!wJVrpwx^wg97I-uEA*+FO z=P~aCg1iZo8RFZw)nENmO%D?zd-WTk$!VR0vh3>>1Gq%j@Mz!1*H-Ox(1YJUgXglOGa&L z?tcEn;A6y~1Hxryi6nwrn)#(ZAY^CT+NaXXldV<#hLVKaPU2${*nQLw3lE?931C>EF zFUKMbrEM(}#|AN3BmwPC*8 zeJ{Bre3Th2re|dIeQPsare)$1R;_PK@9_5UQCH-2ZO7NRw9o}gG4%8RcBpQ#n72>a z{n6hozXPTl=Ig!Pr`IB{g;)dSYAd@AKbu?DEL^+ysTA*WqR^x zIU#SswJtMk{R2#E6y`saYN;mg4eYu4$^P@4d_6aQw_J4P#N8VX-;+Y8 zy2+?b3Xm*x!eQjZWNkR*JGvt{3o-Ptr5=m>9N@+Vq59Tm!n@Fn=R z9UhMPrcI>{=`uz&&UWo?@x#X)e4U_KX6ZHoxr1qLn@`=NrIr>1I^ThHW#DT)!^Ol} z6l>X{NE>9rv; z(ao#=*G=)DO6qx3-z=ka;%*YNPkwb7<{2i~(+XAir>oz_@^|6IEO3B zPA!rXU8Id`XWpCzT$7?c1gOkGEPgrfGZRS)i!zKA4{GTknNA|~-0=a`YyJvu%cVo1 zg|uOOW%2%B=-;Dxlj~$d9p4Vx8vB3kMx-M=UDqY(>buH)cx9%n4vN$#C@ep8U!a30 zbAKpG6nj>3ooR?4J;PP#DAY_E@$w^b_tmWmA68xV=!@c2)z;8|J^yh>H*P;tq|9>I zuno?M%}gV8@C24SJB+*_c{l&&(Kb=viPDG`SgkD}Ra0xuK*=Ga_STnN&OO%9X+AV* z1&ci?rUb$cchBAl7k=+Vt+p0j^1!|~$MlLZpCBJVa6i>XSO4t%;{4L=($cKii4PqI zU;f5AYMY}y+;l@c2x2-TKK{}CW;B~jL@9z>W>-{bb*kz}c7B(Tm7E?o;v1d3D ztZHYnj@>N&iyk}!YC$$Fei>Ec;r`TJIw{g70Ci`XBsMy#;$#_<8Z7Bg^pTU(8@07P z$31Z5+Ox%|FYQP4nWbrZH&9Gr<(iM$xwe&c{df?q^%+qFNyPW09oRYa+F>5iObzcw z&ER|uQ2&iIG^~CHC4Vz88Z#x>Ytg?At&PWt#r-M*Kbi5TSITW))_JP#K2*+!>R@XZ z{_1J4liZEoJ?(fp_eR(pD%|A6f$GRQ@z7kbrW`lM+>z(+^7_y?BLAboRa#-cLL@+&*shr(;DPfK=E_Gc_j}PPm zLw&?Zjc~kh5zjEZAMiA9SNn(VPdi~YxRZzdpiNw(GjK3~IyBM#_%U!w-fUBaO_ym` zpoq2ympr{~u98!lGntQ7e(X|KzGr1c6oAP0X7qc>`O>F);O~lie)xK9{)PhD;mJbA zp0E?+-&T?FnoZ<1wYg(K3yJc9Uj6ycP)?d-10Y!&F``aK!jlJwDCBYDY<;|xfH4tv zgnW$6a@?lP&s8rY->=(3oqYEvt}GL{RxY*u4GR+scb^v@@h_dW2=%EjGAoMI)nK`x z`!`D6Y!pAfQgoXX^kOQF<_3ZUdJL(~0PkRJ@%z5xgzx(!XneAGcZ_tnUIVG)$Sf>YC$we91ratmY5LM>#x+2t_?O_3 z_i*G^mnr5cmhYGig9b45<=5KDNJQCO*oc==`vhZGpU(@aDQcC^u-d3Dw{+F4q^$Gy zh)=BIL5r)4_EJ-Tbabe+f!vu5POJY7_hLI}R&4US28<>vU79BQCtnLe{Z7(5)5h)Y zLoNZacq_cT-!`vswa~c6Ww*0;zJ=uA;+R77ZtL1qaCt=WXCE|#Y2}#Y z%>G(W>lh~I=|*DS;zwqNOnN~SW!{$6*Ut|mcr|)e#QiqOJd2jKt$pj!)D_>Pqz>H= zt;Hhl$ddfgf-m|Z%zpy>ZhR#^&`b64%IFZ{U9F+gwK;%}#+jxk_|0|J2-8yR(g~yT zMaen?feiFSRUN(lp`0xsNaoI&qcc>q=I}aSEi30RSaEM#s}9!W8`#K~9xMG=@<+9I zHB?`@LJ&P1hbS)DKYmm9%TlHcI`r?tbR`5H-G^~s!<%U|%eU???HtOY%0GShM5)0?P)|EVQQBW8(K#u=X$8j?f_z?A@n#|v z$p;9z%5R>My!na;!tk&hglAsxZ}lsgV*vAwVlX763L+X;y>bG3Es zMb}P}4*|e`*Q-8bJA7YPm+)%&;-$4x?(M$%q^}ibI6E^TdXwmc3SEjqTJ> zQFeU{$zbIwZseKHJsmRQr^|o!gUps(Z72(`)OP&8(?nHZd?f(0~6C zf96J*=)>OKJmPH4e)H!G&&bt&NeTLL%vk#o)xBEltKHskZ2v!@^nL5&T|U13v?yJ; zmKRQxBAal$U;egqz;5!aBAw!ilKka!7^*%oA>;#^4#Xi!^yB3 zfp z%VOZMWcG*}1>y?sruY+qXN>x>Njn(cd2J@*;fl+!X%f30YHZ}CzL;zRF&Ed&f!#kz zj~;;S-Ht1?i)05{laqZcPG9>?;$ybPLpS`0Ko3%nsf7{6Vpo&SuaDItF|0VOB>C~; z#Zq4JMQRx+&E97&qylrMae2f*`{mo(r`~c#p4Qd=*Qzzs2)1x5A;H#GN8No<&HH=; zV#BUs?ruXMg_O{p&kk)G7rI##$am|J&8KB#aDu@U7YwhVdftb|b$-Vm87;@FRB)#^ zHqbC%pC6;oFBS)WJU=4f>wwZJ*J;mOHXkGt5AySC{R}hU52+F2(&r<$i68x}Sk)O} zwOai+YHD0GSf3Njoihf^Jh&Lz)kG5=wMX7gkJrV9N5*ikAhVR1GuSMr?T?Jj3=Ot} zpWly!L4=IAU!b74q>$ZR%g*V{X-5yLxA)#uabpY~g$FO0xDlBN#iWIQXgQze?I(A% zUn48P1wd4%u*RR>ecP}SUym%V92+l!@%{M<0u5$TKO*TU4r;i36c`vtRQKi~80pk- z_1SKUzfKTT!LOjL+hIcM%{Rs!h>mvN z0;P$~vZ;UTOy`<^6}?F?GT!@8zK#wrZdte3!(wtkJP5Edm9(IVroPf_I^{(2MY_p`qk42KF&58hE;s{rx&oW8U$h;m1AkD-%I+c zqVn4Nl&|$qP5tWGi)>C2vS;6S&MAi?;7639C@WI99So?5arX2S?Dc;n7oPceA8voe zkpwj0ci})_MGtR$>}}16Afnfc6Fco%0_nr>u)98$lg6ui2(6P6DaxJ}>K{ecdXD#J zkw;JO{Ak#>i^5b&u2uSxh<2hq=xmjOq5z`(FGFMMZt)FwZVA6_=qCl&l5BeXYDuY5R4vB@$3*vVH?)$j2hEx5 z>RwOAFCh(FuF*{#{0&7t^#ZwO&ldBLqr8mU((GW|HdOa<&RQ--J`WAP>kh~rK0ezX zc~6f4sFQ1J|AD;h{DHLF4=AyY*G$u$Fm_|;@@YTk&Kb|$Hd}z=3ivm0opyP8w);=< zh0v{-KiXWZu<))EFc;_VoLhE$DJwQQbBq+e_t$h|?dm+~o`E5Q(!dJZO^P*ZO%_jZj!lQTw^rnLyxgl<(cRcT*pxV&}ib-ZwYa z7#UQQjEkTAIj5{?9iyqCSs7_;JpY<^fLp*=Lo?>M;{9=cH2s6iMH?&c9lU+UI)kw; z*?gvn9Ngye8Pnc(n%oa`u%rp&8+6iB`w{e=vdzL32l=^9`?s@L$#Dy>THRd)j2J~bJR}*y*<)94|1?K&5Bd10Y!zA1B#x8d8}ySrAIjt~kH(_Q@w)|F**D?^A>#nd9HB6s)QG0*e)!KxY)3u9LLD0@yX z&virN60tE6?Q7G+X~TL=-ZgaAzRbkv-t7dns_K+nM54yGI|&qC{A&f@CPu5ubUZdj zaI{3!vjTWItazVUFU|}%ML5072mhU1WRbGs;nuT7@(+lI!^G}L+AsvNxitIpPGg^Hq)Qh$4{6KWC89@wB6s)^!RhrT!R_U$lZ(|$!8MBCDmMd2b24!9^(y8F^n;J@tOil z@z3gCSQP7T2KZyx?y<`-G*Wa6({7y%{V7D>KXd0_vvs$tO6}yT7r0j^ zGV@-Q@B!X`s{Iprr}N2i`?`}z2D7Hp4;Jpe6J5V--q!wkkGcX&d;j^pSM~7&pC4Ix z9tky=WRoItfOx-*H_T9Ycq6JgIA#^vQGQVa8KzD0kJhydGdLt~@6o153+ zTon8ZH16;R`r!ZL?7gG8?&J4yrP7uPMRMOlwlb202Fflwvyxqi?Ah(EL=xGBvSn4u zUJaCy?Ch=Vz1Q!0dAq-#&mX^W&hLHhbMAAzd#~qsjO%edu1EJSs)d30NJuio-OIJ= zlR@LtfddEb+_?kCFgTLp`kB&P8sVfbgUBp&Jvf#&YXIW)_xqcj{d419GWTKx22SXr zw6}kWE~T1}WbO{|m@_R*+QkF=uF*9VQY?z)2B~6&v!jK)8?~q z3Ox;+?Cey2==qcptv<4ET}G@ZuP%?AmkA<9;1qj3dD4p3AoTa4T)HxxUiI+NBlY0} z2M)?^S!rZ8`moh@(;r7I?*c})#b)h`ozLD7Bv zrPfyubmrO&u`;2=h)#$8R0{bmWB+~Xj&XECsIgX|!{|lsQ30wgpmC9ro!}MRBu3KF zpz=e(G9V}@ESFheYe7L&ec8;=WZ~BJ${pEElojJVG?1XtBF@Sh2L&ROQ3NdHjW;K! z6xS-3m9q%e*TfRffMmBg+-n|12W$5+b%)n0K!2q2S!(haH56L=O z=NGCc zMOq|&tZ&Ntqf&Qg*rC4qxsA2GVqsEj@_bG{)oJZ&$|av249bZU5+#oxl+tpPp8s*a z+=*LA%Vka3kc&P&deWK`4Wy_CTC-2dVn;&Lr=sOffT?ahd!f#l)h!%`=`E^FRxe)+|EXZgoh4 z0H2uHO#_z1L?t!D_0!6A=i>6VzRp0NG&cU<`qui21`#<^fqy>ah_$;E=;b8l`P+Cdou_r{?aNGUkO#-iIl zkOtkOXuw2wMQO`Y$(ASRZbDnS*y?;wUteE8JB^Q*!f{=6?_HMhij~D(d7JRE=H^;k&@uvJp6NzI?X1uBL z+zKD>NhTZ!RQ}W9GS|kfmEq|b9o++>3^Wr)d#09_s7*{v&}c~z8Fua5>AE~oN{N;| zq~3gr`Nk^C%e{*>(lYzkOxuglS!B#}N~bMG=#ONt->XZvQ*JZvah>)^XY~9=sqSV! zld29^M3eX?U(2cQuXiWjHSb+mTpp2bp^Q0_rEA}&U_{${u&y@AC`Rwr9`E+PfhpIl z$&0tmhXa>6&-n0s6K1Or9Io!XWE1wH|EY3lyzTL^f@dy&Ckd2G|L*<$r?EME#`6_b z>Z!m+s(pUyz6FkAL(Y_GN7HjF#Gc#q*M=xo-3q_kAnerQlxbtKB=P4!+=D(PZzUn! zCi}BJ_U0z$?)vAloaRHGoy<_npdK@9roHYk)(6u--+gM?vhUHf|Gi7$`m#$wUFKb1 z$R8!yqjB=L;lZAWUQrQ$Tjt*5hl&q%9

    RT3K;kWZ$%OYM?!8Nk4_LWnbp&iYv4A zv1z))^Ed0oY!+4snGxv+(!5gnS>gt6^;+C# z3&t(F+?*2i`>q!8hZ^mTWw82SyF5|ms~i=_=0$txM~+ z&JD4iDE>nm$g)GOVKpN|v0RL?ed-27WYQ1BMA#e8Z!kfDo1NPV!{?Xhy@uNL%JT2A z&-r@>1Oz8!(VNB@?p#4lKQMG^7Y$cJSi3wJQ|~OhE>L~{_U%_;VOZKu`J?L|NBeuB z6{26y54PrQ`I2_A@BXfc68CqN5dPEl-Y!;oEYfAIIDt0splMqASa%akg`nJFN#82p zwJEo?yw%A?nP6wjSr5tMwCn?IRiDy=?b3K6bON}`_D>#&Xq&rz+x&L%s>5PlSK(gu z{o!iScY12#eC;BC91Rbw{JyJvXK=XYY?b#~a>k^$Fm1e)Jg~oZ+jjUQ`eAX6>^Cua zy;ubGm)9mTSKP{Le}A;TBgAKa%VMluX1O5D%|>%tz+mgD49yX}>oQ}jPgh^_ye|7` z7y9ltwz4)VzJB7E>O{6qS-ybP(q&~wyE_%@4Hj9|5+`WKh+b+cxjujbMz9u#XYw#l_m0s7x@LFyP<1y zW$#hv@lRn+dm#+iUlGc*AF7$x&k4HSxlkMTf1;l2!X2eFR}&7E|twxJprXGX zR;j5%=}y{c#+#f;QILSGC>^Dvf6kz+r2BrlPU*N=d|!T$_>uXyHR5Lk9V>pE#M@|; zyqnLPZ}^eI*>6)@)PsH&-35mk&g5YC!^D-SBVTw{ zh@TQqVwROyKC&$KXX-rfn69Ren#IpG$B{;p>e`!DH%(oQMF&3Q;6DGj6%MZwY|Z;9 zyY1JL&Y*&D5OAEF=Gw#amda)IyuOijZ1T-?&FfYkoYINjkGPPNQ&vKNP9BmOrlUU`D3!I|EYt*qRZab7AVqqkt`$?WW&oHd4EjgZr(O9^ym zG)$5kF0I}OPz{YrrmEEcb2_cPo7gpTCt5h^Mu0{ zTxqobYc|eJbl__jlV^%iKwnfmi+rPwB8_S;F5*YSd)&S;{4dZ z^o5^w(q7c$dQQ)duEhToXV=dw>|0h7Eb{(#G3DrDOvmxK#7Uk89(a#g>v5;PaZaYm zACK)^k1XofCk-xSkH|JgWO56UGe`<83@TJ`~iNz1Y2RaX3 znW&3C@5|Bh&4qR?youZy3$%+uHzB(!p&^=PRzrh5pHW);J+DEe=Myvj)SEj69p1Dl z5AmerK6AOzG2>F)S6tS5zb=a2EJVvJg>xpeL32%DI)48}>DZZHQrmowbc=ue(hz+! zivMcRQ+}^^33&<4SLU4Orvf$4INVd(n>UhQ`tWJWy^JIyb(SJ^7fC&dWC1nnoXW(- zs3q79@xu9q2gAEA)%qTl6a7`5t6pzLJt&r``df|N$mm|g4Vf`&w`i*K$K-agwX&9( z?VT)DjEnf61$(ir9bKZD*6tlSHfFK1Ka+{<&A3x$G(XdFSVLY{SMc}nu-8$?g!^-a zQ}^h?)1oc*Fnn;+z9nU&^H_1MxkA;;dSa_HsU+<=8<^W)KajmTd0e1 zhUXEvB=(}D`PMR@zOwG^si#t@jf_;L@oaY6rh*E4{eL^umX`Lf%q-Zx;TvlU6BgXG z3c*Jk8=K1S-$M}wKz#u$W~DoskjN>VEaPNC>uYnGUZG>)Ow2oBaxLF@zFmf zBFW!eqptYt;n#n@Ce$TZ@VGrss&bk6lXOb$R9s)F#McjXtxHDjYtzf|_3g`=U4ngo z#j;5X|7m>Rab@kb)`3jJU_ZnV0X`^jg_tb-=b3>Ibnjn(vYc1cg zbw}gQRGo@!d*+qPcOCBT3I3(o)bO@Y^W&F1!CiNEWH%+A`f(?~L&ketHLBaXoa?&^ zO~S6bd$XI={u$9pT4;IX5PIrZ!mPy+j)HKykU8_p=V4?P`uhA3`GDjQ4K?*12n?ex z1TOepo0XNNAC;9l*;aj!V%W87m*g$T&7L|XxN^xm?u~(L^#C)syo~GvH^i>5vYT)` z(wUVdusuC?zG<(2+f8(Y3KeOYBrkr*8;CMf{P%2iM|>e)>FO72Mz!7JakQ^FM@-dH zq9&PYCk9uO){aii1sbXM4_Z%6JGAW2yd&Gn%JlS1DKMDGy-qpt>U|?g2$CGvDwdX% zXVshc{LnOO`sEI3ng?-k2S?${4=%XXJndXQ}xq^Kj8Kc9ui? zdXF3&+}~UuJ@me|yy0u|$=|%%$CQIo-@Hw}BAoCc=R6DgO)Q)&`buS%>ea2Z?AX4s z{kY5_x8D!wSYY2AHL(nJ#4_w2KXvL96vcX=Z4tzyqY23dH8mPq;-UV|*Tdrr(kV!e zRE|4Y_y0aFE$ts5L0}1$vJqm6O`5GT`ZEjeRU;{H^uStv{NOmB!lB}VISBctnO!kdo7@dU7J?bW}7;u_+8%S^PMBoCyi^r5fMcIXNaPeKQ6R z-}_d%w5AypZYm&`=+&MTW~t8B$mJT%o_}5R$o$l!b_tM@;`g`f-IMA=< za#nd(b48AH${#CgVksPBY-oy3x3XlNs;K^yDNp!%v(!cyx$;U2YKwW+;&(%%%X(CDZ2N|fyIsJ4_Xlx+*7 zlUZ}^8Y`Rl^Q=0~cFI*`&q%KMFvnPcWfG|U>d6{+QRRj94jNykr|;$Kx{bDGql)d{ z{x^Y){nWnWSqOUkyM}t+6RGwI^*)=vBf3T+n0+8{#kbm6Wp6-i-9f)qzp~ro@JFEy z$J5x5Ee)r>dp;|_IQ-ctF0i-%NW|258YN{5A6L9U{2)u*A2TmgfJ|lQ|GgyWq!}7y zRH%>d-gKz&o$1m{)DRr&0V=bU8JB{3Uf#4TbJ=^Q8Q7TmN+$c_gVmi{4B{<%!k$_j zby9ZNRNx~^-$%*0^S|FUNZ%=+oSY0nJUmj8L#Xa|EG&9?GIx|b7DslsN8#rAoA+cs zc*10g?HfP0M^x+A$3HFa)2xG}KNz zto25{XIkhfv6qn$@|t))6{(WHmg7AX;E|g@sU1z z@x7QriNj6I&`;HW=4PF@nVS|TlctZYNm{IC<=CgUaO>i9#^% zQP+lz%4zrk(Bs>;ZyGSzUns?syRR7!A0i|PDu2QjA1-EN@>tm>q9xMUx={V<(wuh3 zF(&%7+bD|cS2hU=-J z;pUD8UCu04CL+^q&2d&4x~iwW995w)`>BaCN-6s2E?2H!!W!|_2#fA`c37U6dF8o9 zcx$LG%b&4Yy`*ZoCl4PSr-6}wv~9Od`rTZAyS{r31FFB(V`*ZWjqk|piDi0^Y*6~f zbJKRA)K%bS1H}VtnImh6LbF?j%(YTRCZ6ribW5Wv=`G=4p5CA-^kFRLo$`TBKukS7 zJy4Vzsu30v5<*)5D=Prmv+T#l*KbX7$Un8;I{iM{9;G>bd$y|y9}&yecW!W#!PYF1 zqQ+0^!oR|@Yz%*g#)%&}eNDtgc)WZfw=|w6wxqwLz;${?hSqj!c-Ev)ku$Zdy*_;? zFSox^{EVIiu7Bzo_U76jQ{MOWLTu^AdaDPDpYj$sEw&-8i;WNB5&ih#=I`piqmQjH z9UpnY%;c@eqw8!nQSEn$Gu6-BwpK)b4IZU7G`>+s+bE3pm1Tylu8?K;fG>mKm5J=f z-)e6tqC3g;?!YdpCMTD%_8}+vc!8+UmOJKv8gj@YHJ&95AGGKH(ePtxEKHpBo?NW0 zO@{(~@F|%qYXgId;yNqR5_`oQPZXVe_1dM;M<;wW&*;RVRF!Ap;oMB9h^U($WMm{b z0J_;$k`iU*Y<*)f1*3>n?%zUGcQb6CPWvW*EcE{9J<0Q-wdvY) zT;dWu$dqqonmxy7(VKG@bJu8AGrg{s$>>w})68~>GMeYODc6VBux3PNEv#00EN{DJ zmLl&%SH0!T)*nNs#)`(tkll*s|0{p((m=Kz-?w z5zlDGsQB)M8>V0DXl%LVeDGuSFO)7L77R07ZImhg9Y<9e72(~mSED6ga?a~mmCo5* z=ab{!-x*jNdm3{R&&3xRiJZ;--6g78n|q-+vuCJtX;CLlJ$FFIV=et z^Ye8>X@9LPRJjqtVvDVtY#Pxyfo>;JD0k8esihw*WeD%o`L~ID=1U9=@8v0jc)MLaT;chsb51@ zYjlT-6Y`wWY6l1Ri%`Okv}@RAS{F7o4;f-J{ns|;In zpyEzh$jT39P8=6{cw9Pn&j&eJy zV$!Z3|ze_mN9I2?98b%Q|#3{>B1f5Z0&)5&NxXHMHqrwJThRU1=FhYTAbXiJFvLYi&J zUi!c*SNiN6Octp;=Btu^gDi}km-k&p(39QlJT$qct4Cooe{TO}dfhhx%`g)P0Xu{uqDZWoe<%LT4IKITk~-JE9slZg^Bk_5EmRP5}(3Qjk8lvnq@Iy>L#A>&9(MgKy4c=T7CJbk#raVAK0 zL$J)#+dIi9(70|pL9@ZP<{)3*#hZ6ULxYV8Dv!4?*8*iY&ImL?nxF+SM*6u#Ojg!B zfciIXuZ{EmdzI;CXKn3Pw75#%st7rWwzXY5b|f~Je)*!0E=e?P?el}~a~DM7XE+Y{ zgih`~y9uG=Mkz1XF3su$RO!N;en6*8__W>VOD~GoCdSnlZr@?yl=brB(jmr5kqKzx zQ=nSzX)F+if@_!)>L?9mWik@blTtamUEV|Ln3xz7m&AJC+srH0qL!v60uqr^R7}e@ zGBq`IaLC4M=}_)_{>hB=a~1KH#>NR&Z{5~b!?4D^_P}b z3z(poY=Yr2Y@(=n&6m(ygrr>;kQruPFYV66@D;VRv=CCoXw%locl@ZErih6wmoFzY z-$7Fxr&_Y5jZKQ4pYe16`!rEW#6dANjr+-o!82;9+71YI@liRLP;=J6%uK|7yc2!P zgod72*|wX;?e@?B#bZ#qZ+o6JQi8(8K~9fP`)Fx{gM#83heXewy>RhjU`jvP){SjV zEZC&`y598S#f$PS<(mTS?LG7JdwVat7MvEomb2bEX#8)T;_I~N>gY?EcUo*wN6%b3 zZ`>JR$G)Cakdf78=Wi0jgd|WovhNMl;M72hsTp7x3Onde**c8{Yy@Cek`5x@^l?+!P|n4rP`wZpIZ zimF2SFG-J~D1yt1gjW+vqMrCv$>{1syB}iS%|HKaraFv;M^D`^B*XkPJ|*88)&FJ3 zP1jIRQ&Y3I&x(bP?-Uys*N1yMXmIw9v$7I0*Gx6=U0@Xeo$KrGw;iZq5fgL8hqUF` z9PmSaVHDxe4~0oyPR^mG^!U&usbgQa!{7vU_xCqI+6-(!C>Cnu(2%)1nr>c32QMm{ zlDD9CY;JDOgbbVy+KH$o-svdN*+Jx~Kr4hXvO}oDIo-T1M;u?%%-=9|Wi{bE9lLm+ACr4GA8o z^uGHIV<$)_k6Ah@Dk}M95lER!etJ52q~;H#ap^d-gvk zyh%s)yuABZV_M|Lj~_krW}D3%*60=LO+O-)WtxOW=L^S@l9IT%I4@ik&%<7l$4a@@ zn@z9C%e&jz*f0p0zx2eQ2T2KuJXD)ymheL)p?}Q+YenUU(Nq|i!yXT9xC$j6Izg#* z_QyHFK+Z3Pko<+nBa{ELXHcnBi=Ds#gW7?6*REZI#YM;X`0OkOhQxdBra#4X_H3&* zb@aq~tr`qs<-?um@x`>FNc-*`A|fKn`cV0AcU`*EcbbzET8^HVF-Z%8c!PRuVPpnd z#Dtq_Y>34?cPz$`(f$EcdC`T8et0RVhlyy74#iJ}#%nd$zE~*WXQEhRb#-;?052)U z!}pJqY{AY!^1KU4^l zpE$6GmtK4A4R2w{5 zC3cKUOV_;iSTAgoxl6^)vQMKfy$nTd1_p+)bArZ`8l`)QcdbjTJ!OTN8QIu!^xHoR z31MSGzb=~dWg#QI3c5?9g^+NrD<9>i7YHg!QW@MF-Z8 zRHqskD-WDeKUaU}l$oA|)FPn30UkXXmD)b^#t9SvfLy-`z4; z4~&sJsB9n4OioDn0|{T0+_!mrs`)4&ldQkKuN?b2lkUPJfx6S%*QaXQ0Rearu0RX; z$HOB(KfkT54PR#kG^%Sc-%7mKOTlF4wr*Z#1a}9{Di;iEfP$JPCW(on+}w9CVgcp z5fK*th%gRZ6YzU5r3~|r2tfpC>F-Ht1&&5Tcak+c*RQt9^)0IpCnqO~J2;S!;SiV< z75L)C3#&miuq)R%ojX^An}BlxeMl?FGC*t=0v;U@T9%;U(y$z?W8c-#;mM}_hv2f1 zky*-a9~Xi)LKjRO6CY~17$oJxAVLHf!o=7qZ_n`XHbfi{jNehqawgooEPx$16GR`! zd{^M4G>LiDH;8NvA>ldq!Y*`kGu<>ZBgkywqjN&LP`lLE*O%-KHHz4=YgdwE7sR$B zoF}EJn)02XK(HGACX?#e#%=sqNL~oZ0L(Xg7vT%x*-Pl2(5-sWs)nH?2mAoYg6gJv zTXD5RWekUw_2Me1$48%vOuZm2{qm)j~`R{ZCh&GvJuN( zadH>89sWf$pF4`N{`J?Z`xN#Vkp#QVsVIG=S@P@m?*oR7j!sUT6d;Nja+zR$5%%h; zivuP*9riwXtX5=PD%w~eC>3h5xP?2^UKhTGEnu=*U0#69x0lEG{rmUf6L9S@E8+0T zn-U)zIe4qQJ=6T0ZBqStBRW5J z)yX{)BX-~*^{PrkO5V{YpVTkm($0GO;ln)u?l^sdA8}p?Dj&9D5fbe9I1@87t%1+j zZ73ycOW^;o9~Bz@`S~t5v8vj?#>MZ5i{q323+?MvM>au)0FTwZ0XMHf&HI?tlV31I z#C$dtl@FLJ!pO+jS0^%-H$`OVS(aUheiu7EC^>lU21iD&1s5(8WdNp0OnJy2-0iGV zUZwaWA~1$|ptWg(^>*xh=W<(^pcy8nNZi6?2<*;Lv|V&y2jFl`1;U`Yi)a z!0~_ToIU{6sN}ffGXobP^{@=DpHlY=d0*uVAEc z-@I-oZQ`wVu7I=|Fl}yT){>}v9HWY2^AS|xu#U85rw=?B$0Cn*6c#?^uc?+B-@6`` zv~r-jee&c4Fhp8E7&&73Fd=GUVgjiF(|cS$4BLr3>wsFfHHKX=DXFP7U|x}R(36s? zDn$W0X$2ZsSrXmx*lX4z{L}Y&dD?{4>Fn=kWx~WfSjqkCw&#i6l)RmXdZj2C8<8_pe{SO2%?qxu&`z4IUlL#&z|k0rdCbktL}YEiltjo z%x^C)mXz4N?V^Ih)5NPg#$jfXe$_%;qXRw$eoQrzqk8&({qq8dk(7OcWVE~wA~o1= zXvkoBs6~(ucwL5(Sfr~yNJa8pK?ys?ar)9TGHCo7b)C3D&o1%#(2OZGR12sx$cw@% z0y30jW@i4nyEtmc)pio-1wZVE@DL)rn7u~z*vrey)6*XDe!wgSL3n*bF$PJ?F(!O6 zh8QR*AVdp^)fPBRIwwWkAiGNThTFC8*uH(AA87r!aU4FFq4!cDzPzlA?!wAf^3 z&S5nrZ$ZVVtE;R3%^`VmQUYWCxQuM9r{@3z!wNF*UrkM+bm%IY!nx-xMG#lka2H1b zSycjZ;G4gs*24z##|krnSPp2{2p!su?-f^qED@6$5Pe<3nVp`S8*9s*gJL^m%`2~R zgdNn=t2Sj}kt{1OE-TwGc^>g%MVucrQ-Xuli-WMh7;KdnktzAy=nglx=4I12F>&pO zeAaz;-MPMlj;~fX(xt)Ff@9-M(A|e1j}tullPA0Lb;kwjgwv=nJ^hQtvzz4)k!Z8HdXX30 z#ydZuBT+6)Akoy3)(?p=+!Wlk>)Ptd3>&3Vh62r0UFHCiY4+SkWNb+ZFv0Ky9h=6iqBv1^;b7_Y6{ zwt0QSrW4~ssxIJElU#OPco~6^sixY-?5R^D#*JlF6#9T z0%xiem1pn|J?_JynQ@SlQ(8XoJ>SMv-4hW;78xF%LP*{>w$|2tLcme%@$}bf;@O)l>zN zCNaO%bIaDPGh-ckpmj9=xBZvp-3u`(ftY>9go-{S@1snF=Di?(4V3A_ot9HRQt7DM zrl?hV{Ol#&%vVoUM#&2eE9rcsrZ9iXb#>u4=Hua&HzBBl5pfdCJUpWqeutFuds*3j zKS=sK`1fCowR&B{fWbL{CseBjtAC-8bm9Ri*E%Qft_S`J7Ve2u^rSvY#!$~P`-4Aq zzZn6b(E8CV*egDo{RS!E9mJRB<>5KSEkRjFv9ZMX6VkN~;CVv}*9{6p!-O}|O)@gm zSk@xYd+`X%KPYhVHpb84Ve_?n{B2ASw`4r5B|G=Wd+fwtf4-&aydkZb^5HhSRP_HW z`Hdu(ohyjp0O4^kjgF2I%0s`D>@XF90W+|2_E&#cSdhLM#mpjo{nbXEuq}C(%nkkz zN%mu8UT^nz%5&TN?(LOuzw~hT9%;Y2kv9wvJGaPOnEI?>Mf`>Jvh0s zwo$s93SPbHMD^+jrtJcz=wz6>(}Oh(VX>p@z< zA_qnmbtfXMhtsbLIT?-E5I^~iou6^pe=h;-2Wv&p?E(T0_Fcl=_cDUXi%zzurKcCU zUgQqRCP%f$tjUtQfCpE|vGZhXEVVmaGH>4ag&Q|sdlJK}04suWoNN!UBNh-1tn+?9 zAQi#7j)|)eq7rLNi%Ty(k;-;$+i;QOajZq9#l<7-`C1Z)qY2ysvr^u;V77u9W|7|b zyoG9^>k%WtHGtQOIY~SS9fYE&5(djtal6>!z$NNpIe)c#-L!pbeA zBS*Z^|1(g?-!;U^7N`#!qch$hqP)wqiqr$=*#`x&L{nfo26)~g=_u=C@J6{d&4u1B`5?pC0 z>V3L?LS--=r}W#m%LWwl0DS-hG#07}sFrO%tBg?(_#)f>eR|gJCCNB02s7&bcWQK!%izIzDs|e%*D=ikb**1ojBxVd;G)Lr*{JW z1EPk5hFszu@F;4&bR~4 zh2G)a4{@jf7{OC@bzz3io7;jEb$)4SUrS|`{Fg1f*FUR>5p;jJ#5+~w_Yu_nMJ6qA zUd7nhn5}7->3?XT+{!`6q9+^I(NrysY90p;!&Ovsopo5?+0h=7P?i4k1H0CzjmS_a zTCAuLfhv$LAVw*&zY&W+tN`BH6O8_Yq{O|I+cpjs&NYnt324+wuqwRF!cdQ}Xpt+lWg_d< zP%^Tt!-7da*AP>beL`+=(ecqW5>DDL<67{a^^u0lQBa_v5*HVTYMphXN5A)jjT3;= zefRD=PmF8%Gvm1aOyhdQ^=ccQL8q`J3I@HMpFGg$Q%Sq=l>khNPkuXwK$&S06^nxh z4~C$hM|lGd5WI4rb7W+s5c33o{`?6fd1j==4$!P2@2<*yB&hn_p1Rw|XsM}fU=k5J z7c7nDLQ@dnswW~A{8Qj04(XBuz^)z>6FyvHqaN#TcvyG`Fztr^SJzIxr8LRqReJ63 z1j2!nU{Do!)< z-vfT=`DZD5p*i^D`}fCWzM321tzpOo-)#~O6cGd#45O_&a%>zFvBTo7d7gjn37t`V zacTsOZ3VHVS#RH*9&{(2t*p>y^1NK);Z#gxL_B`CsO=Bq-N?kAgt?Hqylg!lIlS{Z zLcDY9lmFe?RiV2lOxVqhiu3}NOK_VFDK9)@;^S$FJv}O52B>YAOH%TGptk!6i3vbW zF2eWe(^Gb$#-kg-!y8yY$TQ{>%s-4V4XCM6vJU>&(1!%$)hlIFF8{N2S3eX^F~1ol zHe{?}nXCdv8>gmTDShV(;yUO|B3I;xF>qTM(gRZneFccR_wL^PAw4~5*1zC%4r2N4fjVDm3@YEYrte0K1{;rogT8TZx^|DQz(FL5wf zVFF2Knd2(|W^n>t;5TfoVFB2HqK%>F#xCHE+nd)P!?9!+)?2&WUtHfQN=KvYE=2cV z_{(f6)yaAC&~Buoq7IX{fSBs0%}q~ZSni*O6#D4tD=7MC>#!WZ#K&J;Tl-&xel?9u zPfaZYX&_McDS(@S3g|nmz^p`oygeg!tsH{~car&wa{KmqsQD(v*->Q=rRHMlU_1fNHh}?&0iQ$H*up6F^r-OL zHu??sJpO+Uo6n`JQ*0nw0*e{5CYgkczX1X9-=SS9b)kHlNo|SlUU+tR_;z1WTU&mA zMBH9f$FP(A(ueQ=wh&vtQb+?WnwwW($^_@0JbkLR_YA&$_j%)p&PwD|LUWj1+3;eK&Kp5Z^laHo2VH+UmtNnpXMNqkvimoP_t1vQnfrz~o zUjBke2vTgS$KKu|ce}p=^TAYc34$K__uqe`r6J)SkhVvVG_6l}6i_WFuflpt^tu`Wvr*Ce>k7K6j5Bdv2TsBs+Vk zHm=y;MbUpx&QSYErRoJwTK$*cAV=Qg=i_7VD03||CSHpmU;t-6w0+$8Rfyy|?m|t~ zS$%FJ%U}Z3+XX?GeAktn;NV7V50{SYb4k*j^x8gd$_2a%$Q=5w@?ho%%&0|EbN40UXzELbD6mzZh@ED=`)}NM~eUs#6R8*F>FN?2E zQ2I#Kq9;MQoZceub+s|Z)H{gm%B@A8sg2DnU^jIM1S%SLL@F-)2Q9fr&5}h%MDQ?S zfjPOk^H*lbgierPB64sZw#ikaCEvgXH^4GsGNQw`0BxVw|Nxvec8 zbj;1i{6$3nsJ~uu-AzepXK9&r?Kdj7K-h)PoFNR{&D`|#sgacT!V?ozDQaNK=4smt z5@8*gDdjkNf30k7f0UOmtyeR84=67o#DXBHPH8FnYu8Yt_2a|@0o3ALoa_{{4PYi6|{%LJ(UB%>LaWpLhLuq}mixwl= zBA^Lv^W&xPnQ()kMT{z~#|==`iwFubEV;=AZCkuAJ3$}E!$#|YD|i72xW<6q01B79 z1)qvwVF1Mz2y8T-5sMqupP>-~A`Glj6lny42`7|p-0($!|G&pIv@7t(HC!1`bW^(9 z1(5Ox5~_XsuFv|9o)8xozWdKcc_L=C<=eu-f)Aabj4+HzJIXc~8w&u>LhDA*!6QfV zPz1xcIj9J)0YLx~s8is)fRcgJqw?{I2?s2qs&*Cz07GHJM@pY~cT(6u_PkmBbs7g3 z*X^&KRG65?=Ono2QD7i3f)DrdC(=sH$V0GBNq=e>g@>TjB!`XQgSd1=9c5*%1D>X3 zxL!a2`!t84+b|exOc-%43(Jn;!}WovFe(_>jVg&m-PZXR)k8BrE`3vPFbFUAdR|~6V#7%nAr2{QC5$v^K~(1@*8$KL=)^&LNWZ# z9R?(qVPPCh!otF!rnCau!D<)*hl+6ImWi&G3*|GwB`MqP3?s;;bM?UQo?~ii{z0sc z`5IbUB=*2tw!lrRZFU{1HM#$*t+Yq~%!{C)4CjSO2wcBKwu}&T892SJ;{eN4T?^U2 zVA?%+a4pBn@Kc!#Sn9&paQ7z0OxKU-OqAiyeBCs?55|6;znYuD_q9d>OAU8z@8G~q zvQZ!iFikdSSp)_Kh5~F9%G-(qxDNs{x-Z&HZdhG(^ z1ZErN6n3WWj2axm|FL>uWNxq|IXO9A-|&ke5AKj|jvigtzCd+}ul4mG0e-)cm9Q=$ zc?x`Yd+Z*dpvRZ5UadxG42|Vy0Px>kG&djBvd9+%Qt9-4MyCC=w01LtsE6P~&>+w-%W%Lh1RwdF}M-;d6cr61VQ^DQ&T?3^3y{X?|m3tDLrwp4W6! z&D2z#zgVaGU~PrKlx9O^O_`weccoU2xe9J|K9y&Z66YxT=St-||8f6PlrZJGGGp0) zwrYXtrb~o%#1ziIy4p5-Y2?K~L(bw7z!Oo?&=|Vvykz$HbOn`U7&-x~%^TeTbLw%XU(Y>k*kkQzf zMN-lxVUqWL6~*7Y8KXmXQ^Yh2j>&fw6A5@^p zmevIo$WV7Cn_aIxkkP)-J1Hva0(J%tQ323OfKYq3c~m}fkG{c343i{dQ*vKJ7YSzN z)&VF468l1HcC2F!l_4y~uVn*Bf)Im4VPKRJFIaW|QXq3AHh zV>PkfAe+u7BqT7Y?Z6_eUn#TA&Npv%JB{TZPXwU|CFOc>evHkDG@S*LgPmbT5x48) z+22L;7&GuR-jgS*Z3Vw{kHb~zq|6-dgHymmp%*nl!)0hyhTHa1x z7ar;C#A~2Yeu5F|>SfN)pcDu5JJ2jSZ&f$d@{w}b0F46H<{ zmZq<}sJ>!T1k1YrI}#Kq`)8Mg%Fdjcyt5e49c3i! zdwYHV`nNfjE>?7$nfU{F6L7Ie81f-IQO|hTrv!CEAprQOn4sv%VddxkRf_cGUIY^K zx^zSVIIeASV;)U256yPo|A~+=~@Xs~7IyKg*Dn)+R4s`PK@nIS)OCnVHu@MN& za^5MTB#1z}1?L3(4fM*%a^8IH?yy6YWUG!mhf76-h@zrDu(Lm1U3b&XA%z2!At)du zB+H^hEJ=1tMaVj4)cFl(69E)IqW~(QYY~Vx1Q?INn0VcHPaZB$Ikms%n4+RiPRuUaVN#VX;50rP z<_wmQMIMknavhkn=85`DJfm{?VG2kXUJKH zNmT1oR|NzGZBSRhN>`SZ`3o3U9D)~_*4%W|gRonjZ?%&}Qjp-o*&KnA11t;MS;1ix->7#w-9d~U1p9-&`P^xVI^x7wzwgeE@RR`3lf^$c&_x-26v z@8j*wA3w*T`oArwpANxydA*w_2lgvl)$K!*nVm`mgB^&5{P8jyi11Ci?N1-B=SW2-smJ&U z=$r*CMPER?j@y#lbb(0yzYmeCfbwdeY$RGXBVsj zPQy)x?N3R>WxSn*`x=-t$N`vPJd!jh{^>cv`+CEh3P4i?6RrUh7>ThKeNp33d4}H> zDmP0a*2z2*O!FgjG?CIxjbZ0Q;jdOf1SkCNPc*@ry&*U6>}S=}V>!-w^6Kj9EQtyv zxrqLX2DwYXo_&41jONXLcUh zLy9FV&tW@2n4JS=%JWtWeoLG`hHcyMI$y156O0K)kmGS%cjuMM{fFnXPCJp5uUHc7 zgMX42e>zJ}!j(F&Ad;CHCmDJQmGS+Hz=WV3n(X=I9?BKCDUx~HuWy_k_nmvkQ8itL zw3ydvIUMC3<8P@qf+IvytqvD1+&aO=qtMj!-}=aW;vdKL8fZ+@qk05K!`n^mek3(@ za!UDT)c*Y{1$qA&@~f8acGkp$^S+Y%=UqFl-i+$*-dKjMvVZPg>OM)@$hcqR&aj9C zh=|KWJ$KHMZNo6d*^?7736V0w7Y&2Ag1w9;35Hn;Y=vRs$0CL!qJk46>i_9tz~sSt z5B{n={WhiZy0n*fy4^a8dPOOD_c(Dg8dKYls_qhe=q(FZl0v`z+jHR}ev+TMtC9V$ zD<~2D)1!lfj>rx$Xgst9HUwVX17{X!XNXpg-BHJ2*nUvEfQ$OX z;OM~p8X_@5mYV=(Qw*Z@F}wg-D1^-jE(UXSwLRjZmiMMxXJ1NpV_ZXHP z96DKOvGOSvz6cHe$%vd{Fam+qsYAQr65wCCl|qa5*zaEVycgfG7CAXN$(H^20iXWe zM1W$RnEy41N$L^QxKtnP#@eG)cC>o>tXFMLAA~Q!J+7c2l70pK2Ro@@)2%_9J8)b< zyKMn9GJnikzwY95PWRV38`Yio1o81AazrBj0?6Cz>Qu@fynIP`6PJl%T47OwFFM*P zOayS5=ie2nZW!p-8V3j>F^G(Rn;;xV6%4@&e3-*X6c5@>Mu&`u5hI6-az_#kh{wiJ!53(0zT`#m%du ziXpvuTX5eqj}1fcsB*LOQ`~i53?74*$LYRjV?`NwQ84W)Aq&B??NO4uJU(5Y3WNLD8)W}{y&6b?jFIST zs2n0o{<7iZZuHC*b!?Smt)`>4oohOW?(+3S+{+V5;$21cm|5s^20P4%@H0Quo}Wo5UL3H<9}zwN*{kUI1BN z7`O;>_8D*=TVXc+fhN}?+J%jC3Vv)Bz?-HNdv7(5kAH*|N*_GFIZskf?ukRa?a$L> zWF#)YVQ1tL2pJl2pAf%pqo4rpZGy5Q;bCHM@^&9OQS{?Mx2A%^&fNjuWB83W0(0Ws zis3RVD2P(`>}UMh$J+~hXt{`#2m?6^Di@gE^4d*F)Ugx)$xXCAk9ubR8Gj&RtW#UjE7 zF(b_(?lbbSvUCL2Rp{+QnZX5ZH^Art%w%fF?u0lhC1qy!R|4V1yYcWmo%|yBUlf^@ zOU>S6aO)OS%7spy$|#CJuL&|%_A-s)k?+SNK~#DfgZBadxB&T|gRK*a1}GyF)OKI7 z#RTAA*g4K7K0k^GQX;oKnQ|OnY6+0EHGJXcdrzOBuf!^*UO&*C6+AZM>YI2PCeO6# zeIKGxMsVY00y%_*b<(dEe!8xaWqAYT6MuhyU7RV@U^@Ccf{$(Zh%IUbQal_SrXVgn zkQ=k}LNG$HWy@tKH4@o=53vlHi(2B(aWWJ`X^Zm`$=n~+ZPSbz9K)l+*H_%gsK zhCXV)-=(9tE?+^OLA0jzKCqjvuPkvLaTC=sH{S@Qy&fS2Mr;Nf<_~yb3Q$NtW7`-P zY!7O{N3x7l33Co}qFW8Wx7op&GyekvwcE`{Hxq0?Aa2 zS|K)=Dz5=WuwqQ%5qKZ#a<#_S$1WV9^<8he%Nn|t{#Xa~^ca-(^|Gys8)nEFNc(`N z4s=$Qmm~QAmB4(+k<4rhg{+eHOHyj>ep4TY4b*c0LouL?v2$}*Hn42Ud_qch>_p8r z=e>B*g4?Ey`UgU-d<3Dmcj*q-invc!Z8XfZ7AH4-HOHTc3KI(9*d8>|q9B;-?(PoL zPqA%V`)%b#qWLazZP}<=W~;f^|9rqQtOydwH1o%}6}T6_1O6|{-ZHGJwu=_tsDvm2 z0xBS&bR#Vwpol2lpfu7U-Ho6q2uL?bcS$#hbV+wgcXxbq`@HAJ`FFnOkI!|n_u6aS z>%M1%9f6Z;{CtIQs z*b|>sN)PdRcYuNi`Td*7D|;jZ zzy(cK?ZA{m`C*X-1|ibJfpC_kq48>XWCW)Zeu=#CA4N+>7@|E$_F%ThJ2}`4-~e#% z-ya6w6)SylehvR-h!80-G)v4nOoBR{AaEc@2_qvq`}JRODCn3LA9}e&YC<24QJwqhByG2M$Y{EfA1%o72 zkvofB-|)(9Feg)4Q*o)=cv?eLl!$+9O79;SxB-!T42RKsfwzDD8D36@&gWZCOzBPDFen=T+>n7&R%KtM$Yd;xOu_fTtw zjaGTuKQJ%=frZV+7$-CV?2Ba80n68Gf@)UmUbom4;^&u?oNq+NE^>LPQm4@oKAQR8 z)#xAy1-c5<;_2XmI?!$#KdIi2)TTWolYlb5m241hpt! z{sbh^OWV9G_%Tg>C1!E$;e*?=jxJ+m`Kv>Ny~&W=-pz-}yaEkxn zKq?>djtT4pB>r*vVxtOSVAwe)Ej?!r2X&6(?YoA!#1D$4C<~(7O zeN5p@3S>c7MbGp~MouulDA+PK4{)I;iiP5r*dUXb~`aqsK3cty~ACr7KKGln} z4qf`7Ipnkac<{E!TLAn9D{G;Qvb`n16Gz+I01=hFfAmNi=q$5B6_v(wzl9FSScMW? z)0kAD=2>aWu=wA%9bmgnLjeREXT@WdPHJs~|@L6gx(>9U^%o zz(>FkPOrEMt%Q+$2jDDILRrRUbpULaI3!g7fPBYq;Ot2C zj@wnun{?zt*2gu$;egEyngkF(zRA+8yM(L@1gP`#p;ct@ZMrIj?_U8R3gJr^e_UJ~ z2w906dH;^8~){epk#}@A112qex+i1zXw%m!1{*}N!R|fCqSMRE!7Vpr2`eS~lglS=}%%W?+v&0sVb`ZVqUufsw?ZXsQ8)q$tELrD+=VP%-~Az4iwA z?U`;!aUcsqW;rR4%9I-R&OyFcW5!?IYj_UF8g{KE1Tmt(>jK)^8%arZQ0867e)c^m zXm)AoDBt~}gL)dF^~re)pax#2fvNvw(uwa*h$<}qHiK7NLzZ^nja*$!Um-_M1GWW` z6x%3Fsyj`2ZF5|74vv zO!Q}{K$QZ{(HO{%Q&Uq&Jf{GV?*HINQ^@#uR%&2bU;`mLF!J;QDh2rJV(NiM4IGQp z7D1c+jIlChx#0gnh(ZNy_2MAp@cZ`d8F*F@<4!{S2ST~nKzgN|5r|YFYOMy&E?if{ zp9q=}Lt_eLUl2bPP8p8PO}I(hRl{ik+s&MBx1I0bKrSvSR1ID4xWk+kOh-}BCP|2i z?e6XE?eA9?8X(hSh@5`UB2;&dhNC%+r_=;5Qu5SGbib@E|VmCjFj5%Fp3i zDM4Mc{jv6H>c2WbJr-uXCsH1c%%en^-Oc;>EoSPW^Yv`_XVyAq((<^P!)U3vzEC zxx@cluegl{3(y@G_Wj4<1_80j)<@)@X3R@(Zc~SVrCtHfAs8>Nibq;3-QB0_?EKph z2_KB;h>89D?=p`vmr>jxdxunh_uv4qH&aOAPLbm$KxYXe2ZB77zZFQb_zb zl&5&It5PTgeJw)TrcjYa!@~6cXNV{^!hdAggz4yHJS&7ntiScous-DioDE7_00L4X z@k5w30Yx~k#BtRA=J>NB#b`wD;`+A|IX4(Jw#!!bP(7_ zZ~$Nd{i0%Q_qMkmfZ{#{I5<$*f2U5ar5X4v>11zVK81kce^3FsJ{0Kf)+0}TK)P{q z1oha|lNpHem2Tgdpmy5-lc#>VFaKX9qTUn2QIJwlg1n)=zW&@9bktSl<^Qv>80A38 zrE$NycAAp5`sseLjnP+pc?7-@;AU z!h?f&?iLnS7>D*RZy*Ow=Zu35f9+311^A~JPZU|X>cLiir$mW4fG!;Nd54?-+rXR5 zO10<$Aaw!Sf{3v2Z}3>sunP~;Gcz-xT(trV5A9M?K$#7_Qj&%-CrXEM`+W&^cl+Qt z*1Cp;r6SKSa!`mzm0uW2?SDne(9obDR^8U`5Qs{|K9%i*U>~9j7+QjClH_*e49!Tu z9P2gudXD?|W}sZBH2RH!if%IhCX68+i?~?#YYhX^kz3p^1nUSzKeZx#Nyr&bIzag- z3U>yli8@474(vIS?+gj?NQ%r%DW`Rs+nFS%@p8#9Wm}JQ@@w(FnTGb(0&^p{7Gjfh z2;D&x9K$$3Nq}I%us=1}Wzbj*<|e?6K7%HpP&*ZL*wz9}^MfL-g|nE%z(`>*J_(bu zviM&=mMk|{2=X%JH{K$hsYjKsIBt)l#L8;%vP2|8`R;6Dgu))@Gcc$k3< zS?kq(5ZG_c>k)+htHaW^Yl}N8p>*`}A_;C;k&*vJwf}v@sO@g9diGB}P8WwMk}Spk zdQY~ffx5b%MdGD^p&=0VB-fs4`FBcOEC~j4*yWsGn2&u}migpu*m4K#;@2>w#(!oL zdYb02>iJ$Cznq<2?Wd_Ti;s_f@*?>2?cPtfnluJ?{t|IZWo9-BAhRToU?9_&jYhHi z`Ua;M_0P-wiFxRBsL-EQk&aImBFgV%cx+MHg?xRRQa2oC!lTmrt88Se>`?8w9>7w1 zHLw|Jy4+(P7|IRia4rQK*X#+OMT|gqp>obe^f7gHPRsY&(ndoPJpwUjBQBP2#ET@V zeS9DpxsHBs_&M^!>#08$>JWLHS{^@(U<(5B@~MbfGdzV#yx2CIoILN!=Fg*zk;K;@O-Q6LAk6O*3#PtPQPA#qQzWNUq8-A;CPEUl`l7X9{h#t> zD>6vNWi|SgxH>oVZ;g(wPHR$!e7>W&&|b0eVQjgL#&#!Z%B||{8Y=Rat>BkQOuHY# zB8r9Wj@Y@aDnM4zaP8u}h@T$$ZnaF{-AdWcek7-$GJ-!=X$0CXrH*{673w_Wf(-Ix8_d}6ycD1J1ZkIbXwL>2#C#{qerp7yZQ zO+H6w)p*YypOBEGWW6VbySi~tW!FCu`nPv79?HT`B3fXlZ$44ZTkhhfqGOYL@dEWI z@QcjRW_#-&RdWJoZOP~i5iBa|dt1-p!yY=YGwdS?*`ei$@z6ka{=LJ{3|dEryVn8=+)TXpe$0YdF~>ViyViS#)sxx6C3bCZ?~CL(9W%v-5Ai z;HAu9mKEKvI`o|#BuDBtn(MZwhTA)B?IBI1tf}4~);$h{+*CB;&=m_%n{a7Cn>Q;f zR@PN%mQ9$p%Rj6ZU=zeUisY)ajh=+(pLg?#Ne#Wk&QxLok>s%EaHVaT3oi0*P5_d# z4{r7cS?=AtInMyTU+b?*87r~fsu?SjCUeWc7F5vsQE4MPoaZ!y!b7n6IOgIE9FdwD zJ_(eSnN%QgFENecwJZK9g}fg3zr{eqGIeQi-t3*Kff&T~B(?S&_O#IuBm6i`!a&*d zy;W$>+F>~0fr5!0k%CLH-%L#OR0dm+QCa!F4f&LjtMK|GQQI}+eM6FG8Ix6t6rI*UiC~M zSN0{G_jS{QEtZf>%2ghFsf{tC&57U#$Qz_}gc`umNU7wl`|X2@cr;VRJzsD!?+k?l zi6nzkjEGAD9?snz(z&~h$e@Yp8v+}wqeQsl0koMbT9mAEbTk4`i|E-~BcmqOzC874 zLCs(=53n(}-f~;7hNelg9Bv`mDR=ykUv)jrEHzh_`L1{C)MbA&ms)avDIuCI^}!lz z6yRX(-AaL^R`$b5uYAAKHP4zUH5T<`xnE5TuONTC6eJoJCjT5xFg^j5`PMaFJ{|y? z(7dQ6``SX96#MRL(o%~obgW$*0p>;3W1vScY~cuDdkmLoUuV=(*k$FM zc+|`Jbpz#vPW!Y9SMKL8d!8ll+la6C^aoRo6}*UGekU*89o7;U5g0h-B(9icILv%; zR5zs7#k;Z+ZhKZ@+-3;0Gb(N*>hrJ#E8p#!M+~2vntZyLTnv}W&y)fKP9va`?l;|c zI#A>^ce_aEvs+$Crvg}g(mp|@SjO+3@hCJQ`czr&XjZ5FlK-ljvZBIhPS|!UPrGq{ zzkhd!&NS#1t5L929MnZmFP%SqHa;-URm*H4WBH2AduN`)mN9EM+hU-28B8r*yI?Iz zU2|qeic+h=53WAMa!=eAbyd4U1p{nHG-mA3upWQKo5 z{E5>uGF0>Sl+@K6Z{pQtf@IzPIlVw{dLG)r#)4zf62Q@aIm*% z^yd+is{S}zexdE!0cUwREw2h};sM%&<-{L9-aowHr4U)3;OUAEt9pRkDE4@lEZFrV zbbRBx%%_iEDx`{$l5{$cF=^%vX8RR>=ALRz{tV&H-#`XVMG6u8;XmB)!gZRL8?iav z-P*96O4dzX_k-?`{OnmqK0UNjoC(<9X*pfEub7k3;I+3YQ(OHRE99Sr7b_fi;(jS< z@+BZVJA1LKKfmr?$DiT0cDbv|@V_AZ3YNR;rD4$h@L(l5fW-FXCy5AnNsAkB?Cj%X z+cw%l7G^E`%`VQem6%AL{=gQXo%u-S0`)TvtCTc}-dLWofb(HUSFaXiUv@=1 z9SlzBU2FQVskSkhZ9IsNn`phnVWw6X=n$7e*^$sZ+XxV1Fe$vY`WIj>CLEi;$*RSY zlMj@bxJ`;hl&*)95oUUO$JkZbh=N_PpB`N9h11JYlfv}@Qv)+z{$-h&qx}OOa5H1o z?=)Q%>r;JcX`V?(B^^yigoMEBoNa94TecPPmlOJ!r5FNrTiZj^uL00-Wabg<$IU0N z16ON~7+D9kXn$C@1UMaQQ-RmoO>>=?EYbI$a#aora@v!G<$xDc<0NaceK-W@FR0Lg z4f>-y>50!0IvG%}GvP!W#X_^bl@QdX*?JUUYZ@51?)k?%2+`3I2~^LE^aO>b06Lu4 zLIp3$41HZFApxK#z+RAoBxo!KAve_JLGlT--J=O>X^5gbLg{q2u9=I|(OKDr1GROn zUfrY~p{Kt(eWesa{b^P3L-gwK@cO9J&a1)f2>S%C4+CjkOao6Wez36*R@tL)R&Re? z>>kXoGj0!^8~eiy>k^}H6!qkVc(~yxTdqd#Yf`%K7x_9SF@Q5H!4g?$OhTte`Qg9mDNI_kqcrz@j2+ zC>kSaU|a52k!vzx%a2PO|-YyxK{{|CQCWL+P(uOeI~h`%W|4ziC~4%+nO|~k;Hd(RvpzGgrnV2 z7%wezX;6j8PANY;AweWCJcWpJAU^@#GsxT`BR@vG8EKE^RH;>iib>yIPqtj@T_Wz5 zFb3l>g`J(TDrGk}Dt=x#5jZ!FO@P0Pl&WbG%zYgMLAyUR|ClJZ1a2Jgagjt2^=k{0 zD5Yjy{8HlbD$#fGtX1JheSC)UmHX4e!|!YS=}UFgY;P0IRfT{#`E|$ibl-8){csMRnDj9BG%bTTDJkadfk8j&iAlwpN;UO-GH#xe zX#N%#S8?W#KX5u38s+5)?pk#$U2b~;lmOIX`g&xz5X|9H{?Y5!=5%$%<%u*z7N4>D z4H6b&1EsqE4;Nt4VE&C{9KC3&7*snyeOhTiKNvGX)topFu8!S~O?G$hiZK|i**X}$ zRU0tKtU2`KMKB!w_OK$3fQOkq0{!7k&_wJS7hOSEESZ?rkM8(ERr8g-9`W}R*$}>{ zS5ZPl*qk~H7Ri|Q4)M0O z_D%)UoZt|q4Ndpl?l|kS^K)wy&K#B0*JLFR%M4+cobx(6n_Qeugw;+~8ucbsJM;2o z+kHSqHJW;+m}_kGkX1-l*2*flsObD+VVzM_N{U`xMN||LP{;{K3MNjZ+WS!VfnU?MfHK@84fw6x`WqtLd>S3B?z8_iCUPr;m4$>j;16n3taq zYXe>`)8lBWpV%Bi_KoR;mWzaEP0jQi3M(t8D>NmBL9cxh+=rHXc`csPNWHx*j*SHe zQXfJa!Ll~xYeibK`Y+%B?a!5lROZJTY;s>L@z%vcmdc&M)h& zEIRh$Q8Yp|NLP-ft)oS@J^aYsG~?lPxLyGieo$cUL<bdnPI$gt<0b6rjo=I-nK> zq6IKSwm`9UDP7Uh+>F#hL6d3V_y-pQDG#bON#5;U5&YMy2`*2?2?^m15-==srKOup z^6Kffc1Z>b7>gzb2apYq6pD$<+m!;~Y*#8L9~PDAgRV>MMCmmLudODt2aM@B6H=e)$Xp-PA?V0g0YiC*kz z?RHU9z2Ask!Tl7R4}ZxID!X}XVyKP3L>AQIpv#tiTJp<1Ko}jMFK*Lip0X?N{lP)$(YrD=Coqmh7R*f zi!%faU{T;NQEXZ2d`G7d%qtT-0KC#G$_0{Eh}s;P!O>ec1rx?7f%ej=l3F@^r*9#8 zYUzFUv+rt^=cIK*)bB1ePW0keESyRkLgQxF373?ZmYi+b^qR=(9)t`PSn*Hi9}To# zynG`)e07=LDWRC1=7A7}4fgwH+%r1OvQBD}kMnDQ&g7q1!|Y*^_f|gA{?wXIlrW+0i_E`hrE|kXV!$(@`x%$LQ;ij)HlwyHvR=okByM zrNjjAl3vF^qL)a(z;dtgh)mrjeR}X%`T?K^&ZVSIZ;mFCu?1wtOI6D9ygze|7rpi; z2IM3--c>2VO(iK=ywXNAG11q5uo2tjU^SRjJ-96e61UN!)1B896{Cqzv2UZO=PQPa z2~rOnOzON|n2*F?zq6b40dq*D)HLe*_w7LUa~}Rd#ewWa8sb=kYo^YANCD?feZ3qoSeVG-nnSNv3RS zS_u@KpQq&J{xI^rYEQM!4Y&!^A-B;6^N>IvKOG&fyGC$qERd2RF&4jKR^LAu4%Y$Q zM|ETi+40eIwySf;ju-Z73)!t>lqmV=(Tk(C2u;jo_XErTG^}C+X_Sln+F7|Tm(qq! zV*F+TQ;Eeu*BP3U4Xw#@zT>14vF8rh08EfjyIrQ^Lo+Ujfqikeqm!7)Rt0YpqTtQ zOKI>MAy0pmA9>Qa8S#<|{djh4_GSf*y|drGO&m|i1Gyvj82SJPcKx*%f*p-lny#s3 z)8P$|u(3G}zdI*Waw^hDXBo+V@?IWPw*EUD&O{`jHaco$-CuJAESL^+J(Xywc!eT; zObnT&ZdXm~Qr*Q-_vK0ZqC8!eeRmYH=|lc zJU`Xwhx?85x;4j1l{R#VEg}d6c|Bh_kW-M;u2RJ#1Z-CopfJl6#|K;h&|1Bc15}!4 zJXewbA1fids1s#f=^HUOlJo@xQt^qRtKQG`QEiMowzqe5TAlT=UQ+By z7)B?fWtBBEgPv_={8X=4Surt#0|T9oy0OC1?--stfJYfr{4h3;)_Uv!h@0^+L}pVi z?DoxWle{$-rGRw$`9*VcOIs%JB(ImVPr0vG;>-0fZ(g+c$ws&`a?1!QxvS|kV7v_P zou9XwjTaeBWs;vu4e4==<^8@wwyu_!bEbUDO8Lt-viuBMcrARYgINnARr9()Bva#Y zbT(-Vp75BgtDADyggsy-W`iXrX?@6I+t4px>GBSatS;QI+bNwr@t4iXgQoi@7aVbM zC6>vTJAt?q9QxnfY|O^#*L==T8IC@8Wtf+~S@fy=(tdiutV=VJfR&gl!`7t3vLk$~n69onFYk1xWo0}j zYIu(KO;MS>Q_NdVKyxj!?oF6Q;t?G#aHW_J> zz{8ehM(@kkuL`|`JzQZe=2wNY{&0&>13Gs^^KsjM>eyca2#GobJjP^Yg&$vc!f;zi zmZ~xe3X!0|(dU_~O9MymmPvJg;-$-rYL|EDcW9rfd3qA@Drjh=OVM#C;1UuG&>jJZ z7E7clY{%s-m%UG|g!o`c1^qG4mb!$*D^1g5Q0ZW1poc zj)LN}+cqKSQ)F<^fDYkRL=4p=dZ#o?P0@^Fmk53=WOT%&!ZN ziE(ks8-32kr)tncPy1$Ss$$wyN;E2pUUSOoKIal0sMHvpGbheTu0#HCG!;@-c85ex z=8vTNV$*70>cn^*Vg>_a`wmiA2Hh?p`kU>JLzVb3xeVk>s6Pf513ys2y~mY69k>+X zQ>I>JLdXa?2RTc%`ObgyQ?k2@O(ZC_Rb>_hy9R$MMm)yN5ByvDZvzh1r=^mltfD(~PI9 zUelKPQEBnw?1+Jb<8fE+Sax7)D&M>KH-c_zTeaE696UUkq9G<`QPHj5j(teH!G!p*3ZH=q;vF5s4R zBh0wBT^}jQ2ADP7Wld3xdFFEmCxaSf)>Ad8`*luSb(T5=`((?OOiI|(``dMwsXYQA zV-@m;Y+&mlwPU-DCI>&tGDZ2*t}$QVqKMOx#4z)R-f<^jiDn zQBWYSOiT>FG_TDhWKXH%&aOWFtPZe*f^3D=>G9u*I`a@}qhB|oBw&4kNHaz7C2+Vm zIZGCNQ?vm8($-#Ix?s)Gn(PZtYI{LtAUT~$NQwahTQ>2NP(V#UApDTigu^uQ=g;}2 zbab-L8g3xBO}oiEl($FJ>A&jSoPI%Hy!05|BYHQ4bL@LpQ(y6S9g^<`~LZ@6E= z3#X)`6h4}aINO474N6r63|TIYfKZReho|)j$SZ}rxH8!@xw_2Hr{plk2{5tbuPOhI zlmM{%-Mihx+Dlmq_A+8pMz$-RYctG39`=Td#cIHy<43)`YR9s=b`EvCHO(4Sr`~fI zTkdC8ny6BXY?K101B-~m5a@BG^l0@P$%+ggCNcoAjnv#ZnpTDe3L5HXN7KiV=bW8^ zfhp+>4A~yT1%RHLO@dJD4q3PVb;ncp!%aR$-@pFXkB^ogJ$n0G7XKAW)s0)F4@@MI zWW9PHNRk%{OEBW9u5;S0GYx5en-umJ@vpgMr*j>@&`&NjYvAOfFyimCfU4w&^D}3T z4!fpy2KI~HIu-T?izFllDOV>n8nwa|R9vU|G$apeP@I6+0DLUi)Lh&P`lceH0FK}@kwh{hB@SFTju~k}q z)jW)hIHea>Cxh~u{NHLECZCPuY?fX4FX+6Ro5Q(#O5q&CMCA z_eDi_@dGR^9nK$M5hp3oNyJ7*om(BOcJ2;lGYF7p22#9#sX2kmXRC`hHhQOM7aK=1PyD2mS>F4S1Q(Aqz+w2*_ERa#lXf2Y z)q#et7?t*|DgW*`TbAjjiOlo&_Z_bBJ%;ClYllPzT|{qS-y zYxja(zty@tGWypz-=6Yu(=BhG#z6wTcCEJHPIT6~OW#cS1#Cl=l8?C`^iN{AW%EhJ z!hV%oM%jORW^Q>vPp{;i3>3QdYL=GYT9dpf>B$Oo4o>=)wdpA{y%eF$Sm-{TvqFfze#`A2 z)zwwbi(B_STHi!Q)=z2JzLH$oDV);3E4%X>QTvt{bRkfGmynQvmOAuw=_^AG(C-=4 z%;X_$Fd1Xx>iXK%9WntIn1ceGT0yZ}47jqmNzU7hqS8L;G-ea^oly_ismX}87qN_o zetV%GFWleX%C9N#K_}(^RcX6@pr}brF2vc+^zN>5o~|3K5(5=#iOJthbw<_CFT4iu zzsu3uZQj11x_)wE@Qv6?qmZd}MTX z?iJ&^k4=8P#V=N;>Lm3R-&tkeKz;leg^)0dznMj&zE{>ZOZGi^so5(r%+9Xa_1FqF zaTTI?fmbz7yP4<+53@;&*`pQ;-95(N<44gd9LJUV`iOhPet%>f920Gfj*oR~>4x*P z7bQ|sA2VTXZbqHq-NhjdNRyQPNr`s5b^WJ+4aVOVaSAnPMzTmFK#i7|XfRc6znRx) z^X=%gUL<}Duu;|zL~`;$@pqJmI}NPhFdg3BszG;l?u?7!@zWk+?2Kn3o0ZGFDP73ml6cn)n{@`h^#IyQGR?|w+aoR;KYzaMonv)3xExqks@$87_ zKi-!=DZL~AeTvXBxe+76US!`;@}r5duY5Y&u`}!DahO-AbR<4!J??RoZqSep0 z8+I4Iw`u98@xEVZx>a8O!}Oi$$oCV+NI|k9gT_{jw6yL0PM4Y8LngB+Z@*iyajfoc z`gbQt_?{RAy9l?PZFPh|%caNU*Oc>k#*TNdqw(2q&-j(V%qM72fPn5I$eDnq1c5>&00#*Hw-tJ6fE2~+uL@r9 zeGIg1;a7?Q9}Pa~wERiK%E`sOdyOW&x>UOzUn2h7>4D{n+@k#xEUphiTT>P`!f)P` zN2N1st@by?Ss))8^7Wi4q_@Pm0Q2Q^mN=q<20U`eJR=VZ8zvWXG@bGtHYDIzq6YG zOFe~->`sRR&KtphTY4)wb8-vcqeAQAX}5k``xSt(KWEaxBa*;Ojz77o4TE=P1{umL z9@AqY7CReSgGNK;fBVw^Ov??MsP*^0;Ycc`_w{SWTU;tJ+C`R>)I`w>Zd4H#97GWtFxax6e}$ za_uiDa>Jv7uOV*Ugd>1O+#D4iaBh`hlIyNh5y!`>hdEXI>FZ77f$!5)cz3r~36hea z+bmeq6wxROwR=pzz7f6=_X@(LSL&?JZf>EV$1E{URG`aw8+1}^yKl&-5zF)bRbj2E z7^a+@Oy0xw_3ak23?<3hI%O8q?qCJQs4#-=_%1bqH)~%AqHd>4=}~tLE}(c@^{J_o zexVh7o~ce;W@bc}|7_uP1TJm6d#p{=Zt{LTMuMAfjk6~sws2qH{OBi=hZg4!i`Ro> zlM8fj>av|&e=071(jq{9Eiuu;+RN*DICIy-$3H(1r%40}&H!U{&-yOmSLdUd&48bJ zJB`>;J$E*znm%JiJ}sQ_QO55RHl~dY3bKT`BOuy{ii|{j zZ_@>}IbA`TF?G`hnCefHIOOdH)=QX=js)G@96!|mZAlWoiSh-H#)hAY;$^YnV+5;{ zjfRQxmoGS_rPpxWuW~a-a^79T;WYjdiZ9tjPSFJI^Xd_^TPz}h_vF*Q>@Ag)&93F6 zJ`y-%9KUvXc@0N0#`oLp4dsqcdrO}rs2Lf5Uz`~Sxb68Pgn`A7sdD!|O?w!g$4I5( z!JjX+~p#T>Hv@rd91D_iT+#iEE`y|UGAxh-Q=O5> zJ0l-^bB&lPg%5!xmUz_{pyqet4G9vgH5U%zBQ@ZF@1KI>UW7r)Ito`;3dKANMYx0 ziLnt&oq2+>qq{qa0LqDk?>Lm2-fk=2CqJw=0PY@e)(<38(XBwij_idFl5=Ps_UY3r zV0uEonWUs7P`KLwxe?k_i#F-9jl%RjsGF$2!BTD_bx2AwUJ!G+a@ALUE#*(@|E_3v zz+o6{Jf{<|gu(`=ElH+Cw&03*dn=iow4h+StnZpln-_BgR4rG0r*S&$J-(5S* zumGhZE|||4!J;$9@r_Iqr{sFMHTuEoV~Xp_d0OWV>yv)9u7&!ND1&z%QKR+E1u?12 z>>F_yd)Gf_s;fIO8=!R*gXyyC6YqW~il^MZo|bmJl+ao2k2>u!TEuFs@OVJOaH_gt zyliV-RPxi;uhGQ;Mn+HF9)cg3P>YXwPV_xE$aC4y!QqFlsDqu;G#u6F&T;tKhlGHPX^CS0+5dTBiR6XhB1im2f-pBJ9>{@$?W z*6lZ)ao=wDR#zZ?OAD?zwfm64Q(d`gWGfvXD4$?#jJ#!{f>CW=XxDJ$(+ufL-RqXe z6pKGFjFl2-Xe3$cST24`gWfa0+C&jTd)(pky0SHuh~Gc;qJ2|SS}C-fq*wapnr=y^ zV?5@y!}nMoW*NV>vXZ3z66H%K+r^Rlw$s3QH zj)t5!guVz~DYyzLKk4VjH=DkfmiBAp?fG6H4o?Noe6m#BjYp@41pR3@pATfEUPP%2 zh>7DJtlfGJImG(9VcH~Y zNhPbb6-PiI$yiagB#TyEWn|osh*MbNTUrMx5Bb)*)1ibsD~aaWR87`(oMOAV$qEVy z$xlH+mqYB z%X9MTBx0wFST@xc@!OXGRiO&!{eqaMJ04kQ0t-SmH(IJ^X4j8=@0UOE?#_MSKPvMP z@S<%i2tV5Bov zNc2lW!X#M=l+?Z(r~iiw@MY)kfn14jBRb#xI_ioi1_2(GjA5towTVCMK1&5G9?u8O zMObz}Jc%dao9|hNTCWD&GM7?(o7I_}qjQRy{@&7k;SKT*h3MX7)JM@AbU$|P?`zg0 z;(d{Uw*qAxe@r^zadWeyWB>^%Gb7`TMXt&Fc)qat4j=QulJd-raxsvou>eu6vm`9;YyOPs4ass%-O zqfC*>Z&RL2?``+Q_HtAwA@sfK93G2><9j@{czz%gkY+v~opj~C^M3H9N<>C0i{HJ> z6Ft*$CBz)?m|+3nA9z;Ij@yygBz*_N@?=;t-|C?2!^aHu(JaOHs{RZm*R=L+otr@I{W#hb3Bp7`h&6JwjH5qFEs5Y z%wi)iP`@hhh2Aw|U~$M9oo|a|LyeCwx4Fq9ikq!rOustt;tj^NYd<;?$|tLOOBmKS zaEW}`^mbLMgpZVqZ5f;ny}hmry#!F=l&!4|&@AXKP_(x>*?9m(UeOnml^8bx6?nkK zIR3sb8aKf`U}Kzy1u zIhjJ}rM5}-ig87MKG{&-t8}U3J!#I@J#jDh77GgM5&>`ud=l&v()xcyibvMlpE;DX6h;Fb^RU(Ydafd88Rr3AJfVT)nUn$I}uX?1C5ieTRYfuS<{T5rDJRn;)44d%xlJ*VCv$f&2-_LOl=z_mEfl%=W9_G+7~z@9Z%%q z7oD1KJC5hHebxN6P0QdQWxra=(ru6C^&O4CpGp!KJ&CU16mwo7$s|qM;INO)K$monoUuf^s2@0a%4C~#I6&Lw@ zy!~=^ow4Wd@**`*%FgCb)|GG+WF)5F)~a=+TD@lF6xnDDST=MNJ9ItUmxF_oU&nzJ z*27yYFl`hmCa<;jR4nDS)}1>^5SV?5V!xFnC#Qvgmw4TQoV?v$Tl?3Pz!6II5UEaX z%79_TWt+2s>E;mr^0+c0Nj|-Lh+!aN&E5FE(DB)=kfu~|@+UG61sLy;-D?RjI*BzMB?n!TE>0-zE{1VyYD6@BY~1uoY-EfdjWmb4 zQi~5;We&go9FZUB>j2JqxamyW9sg&^b`zxdz3HhwMJQ#Wemq? zE+?r7;iFBbO!ar*j+ECUG$Av&hBGw8o2&ccB-H_JoB9)vO9J#Mye&MKRU!TSJ4}oEp-xwI40}2jKi!)I4M~qb6PnVj#INBTVFc~f|Bax7_?P}7}_T;ht zQ^VIB(6p40tdRK(Qu`@>Ms-XI3Pu&ECODt)SQj{R16u8KM~{zq0)S`2f#@1XFIF9d zR`WO0eoWeLr2bgM76U@|;6#003| zY$K;S(<21zaQEDt9GRpW;ViFXxT2-r{k-w`afT8WsG(W3O#IqHSS=o%U+lEe6#VS7 zDXDR`JmW&VeOupmccH_iCqL1!9=HK4U_K7l8xA)faJ&&0@_81`4zlteZmuq8OyB%_ zQ*YUAHJ4kytdwjr9 z?#?kQso>BGzO!?A`AoxgiG%MRlUk);l5 z4x`~NyOetkvHJ}}=e@q)eis`d3Jh^OyNZ~G>I9YYn-@ATFd(lQy(cdEVmR-Ow7bDl zjkAUAU?z&b{%VIphKFqO6QjGIu^LmwpKx_w9?UoNr$_SLN8Do$Rc3V;xI_B;>JnX0 zki$47LTu1!_eLCFvVrus2eKAn^tbLbSK1=6;P|`2%n$1EKzMCrzpHB|Wd9ZV9Wd@H zILqKj#ozE~^szj)2$0q+z0Iy)-^X@@b$n)iesCWl{A#vp%Z7h#Llsf(N}R3S*u!#E za@}F~#@-@|*iNvLri&*51FVzVC1H`lQl=fkda)X8_ROCQdWL z`a-WLlj@uI0We4VEielkS5Q=${To>5E(m1JP#l3r2t4Z6vXEdc<19VIuj3tu3l%p@ ztZ&~6EFJ^A%x*BZmS5y5{9gYv?DairM75(=nK^`O;r1_E(7rwVda!1>K{S-tTpJYL3mu(nEm--LYe{lY6OtIFpv-~mcXnVA@v zeYStR^@sBtJ8rQ><(Uk)Uh>#&3U5t~l(UisTUfSDkda~A+dp#;n0ABl#k}hDfhn)>QXQD9 zJ`o%kcMQPoO|@uJ?2=_X6TQoL9-r z!qRmY8>7BJ`>8808LxM$=tIHFj5W+X6qJ!VvKTJdM1QIs0oUTtf~FuuVC}+LTG~P) zxtPX_abvi3R{HD@9UP$|z_b+>dgBGgw}&TE`*-S_(b)B8#?4usI4W&=&s6pa2_LZC z5UTINj^WbY(m2217C4uw3#0e*3y}^o{DT0XW15Uc`rgDJulIhC0wOoNwt1j{D`pW& zOTcVB_zy#@`${8OhqN#PYdmb1sg0aS{rN-M&ZY!MI#k> z3GVS{V<_trU-4+PKKM`7SYFlU8MyAC4&{2)30}LtM>M=K;u%6CTWY!(>7IL!S=bBh z?av$6(dsAFB|T>wXXm{Oo=nuST=fEISZ?t`pkE-{ytbh|tlR>n$UtXRK|Lm#Z*6FH zS5G15ONF&;DyE2k@KVh8yO2T}^>5FiMsqcJUbnDsS?S;P#7Ve*s@ zDZcf8H~Xd7!s1WxjAXvsIc{m`uHsGW_EuUl-d8c4A%QPiQN=r|n+qdY{zB8FatGY| zEQ`nc*U`U+rs%!kBk79tT5Fq1#e=KHvRp++rxaE2_7olmfa(A(-F-6oVzvih9LK?a2_Y)T9 z$TbCW3-HZ8?odmD7uKllTv!2uhV;JMl>pe(BOa4@+#g#d%VExS1WYRfH^JP`ypV|Y z*6Y}!+x}(p)u9nzkzS0M5b%A-c4V>X0cDJl>5tm zI?+shQbVyjHC21$2_o_-?m~ybrui8DWR=c1*ZDpkmpN46Ld#4ek&h%S{A#jLXQb$L zyn)7jxd104rqlI_>aR0e51)EKr-ZFQX7wjF8HWsH{D6jQ!vuC2h~2mMfFtm%ekeq* z1FWf=U)90)$wljW@j7W}wD1vNGSTHWs)B*Zmm{#jwpYM1IDX@Cd_QwEWqWvgi`-WH@hnE_%JQ(n&!`n{qxiZHCYH!0_g0eKczv^Q$uF+i z5tP|Sj~uyLalQ6>gy2NTyKk-RBWDIsL}YPjtV6l< z>DT66FF)|4;h~|4{Xev<5{v4qESW8BVr$Z~ckeNyM+XN5$psppH$YQo8aosbQrfe3 z@6Xqcj$b@fA|?y%74?0c47==)?}NrurTFLS6=wWSFr8fg!GrUE_w9@huM(+fyCJPp znYA{lUK?G|i)csOOy3c)6*Y}Lu!w0du)3c)c0zPQBqDRux8M~kl7XAhcSQ(yUXy~y{~n>TOJwe`w#<1?N`CsKAC8Z?->ui7EdPhRKuedIxpSTbJIcF~H;^adUU zZOPc(3Fpr3A3kkUbxsgxwIYrq@RGDhsFydwZAy6SX;a)LXcMk)y#c(%KMiV1E%zWV z>6P0d2AveqN}Dp~8d)NzDDF4)b!*N))!vHkN7D9cT1V>#5-!W@rDX zfPqllQFHM%Zg;I( z1?Uh231<~&qejS6vOxO2goQmEkT%Su&(E#YBV2NWGuD4tl2U-3(KTx^nJfmIu%*-hW22OKujLN(~JS z65W&UDe+!Tq3#*+j?>V8b1rvvKtS~ObyR~nzQYJEoV2^9pOj!>3XK}{1c?2%CSBOo zmyXRc%||Q=u?c_Yyx^73ucDN?N!$vVdI7YQy-rJ2^&OIv9OxJj#w$eOj14X`TQtwA zjATAB1`gL#JQF&M(VAK&?>TxXKOTUtR6iO!M-4=lgz=P0ZAl_~f=B%KOHNNe#sZTEF`by?UQLHO^) z^SP!sIdzBqE;()78!%PD-J;vKjc=;~)IJ=+iNT~Bm9tFuFH!lx_R=fEoG-a;4k&%} zXim@%gsbAeF~4mmjbg3IXDx!8ke_d2q<6fZtDD=Yw%^rV^S(053-;lR(IlFsektga zXPvR22HJi%KYyUxhTVhPZw#;=r#9>|YPPwnoo1Mpv!u{;n+Q+ulx7Cf%`c~^;4W& z`t+&XbnxbWcT5V7`?6xx&oRdYg(1r`nhMg`3x(|=V} zmYlqN?Ddp{xVzoXJ_EKV9(cc3`{?Y5i#z>ZAqOTqL*xZHrR||Lt?*Cn?T4;HHHqI# zSB^<3h8FT!oqG=9uTwh_!g_b6!glGR>zkY_?~O|c;S6BBbiv(uDpjH{kE|{p3Fl^J z$==A$%oH=Xw3L#O0dKe3&(G$KfpQ_diz0MBHNNYss+xZ|t~-U*5~as0J9~9HrXs%O z8Y=mRr{WsP^B z|CWI~8PswQ<42npPfkn=9p~$nCx*JL zlCG|jXV|6iG9X$K`j(3^O!9BN?gR zI*yqPCM1crEb(=ePkCEg`viN5XQ{p(I!#s83t?YuFkwv>juQ>qV^y_IaUoE#m3;A^ zOYQDdRyrn(x87tO5<17y5IY+({e~-=}L3s`O zL%1I~7t8xf2?+_B@Ks>%+`oTc1Zo-=GQYsOS~RO>WldglW2462lj{NkY?EWvtY3(L z6J^Jjc{vS-Q8NQc|*fao~%i6`g7owcWUR#HD{V+F81-sB02%(%gc3{6nuF7 zh7DRSIrfz&#xLIT{(+WkT_h*E)0Ov1TU$;@-S2L!TJ!Xxt811%%t`geO3S>wyiJ=n ziAqXJqNmK=6|e@aJ8oEDE=E-I4h7|;x*j=V2fydvxx-G)9Q?ISn+rn9%r+;Fl)e0S z^NY++9Re(&qh`VO&Gz>8q*x}wK3|a4az*h{Nx?P0)$Z_4zuTB}k7#TzA7#TnWd7KeVSpA+aY&>>%UkM^j-SX)0*Mf8~Tk2xgN(YILG^#-JoQMHwRV5=)7p$*UFRn#?R&cAmvzJey z%ug(FC3)}!ZEcW?Thce8JGk-B%sxU&-4$oQt(`R+UfExLrbV=>^5@@8&CHhBjn>w- zsZ>QhUQBoQJ6AHuRIAfH!nSSesrE`G(vvMy=dI6dPxs4)Aj#}x0W+OXX2hd4nSN72 zsqQcu{h_Tldo$1_W9~kG-i>jA)byxIT*>rd(W>o1H}MZ`rfI>xHa0e|u5OEvLGg4i z+|D-dK!5##56UU+<-T;B*``&kFw1Pl&Q*C|JpVfNJNkO;PGrK)=ea8bSmUFyT zgl<+L!o$}zPcKOzuNLOSKueXR17$8ftT6vumJcevzJAq6MIwbJLdAmX?)CVClU1et zb_H~ZuZH=TApv$!W$oin@68;;w{I5?v)bHwO>9+tZOs`%qo5p5d~JH`_?AsSAKUK? zhzORcr>CdP@Tm_|_o#$?Y-<~PK2VmU6M7|gnD?cFIEUR+M4pusFrvRlOmy^80$(4W zzu&(fw{g}2C%nVZuw%y*qQ*z{LfU+H^e0}%~Spfy-iF_?=QV900|tf+$k$t_(^St%y2#IqOLBOkx-^$O+Vd>6X(TJypMa~ zRe`FLqa)vpspGDgkLfeLaIZQYzxA)XNYHLk5hIL;?#{_DsO{~43&U(q42k;+5VNkX zF6PjoL2%1n-Vn?15FlE0sh>W4P%AHsRF<7#KBOL;Ch$?h%pm8-2k$Q>C-2c;Q88EG zr$|6&((n~TdVk1Un*POaO)%RlMD|$wwUZJEa=923QXT7fN2>|`^|QUV>fgwrO=Xfv zKeEO`Z6>zpI*uyl47mr+OeAu5uVpF^n$dm!w~yI)@S zLVtNP(<;Kj1Zh`S*9Pmz%jl0jIy$j;801?5R;jEtOAQ1uea*z?w{ORI7?noC*=ps&=bck8(EZ6GUbLNcUoMx?BSy>sOd5;4}j=Za_J)`}=NK@07 zTcG67qm{GL4cc+rE{}Mya_8VX!vd~~M1vmAGWp$BHqsNRI_u;eZ*V(I4@o(7L*&gV z?2rke^C|YSrDfWx>YQ6EM#*WyimtA%7NJ>sjbr;w5mnbiYaTpkBrw)KN~ph>V}NN_ zuHg17QOu9;+sf^9F}cc$^#@fVjA%&F*Kt5aHO8upl!tGrZ8Fa3e#*_#O2k0(-ct zh&PMC*(n%Z2B`Jqq!=(gT2*q zx0`ld7Lx2mCzmf-a+xn?YO61d5qIa`ebJF)ZQ?-mw?*in@cBdC-uTC^xRB}FRR)K;h9?p*}%CBisexG(7% zbsPl7$B!HL7-tSG_x2H4m6W;InaIe+9v)BeLe7PFI)ro$04gBT&J#`0)qT27D$$EL zDo1C)oWUG5x;JjrTk>jwE)i$4izIKEzUnS>s=`Ia#>Q*ME`s?MWAFTm6rMoJlJ{tr zD(t`eba8Wm;{Pa)DYyN*K=eN+j(9%lf zr;eR?!He}X`IiVda%p81m69#OkVW1+ zGI7~HB2lmzQrXHaN3=Z(L&uM$EUO+oTQXaWS@*-5Wm=kG$2mW_>Fy)3v0z98)~q>- z8tCQ)7Z!kaqWFwsc0RC5LRQv#g0N#`r@0YD4=N-B12y{HCfT7v{++0DaG)_2&j{^b zPV*IUFOt|-oR&5r453nN|M^SDj7>~zqkneTX5ZPBm4{+ieQeoC&c zMHH+nIN-`Ar%O+0`L&%GAZwi_B<4F+Yp_h7H^RiIUZ=wVxO8dt+O@t=NjMblnirDz zDw#Stz@&I0B0LtEEf!Z1fGXv#ff5vO5(d8Cym_-|nBjV*Z@_&o7AA1i32A%>P&)!p z%<=)tx|^M>6_-a*gIJzR8R4*>XD2~>#>y&k(=G_;4~vtCoH*0=RBWi>`qf0MStqZ- zP)ZcTm@RNRh5OV#n#`{#oPwuTKfV@h1Hp;_=z07&6}~rClO6(*JmBX@i##fx=>KrA z_i|dAnsZhFIT@O?l;L%C)TU0|L`8!t7|+5JF;wI)dH!$rZJ&?hCvqw$@`k|WWN%OC z`3<;FMVkAL%DTQE8;!=O#4WMt&kQ(7-OD^w1qt&D-41>G_9bz9@cv>-X66@K3{OuG zvzZJSpsVKn@X;gW)Yz9GWDBNQ!5+VyXK!Y97xFpbHi&W$BV70s>J4>LDEW%7UbBY#2HyH8I3h8z zA=%+d>D5*yG`7~R`$TqjHXwsHwHxLllshtz8U4i#&H8Zp`G^oaKY`PxD*P1( zPeBpUtoIdH-cpLfX+#&ra?gC7=r;ugeV@|v6f$k}yNSGUAEZ@2gkOz~%iZ1miRwfG zoS4u3@y8#OmSj$`*di>^!rwE@$g#4$wY9aqJ#h8vYm_$dWu*S7d%d7EORW=HwukRm za=$zT`Ts$WB{NS2$I|d%6_c1)1+^DBMMWbNddVB>+axnv;r$j+Tth7wcS{vSh#`38 zl$1ZAK}-(LLm6soUq&Gr?0Q;CN^|iUGb)D?FOjJMfb%eR6|CemDo?1_jIxO4{!+Oa z3XnV>&<~~=A&xt>*|%@cg;=<`x$(rXqQq<|+Jg)t4Uv?VHVXRq2FDD^1RLYWky}!_ zWxgTRHU5~9eUjaMd&tTbhgyycehIKQ;k!^^Q12KorCjpl2_O&+XY!4w>dTHEJ0^Z9 zBLltfu#cxPIlm1VGHdXj{rl~j?Aqj#bYAi{h1Lum76AkiqB?S9Ji)h(_}O_UQ$x&) z6(}qOk-<|&P;|X`uS8sH8=C7WR;Rb53z?OA-REL)8KI0=*isZOH8{Rwjy7B*l!*(9 zDYmLVw#hrymTK`gO@h`}#s#MlQiJ|Bd$KKl+xu+p$PpuqBeM4H*@G;6RBY@8gMq+Q zu7`#;BvS=DckZ0VFLiY+DV5hlg+ABkyw2BBM`uDzKVb2l0A)rW5zRZ!QP}0}jEpQs zLMUGKY>3y_&JQ+kn}2-oxK4I{-d1yMJ8I4>xEagmpPPLY_`r?y;KsroKK< z2w7`4wl`IPnR1xNY{hasP|)>>&rp$G;p=OcY_IGqBd?>%(F85KC*e_1PuP`0%pK#C!kpFgV?0*RLhp{kqcOdK2V{Q|0vJC0;p!BEBo=N-o2VUAF$sRGWTu*ZdHb zS5%Oij$JNB-K$~E^pKDooyV@Glw{DOznD458MZ_s=2Qe0Ne^6Ej;qxb1~nb)sXm4B zVfymwZR`s0?k-}hdJ`PFzCuPy0d84t^z6~c zK%WSugjdND|FV(c{=ycA(&fmD4{Y%`$C@hHPGx|rQkzI_eowzb(uy%_D=Aq!WFEFZ zjeUrXI;rs@CukxEN2jw;F84}D5j5-J$!WuTzK@tb$i*dM}ChLn`}2WbP#V)xFSR!$#<^5#q-7%iV` zIyiUxUFis89_>M}?WhbFmuO9=z*o1z`PyE8@i}(jd+j^==~j8U{cY=vhAth~Ik&He zCnYsf>W?~jP>6rimK4w~ve5XXR?@QkRAA+dsd;bl>mfhey@kCA1%^#nVp8_yZ&C{^ z+2h!lZ`5o{wyS(_N+mY8RH#TL?S>>2qgY6*4qBA5Ez8E#~0_+s{xId<>TRGKb zQ7E9!GxnvL8U^{Cq{Xa&31$<&Cd}@w{;}hygZmldSi68Ys=qBW4WdF1E4ocG)4u-o z-~kFX{JRe=S{Qo{;_LYx^iB-RQjc6c?!0PR7`JILqCIAQo|;#)@#93uV3<}7TrPW= zdDc~Ao$phlaK6{2pD3xTt7~e`nKbzLW$wdL0e@^2TqN`;y-P+uvRk;}QAnt;JPdwD z^3itC6{6xgs8-8dRT| zAS`fmN7EKoerEQ*?RL>E7*QG|@BaS#Lc5ojm#S#MfC1k5+we^~amte>WioOSW#-u7 zdZAyiQ*|Dl$as-JNhefzET5aCm{AQo*t?b#&PPi@Ufz*hpB|)^{iW&t;w-Zfk2dFIWs`<;R(fTR62CR- z7w8OM_OKKQ?>75kIff`++I!L(X<7yG@oV6g?;fOC=%_?h&+$_-HU^!`*o%i;IgTbY zr}{P1{Itt%%SudTgI6 z7Bc@wV>p)TWOR25zlABjFmXboc0F!3#?xaS@yv(U1%*3k9X-P;tBq5f=(&fM_Bc50 zU`l3vx;@{<27SUq8Plm9#qVYEQr}V=PC5D;)e)&vk%gqRLX-I3-!e+d%BS)D%p-lF zP`(LnUW73KSYskak$_~Ox^yNw+T~zu?BtNn)!xOO?B}rD_?a*$=7j+7T2 z)7Kg|E|nvq^ea*@&$J2%MyQ2!zbh6RA6Ra(cnY30C3N{KYS8#l0YglOgCa9rB3uUj zmRqc=($Y3=lSX`GYEEFYBaySIY01GAs;Nl7$H&YY?3Jv?MQMLJlw!cn_~II3&( zA6`^cRM4=gu6FV8`-~yMv+9O)w(^Sb{seU$C40#Jbdqk^`vJC zGP)X>U_>Y^rg&&Oz;a+lsO~jT{f!zblg36j$BrF~iHS)`@uehqXaDfwzy4a1 zvD_|W+1eI1w2xiE0w!L^JEwEFm68%&lCPZa7oxV3Pd&&{zp4z7I5uoMS*z+4QkFh4 zGO)unr~%P!XXoRz9sgZn~v7{||rfqn6;fFu!}hmGIeFrgKeFXSi+oFB}hrfdBvi diff --git a/docs/willy_wallaby_from_wasatch_files/figure-html/unnamed-chunk-207-1.png b/docs/willy_wallaby_from_wasatch_files/figure-html/unnamed-chunk-207-1.png new file mode 100644 index 0000000000000000000000000000000000000000..0989355fe03122635f763b0ad4913fc0a9b90772 GIT binary patch literal 107271 zcmd?RXEa=K`!+fv2_jmwB!XxWJ$iH@dJw$~Cegb@XS5_p5JVf%hUmTbGE($T^xivz z!C;s$>kP?6rmx$(@IHKp+r_it^ybSb;DzX&vXL7IMF08v7cYA4&u%09MfZ>2Z zvTkv7rvHvWgo;4~*GCz*$QSXh4{Y&ndSAbSj6rw|*RMe|vj5+nk{`*N(a<+Ih(RoD za*D9Ub$`z_;!JpGsyq{GB^K9B$i&^-v*sGfo0m46R5vp`;5g$clH%kpw)b_Qd2X}s zN4KRwf@+Q-w@Bl++M1++*_oLpDFr-R=C42Tug5!dgYe+`6;utnA^GosCkRA#eIQHx z`2RX$8QD5PD{I(ve@cPc5T9_xDDMirf%YR$Ct(&B0)0*-u&%mz$_R_I# z3|;s!`S_s$ZO~~Ol`n8{2E%%=$m!Ve`z=$#rHGLaT`_!KVq!TjJn(G6|IFmgA3vpJ ztmlYVws2V^o6kkm%PuxNTj+m3yECY;>sNyuS$kh1(C@1U0*C$E2uDMoSHzAwg+E9@ zqf!5P=erImmU$0$R!Q=P>kxkJJC5Ir&wn{F|8fFlc>ecxnG&fycKDzgmVdiQ2YX(_ z%rSO*a;%<~u13CyK$iVK+e??5)4ZPu8*kwbn>V0gP2e~N)!t(V1NQ`-beWjI{Opzx z-;wm_`bURe&4u449fb3FK~53>1~OWJiPc`NR>j)*`lz08#+rL~l&6UoW!^VCV7whf z!u;=>PyUWLuUDF96@i8*4DtVKT5oPLfW)I#Fc702e=833<{+G-VgB{YCT1GU*!+Ig zv2B(bV4go40plDpqM=7ieeOhZB}7XLyq$v7RN~e_Cj!{;WQu zH-U|RK(UCI@u^w-LD{@)Vm8;kig)3_c|SveD3S5$($Bz`RojUiE8j9lloWj%2olz9 zbwZ%c$D!Q*@7?Rcdis!q5c1`1%jGQpmL-E*zzN3JS5~E#C?UmtCcsVF$*lDL?;d9c zxRxU$vgElKfiIQSd7H-!+I;>e`LrSCS1ZE<$xH<`_)OVTeEpC`QoyrUuSfl3xzf)_ zoFlm0Ny0(zp(*e_=*IQEDXC47jMg*^&jqwxua>^}`uEj~B-pxZf#1w~Ap59K!L{V( znlGX1oWohE<%=>e#ZW^(+M9L@e3rO4_To*G4D19_F7ZJ@;@2}@@iX^bN3PSD;wD){ z?+h1lLgs5wCM?tSjUw%*SDO8OSAS;B+htD4A~cj+r&G>tA)03n7s2`J}xq zm#eCXx&AhhXD(yX)6#H1C$qhNw|ar~QfDKhO>upFk=GyocBk3*+ppV~IWqX657^Z7 ze~dwX?I^Lb;iOa?;XSc*_-tp26JpvkpUhS4b&@i5nV4Alb{rKaAINXu=XzBn+DF4c zRt3cOA%=gJfOWUm*QJkuSEhaLEvfr4C&R<7=W-B@idfQR4u02sM5%+zQ^j9EGo+p6 z4Q&IK;PeUb7y4BjE3WeXL|Xjc@2WU_=CfW1;?kdS`AiheTNm|FF?ck)o`|e8Fn-p# z3z#4~;KrMea~U6*=RIN%R;?Xsc3Td|3imba*CsLtlV@7?->T)90URC#(r5aKFH11I zy4XJ{XdglUI#%2Mf?I83+!2*&Ul=N~Mb3FWgwX3DSk}=Pp{FCVCJoq^m#C`rM#5dS zUlxu3rp-!*5$(vH&7T=nQoWx%=>+S-Vu_Bs@*=2hS4fZA7xH8o?gy)Ec=^eAn))4KBfjYRqj^^dDUbLE=xXr;; znb=xju+jBO((%;lP=MaV=^D*8+8lT0j6eo>CBYTkn(jqM)cjW#`2IuWyHYxW-N^6g#V0G=#P=0$WEFC+5YHbZ%I{ievAUtQMl0SE`mA7}M$({>D z{-B4~6Q*q8rfJ&Be)%MhLT<7)t>2o?ba=d;o1v-rMcCEL%Iw`epOtyIsI|a@zd~v` z+F4vc;4Uc%Apl~n`tQTO@V`&$K*>oCstCGmW6&GwPAd4^CNn7G6aNh|P=4InIwJ#a zR!~rs7ltNcVperEKo{{&pD!V9+GIQB9M#tzf75#S*AILz_RDE~_uu|wpWN(XuS`&R zJE!3Gvb^Hf>$W0gx%&=-^ZBtKXHLJ~ zx4Brp;u^u+8E+Q%ap+-hwMa=IRkUtRH>}DN^RZf{<5+lWL;wo@3|Q}Ed1oHtyD(X? z*BC8Uia?sdTFR;0tNpTNOk`CsZ)_PgJhc77CbcO44Qm4NH$;FL+j{N!%x+3!O z;pz80U~Ru&uPuq%PS!7{%qF?~>`1$<#hYch^=coTvY&jCxSvK2`h5*794A$hii$*) zUw-zbvr!tEpqg}kRq z7i`S;o~RTq_-ol+^~&wI#@Xn~6%(c@-IGczT_6hi=E3K0CRXhV0{vSAg|TA~8*ECc zXNk|r{GDNs(-Eyy`XTS`BikS}wb+no8A z2=B2Zp3)}}s66+>Y6wyiSDO9^u>?eKdSnO^ebtTPoj(V&qni;GH8o@^$F00BUyaqs z+KbO=Pw^930*;PV*GFQ_N6Y)^%k88Ls7{!mhuH68S-fR)-*jc(hbJ^t=jUt&gUC4n z@6XUs@n5aU!|v%pWvX=Nkr(=^jKgEEvLfu|3j{fVA7>TMJC)fN#)(|~fuu{}$ckks z_h3B9@Y+Q!J;yT)KIq}cYlERuh&pc5baAKN(+u-b{W`266k7~E{F<*TnWyYa5Yz!c zAd*E`&hiIaD}9R6Y}UCmUWw@qp~ifkH~m~WpC}~q2)m;#XV1D}57}KZ*o=r(w|>&N zulIa4mdsw`?v7)2O{A46$h^T0^_SdzlTt+hV2h*)hDDt>iK z;$Dt?3@zj95M^a!Nz{lo`j(aFCJ-Mz79E7i3(uj;ZeE=Rxo9iktctQcsVcTWHh*w( zJ%{OY=>tM0KLcaG$0BYvl>q#tTbCGFRo}W4WcReLI1AE(+MK@f*>%{W-nwevsK_~Q zT0x=P>avZ21ZQU~zyspxN!6tdv<$M+$Y$T$WrcZtcWcntA>v)CyEu;uj|kR> z*>u7~>di~4l)OTF1A~cB>=y$fXUc?obbjNvdpADLoYlsMdv)X0@r}#ckSfuqHSnyV zemVImsdhBR7os*>4i`7x_X?Dqdu9lJ4)R)mHF{<(bkVIl8!FBLs71tRt%kTBoV#)noiu!Hnuz4X{K4wg;6C|yC6QlNHFj3 z)*i;n~AP)a%6%_SH{O;j$BQYLHDXom2G(?|>Os*9;%}BQ z(t>Y(s1=3|a4TK4al9C$Jaz_f%sXblgBUE@O7B_KWKTd9)CMm5Otiy~izTMNPdeNR zeUPEz4$%%jEnP^`tc}eZA5DUmJuOsmX(`&OQnv{HApBpW(`_l^c88e6YCPYUWvu?Z z0MlFE30QHlb{%EJYEDh^I-J~6$sXj#lyyhI79QS9lZ!f&cQI-egn%7v^q1$mW-2ch zx8r#^GBL~ANG@Wq7B>jM*~K3STV%YX`@Z|vU;5K>jE2$Do15U_q{y_|ioXMvnYnfe zs!nHMfJj}ZkDvi7+F%=Etv$!!gERO$wMrzB@P1l--_1SyUmmE^i7ZZ{BrAU)`~#s* zw)2qN?(hW*E`FaXm|QNl#x}4K^V{`um2y1b^Y4M~2y|ySCbM^gLN%Om#>xw#J%ULZ z8m{GDz|GG7ZsoBiGp;3iBgA9V}L$ z9-yO^Zn5ue#k8tU(7Q4{TEbE1y6bXcT2L>eEq;H?-|B-P;mpX#6~xNWWY(I6j0xmR zXMc~&S55!C*Ow5MPsuOC2|#a1UK!vAkuF2cm)BQ&juF}b(Ac72>GPeA;$gU_6U#I{ZbD3l{*x$LMThkCQDWQ_X?N^8A}@m=0IW zygdeL!^@)*&S)$(N1Kqac@db#>dVU&qJQ`IdmqWu!ou@Qe5pBq`*gIMm1QqEv8deD#a%Y(&+P{ewUXzsnh$pt!B zDFChC*?$KZz_9lGfEO>1wC!s8NZLp4;BS@`dv#5GoSo>KuWT1cJIbYwO558p=I#Dk z6Y;_1vCEyFQ%P#7!vUoDE}CFMCh<>is1P*g0r4c!HSfGKiQLoJe*3>1{B z09bkyi_#^$S5>0)SzOFl8)AwYRE{J5&Ty>Rg2NrG-mN-T4E*&9oXT&S-WdEcHo!t@ zUDVT(#zYk9`#?d3v+vGL(Bo^7PCOAtF8AeO)Een0R$}2_0Uq@1Sx+B?(20ORqro7u zjPA2cq1*&A1scv+K4#Ur^yfL5Pr5z8eLTGl8z=1wOH*>{PmiLMFIm+b{q~Q(TP?)) z;`*w&7wCqo4cN>Lo-C83-f-$K2-#pcjb#oVYMgB|E4s+_2bKH4*{I#l_KA!C*1Z9RW;!Dcg#FS#K2ydk|0O#9L7RZ;mbth+3I4 zlJCH@;!M_Yp8(HBDNB(U-M$`h9}+VU0btRJXl_Y}ap%{7K-cv$8(Mt;fsny^T*Tjb zkC1xI&Um%A^%v-E`5OUW-`3BuU}Yl&0`uYGrTvv$t$ZQY8*5=H+UqYT>jdzsrEW+D zS@Q>a`T6J^3tKV%?OpMsoaT0TVbm;BJFhnpZ;dS6QtXb7SPL{YG)8U*@A)~}*TOHh z6a4$~j})d-M=PU?{cRtBM*Dess}`b(Azx~8&(2DP{B3^Q+`1pc16a)vm5C0gWNXL4 zmTE(1_nVZ2KA3?RK%}O2e{gLXJ3e?18?hQvH$d?XO|SkkZ7e$778?Jes048 zXFHcGuKVz%B|pBi{^af(E`a6-A@4z2idg`jt>283*C~=#XMWY{T z&dP#HEvoADSsm7(*ICBS#I8H!KgXA6?gr&%-2e4fv-Af^VO-X)vbQ`k5&I8G_XnOb z`L~T8Mcy7)DW^ zEy=91L8n+^^2XZ&jk;h_q5vO=>vVfFdNLH@oOYba& zKWPk@c7G;1Ja)Xs*?kw0UrZM4TkQ=V8F5BZ=bV&s@|PjU+Ku5GAHQRl=%sc{=P1K2 z(I4nfMh(hFldK&RYv#u$^Ap4)nq)w=-!n1(2Grv-?Hc2k*a;d|gaiJhc5?y9mZow) zvY7{Jem73I72RV#*lT|WMi?($HXG>iV%Q-1;nt_qE{RI}%JHfNYTSptvb-U8n03&SqEXNm@IAHC1bg;CRNA}L`ZT(6cma$2(~ju+#en-x_9XI z5!UI##a>2*Rw!muAf(uGU4hygQmX1-@xFq^+sSOS7l@*vFE8iw&xV`y{S{U(g>wMiA>gqU_jB$zm{c(lymT7 zx?R}(ZQAX%Am1sc+tjzBbXi+qR(NbHxs*^eifRtO~{kLo* z)HfnuQS%og0O7u5r7;)VE3j3Ukv=ced-!UNQM~?u@ofU2AZ{xGtX=5Uq3lmklUP=x zf^?zC+R!puQfBelHa@Xe@U?t5c^5&A2YUDi9jrrmasMbJVI|96$8~+_T9F4mjB-%` z1%<|pr%MMGlcejWu&Ah#lz1oL0p%pWHvo5)j}7?;4_-LEHchNxFITU7?@E{e=uGO@ z{Qu#TiiyH`S;kuGPb4FvRt9LjLPuSfsycM!5*2WFk!!8d_IoL@YWPx#m!9^^L96y> zJ9cV~(;kPLF8X%mc^k<~$Gvhz0F@k#ygv7S`@!e=R2_2%OK+7EOxeH=KHf}T77cCm z<@(eoBp5aFdV6Fd?G(*4aXtGikw!p#R*Egh zEn27xW$iZDyXuV-D1%xqqgK>pJpI*TSLqL{1|ND4CgCGt4%2G@C=z1$XDH!ICnj3E z4VjxWu}EUE-{ezw7_;_%JJ(k+ERm&zK$YX}_37NK{bfrgVvgag-Hp>@ zo1DOCqyVjDI&KXk%Col53wuH@`b2Hx;l)goc2@HEX5-Kc?m;ZN!bP(6%ybY=rOld~ ze%kXRy(u5F8z~*V10EdkTj?5!4nNI)(&@=M0L*KT!+*ka)Fk2)HAd6MI?$LiQBnrD z<%5gP3yi1I`PvQ8n;8|v;H!3v^^Fbi)h(dNmTL!7&G`?t8?Y3RBkZkoZ7|8|wX*?y zkSxBBEK2=R#cl#wVeIkJd>``Ur04F@9VUQ+&eT>-IRfX2lJQChU<7euR!AA_+k$Vt z7-Z^+LF2uWL!T#ZWb@c#f};c$F2;Dawcjyht&S5&HSNiiVeEX{aVs}($whvAu(!*T zt3-KxCR#VDG%=y1SGn2+R1uHJDojQGCPAUly1b_#^8iOL)obK^?cQE^YM@;~_-ja7{t6a@P%=d#b&a<&jNNiO&=c0qtY z7AZXLpRjLS*VAeB^MKR&+GuGi8PmR#HUL~koCJ?Sw&{Gc&3zb*wml5|4)&fc4mKha zADV!lv-|;@AngmiF`^x>=V$kxqc3*I?n!@xxaRr*S^jI zIN5~T2&9zQRZb*z9Z{f^D3XY29P(iqvs|`5-r@uZ;v&%i*ej9L7J-}v?sR^&a0+rd zI9S%SqImbQ#n^}Wz{mPdw4+K&$u`2K_mI6w7D|VY2Af!Imqs~#%x-ql+TI>bo%+o9+p`(sg zBmPVq^DjEpPUc^M6Z%`)p8^{itq=yl1(9bNw}$k2BOiSD+}6VF9ISOmd@Ls^!bye9 z*3#tBnWTI{;%)dn(Ntffy%497B1$5rw#FGXLZ)DED*+k}eh|q02>`2V1@a4Al-UY} zo>^|q&23$)h^mGB>{sKQBz{bg8WjGALCWa6fFx~^aJ#TLUHAY_GGR~#LGDE{|u1D4FE*=u8O?BBy7b9m1a`vd8SzB*z6IzSfRgO1B)e6Cx39 z4OymTuUETZ24EJ=FSx32Em8)C$FsafzNVb#h*t&{1ULU5YW2MjX)^H3sRe^-JsKIL zOru2_@3>bBYC{K=_vS9=^5K(qQWbw|%gg(}gDzyZcTIEoq zY5U^KO{W-C^9<~c8@vS^(aifG`rSJNx2h^tUaqOAT{c!W;|*3m$XP_r#+?Ke^{|r; zNG3;-!58=X*bS+;URu*Sh}JkH7vQ?RKrMOlA;4?^F3sh}2dCm8vS{0HXwFAVkdzda z>{b<{A3X{jsEm?kR#~nz+{Nua)F59;(Zs6wF9rM=;KgZg#uj&a8c8M6vPe1aX)a^t zStxO5e-B%^be|+LYFX~B!bQyxm}s4g0jK=N_OCOi?717x-Rpa_20uw`x64i}rHB6J zq?YS5f`arrX{ABnH`w{JQKtfne}@l#sqlCG!iJdwjV-*gBBXk|Y;9S}Pq{G9G;KP70Hvi9mnG;5Kc2Apr+~ zx{WwK&=U+%s;l4CN7WlO*f0E`OLRRAzjDas-q6gop7*ewJu-&s0H8$!kYmb?+BdQy z$nHCs9hvg`D~*`xjyH!-8^O_TpF3tfj+BQ{r89x7v*EA%M=X5zRs8$a5G?BcTx&3g zi^~?|K^{;RvFTUKt#aRqL)Nqm%vFvaU!mz4An!E34?G4u-HW)L&U)id*&R@j6ToiD zjZo;4wE4vIQbWITWBk$Ozg5K6V}b&ty2Th(Yul5IhY)UQ@*>x{B4stQGJ2#zWy{E) zmGq*KjMqutuG=`L`ex6=g!WyE-qMjplpC>6Vj7^_I)#A9-sq%eI`-e%&DD~eEJu}2 zchtFt1kuEvZJf8iwto=>a3C6dX)`t9OYN^bb1U9!+fJn1vkf15rhC1J@z|5}{BTOC zp0`1^d_ee1Szi6Lpg3E(+qCtGT?6D@|ftbNXm4H z>eKdUJl!7g_Xg8mK83)e-KPtguL;k%Igf~W3p_my49&6Im!gC?|`RqXo}Gn;)67 z`~eikgMK5i-#5;94q958Fx54pm#xpN{oV_`$)|ZMpty^eSf6OFpx$~PI-go(m)1<4 zcWf582G{Hwc_@SaHOl7ld<9b2hS4}^^&&-ZW0&t^!VTmSvZKj%&osAmB*Rp<&oH(3%Rb? z`>(4PwtFwlL%UJx>^{B9qd9#4r5|ZRaA=hD)HTw zw7CK)M_P0yf$I81RYQ7$dN2C3@JXTN@}4P2TH2r5c-;b98K|z9yKK)!aBjoZrvaml z2LG$bQsT*H1=_;_r{7&|ap41gr8A;Kn~k6ht7jqD=dXS!JOXqS&@-#Kj@Y`+ygTr^qZbOX0Z^rNFN`l(D%;t0{QVRv(9Lgqp{gj5*i*4O z@X>?jZ!1mulYJ6ZuJ6*{&xmG^Y(MBdJ*Q1EXwm5E%clIuam+N`N9DUVgtwA{Lnzuv zBbiZ!6GS<+CG<1&z?$fG637HuzvX#KxuBf>9RQZ+Hkmqg?DC}>rzT7CN>B8!s{lKg zba)c3H|hM_j-3eO{gmhY0HTaCJe44glmb`$<}0}BT2cBMi0cXaphWmX>M{t+SQi4h z_nOhie762j&B1G3i6L7MUp5i|CCTlxWGiM?b6R`LSmn16Z^{Xt1>9Bl*4UJP$lyRf zWA#Ikiq<7StVN^&O&!7;zJWqw^wv^)s<{0+wh1F-;pya`o;JhpCm@f%tx#yqz~vcB z&sof~9lRi#f4b3Ztcm!eZ-c~|TT4_d!Dm_(LT}43raFXgeXz+ZgY?T>YIwL)Mk>|6 z8a1@!AL1JxSov1^((Q#Q{o%MhdDq8TF)2BJK6KzNsOB0o-bpZ0xLNY9tS__g>W>?y~q3B%OyzUGr9LjLpXx5DEL>P0j8fz(jejC;HKapq1W7-+QKm zx_OZVb>y7Daq6_+R)VJyzq2@Dqo%BXKu=!q+w@*FNL&!)NDTzQ+>Ex5f8h7X_{dKS zeqop*=`xls4JGTX-<`N#s}fINS@YIvy z##ZLSB9%JyT6>!r*QzhzT@t6e7jt1-5`YR2hZz(+9=v;!ZsC4$c>vko+MgJefhS9u z(}SnUV{*{$URNrCWlVyX23fHawY!ewt|FLPn?rA3aI8gJa(q&hdq_56%B^7 zbyXTA8H%h1WgJ7(Uy11@lH-^B)_juEAc(2eh3obc^{0;ZtP~=hch^6P{rY?>Xb(rZ;Y4dp*ushc#`9G=}K*|HD=@EuY2(STmxj%t&I1^UQl)`w2f zVrK|;&Z_O@c*~~n^!OU82#XK7EBdT%a$M04>-G9cT35b->ePS!3Pd6Ag!w3gY5^h) zs!apKfHFEQT&%VhqWyAN$gMMh&1m7LfdV_<;MG$zZ)r}mzTMdEya53yFV`)G-xGZS zhED?=?N8sMCVvmVRmaRB;9eeibcSQr=jK~)Ee>D&;o&O-h25bT`@B2P6>;(Np#;kf z!oNrKV@EN{J|-o7wgD~BY)<~c`?ZlwN{p3R5|4&yycqC=1cBxXM?QAG=)jiV0s6YA z1uq4x%G{%fyRA+tB~HEO4rUK9zSDxyfecDL(&pbSVt&-|a`>uQpYic>U+uRY7bd=$ zt-VeiOO@waKd}0j(z{Uei&NdD^RB|qej=yK9n-6c)!z&86hYwF-W9&6>Oro|`MOs; zYY+a^N;}g1sU4@@K70xSnxupFgirQX3y%Zh)8g_FX^(usBJYMeyxK?Ww6E2<1Tm{q zXxN8|4N)WeGq?a3V_K>a@^%L;4U9Kz3%P0E^8O4cj8OhHT? zezY<17{ZqZ9x<(!1EV9pNOy~H9;=d(mS`Gs#hXXBjmUQ-C&FAt&uXiWzYIeat}x#C z`M)`O)`RuMi<9D#g&~2q20EquyZ7a5`RysD@q~!37dmU1Nge0}wwK?{k^hBxqbVb@}s0huipaY<;@rJHjxVd-S zl{D2vKFFy7wJKz!&DLEi9%4rLLyf04(o|i>J?HvvafYOOIe(0SrsUM^N(L307&E}Y zBAGfT?P@kjGeF|k-BYzOFe2$^+#{5dcS<}Y%O0cu$D0^GNBh=Ffb;5K##kUgE zE^3aVco{7~-T4jFikHI&R()Jv;d^Ov&vYZPiH~>eOtPiy=~+}fl(TXBy$VfIBDIc9 zjpbt@%Pof{FT~DbOrA#9frRc_x+iJ+tUYQYaSMOo=MD;RiYq{Gq>oS1iSt!R<`vmdC!Yk*Xp_uT3zJms04Z+J7Rvi zZi0RA*VQl3OSefp=`3fz)VU_4HMA3Ln1wXzld`8Gvv<~_u4LS*h|4p$m#43c+_Xdh zIbF|9PLRDuJ>>LD8Cml)QFx3eDD7I(emg|)$p3Nc`9|1YwS@#eX{3u7g^DjEcgH`e zRFSr_eY8HYq4;~ltN$uG#x`J(Q-?6y{|e1|SH7C|I_7YG#~q+P(iUc!8pI-wmq8Eg zG@!b*&sI8W6s~6j{;X#`#tD6=U-xMGzp;~W4G3tMlP{600F8$vEP z|2Iu~!q7{wL7yMivQe;A)|Z%GK5MSZ$ndIpclQIjo!OrbN&|@}RXiGe$>mmvBotn2 zS1cJdT2~~~^Yq7!n=Gw6?oWlU-Qh1RFkq(yYA>B4^*FzF3LJ9k$SGv@MGkZDRHDF?Wy69L)Hs6sB#ImfaPdu%{ru!|=+?b$k&DHO~`nC{=AQ0P!Ut<0C3UC%KNDpD@XoHws zXH6Js$-Jcoe>2trRWpc6_e@1I!o4$e%}&lV)UOhUm!^>klA=fi`sW$yVtYeuN(fQz za8^wpY4UjHZkkV#8L?wpc%VscEZbV_Fz#Yx7&o0?7=`#RGt*OK@A~hfEDZi_ z1HJ2$#6RZD3URdRJD!n%@z$PDfYrt&x)CS;*jnvs1wBY4Sc?c`I1{PXu#om$`Va=^ zYm*t^vx~hv{6J6g)Zs67Wr|Ej&}}6w8tjZc&69a|l>}kf3s5+yIx|12)jjWesFc=Z z<#PlZ1uBTc^uNHzMhINgCkqFLTY+t!^Y3kU2jaYtqaS4oX;kOIWb{d+Tzph@=CD2K zmdjceB?Kct2dcbgt)(Qf4Nx-dR9G|=fr*nB&94kt>MHvTq5i!=Wd#^0{dk{Be{z{j zbrHwJeb2AJY-*}?H$pu@`PXJQ{^kP9sM&wk$@Xi-FCgI1bZ$!ry!Wl@a5sA56|z@y z;rtgCce+oEjQI#l0&-Ys5+Ddg386oo9n5^}jKcc7wd+4dIR1JsBPNVFPy6&?5LLkL z#+U5{A+%FCM3BdR57)&mZ|&ys?UMR2=xWt&1p00A6$+ho|IkbuMn{qGwaG@5)5HTk z#?QIdNVh%b^HD~u6kG4VzLgMJ-uzQ*PCkwI#sZ)!<)O%ph^iA92xqP#fnK=$b4cK?xPnRHY@!cp-0s+F)U!|p`ljqx9|u-?RG>~_SGKV&wO z8*#u`6ky4R-6%O4Au&VWo)}$uj#+QDYHH^sc~o|JIV(E&;38*8^|zk4?N@Jtjl;rHiEe>BuG7dvc%k73C%Dk38KIQ( z5uKW9qPqvrE$zV1GjGkonB{j;rQmou(bAv(?WE;1tK3Ci!Yw;2-Jl_ zY2ow-V+*H#y~J2!#xrr+-Q>y_^GCN|veZ>Nc+}E06G3>=2>LtFFRc_!rL$WkfA}Oc z$~TpfJ-;Y-x;xX6!D0ulH6w7Zz8{NTTxm4B6-fjdEmjI%vs6jqg_+|7-RnQ*Uby{Fr^Fp znb8rSp}T&XhHmy-<<=Uu>%r^e^wWAWjeqz<=`HfC?ux}WCMccj`T^}yOEAZ!D8C=& z1Ch;O*INKfjT-S?kj8s_lP|inHe0>w!JhB=QhQUO&^I=Z?46^Z*MYvI!9M-5>uB`p zt#nrO(aT-BDZhlo_9a(+p3-QDI20$b!E2^8CIyrh5 z{(|05Ng!|Zs9vApFHjrz+P;i@oyjczX6s)9IGq~NA9Sej!0#c#jpMv48vF^awY2fDS|@o(${Mc({GQ5ooI6KWzgkM zML0h8S<=P+IC*%?Ho*2%l|A#>{0djeMOC~Biy_gRf4h0OM{8V zGO<0kI>~eA39tn<6tXqOlUWzw6@6dM7RIi$q69`DxK+fch4qA zcasekrW1UZ4(4$-{~0l0^ep!CGmf(h0osaMYlqp z&ZqMDNu&>srOtS$_^JzF*WI^ky1h?yI6)cx*Qm(JzwjbWstIf9d)kqcy2997Q1+?~ zXI_q6)tA7-<^vnwa+Nw0P>yR=UuTfK$Y53)Wbj5>v0ueJnP}vF49Ja<&?=cPWwSA6 zQD^%ODI#qH9;x)nl)@8Ct=-{Y`a$gtgY-!a^cAL%4G`as5Khd%- zoXrg~95~>q7COC<1E<#edmXFfIx;_*Nl2y{R0hNHa)i< z75(+g^%C&9 zpEq^n|M9-JQmi03&KO`CNqQ5hq&mcG`wsv*w7Mzp>Hb+qs@mp1-74c^A@fIwsT7}rNz3(^m)R3x z$Kom6iZR*5<`=gr`Ze66^G6c0rV}#NohyY(ye${w=S77Z!!=f^)@VuX7$=HDF6yFn z0FLI^=_%e(UXp81GU8F?kNey{(y!JqBl>-&Otqs-lMRxn`fQ#Q)C;7uppGi3jrnr| zG%ftd&+Dn@2$!b`gN|VhDNt-pi-SDR_BW|Y+$P(LaYl#8Pb=;I)M2P=8}>O8wL-aR zlr%^&s{l5-5U*;IqMA$PDgkba->di-HaOZ^bU7) zb*JJZBQlO^Vx~t6txblFY$>fg5W{@`W>c6MMYL(Us#%0(c+ZsOsPrd&5wU(MV`US z*GJb-^L_98g?A*`UTD9p9u!f01W>ne0vx8&qo zrPKTz3<^Kbl7i^|)1;H_4mr84;GDkLv_f?>HRcybH@NIBPl;$wN1K*kY2pc zaxG1KipmyQTm(;`1HbhKdib7>&)z=G`OCW13*cx1Il6C?xE)B-NtE`GI2Rws_YS&cFNpTCAfzQb} zO$lQ;^kPfD(oN*)Ogy1{09b}2kPbl^{LHFt2}12;g!u1;DKw;hJ~@xF)IdC!3Q#>s z6PIs#ba+kDYERyRk#5-?YztYaRhNG1++C8vxIi`Wnoq#2!Qt1YaMwur*2!x1m3x9n z6qUcq_UMr64UwL@oTF@eAVX*CGZ^@LQU37%$?s|0@r`n?j_&?+2sxZ@pD)qHtjAxl zwPH3v0oWk%^H5#pKNp^$Nd_Rl$o@IDoL_#0^GUefepdAN>(8ui?^;U+;=N?Zj_Sb} z)t|orJ?jEQg<<-rd_jy$Qe)=~Uj#76SeIAqQ;DRYTDpHm)LM;h_48fYvBNY7m%(yM z(`PUhuZeGrFTrxMK39WLg%IT7w(6{4=-wJmLBRQsJfIF}{@F)fkM0)`s;$R8;N!Ter9?o~2?Jqj}JB9^NI#d&E@`e>JbX`inlOPJ$>X$=B4plp#0AL$pc1{%e1mc|Eu$< z6t*27mHH?VfuWOW$m~_XlR*|gjTgMpc5CnZQ-^?%gkK2rjC>1btgv8-N-p+_y%ISH z@y^#puqCz%8{D7i^2VsGbet5dLOu6(Pp`}?>Pr-EfX4rt<6+^@{f4Dbx&bNltft>G zWf3pEnCq%w`gtGr3KK#eo|)>ip2T?GrR2FgIs8Qc4@8&Iy)B^>X!aHBmqq;IGaw1s zz2dnREv;>KfDrbzsoA5M-?1pnmwn15WzFXZ0_mZjJd->Z&56ZN<_zw7$(*2SCDD5m zr2AJDFCZWQi^O4(6V1nMl$w4=ap~L%s`&W$?CE|-f`gK%NsH*K0p#TXyYyk0X_IG& z2|G|RbJft`h%EXinTa)A$Tuas{)=s$ijKll?mtYMzt6as^dResAL6=(X9G+*#qw)L zyc9sTP##q7E(U8OgPHP^4qRsIe2-ms!RG36ntA~$&=FT_NwMW& zcHfS%Qgm8P9%Vb#ESA{n!x1=H>d|a^Z`m9O1VVK%A1}oDdJ3ce@vs#1lv^_vvi0LL zLvnKRHWGYb>ss6^MSh|;A2420nh8&PJDK>=mH{6_2TWU#m~UA;8j2~{1%f%Br(F!ck&xc!1{>x_mkpofOF{!cS0t=3<&Qu7d+-* zp0)u}HUIMFr`lp+7h77K4AhHZLD2&liR&wTpdwhgW?Gph^=-~5s5}H@N61=%AIw%= zT`l_o+TiX})8gKM_6hLk)nxRYnK^}3QX8wfe7LD>PSB-z;{!RN5&aD&rXnX5Gb=mA zk+0TQSNvi8Q9Rw@JZeYit3($1UPy0v$zUo{Y{Mk|va#G-Qu+(7m6pHOc-{y4!GW6u z$hII?dEjva1+4(g%);_>>>oz#L0^*K2HA7}3n4k%i{mYS=X7cRi&E1jz~BU4vw}dY z!IKX-eO(vj#b1q9{8_r6XX(gLgL(Yd#kcPM9h%v-rw~q7|I|h2+J$51VGHkfbuFvY z%E@|BJfdWM8EET1XgsnucxG!wW!L9Y@(Q>L+Ym@q2e`kCtu${`dAMa=N__U5mt*QEFTqM?6Mompa(%ZtUnt=ruQ42Nk1)L}uN<5(qUqBNN5&#bVU(>A zh6FNrnUPvSa-XsmJ&~Y3V&<|lwL|@R(pDR?FEL_pKk14;1IFTiH z@eo%f!V2s2ajxE)p20+L*ZNU-{nwV~)`(D<&x(Z+*=Oq=oORQ;hsHl%5BD5P?soh0 z5ROIXwU7E_5<-YvsUG1-o17(Ei~M1Pd}bC-Q(e^-b4AnXV>tf031kO`t$w5u$G3f- zzM8ZKyL4CiMun5;4I(#IN@qWdMj45ew&Rj~8!DG2{&}*t_;Z2}?XTKaOitfXXC|5- z>%}ihIF-J9{ao|A_2pSb=r^6*ZY&Hkh`2ooGD6^hIA>`47}cnEh&==A8|2kaVo%!D zF5&>}HfwYe^bBOaXww^3T(_z$J88GA$Z6t|St*BUD;nx+SG688M-xw0_T&MhzVodQ zs4a8Ia37n?xm%FD4+8{lTet1`kGn}<`oHu4ftI=5B%OgFO5)2(r@}!G=CVlQC6}4x z7p`}N5}z|G{HvBxw+o`ChyLNvt6Y!n5hCSjFlAf%}bHjbF~ZD+woo3-dx3)Fmz zrV^SGi`ch#b=Jfm+Bs=&-^ATL&mUm7s4Z`=A#vo4Mep+z?&E#;4%|bRziu$4SBw&z+f2UyMqS#u$)V_Lub9`VU5vLfPq($h?!JG~?b@4dXrgi`Xfp z%Ir^?ieIL4Xxfx;wZ29!AHRWoN7M>67_C9^6~>ElJ>?9#vN^&awCL1-!h3|u@nqDJ zyCFBLce(T3x5SY#9U_%Cz2klbl>~tjr7!BIo(uc*LM!(6iaYi-7W~!{E27thZ|VY; z`rDvFi$))ZaN)89&!1o3-CP_jHo6_J_wWu8@w-pgjm$>Uf+SQEzHck z3p`y&?dE1(ggR!OC@Lx4?HLR3Je-%5l$;Pz)(o>2Q%TaYAK$=j(cpI#Hy0)4Zd@0V z_W@JmoNl1y-WO+tBSrgw^*$0Ay3nDlzZxmhSX-%~Vl5|et`mBOUY0A;=lU)jH(2b8 zzq0W~>|(Hsawv&Mlz~(WL9ht3J$Yo&FEq*W-jGuf(n!|mGWwOM@EG@)aUp6gGRX@z zac<;ldBcVeyGo`PGyh6SF}B@O1wOOo?@*A)6JZGuN?F#NM>ltL$X1RZl4R$bJ`kg4 znW`&IoES^DeA#wM&e*;Fx?}|UkY8E57vZe#EI0vDx{|oLJX)I!eK^y3V6#~jF%RQI zL+nrCs<>b@zp3+t2OE+?WabsQYAHwjp(Tmvx_$G5a|m{Nky+KxLQnnTvoTze7y>(z z&<*=Ah$su47cS4eRUNwdhvDFb*>s|#s}+$kbA=3Pl=2N*kjO*C4EcPJLS2#LH6*w` zb|n3F7Q4`Y#4{=WB1iR&2=4x(il`gxt-{8~f zLF0lr+`#HP0|SFjgR@OK-rVdgE*{>3lqj|M;Aj znmiCQ`165C`x8qFJ55|*lS4!P`CA*+QgwULPd+8ZstgeJ(WZqND_gsP7)Y=}600%I zL}s797SVaS!@`OF`A(5l!v0VSk6zw`5YL3?y=pRIYD2buAdm3&g1n>8ba`h^;2kYd z9%q(&V1lIWq=;tuYtsukNvCU?k^HBrX9Yznh98uYclXX*mDtu+Dv4~LrTnBh^9G?% z0I%QkSmB$!))O7@iX=7ga4l5PjZZvBvn8S_q<8GrKxP6jSXfX%=y|!4hF98f-0O9_ zMg;JJiMcrq4NZk5ugmfJ(OMT05)wn>>1aPD(DkW>=YJ{OM|No1`pGuJSN5(Z5rIv9 zg}ki)V6Q_qo;rTuTJ#3io_VN>eJ|51Z{J?UO68BrU!X${zczj=6K)LgIP9kqeLWBL%yMT&2s} zs73jeG;z`5r7BYX%k8FaMv?jH(&?X<0?CB$4maM09D z9nFg(<1d1loGdN=XU@LEKB)ZDlauEFyx#v46Zp&6&noRZtfZplFlSK{YHtb$7m$kuwaS^Hf@g0# z>JbY$w2$RpR;!C6x4qFSD$|ZFR(uyO$w>OJ+VIgBql%w)94S(GlsHL}V(y4#O-JJ<7 zPV7sqbdj&|GivC4M{0@@=0bH%Ea*I01!Vrx)iWj5f7qShocD=HM*MNi2xe38cNfe^ zJ^EcFre7HI<|Q=A)Du7Yb)QWKYa@n3Y@4=__Hb1cg;FtU;FX$76%NgylgDVBgfZ~UoF3yGJk8S=c?EV{^}{yebz^~pbMSOMII=1xKuQUd7*=@YKc?fe zax&%?E?{r3wN+dWf% z(g>hdrr)S69oTqDc{Vhj#Bb!ld(_D);r)$#E;!GTaCL6Os)O5d3*{^ z;vPBDo+ok#V(5)l@XJrx^35ue)-UTWNXw*Rqq+QpdY;IiE7*RIa^DVcOkq#74K0>e z4L71uD2rU86(Hw{zLO$vn`e+v;od{c_+DUv)_$g|d+_yoy^y+~vO;)MRWmKC0w~sK-}$o={ZIxTtxNxlWSYIV4P3=jC+u!U*=Q^^OBMSmFuKy z(3r&i2Rd)I1l!P~-{)o%%2DgbVvak*(OxHFYrV$h<@~gJp%VG&BEWza~i&*;ai;l^IDOP``$Vw9@=agH<{bi&`o@;$OO0Ib>SB52jEGLN_II0yS z!;NCT>)e?&x}SpjSqA?=M4~A;{c7$kCY^ynbfH~1{q~ReMk(BF4ya!^U|eDmV`zqr zFS@jauE}%w3e~STp*15i_~B%ly-cBHD_pTXr#SRiR6Vl2o*xz?JxwU_pYO^+Cq5`- zjEZs*&AKHIPRr1kJzS*Dp^?O%YD*~exfs(84fxITf0Lg%v3kCTgb3Gd4mv1sY8f$P zr4HQHe6V=gE;X<2;RcbN$g4bO2O!_pxSvM34Y0AIWwxe*njpJ zgU(93-5gP$8|`?z=GFeW3SaJw!1D>*VR+ZZXU3fjs$+AY+TI-2kLs3Y=5PL>uO$}I zoh~lHzmx@QK22fD4d6O?nf60HEZ&P=GPQwQma8N!J}YNq<*OEvCSSi7?`_V^SQ+ka zs;Xne-zgRaM#0C7#)4OMSsMT9ufs-p2*W5tuN31Ov+2>4o+GC94 zr`udYb@M4&mZRU(OKOS6M)v#IClFsy0HaE*AyqzN-l@2_gwE;ZxH0+k-YM<)P5!Hp z4>@EI$8Qr(Nq1uF&gu0f4zuYcr(aaD{Me+4_4t9I%D*jswOx8sB~;(24&a(M0cHjP z$$0U$M!UEVnhPbmw0nrAm8G}p=Bg#bGK22Syt-5*2bN8kap8xi19Qw~*Q)rIKL^f+3_}x}%Kk6qrh|J=$WzH1f$RnK74y z))I@cmijwcm7`AR$zc(O-_~%~F+UoVnUQ2ITyN9JV7Jd(a zIKj1;z&0X8XyG8&*AF{5W%NNE`^xcyH&)8#i2LMl9L=A2nLE-taiC;7zP0@x#3mVt0Hbe#l;QI;|=n;GG_?SDsH`BYJ8R6M&bc;X5E59nOQQB1(tH9R?%-c$cQNu(8IlDpjdbJ1!(DI~w*9(}p^?lDFI^l`re8Z@5k4;bYe8zYh=MN5TxAz-?RTAfMD|R^|)ZK=R&?HX&Omp7a6%$o&AL0(Vsnr%xa&s%K~k= z^&>Juw1z}}Vx0SWpyT|w#Wvq-U5*51M5qqBy?!(myD#Y+l70ctgu?CunGNYy(Hf*M zHH)c6ELQo27XK)0DJTwX4JMq6=r%=E_{&^I3coHn1swQKn0)@9`6a7o*Xj{t_T}xP zSRU!9BC(uTW2vP>3{kpG*rDLl@tz4XOZk5JFvX1jylfm=^6A5vBo-e|7&5PH-QG z(t8hkDz`JTBSpFj&eMNdqduS<%3t@Exj3`vSi@geVp(fb*L56~-cY@%^PuzE2_h)7 zG(8pYI=j>f`v!X_dYEM4qc#a-ovsQ)fK|U6qw{-1cA!X+UyM4{j9C84DIU{5q12;- zNdum3-}tfkO#d>`kja3KnhxgIiTEFoT?=-m_deQT73XNE2bs<6PoTp^EAFd7m^F#6bnMP_YL&gAM8{61t(A@A+$u$a7_b!w9IqDq&uJf6fY z_ESu!YPjT|tc{5C@tpWMHT>iS?b^z!^@7(N7N;j;+q7{MVcu6pW2`bx%^ocm_ICg7 z+DTC_(^5AeL_VQtpa_e9ykEQvL#CYQkhIL9=HRdW#~V*S<;MJyED1Fuk$-i1;ud9( zi`myM=X~sV&k{K*DoLkoLg_$#H*85W0j$hfX)+5-1*5kdJ-8J!y)**(1S)RtAwG>8 zn6eO0kAVMUsfBCTd7pdGW}f)>C#Pl?Ru{Lnnb}x%TgH5^)ScKYY^9zvx-JL}{pqbJ8B-ve+5~{O*DY{lC5G?vpYGhUs_Rdj%nbg|Sp0 ztlG?^Oa(BUSH3K}NT5&<_Q;El^)~YtuJY#>K){WLEt&Iu$a$A)zOp9PYN4$Wx6D@_#j?-NV(*3Ujs zu2Jc1EzXe!yuME7nkx0I;u$q`tQm=8#WkI!KleVqY$#4^=U=UtSE~GVq0mmp@!-@}{;#2`ZAg zu%1dUq%lwSo+51xu^XyGQn-*_+2l-h=uSo`&KpIMqR^*2;oQ8d*J+f^3S$2ZOhN$4 ziCi(89$5)bfMEehf)2}YB2Q+ii#3gi-n9>suG$1Frdr2tkHnevSNZ`$m;Xzgd zES})op9$eHpg`F67UJpIJvWO5#hhg9x_ol*Pf&>dw-;dH3ku14=Buc%5vCy0w@x!# zfoc3#4!UJ58}l+>r*r^>W%`T|5Bk1wNNp@aL-KdE)q4rEt-N9ew&!V|bu)N4s`dtF76Y7f zKm2!~n6P*UXAoveoD&@55wi~5E}JBeXck0{T_rV;9TZb!oDAI=kOg<$30^&rwkGsO zDPiCHl}A`65sB!Z39X0z!I5(D;0e`7!awIutB-W5Bd$R$(%`n~ez;wkY&Q2#an25Y z$J2Pb{|@t_MF?aDxH}k{2+apJ#8te%#MkT8$QsLkN~xCPt;3#nL*ZE> zUAQmL1s_-o>0y;a$bW<%ujpO??InGXrw!QQyiy2enZP*>UmX+6vWVH zLeSR2I3iY7;c*P5-f~25D{-6!iJ4+h4H3eZm@w)|b@(@<-@|=I?g~=X)fglu6Qzedw8)1$G7;9R%=C#J4zIppiQJ+ zU%rvp634)@xmUN;!R9kp9-gh+EAIE{_^%qQvpuRT+uw%Me(f4Mo(FepiEjK~Ol^RL zorobRgU%{O(dm~Vgxl)J0U2M*AND4XY+I_oQkeE)lN@Atw6ZsjdUd2f`6q2N z%#Kku3;wJu$lr9t&v0?99_%Q#w#hNbAg_G5 z(Yx7RdxCUrKDexbzYkn(%ONZ|b;~v#mH?P)_PVk39lHM?kY5xe4^lQD@hujlTI!F{ zmyc(F&?;~AHT0P|;|A^;AT|n$Mfwgf-da)CUzC@zRAzidoXLNR=OowgwVD-w?%<43 zs!}Ct5|z2;Esmp~Pw@t1gT91nrOBQ`X!q}5ojRc#GBp-y^tR1E{WS{+HCTr(cWrwC z|6dtz&7*djiTuAiQF#O#CR5uF^)*))6x?{xQU;En!W+DHya-HnJWftkp!VC{aq%?v zFXQ2}@qdA;uE9rSJ6Gnf39uZAK##grD{sn#jXf+nWu?>d6keEr{ZIjso(@6P@MIKht)>MoC0n+2OFu=GWQC z5bOHZfMdlEFp~AMs2UPEp=q9^k>b3IOY z29HS%>n~#;E2%6AMget1E}l&=8%f$5PEvgC+{@Ok%XZTtGm^@aa>wtM^zm}{YRAgg z-%RXoXv5iuxCs|sh9v8OHn^|?I>B=#pQri9v{eAfL+}^DPC<_)54>`oH8uR z-pdQ3I`)$W1<-@H(X9Sy%dJ5fOBx07MLwsgh`7Yh+yd5Q&bSkdZ|;4m6X#6atOs3aJECigz>kHR7Q-%3B+Og@rKRF)G4Lw9HXw zw6`pz{gJ=y#D=f5hatkbPx+iW^nTMgR(5-i49BLh#8?V#OOA#SUU|!qwj?7G{Om#T zo+kb|;L`(*pPi|D>dAG0Kkn{Egy;9ulA^9opZGMF*t0a73d#`aXFz9Pdv*|TI}~iA zvl@x9lAvA0rB{Z}gfasRNL5aG@YA8SBDE-udByAv_>ew+S%3z}AT zYUcf9Zxg0{Q)vVNJPNV|rI_*eDWM+h=tAm{?SY+WFFvv~#h7scbF993?G}bD$SX6% z`}w@%9NAoVHIf^@jj_P8|9qnxi7uERooNp1afragu)_5w>dqVM^s9R>@I z<^>?@SWk(1^KfC-^GI!LuO#kd>udHWAS*J+Dl?&SgcOQ{y;$G03IFb*FlZ6zVCGOzge4I{R&PZXGEwgiV zHC*Se6VGxM#za5k;p{wP$?{YUn+oW-{$p(ha6(BcERr=;*otq>4FO=uJITHr6qv@; zrryRFIX)=-1N-&W1G+-AyAOL*y0+&+AazVyGtn=Ez7r z0*C)AhTv+ejpudgUs0xt7@Yo8T(5(tWLWjtk8Un*VY5uIa|_Q8vEVwGCq{O5*rP~G z?qE)&Hu`P0a?Z^g8@dFGO0Ea0oM-lqESi5qZ@YTN+a{aG0|O9{@w3%s#Py`hH;e?+V$R{h2PQ zR$ty-_ZUytrb?((2n5*-#oS1F${gUdwl&NR=Kfg2T`f1Z4jabJD9YqPBPM^biHeZF1rKN0 zsDokX#gC(XJVx4uVj=*{zLUUeln5(EWF6a)u?YmEiyGb$OdhiZ@wf9a<#Su%S1*O@ z#Ht%wnb~L;Wz-vwqQY6vkPx98xEeWs2nUpN^`8B;NwJ%IBScl4+SLnzI1PAiW#x8- zJB*um&RnfCmuyBA5J^$hQ8&InkvN)=o6LHCF}()Ojzwb{U8kTfKJhM`ux`WlNw!Mw zV0PeDQE3U?dvBN4!VyyFSzd8-_0_4cK#e0Va@{HD6(POjnCk0OO%s3nY>P7xqg$u3 z1I5re`5*=z--diZLG+kF;KX0CM1T9hQ)lnsuyOr%)v?}CEv(}?2S9@XYqfLGPl(h( z?-Bos$43wOuXo@p%#uoeZ=ZSU$Q!oNPySiB)qsfnin>RvV@{l9p}6to^Gr6dfmHoi5`G};1vL0O2kSIJ^@p@#b8}%(W+GLWA|CtZiSk~|m zhKps5MADO4X?0s|uCA`GP``qU5VhTef_>fDtupbm+oO>s@5%kWdy0+oX4asJvtkQx z9DS(Aa6q$K*@E0d_{N&px~|*i=#N+8yuD%=2Vx{grZIo;>s9%2k*XW1g=$OaM!W}^7;zl+(gZWMVDP_NUE z`~;+U(S=o&R{la30}5VYZ!T>HgGu=4$W9mKlK&dNQ1;#kTY&oJdfhIjKrGy` z!4r^X$A0@bd=UE%{DZcRFzUp7y6DxU!@)_jJ=>s69u#=O1Nz#9Rau`o?u%)&3U2@cZgBj4eoMNMB{`QYwy$P8!V>*i zIs5X<|7|0olZ~dCQo5v`nH7muh09627ILfRe4~Qv+kO4#aOUggT;BA*x~ENHTNFZ; zMBKXD{Wp1@7pA)p?NwxMF`WO(6Vnd*i)tx#I4p;eP48CbG&(#yET>QmtxmjxZTq5v z)89q@1y8Dp2+I)Pj`}9U9=pMg_6zM?o3l4u@#dTfxn*a;2dU|cZu<+Kw_+21Nazh` zD#Go6e%GK`fYSgZPy`Htmn2DW!b2*RbZjgHuQ^fUNX^-Gf*u%GP4b?LYLo2c|C4_J~PITfxi zR@H}ztDv#CgC9+X=m&!8$ z(`qD+zY3-%U!UCr=f&!+#=pe?Q{41t_i@hs87(?sTNNOB9?OGQq2)znJncto2`Pzs zXhNPVYZ#q-X&VsPw?9?ig2`+GFbIYj4jTEu8k2b3t^dEsH< z9MjwN7;{03I^^94n_{R5!XYoH^WiT=(%yf#re5XvWrzab=kR;+wvOl}uf(&~q z>7MrL*}Ta~MS=He%=|h~3Dl#YSwy3h;y;|0lZ06%h_>6d`U6Fj}ggT7o-ar-ZMvq^gA~*fT3)O>v%N@VQ+n zo=NKtdmJlaKzyAhA(ws3m*@ma_QG86<|~^wdY;vnt192lEMCm=G!(>=L%8bQ67p7 zsjOps`JrD?ItynTaW(QsMR6;$Hhmvu6xV-hIVf`Oqf4{%@wmi*I%jugr^!z`gzgisLWK(CpTF`SAVT z1vC^yAM=L+vUDR`)AgPBX{vY;N}G6FL&EzV&6IdtZ;Rjmr#lCzW6A0lliv~R2VL1F zvzQ1cl%Hns8CMp}K8MLP&ygu=ROAd&K~A%&&~5Ta*v_>r@>uX(O6ShHlf}Itwl-_H z=>$C4hf5cbF~rIgZ5Ary;r}iww9@Hbv4*{|yjcg)4rwNkR13G;_${Rjug&T+iKrhR zf7RAy9Imvn81;WdlulyZo-WhB3v1Y~+g)_oB6yUoU5+crrV4x?t2d-NHT%&?d$=-x^9wORsW=6Yw~Wh+?Y|sVFP6v$5$; z$6HaSdEH;#?AJ5oSY4)UfoYyaw*!LlyziRNVArn2=!?Y(WJa%R=IHv45eQR@yR0Fx z{|6^*o;(+MBzEh&#w-bh++LfGRifyY@zZ+p{+^cs8jzB(Eu*=2m!_yS3A2Czoci=| zQjR4BeBu~zFbN7*Ln#A*{2vTxlA5`u)AV+tpwOZXTYY<`}f#4oB#$p+|#ZUz{h=`f!GD6d8%HnVFeup3@ks z*Y#NX{U%5A$PmvmI}Wnt^q-Ybj>XH#nx?aPyP-Gj-;s8-J+C&>!Zp;@`7f7!3QOwD z7?uy(PYI2f-P(fWi2<`t7 zFPf-aLBV@pe8d6^w zM~m6H<1TiabcajGm@-q&X_yNvEpa@ZzBQT^QOg%&VVtzf{PRNo6xv*4JO-IcOLg8IhBhA3gX3 z|KfEDzXvXfqKZlvV}6O;u`@59#g+t}W-X)OcQ0ULZ-l@&K+us!jitx(j~CY=SO2oY zi%?4-n|;yL52twoYTF)zDO{yM5L;`r=IYe}yygJN^pEIj=52w1UFYF?3JHr=wa$L~ zs)~Tqdii8XKn!ugX~|=Eve?|h!sBd0ZL(R?Kn^pPAlIObg)5(!>j$N*c zKi*QykYcVxXd+9DM_XXojRI%aOTrBGzc!r4m35>?uiqpJTz3%&S#0d=J9SOK(@qr1 za`L?H!+7>_a+zIo2i(**+$x^l0z-5`iLl8#x=TgQ?F!Wn(e%vTP%LRBh-=N~YQi7U4XETxy;e*$=#C-Qfa zt)?;^E4uQzu{ys{r)sB*R({f zg2Lvz6GSr@n}INwIafMaSy6F(Vxr1;h){Y={J3v?&|cM`~6aU&MK5wGeFb%dkD4gq;x#^hJFMU(G4gL_(#ri0r+ngDbt@#>3u;NKQHw4 z)GrPR4u=b9mm#Y`G_ZgQvQ1@GRcvhR*}KCIY#n@lcN~&Ry^dE&ge4CLlgyV@=2M+| zWu2X!5WB7O`6`ba$1&D7Lcv&cA82Xa@2|GnRu0qn+%|{OmsmD$?_I`NA3!Q2_T?p% z9lcwGotTP}5{a)&D))r3*fp5%Oy#nBPGUc$1y+0lKl}o`eq`k6dbjjpTh?pGBjd^U zlh$2$xCjR+_zs@#$)U1g3609ci4E2^tWOEm%%R1KpuweVf`gF3SA)QIN5=$^ffJAu z9DG(W+i*;)?@z;nGzKu7>VmCa(OP&cSj#GRXsRm!BCls=v*Y8v?ey;^DSH zsSr)*nc%$p$-of*%{TCyabW(+vUbHE=0EC>;1*%*qa-Eu4Rw#K}Hk{IWZau0F5*v;&qsu`7ciaoDS-J;cMrW*&&3qCW zV;M|BVAH9y78#yuKYuuEL%o={L~!0M#Ky!NNab-FO5+1@_wK^$;UbM`66+MlUoBuF zOFMZ~Hi3}@8fCBL1)}F0Ar|975X0*n)_P-U(ml@Ae1~@?3W18V9hR~`h2-no(ok1@ zDG!f&nw_8XuEM(4fcLR)X$AzoryLnC)X0i@Ne*14keI2O==S${6_Xnci`&E% zZXRy0Y!8kVaCWENt{>b%6wpMX1PzbR#xY*zaa;Uxw#I;${zwrU6Qk*Mcb3dy<$2I} zp{}9fmTtXyaRWkNC|Vtunawxg(K$a_kCwHZt)xCmfEg+uk!(8ug8K8N=*BlfusrRs zgTalx42&I{VX?w^NQ}sX4DspE2_i`w_bH6a)$_VV}8;e&Zk@yxh0gymjlv z6L3mlO^V)orM8ozwsy6@9;Lwyxg&%lJ^^sFWFVOX&;MxFy!0&XPP^H&vFe^T^!^HX z;#qx;yA?2JHm@cC>40)H6V|=k!dwblMDq~EB4m>y*bJ@*k&+Wx!n!`b)xWS)T4Ght;Ajokq z&5b64D#SLYN_g=<8;AXL9-O})y*|*Y!U(Wcg6zb72@Btaue_iGF-yZXTb$wEXOflr z3l9&WOz>v|Uf>7np&D$oc0E`w)ZViH;u1gShQ`U)d`)?fcDel#K4?#Yo?a`l%oW+G zjNO95rFP^xPekHI67_)YflNsBt=D^L>#@FOxwXdh3MCY;L_jXJ+RT?5_GtqH1YUc6 z=NaDRgSNB3?+Kvf8zBIMKI6_6KiqB!05H_d|66uj?YR|#a~or|)l!)285jhk3aSGG zMI7Iz|2XU_@5ZGO?76~MCvR!WuuQ)9d{-%$V$|Mce*Hb#>`C}M&A}ry_Qu!J%sh?)CXeveeG%r??WY| z=Py{52m{lLOaKAo#d@Vuq$K8}!9^%37ptEEvro*u0)O&>Nm0ZM6g+m5B!7Rehbux* zxc!NX*J!+Wfl9cyvm>N~a)_AG5sYmGDiBc1ff)9V#V7=GnW5=&b+PcmEv`vptBo?Z zU2`=#`oIK`rQ? zts{50W=3DSl*7WEzkWAK57Qs^*`x?a)+?2ys2lCHCz=5_)T1Y5z#hnRA-ym>e^-*Y z*IXBnE--h~a+!%|qq_UFg{s(ZQHs zP^RkL35KYa$C=7tv1WsF*;(skv4%>C=GE0!dRFQA(8ay+(5?GZa9HlEfs|iL2X-+PA?M%Q_wy>*yE{$me@dK8>o;CjE`d~2s>B1~`TD~GR{Q`ws#F@d^{F_H{ z9im>p_}L5S`=5B6#>IUu#chIH{{*dt4H|CzK_i7#IEWfbHV7zrWb9wby=CVPHy&O) zA%qgVQ7!r4?~UDvik2CAq%44`8&%9$f*lCh+RnODC0cZJbe1)4Cz}I^8L>1f=pvgX z`)Ym5Ye$!0(M2(wfY-fy-U&^rAgIE?V5+5=*=)JtXC}e3kGT42P;P4# zn{zLVx8_1{m3An*`M}0CQ*w^vLFYAsc0|?J)ktI|&aB&HAWL(Y{Q#4R5mU}POlg5x zcJB#M9W5<733Q=<=W|v=1}J=7x2aBxGY~sU?ozMCauB=$S713;ZGo-hUgq%m0O2r> z=yv%9jC^<_)O^{&P!#;;P99?ekQ%~>x;RGAlmOuVOW@OA2V^bD78MtJfTEOKV)l=2 z8-OqJ_gJk|ptbcK-;uYZfkY&%W34W*z|@v$tD&7gMk9?%PzVe|Adz2D(Rt@xQQ>3; zvrZO*kk?s=PV0Nli_Mcf=U3BVvI9w_Jj8P;0bP0gvlFOVCh2j{lLr>iR^9fGayB60 zv8lfJL2c+1!TAmH4DR19NMT@D)7;!#qrzyw_vKqOp^c4=tru&$Y<8Pc=I?Ha9xlap zUkQ1`5g|V>-s1z4xTRBb#{J-dJCQt?B*y;d?#CaUPV1 zCQ|3ru_F;f(U*z$$dLb~_mW4-34G+A|EJz-A@+#a>GZUxnfm8Q%+0zQe*NsqZxVP75yYN5Ygu1uwwOzM!wovQuuXkrcx(Jh5P34EagGB!ThO;Xu<_KPQt%VC%E`a}AN)zNA1zoIhKGF+Y6Oo&* zG*tkTj1bLjuQ_5_P___XBp@zK?Jbd=+CASs7e9>~8?XZVS$@VmuqKJWM4y~IDl_Vh z7(<2rk6Dk8CPbKIZSFkiT14p8pWe&&Zq@MExW7Co+JNkG23W*EKgU-o@?AfY@k9O_ z3ZU&+3tf$ht)L;IDwln9?-WN+DDWM%zs)uMlqF}cdv^kA&vj77p@(Ns^>Tk{tO5K< ziB5CV)|#j6AG`Vm;QBY0YvILS=Xe0P0XE7FFfms3P!|jC)|4 zMQCW~GrkhKF(H8|pDTf= z=S>!$<#M3WElarYLp^4oBSNdhTYOsrx&W{1fZ`VXl$FiZ zMm*pI&ri3<5J|$j>^5Rma__#>J3`B4d_I5IP4I+wAL_b0AJV{L&`x1((P-lbwAwzG_4StNp{~lBXNlYfG8a?#P>|5+#SJn{UVg8`*T!?fCt1oFX z7!B|^evJ>CrcbkDj+m>E_)lM}Pq1q*EX`X!&EycbwFvs4lr|JFy~J@3$fj+e#C-1? zN3WIi%PTFDipCT;W8(!BczHe%kJ#Ah6$fB+?0TZe%V|k{<7m~GhqoptCe-TZAq$q~ z(Cd%6Qi>TR*Magva+(m!ahdinmIm?>r>HSth1S4o)7nTllcw+#5#>$$J46W z0;AZiD=}5)C9IkG#^(g8U$r6OgR$*a6Q4xt14dTd|JmklotbHS-eprW7m6m2A&Tbv zCO>0mOkd*S`OVkS+8*}VaYn$l!j=v66oY~$G1VF-NC50Q4Vy-OQsPF*p&p?nAA^yS zQrn7%iXX<9J0-dWq8NKn=I^|uxpzPaX+M5EXh=U8IkOQqLouh_|Nk%N_AkF!{k+3q zWc_2_(g{2~ZbqwhZ3MQRjm^;^1_E67dq#NFk71G0EH8Vw?p7^#+HGvX0JU;a z58E`Nd);%BzA*A+Yf6h zs(*WCd-UDLQql$_JZJtt=>Z%yke+_rhxA)|aiDLomFFg>6WC7JpG1?so|kgUU9eOg z+bdG3!rWGyCqwd&w@l*6Dq|zDiKco56qk7ZT5GWL&7mLUs=HaQh-0{2d9pM{>60wQ z(i2`4lFT=Ev)rD!@l3|>QO^X2{0jD3fiTE=d$et{#4Rm_m1>$t4ja6I$@j#nVolIy z#aKERIi3qM)Kpubzt`20J*vP#)@bK3cd=yJ!r9oJtN4GoI?J%AqpsZ#JqSojHwcJy zOLs^k0!mAVbax}2(jC&>-O?qUf^v}XZ+v?6=(jl_u6Z%`?v0$U{y%# zPXxL!FQD$&QN1i4mg_l$+8OH ze>bkPN=^OY;$v*|!!;g>{2N^&-ki+<=M~@a&7Hw=2+uZb5KP9n?(NwK-){orpquvb zbB~TF+Q14d{TIJ`5k)ZIMUO`;a*Q^%lY`^fOY149n-G&aqv12kPid6H5j*Sm3?>eui4L<5Pp zj~?Y9qT2nZ=V+Sde336YC8A^^>?GfB-d63qD{K6~@*{`(1rb?exzsG&_n-^K$H&(; zH1zlO8X6l11qLp7t7>aMoM%3P%(|BqUnE+J)O8c+7(QihC<+|_1z;|G&{hx#e2g=Q)2ip zc$#YVP3JNA@550(r{!pq{g6JDw*=1u10Ib>+-6a!xd3$21wg-w+iKZ{7&Anix#Ru} zxIeTcBqRXKsKv#_jD}z4b@}=E=5-z1d1e+C=l_sTxy|cRa_GYP*rt8MHRwVYbq6`* ziBJ)l$2tjFWEu5mFlTxFI9-!TDCo#x-?90{NV3OD6ULz)&wW+IS$Zjsm?6HhvT^ul?^dD&aIR~HCEVYuaT&W9|r2E2>4E=McLHhsP@Z`x_ zd~1v_I0#+1;JiUU7H9@>JKk&^;CKL9oO__C^2d)xx}?{f;o?2-b4R3OnFjVZYzfaQ$w1KNU62r8ZC3zvYYx zn}C4WT88XjA_W##R!qM?fr|)+@mQMc<>4ntCot=)0quNHW@xv$U4TUG5vV!w?_bML z;Yd6$HIL00DdL74v)fhoW8n5d2U!CJIcV$>ga@-Kt)cR7#-_5Lz@l4@HrDx8Rz&ZZ zS;slak@tL(p%ybIkbw_d>$$tw`xRorjKl3Qan2Kh#2~YXnVq(M7ky7g3l&N#yFuK5 zQ9N@})iYS=7T?x@_4LCDzw}4W{&e2ofz%+0-UxGJhzs9der_0Ibr5lBt!3p9rKF%Z zSssSc>}~kO6-7Au*5aUPH6vUaR)~n~tzR1!B4L^@nWYUPsUP_czw{cjR%3Kp8Y)Di z-Q#8~op18#QF+tVYisB`Fl&1UA?M_*tE_bIScr{{jgQBHbY_bM{{>$vOqeG|Cp@X$ zpbRYYiH}tuV;y`xRVlAugKKK-h96JBchgTYIE<6kP_9RGbsh!n+NivRuu$4F{ndJ2o*`*=R^$zL}tJl_uW%4?kB@U z;OrmMviE2Xr8*_-);nob@REg4?JnHw=- zl0=UMd{7qOVDf=?=m`_LO?G{cQE6$!wZpH!;z)qB^0hmx5Xet5xYIbyVl^1*7c4=( zaTB+V!mR88eJR_Y9kF*e<5_T*flnDgC? zamJHh;C8M(dCj|-onT`Qc! zEBc#ddD=Nf;uH*Q&FXITDa-_4(Z>3KxFTcQLq-*e zIa9z%2Zih%3>BT|yYHN5+bBH8Y{GY7nH^a?O`CHr8b}fNk#}`IbpEcI;;LSWy{o;m7v2OpCj6sYKR2EO*{l<>vmh3XwCY;g zo0NWonR6+VJrMyD=AwhcD)lAP-|%qH2U`33X){}C)T*c_NpRMBSPf+)&5;8UlgpA0 zN!;nBnNvstW>J&!jr5S0F_=lbLDj%%8VeTdQ<%r|pwjRS)@g8Xoh9Y0~BP(Qb1MNuU_}&}xSPE*0l4GTVZR-kw(C4E34gWmkuL(ms-sTm`oJ=k;QMqSmFg~JToVwQhdMunQOjuY* zI6e-^9Aa;DOI3mzhn7fwA}|*>H;oTMkTLU=f3&!A6=r^x!+Yq5~ATQQ)!Z1jag7j$T$s&C<+`Um%3SHA zQoV;iYr~Yy%8Yo9zd`yvRz79g$u+x&)cVmdHd>XVYK`Z4TV=KY7cZ0hDZt%v$G^Ye6BGL?n8n`Q7rDFPCd(zX%KM zv{QVmfO;9qmcg*cYboR%4WU0-K*L24jkVNSIpE< zLLS(xC06%1Ckwc)5hd~$#z*W&)zVDjL{LbU*{Yt;!W|?v=Mz!3+}qN-j+Dq; z4ik6?{QEyxqKV+Y4r94dlA6hbt z)Uc*geezp)a*QMn6;3|RGqW=DZJ%X?=?}qM`^hCw&P97dsMfZEDl{)T81J(UkFcpp z*HY($DvCM#p<2&t?`PySivLeWZGXAs_O*6tR3r+?F<$x6K)NtV2XuHGI&N8(v&N{9^(oR*yWZCP0S zsNip}o+HOIC}b)x_BDOqmwYW*RFBnF>_{*#!DgjC=D*y3>M%3vxkI#S&8h1o_^W+V zfbmuxGnC_ZFBjz-Tdk0dlRnhvC2UDbeF7TwNkC{?3xpLR#pKbmpeVig&yGXRxa2bH zM_@~<<2IwE(tr<^;hSbQdKz6?sgMnZuzaK+RHDayyA|Wl@_V&p3U^qOd#QxJe1ad^ zJNXIgZ0noSJ1spnj8eIS{2${?9A1XioV;Xjx^Y7EJESMAlwy@Z5U8-howoTT$GHI; z@kJ$Lnk5vTh~AH^`+0#*(>6+@6Dq)yPLws~CuBJUe3UxjI*xO)g}$Avj-~sYV}yz7 zr5kWN+^*o&v8r)&!+Wy@Q>>x81#H_AkkFS41q`ZqBWr8c=#7>pi@d2Q`8CVVK2cLC zmfYCc_9P4?2&Aq3F~T4xNG)}xWIe=#r#Y9me3bMS`Q@fUZzJRtPV~0-s31T4T&`LJ zeO3HFd!`s5+_C%4kI;XL6IYvNQK7`Jf^%qU8c zuh2w#>hH?ua5UZbP`h8OaW~Vv3u5}%wc3<{`M>fywaD$q8^rOm%QskTVnqW`CGk5$Z}*DJ zgNQ!d7yoQYgiFN5L+3JD~6kMV+J5IKGC~BH7cic z*3i^mxSQz;z2cd$Qx&9XF~HIN{k;FO1YM&$GvC>7KbH;Oe;LThAv7mSx-Nv-DK-}A z1B(+KdD`7^>kLKud6VHh0Y`6@eaV^Ucb3lo{iIhsyBx%_3sK1^Qh6yU;HubWU(+_u z9SDKIj_^O_cR5tp&2t|SS<MPcw2HlsP$oz*Hfh9`=% zT@E_MEG|ux`Q0jsvFj4axA3CjxoNtt;@$DyGn( z`&lmkbu)Ug)!z*lyy4cRssT}3wJH}@xPi=jVMyZ1`l3$uQz5R&Zvl7+2EPKEk%Zz;uUZtg_02Y!@WfdmB$&A|GGM#bLd z2MM9g4A)}2LVH?ulXnTVN*ai&%7ej%-Yv4VYqyMZW7UG+4CmJo{(ino1ZfF3;n+K} z5Xj~W)`zs~9rcjCthsR-c>Jo#J8HfPA@e7Kov#rp;JrWz7eERJRAqg(x$Z?_B{cYz^@nyzN}raQW5!qtoO_5Q zROF?)r9(zn(wrIHm0`LE<(o03H*E2tJl-kR$06cMsU>Ligs@Cwx5z2%I_9^EGCjeF znq}uPJnt2t+GE*KZC@NuiHiw3ywXwW<@Rssv1kG_|7XRE+Zb^DNCi_Rz=iv8p{5`w zr`wi=mv>Q+RtRVmS38>z@|v2dc#7Y2l}l!DBt9{Sb@0S->Vdba60AuCon8C-D-sfu z5@(8k_rg_+>!0qP63)kx83-ih2O46JQZRe$&2^tvf2A&AxU1Vn6VtvC2jN9GM_d&w5=Qxu_)pwXQ?gbVjCAl{R2YH zUpQX6em6o}jm722>Uv2OBx^W~iRaEiNp?uxBcPGDc5P^kFIh2n^;U4;P2K0OH47ug z=SoOyUoLj~h>3~$c|=@0O*vjVLL2CCNbw9k_wp0n1W3UP504vs+GI;8q4!_#5K2oA z@j9+0L`D4y8eYM#gWkI~pzeA@u#u>^Or>4oI{CEPJr*(sbwPlzRmz=Y0Dla$b^u!Z ztkhQ%!=*WS=mm`kIFOyoqID6rJsoYlR{)kSgp!1PYiHSti!FZ0t>f3*eI5}%&M$uO zw`GsSIi6ulpMqpcEFYiAwM8surupP^SzOuMb$BmUim_nKn>Xp}mbMQG7Du{g<+uz1 zL$tIOvk~Zny%T$EAz;e=zrFxux~Stg_13i3myXz~!Y>h`J>V5CT9o;2)yZkpP@t`*o%pY zk-4n-f~^R8_C)C2g&&U* zM-S(S&Mtn2Wo#Z@-dv*QQ~DB@V^0@h4eodP938cd({4cLStSoPqVHcY+i>$n;>@;9 z{r~CZ;muT9UZ;KH$VbED#7wU~$?(NP5{OS|K&9)CpZ-HS6%l$^6b|K>37iB~@Gu>p z1M4`Jj7JIfmB>4KtG~6w+3+hB!L#nU-IEUt4kAOLHus4?_5ez@(&9WfH&+4RJT5LL zuw%rqlJM;8?RQ3!;cBihf9SHG2+YjP9D%o0AOpV@k zvjLDlN7Mn;y?>=UscJQS;(*}#_SUVe{;tjvbm@R_H8eDIbH4K&`Bnor(;r7V`+xp? zhNoTkiwY2_SWF91ZD5Zl0ODy4FeLXYQa-2f2Wydyw6rv#r<-wqTW2@~niEe<7vY9K z@sXAL;K2Kq+nPpPMSJw$vL!V5cMTdgcy66|_KZE72NAAr1|hVbYd=FI2;NIb`q=%k z(WE_o$=+{mb-%3u3m6ZidhM#*~)b34A$JE6j zJox7n9v&Xf&(D{0n^{{wbv)e(K{}sPk}8eEv0VF`O<@7FS*-TT%F2s_xeDEm$IZbw z&;fKYGTI_v!Q^m(e>wjdDl4GCi4h;UMgTf{TI5vonll?cpWw;7WcWDbW>_uTSECEK zaouAZ`a>ixn;k-WoYGtQ4DoNZTu$i5T^)jIz&BP=@45a{^6JfCNCljwb%0d^k#|4eM^+dA>lzfHO?>) zag54bT@(ua`^Hl-tk3G84+#SS9dc)Zq|D4GFbFX-_tfZsA`<=Tt(ph;o+3$l-LL+x zw0pXPvs*!742%(=Lu?$5e;r&z0i#+#loE3%4jPUNJw5CD{W=DP6&2%ygGD7JcR zCjDLiRTKQPX2*kO(q#}-OFz{dF0=?tM+x522?#u>ciet#c)xjbbE91WZ4BQBQZMi@ zm%`WtgzKSO6!@=42M3ei+5hkynoXD5*&Ot}Wz#_R`1EPvF9SZ55JfoEPzZ4+ z<3F*N7KPZa(tcK``U&E4Q8G}q+pH!CT3%v_=?Ir=ccC9S3yheJf$JH-Z8MNgaMytYI7Gtw{!~|20}{|1$iX2Y zAuF*b&*LtDc~|5=6*o68_6%Aut{F{Y&8@Cxd-LWEA0Hn-e`{8j9{3y2q{*bI2M9Xu zKcfLA48}6Hc6RKE=Qx2mm_02-5A5vdV-1}N)S$GG6b756kl%kLTFEbP;uGXb`bAw zXqea5vkbmm#LK@~Fcd7`>M$F)FlODlXTwp_5_LUTs)Wt)L3gMwoF`tQb(s)dMCTXl zp@{$FL91u8Ds0C&x-+fZwz}qGXfp?F8vW}h3__2Bd$kR+w^L4_fX5>gVp||g&wL;7 z2CK&_CTAs!Iv0`kOW|(~s>RE;5eJF;-yd;!m_8uM)fLBd;%2~Hj| zy0w5#*Cqp4LMDs^5nevTHF4sdS=Unk#-XCz+zo)%B0?O}w;umZDI$nGD-b}l*x=y- zJ7Z*IBo|D&%C0q7F4$e4tOK%|Hxf9LKUG>6kB^TxF!PL$jp+bY9?0ISZUeVY{Wo75((+wSeFdqW4<9yjfC%&EtB5n}5 zo(YPDC-w+zxPV-&p{=bg;8z_SYB%2S0uez316}Yst01a; zCl2l%D+{s*_;V|(zhGCCl$4Ml9i^`WMI|F6L(qe-CrXt+LJ18Nz8XYOO?J?*EJAzj zT9g3FaXh3Nn_qBe0wg`8Ecl>u5(SkLD<#OvMwEu6zK+(%?BrgK3GJt}cqXMcC=FXr zKfR`OXsT^fy|Vo@RlsokMJO~ z`3c`NEQQyGgmgvlQyN`->G&h|-L37`$)$}(ra5@u$wzl?DMSkaI5AG;W+OmIe(K`l z0)*$?-CdX4y#nu>VO?G?4nPbvHZ~%Wx?~-Uf?*qUx6)c92!mE*rKaC4kXv9OT-K|H z^Gq)xNuXo|o2eGS{vesVk^@K1r%#{2S25pUok`{EEZ5;I-w1LTlaZwLfpQXD#?PAr zt1u8&R#s$WWFsRZZtK-r+LOQH`AT#Edyh_he(ku3o}HcH2MrDkKwrE#`!{*vVMk7T z`@IwJ)(hvCCF&KtFMsy-GJq22=4@-Kpa&uNF|Yu$7`b6Kq!g{OEYPNIt41*?fVSt8 zN{@6dfh|;SF*|#5{LehF=a!3ykj0mDArA#|QkZf{%ufNWQRQ_XYm>1DklB z>EV--obppn=9Qka{y@~%#oD5|4RBO?0&=98W^hHtIl&0PTK0B!Zf!SU2drLn03@%h ztPCtrw&c>Lrlx}^8o4A0gpPsX@PZ#a!DqY@zj>a+Xx;hrjr|JV7p0G{~p zmQX^aqfqhIaH6y9T1$E6)xSwccRTW()*A>T9bM(l5jY#2iNpU>1%ldP7?43+HtQk} zeV>$p-a5T8i|A+7qwVQ=gw3?`Iwo9v7*r2=ms}UYSzZ=9jn=3?fR3jbmu%l+5)f)l zI-Vw-7bQ`J2ZI6iWQi8wEwmLCtlu-%Y*2?oOrpMBN}l!>Z9IVR+@16&O=?_p7sV(B zy;VWtls45P80rh}TA;rf`@+UD>KCeqnb768YKi$OgJx(Lvhybl2Je+vL4J+F28>oL z@=Y5UUUJQt+j9U9L@JUv8Nm0+>?X})<8#x~9$;~T(WqjuQ%g!VZoQZtQc)|4i(w&P zY$Y%__=>lEhqzt)%a<>rqM~43$;imq*`0w10w}1A_;?Q2GsFHQu&Tfd{DchdsPOQ^ zx1v!wp8?aU!64{$?;<57rKgvrQ}Oh?zF}cO7&z?9J!n#a*me8Ew-*=h19^M7&Ap*5 zz<&jZJx8v5rQB;m8th-1DeL!b-b>9}xn!qj==K>S z4JFqQw-ehx) zOB4amI@oerA{CCY0&wp@FuCPGU=K><7`S65dM!E&QkZ(mfI zjAzW0sDHqDg@Y3hq8Q+Ql9KSr$c{CZSHG*?#$}-JkHbO0zX9`4Qx^Ej%ARU^gF{1Z z035qIT1reyv(y#pFrO(dhXXqud`Y}u%qOEkLfK?aU?u8(k?L5c&{L-`9KiLS$4GqP zkW5p+2w?CL%C>-~!%7A*&;qfpK}c@BnAN=W?(gqkI==+i*^+uJPt)3_PflT`b&z_I zsu!uguBDq^voF3y^eth4`MKAJWNQ3@erPDG-Xol!Ze`xQrE{a@)Ny8o4f5eo6KsW&hF z`eeBMR?8-)24Qex>I+w1{T|?jtGmPWaa&eXm^+Hl11nhd@^S9WQYiX8hlx(Q*0s*(kSaq=fSE-;djcvsPTGfYy~&o(3D%>p2pq{B%s;bLd`utCOuX&sXl+8`HV7 zxp}x)H@wo-DJov%9}<7^vDD<{T2;i&g?<~!>XbR$yDjEa0EWgm6l=#X z!}IXihkMz7D>fWO8oB*uo&RXKHj-@MO+w~I28+=N7j)GJC#4*}(YSs=Ym6@}!F1Pm z3higHL0Ay3T=38@zdohBwb0Ef*0@OgO;Ks=WDQ&ox>n3<1153KyH zo73U*xIVET7jOnV#HHI((T4(XlmN2(6pTQsi;4AsK9#|jFT_G+<>mEP21kgZ*AZH<=2T@I`%we#nL$&;`#& z*l{GvKGzLZMK@)FCq3p5FZe$t4Cqe_2($@t0~ZzHh7FMRsziO@^{a7Nn8g+G;mEq@meb;ZvbqUkST&|Q=JS<;Fm1@7jjXl> zb8=%L6od>7`#3iTBa6(l8tm0NFwu;uXNyq9$c>G*C8oMFf2l!cfXid^e(PGOLc&lJ?}4ggA2>(k?%TFJ#$qHaC6@VV3}baP^VQzfpqJ(x*h9!Ab5c9gob zR1ks0-9gzbrjSXqFP8y%Y}$~J;9W=b-;*SJHrgF@X8Ul3?Gdo zu{_bB%{}2Ru9NN#*zl#>&Aa#S?#;i4#o+%`kj^O>|KqprY-`#H{la=O97=I|dJ6IB z?(S}FZ4D1c#gTCSyPOOPFi^vK051QM)frd~fKCKxOvuX0?u@27pR9F(Lb{^@n;s_= z^ge<~-F63Js1lj9tI6UA4WI8g0%NhXJ6+(Gst>u7yAI6=D7f%QPa5mTpkW~)A@THh z)zRjaL@Mz;CPrWP3m3@mLa=GwJV^(rY>?YJF#g}hk>cW@Qbu@f3?|5{vc3Qf=P^Hh z>m`9a;D%)4`W_yRvU%FY^EGl*N1`7M+uPJP#Mnk1riu=jFXT=*hW-J0GrVz;8|8*V zN3Wumln%(-hWy7Kd5{V7FSDP{x(N9ZAQ*lInc^08;+bZo93FuyCZ@buwOkHQTaiSs)T-$&p>f+)8s^7N38?fkJlJd_lE`rMpUqGJc!$HM{g^lf=!OqWr z4~!lFp3~RY2iOs?=TpJpdc(rrvo?~%(&c501TTVxg#{W<4Y%ir{?7&myBF<%RK_A8 zSg2b{TUo5Lc&-#K6jnlp6!KM`!L&=y>s0|>TE&;sc)jBd9(NjGpjt_QLOvMhhdDDd zH)SK}GsrMfg&0!O(3>Xs@i8Kyhvt~q5kiEgYPbd|)`$31eVKfCanrmee4&Krqylsc1fgW4z)rXi zugzNJR2X>_;Y2B-V;qD-KN7>rBD!^(V}F8{xPtCL;V~y&GrWa*Xqds zJ|SbH{fx^R1ePZ`CMpc^vX^;uLlS(}rUAvr{2!Mqw*W%qw^r+h3t11H2Bev7SC|sK z)eC=ebiL7toSYgL*W?32M}v*MqN%2&D$qIZW+ubnh<^mb*A5O2tzd;gddR=6eKZ|jEQigeL%ULX48G}ih7;7wV zdSKgBhq?0=8VaM;S5)xH=+s8BRCmMW_VbYEn0b}c z&2B1|9_Gt4MkAj+_3NgS{C5DNNHmyX_y^_=AwFOh57FtGFdeZZB5l<8EvP`5nCdkc zK%WE%2q4UT%XIWVc?*5fOZd}H8y?b`$4`J5=VUt1Vso!9X^^q{FMUw1MOTru|L=zZ zcGb`YwxZk&{+R%7?;=CgNCHxv>6lxTJjyOy-noFazZtE$!oR#eZ+{CkfSL2-CRO?7 zmmh=flx6c{_DLhTURM68Y2vzS@Q1yiO28m?k6NB7H{D>S79Mz7=M?#Mr~SW~X1);( z_ZjjKd1b=_`MZt(voLyr=$(}}p=5J|&JUO?k$aaV*Q5tZ5XjH>tmEXE zVJpRig&orM599|IQVws`u0E|LS>oyHQ0SVotK%F97Md=Mo-)l!6%Hf;4kkqZt+>Wj zeKUW=?q$*LrB3LKvV5aeCjA*+dLb0|>k)I$>$B}D9XCQ0V!Lxb0%3$1>Y*!T5he;_ zY}?1Q>uy&)M`Z;+b)?CuUkiC}U|&`^h+@Fs7$&xuzl&_P9C{Z5^Yk5<2;ZjPL>H*Mr+t+eC)JsK{@p? zhJWm0Dep-9c%g-4%9mhBdh098D83tS&q8;z_p<4eB!o#v8gppx&7z~N$kj#NWLvKE zFN;enDQ7*Bz}T!$l~9ik8=PNGzZOzUj^gDo6?3Vz4%eh6BAx!+zsvc@-`!kjGi;Wd zG)*9P58pUU;zvG^iFUj7(T^O|CC}Fp0`nhG&r3*2pv$wXU=?vNp-6W$AI3#&a-eeBT#h-14Q zDppkvwKf)s77EdKqc_qY5cWKLs?tzX%zXJ`Z+q;tmur+{)%hcm%D=6p4lC-_%}F2> zoF1tAo-11782zDT0W~$WtU}ZuAQG)RVHe8a0`2j`r^52`AY^`cH0F46V(w(1vic+D z_POT&VN%7f7yeIH#26Q{Y_Vq564I0sh)|zeE>u0j88FNmjycARhK6EaquOxf_tzlw zD}YYJ-_iyN@^GHL37mIS=~9+XY({P~kf)Q%PJ&l>=^kaiXCLA3&u6V-YGX^R?3hf) zOGbx5X{^G=&X}%_a81MU3@1E!oCd3v%*0a5D1(beX8%oi zHboSPKN`Pue)JwyyHLrnaMO9vh!Xv37#e<%mr|ge8CS3(J?6@Teqg~pYLwsnb&D~x zrD{@u>r=~R{YkLR^vYxI-&U5x9+vrEwPefgaNXQ<(IS>yIsaNgveuggK1lT2-6FG* zcvas3Yh*x_=>Nn+=Jz89qX{@hHC>1b7@KpYttSzKwH;G7y@pO5OzE&Dp>ntjth=ckuAjv_RHWq&{rRxM zK4GtjziC#m4j_we-$uY?Zc_Lr$&1UxfcFz%HwpDYNByGmgLSY77IE&qFqjFDZNI*@ z`udfVFaqLpeeFWFOad>GQH|W~Ok>}^>M)?k(bT4(<45T{U?>}Z)<+*$`&%NaSD)cM zlz)T4K2E&ZDm_Q8+7Y)U=>w&hHjcgDZPfZmVyZI^x8=--x%8-BrPF|kEdU_;8DY1z zFvH_fuUpO`+6lyfAJK`z{UB;?rnL5F^tI!L{0lVl()`ryf}kKiHSgUa;q$p@^mn^*5KB7rYG;)Z_i0!Sb2aI~}HnU#l*l?VFZt|TzJ z;Aaera|_4+LArd*s}Go_a>#MjZA(`dj91p)8C#16EF1ht1-PkQ_9<1U?PR&4C_I=P zk(=db!VbJTSNQlw%_6sZ(8mX7KBu*VS~;99DPQyMew+NoA8G?VzE_zT@b^DIDtDQc zVvPdBg8yLwe)b`-`RIe%k@qVN9@ci;z%xVCwG{^c2<+ z7jTnurHoqb9I@e~S40~X;#|>fZm!9#6;3g#k%(NdMiKJ&Q>(P-kfK8-jCU=ba&d3* ziFjt{U-+?~_~%S2DJ`FL&D1lyboH?lGsvb8HhFmrsG!E*xumj%uxRbK;kL2G z#P;_HXgvgGkz4vIH_-azy4}DF_laJMQo*8#zhn#8I5`W;@nbi&2s@AQOEUbC_v=Fe z$_taZ`seq*Ze)Y)5}LzhIdAiyQ3K$5!;FPCg2xFq|C$EvQ~sps=JM#0416+GBBSyU zUJ*{Uh}FyRHNQ(3&x0#AF@gooho7&sPKawf(puQv#mUJYl-zrCURhiZ^7H%oW-X1{ z&^UFT!n)^Q=mD6G#vtZA2I}_{aZGl$JnTiIL1z!N4Lk&UlASzl&2;@Q6fC1yEHf{N z2GF3kEODga@ zB-gH04*(x3hy?W3z^(7ARrsr~Q%BWwbi9BFmXZQ8W<$eGFdET@jNzB{n2oTWwBy=> zFY@=_$JNzWqEHIwjRNRO%*obcl;o;@#=xLR~&)VySK*S-mAz#*J zuY4uoZ59~>Rw9*o{c=tq6Jj(nw5H9u6OWGpnE+kK3|sxz&GN3kN3}+%!r_ z+QLYg+Uq%KLHHVeL8B4PeIjH!EmiY#)?;T1k%WZF=^5LaM+>%}jlma?Xf8CKw!c7o zMzRR~TLN^L5>1SZ+CaYpH23}8q%hzY-fb^+uo0qVl*VP2fTbUQ{;K+ME-v`?%jWgD zT1mzZoDzowrLtbnSyB0%x9)S2d#5jAa|(;1E+;=(9)Oop zX zw3Mn5@7^XI3-M%>H#b#iTy@D$N$gN5!xIrX{GJ{_15_)wqE!*qqQa0RxA%Qqz+0CK z)G)S?Gr9qPw~eRzTAiu~9yl^u+%^o{YUwvuyZlsGUuzP`$|)hN%Yu~ZA#^5UYj1*{*Y z;EP7MtDdpm$4<~z;(C5LorwC94}9QYPGivo3_o*(aT^#r6L{eo^uAq~;%e*hQn|!e zA6K`cB-gtd&u2T3Bt7T@?t$+-9pm~e{Do5X{N=Bo-rx5h&3>e&N8xM?<(BxBLV6<%IrB2->n1ABOBfMXlAiokL zbY-CF`JP@n;-NU_4;6kj+Q99#YNcsM#X!H9eoijw|F7n2p~zEQF0oX+cBh4P5@;vK z>nQAree09M!TT19f>9DCn(C32l94dB$|ceHaGMMFygpIeO{tJXCOW^sujegPcTAn+ zPcHAttUj^_GKdvvWX1Q~IO%@YotpcAMdrW#aNpF8`hw%UYzwx5^5-(bB&~ z-^1BnotwZcz1>c{&62`jjzE!#WxjMAUwuMh-F=O8^;xYH#(&D#4ql9JDI~{U+>Eum zxmXzwS41&=j7Gtab05SqNk%YSFj2V+iUkbi|RIAc$kK=sS_9R;OA*-%B`yOvS zrLw2f;UCl0@D3t=H!TR7V_%P@%AfsMN=GFxtD0jbj>zO%xN&9C7En{)uoM;}QPb^_ z=ySo`+zptS`SwC#W3JeD?O#Mt&PXFXI1wvC{(UqBt1CJa6u{P2R=`pNFuy>i3Jwgs z*g?hMcv0nN^UH9M8y}rO{s^FG&cjBss%L>O09h}aZjAi zu0Sp&Ta`g|=wnYF@8ZDhZm~5xM0n@YF}dqG`Pda~pT9=#Yfc(Uz}N=}RDmgJTuAqw zYXK}0Psu@vY#ITuVQ*kzFv#pYu0&JU`O3bi>HuS>DUAOSA zH65i~9MP%XzAws?pTvLfdE}ADv41F-`kDsh>4w zM>X}KvGf$6YAuBrVT>$2g!D$}_D?9Fa692<7Oy(UG=yzv*D9*ZZh)0R6ZAkVg+!Ap ztOVL0HjX#2U0MtqiH!|E1H`2^{_#N|wmXUfe{bJo+Ze>P`t%&XY7-lw$;p#e^%zSh zbd-Bf$o~43U^4jnl@v+uziLz)${6OEq*eee;t?1N)z`1mGX!*}yv3MCQo!(H(L6`{ z3Fl^z3Nmwezh#OzV??^+j{~rJtA7*!tR$VK3%?X;kr!FB;8P$izBl@_uo6WHcgrRQ!5sKiHui``flm&r)k-yuj~i|90|{?_aOcLA@zQBMHW z`0aMU=zhtB6#`n(O}`&68%U9$B2A6mVc~6z!uA3QOi(Vpn*W7F&5O7sM9ZnLcS$11zlM+GH(L?}mI(2#8Rv zX30sZs;ND@ssc}=%coV~?E==hjI=a<_#Mat2dWeVb*;@WJ<{*t=#8KwOTi&f*r$)u z6GV)jPCV{h9CUsslOvTe#T^sZ&u9i${gr~=OBG7P>g~xvObcr9ldK?|BvFE4nIE}V zbL{k3e#TAZuu#agqv;8ZSmyB$CE_Q?f!!lGW-Qsx>GQSa<#xoR-1e}jjLkECKe@4y zRxyWt4PCgTe+!l9;7T$ZVjUeCdj5zNOItxDjIiXv44o6LMVI2SGJ=YXvh zNWFn?1H=cM+6@4ZSq~;7op<-m(htS!eD}N;__Mg1|7WCdROt=q;>@sElLdK^5E`yE zF_w|6Y{y_iW|5Et58`^O`y8^x8|t3N3Z1KF-f1N8NSx0HOk{4*?fG|NXiW&}O>Di{ zt^eyC#qx&CD>DY_U}5y0CGsdT;Wod75))j$2O;}v0d^7k?q3Dw}}T= zE^|e>&zpa1n;1R0DwV;xyt0%*0Zr%a;j$m|5$E%u_-{7L?~kvgMTKo>2eR2XUTt#y z2wu?3*#XnA-jA(adf&prQi02yrDf?|183?eAcH{{i$k84ni?>QROFqhxLCk@tFX9u zXl(3qca?(79eAVyW}NKt1gSfMkQEM!nbtsh(Hlc0l>xXr;CKUp_>lujZa3Asy-4k< z$#@*SuC5Mk4V#S`%t%ToQ)ZrP>mK4S?`<3Lj##olNA4zW)HThqR@W)~Yn~oL~SnsxtRFr0P3%*D`fcaC? z+l=9iZdC2xNb3$SAVqXi1Py=rP?u!Y15f%5*5wmJS(;^^^tfR3DF4&3(}RfrDWN+uw@2Dz&A}Y&JeAW$9 ziYLcCk1?D&)()_tvuN-crwE2ReQD)9e>*(z)t{}t*xbX$m6F!52O-Z_ajc{ zu}dJt0E;wW6Fp`tUZ&P@yI%ruIeV`jP=$B_pzVBT{AtY}{ZI-Z79K#Fp^}OO?3_S` z;EMqhpw8?c9LUqe$Tx3XO5MhB7Sks&xW&tLEK2eVEKI^x-2+t=3@h<*z|Z~;#e@VY=J z`LkJ%BeBwGo14GYFV79q32Nj6Z?U%hV-C?oj>n4% z&(>&&-~D?8qhW4*XGcPFQ*K^%7!XillE7qQAr1k(1imE+{Dr@7ant<2 z7wN1{Vs6XK$l~v);}S49!jWOU%08>w5SEnKvSRei_saaMbr9s4=FS$GuraLX%xpSs z2u<)b)&z%qz+gY(xMI|eV_~;rU?j{buzwRPn0if34J`2}IXG}2(dp@Jz#1CZ#vLv< zor9pH(`b_&m-_VAxx?VoWf8F7c~%IXogretDE0H_@CMKyFvAi6r2E_f5^RLa!w@!; zPHW?T?HHGy+`$Au8*nhTDM94mMTX#fDh1;oF#bi^{Y=a4S&+Qpy#F+CBo=&QGaPxt4*x3Zt~`q@2N$QuBt z7hI`Fz^jroSq{*54ZpqIJv{g!!WaPng6-}W&7Dk2OXD|DINp*j*KPrhz^WfUJWm+{ ziV6|&vifrwpuK@Y0_@-U`T5_ya|OQ93YqK>hzp?HM8(ABfH59`z7rF{a6d5V;sQkH z?yf0wHy{CZ&CK@V6a@*eu+G5v24Ktg0jPNl;+=jHpekNLPCz%F*NGb#?k-ehJP*A5 z$|I||yFaq_W>1zYQ7a?lwn~#454!7VM!W=`ZklDYpg=`my+mThMhxpa`z6c#EXf*u zg7`@0x&ovV7|(KLRf$ZnkV9-%zl?Gx0Ur@J#qA}Nd*^iiG;1=n%LJnt$22f^u|A)= zU41WGk^}o!gSV(dec@e3`|RUK+n#MnvF!hJXW+-cL_H>n3oQile@J`Fu&maoYxDu> z5D*0f0THC7yFo#^MM6><>F!362I*9!q+1$9K)OL%0i{7&(lej^UGKTRbH4NM@M|lw zSAss!}zgPsUA>`8m6L7S;x9p?Wd> z5|ca>3i1@epoxOo4Ko9FV;wsH1(pPZac`lFPQC ztG`51Z1Kk7qAwZM7;YrLTQ-%ra53JTmWAP4*w@&SM89h{q+>+J08iK3V; z{hRD7$JOJgRa{$+c0I_yRO7x-R#*tG5}g{$QFsU3cIO*FP7HTzj`;88jEZ~+El(rqmYy8`Uo>AHMIc}URxj3dqWx*GEXd__XsxgeSNoEqE|=b>3p`Ejf``d z9NZv)x)VFxSM>Ft){#-RjE zwo-$PnuR9rIqk=u{X>VxdlV4L@boRm+27ba4O)IQ`EEgKZ2UJ0?`+MKS`zp2u$I&k z&VK4HET$Cx{GQzU*|?v8|GIw7(clL}Q3U3iTCa|`#`IxA|3mHP-R2TB$4QBh=_g}W z+^LS(2!s2kP|PJM?)N)PS}yI!tZp2c6D@eimnNUq*l&|H)DSDm<2 zTF|t^r$!(i&}L30kOiC&$165VXJe>mddWR*C(5(gdZ(rE3Km`ppjF`7?8zGSebllbOuC0Mw6(p{1 z2j5W;AU$il+OdDeJekI80|Hlv>&r9bTmX2;cFxX+u*-#rq4a8tPeD3%VnNKn&1=&O%#LPCb2^KlpisXj%C!6wHhQ`FbY;F+ z_v?uXWb5m+Wu^+MOdK+E($-SNxs5b~)>?s#B2Qn~WOnuUznCth zpiluiqbu?l*n!nOMy;h+tQ-G4$%eLvJU`rMDbH3S{Q5FRbn4IpBGf^_f*JNI3_N-3WCn6%Ep#kG*_>&KlLd681G?|Apg)OK>Kt2V2D2Vc# zn4U%orl%W+)A`FG(hfFYLGP0i+Q5Gh{0|RT5dP)mrB9_c9^%{osjGjImhZkZ8*2O= z`Ve0_J9qdYyZU^-!TIpu;0663^J<+^t!x3uFRi*LhzhVUcpm=VS!`*lsBi#fxn0;# zfuFDkUuU;m+-d&G6`#w34D{ zcz(*)ve73>tZJ3dOBm&!-mbjQ++zQ7Yum?P@e3_hb&ljL?2*m9Q(E_%oR_bU zTEef!aNI_o@7)XvklmxUc)hr>MRH@Z>EWG?BmO%wZjr(-%*N>7+4hYea0><5ZGMK( z2X(hTld~Gu;J{* z61k8|VYo%FX%&E|0mwH78<8TtGi2D}Ka%I420uj6W+s=z4~Y;}*b8ekh>Ei7H<|`6+#6(MD4fEYXoKAXV#n>%coE0Ckg)?h$AA!e zz?AfHf;e-c_+*NZ3Kh*nLVq|3cD{DR!c4_^Wi?BAnT4`GWnLeq%u6iwi^5fHr$rT+ z?AXAc3$J8C8n4E6G>5A=jZk!~qeh$f2zba9K8}_2DSEt5dPWgrwZf4Z?aCgN=#oxT z`XPp{^lxL1s;w$xZ}t4mI0OQ>u%zS?wo@>rK>8@CpH6nTU*@Gt-2dL-T%gI^0$Uyk z^%;~iK0E#DLn_6ThCf*hw;ki;(OXTkr56=XUsplhQc?m3R@CqsDJki+-?yF~Od8l* zLkOK<3w3uN9vq|;a#@F9_#!)7cxrH~>3{QCjeQh)3Yxw;vwjk5v%qkxiJV=fk?N|d zw{PA+0UQw%Lk$5}@TiZxe4jiCX=%9{&3$S=|Ctk%egM@#8y6_SVbqQoALrnoLq?xL zA@vw*=KFU6AtCa?<_#ke!T>MRPrQ}to+X+m1FNkrWKtbk&6aWZb?LTIU4{QfM00R; zQPO;5*f7y-th$iKgg2S7R{ChTvpclqUI9iE+a0Hkq|Lx2Ev2!c+I$o8ETbj!4J_Jm z973t1$49pJ@--_ocB{q`I1}Rx^@6`DQ_LFfO~3y7)A=@na`0#8YowERY)l=3H^I@? zoPTy#s<5W!oi4K>Ei?0{A#0PHuQj+S{{6e&`v-ErRwyLF-!A&#!#?*VlZWquiVXo# z+a`QwESgWnJ}oFf0ZfSBq!i)-FbH=XV$F@ney^>O-o1MUpfUZk*6hMU%McL(0YQkW zvU0SJl`8qBk>OKQQ_~kO<^hJt#86REQ^ydJ3pm_&#kRidIF-n*)la+UaDPb`h)ACyo|&$}rW~ zJN&wIktEm)Id+)k6(%!slf#i+->1Es6+Oit+Hd8K<)T@wqFZ$y;<@OrZv7>RFrNJO zMDmc+%qINgdw2XWpIF6xBVoP!3FbY8B}RGkW@aDBcGRAJZrqSM-|+e?5hH|%G~q1y zD&n4N%&Z@(rBeS1%fbQe-ip88XI5H0pS#koCZPVjAH4$jPQTtxKXYRQ>QT+?LvpYJ z@H{MQYx@U|27fSof~>Sgp7Kd;sJpv+Hn)Xx3ks0e6XJmGk-jttm;-Aj7=ys61z+~9 zXY};+zd>5b@AZCGX^%;fo|xw2sO5lsu!))3*z449-@ZYW8mo)pfB0JoBbbPaitRQI zxL)iR8l?%JWS$+;=NsGA85kJAWeI^S8=NW-PR?sJMrX5?q+f5@jroR%R;Fg&%?(lW zz&gvPMDzMvhUY=6Rx@MZWct(t0=}kSPB(Tkll3*f1+LY1lIGKMN zLC3|Vyv2%x>!iPGB^(`CJ8j!5l!c(0ChsZ63ZggXXuDEs+_mPW+s}*h!?&9ntezM* z`Lj0sJ-j}pLGaDTEB%)oE{m;=BMx7ByVQ$NMs^$KZtC%HbXulJfjP*{Q0WG*NLYnj zAm+lu&UE3X#v`w*cT5`Ks@+*xIov95Q!ky~{Bv|v!tI+z&Vp==7a6-*J0Hvzo*Z-O zY#oWdcM~JU3C%At`RyQJ-@M;VVc#4`=mqvbfMj)0GJ|UeWLZ(lhLcl`Eg4v^zZVx83o8#EJOJ|rh*Z@I0{A)covY!bE2*wN9LrO1 z-~Jv4VBLI>T*J`&zYfA+H%IYo<(7<1E( zeMdiRG*FdKD`_(X2wII&*)ocd{aV!&>>@S-8mo%*U%TMldQ8@yk6osnh-h2 zXk)zKv6K`9wd>o%-z+;d(D*-fO{}h~8yXl8ff%{s0@aB2tE($Lp0h9kVd01;AD{vu zWjC1cT!#vp3>y7V84*dru?FK;%#}I?Lsa3|gg=UcAt*#uUjFSUZp-ZS4(MFh%Uv(v zb!Efrl!+qG8{YuMAX`GO^=|=H{Tl$hzy!1X3sB5&eo?sZu5NBJSe&T?0Jl+SY5*I- zYLW2~&gaez5C`%HY`06^Vx?i%3c$GH`rT@nE zWNq@(e?EY(5OH+f8*S)LN<1R-^}Adc_SVvkq`(pWvgn1`pKmGfj9>cH@zf~Xi!tM$ zvJeidJq|=s%r#$XdH8)F;*f-^Ss6EW=28rNQmP!xZLw4`C^(|1fw7yY1JS zNL1**v*d(8&L+7qC0xO|_0okwkfYUp_J#WwmGV#I)pw3SiP*R}$!Dd!pi1)tJq3M& zl{~FiOXWk6r~qSaMa4*ia@b;SPL^iICncT0B#_0=j=^dFhD|C#rhW>t_9-6Lr&xFK z!ajU>5TY7GB^qh44p|SmNzoc0bpv=DMu=pNlvHwwKr>4u1;x&jS`r%#@f}z<8!nA$ zKi$X2EnVs!zrLBtW%kyfJSIHcYQFyE1Md70NSy%BJ2Z>P1nz9)9bFgU&H3iwt{Kq~e~ zT$(J>MX{##0DrUUThFyphU9c5Mh*HajJmest?u{(+h2)qb%kTkoCLQrw=r=y%Xj9) zIO{KJ+7+gqw8S5YKfazUCU~6kdguH5EJG{-Tdx3e4Q6et-tBiyDYLRMLSGgoNpI($ zzNT_TXh)~a*=8)&akh&0O|WCBGyHCxynRUGu(#L(uk{~~W`Kj!OkbFwG9ac`v|M)y zl&FA>1l;9{nL@`9mHqSQPl0Ga#@CmJ!x+KF3yrRei;FzZ3z8z>FSCT*hq}7*PU;+# z>H*9EV1t0hJODT_7~o`o9}EdgPh&&;bf|4HGql0~Yo8tw;;F1Ae2x<@T z`V)eAY-7X91)n#}m^qPIM8pS5I@paA<>VlTzW`Vo@J-w7zDS9ep;dLhfcZLVrHjXK z)jnd`jTfp-0sNv-G8Rq7oncViA;afk1;Rzx&GVbxZsrcWxf`nNuAKiSv2Ra{!XCpQ zcouUPzp@Y$)BIbo`H%Z}X{2uD+F4op=dX1QyUk`j=afpONPEQpZA<^+J$L2|X_&#g z>7X;Be=Gcbqmqe4de@f<0?M6hWA=t(&1X&mcrC4%H@3Qd)tvs5Lh13$*c+C&RSVoM zlQHZv))mHmthB*C`S#DUl$Hr9%Z1pW+b$MOh;c~M>wJ6vFAL*vQXsA|EGk}==>8;) zwN5HiFPbWcsPuLyIIM7BKnPT&Ig{2vm(@ef4S5vmm~bY-A1z$2onV81WorxI4(8@i z+_~cl9Gi?U@i8$@Xm4*XOb}{z*%$+hS5dH!m_Ix`h_CQ3WEZ?Q1Yion`oX(*4P;>P z?|DFo01VY}`>lQmz~Y+$VliS?EVIg}1Jzo-VZ(|isJhx2e)}F!tl-B?ZeD=6X+ISg$76(3bktJi+iFj-sm|}C zJh)qtq`0}#QtX9ulDH1HF$8dK)P#ssTP~%(cK(%?e6+8KF+9CrDv=y|i)E#HT4L_2 z1xFb6*{yJB5j=lRu@qL?@}oM_>$`Nl8%cL8-QkRe+w!AJ>9RQ4d(#)^Hjg{8n`EL< zQ4q7d{QUJoJHNKDZr;rI$H41^DSs_+QvkWB2A{RIwsv%MV^&>zW%K9R+%^h?@&(hXKqtm71})x7P;r1$=)NGczND4FV~^fRmP{tF68M z>ld$4dw@{qH$OkWqM{;5w(z+8>u6;KfeXdi+1aCejomqt!C7LyX@KVAQZvvX@9A-@wdQn1rG>+zPfe1FvI~CO`>Xm$C@`+mp1mqS zBzs;fS9_F@ymHk#G@06PQ7@|b#b%0gL+%~p7|SQ>xHp8(je-$U!ejD*@bh*yo6RYl zK}KTVj2Iqr^vQ3i&^3tYOT6I|cz1hZTmrXQ{y`j6^Lgqsv*9kc0EPJ5_6jO z6=VmOk~I9LB*ttXg%LUtqMFT$Vn3>vArO5$63p8Xu72gH8L?`Pt~zW@>vez2|E=^3NR)XJ&6npQmK$ zy${38zu&JAPVpRH&Rsx=mZboTfmHuHLZ5*8Yd|sy{4TfdZBzs1#MZ+h;i#tX;6p6j zK|`^8yQZsOqUk1;FZ;YmNxT^IwbEe1X=o#5jYwkIgX;C1@E ziMrs9UeMkodBme|N=T{FeqS-zPRQ6Y=k4fx{|1_CDy|eJW1lvTdMt0;iW;y~Tf>}dM zmi_y2#cJ%Lk*V41lr5_D#`a+=j0j^hc%zB%jO(<-9=^p(Bcl(t89c1eme0Zea%fn! z_m$h3iZ=e_3Z;`w%x^Wvc+!MIArBrZLYo{OJnx-ro%;TSPtR&vg8${SMDQ-^l0CYG zA+>OmHG&SmAu(C~u)d{)tICOjX4PDj@tbR9)1d5k^r*0yPSbPL#c{iP^B3zMW#dy1 zQxY74)ne`GV!P-sz4W5UBW48GU4-Zob?GJHm$e_{1Ynlt$QFn*X?&G83;#W_WA{p{ zMpnRp6)&ifQv06qhzPT2R>tXLPgg$VGxV12MsxQhC236m5Ys=8+0~ZJv^Rahn9Ou{ zWM_#)8M{VEG1quO!Hi+4!&FAzF`j)SEMx;!ii1VW@Ij>_a&Wi=OAsoRwlccpi+4=Q z-)=kNwGoskV+T*Fi<*CtKo>Y!fe4|J1ORSHPhUO3w^egDXOy zJL`x4pQB)RoL}LtRJWYwKX_fSE;rW7o8&w*l1FEGl$z;8szEUmI%@Q?78;jj%J>D~?uTwHD9kg#^j5V>7llYF2YYeAOlvz}U z%;hMKPh?n(&G{r4+^|LXx+y67$6%bNE;Lt1jBxscn^>;rBb4+`X0f>4t=e3_auCdwNc zE}`7O;2;}mFCs3QASPXXOU%`ECsf_SX<>K3L?#?fvfY8R;-W>lkS4f%Wq)Y2<1Ay)w4@Pblum;Djdi;=`}E ztR@!>6@&T9Ktw|J5kRY#mD;>LKxM;p6)%8TX|%h$yKPq|dN2?rJ&nn7t`B?pou;1@ zea!PT&c)rYCapJJg)MKGrjc5m3Gb5bOy6=kmld01YB$Sj8)v1$6*3QWqgTjX&@u2- z(9zFl2oa!t)1PTfOGUq`Cx8#Hp;>+mxmkm9W``p zy%u)AnYOY)or2p5+)`=jlejFc3el$3!oZ2~r?ZDw7oVF~w*HU|{m`c8N2HDBj~%ef z_t}uY(IWb)P`VqUc%5In?oG;2Cw)!A{qb4J$GH-3tCS&GL+mf3I;4COYdaA}x1jHg zP>t|J7m*|c@B|1br*-x4OuLkXE8FJ)`&kS46U)G*Jvg`nVsZ$Hf(c^C27h_^9z;H* zbGnYa{e-pLJh1I~aEo3{{r(#gTp6stik4z%x`}~wPAu#XTP|Bwwd&72DapDW>hIoN zuc{~>BRX?|K)M+80EQ)vJ+}3+uK%x|&7CWrm+~pM?g;z7E$f>wD`j}!i z+rim>)CVp!z^$(t)eo$wn#WW`MEfuu;QZoZ`lx2yk_RqL25aA=d9L36bba&&4Ubvh zJ4hFSD4;rMnB+|*6i zeePMlo*el_A4iGie;wHwBiAk39&yx$(50$W z&4kzQUna|-I_58y$JE*T2bby}7JxDX;d1-;oj(Bbq_IFC3I$ha@8w(rFk;LsERV53 z?h^TFk;&ND7!v-FlfIxq2T3wYuYxgW-!TUXuso+Kk6%#LN*7zSJAJ-Dg?vvbn4hVP z9n@Rgp`N(=X@ge!MLlBe%uktHKrznD=|;}47ix?w$1dL0aafeeed+Z9JutnqZ*@M( z;@!0P!N0a*=g&fIVzU*2Be|f&LvZv(qKZME5SS9I_nIN6NHsKRfKO}q-f!>MkLjqfyt%r`Q zD6hYb-7mBFkD09zvDJ(`aBdXr*x;4KiHf0E+E7_VvI;!KGArgCbQLVo?Uh$DEuc4H zzT^9`L-zyhWv2x9Q(4bp3>3VZ1Ajx#S9^&=2%}d(O5}4q3TAiO5a=shU0!aAOqA-u z_%GRbVBbSgTX%9dPqb4n)2e2V8uf!#)J%SQ#2MXR!s(uqq>zlA-HsQB(=3UJdgLh= zM?TPsv0Ek#G?!7Rlu>DZGlJlXNUZeM%Qw<|S4m6^JsFNKvI??aCY9#+Uo7P^1f7ET z7SVnWwpWO;#G-*&w$EQL_4oA^p19J=L>8`*E9RG$+CrKdNTAYVnJsm=zBnke#UwJ9 zF~8GwJ|AnLzYRqc?O&~t`UIaROa9;qu|u495u! z-*GfQ&Lw-0x*E`@jAosbK@f-K8H!C4A+4=wO2Y;>O zBn9Y#fN5+c00YK}X*Bm3&tWbZp%hsdKBdsn@I0NPDgHtC{-~5>jW$Q*aQ#4j2O{Oolhwnw{YB_{hFS&;QSX=wBV3=v_J7!o zm&FTylrPyc9W|JCpM)x*j{@hL9LETW70J>` zMqXNb@c`QoDCLNWi6KwaCK^N!AZMTnxxBifym!w)bRVQx6Mvf_uTSF;T`8A?tng}% zz5fn*LeE^Oqfcb4xB)BqKs}(78eX&_Uri@g!GAqYzaF2<1Y15;FyfB`)yXZbZ zdEKL=1TFwPF5a6HOxxxB7clZZbXLM{qEM|o3+zacqZ&w9Rb5@Ie{w@Xog8oN*IQ!s zF0%$HTc-uK?)}OKk^rF4Hr+<(pV=sI;hcUX-B#aXS*P`6)e|q+Kp8qjyGM%LYST@k zRxCst{S025D3~Q~mhXPV3;FQmi%Q0hz79QaV2^?Vkm9KX+(eS~Oq5b@)@Su8$jkP2 zvHt@U0l&>;P={%W&Ef&oUf*($FiV5A_u-k46?{|Ao z7^f`ezDB;15jKZ(NIVeG)+w9N^O8u4Jc^!?k&%UEYw|S=h;ase&VKy5)B zB27p}W+t_m596)yv@FL6W{4oz!W_epql?EebA2Ue1x20Qd(G5lf($H+NFtovF4V%g z4Nfw=@FzipFj)w)y=uodzR}$Z*M&TpvNA!b_%Mq^(s5k3=E7$4BNq!bZNvH$&bVd)tQ<4{d8s+pZ1>V-M@{K{ud7to{%$2#C-k&N1+%l?(n01r6ezYL(*67CDKI<R(E=LPFQPsaS8uo%jY=I$+#U3_oPv* zgYNY>>KQ6RVvBL|LJhUG!$Voas-66hFu5KN%oK@nX z<=1+`hkr9jth0qr256WmTo+g=q8r;Q`@C$FGGJc_|x2IAjbqbV|RVSJRcv$KNyHUD&Ymx*J;3vFM#O*1!?7{V_c$ic3FnR3HH zefl~xGZTC&iFbU$stfsd=-f^5Z>IdY8#yWk#k5g~YP;1=TO?wHrBFWF|8ghzODBf9 z3bobn>E6mgba|(qz>RLiBZd~E>B$1>DOj*IaEd6bZV+GG3ng#ei7hgu8X_wgxjiLl z@Zp<4hL~N@zYex3A`dl`t=&gcE3e+NN)2e+OcM<+gwj96_P!X^P0^w)eQ{T66L***7u>y#xEKOIg)%=auQD`Q&1Og zAKPIOuo_jh?DS=lVh{IM_z^_SI8;;J{!xV>Y1A{w z+w6_9!1w#i6wk2adHw9kX)VbB)=0zU>OX_hMJ5dO)4x*%qM!0P4i7i7L%urtmM_k> zUg5hPzq$($ASx+OUB&NUJ5=krGo4EDoSM2@PKil+lU_}fbh-^#W1($TJ+lq1&57MV z(tGeHtoJel9{*t6C^t7ZkrV;7*o%6*%=-FM&{iO2Z9smf@&19)>p-9DKwzCr?iqZ{ zJ2=vy&V%U2@3~J2LI~yJK~0Q|y=xHR-~h39$FTt;C@};D&)_`;s+dC1>0?yXU@<2g zEAq<9erE@H2?-iTMn-_&B7gsb{)>=+0IO~dBfK|!d?cM5{{H_!GU^K4BaziTVK-J< z+Ls$+c_$|)&>w4oo))|_K|w*ABH_vf7tl>0l0xdc2hmVfeclYe`QkPP|fT_ z=^7^L9NEZpur+kc(sX<|1IYo9Hn*LP@hV*5AbWr(lok;AMf9r4_nWE1qoiAwvYxJ5 zExyz`82SE1D*rx6=q2JyxwN^^#T4LU0gV&qBN>s0jadJ<8e+VhU~x9_2FNcP>$rV}D?fZn%*Tr1GAl?e*KtpIFlRBJvfpEm~g?rOh> zn)m5npKBP$dwsR%KYg#LtPI*OyD-bk`t&f1!}tsIfV+Qf# z5`TudW-ZWq&p%mq&=?c&I$DPfnO-Ta1~_a;vZW2#TnNkPGC}G|I)X9Zzkd(mZ{)6j z0Q%WXmq}w4X?_AJNW74?x_Xt*nG;?o=*&&L$n#%M$`F2q6Tue*RZx|)8n!lfQ0pY{ z76l~t|ANenHm_rANC335vXYh#1#1MrlI(0l7ndV2(4;S36Y+eRbG!8vgU^8V?QI;7 z3V&fsf{&Z?HJ3Z55hq)CGP5p+GtodgLpRw$N5(EFh$NArMUFa$_FdP3=5xo-W(O!d z2cMR=?p2N*qAGZW-a*Hz@i}+KH?h%|71z%;8mpoN#)h(;Qu=r;v`eNl%Zle%Ywvbx zV8$N-hLI57hS8}Ho+ov`^ZvdC{P7CZ_b>_(nJ^DpH{sCCFMwHT!`aVPj*X;Cr<`Te}4o5op=&gGdol#o_6} zbD0_*-a}5og~V}CpTyh~|CSp^tOE~L-Ge?n#h5;D&kj~7XA@1Rt~;kJ|gs) za6`X9FyucNtM{?Z_D0$QDrVCLs}%ASOnA5|{VEi)-`t!M{znTCBsL~VNVs0r zKIivfC_w7NF%z%M$(_eoOtszO^(eT7SJcMEb-T~WCYmN*n{gbVwEq~B1}J&B8E%A1 zipWvxJ342<=Qrol&&%TY<$39$v=gkMuFmDY`{L=-9*Brc@n+-w&j1eUG1=ltbxa*r z5?=FuJCBEL)z#H4EuvCydR7`58^>Nq(cM~T@lM8xj*J`_8fx{s@&ciWlM@&3v@f7% z&{Y7uYjIvvRzhimDY@Wz2DJ*PCLILsg^WP5@}l3x4y+#-dv6Q?2~wHiUl&F%{)$)Ez*<42sUK1--#5|M2H(s7pP>4VLF;J7E&>C$odAul7;kcqvB&hgZ zv_{EjMJgmVecsIJ>xl47)LZ@xZhiwTgs}I?D{GTWcwNaiA6YblFKX&KKJBA^(<>P)Y&8F- z76Wy4buF#mzkheH9TpvwhQ-7XNHH=oVbiR_RF>V{tn_qYP);YZJXc?sO;P*}_g8Vi z7EUeKXhqQO1LXv-S$=_9a3jCua99QQ@+Aa&!kr7E4Dl)W zCiIe!FAm7-AUw<)bZ9Zu;#tYb{Lr=qx{f<_gqt8>s8}&IF{y3Gc!TIm&ik0#&I>p6 zNpBwM$ZzUQPReL?1#A6W`7@PJHO_;gU;6rBCdq^127ZT&?(bk^31uuTDCIKg&8v_{ zusMtsLFbudLk#q*Z4XjE%Wk;)>sPJgicDGss1ZL^>qddiB{r6DqP^Q5)@ug%RUot$ zrfrWOp;GniKPQ-?df+z+J3pb6=JPYJ8_dkih5q-1Tr5HR3b@ee&N{ZMoifF{!;~X+?f~MkQK>=-u`Eb@1EQ^;l6yb0u zuhvI%oqrF2C&Li7ayRJn!@M~9hqwsK*RoH<$|Ds9m(HsW$irpNjh^oZ+@F@FisvNj z{=pGy-7?E1hwAJ(8+}|uc~R4RedQ{kIBFDg4E>ae_IEgLefnCuT=`m~Pm}hUcLfUe zr3`RgsqI&ji6MXVtGUtC@nnW;Asz<4+$ky6(-cDf){NKFArHmDtM=H{wKh9@3Ca`e zo9g2g@Y#>NMAHWiYOSBi$ze~fHHRf65c`HxP*FjQEy|~WDv?i*nwy#~Anbh;-@*C9 zIsT=kKWKxAO_wC-WnP9BkO0f`1960=&ss8>gP|3X}pf;#`H3Ogw0F@~O zz{_7pnH!+|1Je>s2q^I2ds2oi8XS0Y;8%cEV*mqM;d_!O1QG=w_zSDv66D=^Rb1$& zNFC1oeer}C(2gKh>Ez8IX2PjHeeVx?~yt#bf0g>Y9CuBmIw*TSYZoq@UgHE z6Uj76iIM(d5GYvv-n>0`eAT+3rA;r2kSO4ir}H7Q^m#OiTgrCiSNE192g5p|g-Tz~ zi^b!Uc?6^{Mn#n#s=h#eWk}rDuC-2t9Vv5TT&Cx3(TS_fDS=}SN+DZYiKmsI0mgLGreGumda57}-_lXIS`2SyR7<$+JQC^>;P9%Bn+TOXNFlJ^gB}m6YyJn)2#F?*QWhnT>&Y_VKu49N@#lN@@)<|WI<{Ng zX7Bw=*=_C= zEZmv-UJRV?wpc}{*pK@HZaHSJ-}SXW-mAN1v#tTIwVLL#S5v#Y`hxh8mE|sOuQoO* zX^y}gv+a%kthN;hEaYrO7~~-JrvF#iCFSI^T}VZNoRH(Pd1Xb#Cs^ocXf;Rms_YgP z7ICugAb-?aWgk-8O#m>!At8+g>8`FInB~P^Ba}iasQ^L%&LffnQ$6xMHLQo7C%0{C zCp{&sfdh|%*ZQ4u!Cb3vlOlb@n>RVp8pU_O?EL?f73R6&m~y4UTDl=+2^*zm@4V9h zSz!6bZ&FH1+bP};xw#NsOeh83KB)J&w_YG5F4RBBsthijd}dAJu-u@vE%m}^WIJ+9 z`C{6xoUoHp%`u@vHan~1VuIG;6+bB797Xzj95bG^uFO3LS`>PfwQ-BB2!xT&N{RGy zFR!vkjE$@sg#Jnedz~v~ZjN4qv$rVHLV)ZJ4V@HBiH&stE(mlqP|=QOL;ppR@GGn` z@P5HLwVk$7Z@;uyZ4r~0_-ruWjt4~Rv|I(RkxIY;Ks+$y$a(o|cp8a2lo;85=S5n@ zqM%updedbBP!|ce_nHrtm1wh^pf5+=P2zdVgEeV8K8Y-@A zg&OVlvo&%BSWV#>}zYV-E$nih#eY*{uz2=?}xV-I%7NKKs^EJd9 zZyxR6K)8_6S$XLC@s%3swH`s=IgEop{e4GSMAIx@u%kGF9Dlq}XM+hT)q5dBEr%02 z`-4MY0;@ML-3$@H*SGZ6{cCa4yZcp>ZHG$O&aRICJ)ZJp+Z} zL#zj!oQ|+jd$nAHa=7d+S{Z5W3lUDveem4CnOj>QYSRci&hBE%1=s{&8_X8*tcr;U;6KY}+GvTvKO&$oC>FBIi+i3)XY!DB7d+B=7nP%K+z!M9Mq{l6Qy z0N4m%39w8srNdZbca|smE_ySP$;U=R)zE62rUB(Bd6aq)&*>9{5LJt0rp}RqTQ7B9 z-vvgnR{O>>+g+;F#;;-1Z1uVmM19+1$fgUq;d*ps3ne{TQ>#VKX|1B2ZC@$t<7XJt zdGqo)FPXK^QRl3;#5Pe{D|+3Yb{Dc*Fwk>2XK$LS-?Px25nlJbaz&vr9X~PcFjaRu zpwyDV$>-%74y|yX2n&N|bS$>-N5vkWlXr<}Z`RtrPB6#!)4o`63G_XtG8pv0hNu#{ zZb%h}ROi_=m}v32sYwO!51=g=k5h1Sc4p=Z*PC?Y$w?a_* zansY^#z{y>0OO1&0A2VQ4a_s2Yd9mWp>k3+bP~vb#ST9x3QZ|w&iG&h;3lZ{KawPk zw>w_K#sVHal3HIOGcbZrymJEGDjLL?1ADU6Ia5A5H)5_ zN`9|?*Boq?=sq=AWSPIn`Fl!au=H0**{W-P;GssFBKy2&n`O3)y(wbk?GE7&uJZcv zSD<}B3X;5nM4aPiwi@&Y@$~|uH}vVGq(2bPB$l_7SuAr9zrSI!IQ%0^Je0^^DSTq< z&ljI&j%-)&qO9OM{Oo$dx~jN1gG6b+orQyme*Y z@lRGSPEn!e%oZEFKrifWj!qjvCkFw+JIJ>f92y!r+F1&_Ntp&v0nY!-%t7<&8>pyA z8tT=60XqdnI+U&OT!7^G@xv%&WTHsp#*G^`-8;FtxlJw`>K3McK#TzoQkswjrUwA> z$Zu1-I5{;s{FH(vV{L71Z2Uesc^Yo@CLZm@*;!tGKIoEHzzqe{cYwkNA)qE#J{gbY z$asNDLVP>`Y-rMSe)(dM>0%^L4i)6l!2vvx`cXOpi zZfa=1{6B?%v$ho0!mu`>o*|T~w_Cs=AmG0uhmBnmM5Jr9w6qj)d0k+&PQSikmHuCQ zA*{4o@8ZyMNWY=N;;(%%$j5Wh$QTqtOuVFSp>-$k9 z(^V@M=${8{=cP_>9-4?bMvt=Sjg90IcxgR}dH<+eS8*^xr(j$zZ%nKAeeM^-Rb9%( zY1LHQ3TM?JU8C+o0lDSj>`IMYu52+bH{%z3D+Txz65n~9gdT_v)hcW^)suT^&egY> zYi~98cmCJ4z#b@irKaS`IX5#?tEg_CW4>L7&TG>k8Sx>^)m2E}hG8r`g%gz;j*mxX z(6b28K^NhT3hDnfRYQWvac(qhZhH?$wdwuvtzj`g;X?Fu10Etd@~meDie3#xs&2_* zw>rt}rh-vlkhmm2aXSdzWoMS%>v2i%Ir1?*d;sG^K5G1XCqy+g8S%|Xg6`AluVF?O zW3!LzL6Q9bI!Os zH#1Z3qKjk?BYjxF0m1=v0yf0+zkgGlwA|cI06-8)L78Cp-VE>tX-vQJ2W zo!?FB?QzkSy3YAq4-!PKV})~BlA%2-ai(vsabNr9V6i!-62#UAmI?I8u^x&|xyE*J zp+1UMdlpVg`askFSlQUjVwCg`FbjQ(77UADBJnNmJ~7cZGSoMUI4yiB+RInf(%jMR z^DYmccZEb8*;E`bF`&)_t}9E#Q@vmu7!dt7zlE-@P_b5+XazfmZYfD;1z4(iXjBkF zL4BJHGA}_QSYL!{kniUQk_YclZ(c82;{#d-Rs zjtzAOC%0#wzA7vP)3RE*XvEPza@r31$;^pn4z>>rYVA!o@?X*r? z%+>O!sw93X;m}Xr5@(%u=5tQ+O3e>8g^&!b)`gK1*JF+xZyA$$5$up3F{V9|0s?V9 zDEXQrUsap(U!nzCm^tl!rO5PDJ}7|>Wes#Fb0*K3x0<{w?G1QIR;=P)z@rXO?t?3& z30e6ISP$Z)N&zY_LkRz|9YwoIB|W$&VSY?NKma`hwge{8Y9mLk1OK4O0Z08QXUQEm zwKDZz!1xca&KB?&uG31-+qGnxAG>7@8uOWxZYeXiXF5(vPUU{=pqc5pSyj2K+TzYi zmXA&SwZ*0GeUBX5^S!O%R-zs`6-j1ViihNMbg~blOuqd~ z^0PPji=X+Ba*7xU8u*UcY9OC?02d6>Z`l3Si!@nDVAn#Uf+k-cwOl=({c=JpMO zNb5eaKUnWc6Q`BMQgrMyx7P%GLr&{S7F|j~@w)LV|35FqEJSvwKh$yJDR>FT_EE8A0`OWRs-| zS1#}E>LCZUMF|0M?i}O28h@dGh)jP(ZqI9sC$zEnR(99*HecG;#D$X(r5%L5ysqJO|3T@6Vq<-lv;& znnT~dk--UsY$g0ox92mogSP1nxz%S2tXdzfu^aC6FFw1sdw-pE=MK0dOO~aquct(; zL|_++jh!1DcY9%2@2_2{>rYN)Ny^c3bJ(+w@liKkh$Q2Kd6>4z7v(|4B>9!;6SWl& z8nT9Yjs2IcL~D0u>UN4^pK^nqcb4a4xF!0QpNFQ~DDh1a-mu0${N2B6dGzdFer*Kf zfX;36pVvsytHhe=VoxrxfgovfyhjZUNoXNKR|B>u92^|1Ac)HAg1KB&pgzwZ$d9LlIpjCWVpzDH)z=igP{8dLoW7*fc zB}vM+YHVQrWY~~$J}b?SB+EbCACDwcI8daNqbHhup;h_*;|pVs%e_?2S7A{enF%*y zMu+;Onm@it^hi&Dd4xBAMLE>S|#q2cT>yS^*9oSbLC~KE^5q${MWKHbtg-R6Lbi9!|_d*?t^75}Ss; zZ~EUUVvE>z9&&AAr!RgTLVY*849Dy^uKOp~#G^QY;PH`2FZQNs!^4>w=DAt5A6@ff zY2QyH5*!kIY0G{qR80l>eHoJEwY}NSt``pcwY>h(aU)75nD87DR%|}Ghs3HfLS0%A6@@Rw`0!!gj0RfHD38}h z-qZfg(nTUEYu~P}ZL-|Cf?D>_s*l_{kYa@2e7J0LzPS*+)IYp$n|AyMtQ=yjq7$fs zLD2~IN9H|-rM@5C;-Zoh9qbG^e|eUDOT>S+hxJ~sZ!^0uaz4b`l}xf?H*ep6(c1Ng z#qaT_Jd@scB5Es}?1yR8-nY_NP1)8f>BBL*D|O-xtwAiB7D?CZpXy(7Y8|wYPpZH~ z2Ib%SXm+&&G}2$J(S$$Y0+V9M2vlb2EZm5+U9}C)#Ig+f9MvrCPv0uE#U8;EqE7Cj zlK0cncLclXLo78SZ& zgmQ$P*Y^Cd;O6t+CI7mzSWL26NyBts_RyoS%%?8TS{6m$wyXB`dGJ1zHzWI+5CuJ zx=Q=N_cZHi=KPVrA(n$)k@fmJzT)ARroKjEh5VXP*ZEr;Dhn{>uK%p=}JjbGyV9CCNO;Hp4Mx2RWABj*U;GI3=u9{ zTXqk};}3Hfb_+Jyr2ILboo@U)A{U^0xPmy!HKY~3MBvW!x~J%*9uX7D;x(CU{n6mx z8^3wX?!rz>op(@gpa5mCvW?vcP)3)4WwwCAlyTGrQf z8E>8GtUmsAyo~|6f|DxWt4Y6u{7VPbi_0?vE{B5vUSb^iq9w5~Uzj7OqHT$5;pa!& z)CK#2<#Z>GhVYm5j zTK;Hb%G|;;k@k2d-e_Gz$MQemNXnGA;|MF8N{;|fh z(ELQQ;Qbcm*FHZ9;+)eXdQUBxY>URPX{`a-uO8Wz+OBdu<7J^u%=E$3AlJz=A=KM~ zv;r{F(Bsn3Nm@q6`pe6>PvOa3P!ye1P!7JF8FRt-zo_~PpsKofe;h_YrIZ$=OH#U9 zknR+uTe`bJq`Mnw0qO4UZt3pshW|Rxz4!OdyJsAo5zg7M_WIT*wkphd$n_&1Db>H1uRmgU_`ABd>ju+iU_-9JB~h~hI57;Ta$Zbt0E-ot z2nCSn-h#G!qMI;OGRZ#&cz^7(`iz;60kwK_qSue*0MBanNhCsOGi0Es;7%RTq8R5* zM}(*70XjP1NcD};a-kae&jIipG@TQ${}ZL?lwPM1dc9I2T+Aow~W7 zrjuwaL4__X%@Qa@y@`!IKRLbi^thg_@I5WyUhS;L$tIqGw&5GzsM|=3TMD4&SVHp% z2DXp6TuogcSy?+2-Po|R9V9j`_%?L}?6*E`wWB1HmTR4Cj)iinDp46oySrG<0&g?0unU*j1AAYgj+w#O8 z;XVhP=bXt7%BU0qWDXk-GJWG8$MOoODkeOY2)v?qhlB2kK?xZ8u(~KgfyY{0rRfg- z*?VUnLoF;vLXOO3#J1*e!qUl><_^U&b^Ic&>I*L3z#Pbb09`39r7whH6{@Z;UjU}= zjTz|?v3p%=QTo!zKwQFBu%2Hc*}3i-dN)aiW8HmoP5L!@;~Wy`phVM4kD(~zX&jOq z@96V?ya$Co6x4^YJPKr(F^yaE-WE&7;~<`LkrL@C8Ia@XOay$}GJMr_#=eI+qDnA*7i zRB-WvrBoMW+ZjJgkOiW_{|~y9P4}eP(Pyygy>$}dDpPAU`hTGVpBdp6Fb3$g7IR*g zL%%vCf5!z~-Ts>iJK5%ySpl#Q@lto}m-n5T5S>45g4V>k=W0=QyZ0WEla7YPWj9f; zbIg$_r7)?Te`67_{F@vpIj_{KW;5iYV&5E<)AzmpL^Wwbld{U$|Io_;<^gW4kgxqs z%9!oFb8+YH`C!%n^TlpFa&zQ-;HPrM=ju1aUA<>;G|P>S(bjyWHnL&>5Q~YWfLODo zyu^ZYXvE9$AAt-G>A}vnLP8}z^=2cxLbaX?nYhbPqCt}6Ya?CdnhPsk1{;ZH?y{1a zjc`FMIzV(&1n`P~Rx>vv?HW#ud-wUPWS`|Ge1U5)irz+tC0plxLpKK;^pGpT ze)nVDp`X+f8gZAtIOH=My2SpvRte2JgGt;O+l4MU;D9oz{QN(ZbC6mAIv?b@ezCw7 zU&+spp`xd^I^mCe&n91aL^*Sizq!pTH@g(|{_C!X+X5hx54p18XS&Nhs;jk!Q!Bss zAPX;d@4>xhw|`9;d)$4KOEr)%X7aHikIE2XL5m@sY=MaY67W8y6jpo*d+ z=c$-U?&I}-OqE|wah;cCVK09ZMFvJ<;xcV^cA=MlI&IRN=rxVK6l*HjUq2dJnn=4d z!c2;jp@oTLE6udp!5TN@H68i8XK`ViSczf3L2_wNhgZ1;=!Y) z8qcDcRiUwGQH4WMTFjl&H+i4R&f`RU&wykEoXf~KR(A!5?63oz`_D3E9&Y!1+J;hr zj^Q+C&aqam56}N5wpmKKM$6b|*a;F*i~NG*M%xgtWeY9wlAB5WV57LLk0ZDY?NhZ{ zVp8Ld?k~H-WYj}0d_6|@!deCrK*&Yx?9p)V8NXV>I6o2ASaf~%bSbB=cJj7rWO;RnEDJ{BDt^y;BD(BCH=JGdJ79H#$TSBy9H1X$9sEwva7Emq5eC6c5%iB&G`!nmS{VvAz{C?-I0mlN$r@r;CaO@uo z)}af}IePl0{5OAoO*(YZSeJrj1(0c-XgIw6f-I@&*d^}}=q&fwnmi}`0mReG<@M9% zY_-+Z!XhFB+>YjlKOO*#@C=B60YGiUlhW!04CE^Iv6DFM{N#$$KE}2!a83rz+zhDK zaO~TP|F(9p?VG<0bki*0rwB~#_Y|;1vZ_c^xp!8C)}}}^41&}BGxJLu)*3lz@5YmN zMr{vv*(OsWql?i<2%cwrUJ9REB8~4?P9wGsHal&4e|m3kSA(Z5xVszStZP_~1?ETm zR%F}7e#LPNu?kVMY94w92;r{2z$wrVfB+W7GMoX5^ZeY_PYz6xzz_?}NE8$mIm0XG z%sXT=^ng-?rHKh}a8{s>0{}s=u!4eu!1N#2IlZBwVR?BOK;-`@b$(C*z_L~_7y(Et zxv|xwq$f2sH5?oq!LMJz1VVFe-U9$N!bD;~_E|X>j>qY%t12NOVQpR3rAXERWMVDN z%mznBfF%{Ej4LXb07{0O+-Qs;eB(VKUnq(iU=WBuCyve^UHke%L&SCl+FVh<$dN6@ zVoU&Q<=t$+6%92tkevmJ4(LuVuM4?gh^&mIt25tt)+8Qcy`O!#YVBouKg0Vet+{A# zwSK&{k#y%aYj3npl~)Vnb<({$kmZ1lvfDCG|C#Wd$$-@Z#f~O)SE@)7jZYeE{77dc zPquHS+*nhvvLvXS5jn67?<<*P@jUH%P?$pO_7=*!|5MqY=gyFd8_y=p#xhC6lS^N} zEl~FwE5Sbk#rPXA6D0KmbL7~JjF*4<5yXB3UXN!$Q!zRktv3-E9kTKBzX0t89zYG}E*T-Q2&0m$dq(XY{)((Fs%?6sAYro6yp48o2D z2T-cs<2PnY0R-9ei;Knv&t>=hfyTA_GM{wN=>l@V%g!5NPqmoj&_iu7MD)pdso% zVDUi%W?P}Kh)W3jPv5}o6IaM4|Kib`)IA(FVXW_mj{&9*Iggt98O|g0hOsnPLq-=s z->@3!*~sr`J~5wWQL+I*U`-8>E^APLe*uorG*nbBz+E0-@4X%tRz!q^kl4>5K#<*C zRUnoIAkIL%7U0?}>$l<|Py~PpZa65B1WGo5QVKS6LO^K(FcvVEyWu~}OR8DThjBXz_%L3Mi&6!vmFo$EM`~el1V-=#?xzB|3iSY{ zx<{Cqn`;Qb-sCYu|RZvxR30Nlp01D8S7o-t9Oj!(g09Ek-RMg4I$o{I9>+N1P zb^`nom=rd60X@Qu%dt$tMkYcYXNW|2^F}Mc{14bH0y+UuP7=ZJKN!DW0NBE>$h9$NFNu9v2dJ6?xE-g%K89dldEz6`oo@nbpDZ2B z5~u-t0r-4!QB#`&EDRbcHy0N~gBZx)_>C#b@`5SXUw{(_M(Sojn;wYc&d$h1H3PQ-ug61-+DMo4bj@+dpzv~xNy}fV|h8bNXlTyf68(h)|0im@!tpo@2W&ebNq-umanqoekqd9^fgha`QQ)MZj&X?k_(4r+yQJHc*a^o zC%I;fTn56A{vBybUiNJ$Z1meVGK3wCjj?VmQ%%p%9?lu|*K~fO^CjvQEpDVKQ807B zBgH$@o@)tWw-9d#*dS+MF1BFVXt6qPuEG&N1{&25OG`_$8g|0~YX@*k;PnD-0FS^U zJXA8$Y0)Nt)J+tQ3M1I8r2Y*tm~-#+fvzG)IB>Q{Af6gKy?7o5x?On0Re)#VW7mPq zdpQP{b3IAW@!v;?AIAA1)piNT&vi$raao?`cpBT zh@zZ`h~5fDSx00Nd3Z&gA6uur-xe}Y{Ae3PvolUjS}Nl?xc80Y>0 zqRYh;0It}qcF;V_-g>BQ#ikb@D13p9ZW$O8fSn(>Wr+KY0Ybw*n5k28bGxm+ z>|eP8>}o9T(^Q4bV}Vg*zX1djBPTbum4KYiWh*7YEz$nV-`|ua8DthfwZF8uc$o1^ z(KO%Hlp;jwUo+?k49yAbH(MT#zrtZ+VKownw1SaQT=vR(Uw_Vg+x7__ z!SyB7WV8y`@$wROF4*>PA;0pMSjqFZL^=x6^I4nwEe^Jkq#u7h=fdn-XgFT%)7I3r z3?NM9fP?LA5q`A21ooEFEg7$Oh$GW~w03u?E9dbNsV!0aN1m#4~v*X_+s~x7+_*&t4|Pgj(aLV z2{)Bw+Hgy`yh!FD7?o2u#X?S_a!P1(!3iRq;s@zTutfnfktQIHw*rR;$f&$#!P4z^ zEdba_ECcv7pfgJF)MjNt6X(cL-bbLop6&3zxB%&2Kn}YFK>{zPL7(wYQxj?T^q&1! zR^z#Akn+vMFC|DV3Gbc;NOd6e+bCh)PuzD_7=vvWmr4Wjl zY)v`_RiwO1desI)LS;S{kcq}HA56p37(Kf)nqfx4>|8U(=TZ;;SFRd0;y#2Z_g8I+I3Lqh5C~U<&}G zsI+RC0$%PcpuYh?^X~wVP6P0w(Mamv?FxjF%g%lSl9C57X#vs!;LrkF4ggg`bkji! z5bu$hxpj|+2+PIIEsEj)Gb2M%ojd`^L_k<&4>$bheWc-3HxL8+H*Y3bJWf{ySPY(; zLUcn})bM&CV0;kMnt_roGSBH3;L0HM<9B*`8zF#1g3z_$LO_gxhX=%srswBf!MOvl z$g=yz@KY!FvKQc8H8VX;?C0*`Aq%`c09X^GhrlCXDDFH0O02RBA9MJnz}RJBU|_25 zTeN3^Zf)+brLAl@Cs}i{5(GI}rHp>@9E!j_HNuo(qjFVp?hiyi+xDQOC;p8xU+VP? zdDxqvDr@ z2~0-lXJ%%OS6W|wBJ=$NTRcFK12)!Y^|)eS6D^0>e1GqHIjM0U*kQfo`x@oHrGA_c*eD@fJu@?W6L&A@6FqFwuxqd_4h#-LGQ$=RH(}v!@Dh1K$P8zq zG27$X_Msvhi1bZu7mt7jbJB1?*Q_;q@d%QVmxq-XKojF-Wd&zH7aj$|iMytzCMbwH zk@@kkvHvYDHi5kwNIHW=<)Hm7Vd`Q=hsEJh&K8e_9>c3_czyx=9~7}#D=aDK-6s!2 z!XMO;M<39;G^$ObB9dblFqy--hA9S8UKE#}i7Ec&h~87@=ZY@Ipt*(!3Dd2TWEKDV z%iYWDh}TKU&&$pGs$f;~O0Ef$d6w!BA9kHlaL`OoRk+N|Vu8LV32aKqdnA#8@?2U) zR`RnS=a+0A_vPp={BLUhbX3uJSRWPM|Y15-NPSNZbAqg0BZuIwnIM93c>;GCODQrXamh89>9MAP6#w9Z6I5=mV-VqPCBN9 z-T}|^X%7RT8!#xIr$I{hc-9M?F`kq!=^76!0A3eT2!K&FFzo@2eNtC;5u9TEl}3Om z1VkjD_W06;F8dcyI~c5?8wFhKp#eUKz+X2yCWg>;m$+8})|cp8iN@UY^c6@uAkHu0 zuq~kSuBe#k?cG84y8XGK`5qY=!u(_ht`{Kq0>6cKw3l8%YeUWtwS4i2Pibu8e(u&1 z*`w9B!{rTfZ|A?_v#Appn>SnAXqRIryK<74ryJ?b_6BE-joFp`y&JwS;mtzCUD^@) z8-c9Kc#ON0Pc*cc|G2MMaf~M@_66>%SWu*IR^sbBsc&?)8_l>xvO>JQ%Kp&Reb0|> zdtAO#j8dl)dhC@L1i@TPE%y_sU&vy?J;RqI+OCOd^N+W9vhx~QFhX*A!%LWE2!*^ zgaL&X!3C{f)mJ;E3bEhd)Y@)G2@K6{#sU7uWh0Co!pS>6wgjm?#K;1q3%Ie#??J+F zGY5)ofNugM%7LMwL;mIQ@p7P<&hk4f$?tid;Uii0L3}-yx1yb?XEKhvHfNrs-N7qZ!>LfWS|WIWIwYgKVV?g z(XaQnsh{QHx4=}~5W;_+agm#JP-3Q%4_j*CyGXtW!}7O(%|BXJw2%8fiNf?dD~&33 z^2>!~|6@eaw`m6tnyBWOIz6L4=>w+gQt<~pk=5$R<_LVF>OU_h7NXal<2LcihYQ&p zx1Bjcul@Bovi5lXd`xWSRvnp?=5}`ha+mZ+D<+K*1*oGBh`s3lcJk~>_x%*n<{HJu zr9@;_mT)av>)Q1zop5KXv)8eHRyg^g(5Swje!8<>(kA~z&5&A>ojd}Aqr2Pfu4{nB z45c&0BJaLv5$|`ky?anBPg88u%0yw#&`9fs1*;2;-X!Bci%Uqjf|mn;cV7TUnw^d9 z8CcLn4uetv$k>9Lnn45yUd=1;EJo6K)fE*lKzDU(a}yiMAJ`P;w?3u#0iWZ_xzBJL z?!DHazr`ZmdfI^q-u4HOeE{96mZ$3lQ1jK+)Ih9a>|-%gxSdJ>M+rbV0q0W@G)3m- zzH4eiIK(yp$$=9l(h$)Mfxd&T2xNl+R$d@y-rn2<9UL+WiWS>I+L8c9gm6RDa5R`F zt8;+W6_2*Z!>h7{q^DT4;TvF$F!}3({t6W>E&h`1CPGBI&khF=^R_nb7Y)ur5eyGI z+s!}se<~uqH#9H+)59>yg*UR`VGrX3NaHsKEAKluGIb4pcr`C{)}F^EV*UIs9ku%{ zj(RwG7gdh~-%Ll9N>!8Q_<4NXQ8Yn;Jgi~YV$#QI0R#6QMyTKaUhtQh4^w2Z@OASk z;MY)(k>HVfXCxdGkQ!0^Sq~i z%!bx$RkHxMs;aZ;TkoPy`?=0{P{#&lq>aJ3bVi-T(x($zPX4fMvBGx4nl+c6Bg&b0bJN0{R^AzMyvqbcHp+ zo)4m<=pzggJsaCKh}v~mGwLf4f2U=a{FhrmrmnGE+P}2|`vC}yW}waokb2M^`|*NZ z{0$TupmGwGY=sq%PGPrtUOMq~x|-&{`3+=YAAtG9{SRGDczq18C4*ijpp${Z0uYVy z-g>yZ1J;h+;Xg({NKN@B{J&S28<5A;t(w6$TnuEU0n@h3oVjWIY!}wE&JhvxEa5T% zxuW-ljUDjs{f}9!@k~skO*l5EVYz3V=Ea#$GlKe*GL;%mS?9A4_gLBACAks)@!4uQ zI8x8V%|Q5d*D~iB0;#yh5nd01yZ3X zI{F#`=#!73aT0r&#~!S~zXyjoXj$QyKkgBR=a&ilQ7Y>DA|vQl*w$|(QeP{AYWf^R}i@GfyjiUq$Ci02f7TF4SRV49w!>`q=076&+jF) zkTDCwE@0^c+WcEWIfR{fb<1&MmZ+#Gz|;QO(}?CD(g`y1+3|5kfrS{db{L-uKCKx`1Tj2 zx9!kB0|TB{)1C7I<+X?6N>SL7(vc|7RD9_i6C)D0MV?o+$}5oio9K)>Vy0;5Rg6A` zQ#SQGEL*R(1{i+8+h;vP{#;@s%cI!!hkmP)RD6wk+Rx>rhLpzxWR-mnSMF{t* z`w;#p#?6h_upWUW%NDtJ!1$Z(`nyniutO1cS%O@w#RKS2`4Ry=;*qH4?2HoZ=}64D za4rPL#-+W2)RCbLj$?SQ2Mvlo`)@!fZRQYlchjZf|F8gFW1=}BF|8XqtGDtm8jdBk z6=y#?e>~64QCz%E?!qjHGW#&%%%YmWv`dWC*>oIsW#Bta@}Q12>~Wed>nE=9Xx%g(H<;-NKm)bp$w__uiBJE=`R4bI z*nNpoN2);gO;ZvL4pY!JH&pz}FSa^W?m706B0il=^vupdeLwhM<#UPK!nO+wSKt89(uR5huC-DBX(d%E#PvA_ckdLuV) zEh&;AM&$|d%K8b5d9Xqx+eTXLph35rQs`&twzPbRCB(b8jP`CC{QNoe`Iq*+FbmqD>bEosoB> zR%z|Rkx}I$FEWFgetjZE)fEd=+gE2-=#}N=XRyQLAo=g^?t-G>J$3);>dsC8%Kko1 zlRH*q95q6k+T%K$%k@e#HY*ke;`}3y&?~xo1Rgp zy-Mi)(U4a0arf&l1H+F!{pMlJ`i%Hpn7q!(9A)(dfhhh}%obX+S@Ht=MzC5Gn!R+a zm)~l;VRH5qEb0%YOjC_KSHjMlbn}PbReRN1>e-iK!DD1^mKZ4XeroaRUkdzWtlXg|#vRpf z9+_n=v#YrK!x%>kyCH_zem_$4ktWrteXMX)l-X^!w$Q+IN*9|De}R0vwmOg74P*RO zlV=GcZY-~?)}f+w4{5XGj)k1juO))Lgq`ikHtX26zuL*g`HOemQulB^h`KcZK$zMH zYCZ~XKdRnr^jkphEGa0U0_{nkw*-;c@i8$MpkTtl!df_T{I%YjvP&LQP*D+`w+E8$ zt{;lxgv5GrX|$QbxfI-so}H1NSr-;@8l&#QcBx`R#(LD~MckdMTEvAP<4 zam_i%N*X}Jag_=@_!q#@Mim79e|RM!{@F{~pv4Wucz{csBz-f8j>xF6y0r^{HER-Y z2=pPc#_Oua=8NQ8+&7a6Bw-BezihDOdY|RAE>W}}AFXhfOx?$lcX=7HV1BpY#XFHj zvzso|zBP^OoHaIK!;EKbb{RSSMKqwqf_yD5iQG!Tc>4zLV%O`nV%X}zkCwal>w>)B zy<~K&;jnvlVdlD$vPPULzQX*}WZI!iM=;Lg_JhRq1q^SF5x%}CTwjh-bjJk)Q* zT!C+hh$@Ip{DyY_LbCQcO_@Q{kCJF0L8X_ekRizylRzdvP%Gxy@at&%#S;HNKQ^SO z=ir==KC`BAQ8D;9*hro7Z9E|-pVe1Pel^Ena0>0iIun))L$H>XTlKj@F6pgn z{rtQ(ZznYua;W{hFrQf z4;$Gjt!8jJ`*1Fu*_ceWf~HBXC>LA9578&tM7kgLrLTn#7mZA{hK^a{6~vzyLXxOy z6=LIviiWHtPLXvg?CN7#qEc<0hFA5{jW=u=VN|}LHU@1U2_OCjSQYePyH;TbdwVg1 zlb?Vx-3(e#FE?H{;XANI>8<7_ zepp5E!Alz}LVq*d^~$|UBc8bQdyVML4Y%Yt-m%ugvc1v75g4NOv8My^RrWrQ-;KTe zCt)RFDmM3K@wa?rq5XYpMk64Y(e(TxvNg(kN1T)A3-5k;646IG5~|NZG6lUI-ezd+ zJKa$gQzs$)b}#}W0^XL}XNP_Zgk=D3f9UGWwHYF>i;jhC&98;=vM_z&SX(aSww?=odI*vcf*L)-JkXOaVB;_U^?&M0y9Dnwvy5)%V0gI;wdR^^9_@ z8ao>UlR=1Am*v?8OW(7bYp>!t*xuezc0~W2 zH4^6ig3izi;af=2rZ@U|Dzu@wj_RWNrHr!UvmrkTbeZ)Rn$<<%Hrxb?Z6{#n4JM)6 z_x|O@S90qe69sQYjadodUM$h)Os5pdZ9M{d!P@36kBNZ*BoDECKCDX`!cfP86*ZW_ zB{`?eaGQyY^%1C-{7&D1W^^JrrD6De)mP)>m8-%z#*W{(QsQWC@9WVf>Npney4hKA zIdrWzq|j8fk}l^jz0EXuA_^qt%4bFKq6;?=7!Pq`sBKw1ctPF}BDUytjJ!3h?$4K_ z*t5vsmLP-58|+NSTjsZHp#%RU^z+l@#_Ra8pOSGqBW*vajx)Dw1m?JZnO7cZ@BHIK zLZQ=$bf@!~3=){l7yXMokyBV}^(oa5pz8~_b9Q&ETFuXHh2OQony&PyyX+{=kxTMC z$ycvPpKC91$ZM#qtg?VP-c#=wzNq2>%o}#YwnImN#L-j;OfqPZ4#puQXAY1a#E7Gw z)9Hz7SGaJvYHsR7@(4sX*UnJ$noK=`VG*y`DQA;E;=d&tANPAE@x?d0)4=jDV=u5- z;c<7l_>e!xE!bzk-TKf)%bw@OA8+xW=Ikk5WE+gE{eaQR^u@(y&9~jYij#pduR=6K zqb6mgw!dTAxxZGn{*g*d&7xuLf9jd+el7K_K+#vn8pq$6f~{|`KE zF7yYUgt%_9b7=vs8S;1JJh=gl6f;t>v{jg)yYr936$Jr~vg%%G;DeJq(8Qx|{SapB)zG)Yof zcgec`oRG}cGqaRs7UYcR}u0a;V z&HQf0xr@5Qx_)YE;y?|}S%N*rE6tTNXd2${!|IG?l|$w=x!-&it;Ly@BUB6}k$g4O+_3e_%+aNgxO!{yf0X&%8b>|9tl^l9TywzT|cF zDM8XnFO)O%2uVy5sb8X{RLxnB%eOo|9`fMz`pNxdiFNn1p?Wx9NeSJGoihQMnyto| zxFdU|;+Qu0Z-|rROFz>NK5ZhH5!E1Lx%ky9S?B;o)V(E8U>HSlT_YMwzwnLU^L5@xXeW3$e<@$T7mjkGd!I(SQlP?TaFRwXj2xn@9$>AABj1JQE`!mfrdXe?9m1 z!ez;Nq$cU3if3g02q|-zSw!RHg;z$9cge%gZc$Tt?OB|b-xn=@K!%(cIX*bCiBo8| znX(7x$pee#-dxdOIR3A+lu9L>vLiNl_2Vn-0s@1rT~!=>SLwj7_!aVW6{o6O%S5Dvj$h zb{nB2CzxGSo`Dcj~QIVswbwVD%#CkUq!_&jHeNZf<$Gg&_#amao1 z&l(1Ls$abQ|L>g97~D_#Ymg;Uv<|9S2SU4)l&2;Hx71)zC%sOY2(OrROFj%&JnO~n zcNPn|EsO>H>q@$Aa!<&9qZ9qQxclzKtlBp7E8+Qc>V@W~R$W+PA(BffD~r#ZETzrD zMw9~9;#AkEcY->nlvxhU&0M-J6Dis%r8}?-zXQZHTnyLEN?3dVrlE80x3CBPgZo${ zCVZVB(!;O*H0f1T!OG5){hD;Ud4y#Z&ZCB@8yh6PUNEQkrh?_W^-=$5ab+lNm2T{j zfoV`PZ@L>9JGLP6XC{@I_~%jY_iD1-b(!WBi{}=xB=HeO(mOG%38s=ZYYBP z6L4t~QaH_!_z|FvMQ&+Gk5OG)Ip&s7dIH1e-(>}xU&tORXruaBCwXCD_eLoj+S~Zl zG}1`e=KN7$+?z8Osc|Zn4MMKF&*S9BVC7LUyPYp5OebNnxu>zqdQmA&U%Pj)FJKd9 zm{rcHNzE5}{}m1uL7sW(n15E#WC;SBW$@eTec_$wZ_QcTnpxKCG5q z;)FCrvPwElQV&jiB*Fd3y~AQcbo5qdsvIH`H*QX-B1959$5c-6WF-?+`kRm?1GPHw z8lETd3s%F8r=nf!ucJtUJR%RpuU68Cjb<7xc5c74)HjkwM!9GyoPEZrN%S8}11|?V zQ>TWYV@7Sz3juVt!+tD>PAP?v^pxv0mG+5;uZ|<`ce`j~OTkbu?Y!+Xc624du?+|@ zHvKJuP{2Gb7AYmbh$Jh4dT~xmcd3yo=YW5dp>51FY||8&CGKuwEoW49Ls3Lh!J;s8 zVV{7Hjw*c{lKj!}rnK`dovGg3Xs8Fpp<^SFklYjr;gumD&TS(gleA^Gq1}goKWEcs zR?ZKqH9~eAX-r1nb}cG9JadUWmOlPd4Yh%9COUt;lR;F8cGr+&=ao+VfBZ2O6z;N-Hn68&|ifuEqFP~)zXJ@F}v%ffeLwq`xS-GD_9E{Np2 zn~1zMoK<`Q3Kbez28&|zquOUmXIm9~X1oWb{$%9fVftfzFtOv~V8PP9<1_C4>i z9#>5z@Rr%rHZ!JR?PsLmWiVC*Ks-;aKhMOi}6`CQ*49{-%ybK^g`(;_B5 zQ>b)|Y#z$Nyj(Y#aDbnh7s9j8mA7u8I)=$~1?SEElv%yO?|+=xD+ye34(QzRvCw7O zZ$5!_NAM~Ss;w&-ySUjE~*M>q0{*pCz6)@9G`xNSGWvYqXq^e9YV zFar6UR0Y%L@5);v&OIS(kw(VX%cH6VRcswOCHa(9s%b_i?`c2ZC>1?C=R$Dz6GT35 z(P=aia0`dOz)^{5y592E{=jeKx+?~FeS=$8kTg_JvL|2{q%O^bK7JBK9UoKBwqxQ9 zI{qSgUBAl}_R@8{|2j1qKJR;4?jNlCFCjw8sk)fA9$_#Eg#fO6psyC~s}ggo^wKe} zM;%YlhG_1l?Gn3Ucno$d<2~9a{7F-45md{-*TjAXH&xuCDCX-@Y;FXEhneck;hZ>M z1C8r&!g(>2XXm0&aJEPO^aSn^1<*g8k*d?UKf4C&g($m1Q3P+XJ&+I@>w&yj=e!*Y zj$titFG0!?F5TBg*KrASrgYvMubH%dUgSH}^UN3e<(rn)Zm=0hpaC1c#epb8!eV}X zJf>gSNRfU<@^4e6ww~!tooEDN<-w9|7&c_Txq-k&Z{#DNE2uJpQ#Z5p+5`md2gBv% zUs(or3FH{7i2gs0K%^B0e?4XNs5)7{6OHU*89gYUaR46$9WpY%1KS?Mp?_=Sq{`oJjTV zr!pvghgSrR-Dg z4DO0`f$t}G@I|J(r3^r~K%CgjxTDtstk2B(A)<`$XfUxk+Ja+O=r}Or^M4=B^S4*% zCw~Pnej{H&f4tgmJl;{5Y@!5Z&e9r1Wyi~VAL6Ec3PHHF*gw+o8rKoTOP`ZGX4@<* zXCK79J~n9OPotiC03$JPQEy~2s6`5?gt(9OHg%#;%4djqylW%Q-mplW3FJ*e=<{m? z&3ZljN>g2jPAJ0X=3)0gYo|{vGfibk1U!qY|H%C-(F|kgFWw|D`L4GW)U!=b z4f`8MLL0BKGP>WsQMYPJLw^79#(8n0{%`0@^%XT;i>y!3`aa)AKml`HRZ+?dFG407 zsmjOc$v5;vJ(P?Lqj(G}JXG>NYmdZayQRAAIPRwjSv2k}8QmgpofmLW!09D0NMYov zy_IDJ-D-fdZPu~^!`u&p25)WqJN|zG-am~q*2u7NzoF%Lw`g5hq$n_TyH(Y-`g&@U z7^{*J;T`YWlu*&n=J~!;{-(_OFePMVHY3!GMZXF;WD-8;dNebg-QJ(cPYV5lw)y*e z<}`}YUFvXmd-;LhDBdtkQ9kvEljABqa<>9!Z)O9yx{Lmjb2_f|<8?DF8Dr&`{fh1G_g~KJyI^CG@K6m!u$Po|+*)z!p zROzZ>&+t{Ze!cHNSc-SR|67onPfQz z60UP#pC{Oa>9)mTu__n7K=Y4-tqn@-C-o&%Xef~_esjL+au&%~6yN@&c7_y8_9P)B z^rt7`Xq+n-SX@g>kyP&x1V2r|L6x8$MCVfZvHSC=aUpd?O$D*^!sxlJa9xc>By!S# zj1M;(sPbKI4U9c>f2Q>6g=-*hJQNS80ifyz1{^=6dgV2EP;|c3^>KcQ ziAA+}8H&WEU}r!UNG(;wo0DZ>C6X6{3xaL$reJUh@$-(7hPzD51t@}}qwrQ4iBKiu z9}LeIL$Na<4@MhE*^kcL<8l!q6+NP4aC}tV=_wSuUhc?@{+SU3IW!TZ2N8Dl04|=w znUEC2n`PocV@Ao*dY2XQJuUHLH6xXCv3)jV;{fp1AkfJ=L17?yE`GCV1!Q29q@7W>b=8 zGLb*MStk{ibl@umxEKN zo2)X}Kw8Kr@G~@u?F7V1OXba~(u z#kRSuOyOmwW(ysgO8?Q=oe}v7RrlB~@W!W{=%D|2UmKUkZn20pv}nW8uLmw%I=7D+LU6Ov?6&J|)4g9C^(t8Zc-lqT>OC-Wkt*<#Co>!UQ){Jc^~CN_C}H{vT7W25VO>h_ z3knuG{e!Q&5C%J`xE@qWij(|8y{;5;9l2D(Cq^BCLv+|<>2Bcm3y!?^5KMi5?ky{G z?Hf}gE*CgmyX+SyfMhz37_d9lMXf1P3C@Y3<;?2oZP+(bL6h{0B#(!0SPWg0sqs^6 zale&77wJvBp-;2NLK&@)!J5K z*XCuT%I}}!``>S(bs=6?Y|~C3Exg+{MGa!f47wa144sEN2*W=ukr7_gJzxB2Pz~0# zHb0vUc?`3*J+0fx4=aks5asxbsx}=s;(?4=^7*H2RU6mn7YjW{TxDe$LT-`;OU<#n zioNgVu6ciMO^!8DV)@;_rv3fm_z)qAf#A4pzJRIHp!WJS|3WU#5WJ7$%fUkOzz>)y zxj%KV8#KXxti5(~xlqraujQmO3YHVS2kuRlp{EWe#jw~^w}U+6)_AiAS8&9T*tcO2 zU`I81a0N#E=4`T#L(FBQt!zE3VU{eWPg**0 zDco8kr>7zROa2V=-{Bsa=kCx{J6MbV!vfsck9Yq!6)m42y(@zQ87()d$-k=94Qsp7 z-{T}d5I=}`?=Z%rnsoP_zl2!PLf#qsWIvXG zs&MM?kW)%z#QHU0ouW$%?IvB1U~x6ny)rYXA-S{n zFtJ-I?gQ0;1fK9M8Qk+L2qHRqyXuq=s?Fkge&)JbW!lz zZ3k_&vQTYG0Zxu%pI&bUA%yPaih>|7bJmpFG$6Vu2VHD5V#cPYE zi;)n>sVDe?Q)h5Mm42rrPV8_W>mJrn_fg7{fRJFXWDE+5iEoz;AsQxJ-6H=`s>0V% z@eAbZx**rfa25E!#dAWjyy*DikP^8Y!PFm^w#nE9^9ucN^}k@_2YYht7tlaX6l7>j zE98+$z|bR4+HL^gvi>36NhaY>Q|-`g=pSA7;GI=^!AC=B!J;`swZ(d0CGSp<6gFX0 z4i3%33$#nk@UU@WqWZb%!9axy5>dU-v-JI91V8VMsJj@)B{8NE8hvVRG4HuI%C=kX4MF&6Vz!&1d@|JdH65D9|mt1sTM>$!pUd0Av zG(rvwSzRao?Cj&+qM&zUd|O-wv9E{ZecTBV$=i8-V3I1Z@lPRbRIQnSxa+^}78yj6 z#(Lc;~^+kj7-X!_J_U8`iRcyn*@;QcwSgV1j5Coxkw+-sxiGGV4?|$v< zlrSV0H=mx~Z?qX`8m2A?Dab)B3jNT9rLHq?3lEo^g^aU~-p=rYJ<1((%eul@A_Lze(^7-)V446f9xj{mLhxz-f9F>Ut~3$lTTNx*)f4%K zJk3G@kprlu(2v8+Wf`k7g$=1iL!5p@$+cSc9V*7GRBb7gUR>LtE zf+*sfg~|tyPtfB3`!?|Wf9<_hRF&P^KDrcClv;>NNG(7yD5azo5TpeJ1f&I|yHlhD z>23u90Rd@gloCnlPNln>|NZd${@>nX?1Me_!9Lhyf9vG+g~fd4yyx9>KG#J8hZ#q` zHcJO5t-_4vD}r5cTmJLRQM-j=@?U2QGh{R1-^|XMn%!bGGXf( zhA#|o17u`H9tli+{1J1v<}mCY3#Ai9p=v}$iHiBrb@1vZUuq*ZUioTUJI{7byLwt^ z>IPhGQkpAT3BCdoLH6?9I;1hsox^V%JG{}u?*32`IPCf1VaK9eec=C4~Ozj>yI5{v=2N;V&Znx2jvLM zndn&={g&lbHkNK7b)*{7VnrUez^>dJWVo7bAx!`HhJK;;?bKG% zaH7INkw~L8%P5SjQkhb!%-KB_E1x86sVkMz3pa*?b-_4@Gk@zxQz{r13PX59q42=T zkjmmQD3s!1>Ni}?{IGxhBpqDsq!l7<9Jn+y2B|$WQ_61%)_BE(W>0~T=JtQ``T^KS ziR<)ubiCMwEwjp~52`GJOPtw6frwqD)9mTsH-^VWnNKbMCY`vLqWX?i%8G--hgIr@ zF~#XSxP{kn;cun{g!+zFF=<`6iSnIH*HGlrZU11x1(Rs1^GwIJURUfs$tB#^h4DP@ z;#_rLO`>%r`ObQ~13wg{Li_Zx_GH&^zxOws&=kJ|Rd^dwMf1KwDnZkr*K4HZNG{Nx z@3~$lCQY|soF=M8N^*TV*P`}CM24LC;)CWOOIcQATzd$>A2Oh67V730$nWcO^P{2& zJ+}I6dx!`2Z8qh>qtJSk0H6Hvn3_nyt3V!qeSV)#a z-4OkWA715s%DcOE!o)2ETv~5M`PI^F43KyJJxV=pw`wIoDK9R}%i{iY>UVaS27~1t z{h7OPWM^V@@QqO6n&K;(#Hs+*oa2p#8BM`Anl9I0uWCGbVZ5ZQ9$+!+essb0z^ z8Np>Kj(Zf&{P(1|TW`ca_jlob*xH6cWIlEtMATm7|K(Q^j~>>Q<~Y)myXH|3e_?LY8BQ(x1ma?%@zJBi%5A{SaC9f>(Hg9QgS zFf?l^sgK&zT1!S;F3JDs(qo;p>dj)A)}499*A|4aN$Z=9{~mMaO4KDJNVjXJY^C?z zZPa3QIJ}wCzi7%C-=Gt_^f=^?QvYklZ>*H;nfSf>7Y|#Lmi!to1x-64ftvM0GksxT zx0)8f=2I}pM!<3Z!<(ZIJ_Ni6h1fS-C#H;?ebpL$TaI2WzPL)Xp0zSq*8K6eFTK)s zQqhdxV|*2~iz8o}R`F4b$dwr>GD-)o@EyCyszh;+>yf|q%Vxx@RwZ}nsuFft(!nOf-6GE|n+)88R-rRpYM`s~k%Zj1(@5}&Dj;QOH*>mCA zB@&m1M-gH7!+rL1BCl8JRp^2y{Rk%RYQ|Ej6XQ=iR_0t-<#>Os&DTrg;lHotkE%!X zSFi8UcuFbXRor{AY5{4sV~QuUfd5S%IE9wD9v{{G`gMSS>TdnTk-DEn_C-{BjsZSb z6m{3JMP=>a4$-5!iHv*A%WT?r<>F8EjG;s$t*0EJ!IgIbeF? z7VM2}2X_s+*!vRb$+@v9d!9Z zrX+3+1H%I5yzh9iueM6B*lIXF{#=IU4S28qi~MkuW>Z5)%t$9l3Hm4BE}hVtjbgE+ z>`=Q%DYi!u=-hFzOlEH2^e+r@%DB~-pG7!M7e}5TA|cn;Te?Ohm+EKGl~}-bf05b7 zQl^ii^~F7bz`zYUKcd9W?A_YwMyjmu9$9=r;y1nsS zEBs)HG*fimdIw``bYhW7GdRZ zEfthz&X5{+rT^VZHDFDYf7Agl?9eXk{qTM8fia(g&gsIuitWNzS=&W`n`iM_%tg!TaD=xF(pn2(N!(h z(#KbQ32~D4P!vm;lBOW>50uW~>-X*S6JzVemRbkMn(~&szD9r5jS{!uB&dHd06=id zZ&BwjdwpLOp6PtMlG>}@+gt2zqZE~3GCEk<=(%w+QxQ*U6pOqiY)(;Xa3FI`BTWwA7< zXKaM0ZuotwH`ZJA$)_J!wZ0y|8#{zZ=G?Z0x`Qm|1KQ>!?AB1&`8in!;-@4%p(~f zK=m;Z`0yVcVu$HeeIQ}*c2|1!3Xfm5UCDkh-yr(<*SWhRyd(#6#yvmFs*f(G5Ngua zU#)2C^(E)b40)x z>Bi336egOP@-(K*U+ZMt-?R=YNZcvRcskc~XTW4=@4f2eD}Lt)l^jg@=LXp?X86|H zJ8NF=XV~9$y3pmx5%>(d&}05YXaJ7f3mSM-n>PL5EYl3(r%m;jXVghEgDI9}B}$61 z$J-vQae8uE78oAmMev3bpU(a}K)>hgWp=;y;76s;#f04Z3_*AFDt<26-17SLs3?=I zXwdqo`GG^7;DMP?W1^VR2|9(Q*|E(}yHINJ5@s)5ikP~FL}yhTQ@N!o7J=2Uu zK|H4yVJ2LGhncU3MtO1C^c7~uCg8>L(rHl_Py3ZXeLE=JJW zt<>D-?Q$KIRHJv^JN$bo#C)VUNiFJ;;K}o@3iE}TmgI@DV;Q!1vd1iX22Mwf)bZsR zuPW&I6C>T+jjTk<%gN;>3sWm!aDS6sIG*o3=j4>G^hi0L{pqjtC2yR7?|M_&)Kso} zCUs4teR}bm_op@5B`a>xs$=e~R>pshy75oT8$IeyOS->Zez+q!Tvg3GWsXkNNFiO# zDVyzSm{Ks6rM<+|DtUYhFJDf%U>$RZq@VjECOw@5wysfiva%A@yZ7aSs4uU#)}Z*g zG?@~nZ-yI;o<72(YPeBjv$5LII zp@{i*S%HGy{>$po^v!<3{pnJ>8a2;fZl0$FC{ptQq5dWc%<2PG^ z!gRy7Htm&$+Fq@gOE>FG;$X16{^7APs(DeJzFs_QAS9Y#zd++}agoYTr`$}?=~r1s zy%Tey&IDmqbX^tiua3M7UpK3{~eO zT0(!UY)-wkSw8i-`f7DJa^K2sntAf-QzJ>0jZeBr4;Yw zt_xx+@2wUb=9&a>#W(&1D&x-1ahKh;^_g1>k88>6<=T0*OWr7`nI&E{cp`AC`6w6W zu0?*H4{Zq2vhI@8xR=aJoGu^bO3dIKFtgvyCOwZQ_xH2%vF*1lDlO8f*oC3t@T=;a zNq~!kLKniK>z@gt(WeH^Y92MA1HM7{N%2_+l^nLBV+C-BUWWJ5J#u6 zYU+uCQUrOHN2ZB1{_ipQf3*V5Bxb?oUrmZX{<0r^vGe5}Iz_#|uHS2nAIZUPfH0(J zHTm+EMspJQfZlq+e|9m|s?`YpMu;Ba=`v4z3p;#Ghx{BB>1($dtEn&Ype34D)Zg6k zH(!vlRmvB<)tk_#h2g@Nd^z58wS1-#JDR%<{)C1WXRTH%l)cyOZ%C?%!5;s@6mbt( zVXvpqq(4^%*UNUw=JzbUeTrFCV~*>+_1gW7+-)XEo9s4SEf$unk*RG&^u0?v5>&Et z67EjvJkCL@<(WQ=@!dBWe`CZ?84Ug=8g!IisJKAa|LI8QMby4Jtfy8A+d+$_Iog;~ z@79j4ugG_%rmY9Mpv=`HE|~QRv#n-4<&ma+UALEt6~#GK)_<7ATNJ|F!XiWUXZdOV zOF1rjYX6#Tf3&+4){#g6<}g&;Z|H5k-Aa^h;}i3+eSS}2Wv+w6G;g>&YjNJL?2dpy z;vBrP!^)1Hc$=NN_&al2V!nONxrdu#D1yvr@)h0P!Gu0^cCK86aoC^*^og~8VQR6C zd;d%hRE^4M+%7KmVP+2t>-D5HRed}8I`{!9|D$?NuF>7iJ@Ukzh^6|U+U-lv?6Fa` zA&)&S<;MIO_-KC0l3G-s=_IVY5KD(dIO9CxCAmv9w~HLJOOp2g%pVZqPn#d!l@R)U z(``|D|1@ci;`-niSdSyN!^D+ez0nkUw-(-~(hkT>Xok>_54aRt34U?Oxs%Q9+$KY40vIVRJzy>9|`e~_N$KEBvozIoGga9ybS?<;?95fM2}IGyH$z>^Y6DVwPCsfOXj7Y?QQ!BRy1dJx ztY^Nk$di^%JBGXJoJTA(=jvB8CPKOA!*&xc-6h8IBr&~2D3mb4IVv2Pg*l6*`wBae z;ww9yktY!^YiW=-hmyX4Ir6IMz5`=B{B)&x7FT_qnp>4A zAv>@_Y2f?oYmYw|D;xjpE6P;7ZF3$Z`_$u7HBJ8u?wyqqYW{0>!92tD&7$i^J0BPk zncyh=-4p1ckLd&j6~1wf3jK3cS%%U+V!24YoV2|2+x>$cvURTUduaw=vhJy>wi=>Q zw9NdI@apgSq%nm48ls5@pT*W8yNYGo3LFX5PJ^^tFh%J$$3g{e|L zryH$v&p8jYPU9~z_R*^B4Fk7n$#Fr`uvac^gjYn}x#W?(5`iasA+`XM?nor^Ao3G)WL|Z5ls-a{CAd7%nh(hU<@u%^Z zaETJh>(1Wk-~vR0Zfu%g4Nt$r8UGNSH?m(GsmG1MXwwMLd)~ZPN5PF(Xh=HqqmeaU zG>rbK+#W3ZejbJKYtY6)8DUl zj2jjeHM8z8VrKvOCfS>_sF&~T>$2_=Bc#&DAhUC$Z8yUJPn3=`o{5P`8s}4%5xkM8 zRf&CG$1WySbe7dPu5V<7V%|=Ntd4JhRKNRkTAE|k*v<~CeeS36Tz!BP>AsY47bWK< zIY+cipR6zVa&n-srLTt+TtY)O8*Z_uP~CO&|7<4`c~heQnkoO5OuX*=R4k@j>7-}U zXp&<5@4Qc9C?aHZ{4-uZ(US6(ApVLaF@k61RORy2u7RqL`lAJ(Sd5dvS3F`wJ;ZE6 zS(5CR#vq741be8iTpyd-Kau0sFFxC;P!agX+F|6hZ)AbGm9*lVkg(+&1%o%BUEvD< zwc{!0&QTHFcC7mS!!@X?A}xwlErF`GOH0hOPgYXQPj92Zl{~Q(eC3tUk_@&oAZg~*MuZGI?L-|-&_oIDQlEd(h;mh$lea%IJzy;U*!fJV{Gh-0XeBtvJwwfDwPz-0$ZF#~ zLjX{>tVL9yJf^&!)(-h=_ca!}RHXpqKW- zmdemq9Xe*Sj~d5+|CJ8+E}$>W#+URKQ;W-fhaW;rL7^^QJ~uY>dz2F7Y}00E@5|k0 zqibv?zY}ri(tUbqT<|P<#!okkE{A)yaem`ZFdw2SnGxEwJCEGO@R$?xC@pEJW{t1C zj4^T*Am*9>Z8x8iYLa2K@EzxP#*==3qcPjtdMjb%$wGaOI)P@YPgxc<@>o!zu_2%4 zNm&)b@QZszc6oSpZZn^uei5+0EtST2cpEXV%?twqBQXNUmjmQ!girv3j$2|o7{@Q! z40++W;vX07ovTQzqN<{1L~JC53i;tWI40gNY8vTL_!^W zl;48{>8>40*G&l;Tl;<~V{k;3Sg!_xb?roQLujScUMbqm_?%f7;HqFiYNxk$Fh4WY zTEoJBDrn=L$2G~oBNixAJyzs~jP`LHXDw=1$ga=S;GWvnM8;$^J;PWW`<-aNVe$nE zXS+c{%8)JA-3p!}$e?LZ;v!F83_yz6GidEPop(Kqoix!agd$okX7gA*lI0S=QTSi) zX^kjTBDcmf9oo8ZZfxWBBg&hNA5M0>vv7@eH(QaM?SWCfM@@>$dS!uqsny2Il1(SUkIbl!zAhH~{Dncz-v zU|Vu=MfR}|ZM`jlo}<;&l0*OF>JQ_236HWOA|X){A|dymisw;)3tF<`l)o+>JiXeL z@Zgm1f%X1m+_-XSRCaZr18|f6byN@?p*D~Wu(3y;(W#8RM@qyXe!ojz>2n(5`S_?x zd3)5?l_lgAA!zAA5g=(jCjmK+((}(U$$H-}ZdJX@j+*)%pL}TY8D8uG^*W)erMs3{ z+?${GVz5Se!vQk?ZnoY*YnOg$j@N_#aWh)aFtv1#_N{UX@{01Dd#-ov{tM)C59Sex z42avQStREwEpg4=e~keVYD&tu++pI7D|uR8oK35__iC_W*lp6fdDQGt;>nWSP|Rcha12)cLax2Yn@t-O9e*|orG(RhA_0$ypDQ!NFy12$4EZqKaNU287joaD)Q zlbgi7cKG*)m4KP)sW}O=tpNTLlBtj;=U>yu>cX2hbw$;0&VDd)KAssVZ-6Qjk8Vx5 zc`>phF|;Ewe^z#FB!A>v`~lO8js2;UBZk9yV$HALha-u_BEy>?w;hW&<19ojQUPkT zK_lxk7w$4q9UXHq{0s%AgyEev-K1OgVPhDtM@#Dhd;1MHG($43y5t(#D{IUxKk$`n zms6U$GddVMR#A{FFnKhm9)4))wq@48b#kH3B4WiRA~I$KIlqyD$iQ0gwViRH%!D3q zv%MnoICRyK&XCpHOdcX<*k?n zl7qxf4phYRFQ8S^(hYwwKU`h-kTvgvQ8V%~!QaM`XM1jfZ>w!gPkabfk;gvH+FYb6 zJFwe!Cnu53i=ulgPu6xz5AlUbeyS3ZvY+qJWEC+ly$DwVa^rhtk~JhB79w6MCI}U* zK~{G)P`k=Z?-KS0rHF4Fv33=8@)_gaXi<58GzL1Mq6Ln;Pqy^>FT2Z!T~36*qLKf8 ziKmS&=UW;s+Ka{A)9`q2qtVyRD3#MOF4;gB{;Hn+JfBB) zF+KPCdyR*cwBp%f2D980w@cY+D!ImIs3VEDLPBGJb-eUj>>g8({w8C^@H0HO&`S3g zLAx`<6F5VzA-13$q-~~38C9b?6xT5(^RKr zD(_WNqpQ>iWtE~%GF$M5@lbvgu-vy1)6cCaCU?_HN#mtEeqAH`v+?YyaLM9eD(xtu zZ>^#pmrz^D9pjd{lilSzb#B6vHbLdLZJ$6Q@i`v6zn?US`mrD-K&#~$xJ|8d*yKJ7 zGZi{0xUkRCa5t041dc{&Zd=aMqdp4$ZqGZ|tDjbfP5#Mx)Wl+b)crQk(X}nq2>Fr{ z(Zo5F^4HwswcOv)PQ0qC0;=-+tNJ_8duF|c)L_acr)j5KH=X=E>w0VYkIcFcKo?1` z;H;rM?%QSRDbJlx9#`m=H#<{oVeE5oi>HC7sQeqpia-x6<>TtvH)le^fA-Z6hD*;u z^w*lBsl`I&zMT0s{|VP5Ea8Dk+*z{$j|f&(dp(|S>ZJ36Znv4`-MThI*k3Kbv+obY z^mr4q)r<|xMTfa5oZ$YSCV6bRxu@7ICO|Jf>K=qA8{^R56ZpP8!OsSyiLP4;HG@IK z3)=E&PSY|636s8zI+PuUn0)F*C?%A5c(gFH19OL?Vry&nYxB&o z0v~MVPhzY-7)Ks@^`k{ZeM<8JEWw?Lx;JZ1{Xb zoh5+(GsyqQ1hnD*(34uKc|hO=06R$u3E<5D$UuYEms> zy?`@$>7*R6AOXyoQM398LA#;9TTpeTU>4gt^tClRL&gAOCnGDX39JEbZYRJh;IIgE8=k^4G#E%F7P`zARwW%pE8Q0J_%; z4bW+^v9TS1!3WrO#WwR@dwYA&c5zUTMdr(`ckR+DUo6bwFp?IGf~oB`=8-bHNt{P% z;pcyQKMK^=_;oP?u1+*nms%6Jtzl_Y!5MHC;0cpPTZW}K<^h02elc4W(7lajP~4Gy zxofPZpo{Gc=HyC(d+r1PS%v%nT-{y`Fubn`QSdvlpNB>?nj<0Eg-!O+)Y7iVhw{pt7z;bn|h2uuHvEG1Xr zEl9fs^DPKnBqamTIs@OdNBaOXmM7TGdcG?Szyh;p4*)98eEa6iEphp4w*ZW=E3p1+ z`z-mdg4TY==MP%n1!{9jzxGw(aeGNBI!E5WFLBzsLQv1ciB3>V(R=#-^5y^?0nUw}ITqE=iMhk5nJ^q~YhSt+%UY0?B#9<>`h(et(2;AA{hMzaT05GblscBA5&S)RjefS^#(`JBJblc9>+TLr`; zta0qp^73I>Sx@qKYdyXVngG>{5KeMOKXPL8-~AX7yO@7o+Wt5uFZ+!H;}WJyHi89x z2H9NFPNQQZej)M8$uZcRLZjt*n69^9lMqN{OAe51#~(I2iQNr+D%#nr1-a+ zJAHbjKDmp~(@KeL+bLaCI&u@#oPE*&Yi`%(#Cw;w`O$xg_0Y;{StjG}w$GRKp!m7c z4G=*31H50tO{8e-TME2a^dbV{giF}Choa;VuWhpa9 z4Wb@|uzdjm0nm8@c>fy9>Bja3fV(a!DQQpOX0vdk0tSOt=Q`z)UH&iX&WS|HG+wL; zGW=ZQ)=e8hysHleg@blyAq(xSqfLw) zG{IR7)&F0j7~*VG0^0O95067vrm{$Vz!lKn1)FB9V+GVoEcb~;zAyi2nfho`nLz&( zuWlj$ex#53B>r~%9%@StL<6=a zltmSL{r#_CM69ef00{o(yq(9%BE`(-&z}L8Rup+0seb*3c5N9csRVXI`Om}W(6;}b zwnTUS-=-_hpKqcVYp2)w*6WF3f$^NC3yxdh4NqG%PvjL9IjsKv5)Z#UYTdyP4rPYx zGPdpB+9dVyoY)gW*WHToDga@v$YpKya}x`qK)_goD0yeLovZ%gHC$pU3UZ;M*A|%^=VRsDVAJ>I`6^tz79RX_!g1sCO5dm<~c7Q?(y=nwE z5eQYle3bJvOiMQ}lF_aLWzI3sc_Rqj%dacYLUIx9n#5%KA~k@of4bKsSTejU3&Mk_ zckL-Rz=s2<`-EME`(7P&dpsA*^F~&>-d_^Qw_4{7 zCP2hfPIFFSzS^Q&!{04awMh}OIDD>=k9UCM@OQ*r!1VV(PXX05ZV$3Q1iq=WTz?^S zaE&>n78h@Wm%{yQ50UFRoQNs=Xn$2L&dSMIt{6Wy>qxzR{kmBxFF(KQLQiI5fPV@C z$tvUr@WyfdcRDMwvhHC-l$4bG$bHEEtWGrqz}I2&FtBd__S1kH$4dr+6#%)5<*pZ_ zEqA!PIu1~-0IAOh0IIcD>hNgpoagJzwjX!32QWi`@12dftKdY%Z@UnI)&aQP(LUTO zSG>@`OUC(0nC9!kf{}RGLpCFt;UU((FX3}D!Hb#=k~b^#sSzU&oHb^+pSoS?g#0a@LLIs3_<7cjuGAquA& z039ov#5dU0^$@UZfpo#vH8@h+=Q<#PXF`S*Dn@nr6(4miD;ADFiV*vg;ZTSBi8ZEP z@VHxWyF^^d*|`Eu``|(Tu#;8;>xZBq0=EL%0=U)YjRuB4z?m(e7YD{e__wXCt@F-O z8Gs?WgI-`$RX4EG9TXQ8;b4Y_hm&14nrpRiF+ikt1VkG^cCv>&_$wGd0?--&04VI_ zF(Ba1m$(1dO!6T%0hxp^mD3x@)v9{obkYkA)w|Vc&KxJdj!t(7%&qrKKr>giou%tkfMGE(acBOo+9eKbHiS0GxJ0 zIKFl3z~~S}`hfP^_t(u=0bLaQHLUothe3=s6NnQ?x}F#rEdXY8Ii*g_IqZhpcD2!8 zvdWEmvmogOCMH@7TT63D;Jndi*%Y9Id!d0afPb9hl2lYuk_PbE#E$VIt14gwv+nKd zI}-y534_-iX66;C)pKvP&CJYlv$Kiu*uLrmr&CrI^Q-UQhF3m-)pz2y*J5c-*;U*m zCkI?~<=iE^XZ_o?-21?D!>m@qxE9Xwd_+-KHx*6G#MJh|?yXy>o+G&3;^Ja_e0)gK z@z_!{MMZsZb{>nUL_yACY-Gf&S)GuS)XrKbgjY9Epa*#_@VfB_Km3n~HFXLH!K+m; z9UPwkgfm8D;<=EJCz_UlK@3MuPVP_6qiEx% zV~CC$18*{T;kevik?7xk{F98^3NgC7Ku-ls6y#iHuR8JVkz5rVItw7RA8hBjkDBI# zp8<`G)+<~51e_JXv!JM~oGjqVBPb~N?G34@xcGZ?%CJVkV-X0I2H@-=XB#R(@|FL-~OQr3^={)OhD4c&8-4@9svvtXF=GLN)Opj z{xKtmf&q=KcHkz0p3}K!&2*7RMpvcZu4C6J^>YUUZnF}47*{dhkx{j@&>XWOy3jSwXP z*7P-@&*|w-d`aLmF!hkC@Qw#w#baZf+W6N31Xv-!v{U2esukRrgv7sQghb@CGb#l< z9Uzt$4Vr*mn)lnPM3FGQ)G8fbTwDaNF=z1z(iq@$YObuTd{%tLCOkf#*PuONXJ@Ce zu`xY8orq2ps=LkgCh8jSL={npc4l(Du0n z5q4>;ikF3j#UhUX4g?(81bBl8KDFneAAFV+{lC;#LGnQ8#H>L~TWO7ju6f1iNsof8 zcvw%9`@qmpxp9APcZMu534rJ11C<0KU+{*UYI&)^^!4@C)YK;Oc!Al7%Uol?y%>m(l25iWK*U@7E~5m1HoTIVk>oV- z_R`W0%_^O;k`hFho%~XuP(mU^a09G9s*wyDucScUq+tMeg@NG%W8&dHpBC)~>=1BL zUKW;+Fw)nT4hlNelc@mS!ezaI1mU9tZQ9v+@`_m*5YXpHB*ew%yJulQEmuUoUY7mp z_VAEqxY3uM-q+u6U|^v6-2zk3v4&_O;OF1*3enq#p1}-P^?D$LF78B`3*Mp06tg(XQWp%<yS|e@$p$c?V|iHp@HC?X)!q=_ZTU zBWQ*Q9@PoPY{kOJFDal{LH`D)14T7@3fMi=fI?Brvd#zpNdUM_x2;s_%b06V5jrZs zX7`M0E;s^q7$l<;@dtIUW+g?199pxC)_7Nry^}{jSIQS zvQkp7Djh7f^nQ3=09ruMnV}+a#opWo2AKq$qfg(z4~%evvie{rVG*KmR$x=9B>qDLT+#;-%G=Oz=r1dayC4ZP7nia19;5s#Ew7LO>N z2u=X6ALIihTb)2>Q*R za?;Y$tCgEUP1BI;;eNFr`%N|ASDo?U172`bw{045fw1lDzIguJqKVSp%j_XoZEFTB z9ep2>6^%?1pNfVCv)8X*zZ@pLNy;hPe*Jm_V9BPY$3WJKb=Y<32A2EWa@h*pc^y?c z*%Un8{NziukmKv=={cL^oa=s!Y#0@67>!@;ygjCV62wg{FrqdHYv#o|f}n-v4h)QH z)S77}B|G<=pp{3ediLn^haJ2WK@jsUFg=1SQ%tN0sMMg#k`xj`;|1}jzWzNiY*Iqo_Ap^KhvDx05AA0$j|9*)B5DJ|U9B zV7xkJ)ryT22b8%jr$h#%XJGA}?-4a+wm4A7v4m#cmQtb1{zfe|r^!HP+1@X@V0t9E zQp!;+hT;m%GP!#Q<=CL9EYc(JLK@#L@3l$aqLM>GETFL6Robz4-?e}H^#q8ffY${3 z2BRa|G{w%aa4x>=#DUrJX2*JkYOxs(w z<+ya9kh6ez3m$D0p`wF|2>8$l4`yL|(D(lnYf&`*F00PtxCu|kA)R&X;=JtoJ+Wi6IBOXv zzs;y<0-m|FG;h;A$lEs|gyhXx!ybR51(leJ|P0 z0f!oJrl2K<=)jsc(BaiySOXf~PjK2s%gk&B=`OT}uM73`_S)jE0d46KaQXocGx%G( z^{EEv27whr1FJ1rP&4XZdjUccI4WQQ{9y>y(QeY=MkTmK57Jr)JIII8Lwf1#AOD{? cx~M(H`tX!x=N2ndAu?(4r!peBLRy~x3rzV2jQ{`u literal 0 HcmV?d00001 diff --git a/docs/wrap-up.html b/docs/wrap-up.html index d577f8a3..2ecf8c5d 100644 --- a/docs/wrap-up.html +++ b/docs/wrap-up.html @@ -440,7 +440,11 @@

  • 7.1.2 Slides
  • 7.2 Reading
    • -
  • 7.3 At-Home Exercises
    • +
  • 7.3 At-Home Exercises +
  • 7.4 In-Class Exercises
  • 8 Wrap-Up
    • diff --git a/resources/lavaan_summer_school/multiple_group_models/lab/Exercise with answers - RMK/Lavaan - Lab - Multigroup and Meas invar.Rmd b/resources/lavaan_summer_school/multiple_group_models/lab/Exercise with answers - RMK/lab.Rmd similarity index 100% rename from resources/lavaan_summer_school/multiple_group_models/lab/Exercise with answers - RMK/Lavaan - Lab - Multigroup and Meas invar.Rmd rename to resources/lavaan_summer_school/multiple_group_models/lab/Exercise with answers - RMK/lab.Rmd diff --git a/sections/week7/home.Rmd b/sections/week7/home.Rmd new file mode 100644 index 00000000..ca1b4200 --- /dev/null +++ b/sections/week7/home.Rmd @@ -0,0 +1,453 @@ +## At-Home Exercises + +```{r include = FALSE} +library(haven) +library(dplyr) +library(lavaan) +``` + +--- + +### Multiple-Group Path Analysis {#mgPathAnalysis} + +--- + +To fix ideas, we'll start these practical exercises by re-running part of the +moderation analysis from the [Week 3 At-Home Exercises](#moderation) +as a multiple group model. + +--- + +#### + +Load the [*Sesam2.sav*][sesam2_data] data. + +- *NOTE:* Unless otherwise specified, all analyses in Section \@ref(mgPathAnalysis) + use these data. + +
    + +Click to show code + +```{r, eval = FALSE} +library(haven) + +# Read the data into an object called 'sesam2': +sesam2 <- read_sav("Sesam2.sav") +``` + +```{r, echo = FALSE} +dataDir <- "../../../../data/" +sesam2 <- read_sav(paste0(dataDir, "Sesam2.sav")) +``` +
    + +--- + +`VIEWCAT` is a nominal grouping variable, but it is represented as a numeric +variable in the `sesam2` data. The levels represent the following frequencies of +Sesame Street viewership of the children in the data: + +- `VIEWCAT = 1`: Rarely/Never +- `VIEWCAT = 2`: 2--3 times a week +- `VIEWCAT = 3`: 4--5 times a week +- `VIEWCAT = 4`: \> 5 times a week + +We will use `VIEWCAT` as the grouping variable in our path model. To do so, we +don't really need to convert `VIEWCAT` into a factor, but, if we do, **lavaan** +will give our groups meaningful labels in the output. That added clarity can be +pretty helpful. + +--- + +#### + +Convert *VIEWCAT* into a factor. + +- Make sure that `VIEWCAT = 1` is the reference group. +- Assign the factor labels denoted above. + +
    + +Click to show code + +```{r} +library(dplyr) + +## Store the old version for checking: +tmp <- sesam2$VIEWCAT + +## Convert 'VIEWCAT' to a factor: + sesam2 <- mutate(sesam2, + VIEWCAT = factor(VIEWCAT, + labels = c("Rarely/never", + "2-3 times per week", + "4-5 times per week", + "> 5 times per week") + ) + ) + +## Check the conversion: +table(old = tmp, new = sesam2$VIEWCAT, useNA = "always") +``` + +
    + +--- + +#### + +Create a conditional slopes plot to visualize the effect of `AGE` on `POSTNUMB` +within each of the `VIEWCAT` groups. + +- Based on this visualization, do you think it is reasonable to expect that + `VIEWCAT` moderates the effect of `AGE` on `POSTNUMB`? + +
    + Click to show code + +```{r} +library(ggplot2) + +ggplot(sesam2, aes(AGE, POSTNUMB, color = VIEWCAT)) + + geom_point() + + geom_smooth(method = "lm", se = FALSE) +``` + +
    + Click for explanation + +The regression lines representing the conditional focal effects are not parallel, +so there appears to be some level of moderation. That being said, the differences +are pretty small, so the moderation may not be significant (i.e., the non-parallel +regression lines may simply be reflecting sampling variability). + +
    +
    + +--- + +We will use path analysis to test if `VIEWCAT` moderates the effect of `AGE` on +`POSTNUMB`. This analysis will entail three steps: + +1. Estimate the unrestricted multiple-group model wherein we regress `POSTNUMB` + onto `AGE` and specify `VIEWCAT` as the grouping factor. +2. Estimate the restricted model wherein we constrain the `AGE` $rightarrow$ + `POSTNUMB` effect to be equal in all `VIEWCAT` groups. +3. Conduct a $\Delta \chi^2$ test to compare the fit of the two models. + +--- + +#### {#fullPaMod} + +Estimate the unrestricted path model described above. + +- Include the intercept term in your model. +- Judging from the focal effects estimate in each group, do you think moderation + is plausible? + +
    + Click to show code + +```{r} +library(lavaan) + +## Estimate the additive model a view the results: +out_full <- sem('POSTNUMB ~ 1 + AGE', data = sesam2, group = "VIEWCAT") +summary(out_full) +``` + +
    + Click for explanation + +There are some notable differences in the `AGE` $\rightarrow$ `POSTNUMB` focal +effect between `VIEWCAT` groups. It looks like `VIEWCAT` could moderate the +focal effect. + +
    +
    + +--- + +#### {#resPaMod} + +Estimate the restricted model described above. + +- Equate the focal effect across all `VIEWCAT` groups. + +
    + Click to show code + +```{r} +## Estimate the restricted model and view the results: +out_res <- sem('POSTNUMB ~ 1 + c("b1", "b1", "b1", "b1") * AGE', + data = sesam2, + group = "VIEWCAT") +summary(out_res) +``` + +
    + +--- + +#### + +Test for moderation by comparing the full and restricted models from +\@ref(fullPaMod) and \@ref(resPaMod), respectively: + +- Does `VIEWCAT` significantly moderate the effect of `AGE` on `POSTNUMB`? + +
    + Click to show code + +```{r} +## Test for moderation: +anova(out_full, out_res) +``` + +
    + Click for explanation + +```{r, include = FALSE} +tmp <- anova(out_full, out_res) + +chi2 <- tmp[2, "Chisq diff"] %>% round(3) +df <- tmp[2, "Df diff"] +p <- tmp[2, "Pr(>Chisq)"] %>% round(3) +``` + +No, *VIEWCAT* does not significantly moderate the effect of *AGE* on *POSTNUMB* +($\Delta \chi^2[`r df`] = `r chi2`$, $p = `r p`$). + +
    +
    + +--- + +### Multiple-Group CFA {#mgCFA} + +--- + +In the next part of these exercises, we will estimate a multiple-group CFA to +evaluate the measurement structure of a scale assessing *Prolonged Grief Disorder*. +The relevant data are contained in the [*PGDdata2.txt*][pgd_data] file. This dataset +consists of a grouping variable, `Kin2` (with two levels: "partner" and "else") +and 5 items taken from the *Inventory of Complicated Grief*: + +1. Yearning +1. Part of self died +1. Difficulty accepting the loss +1. Avoiding reminders of deceased +1. Bitterness about the loss + +You can find more information about this scale in [Boelen et al. (2010)][boelen_et_al_2010]. + +--- + +#### + +Load the *PGDdata2.txt* data. + +- Use the `read.table()` function to load the data. + - Convert the missing values to `NA` via the `na.strings` argument. + - Retain the column labels via the `header` argument. + - Specify the field delimiter as the tab character (i.e., `"\t"`). +- Exclude any cases with missing values on `Kin2`. + +- *NOTE:* Unless otherwise specified, all analyses in Section \@ref(mgCFA) use + these data. + +
    + Click to show code + +```{r eval = FALSE} +## Load the data: +pgd <- read.table("PGDdata2.txt", + na.strings = "-999", + header = TRUE, + sep = "\t") %>% + filter(!is.na(Kin2)) + +## Check the results: +head(pgd) +summary(pdg) +str(pgd) +``` + +```{r echo = FALSE} +pgd <- read.table(paste0(dataDir, "PGDdata2.txt"), + na.strings = "-999", + header = TRUE, + sep = "\t") %>% + filter(!is.na(Kin2)) + +head(pgd) +summary(pgd) +str(pgd) +``` + +
    + +--- + +#### {#oneGroupCFA} + +Run a single-group CFA wherein the five scale variables described above indicate +a single latent factor. + +- Do not include any grouping variable. +- Use the default settings in the `cfa()` function. + +
    + Click to show code + +```{r} +## Define the model syntax: +cfaMod <- 'grief =~ b1pss1 + b2pss2 + b3pss3 + b4pss4 + b5pss5' + +## Estimate the model: +out0 <- cfa(cfaMod, data = pgd) +``` + +
    + +--- + +#### + +Summarize the evaluate the fitted CFA + +- Does the model fit well? +- Are the items homogeneously associated with the latent factor? +- Which item is most weakly associated with the latent factor? + +
    + Click to show code + +```{r} +## Summarize the fitted model: +summary(out0, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE) +``` + +```{r, echo = FALSE} +fit0 <- fitMeasures(out0, c("chisq", "df", "pvalue", "rmsea", "cfi", "tli", "srmr")) + +chi2 <- fit0["chisq"] %>% round(2) +df <- fit0["df"] +p <- fit0["pvalue"] %>% round(3) +rmsea <- fit0["rmsea"] %>% round(3) +cfi <- fit0["cfi"] %>% round(3) +srmr <- fit0["srmr"] %>% round(3) + +r2 <- inspect(out0, "r2") +ly <- inspect(out0, "std.all")$lambda + +bad <- r2 %>% which.min() %>% names() + +r2 <- r2[bad] %>% round(3) %>% as.numeric() +ly <- ly[bad, ] %>% round(3) %>% as.numeric() +``` + +
    + Click for explanation + +- The model fits the data quite well ($\chi^2[`r df`] = `r chi2`$, +$p = `r p`$, +$\textit{RMSEA} = `r rmsea`$, +$\textit{CFI} = `r cfi`$, +$\textit{SRMR} = `r srmr`$). +- All of the indicators appear to be more-or-less equally good indicators of the + latent factor except for `r bad` which has a standardized factor loading of + $\lambda = `r ly`$ and $R^2 = `r r2`$. + +
    +
    + +--- + +#### {#twoGroupCFA} + +Rerun the CFA from \@ref(oneGroupCFA) as a multiple-group model. + +- Use the `Kin2` variable as the grouping factor. +- Do not place any equality constraints across groups. + +
    + Click to show code + +```{r} +out1 <- cfa(cfaMod, data = pgd, group = "Kin2") +``` + +
    + +--- + +#### + +Summarize the fitted multiple-group CFA from \@ref(twoGroupCFA). + +- Does the two-group model fit the data well? +- Do you notice any salient differences between the two sets of within-group + estimates? + +
    + Click to show code + +```{r} +summary(out1, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE) +``` + +```{r, echo = FALSE} +fit1 <- fitMeasures(out1, c("chisq", "df", "pvalue", "rmsea", "cfi", "tli", "srmr")) + +chi2 <- fit1["chisq"] %>% round(2) +df <- fit1["df"] +p <- fit1["pvalue"] %>% round(3) +rmsea <- fit1["rmsea"] %>% round(3) +cfi <- fit1["cfi"] %>% round(3) +srmr <- fit1["srmr"] %>% round(3) +``` + +
    + Click for explanation + +- The two-group model also fits the data very well ($\chi^2[`r df`] = `r chi2`$, +$p = `r p`$, +$\textit{RMSEA} = `r rmsea`$, +$\textit{CFI} = `r cfi`$, +$\textit{SRMR} = `r srmr`$). +- No, there are no striking differences between the two sets of estimates. Although, + there is certainly some variability between groups, the two sets of estimates + don't look systematically different. + +
    +
    + +--- + +#### + +Based on the above results, what can you conclude about configural, weak, and +strong measurement invariance across the `Kin2` groups? + +
    + Click for explanation + +- Configural invariance holds. The unrestricted multiple-group CFA fits the data + adequately (very well, actually), and the measurement model parameters are + reasonable in both groups. +- We cannot yet draw any conclusions about weak or strong invariance. We need to + do the appropriate model comparison tests first. + +
    + +--- + +End of At-Home Exercises 7 + +--- + +[sesam2_data]: https://surfdrive.surf.nl/files/index.php/s/dfzC7Tf5HHiTX8M/download +[pgd_data]: https://surfdrive.surf.nl/files/index.php/s/xxkp4gZY682AGyY/download +[boelen_et_al_2010]: https://doi.org/10.1016/j.jad.2010.01.076 diff --git a/week7.Rmd b/week7.Rmd index 4d6695a4..271e765c 100644 --- a/week7.Rmd +++ b/week7.Rmd @@ -23,8 +23,7 @@ During the practical you will work on the [In-Class Exercises](in-class-exercise ```{r, echo = FALSE, message = FALSE, warning = FALSE, results = "asis"} knit_child(paste0(partDir, "lecture.Rmd"), quiet = TRUE) %>% cat(sep = "\n") knit_child(paste0(partDir, "reading.Rmd"), quiet = TRUE) %>% cat(sep = "\n") -knit_child(paste0(partDir, "../home-placeholder.Rmd"), quiet = TRUE) %>% - cat(sep = "\n") +knit_child(paste0(partDir, "home.Rmd"), quiet = TRUE) %>% cat(sep = "\n") knit_child(paste0(partDir, "../class-placeholder.Rmd"), quiet = TRUE) %>% cat(sep = "\n") ```