Data wrangling with dplyr

# Data wrangling with dplyr
## Amy Perfors
### pretty much taken from Danielle Navarro: <a href="https://djnavarro.github.io/chdss2018/day2/wrangling-slides.html" class="uri">https://djnavarro.github.io/chdss2018/day2/wrangling-slides.html</a>
### 18 December 2019

---

.reveal-text.bg-main2[.pad1[
.font4[Data wrangling]
]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[

# Data wrangling

.pull.left[.pad1[

### .orange[**What?**] 
### .orange[**Why?**] 
### .orange[**How?**]

]]

--
.column.bg-main3[.content.vmiddle.center[

[@allison_horst](https://twitter.com/allison_horst)

]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[

# Structure of this section

.pull.left[.pad1[

### This is interactive: don't just listen, code!
### Goal is to deal with real data

]]

.column.bg-main3[.content.vmiddle.center[

[@allison_horst](https://twitter.com/allison_horst)

]]

---

.reveal-text.bg-main2[.pad1[
.font5[The "sampling frames" experiment]
]]

---

# Property sampling: the robot only detects plaxium spheres
 
<img src="images/data10_property.jpg", width="70%">

---

# Category sampling: the robot only tests small spheres
 
<img src="images/data10_category.jpg", width="70%">

---

# Small sample size: Elicit judgments after two observations
 
<img src="images/data2_property.jpg", width="70%">

---

# Medium sample size: Elicit judgments after six observations
 
<img src="images/data6_property.jpg", width="70%">

---

# Large sample size: Elicit judgments after twelve observations
 
<img src="images/data12_property.jpg", width="70%">

---

# Seven test items that vary in size: Smallest...
 
<img src="images/test2.jpg", width="70%">

---

# Seven test items that vary in size: Largest...
 
<img src="images/test8.jpg", width="70%">

---
class: split-60 bg-main1

.column.bg-main1[.content.vmiddle.center[

# First let's pull all of the content for today

.pull.left[.pad1[

### Go to your chdss2019_content folder and type "git status"
### If you're not up to date, type "git pull"

### .white[Then copy the day3 folder to your CHDSS directory:]
### .white[Change directory to CHDSS/day3/]
### .white[You should see a file called data/frames_ex2.csv]

]]

.column.bg-main3[.content.vmiddle.center[

]]

---
class: split-60 bg-main1

.column.bg-main1[.content.vmiddle.center[

# First let's pull all of the content for today

.pull.left[.pad1[

### Go to your chdss2019_content folder and type "git status"
### If you're not up to date, type "git pull"

### Then copy the day3 folder to your CHDSS directory:
### Change directory to CHDSS/day3/
### You should see a file called data/frames_ex2.csv

]]

.column.bg-main3[.content.vmiddle.center[

]]

---

.reveal-text.bg-main2[.pad1[
.font4[Getting started]
]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[

# If you don't have the packages...

.pull.left[.pad1[

```r
install.packages("tidyverse")
install.packages("skimr")
```

]]

.column.bg-main3[.content.vmiddle.center[

]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# New R Markdown

.pull.left[.pad1[.font2[
- Create and call it initial.Rmd
- At the top is the YAML header
- This is the whole document...
]]]

]]

.column.bg-main3[.content.vmiddle.center[

.pull.left[.pad1[.font2[

````
---
title: "Exploring the data"
author: Amy Perfors
date: 18 December 2019
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
````

]]]
]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Write...

.pull.left[.pad1[.font2[
- Load packages in the first chunk
- Write descriptions for humans!
- Document, document, document
]]]

]]

.column.bg-main3[.content.vmiddle.center[

.pull.left[.pad1[.font2[

````
## Getting started

The first thing to do in our analysis is load the R
packages that we'll use to do the work:

```{r}
library(here)
library(tidyverse)
library(skimr)
```

The next step is to load the data set itself. The data 
from the sampling frames experiment are stored in the 
"frames_ex2.csv" file:

```{r}
loc <- here("data","frames_ex2.csv")
frames <- read_csv(file = loc)
```
````

]]]
]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# ... and knit!

.pull.left[.pad1[.font2[
- Note the output messages
- Helpful the first time...
- ... but eventually annoying
]]]

]]

.column.bg-main3[.content.vmiddle.center[

]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Tidy the code chunk

.pull.left[.pad1[.font2[
- Give it a name (like packageload)
- Suppress messages if you want
]]]

]]

.column.bg-main3[.content.vmiddle.center[

.pull.left[.pad1[.font2[
````
## Getting started

The first thing to do in our analysis is load the R
packages that we'll use to do the work:

```{r packageload, message=FALSE}
library(here)
library(tidyverse)
library(skimr)
```
````
]]]

]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Tidy the code chunk

.pull.left[.pad1[.font2[
- Give it a name (like packageload)
- Suppress messages if you want
]]]

]]

.column.bg-main3[.content.vmiddle.center[

]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Import the data

.pull.left[.pad1[.font2[
- Verbal description
- Specify the file location
- Import data with read_csv()
]]]

]]

.column.bg-main3[.content.vmiddle.center[

.pull.left[.pad1[.font2[

````
The next step is to load the data set itself. The data 
from the sampling frames experiment are stored in the 
"frames_ex2.csv" file:

```{r dataimport}
loc <- here("data","frames_ex2.csv")
frames <- read_csv(file = loc)
```
````

]]]
]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Import the data

.pull.left[.pad1[.font2[
- Verbal description
- Specify the file location
- Import data with read_csv()
]]]

]]

.column.bg-main3[.content.vmiddle.center[

]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Inspect the data set

.pull.left[.pad1[.font2[
- The data is a tibble
- Printing it shows the structure
- The glimpse() function is nice
- Name each code chunk!
]]]

]]

.column.bg-main3[.content.vmiddle.center[

.pull.left[.pad1[.font2[

````
The data set has been imported as a tibble. Let's take 
a quick look at the data. Each row in the data set is
a single response, and each column is a variable:

```{r printframes}
frames
```

An alternative way to view the a tibble that sometimes
looks nicer is to use the `glimpse()` function:

```{r glimpseframes}
glimpse(frames)
```
````

]]]
]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Inspect the data set

.pull.left[.pad1[.font2[
- The data is a tibble
- Printing it shows the structure
- The glimpse() function is nice
- Name each code chunk!
]]]

]]

.column.bg-main3[.content.vmiddle.center[

]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Skim over the data

.pull.left[.pad1[.font2[
- Gives means, std dev.
- Gives quantiles
- Even has little text histograms
- (Not always useful though!)
]]]
]]

.column.bg-main3[.content.vmiddle.center[
.pull.left[.pad1[.font2[
````
Finally, as a quick first pass, we can use the `skim()`
function to get a simple overview of each variable:

```{r skimframes}
skim(frames)
```
````

]]]
]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Skim over the data

.pull.left[.pad1[.font2[
- Gives means, std dev.
- Gives quantiles
- Even has little text histograms
- (Not always useful though!)
]]]

]]

.column.bg-main3[.content.vmiddle.center[

]]

---

.reveal-text.bg-main2[.pad1[
.font4[Exercise 1]
.font6[ day3/exercises/dplyr/wrangling1-task.Rmd]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[Introducing the pipe]
]]

---
class: split-40 bg-main1

.column.bg-main1[.content.vmiddle.center[

# The pipe, %>%

.pull.left[.pad1[.font2[
- Take the frames data...
- Do one thing...
- Then another...
- And then one more...
]]]
]]
--
.column.bg-main3[.content.vmiddle.center[
.pull.left[.pad1[.font2[
```{}
frames %>%
  do_one_thing(.) %>%
  then_another(.) %>%
  and_then_one_more(.)
```

]]]
]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# Regular code

```{}
do_one_thing(frames) 
```

]]]
]]
--
.column.bg-main3[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# The piped version

```{}
frames %>% do_one_thing() 
```

]]]
]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# Regular code

```{}
then_another(
  do_one_thing(
    frames
  ) 
)
```

]]]
]]

.column.bg-main3[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# The piped version

```{}
frames %>% 
  do_one_thing() %>%
  then_another()
```

]]]
]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# Regular code

```{}
and_then_one_more(
  then_another(
    do_one_thing(
      frames
    ) 
  )
)
```

]]]
]]

.column.bg-main3[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# The piped version

```{}
frames %>% 
  do_one_thing() %>%
  then_another() %>%
  and_then_one_more()
```

]]]
]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# Regular code

```{}
then_this(
  and_then_one_more(
    then_another(
      do_one_thing(
        frames
      ) 
    )
  )
)
```

]]]
]]

.column.bg-main3[.content.vmiddle.center[
.pull.left[.pad1[.font2[

# The piped version

```{}
frames %>% 
  do_one_thing() %>%
  then_another() %>%
  and_then_one_more() %>%
  then_this()
```

]]]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[Exercise 2]
.font6[ day3/exercises/dplyr/wrangling2-task.Rmd]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[group_by, summarise, ungroup]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# summarise()

.pull.left[.pad1[.font2[
- Group by variables
- Create summary stats for them
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
frames %>%
  group_by(test_item, sample_size, n_obs, condition) %>%
  summarise(response = mean(response))
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# summarise()

.pull.left[.pad1[.font2[
- Group by variables
- Create summary stats for them
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
frames %>%
  group_by(test_item, sample_size, n_obs, condition) %>%
  summarise(response = mean(response))
```

```
## # A tibble: 42 x 5
## # Groups: test_item, sample_size, n_obs [21]
## test_item sample_size n_obs condition response
## <dbl> <chr> <dbl> <chr> <dbl>
## 1 1 large 12 category 7.60
## 2 1 large 12 property 7.16
## 3 1 medium 6 category 7.32
## 4 1 medium 6 property 6.66
## 5 1 small 2 category 6.07
## 6 1 small 2 property 5.78
## 7 2 large 12 category 7.51
## 8 2 large 12 property 7.20
## 9 2 medium 6 category 7.17
## 10 2 medium 6 property 6.95
## # … with 32 more rows
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# ungroup!

.pull.left[.pad1[.font2[
- Get in the habit of doing this
- Avoids mysterious errors
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
frames %>%
  group_by(test_item, sample_size, n_obs, condition) %>%
  summarise(response = mean(response)) %>%
  ungroup()
```

```
## # A tibble: 42 x 5
## test_item sample_size n_obs condition response
## <dbl> <chr> <dbl> <chr> <dbl>
## 1 1 large 12 category 7.60
## 2 1 large 12 property 7.16
## 3 1 medium 6 category 7.32
## 4 1 medium 6 property 6.66
## 5 1 small 2 category 6.07
## 6 1 small 2 property 5.78
## 7 2 large 12 category 7.51
## 8 2 large 12 property 7.20
## 9 2 medium 6 category 7.17
## 10 2 medium 6 property 6.95
## # … with 32 more rows
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# ungroup!

.pull.left[.pad1[.font2[
- Get in the habit of doing this
- Avoids mysterious errors
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
frames %>% 
  group_by(test_item) %>%
  summarise(
    mean_resp = mean(response),
    sd_resp = sd(response),
    count = n()
  ) %>%
  ungroup()
```

```
## # A tibble: 7 x 4
## test_item mean_resp sd_resp count
## <dbl> <dbl> <dbl> <int>
## 1 1 6.77 2.56 675
## 2 2 6.88 2.10 675
## 3 3 5.71 2.41 675
## 4 4 4.48 2.68 675
## 5 5 3.76 2.81 675
## 6 6 3.43 2.99 675
## 7 7 3.26 3.11 675
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# pull()

.pull.left[.pad1[.font2[
- Same as $ in base R
- Lets you select a single variable
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
frames %>% 
  pull(test_item)
```

```
##    [1] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
##   [38] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
##   [75] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
##  [112] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
##  [149] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
##  [186] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
##  [223] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
##  [260] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
##  [297] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
##  [334] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
##  [371] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
##  [408] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
##  [445] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
##  [482] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
##  [519] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
##  [556] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
##  [593] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
##  [630] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
##  [667] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
##  [704] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
##  [741] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
##  [778] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
##  [815] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
##  [852] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
##  [889] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
##  [926] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
##  [963] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [1000] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [1037] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [1074] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [1111] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [1148] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [1185] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [1222] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [1259] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [1296] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [1333] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [1370] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [1407] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [1444] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [1481] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [1518] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [1555] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [1592] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [1629] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [1666] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [1703] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [1740] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [1777] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [1814] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [1851] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [1888] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [1925] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [1962] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [1999] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [2036] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [2073] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [2110] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [2147] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [2184] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [2221] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [2258] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [2295] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [2332] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [2369] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [2406] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [2443] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [2480] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [2517] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [2554] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [2591] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [2628] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [2665] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [2702] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [2739] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [2776] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [2813] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [2850] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [2887] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [2924] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [2961] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [2998] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [3035] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [3072] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [3109] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [3146] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [3183] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [3220] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [3257] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [3294] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [3331] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [3368] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [3405] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [3442] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [3479] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [3516] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [3553] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [3590] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [3627] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [3664] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [3701] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [3738] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [3775] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [3812] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [3849] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [3886] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [3923] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [3960] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [3997] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [4034] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [4071] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [4108] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [4145] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [4182] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [4219] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [4256] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [4293] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [4330] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [4367] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [4404] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [4441] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4
## [4478] 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6
## [4515] 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1
## [4552] 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3
## [4589] 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5
## [4626] 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
## [4663] 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2
## [4700] 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# pull()

.pull.left[.pad1[.font2[
- Same as $ in base R
- Lets you select a single variable
- ... And assign it
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
ti <- frames %>% 
 pull(test_item)
```

]]]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[Exercise 3]
.font6[ day3/exercises/dplyr/wrangling3-task.Rmd]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[arrange, filter, select, mutate]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# filter()

.pull.left[.pad1[.font2[
- Same as subset() in base R
- Lets you select a part of the dataset
- e.g. just the small sample size trials
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
average_response <- frames %>%
 group_by(test_item, sample_size, n_obs, condition) %>%
 summarise(response = mean(response)) %>%
 ungroup()

average_response %>%
  filter(sample_size == "small")
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# filter()

.pull.left[.pad1[.font2[
- Same as subset() in base R
- Lets you select a part of the dataset
- e.g. just the small sample size trials
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
average_response %>%
  filter(sample_size == "small")
```

```
## # A tibble: 14 x 5
## test_item sample_size n_obs condition response
## <dbl> <chr> <dbl> <chr> <dbl>
## 1 1 small 2 category 6.07
## 2 1 small 2 property 5.78
## 3 2 small 2 category 6.26
## 4 2 small 2 property 6.21
## 5 3 small 2 category 5.87
## 6 3 small 2 property 5.24
## 7 4 small 2 category 5.11
## 8 4 small 2 property 4.68
## 9 5 small 2 category 4.55
## 10 5 small 2 property 3.99
## 11 6 small 2 category 4.16
## 12 6 small 2 property 3.68
## 13 7 small 2 category 3.98
## 14 7 small 2 property 3.67
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# arrange()

.pull.left[.pad1[.font2[
- We might want to view it by condition
- arrange() sorts in different orders
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
average_response %>%
  filter(sample_size == "small") %>%
  arrange(condition)
```

```
## # A tibble: 14 x 5
## test_item sample_size n_obs condition response
## <dbl> <chr> <dbl> <chr> <dbl>
## 1 1 small 2 category 6.07
## 2 2 small 2 category 6.26
## 3 3 small 2 category 5.87
## 4 4 small 2 category 5.11
## 5 5 small 2 category 4.55
## 6 6 small 2 category 4.16
## 7 7 small 2 category 3.98
## 8 1 small 2 property 5.78
## 9 2 small 2 property 6.21
## 10 3 small 2 property 5.24
## 11 4 small 2 property 4.68
## 12 5 small 2 property 3.99
## 13 6 small 2 property 3.68
## 14 7 small 2 property 3.67
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# select()

.pull.left[.pad1[.font2[
- We might not want all of our variables
- select() picks only some of them
- Key: select() is columns, filter() is rows
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
average_response %>%
  filter(sample_size == "small") %>%
  arrange(condition) %>%
  select(test_item, condition, response)
```

```
## # A tibble: 14 x 3
## test_item condition response
## <dbl> <chr> <dbl>
## 1 1 category 6.07
## 2 2 category 6.26
## 3 3 category 5.87
## 4 4 category 5.11
## 5 5 category 4.55
## 6 6 category 4.16
## 7 7 category 3.98
## 8 1 property 5.78
## 9 2 property 6.21
## 10 3 property 5.24
## 11 4 property 4.68
## 12 5 property 3.99
## 13 6 property 3.68
## 14 7 property 3.67
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# Assignment

.pull.left[.pad1[.font2[
- Let's assign this to a variable
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
average_response_small <- average_response %>%
 filter(sample_size == "small") %>%
 arrange(condition) %>%
 select(test_item, condition, response)
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# mutate()

.pull.left[.pad1[.font2[
- Creates a new variable
- Rescales response to go from 0 to 1
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
average_response_small <- average_response_small %>%
 mutate(generalisation = response/9)
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# select()

.pull.left[.pad1[.font2[
- When used with a - indicates everything but
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
average_response_small <- average_response_small %>%
 mutate(generalisation = response/9) %>%
 select(-response)

average_response_small
```

```
## # A tibble: 14 x 3
## test_item condition generalisation
## <dbl> <chr> <dbl>
## 1 1 category 0.674
## 2 2 category 0.696
## 3 3 category 0.652
## 4 4 category 0.568
## 5 5 category 0.506
## 6 6 category 0.462
## 7 7 category 0.442
## 8 1 property 0.643
## 9 2 property 0.690
## 10 3 property 0.583
## 11 4 property 0.521
## 12 5 property 0.443
## 13 6 property 0.409
## 14 7 property 0.407
```

]]]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[Exercise 4]
.font6[ day3/exercises/dplyr/wrangling4-task.Rmd]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[Exercise 5]
.font6[ day3/exercises/dplyr/wrangling5-task.Rmd]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[gather and spread]
]]

---
class: split-two bg-main1

.column.bg-main1[.content.vmiddle.center[

]]

.column.bg-main3[.content.vmiddle.center[

[@allison_horst](https://twitter.com/allison_horst)

]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# spread()

.pull.left[.pad1[.font2[
- Separates a single column into two 
- Here: each condition is its own column
- Value is another variable
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
wide_avrs <- average_response_small %>%
 spread(key = condition, value = generalisation)
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# spread()

.pull.left[.pad1[.font2[
- Separates a single column into two 
- Here: each condition is its own column
- Value is another variable
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
wide_avrs <- average_response_small %>%
 spread(key = condition, value = generalisation)

wide_avrs
```

```
## # A tibble: 7 x 3
## test_item category property
## <dbl> <dbl> <dbl>
## 1 1 0.674 0.643
## 2 2 0.696 0.690
## 3 3 0.652 0.583
## 4 4 0.568 0.521
## 5 5 0.506 0.443
## 6 6 0.462 0.409
## 7 7 0.442 0.407
```

]]]
]]

---
class: split-30 bg-main1

.column.bg-main1[.content.vmiddle.center[

# gather()

.pull.left[.pad1[.font2[
- Combines two columns into one 
- The column names are put into the key
- The values are put into the value
]]]
]]

.column.bg-main3[.content.vtop.center[
.pull.left[.pad1[.font2[

```r
wide_avrs %>% gather(
  key = "condition", value = "generalisation", 
  category, property)
```

]]]
]]

---

.reveal-text.bg-main2[.pad1[
.font4[Exercise 6]
.font6[ day3/exercises/dplyr/wrangling6-task.Rmd]
]]

---

## Hooray!