如何在R中整理这个凌乱的数据集

Question

I'm pretty new to using tidyr , dplyr , etc, and I have some data I can't figure out how to tidy in R .

Variables are mixed up in rows and columns, and the spreadsheet looks like it's split up so there are different kinds of information on the top rows and the bottom rows.

A simplified version of it is below.

You can imagine this is an exam with 4 questions:

• The top few rows give some information about each question
• The bottom rows show whether various students (given by their IDNum ) got the questions correct ( 1 ) or wrong ( 0 ).

Here is the raw data:

Question    Q1         Q2         Q3         Q4
Topic       English    English    Math       Math
Subtopic    Grammar    Vocabulary Algebra    Geometry
Difficulty  2          4          3          4
IDNum               
512         1          1          1          0
102         0          1          0          1
321         1          1          1          1
246         1          1          0          1
248         1          0          1          0
136         1          1          1          1
290         0          1          1          1
753         1          0          0          0
752         1          0          1          1

I'd like to tidy this data set. It would look something like the following:

IDNum   Question    Topic   Subtopic    Difficulty  Correct
512     Q1          English Grammar     2           1
512     Q2          English Vocabulary  4           1
512     Q3          Math    Algebra     3           1
512     Q4          Math    Geometry    4           0
102     Q1          English Grammar     2           0
102     Q2          English Vocabulary  4           1
102     Q3          Math    Algebra     3           0
102     Q4          Math    Geometry    4           1
321     Q1          English Grammar     2           1
321     Q2          English Vocabulary  4           1
321     Q3          Math    Algebra     3           1
321     Q4          Math    Geometry    4           1

and so on.

Thank you!

Answer 1

It's not entirely clear in which format you have the data but hopefully the following will help:

data

df <- read.table(text="
Question    Q1         Q2         Q3         Q4
Topic       English    English    Math       Math
Subtopic    Grammar    Vocabulary Algebra    Geometry
Difficulty  2          4          3          4
IDNum       ''        ''          ''         ''
512         1          1          1          0
102         0          1          0          1
321         1          1          1          1
246         1          1          0          1
248         1          0          1          0
136         1          1          1          1
290         0          1          1          1
753         1          0          0          0
752         1          0          1          1",h=F,strin=F)

solution

library(tidyverse)
df %>%
  # collapse the first rows into column names to prepare for gather/separate combo
  setNames(apply(.[1:4,],2,paste,collapse="|")) %>% 
  rename_at(1,~"IDNum")   %>%
  # remove useless rows
  slice(-(1:5))           %>%
  # change IDNum to factor, only useful if the order of IDNum is important (probably it's not)
  mutate_at("IDNum",~factor(.x,levels=unique(.x))) %>%
  # wide to long
  gather(key,correct,-1)  %>%
  # build your columns (convert to TRUE so Difficulty will be numeric)
  separate(key,df[1:4,1],convert = TRUE) %>%
  # convert correct to numeric
  mutate_at("correct",as.numeric) %>%
  # sort
  arrange(IDNum)

# # A tibble: 36 x 6
#     IDNum Question   Topic   Subtopic Difficulty correct
#    <fctr>    <chr>   <chr>      <chr>      <int>   <dbl>
#  1    512       Q1 English    Grammar          2       1
#  2    512       Q2 English Vocabulary          4       1
#  3    512       Q3    Math    Algebra          3       1
#  4    512       Q4    Math   Geometry          4       0
#  5    102       Q1 English    Grammar          2       0
#  6    102       Q2 English Vocabulary          4       1
#  7    102       Q3    Math    Algebra          3       0
#  8    102       Q4    Math   Geometry          4       1
#  9    321       Q1 English    Grammar          2       1
# 10    321       Q2 English Vocabulary          4       1
# # ... with 26 more rows

Another way, with a few more steps but maybe more intuitive, would be to separate from the start the header and the core of the table.

We create a lookup from the header (that we transpose), and we'll use it on the gathered data later:

header_lkp <-
  as_tibble(t(df[1:4,])) %>%
  setNames(.[1,]) %>%
  slice(-1)

df_core <-
  df %>%
  setNames(.[1,]) %>%
  slice(-(1:5))   %>%
  rename_at(1,~"IDNum") %>%
  mutate_at("IDNum",~factor(.x,levels=unique(.x)))

df_core %>%
  gather(Question,correct,-IDNum) %>%
  mutate_at("correct",as.numeric) %>%
  left_join(header_lkp,by="Question") %>%
  arrange(IDNum)

(same output)