根据面板长格式中的观察数将列单元格划分为不同的数字

Question

I have the following data which is of a panel structure. I need to normalize each cell so that the observation for a country is divided by total number of observations for that country divided by total number of observations in the panel structure (here 10 - in my data 1100). Also I have showcased three countries (AL, UK, FR) but I have 92 in total so I need some general formula (mutate: by = country?).

This is my data

df1 <- data_frame(Country = 
                    c("AL","AL","AL","AL","AL","AL","AL","AL","AL","AL",
                      "UK","UK","UK","UK","UK","UK","UK","UK","UK","UK",
                      "FR","FR","FR","FR","FR","FR","FR","FR","FR","FR"),
               Obs = c(NA,NA,2,3,2,3,2,3,2,NA,1,2,1,2,1,2,1,2,1,2,NA,NA,NA,NA,NA,NA,NA,NA,4,NA))
df1

 Country   Obs
   <chr>   <dbl>
 1 AL         NA
 2 AL         NA
 3 AL          2
 4 AL          3
 5 AL          2
 6 AL          3
 7 AL          2
 8 AL          3
 9 AL          2
10 AL         NA
11 UK          1
12 UK          2
13 UK          1
14 UK          2
15 UK          1
16 UK          2
17 UK          1
18 UK          2
19 UK          1
20 UK          2
21 FR         NA
22 FR         NA
23 FR         NA
24 FR         NA
25 FR         NA
26 FR         NA
27 FR         NA
28 FR         NA
29 FR          4
30 FR         NA

Now, what I want is to divide each cell with number of observations available for each country / total obs like so,

  df2 <-   data_frame(Country = 
                           c("AL","AL","AL","AL","AL","AL","AL","AL","AL","AL",
                             "UK","UK","UK","UK","UK","UK","UK","UK","UK","UK",
                             "FR","FR","FR","FR","FR","FR","FR","FR","FR","FR"),
                        Obs = c(NA,NA,2*7/10,3*7/10,2*7/10,3*7/10,2*7/10,3*7/10,2*7/10,
                               NA,1*10/10,2*10/10,1*10/10,2*10/10,1*10/10,2*10/10,1*10/10,
                                2*10/10,1*10/10,2*10/10,NA,NA,NA,NA,NA,NA,NA,NA,4*1/10,NA))

df2

  Country   Obs
   <chr>   <dbl>
 1 AL       NA  
 2 AL       NA  
 3 AL        1.4
 4 AL        3.7
 5 AL        2.7
 6 AL        3.7
 7 AL        2.7
 8 AL        3.7
 9 AL        2.7
10 AL       NA  
11 UK        1  
12 UK        2  
13 UK        1  
14 UK        2  
15 UK        1  
16 UK        2  
17 UK        1  
18 UK        2  
19 UK        1  
20 UK        2  
21 FR       NA  
22 FR       NA  
23 FR       NA  
24 FR       NA  
25 FR       NA  
26 FR       NA  
27 FR       NA  
28 FR       NA  
29 FR        0.4
30 FR       NA 

I am interested in solving the problem obviously BUT I would really really appreciate it if you could show me how to do this for multiple columns as my original data needs this same operation done for many columns where the country tickers (AL, UK, FR in example) remains the same.

Answer 1

You can do:

library(dplyr)

df1 %>%
  group_by(Country) %>%
  mutate(Obs = Obs * sum(!is.na(Obs))/n()) %>%
  ungroup

#  Country   Obs
#   <chr>   <dbl>
# 1 AL       NA  
# 2 AL       NA  
# 3 AL        1.4
# 4 AL        2.1
# 5 AL        1.4
# 6 AL        2.1
# 7 AL        1.4
# 8 AL        2.1
# 9 AL        1.4
#10 AL       NA  
# … with 20 more rows

sum(.is.na(Obs)) counts number of non-NA values in the Country whereas n() gives the number of rows for the Country .

For multiple columns -

df1 %>%
  group_by(Country) %>%
  mutate(across(col1:col4, ~. * sum(!is.na(.))/n())) %>%
  ungroup

This will be applied to col1 to col4 in your dataframe.

Answer 2

Using data.table

library(data.table)
setDT(df1)[, Obs := Obs * mean(!is.na(Obs)), County]

---

Or using dplyr

library(dplyr)
df1 %>%
  group_by(Country) %>%
  mutate(Obs = Obs * mean(!is.na(Obs)))