How to remove insignificant variables in a correlogram matrix

Question

I have a dataset with 29 variables and I have tried to see how they are correlated using cor() .

This has given me a 29X29 matrix with the p-value for each product pair. Most of these correlations are insignificant, and I want only to retain the instances where the p-value is significant for 2 specific variables.

Here is a toy example, suppose that I want to maintain only the variables there are significant correlated with mpg , ig, cor_pmat(mpg, other_variables) < 0.05) .

library(ggcorrplot)
p.mat <- cor_pmat(mtcars)
corr <- round(cor(mtcars), 2)

Any hint on how can I do that?

Answer 1

Here is a function to select on the data frame:

library(dplyr)
library(rlang)
library(broom)

select_via_cor_sig <- function(.data, x, p.value, ...) {
  x <- rlang::ensym(x)

  .data %>%
    dplyr::select(-dplyr::all_of(x)) %>%
    names() %>%
    lapply(function(candidate) {
      c(rlang::as_string(x), candidate)
    }) -> ls_pairs
  
  ls_pairs %>% 
    lapply(function(vec_pair) {
      x <- .data[[vec_pair[1]]]
      y <- .data[[vec_pair[2]]]

      cor.test(x, y, ...) %>%
        broom::tidy() %>%
        dplyr::mutate(v1 = vec_pair[1], v2 = vec_pair[2]) %>%
        dplyr::select(v1, v2, dplyr::everything())
    }) %>%
    dplyr::bind_rows() -> tbl_tidy_cor_test
  
  tbl_tidy_cor_test %>% 
    dplyr::filter(p.value < {{p.value}}) %>%
    dplyr::pull(v2) %>% 
    c(rlang::as_string(x), .) -> keepers
  
  .data %>% 
    dplyr::select(dplyr::all_of(keepers))
}

# use it like so:
select_via_cor_sig(mtcars, mpg, 0.001)

If you want the p-value matrix, you could run it on the subset data frame produced by this function.