R：高于和低于基准的值填0

Question

I'm currently writing my master thesis and when I made a regression I found out that I have some outliers which I would like to either delete or fill in a zero. I got a dataframe with company names and their daily returns from 2010 until 2021.

The dataframe is called xsr. I want to find the outliers which are above 0.5 and below -0.5. I managed to create a dataframe according to this condition xsr_short <- xsr[,c(2:214)] <0.5 . Then I tried to pick the false values outliers <- subset(xsr_short, xsr_short = FALSE) . Which just gives me back the initial xsr_short .

I also tried it with the select command: xsr_short <- select(xsr, c('ABBN SW Equity':'ZWM SW Equity') < 0.5) . The output to this is:

    Error in `select()`:
! NA/NaN argument
Backtrace:
  1. dplyr::select(xsr, c("ABBN SW Equity":"ZWM SW Equity") < 0.5)
 22. base::.handleSimpleError(`<fn>`, "NA/NaN argument", base::quote("ABBN SW Equity":"ZWM SW Equity"))
 23. rlang (local) h(simpleError(msg, call))
 24. handlers[[1L]](cnd)
Warning messages:
1: In eval_tidy(expr, context_mask) : NAs introduced by coercion
2: In eval_tidy(expr, context_mask) : NAs introduced by coercion

I need to fill in the second condition > -0.5 and then delete the values that are out of this range.

Thank you very much in advance for your help and your time!

Answer 1

It seems like you are less concerned with an actual subset but rather just switching out unwanted values in your data while preserving what you have for the regression. In that case, the tidyverse package may be helpful. First, you can load this package as well as this fake dataset:

#### Load Tidyverse ####
library(tidyverse)

#### Make Data Frame ####
data <- data.frame(IV = c("Control","Treatment",
                          "Control","Treatment"),
                   DV = c(-9999,2,4,5555))
data

Which gives you this:

         IV    DV
1   Control -9999
2 Treatment     2
3   Control     4
4 Treatment  5555

From there you can simply use mutate and ifelse to remove the unwanted values and replace then with NA missing values with this code, saving the data into a new version with the replacement values:

#### Swap Outliers with NA Values ####
clean.data <- data %>% 
  mutate(DV = ifelse(DV < 0,
                     NA,
                     ifelse(DV > 100,
                            NA,
                            DV)))
clean.data

Which gives you this:

       IV DV
1   Control NA
2 Treatment  2
3   Control  4
4 Treatment NA

As some others have noted, its generally bad practice to delete outliers in your data unless you have a defensible reason to do so. So if you do remove them, make sure you have something justifiable to include in your thesis that explains why you removed the values.