Remove duplicates from each row in R dataframe

Question

I have a dataframe with 1209 columns, and 27900 rows.

For each row there are duplicated values scatter around the columns. I have tried transposing the dataframe and remove by columns. But it crashes.

After I transpose I used:

for(i in 1:ncol(df)){

        #replicate column i without duplicates, fill blanks with NAs
        df <-  cbind.fill(df,unique(df[,1]), fill = NA)
        #rename the new column
        colnames(df)[n+1] <- colnames(df)[1]
        #delete the old column
        df[,1] <- NULL
}

But no result so far.

I would like to know if anyone has any idea.

Best

Answer 1

As I understand you would like to replace duplicated values in each column with NA?

this can be done in several ways.

First some data:

set.seed(7)
df <- data.frame(x = sample(1: 20, 50, replace = T),
                 y = sample(1: 20, 50, replace = T),
                 z = sample(1: 20, 50, replace = T))
head(df, 10)
#output
    x  y  z
1  20 12  8
2   8 15 10
3   3 16 10
4   2 13  8
5   5 15 13
6  16  8  7
7   7  4 20
8  20  4  1
9   4  8 16
10 10  6  5

with purrr library:

library(purrr)
map_dfc(df, function(x) ifelse(duplicated(x), NA, x))
#output
# A tibble: 50 x 3
       x     y     z
   <int> <int> <int>
 1    20    12     8
 2     8    15    10
 3     3    16    NA
 4     2    13    NA
 5     5    NA    13
 6    16     8     7
 7     7     4    20
 8    NA    NA     1
 9     4    NA    16
10    10     6     5
# ... with 40 more rows

with apply in base R

as.data.frame(apply(df, 2, function(x) ifelse(duplicated(x), NA, x)))