How to create R output likes confusion matrix table

Question

I have two of directories. The name of first directory is "model" and the second directory is "test", the list of files in both of directories are same but have different content. The total number of files in both of directories also same, that is 37 files.

I show the example of content from one of file.

First file from model directory

Name file : Model_A5B45

                               data
1  papaya | durian | orange | grapes
2                             orange
3                             grapes
4                    banana | durian
5                             tomato
6                     apple | tomato
7                              apple
8                        mangostine 
9                         strawberry
10                strawberry | mango

dput output :

structure(list(data = structure(c(7L, 6L, 4L, 3L, 10L, 2L, 1L, 
5L, 8L, 9L), .Label = c("apple", "apple | tomato", "banana | durian", 
"grapes", "mangostine ", "orange", "papaya | durian | orange | grapes", 
"strawberry", "strawberry | mango", "tomato"), class = "factor")), .Names = "data", class = "data.frame", row.names = c(NA, 
-10L))

Second file in test directory

Name file: Test_A5B45

                               data
1                             apple
2            orange | apple | mango
3                             apple
4                            banana
5                            grapes
6                            papaya
7                            durian
8 tomato | orange | papaya | durian

dput output:

structure(list(data = structure(c(1L, 5L, 1L, 2L, 4L, 6L, 3L, 
7L), .Label = c("apple", "banana", "durian", "grapes", "orange | apple | mango", 
"papaya", "tomato | orange | papaya | durian"), class = "factor")), .Names = "data", class = "data.frame", row.names = c(NA, 
-8L))

I want to calculate the percentage of intersect and except data from files in directory test to files in directory model.

This is example of my code only for two of files (Model_A5B45 and Test_A5B45).

library(dplyr)

data_test <- read.csv("Test_A5B45")
data_model <- read.csv("Model_A5B45")
intersect <- semi_join(data_test,data_model)
except <- anti_join(data_test,data_model)
except_percentage <- (nrow(except)/nrow(data_test))*100
intersect_percentage <- (nrow(intersect)/nrow(data_test))*100
sprintf("%s/%s",intersect_percentage,except_percentage) 

Output : "37.5/62.5"

My question is, I want to implement my code to all of files (looping in both of directories) so the output will looks like confusion matrix.

Example of my expected output:

##             y
##              Model_A5B45       Model_A6B46    Model_A7B47
##   Test_A5B45     37.5/62.5          value         value
##   Test_A6B46      value             value         value
##   Test_A7B47      value             value         value

My answer:

I've create code that can process those thing, but I am still do not know how to make output looks like confusion matrix.

This is my code: (I don't know this is efficient or not, I use for loop)

f_performance_testing <- function(data_model_path, data_test_path){
  library(dplyr)
  data_model <- read.csv(data_model_path, header=TRUE)
  data_test <- read.csv(data_test_path, header=TRUE)
  intersect <- semi_join(data_test,data_model)
  except <- anti_join(data_test,data_model)
  except_percentage <- (nrow(except)/nrow(data_test))*100
  intersect_percentage <- (nrow(intersect)/nrow(data_test))*100

  return(list("intersect"=intersect_percentage,"except"=except_percentage))
}


for (model in model_list){
  for (test in test_list){
    result <- f_performance_testing(model,test)
    intersect_percentage <- round(result$intersect,3)
    except_percentage <- round(result$except,3)
    final_output <- sprintf("intersect : %s | except : %s",intersect_percentage,except_percentage) 
    cat(print(paste(substring(model,57),substring(test,56), final_output,sep=",")),file="outfile.txt",append=TRUE,"\n")
    print("Writing to file.......")
  }
}

The output is:

Model_A5B45,Test_A5B45, 37.5/62.5 
Model_A5B45,Test_A6B46, value
Model_A5B45,Test_A7B47, value
Model_A6B46,...... 
Model_A7B47,.....
...............
......
....

How can I transform this output as looks like confusion matrix table?

Answer 1

This won't answer your question directly, but hopefully gives you enough information to arrive at your own solution.

I would recommend creating a function like the following:

myFun <- function(model, test, datasource) {
  model <- datasource[[model]]
  test <- datasource[[test]]
  paste(rev(mapply(function(x, y) (x/y)*100, 
                   lapply(split(test, test %in% model), length), 
                   length(test))), 
        collapse = "/")
}

This function is to be used with a two-column data.frame , where the columns represent all the combinations of "test" and "model" values (why work with a data.frame structure when a character vector would suffice?)

Here's an example of such a data.frame (other sample data is found at the end of the answer).

models <- c("model_1", "model_2", "model_3")
tests <- c("test_1", "test_2", "test_3")
A <- expand.grid(models, tests, stringsAsFactors = FALSE)

Next, create a named list of your models and tests. If you've read your data in using lapply , it is likely you might have names to work with anyway.

dataList <- mget(c(models, tests))

Now, calculate the relevant values. Here, we can use apply to cycle through each row and perform the relevant calculation.

A$value <- apply(A, 1, function(x) myFun(x[1], x[2], dataList))

Finally, you reshape the data from a "long" form to a "wide" form.

reshape(A, direction = "wide", idvar = "Var1", timevar = "Var2")
#      Var1 value.test_1 value.test_2 value.test_3
# 1 model_1        75/25          100        75/25
# 2 model_2        50/50        50/50    62.5/37.5
# 3 model_3    62.5/37.5        50/50    87.5/12.5

---

Here's some sample data. Note that they are basic character vectors and not data.frame s.

set.seed(1)
sets <- c("A", "A|B", "B", "C", "A|B|C", "A|C", "D", "A|D", "B|C", "B|D")

test_1 <- sample(sets, 8, TRUE)
model_1 <- sample(sets, 10, TRUE)
test_2 <- sample(sets, 8, TRUE)
model_2 <- sample(sets, 10, TRUE)
test_3 <- sample(sets, 8, TRUE)
model_3 <- sample(sets, 10, TRUE)

In a real world application, you would probably do something like:

testList <- lapply(list.files(path = "path/to/test/files"),
                   function(x) read.csv(x, stringsAsFactors = FALSE)$data)
modelList <- lapply(list.files(path = "path/to/model/files"),
                   function(x) read.csv(x, stringsAsFactors = FALSE)$data)
dataList <- c(testList, modelList)

But, this is pure speculation on my part based on what you've shared in your question as working code (for example, csv files with no file extension).