大数据集的市场桶分析

Question

I am creating a market bucket analysis on a large data set containing 2 columns (OrderID and Product). There are over a million rows in the set, and using the apriori packagage I was able to create an effective rules list using a smaller subset of the data, however when attempting to use the full set, I am not able to use the split function to aggregate the data by OrderID. Is there another function with similar functionality to split that can handle this much data? Code listed below:

MyData <- read.csv("C:/Market Basket Analysis/BOD16-Data.csv") #Abreviated for proprietary reasons
View(MyData)

library(arules)
summary(MyData)

#Using the split function, we are able to aggregate the transactions, so that each
#product on the transaction is grouped into its respective, singular, transID

start.time <- Sys.time() #Timer used to measure run time on the split function
aggregateData <- split(MyData$Product, MyData$OrderID)
end.time<- Sys.time()

time.taken = end.time- start.time
time.taken


#Using the split function, we are able to aggregate the transactions, so that each
#product on the transaction is grouped into its respective, singular, transID
aggregateData <- split(MyData$Product, MyData$OrderID)
head(aggregateData)

#Need to convert the aggregated data into a form that 'Arules' package 
#can accept
txns <- as(aggregateData, "transactions")
#txns <- read.transactions("Trans", format = "basket", sep=",", rm.duplicates=TRUE)
summary(txns)


#Apriori Algorithem generates the rules 
Rules <- apriori(txns,parameter=list(supp=0.0025,conf=0.4,target="Rules",minlen=2))
inspect(Rules)

EDIT: My data would be as follow:

OrderId     Product
1       1234
1       1357
1       2468
1       1324
2       1234
2       2468
3       4321
4       5432
5       1357

AggregateData should be:

[1]
1234,1357,2468,1324

[2]
1234, 2468

[3]
4321

[4]
5432

[5]
1357

Currently I am using the split function to achieve these results, but when applying it to a larger set the runtime exceeded 30 minutes before I stopped the script.

Answer 1

Is this any faster for you?

library(dplyr) 

df <- tribble(
    ~OrderId, ~Product,
    1,       1234,
    1,       1357,
    1,       2468,
    1,       1324,
    2,       1234,
    2,       2468,
    3,       4321,
    4,       5432,
    5,       1357
    )

    df %>% 
         group_by(OrderId) %>% 
         summarize(Product = list(Product)) %>% 
         mutate(Product = purrr::set_names(Product, OrderId)) %>% 
         pull(Product)

So for your code you should be able to do:

library(dplyr)

MyData <- read.csv("C:/Market Basket Analysis/BOD16-Data.csv")


 aggregateData <-   MyData %>% 
          group_by(OrderId) %>% 
          summarize(Product = list(Product)) %>% 
          mutate(Product = purrr::set_names(Product, OrderId)) %>% 
          pull(Product)

And that should be the same (and hopefully faster) as doing:

MyData <- read.csv("C:/Market Basket Analysis/BOD16-Data.csv")

aggregateData <- split(MyData$Product, MyData$OrderID)