[英]Best approach to combine multiple MySQL tables in R
在R中組合多個MySQL表的最佳方法是什么? 例如,我需要重新rbind
14個大型的MySQL表(每個> 100k行乘100列)。 我嘗試了以下方法,該方法消耗了我的大部分內存,並且使MySQL超時。 我想知道是否有替代解決方案? 我不需要獲取整個表,只需要通過幾個變量對整個表進行分組並計算一些指標。
station_tbl_t <- dbSendQuery(my_db, "select * from tbl_r3_300ft
union all
select * from tbl_r4_350ft
union all
select * from tbl_r5_400ft
union all
select * from tbl_r6_500ft
union all
select * from tbl_r7_600ft
union all
select * from tbl_r8_700ft
union all
select * from tbl_r9_800ft
union all
select * from tbl_r10_900ft
union all
select * from tbl_r11_1000ft
union all
select * from tbl_r12_1200ft
union all
select * from tbl_r13_1400ft
union all
select * from tbl_r14_1600ft
union all
select * from tbl_r15_1800ft
union all
select * from tbl_r16_2000ft
")
考慮迭代導入MySQL表數據,然后使用R進行行綁定。並確保選擇所需的列以節省開銷:
tbls <- c("tbl_r3_300ft", "tbl_r4_350ft", "tbl_r5_400ft",
"tbl_r6_500ft", "tbl_r7_600ft", "tbl_r8_700ft",
"tbl_r9_800ft", "tbl_r10_900ft", "tbl_r11_1000ft",
"tbl_r12_1200ft", "tbl_r13_1400ft", "tbl_r14_1600ft",
"tbl_r15_1800ft", "tbl_r16_2000ft")
sql <- "SELECT Col1, Col2, Col3 FROM"
dfList <- lapply(paste(sql, tbls), function(s) {
tryCatch({ return(dbGetQuery(my_db, s))
}, error = function(e) return(as.character(e)))
})
# ROW BIND VERSIONS ACROSS PACKAGES
master_df <- base::do.call(rbind, dfList)
master_df <- plyr::rbind.fill(dfList)
master_df <- dplyr::bind_rows(dfList)
master_df <- data.table::rbindlist(dfList)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.