R: How to read a csv file containing non-dataset information

Question

I have a .csv file that doesn't show line breaks in notepad. Notepad++ revealed LF characters at the end, but I can't figure out how to tell R to use that character as the line break or how to replace it with CRLF or \\n.

**Edit: here is an example file.

Answer 1

Using our fast, friendly file finagler:

library(data.table)

url <- 'https://dl.dropboxusercontent.com/u/8428744/Collaboration_vs_Publication_Year.csv'

# ignore first 14 rows per OP comment
df <-fread(url, skip = 14) # in this case, it works even without skip=

# put first 14 rows somewhere else
other_stuff <- readLines(url, n=14)

Warning message: In fread(" https://dl.dropboxusercontent.com/u/8428744/Collaboration_vs_Publication_Year.csv ") : Stopped reading at empty line 23 but text exists afterwards (discarded): "© 2015 Elsevier BV All rights reserved. SciVal ® is a registered trademark of Reed Elsevier Properties SA, used under license."

df
#                            V1 V2   V3   V4   V5   V6   V7   V8   V9  V10
# 1:           Brown University NA 0.80 0.84 0.81 0.79 0.79 0.79 0.76 0.64
# 2:        Columbia University NA 0.98 0.96 0.95 0.96 1.00 1.01 0.97 1.26
# 3:         Cornell University NA 0.94 0.92 0.93 0.95 0.93 0.98 0.94 1.26
# 4:          Dartmouth College NA 0.74 0.79 0.70 0.75 0.74 0.75 0.73 0.60
# 5:         Harvard University NA 1.08 1.05 1.06 1.10 1.09 1.10 1.08 0.97
# 6:       Princeton University NA 1.04 0.99 1.02 1.06 1.08 1.05 1.06 0.87
# 7: University of Pennsylvania NA 0.80 0.78 0.79 0.83 0.81 0.80 0.79 0.83
# 8:            Yale University NA 0.93 0.90 0.92 0.95 0.91 0.97 0.90 1.07

cat(other_stuff[nchar(other_stuff)>0], sep = '\n')
# Data set,Collaboration vs Publication Year
# Entities,"Brown University, Columbia University, Cornell University, Dartmouth College, Harvard University, Princeton University, University of Pennsylvania, Yale University"
# Year range,2010 to >2015
# Filtered by,"not filtered"
# Data source,Scopus
# Date last updated,16 October 2015
# Date exported,19 November 2015
# Metric name,Specific metric,Self-citations,Types of publications included,Other options
# Collaboration,International collaboration,-,"Articles, reviews and conference papers","field-weighted"
# Name,Tags,Collaboration,
# ,,Overall,2010,2011,2012,2013,2014,2015,>2015,

Answer 2

As you mentioned you wanted to keep all the data you can try the following. The source file is messy so this is not a fully automated solution and future files will need extra massaging.

myfile <- readLines("https://dl.dropboxusercontent.com/u/8428744/Collaboration_vs_Publication_Year.csv")

df1 <- read.csv(text=myfile, skip = grep("Overall", myfile) - 1)
df2 <- read.csv(text=myfile, nrows = grep("Overall", myfile) - 1, header = FALSE)
finaldf <- data.frame(df1[, colSums(is.na(df1)) != nrow(df1)], t(unstack(df2, V2 ~ V1)))[-nrow(df1), ]