从html源将XML数据读入R

Question

I'd like to import data into R from a given webpage, say this one .

In the source code (but not on the actual page), the data I'd like to get is stored in a single line of javascript code which starts like this:

chart_Line1.setDataXML("<graph rotateNames (stuff omitted) >
<set  value='699.99' name='16.02.2013'  />
<set  value='731.57' name='18.02.2013'  />  
<set  value='more values' name='more dates'  />
...
<trendLines> (now a different command starts, stuff omitted)
</trendLines></graph>")

(Note that I've included line breaks for readability; the data is in one single line in the original file. It would suffice to import only the line which starts with chart_Line1.setDataXML - it's line 56 in the source if you want to have a look yourself)

I can read the whole html file into a string using scan("URLofFile", what="raw") , but how do I extract the data from this?

Can I specify the data format with what="..." , keeping in mind that there are no line breaks to separate the data, but several line breaks in the irrelevant prefix and suffix?

Is this something which can be done in a nice way using R tools, or do you suggest that this data acquisition should rather be done with a different script?

Answer 1

With some trial & error, I was able to find the exact line where the data is contained. I read the whole html file, and then dispose of all other lines.

require(zoo)
require(stringr)
# get html data, scrap all lines but the interesting one
theurl <- "https://www.magickartenmarkt.de/Black_Lotus_Unlimited.c1p5093.prod"
sec <- scan(file =theurl, what = "character", sep="\n")
sec <- sec[45]
# extract all strings of the form "value='X'", where X is a 1 to 3 digit number with some separator and 2 decimal places
values <- str_extract_all(sec, "value='[0-9]{1,3}.[0-9]{2}'")
# dispose of all non-numerical, non-separator values
values <- str_replace_all(unlist(values),"[^0-9/.]","")
# get all dates in the form "name='DD.MM.YYYY"
dates <- str_extract_all(sec, "name='[0-9]{2}.[0-9]{2}.[0-9]{4}'")
# dispose of all non-numerical, non-separator values
dates <- str_replace_all(unlist(dates),"[^0-9/.]","")
# convert dates to canonical format
dates <- as.Date(dates,format="%d.%m.%Y")
# put values and dates into a list of ordered observations, converting the values from characters to numbers first.
MyZoo <- zoo(as.numeric(values),dates)