How should I avoid 404 errors when scraping with R

Question

I'm accessing web pages by looping over a couple of variables to insert into the URL

There will be occasional 404 errors.

How do I insert some sort of catch for these pages to avoid breaking the code. I currently use the XML package but of course could load others if appropriate

TIA

Answer 1

Most of times I use RCurl::url.exists() . In case you have a list or a data frame containing all the urls you can try this:

map(p, ~ifelse(RCurl::url.exists(.), ., NA))

HTH!