How do I login to this website with C# to screen scrape it?

Question

Greetings, I have scraped thousands of pages, but for the first time I need to setup a scrape for screens that are behind a login screen:

https://wmars.cwmars.org/patroninfo/

Anyone care to give me a lead on what I would need to do, in C#, to pass in the credentials for the above screen and the get to the screen 'behind' it?

Thanks in advance.

Answer 1

Generally speaking, once you log in one of two things will happen. Either they will send a cookie back that contains everything needed for the site to ensure the user is logged in or they will pass the information back on a query string.

Usually all you need to do is capture the cookie with your http request object and post that cookie back on subsequent calls to the site in question.

In this case it looks like the cookies to capture are named SESSION_SCOPE and III_SESSION_ID. There might be more. Just use firebug to look at the cookie list when logging in to the site to know what you need.