I m searching to build up a course, which collects data from 6 different websites and exhibiting information which changes dynamically.

To be able to develop such program I need a screen scraper utility which has the capacity to extract data concurrently from 6 different Web pages which the information changes in a high frequency.

Begin to see the link below to obtain a concept of the kind and also the dynamic way the information is displayed.


The data is displayed somewhat in a different way around the 6 web pages, however the concept is extremely similar.

Can anybody please advise what's the correct utility software which is fantastic for such purpose.

Thank you

Ethically screen scraping could be a dubious practice - but you will find legitimate uses. Search Engines Like Google perform a type of scrape to have their results, for example.

Some sites will block utilities which make a lot of demands - the industry valid reason to request permission if you are planning to create many demands - as it might otherwise seem like a DoS attack and also the site may do something against you.

Everything aside for those who have the best use to see the items in a webpage (to ensure that you are able to process it in some manner) the code is rather trivial.


In C#:

// Initialize the WebRequest.
WebRequest myRequest = WebRequest.Create("http://www.contoso.com");

// Return the response. 
WebResponse myResponse = myRequest.GetResponse();

// Code to use the WebResponse goes here.

// Close the response to free resources.