Ajinkya Mandhare: Simple Web Crawler

Showing posts with label Simple Web Crawler. Show all posts

Saturday, 21 January 2012

Tagged under: Code for connecting to a Web Page and Obtaining its Source Code, Crawler code in C#, featured, Simple Web Crawler, Spider Code

C# code for connecting to a Web Page & Obtaining its Source Code/ Web Crawler Algorithm in C#

HTTP is the primary mechanism for communicating with resources over the Web. It is a Stateless protocol, used for simple Request-Response communication. A developer may often want to obtain web pages & their source codes, for different reasons like: building a spider, obtaining info on a particular page, etc. For this purpose, the .NET Framework includes classes that aid in this respect.

Requesting & Obtaining an HTTP page: