I will write a multithreaded, command-line Web client that can fetch data from multiple URLs concurrently.
I will write this program quickly and I will provide an option to have it read the URLs from a file (in addition to the ability to simply process an array). The number of sites that the program accesses concurrently, the agent name of the program, and the time-out period will be all configurable. The program will be able to deal with cookies and also with authentication, should that become necessary. Finally, the end user will have the option of writing the output to disk (in addition to simply keeping it in an array).
My program will be free of bugs or I will perform any additional work that relates to fixing bugs free of charge.
The the program will be a stand-alone program, it will contain a grab function that performs the job that you describe in your RFP.
I will use the LWP::UserAgent, threads, and threads::shared modules, all part of core Perl. The only requirement of the program is that Perl be built with threads and that the version of Perl be reasonably recent (2 or 3 years old).
Thanks,
--Donnie