I need a [url removed, login to view] URL scraper and I want it to be fast as possible.
1. Use proxies.
I will be using a list of 200 proxies. Proxies must be used in order so we can get the biggest time between a proxy being used again.
2. Be Multi-Threaded. I want it to support 10 threads, 20 threads and 30 threads . I want each thread to use one unique IP but in order. Example: If I use 20 threads: First 20 IPs from list then 21-40 etc.
3. Make Bing believe is browser search, delete cookies and any trace after each IP use.
4. Must have option to set the thread timeout for each search to avoid proxy banning
5. Ability to support any type of query, especially this one:
site:[url removed, login to view] intitle:"keyword"
6. Load unlimited keywords.
7. Grab all the URLs from results for each keyword. Full clickable URL with https:// in front.
8. The script needs to handle any potential errors and try again with different IP the same keyword that failed.
9. Needs to run on a Win7 x64 machine.
10. Export the harvested URLs in both, .txt and .csv.
.txt: all the harvested URLs one URL on each row.
.csv: in one column the keyword and in one column the harvested URLs for that keyword, all the keywords URLs in one file. I want it to show only the keyword not the phrase that was searched on Bing. So only what's in the quotation marks.
If something is not clear please go ahead and ask, no point in losing time working for something I don't need. Thanks!
Décerné à :
9 freelance ont fait une offre moyenne de 1723 $ pour ce travail
Hi, I have great experience in website data extraction. i have done the extraction of many sites like [url removed, login to view],[url removed, login to view],[url removed, login to view],[url removed, login to view],[url removed, login to view],[url removed, login to view] and many more i have read th Plus
Hi, I am ready to do this for you as per according to your requirements. I have lots of experience in doing such kind of projects. You can check my profile for my work experience. Thanks