En cours

Multi-Thread PHP Bing URL Scraper

I need a [url removed, login to view] URL scraper and I want it to be fast as possible.

Scraper must:

1. Use proxies.

I will be using a list of 200 proxies. Proxies must be used in order so we can get the biggest time between a proxy being used again.

2. Be Multi-Threaded. I want it to support 10 threads, 20 threads and 30 threads . I want each thread to use one unique IP but in order. Example: If I use 20 threads: First 20 IPs from list then 21-40 etc.

3. Make Bing believe is browser search, delete cookies and any trace after each IP use.

4. Must have option to set the thread timeout for each search to avoid proxy banning

5. Ability to support any type of query, especially this one:

site:[url removed, login to view] intitle:"keyword"

6. Load unlimited keywords.

7. Grab all the URLs from results for each keyword. Full clickable URL with https:// in front.

8. The script needs to handle any potential errors and try again with different IP the same keyword that failed.

9. Needs to run on a Win7 x64 machine.

10. Export the harvested URLs in both, .txt and .csv.

.txt: all the harvested URLs one URL on each row.

.csv: in one column the keyword and in one column the harvested URLs for that keyword, all the keywords URLs in one file. I want it to show only the keyword not the phrase that was searched on Bing. So only what's in the quotation marks.

If something is not clear please go ahead and ask, no point in losing time working for something I don't need. Thanks!

Compétences : .NET, Programmation C#, PHP, Web Scraping

Voir plus : example website quotation

Concernant l'employeur :
( 8 commentaires ) Dragasani, Romania

N° du projet : #8523603

Décerné à :


I am an expert in delivering custom software and scripts willing to discuss further about the project needs

400 $ USD en 3 jours
(216 Commentaires)

10 freelance ont fait une offre moyenne de 1660 $ pour ce travail


Greetings, I have gone through your complete project requirements to create a desktop software for scraping the URLs from bing search engine using Proxies with different time intervals that can be manage custom opti Plus

3000 $ USD en 35 jours
(84 Commentaires)

Hello there, This is our latest scrape job for movie /tv show site and it was developed as a wordpress plugin http://screencast.com/t/rjFkOFcy1 http://screencast.com/t/Jpg4o4v7o5B http://screencast.com/t/nTYvf0 Plus

470 $ USD en 6 jours
(96 Commentaires)

i have 8+ years of experience. Can we discuss the project. Please initiate a chat with me so that we can discuss the project at a broader level. Why you should hire me- 1. I have a very good communication skills Plus

1176 $ USD en 10 jours
(34 Commentaires)

Hi, Greetings !! We can surely meet your requirements for PHP bing Scrapping. We are a team of 25 full time, dedicated developers having expertise on native iOS & Android app development, Core PHP, Joomla, Mage Plus

2224 $ USD en 29 jours
(98 Commentaires)

A proposal has not yet been provided

3529 $ USD en 30 jours
(109 Commentaires)

Hi, I have great experience in website data extraction. i have done the extraction of many sites like citysearch.com,yelp.com,space4lease.com,goldenpages.ie,tigerdirect.com,amazon.com and many more i have read th Plus

588 $ USD en 10 jours
(330 Commentaires)

Hi! It's good to see your new serious job posting on here! Freelancer support team introduce me to your project , because of I am a Preferred Freelancer in this site. I read all of your project description Plus

1094 $ USD en 10 jours
(45 Commentaires)

Hi, I am ready to do this for you as per according to your requirements. I have lots of experience in doing such kind of projects. You can check my profile for my work experience. Thanks

1764 $ USD en 10 jours
(141 Commentaires)

A proposal has not yet been provided

2352 $ USD en 30 jours
(40 Commentaires)