Hi,
I got a python webcrawler/scraper written using phantomjs and such. I would need someone who can optimize this with a short notice, preferably within some hours. And that can operate the script on servers provided by me, or if you would like to set it up yourself I can compensate for that depending on the price for it. Today the scraper can retrieve about 2500-3000 items an hour, and there is about 300 000 unique items total with duplicates it is around 16 million rows in csv. I would need this in less then three days, maybe three. The script I got might have som bugs as it is not the latest version. So if there is some improvements that can be done, and of course running multiple instances on the same/different server. What ever works for me. But I would like you to be operating the server, and making sure everything is running.
Hope someone will be interested.
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi
I'm a web scraper/extractor specialist. I work with my own python scripts and with known frameworks such as scrapy and selenium webdriver with chromedriver and also phanthomJS.
I've worked in a similar project, setting up the script on a vps and scraping the site www.zulutrade.com. I cannot guarantee that the quantity will be accomplished in less than three days, it will depend on the capacity of the server and quantity of the instances that can be run..