IMPORTANT: Please do not use placeholder bids and respond only with specific questions about my project. If your bid exceeds our limit, then do not bid.
We have a csv file containing several thousand tool model numbers. We want you to build a scraper that we can periodically use to scrape the individual URL for each product from 14 retailers and append the csv file with the URL's for each product.
If the product is not available on one of the source sites after attempting various combinations for the product model, then simply skip.
The 14 sites will be fixed but we'll need to be able to omit one or more of the retailer sites from collecting the data if necessary...a simple checkbox option next to the 14 retailers is all we'll require. That is...by default they are all checked and hence all scraped for the product URL however if one or more is unchecked, they will be excluded from the scraping process and hence results.
You'll need to factor in how to prevent the scraper from being blocked. For this, we'll provide a set of proxies - we'll need to be able to either add a list proxies or add a URL for rotating proxies. I would also like you to build some functionality that reports or logs when proxies are failing. Preferably, you should be able to offer advice as to the most suitable proxy provider. Most importantly, do not build for one type of proxy but give options.
It is important to note that some of the supplier sites below use CloudFlare making it difficult to scrape. You'll need to find a way around this.
We've included a sample csv file with product examples. So far for testing purposes, I have provided a list of about 60 items. In the final csv, there will be many thousands of products to scrape. Your scraper must run fast.
The structure of the input csv is as follows:
Product Title, Model Number, Brand, Category, Sub-Category, Product Image
The 14 retailers are listed here:
1) Total Tools - [login to view URL]
2) Just Tools - [login to view URL]
3) Vek Tools - [login to view URL]
4) Get Tools Direct - [login to view URL]
5) Power Tools Specialist - [login to view URL]
6) Bunnings - [login to view URL]
7) Discount Trader - [login to view URL]
8) Tool Mart - [login to view URL]
9) Melbourne Tools - [login to view URL]
10) Trade Tools - [login to view URL]
11) Australian Industrial Supplies - [login to view URL]
12) C L Tool Centre - [login to view URL]
13) Gasweld - [login to view URL]
14) Mitre 10 - [login to view URL]
My suggestion is to use the model Number in the search bar of the retailers below to find a suitable match for the product. If you have a better solution please advise.
Please be aware that should the first search for the item fail, you may need to use an alternate product number combination. For example, if you can't find D26500K-XE, then you should try D26500K(XE) or D26500KXE or something similar. It will be your job to determine the best way to find the products taking into consideration the unique way each site represents it's products.
Next, you'll need to modify a price comparison plugin we have built to allow it to import this csv data into the database. We will provide access to a copy of the site for you to review the code after acceptance of the project.
Time Frame: 2 weeks absolute maximum.
Any questions, please advise.
39 freelance font une offre moyenne de $618 pour ce travail
Hi, i have done a script to scrape 7 websitez from your [login to view URL] check my profile portfolio and you'll an excel with green/red boxes. I will recommend good proxy provider foe that that has auto rotating ip. Also re Plus
Dear sir, This is Lin, i did more than 380+ scraping project, please view my profile page here https://www.freelancer.com/u/mantislin.html then you will know. Can we discuss more details about your project? t Plus
Hi I understand your requirements and we can utilize proxies to avoid the blocking. I bid each site with 125 USD Can work on a demo before you award me. Can finish it in 5-7 days Thanks
Hello Simon, This is Vaishali from "Hire WordPress Experts" and I am here to help you with Website Scraper and Import Tool . We have gone through the information provided by you. and I can assure you that we are ex Plus
Hi I am very,very interested in your project My name is Shan Bin and I'm a Chinese developer. I have 6 years of web scraper development experience such this projects. And I have good skills with this program languag Plus
Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, PhantomJS, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database Plus
--^^-- Hello! ***************************** Nice to meet you! ***************************** I am a Web Scrap Professional developer. Scrapping and importing tools. I can meet your requirements fully in a short Plus
Hi, I can start work on your project immediately once you confirm me and i am also open for further discussion if you want. I have 8+ years in web design & development with good experience in following skills ... Plus
Dear scm1971, I do have relative experience achieving something like yours "Website Scraper and Import Tool" thusly I am expecting discuss and progress. Relevant Skills and Experience I have 10+ years of experi Plus
Hello sir, please reply soon , i will start work right now and complete all soon, please share website details, thanks
Hi Sir, I’m a Python Programmer. I have created python script for bunnings and totaltools for searching product sku's. as you know every website has unique structure. hence we used to analyse everywebsite paths and co Plus
Hello, I can scrape all the 14 retailer websites and get the date for you CSV file. As you said you will provide a set of proxies, I can rotate the proxies and use it with those websites which blocks us. There are se Plus
Hello. My name is Maciej. I am a Python developer and web scraping expert. I can create the scraper matching requirements for the price bidded. Please contact me if you are interested. Kind Regards Maciej J
*** I checked your description, website and attach file. I can build scraper tool. I want to discuss with you about this project. thanks.
Hi, I'm a full stack Web Designer and Developer. I am professional in Web Designing, having 12+ years experienced in Website Scraping and Importing Tools. I have see your attachment file, reference URL and expert Plus
Hello There, i go through with your all requirement and understand it very well, i make sure i will give you best result, I've experience with scrap websites, i will also build some functionality for reports and Plus