We are using Chrome extension [login to view URL] to scrap a web site. It includes two scraping sitemaps (in webscraper's terminology), one to retrieve a list of all products from a web store, and another one to visit pages of new products and extract the actual information we need. Both sitemaps are running fine, but they are fired manually, and the list of new products URLs is also set manually.
So I want a script to run in a Windows machine (probably a laptop) to do this:
1) Run the first sitemap.
2) Read the results wherever they are stored, and send them via HTTP POST to our server.
3) Later, poll the server and get a list of URLs to visit - about a hundred each time.
4) Edit the webscraper's sitemap JSON file to include that list of URLs (if any).
5) Run webscraper with the new sitemap.
6) Read the results and send again to the server via HTPP POST.
7) Repeat from point 3 until the next day, then repeat from point 1.
The scripts also can be run manually, for testing.
If possible, the new scripts, or macros, or whatever, should run in a standard Windows machine. And have to be editable, inform about what is happening, and have a verbose mode for debugging.
6 freelance font une offre moyenne de €206 pour ce travail
Hi, Dear Client. I am expert Data Miner and Site Crawler. You can check my skills from my work history and reviews. I can make that perfectly. Thanks
I am a python developer. I have great experience in web scraping and I am an expert in it. I have all necessary skills by which I can scrape any website. I have even scraped sites like google, whatsapp web, etc. which Plus
Hello Sir We are expert in web scrapping using java and selenium web driver (chrome/Firefox) will write a software for you which will full fill your all requirements and also we will set up at your local. Please ping Plus
Hello dear, I have read your project description carefully and i know you want to automate [login to view URL] scraping tasks in 7 steps, In touch with your server HTTP APIs. My last scraping project on Freelancer was Plus
Hello, I'm very interested in your project and can meet your satifactory requirements. I have extensive experience in both web scraping and process automation. The solution would be developed using the python program Plus
Hi, I have full experience and expertise knowledge about web scraping, web bot using python and chrome extension. I have create many bots using python and chrome extension. I have made [login to view URL] file by python script th Plus