Stagecoach Software Pty Ltd is developing a website to contain statistical profiles of a particular industry in Australia. Our stack is Ruby on Rails 4.2, Ruby 2.2, Passenger 4 and MySQL 5.5. We run a 3 server configuration on Engine Yard with dedicated instances for our db, web app and utilities. Our database receives about 30,000 new rows per day through scraping operations (22 different scrapers using nokogiri). Presently our app instance only provides admin operations; the consumer front end is under development.
Currently our team comprises 4 backend ruby developers and 1 front end developer.
Each of our scrapers records the URL of each relevant product. We wish to record the last date each product is advertised for sale. We therefore need 11 nokogiri micro scrapers developed that will log on to the URL of each current product and insert the current date into a designated column if the advertisement is still current.
Each scraper must run as a discrete *.rb file and be capable of running as a cron. Code must be included to make each scraper record start and stop times and number of columns populated on an admin page that is already doing the same for all existing cron jobs. Further instructions will be given to the successful bidder.
All code must be uploaded to github and approved by the lead developer before deployment.
This is one of a pipeline of projects in this enterprise. Further ruby projects will be available if the above tasks are carried out well. Applications from a freelancer experienced with Engine Yard are welcomed.
10 freelance ont fait une offre moyenne de 478 $ pour ce travail
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi