277109 Daily Page scraper to MySQL

Stage 1 Scraping is to be performed on the nine sites listed below. It is assumed that this number will increase, so scraper and database need to be easily capable of adding other sites in the future. Create a MySQL database with individual tables for the freelance sites that appear below. In the first instance determine the fields required to capture all the projects and all employers scrape data into separate tables. Guru Elance GetACoder Get A Freelancer Rent A Coder Go Freelance oDesk ScriptLance Project4Hire Stage 2 Design MySQL Schema with translation tables. As an example categories in Elance are not the same as the other scraped sites Company HrlyRate (number only) Biz Location Summary Skills Employment AboutUs PaymentTerms ServiceDescription Jobid Website (Elance) JobName MinedTime (not in UNIX epoch format but 2003-01-06 00:00:00*) Category (becomes CategoryID with link to translator table of generic categories for all site scrapes) Description Details ProposalsReceived Budget ClientLocation PostedTime* EndTime* Originator Stage 3 Agree feeds & database schema Provide automated update to content every 24 hrs