I need a script that will crawl several Craigslist ([login to view URL]) sites and collect data from posts in specific categories for the last 30 days.
I am interested in specific **sub-categories** within the **"Services"** category. Specifically:
- household [login to view URL]
- labor/move [login to view URL]
- skill'd trade [login to view URL]
The **cities** I am interested in crawling are the following:
Atlanta, Boston, Chicago, Dallas, Denver, Detroit, Houston, Las Vegas, Los Angeles, Miami, Minneapolis, New Jersey, NYC, Philadelphia, Phoenix, Portland, Raleigh, Rhode Island, Sacramento, San Diego, Seattle, SF Bay Area, Toronto, Vancouver, and Washington DC.
For cities that are divided into several different **"areas"** I will need to crawl those as well. ex:
**SF Bay Area** ([login to view URL])
- [login to view URL]
- [login to view URL]
- [login to view URL]
- [login to view URL]
- [login to view URL]
- [login to view URL]
The data fields I am interested in collecting for each post are the following:
- city
- area
- category
- date posted
- time posted
- link to original post
The output should be in tab delimited form and I will need the source code for the script once the job is complete.