Craigslist Website scraper! It’s developed in PHP and MySQL, and only needs a a hosting account to be installed on and run through, and will go through every Craigslist city’s website, and do a search query or pull up a certain category page etc., then it will scrape the results displayed and parse them then import the parsed data neatly into a database. It parses and stores: the specific Craigslist site/city, the Craigslist PostingID, posting title, email, posting content, any images included anywhere, and it will also parse the posting content to look for an email address and then store that. This script can be setup to run as a cron job, to constantly poll Craigslist for new postings that come up, and then you can do what you want from all the retrieved and parsed data.
NEW:
Now has:
- .csv Export, now organized by state and city
- Simple Admin screens
- Craigslist Query String Link Builder
- Quick & Simple 1-step Setup
Features Coming Soon:
- Password protected
- Proxies used for scraping
I also have a version of the CL scraper that is tailored to scraping autos and will scrape and store the following data:
- Year
- Make
- Mileage
- Price
- Phone
Contact me to purchase
If you have any questions about this script, drop a comment or contact me.
Thanks,
Josh
Video of newest version in action!
Video of Auto Specific version in action!
Showing actual scraped content for CL, (Notice 30K+ records!!)
Contact me to purchase
If it’s out there, I can scrape the data for you!
Whatever your data needs are, if there is a website out there that has data that needs to be harvested or scraped, I can get that data for you in what ever format you need it in.
- Provide the website URL and tell me what data you need, and if you need updates to the data in the future.
- I will review the site and determine the project timeline and technical steps required and will also gather some info about the site to be used for QA later in the process.
- After my initial review, I will code to the data need requirements and begin extracting the raw data and storing it in a database.
- Once the data extraction/harvesting/scraping is complete, I will analyze the data and perform QA tests ensuring it is complete and accurate.
- The data is delivered to you on-time and accurate!