Dear Client:
I have over 5+ experiences in software programming involving following tech/libraries/languages:
• Python, Flask, nltk, numpy, matplotlib, tkinter, lxml, XPath, beautifulsoup, urrllib,
• Smarty, PHP, C/C++, Java, vtk, itk, openCV
• Ruby, mechanize, nokogiri
• Regex, JS/Ajax/JSON, html/xml, PyV8
• mySQLdb, xlrd, xlwt, csv, minidom, Image,
• Csv, excel, mySQL,PostgreSQL Oracle
• Selenium Webdriver/FF/Chrome, Xvbf, etc.
• Linux/CentOS/Ubuntu, Windows
• Parsing XML, HTML, JSON, JS code, text etc.
• Scrapy, Clover API,Google MAP API
• Random/Rotating Proxy, User agents
Scraping/Scrapy experiences-
* written over 60 spiders/scrapers in Python under Scrapy for extracting data from 60+ websites – with JS/Ajax/Dynamic data contents, multiple regions, countries, currencies using.
* written pipelines for pre-/post-processing captured data, collecting statistics, and middlewares for rotating proxy, user-agent management.
*extensively used AJAX calls to retrieve data in HTML/JSON format and parsing of JSON messages.
* extracted product variations by colors, sizes, inventory/size data.
* extensively used FF debugger/firebug.
* installed and configured Scrapy on several platforms: CentOS, Ubuntu, Windows.
* outputted data in various formats: csv, JSON, mySQL
*few example websites scraped: amazon, neimanmarcus, walmart, calvinklein, toysrus,
Pls contact me for more details,
Thanks,
Malik.