Develop web scraping software

I’m looking for experienced data extraction developer to provide me with custom project.

The goal is to automatically retrieve information from some major search engine. It’s not about regular search results, but information from snippets about important places containing place name, phone number and opening hours.

My original project analysis shown, that search engine is using content generated dynamically by obfuscated/compressed JavaScript code. No simple wget/curl will provide results. You must have experience in this matter since it looks not trivial.

Input data (string):

- Search query

Output data (JSON):

- Place name

- Address

- Phone number

- Opening hours

I see this as a command line script where I provide search query as parameter and get JSON response in STDOUT.

Script must use proxy service provided by [url removed, login to view] and include automated dead proxy detection and rotation. Connection timeout must be a parameter.

Search engine is using HTTPS encrypted connections.

Interpreted languages preferred like: PHP, Python

Script must run on headless Linux/Debian server. It must not depend on web browser or any other GUI application, so for instance Selenium will not work.

Script must be able to run multiple instances concurrently.

During the tests you will provide online demo of this script where search query will be passed as URL GET/POST params and response will contain JSON.

After finishing you must provide full, unencrypted source code of the project and build/compilation instructions if needed.

Compétences : Exploitation de Données, Java, PHP, Python, Web Scraping

Voir plus : work online software developer, get python code, get web developer online, web server languages, web search engine proxy, web develop python, web developer search engine, web developer python, web developer line, software developer search, software developer phone number, script php proxy web, python software developer needed, php web developer software, php developer opening, looking software developer service, demo web developer, curl python, code python online, joomla web develop, convert web site languages, asp web develop, software solution web design

Concernant l'employeur :
( 5 commentaires ) Warszawa, Poland

N° du projet : #8472130