Fermé

Develop web scraping software

I’m looking for experienced data extraction developer to provide me with custom project.

The goal is to automatically retrieve information from some major search engine. It’s not about regular search results, but information from snippets about important places containing place name, phone number and opening hours.

My original project analysis shown, that search engine is using content generated dynamically by obfuscated/compressed JavaScript code. No simple wget/curl will provide results. You must have experience in this matter since it looks not trivial.

Input data (string):

- Search query

Output data (JSON):

- Place name

- Address

- Phone number

- Opening hours

I see this as a command line script where I provide search query as parameter and get JSON response in STDOUT.

Script must use proxy service provided by [url removed, login to view] and include automated dead proxy detection and rotation. Connection timeout must be a parameter.

Search engine is using HTTPS encrypted connections.

Interpreted languages preferred like: PHP, Python

Script must run on headless Linux/Debian server. It must not depend on web browser or any other GUI application, so for instance Selenium will not work.

Script must be able to run multiple instances concurrently.

During the tests you will provide online demo of this script where search query will be passed as URL GET/POST params and response will contain JSON.

After finishing you must provide full, unencrypted source code of the project and build/compilation instructions if needed.

Compétences : Exploitation de Données, Java, PHP, Python, Web Scraping

Voir plus : work online as software developer, where to get python code, where to get a web developer online, web server languages, web search engine proxy, web scraping python 3, web scraping https, web scraping application, web or software developer, web develop on python, web developer search engine, web developer python, web developer on line, web and software developer, web analysis service, software developer search, software developer phone number, software developer by hours, simple scraping software, script php proxy web, scraping web content, scraping a server, python software developer needed, php web developer software, php developer opening

Concernant l'employeur :
( 5 commentaires ) Warszawa, Poland

N° du projet : #8472130