Python code to automate Web-scraping for images and articles with Google search

Please review the project description carefully before bidding. In your proposal, please mention how would you approach this task and what limitations/challenges do you anticipate with this project?

We are seeking the proposals from freelancers to develop a python code and demonstrate its

functionality to carry out the following web-scraping tasks.

1. Web-scraping for images: The python module needs to be able to scrape the web for

specific images and extract all the associated metadata. The images of interest would be

mainly related to rural and agricultural background. For example, if an image of a ‘tractor in

the farm’ is searched, then the images and the metadata for various tractor images should

be fetched and stored.

2. Web-scraping for articles: The expert blogs and articles need to be searched and

downloaded. The articles written in the domains of farming and rural life are of interest.

3. Google search engine: The articles and images are to be searched mostly using Google

engine. Other good search engines along with google will be an advantage.

4. The code must be deployable on the AWS: The deployment must be with the docker

containers and the scraped data must be stored in the S3 buckets. The complete devops

pipeline must be readily usable.

5. Scalable solution: The deployment needs to be horizontally scalable.

6. Demo with personal account: The demo of the code execution along with the CI/CD pipeline

must be demonstrated using the developers/company’s personal account, we will be

testing it separately on our platform.

7. We are not looking for any UI development, basic text entry even with command line is


Compétences : Web Scraping, Amazon Web Services, Amazon S3, Python, Programmation C#, Process Automation, Développement de logiciel

en voir plus : scraping google images python beautifulsoup, free image scraper, image scraping, web scraping python, scrape google images python, python download all images from url, scrape images from website python beautifulsoup, scrape images from website online

Concernant l'employeur :
( 0 commentaires ) Jurong West, Singapore

Nº du projet : #27366587