Fermé

create example scripts using celery and rabbidmq to run different tasks on different remote nodes

I'm creating a distributed system. But I'm not aware of Celery/RabbitMQ setups.

My plan is to create 5 different workers with different tasks (example):

- Worker1 - send icmp request

- Worker2 - scrap a url content

- Worker3 - execute a whois on a domain

- Worker4 - get a list of N urls from "somewhere" (could be the same rabbitmq?) and fetch the title tag.

- Worker5 - fetch all ips related with those 300 urls.

When the celery client send a task request through the broker:

- I want to scrap this url: a.b.c

The worker2 should respond and execute the task, and return the result back to the celery client. All the other workers should never receive this request, as those don't have the proper function/task.

When the celery client send a task request through the broker:

- I want to fetch the title tag from a list of 230 or 1000 websites

The Worker4 should respond, fetch the list of websites, execute the task and return the result back to the celery client.

Each worker runs on a different linux box on the internet.

I will also need the ability of have a pool of worker1 boxes. Let's say 5 linux boxes with the same worker1 software, and the first to respond will perform the task (Round-robin).

The celery client must be a Flask page with:

- a table with 10 domains and 5 urls

- When the user click on one of those domains, and hit the button 'run whois', the celery task must be triggered

- When the user select 3 of those urls and select the option "fetch title", the celery task must be triggered

The response must also be presented via flask webapp (can be a different page)

Compétences : Python, DevOps, Data Scraping, Scripting

en voir plus : create flyers trifolds using adobe photoshop, create floor plan using autocad, create online admission using php javascript html, create dyanamic website using php, create website voting using java mysql tomcat, web accounting example scripts, web banking example scripts php mysql, create contact form using formailpl, create simple banner using css, create registration form using mysql, create report php using pdf, create image swapping using flash, create xml file using mysql, vlc example scripts, create acknowledgement banner using scripts, example jboss create web service using annotation, example php code create order report using fpdf, create ear file using maven example, how to run selenium ide scripts using java eclipse, provide example scripts for automating the ui testing of a video player like youtube using selenium

Concernant l'employeur :
( 0 commentaires ) Brazil

Nº du projet : #28332447

16 freelances font une offre moyenne de 544 $ pour ce travail

(15 Commentaires)
5.6
benliao

Hello I have read your job description carefully. Also, I have rich experience in python and web scraping So I am interested in and I am confident in your project. I always provide perfect results to my clients as you Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 8 jours
(6 Commentaires)
4.2
DRG1990

Hi, I summarizes all descriptions and provides clear and concise explanations. I am an expert of Scraping with Python experience. I have done several similar projects in the backend development of numerous internet s Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 5 jours
(5 Commentaires)
4.1
(8 Commentaires)
4.2
braincenter

Hello, Hope this message finds you well, I checked your details and I believe that my experience is what you are looking 4. I have been working on similar projects for the past eight years, and I have the essential sk Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(10 Commentaires)
4.3
kholmogorovadev

Hi. I am a Flask expert I know Celery/RabbitMQ I can help you as well I'll provide you good result Please contact me best regards Aleksandra

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(3 Commentaires)
2.9
sharmashivangi00

Hello, Hope you are doing well..!! I am a Full Stack Developer with over 4 years of experience in Python backend frameworks like Django and Flask along with the frontend frameworks like React.js and Angular. I have b Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(3 Commentaires)
3.3
rostikhr111

Hi, How are you? I am very interested in your project. I have rich experience in web development using Flask and Django. Especially I have rich experience in Schedule, Asynchronous using Celery, AsyncIO. I am sure th Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(1 Évaluation)
2.7
nikolai198061

Hi, client. I am a senior Python developer who has rich experiences. Especially, I have professional experiences in Web Scraping. I've successfully done a lot of projects to scrape data like email, URL, ads, contact, p Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(3 Commentaires)
2.8
AndreiiWebGats

Hello, Nice to meet you. I am very pleased to present this proposal for your consideration as a fullstack developer. I have deep knowledge in real-time data processing and experience in RabbitMQ, Redis, socket.io. One Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(2 Commentaires)
2.9
demydkasilov

Hi, sir I hope you are doing well and staying safely without COVID-19 effect. I have carfully checked your requirement. I am a web scraping expert and have good experience with scraping using python, node.js and php fo Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(2 Commentaires)
2.4
Rustamdev

Hello. I have rich experiences in python scraping. So, if you hire me, I will do my best for you and for your project. Thanks. Rustam.

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(1 Évaluation)
1.8
stepinnsolution

Hello man. I am a full stack developer and have around about 12+ years of experience in the development field. I am the CEO of Stepinnsolution. Stepin Solution to award leading the digital world. I want to tell you We Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(1 Évaluation)
0.6
iwtechnocrats41

HI, Hope you are well! I have gone through your requirement properly and our company's main policy is to create and maintain long-term relationships rather than earning profit. This would be a win-win situation for Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 30 jours
(1 Évaluation)
0.0
idevenster

Can I have a go at your project? I have been doing Python, Java, Node Js for the past 10 years. I think I can help you out. I don't need money as such for now

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 3 jours
(0 Commentaires)
0.0
shanarana001

Hello successful buyer I have go through your request and it is fully understand and i can do it perfectly and the work will be done professionally and perfectly because am an expert in this field Kindly contact me now Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(0 Commentaires)
0.0