Need To Process a List with 30 000 000 Domains


I'm looking for an expert in crawling/data processing/data mining to process a list with 30 000 00 domains.

What I need is someone to crawl these 30 000 000 domains ( + up to 100 pages on each domain so this will make up to 3 000 000 000 urls which need to be crawled) and to collect all external backlinks found on them.

This will create a very large log file ( maybe around 50 gigs) and i want the expert to help me upload that log on a server and to be able to search inside the log file.

If you have more question regarding this project, please contact me.

Compétences : Saisie de Données, Exploitation de Données, Traitement de Données, Excel, Web Scraping

Voir plus : want need data, need write poem help, want write article help, process get project freelancer, process center need security, need urgent money help

Concernant l'employeur :
( 14 commentaires ) Pernik, Bulgaria

N° du projet : #12685180

44 freelance ont fait une offre moyenne de 620 $ pour ce travail

600 $ USD en 10 jours
(230 Commentaires)
789 $ USD en 20 jours
(274 Commentaires)

Hi, Thanks for the opportunity to bid on this project. I have done such data gathering and research in the past for various clients so that I will provide good service. Regards, Jai

555 $ USD en 60 jours
(259 Commentaires)

Hello There, I have questions regarding this project. Would you be able to answer them before. My name is Kal. Looking forward to hear from you soon. Thanks. Kalpanasekhar

526 $ USD en 10 jours
(138 Commentaires)

Dear Sir/Ma'am, I am a Web research, Data Entry & Webs Scrapping expert. I checked and understood your requirements. I can handle this job very well to your appreciation. I can find and extract the information Plus

750 $ USD en 7 jours
(195 Commentaires)

Hi, I understand the project requirements and willing to start as soon as possible. Kindly share the details and I will deliver with high quality. Looking forward to work with you. Thanks & Regards Khushi India

526 $ USD en 5 jours
(236 Commentaires)

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. Our late Plus

525 $ USD en 12 jours
(6 Commentaires)

Dear Sir, I will provide you Current data from (source website list for 30 000 000 domains). Also, I can scrap after login from current data Scraping source website, I Will follow (name, details (Email, phone, website Plus

250 $ USD en 5 jours
(149 Commentaires)

Hello, I read your requirements and understand very-well and ready to start work from now. Let's discuss more in chat, waiting for your reply. Thank you

750 $ USD en 20 jours
(24 Commentaires)

Self-motivated professional, certified data entry specialist, a proficient web researcher and transcriber, blogger, web designer, Wordpress and various social sites marketer, knowledgeable in SEO, and technical / help Plus

400 $ USD en 20 jours
(37 Commentaires)

Hi Sir, Iam expert in Web scrapping, Research, Websearch & Data entry. I will Java code to scrape the Huge data can be scrapped in Minutes, you can see in my previous project i have scrapped 80,00,000 Urls from a Web Plus

750 $ USD en 10 jours
(101 Commentaires)

Hi there! Have done this type of work before and am a big data/scraping expert ! Please feel free to message me with any additional information or with questions if any ! Thanks! -Steve

750 $ USD en 5 jours
(75 Commentaires)

Dear Hiring Manager, I have gone through your job posting and become very much interested to work with you. I am an expert in these fields. I have already completed several projects like this. For evidence you can s Plus

277 $ USD en 10 jours
(53 Commentaires)

Hi there! I am specialist on scraping data from any kind of websites including frequently blocking sites. Also an expert on all of data entry & research jobs. I’m ready to start it right away. I look forward to hea Plus

3333 $ USD en 30 jours
(54 Commentaires)

Hello Friend, I have gone through your requirement. I am EXPERT in building scraping tools so I can SURELY work on your project. You will love to check my work at: www.CrawlerMonkey.com Skype id: crawler.monkey Plus

2222 $ USD en 10 jours
(6 Commentaires)

Hi I am Furqan I've reviewed your complete job description, and I fulfill all the qualifications required for this project. I have 8+years experience working in python languages i have completed lof of projects Plus

260 $ USD en 0 jours
(3 Commentaires)

Hi, I have gone through your requirement and I'm very interested in this job. Yes I already did this type of jobs and its more easier for me to do this job. You can also check my profile's reviews and ratings about my Plus

366 $ USD en 5 jours
(24 Commentaires)
580 $ USD en 21 jours
(13 Commentaires)

Hello, Im interested to work on this. I gone through the job description. Im looking forward to get more details about the log file and format of the logfile. Thanks, Sheik Mubarak S

500 $ USD en 15 jours
(12 Commentaires)

Hello, Iam shahid from kashmir. Over the last 7 years, I have worked for several clients. Joined Freelancer with over 7 years of experience in , Data entry, Linkedin Lead generation , Google Research Expert,Web scrap Plus

250 $ USD en 0 jours
(2 Commentaires)