Need To Process a List with 30 000 000 Domains


I'm looking for an expert in crawling/data processing/data mining to process a list with 30 000 00 domains.

What I need is someone to crawl these 30 000 000 domains ( + up to 100 pages on each domain so this will make up to 3 000 000 000 urls which need to be crawled) and to collect all external backlinks found on them.

This will create a very large log file ( maybe around 50 gigs) and i want the expert to help me upload that log on a server and to be able to search inside the log file.

If you have more question regarding this project, please contact me.

Compétences : Saisie de Données, Exploitation de Données, Traitement de Données, Excel, Web Scraping

Voir plus : i want to need data, write a letter to some who want to help u continue your education and write the school you like, need to write a poem help, need drupal 6 to drupal 7 migration help, i want to write a article help, i want atranslator hindi to sanskrit and please help to ned a translaor, i d like to visit volgograd i need an interpreter, to need in spanish, want to give advertisement to need an accountant, process to get project from freelancer, process to design good web pages, how can you get imail adress write a rich people need help and how can are send my imail adresss to a rich for help money or fix, uk debt management companies outsourcing their process to india, process to get a project through freelancer, outsource your assembly process to china, need various it help around the office hourly mostly tech projects also project management for outsourcing web stuff, need various it help around the office - hourly - mostly tech projects also project management for outsourcing web stuff, i want to need net web developer mumbai, i want to need a graphics design job in dubai, i want help to write a conference paper, i am going to need a consultant with expertise in managing surveys using tablet computers, hi there i’d like to be considered for your writing position and i’m glad to answer any questions you may have thank you for you, d designer to hire, d character to 3d, companies that need that want marketers to be paid in facebook and twitter

Concernant l'employeur :
( 14 commentaires ) Pernik, Bulgaria

N° du projet : #12685180

44 freelancers are bidding on average $620 for this job

600 $ USD en 10 jours
(230 Commentaires)
789 $ USD en 20 jours
(274 Commentaires)

Hi, Thanks for the opportunity to bid on this project. I have done such data gathering and research in the past for various clients so that I will provide good service. Regards, Jai

555 $ USD en 60 jours
(259 Commentaires)

Hello There, I have questions regarding this project. Would you be able to answer them before. My name is Kal. Looking forward to hear from you soon. Thanks. Kalpanasekhar

526 $ USD en 10 jours
(138 Commentaires)

Dear Sir/Ma'am, I am a Web research, Data Entry & Webs Scrapping expert. I checked and understood your requirements. I can handle this job very well to your appreciation. I can find and extract the information Plus

750 $ USD en 7 jours
(195 Commentaires)

Hi, I understand the project requirements and willing to start as soon as possible. Kindly share the details and I will deliver with high quality. Looking forward to work with you. Thanks & Regards Khushi India

526 $ USD en 5 jours
(236 Commentaires)

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. Our late Plus

525 $ USD en 12 jours
(6 Commentaires)

Dear Sir, I will provide you Current data from (source website list for 30 000 000 domains). Also, I can scrap after login from current data Scraping source website, I Will follow (name, details (Email, phone, website Plus

250 $ USD en 5 jours
(149 Commentaires)

Hello, I read your requirements and understand very-well and ready to start work from now. Let's discuss more in chat, waiting for your reply. Thank you

750 $ USD en 20 jours
(24 Commentaires)

Self-motivated professional, certified data entry specialist, a proficient web researcher and transcriber, blogger, web designer, Wordpress and various social sites marketer, knowledgeable in SEO, and technical / help Plus

400 $ USD en 20 jours
(37 Commentaires)

Hi Sir, Iam expert in Web scrapping, Research, Websearch & Data entry. I will Java code to scrape the Huge data can be scrapped in Minutes, you can see in my previous project i have scrapped 80,00,000 Urls from a Web Plus

750 $ USD en 10 jours
(101 Commentaires)

Hi there! Have done this type of work before and am a big data/scraping expert ! Please feel free to message me with any additional information or with questions if any ! Thanks! -Steve

750 $ USD en 5 jours
(75 Commentaires)

Dear Hiring Manager, I have gone through your job posting and become very much interested to work with you. I am an expert in these fields. I have already completed several projects like this. For evidence you can s Plus

277 $ USD en 10 jours
(53 Commentaires)

Hi there! I am specialist on scraping data from any kind of websites including frequently blocking sites. Also an expert on all of data entry & research jobs. I’m ready to start it right away. I look forward to hea Plus

3333 $ USD en 30 jours
(54 Commentaires)

Hello Friend, I have gone through your requirement. I am EXPERT in building scraping tools so I can SURELY work on your project. You will love to check my work at: [url removed, login to view] Skype id: [url removed, login to view] Plus

2222 $ USD en 10 jours
(6 Commentaires)

Hi I am Furqan I've reviewed your complete job description, and I fulfill all the qualifications required for this project. I have 8+years experience working in python languages i have completed lof of projects Plus

260 $ USD en 0 jours
(3 Commentaires)

Hi, I have gone through your requirement and I'm very interested in this job. Yes I already did this type of jobs and its more easier for me to do this job. You can also check my profile's reviews and ratings about my Plus

366 $ USD en 5 jours
(24 Commentaires)
580 $ USD en 21 jours
(13 Commentaires)

Hello, Im interested to work on this. I gone through the job description. Im looking forward to get more details about the log file and format of the logfile. Thanks, Sheik Mubarak S

500 $ USD en 15 jours
(12 Commentaires)

Hello, Iam shahid from kashmir. Over the last 7 years, I have worked for several clients. Joined Freelancer with over 7 years of experience in , Data entry, Linkedin Lead generation , Google Research Expert,Web scrap Plus

250 $ USD en 0 jours
(2 Commentaires)