Fermé

Need To Process a List with 30 000 000 Domains

Hi,

I'm looking for an expert in crawling/data processing/data mining to process a list with 30 000 00 domains.

What I need is someone to crawl these 30 000 000 domains ( + up to 100 pages on each domain so this will make up to 3 000 000 000 urls which need to be crawled) and to collect all external backlinks found on them.

This will create a very large log file ( maybe around 50 gigs) and i want the expert to help me upload that log on a server and to be able to search inside the log file.

If you have more question regarding this project, please contact me.

Compétences : Saisie de Données, Exploitation de Données, Traitement de Données, Excel, Web Scraping

en voir plus : i want to need data, write a letter to some who want to help u continue your education and write the school you like, need to write a poem help, need drupal 6 to drupal 7 migration help, i want to write a article help, i want atranslator hindi to sanskrit and please help to ned a translaor, i d like to visit volgograd i need an interpreter, to need in spanish, want to give advertisement to need an accountant, process to get project from freelancer, process to design good web pages, how can you get imail adress write a rich people need help and how can are send my imail adresss to a rich for help money or fix, uk debt management companies outsourcing their process to india, process to get a project through freelancer, outsource your assembly process to china, need various it help around the office hourly mostly tech projects also project management for outsourcing web stuff, need various it help around the office - hourly - mostly tech projects also project management for outsourcing web stuff, i want to need net web developer mumbai, i want to need a graphics design job in dubai, i want help to write a conference paper, i am going to need a consultant with expertise in managing surveys using tablet computers, hi there i’d like to be considered for your writing position and i’m glad to answer any questions you may have thank you for you, d designer to hire, d character to 3d, companies that need that want marketers to be paid in facebook and twitter

Concernant l'employeur :
( 14 commentaires ) Pernik, Bulgaria

Nº du projet : #12685180

42 freelance font une offre moyenne de $637 pour ce travail

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 10 jours
(247 Commentaires)
7.9
%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 20 jours
(310 Commentaires)
7.9
mirniyazuddin92

Dear Sir/Ma'am, I am a Web research, Data Entry & Webs Scrapping expert. I checked and understood your requirements. I can handle this job very well to your appreciation. I can find and extract the information Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(304 Commentaires)
7.6
Kalpanasekhar

Hello There, I have questions regarding this project. Would you be able to answer them before. My name is Kal. Looking forward to hear from you soon. Thanks. Kalpanasekhar

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 10 jours
(161 Commentaires)
7.1
khushh07

Hi, I understand the project requirements and willing to start as soon as possible. Kindly share the details and I will deliver with high quality. Looking forward to work with you. Thanks & Regards Khushi India

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 5 jours
(274 Commentaires)
7.2
Harun1986

Dear Sir, I will provide you Current data from (source website list for 30 000 000 domains). Also, I can scrap after login from current data Scraping source website, I Will follow (name, details (Email, phone, website Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 5 jours
(207 Commentaires)
6.7
asifdwan

Hi there! I am specialist on scraping data from any kind of websites including frequently blocking sites. Also an expert on all of data entry & research jobs. I’m ready to start it right away. I look forward to hea Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 30 jours
(195 Commentaires)
6.6
rangacbe

Hi, Thanks for the opportunity to bid on this project. I have done such data gathering and research in the past for various clients so that I will provide good service. Regards, Jai

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 60 jours
(272 Commentaires)
6.7
devi222

Hi Sir, Iam expert in Web scrapping, Research, Websearch & Data entry. I will Java code to scrape the Huge data can be scrapped in Minutes, you can see in my previous project i have scrapped 80,00,000 Urls from a Web Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 10 jours
(147 Commentaires)
6.5
mmadi

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. Our late Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 12 jours
(6 Commentaires)
5.9
dynamitewebsol

Hello, I read your requirements and understand very-well and ready to start work from now. Let's discuss more in chat, waiting for your reply. Thank you

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 20 jours
(32 Commentaires)
5.9
catstevens2009

Hi there! Have done this type of work before and am a big data/scraping expert ! Please feel free to message me with any additional information or with questions if any ! Thanks! -Steve

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 5 jours
(86 Commentaires)
5.6
abdussamad

Self-motivated professional, certified data entry specialist, a proficient web researcher and transcriber, blogger, web designer, Wordpress and various social sites marketer, knowledgeable in SEO, and technical / help Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 20 jours
(40 Commentaires)
5.8
shafaqat11

Dear Hiring Manager, I have gone through your job posting and become very much interested to work with you. I am an expert in these fields. I have already completed several projects like this. For evidence you can s Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 10 jours
(107 Commentaires)
5.7
Anasra

Ok ! I will help you to scrap the urls and will deliver you fast results. Moreover please check my reviews. Thank you! Amir Ishfaq

%bids___i_sum_sub_32% %project_currencyDetails_sign_sub_33% USD en 1 jour
(56 Commentaires)
5.3
shahiddar

Hello, Iam shahid from kashmir. Over the last 7 years, I have worked for several clients. Joined Freelancer with over 7 years of experience in , Data entry, Linkedin Lead generation , Google Research Expert,Web scrap Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 0 jours
(17 Commentaires)
5.5
sheikDev

Hello, Im interested to work on this. I gone through the job description. Im looking forward to get more details about the log file and format of the logfile. Thanks, Sheik Mubarak S

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 15 jours
(21 Commentaires)
5.1
utkarshkatiyar19

A proposal has not yet been provided

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 10 jours
(14 Commentaires)
4.4
crawlermonkey

Hello Friend, I have gone through your requirement. I am EXPERT in building scraping tools so I can SURELY work on your project. You will love to check my work at: [url removed, login to view] Skype id: [url removed, login to view] Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 10 jours
(6 Commentaires)
4.5
AndreiMotpan

Dear Freelancer, Working in a multinational research company, I gathered an impressive number of project in this area which you can indeed confirm on my profile. I have experience working with many platforms incl Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 10 jours
(12 Commentaires)
4.2