I'm looking for an expert in crawling/data processing/data mining to process a list with 30 000 00 domains.
What I need is someone to crawl these 30 000 000 domains ( + up to 100 pages on each domain so this will make up to 3 000 000 000 urls which need to be crawled) and to collect all external backlinks found on them.
This will create a very large log file ( maybe around 50 gigs) and i want the expert to help me upload that log on a server and to be able to search inside the log file.
If you have more question regarding this project, please contact me.
44 freelance ont fait une offre moyenne de 620 $ pour ce travail
Hi, Thanks for the opportunity to bid on this project. I have done such data gathering and research in the past for various clients so that I will provide good service. Regards, Jai
Hello There, I have questions regarding this project. Would you be able to answer them before. My name is Kal. Looking forward to hear from you soon. Thanks. Kalpanasekhar
Hi, I understand the project requirements and willing to start as soon as possible. Kindly share the details and I will deliver with high quality. Looking forward to work with you. Thanks & Regards Khushi India
Hello, I read your requirements and understand very-well and ready to start work from now. Let's discuss more in chat, waiting for your reply. Thank you
Hi there! Have done this type of work before and am a big data/scraping expert ! Please feel free to message me with any additional information or with questions if any ! Thanks! -Steve
Hello, Im interested to work on this. I gone through the job description. Im looking forward to get more details about the log file and format of the logfile. Thanks, Sheik Mubarak S