Hadoop/Cloudera Search /Web crawling/Twitter&Facebook API

Fermé Publié le Sep 11, 2013 Paiement à la livraison
Fermé Paiement à la livraison

We have a Hadoop Cluster running CDH with cloudera Manager Opensource

We need to use HDFS in our hadoop cluster installation with 6 nodes, to make a web crawler, to search for words sentences and post in different web pages /ads web pages and other post (most in web pages like ebay, and other spanish add's web pages) to crawl the web and search for some patterns (the idea is to work like a spider in more than 50 website with out stop)

If the pattern it the same we need to store the HTML of the web in the hadoop cluster (cloud be different websites in time with different word search's ) just the web page that the pattern that we are searching exist.

Also connect to Twitter API and facebook and correlate different Users and hashtags with Flume (could be different twitter accounts, or hashtags of facebook hashtags or users comments to download )

ones all the info is in the hadoop cluster we need to put all the info in the Cloudera search to search word's and index And index all the info in HIVE or PIG For extract the Info in [url removed, login to view] library

the idea is to centralise some information and search in sorl and report in [url removed, login to view] if we find some word over time and the frequency of the words.

Marketing sur Facebook Hadoop JavaScript Map Reduce Twitter

Nº du projet : #4916994

À propos du projet

7 propositions Projet à distance Actif Oct 18, 2013

7 freelances font une offre moyenne de 2366 $ pour ce travail

NeoNoXtraOrdnry

Dear Sir, please PM me.

$3684 USD en 30 jours
(27 Commentaires)
4.5
freelancerj2ee

I have 8+ years of experience in J2EE programming and is familiar with Struts/Spring/Hibernate framework. Please kindly check your PMB.

$1666 USD en 20 jours
(4 Commentaires)
3.8
userdatacode

We are aggregated with innovation, creativity, ability, Knowledge, experience, convergent thinker, skills, professionalism etc. It'll be our pleasure to work with you. Check PM for more.

$822 USD en 25 jours
(0 Commentaires)
0.0
nnramkumar

Please check private message for more details

$5555 USD en 75 jours
(0 Commentaires)
0.0
ranganathp

Can help... I am an Expert...

$2000 USD en 40 jours
(0 Commentaires)
0.0
metaruslan

Hello! I am an experienced Hadoop developer. I am sorry for bothering but do you accept hourly payment? Thanks

$1250 USD en 3 jours
(0 Commentaires)
1.0
nickgonzalez78

I have been CTO of many well known companies, and I am Chief Data Scientist for an up and coming company. I have most of the pieces for your project ready in my personal library of code that I have been building up ov Plus

$1250 USD en 3 jours
(0 Commentaires)
0.0