En cours

java-based simple web-scraper

Hi there,

I would like a web-scraper which accepts a property-file as input containing

- a first list of domains as input

- and a second list of exclusion-uris.

Apatche nutch would be preferred for doing this. Nutch already should offer this functionality,

but I do not have the time to verify and set it up...

As a deliverable I expect a simple java class (including source-code) that I can run via

command-line, that accepts the name of the property-file as input:

java -cp . simpleScraper C:\\path-to-property-file\\[url removed, login to view]

The scraper must extract data out of the scraped web-sites and write them into a mysql-table

with the following three fields:

URL (Long Text or varchar) | SCRAPE_TIME (timestamp of scraping) | CTEXT (CLOB containing the

whole html-code of the web-page)

needless to say: the scraping should be performed as performantly as possible ... :-)

For example, the property file could look like:

/*

DOMAINLIST:

The entire web-site of all specified domains have

to be scraped - except below exclusions

*/

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

/*

EXCLUSIONLIST:

All web-pages

starting with the following uris

must be excluded from scraping: */

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

Thanks and kind regards

Compétences : Java, MySQL, Architecture Logicielle

Voir plus : software web sites, long web pages, nutch, java based, all web, java code example, html based text, html table txt, html java mysql, java html table, web data table extract, java simple example, mysql data path, table extract, table based, java simple file input, code example java, command web, extract web data java, java web page, java file list, simple data web page, extract domains text, java mysql command, web sites list

Concernant l'employeur :
( 15 commentaires ) n.n, Germany

N° du projet : #2363535

Décerné à :

JavaGuruLV

Hi, I am JAVA expert with Sun certificates and more then 5 years of working experience. I will do this task for you.

50 € EUR en 4 jours
(4 Commentaires)
2.8

21 freelance ont fait une offre moyenne de 187 € pour ce travail

eperfections

I have a message for you in PMB

200 € EUR en 3 jours
(309 Commentaires)
7.1
srinichal

I like to discuss further and deliver the project

250 € EUR en 5 jours
(67 Commentaires)
6.7
tomydeveloper

Hello,We are Scraping expert using with javascript.Please check pmb for more detail.Thanks

200 € EUR en 5 jours
(55 Commentaires)
6.4
dobreiiita

Hi, I am JAVA expert and can surely handle this project perfectly. Thanks

170 € EUR en 2 jours
(169 Commentaires)
6.4
procoder898

Hi, I am expert at Web Scraping in Java, can surely satisfy you. Please check your inbox,

190 € EUR en 0 jours
(93 Commentaires)
6.2
zeke

Please see PMB for details.

200 € EUR en 2 jours
(57 Commentaires)
6.0
poornachand

I have 5 years experience in JAVA, J2EE and Web-Services. I am interested in working on this project.

125 € EUR en 3 jours
(26 Commentaires)
5.1
csanuragjain

hi i can do this contact if intersted

90 € EUR en 10 jours
(22 Commentaires)
5.1
manhpham

I am ready to help you. Please contact me asap.

200 € EUR en 3 jours
(19 Commentaires)
4.4
jitendraparmar07

Already did a large no. of similar projects. Moreover, I code only in java. So it seems like you were looking for me only. Please check your PMB.

200 € EUR en 3 jours
(9 Commentaires)
4.5
javaguru75

Already done a very good web based scrapper also can do as you asked. Its highly customizable also video scrapper and many more functionality. with regards

500 € EUR en 20 jours
(8 Commentaires)
4.2
monumichael

Hi, Experienced Java developer for your service. Please have a look at my review comments to get the quality of my work. Please approve my bid. Regards,

100 € EUR en 7 jours
(14 Commentaires)
4.2
Specialists786

I have a message for you in PMB

150 € EUR en 1 jour
(8 Commentaires)
2.9
MagicalDesigner

Sir Please Check PMB. Thanks Sir

250 € EUR en 10 jours
(4 Commentaires)
2.6
dewanmomen

I can do this project. Please check PMB.

200 € EUR en 6 jours
(1 Commentaire)
1.0
tmotion

Hi, i have experience with this type of projects.

190 € EUR en 2 jours
(0 Commentaires)
0.0
Sigunejay

We checked your posting in detail and based on that we are ready to do work as per your budget and timing and we are ready to discuss the things with you

160 € EUR en 3 jours
(0 Commentaires)
0.0
yugilancer

Hi, I have 6+ years of exp in Java & J2EE. I can complete this task with in 3 days with JUnit test cases as well. Please check PMB for more details. -- Thanks & Regards, Yugilancer

200 € EUR en 3 jours
(0 Commentaires)
0.0
U719OpHWi

Custom software development - <b><i>Removed by Admin</i></b>

250 € EUR en 1 jour
(0 Commentaires)
0.0
atif038

I am professional Java programmer. I am serious and I am ready to help, please contect me for further detail.

60 € EUR en 3 jours
(0 Commentaires)
0.0