En cours

Python script for scraping a website log

I need a script that will access a website, and iterate through a number of log pages that are organized by date. For each such log type, it should save a file with all of the entries sequentially by date. In addition, it should save one combined file for all of the log types, where the log entries are merged across all of the different log types, and sorted by each entry's datetime. So each entry will appear twice: once in the merged log file, and once in the specific log type file. The script should accept as input the oldest date from which to capture the log entries, as well as a few keywords. The keywords will be used for first running a search per each log type and only scraping logs which contains at least 1 of the keywords.

The website requires authentication, so it may be a good idea to use something like [url removed, login to view] with Firefox where I can log in manually and then fire the script. But I'm open to other implementation approaches.

As part of this project, you'll need to thoroughly test your script. This will include setting up yourself some free account on this website and creating some activity that will generate such logs that you can use for verifying that the script works 100% as it should, before submitting it to me for acceptance testing. The code itself should be of good quality, easily readable, annotated, and making plenty of assertions with descriptive error messages so that I can easily tweak it in the future if the website makes some minor changes to how it works (e.g. if the id of some key DOM element there changes)

In order to help me filter spam bids, please answer the following question at the very top of your bid message:

What do you think about using Marionette client for this project? If you prefer not to use it, what alternative tool/framework/modules would you prefer?

Also, I need this done quite quickly so please indicate how soon you think you can finish it.

Thanks!

Compétences : Javascript, Python

Voir plus : website makes, scraping javascript pages, scraping free, order a website, help making a website, free website tool, creating a website free, php website scraping script, website scraping script wordpress, website scraping script php, youtube website scraping script, website scraping script, linux python login website post script, php log website script, create script log website, script automatically log website, script log website

Concernant l'employeur :
( 67 commentaires ) Netanya, Israel

N° du projet : #8511290

Décerné à :

JoshuaCru

Hey, to answer your question regarding the use of Marionette: I usually just use a python library called requests for projects like this. It includes authentication [url removed, login to view] Plus

166 $ USD en 3 jours
(11 Commentaires)
5.1

8 freelance ont fait une offre moyenne de 175 $ pour ce travail

kalpataru44

I can use scrappy or selenium or browser control for this. Hello, I am kalpataru is a freelance expert developer and have specialized in bot, scraper, automation software, web apps, desktop software, android apps and Plus

222 $ USD en 3 jours
(4 Commentaires)
3.9
dpappu131

A proposal has not yet been provided

111 $ USD en 3 jours
(5 Commentaires)
2.7
ssumit2606

I am an expert web scrapper with an experience of more than 3 years in area of developing crawlers and scrappers in Python. I have scrapped websites like [url removed, login to view], [url removed, login to view], [url removed, login to view] and many more to count. Plus

222 $ USD en 5 jours
(1 Commentaire)
1.2
alpha348

Hi, I'm not working on a python so I can not answer your question about the Marionette client. I can make your project on C# or PHP

200 $ USD en 3 jours
(1 Commentaire)
0.6
mirzarehan987

I am an expert web scrapper with an experience of more than 3 years in area of developing crawlers and scrappers in Python. I have scrapped websites like [url removed, login to view], [url removed, login to view], [url removed, login to view] and many more to count. Plus

157 $ USD en 6 jours
(0 Commentaires)
0.0
proseigner

Hello I am an experienced developer and would be able to help you in writing this python program I have attached a bid that is my time and cost includes testing and ensuring the project meets requirements If you are i Plus

222 $ USD en 7 jours
(0 Commentaires)
0.0
chadjcsmith

A proposal has not yet been provided

100 $ USD en 1 jour
(0 Commentaires)
0.0