Stock Data Crawler Csv To MySQL

The project intends to gather stock data from the end of every trading day by using the Yahoo Finance API and import the data into a localhosted mySQL database.

The program will be written in Python and/or mySQL query language and should be compatible with the OS X environment. The code will run manually at the end of every trading day.

The Yahoo Finance API is simple and fetches a CSV-file based on dates and stock symbols:

[url removed, login to view][[url removed, login to view]]?s=[Stock symbol]&a=[start month - 1]&b=[start day]&c=[start year]&d=[end month - 1]&e=[end day]&f=[end year]&g=d&ignore=.csv

The input is a list of stock tickers in a text file separated by line shifts or any other easy to use solutions. Example of stock symbols:


[url removed, login to view]

[url removed, login to view]


[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]


The other inputs [dates, etc] shall be easily changed by the user either in the beginning of the code. Example of datasource for the GOOG symbol from 1.1.2000 to 20.09.2015.

[url removed, login to view]

The output will typically be, but may vary:

[Date] - [Open] - [High] - [Low] - [Close] - [Volume] - [Adjusted Close]

The output shall be imported into a local mySQL database, each stock data into separate tables. Date formatting should be compatible with mySQL, the code should be robust and report any errors in the process. In addition each time-series should be identified by a unique [SymbolName] and [SymbolNbr]

- The primary keys shall be the [date] and [Symbolid]

- Table for each stock symbol, adding [SymbolName] and [Symbolid] for easy processing

- Separate tables for symbols and other if deemed relevant by the coder

- Separate table for Symbols and other relevant tables as needed

- The code should be simple and easy to modify, with comments

- Use Django for mySQL connection

- Use only well maintained/supported python-packages.

- Use virtualenv and be compatible with OS X-environment

- Make a separate table to consolidate all stocks into one table sorted by date, looking something like this: [date] [stock1Adjustedclose] [stock1Volume] [stock2Adjustedclose] [stock2Volume] ... [stockNadjustedclose] [stockNvolume]

This is a small part of a project, and a well written code will get you hired on other parts.

Similar projects and inspiration:

[url removed, login to view]

[url removed, login to view]

[url removed, login to view] - this one is almost identical to this project, but uses a different connector and stores everything in the same table.

Compétences : Django, MySQL, Python

en voir plus : using stl, stl list example, stl list, stl example, stl data, stl code, start blogspot, sorted data, small project in python,, list stl example, list stl, list in stl, list get python, high volume database, ge com, everything sorted, datasource solutions, csv import data mysql php script, mysql csv create table insert data, mysql historical data stock, export data mysql csv, php mysql data entry user admin edit, mysql export view data csv shell script, importing data mysql csv file

Concernant l'employeur :
( 1 commentaire ) Bergen, Norway

Nº du projet : #8515212

6 freelance font une offre moyenne de €77 pour ce travail


Hello, I specialize in building solutions for stock markets and trading. Here are few examples of my recently completed jobs on this subject: [url removed, login to view] [url removed, login to view] Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% EUR en 3 jours
(30 Commentaires)

Hi I have an extensive experience with Python and web scraping, so I think I can do the job. I would like to do this project using python-requests library ([url removed, login to view]), official mysql-connector-py Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% EUR en 3 jours
(24 Commentaires)

Dear Sir. We claim to get it done perfectly for you EXACTLY in the way you want it - Kindly give we a chance and we will prove myself - Ready to prove our words, let's get it done right away and I mean RIGHT AWAY !! Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% EUR en 3 jours
(23 Commentaires)

Hi, I am a Python developer with experience in creating data processing and visualising utilities, programming Linux daemons, web scrapers, CSV-files processing and databases interaction (SQLite, MySQL, MS SQL). I h Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% EUR en 3 jours
(1 Évaluation)

please let me know the details about your project requirement and we shall have a discussion . I hope you are requirement involves a command line tool in python for importing financial market csv files to mysql.

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% EUR en 5 jours
(0 Commentaires)

Hello there, My name is vijeesh k. I am a python and mysql Developer with more than 5 years of experience. I would love to work on your project. I can complete your project on time and within your budget. I have Plus

%bids___i_sum_sub_32% %project_currencyDetails_sign_sub_33% EUR en 1 jour
(0 Commentaires)