AWS S3 text files to Relational Database

A little background - I am using AWS to manage a mountain bike timing system. Competitors tag on and off a number of different stages and the tag off time minus the tag on time is the stage racetime and the lowest cumulative time over all stages wins.

Out on the racecourse, I am using Raspberry Pi's with RFID readers which collect and store data in text files which automatically upload to the AWS S3 bucket. I currently have two files in there: Stage 1 [url removed, login to view] and Stage 1 Finish.txt.

I need to get these files into a database to join with a competitor registration database so I can ultimately compute racetimes in AWS, extract some analytics and allow competitors to search and compare times using their surname or racing number to find their race times.

In short, I need help converting these text files into a relational database and this process is automated so that when new items are appended to the text files that they update the database.

I think a Data Pipeline is the way to go but happy to listen to alternative recommendations to achieve my ultimate goals as described above.

The text file item/row is semi-colon delimited as follows:

[url removed, login to view];1s;2016/12/30;13:56:23.235;00235689

Meaning log information (not needed), 1s (Stage 1 Start), Date, Time to milliseconds and finally the RFID code.

Any help with the Data Pipeline (or another alternative) is required as soon as possible.



Compétences : Services Web d'Amazon, Administration de Bases de données, MySQL

Voir plus : translate audio files text, auduio files text, exporting ning database files, relational database video rent, psd examples files text, relational database project, transcribe sound files text, project database files mega retail store, type picture files text money, perl search files text, php mysql relational database frontend, transcribe audio files text, mysql database hold text files, create html files text files, audio files text

Concernant l'employeur :
( 9 commentaires ) Mandurah, Australia

N° du projet : #12676953

Décerné à :


I am highly skilled in the AWS environment. I have been working with AWS for 5 years now and run multiple production workloads across many AWS services. I have both the AWS Devops Engineer and AWS Certified Solutions Plus

250 $ AUD en 3 jours
(15 Commentaires)

20 freelance font une offre moyenne de $170 pour ce travail


I have 9 years experience in JAVA, J2EE, WebServices and enough experience in AWS webservices. I am not sure if we can dine the pipeline such that it extracts te file and fills the required columns of a relational dat Plus

166 $ AUD en 5 jours
(25 Commentaires)

A proposal has not yet been provided

150 $ AUD en 2 jours
(22 Commentaires)

Please come online for detail. My Skype Id is [url removed, login to view] I am a highly skilled senior developer with extensive development experience in system design, .NET technology stack including C#, ASP.NET, MVC, Web Api, SQL Plus

155 $ AUD en 3 jours
(15 Commentaires)

I can write a php script which can load data from text file to database. I would love ot work on something new and good thing ..

155 $ AUD en 3 jours
(12 Commentaires)

Hello there. I think lambda could be the solution for you. I can take care of this task if you wish I am an experienced developer with strong aws knowledge

155 $ AUD en 3 jours
(1 Commentaire)

I'm active in graphic design, web and app development for the last 8+ years. I believe in delivering the best possible solution that can be implemented in your business strategies. I'm serving more than 300 clients aro Plus

250 $ AUD en 7 jours
(6 Commentaires)

sir,this is easy task for me.i will complete your work as soon as possible with out any [url removed, login to view] give me the chance to work with [url removed, login to view] for your reply. Thanks

35 $ AUD en 1 jour
(4 Commentaires)

Hello there i have read your description, i will recommend a web application for this, can you please contact to discuss in details? i am waiting for your pm thanks

250 $ AUD en 3 jours
(7 Commentaires)
144 $ AUD en 3 jours
(2 Commentaires)
111 $ AUD en 3 jours
(8 Commentaires)

Hi, I have over 7 years of experience as a database developer. I have completed three recent projects. All clients were 100% satisfied as you can see from their reviews. One of the projects I completed recently was Plus

170 $ AUD en 3 jours
(3 Commentaires)

i have similar experience working with S3 and mysql db.

222 $ AUD en 4 jours
(2 Commentaires)

Hi, I have read through your project description. I will work as per your direction with 100% accuracy. And I will deliver your project before your timeline. I am waiting for your response to discuss more about the pro Plus

150 $ AUD en 1 jour
(1 Commentaire)

Hi, Firstly, Thank you for reviewing my proposal. I have gone through your requirements and came to know that you need an expert web developer to build your website. I have strong knowledge and 3+ years experience i Plus

222 $ AUD en 3 jours
(0 Commentaires)

Hello! I can do the job u need. I think Python is the best solution for you as it's covering all your needs. On the other hand maybe it makes sense to use logstash to transform and send data to your database and do Plus

222 $ AUD en 10 jours
(0 Commentaires)
155 $ AUD en 3 jours
(0 Commentaires)

you don't have to load your data to different database unless you have compelling reasons. Now amazon athena supports query the s3 data like sql queries. Much less expensive and less maintenance than using different da Plus

155 $ AUD en 3 jours
(0 Commentaires)

Hi sir, We assure you the quality of service through various checks that are employed to avoid any compromise on quality of work. Let me introduce my team to you and generally what services we offer to our client Plus

150 $ AUD en 8 jours
(0 Commentaires)

Have 4.5 years of experience in Java, REST, Data Structures and Algorithms, SQL and NoSQL(MongoDB, Redis), Spring, Hibernate hence I will be a good help for this project. I had recently worked with Rajhees, C.E.O, MD , Plus

155 $ AUD en 3 jours
(0 Commentaires)
222 $ AUD en 5 jours
(0 Commentaires)