We have a series of text files which contain lists of log file information in this format ->
There is no delimitation around the fields other than the full colon ":" separating the 2 values and obviously a new line at the end of the 2nd value. There are however some errors in these files, some of the lines don't follow the correct format above and we need these removing otherwise we wont be able to use LOAD DATA into MySQL when we come to import them.
These files are stored in multiple nested directories, some of which are 5 or 6 levels deep and each directory can contain between 50 and 200 text files.
We are looking for someone to write a simple BASH script to run on ubuntu 16.04 in the root directory of all these files. The script will traverse its way down each file and directory and cleanse the data. The script will need to open each file, read it one line at a time and check that each line conforms to the above format. If it is in the correct format, add it to a new "cleansed data" text file in the same (value1:value2) format in the current directory being checked. When its finished reading all the files in that current directory, it will save the cleansed data text file and move onto the next dorectory.
We have 98 directories of this data and hundreds of files within them. The output of the script will be 98 "cleansed" text files, one in each directory containing all the correctly formatted lines from all the files in that directory.
We'll then use each new cleansed data file with the LOAD DATA facility to upload the new value1:value2 checked pairs.
HI, I have ten years commercial Linux experience as a systems administrator. I can write a script to filter your files. Hope to speak with you soon.
11 freelance font une offre moyenne de £57 pour ce travail
Bash is one of my best skills. I use it every day. All I would need is a few sample files, and some nodes about data types in the file.
I have worked as server admin for big names like [login to view URL] and servercentre.net. Over 8yrs of working with cpanel, directadmin and plesk servers. Can help you with this. Let me know.
Hello I can do it immediately in python Thanks ...............................................................
Hello Sir, I am agile safe 4.0 certified I have 7 year of experience in Software development including technologies java, j2ee, spring , hibernate, ,Rest apis, php unix shell scripts, apache tomcat, jboss, server co Plus
Hi, there. I am happy to apply in your project. Your project is interesting to me. An experienced Bash/Shell script expert here. If you want to review me, you can go through my feedbacks and git repos. https://git Plus
Hi, I can write a python script for this that will full fill the purpose. As you will require data cleanse, so I feel no perfect script can do. There will always scenarios that are not present in current data but can b Plus
Hello, I'm interested in the project. Right now I have a small web development company with 2-3 web developers. We are currently working on the back-end of our website and when we fully launch our front-end will be com Plus
Hi, I wrote many such nifty content aware scripts. This is doable with shellscript, ruby, perl and almost any language. I will choose the one that will get the job done best. I hope you decide for me and be amazed!
6 Years of Experience, Expert in all kind of applications, Laravel 5.1, 5.5, Spring Mysql, SQL Server.
As your description of what the script should do sounds quite complete, as I understand it, the script should: - take as a parameter the root directory of where the log files are - recursively traverse the entire tree Plus