- Programming Language: JAVA or Python or C programming
- Required Experience: Network Big Data Processing
- Open Source Management System: InfluxDB, MariaDB, MongoDB (You can choose one of them for your database But, InfluxDB is preferred)
- Library: nfcapd & nfdump ( [login to view URL] ),
- Protocol: netflow, jflow, netstream, sflow
- OS: Linux, CentOS7
- Server requirement:
Basically, an assigned developer must build own server and need a confirmation of OS version, programming language version, and DB version with Next-core Technology.
Specification of our server being currently used is below. This specification may be improved. Thus, if an assigned developer has a suggestion regarding improvement (optimization) in server’s specification, we are open to consider the suggestion.
CPU: Intel® Xeon® CPU 2.3 GHz
2P24C/MEM: 128G/HDD: [login to view URL]
Development Period: 3 weeks (=15 business days but negotiable before the project begins)
Inspection Period: 4 business days (included in development period)
Development Cost: 1st requested way: USD $ 1,840 / 2nd requested way: USD $ need discussion
Project Objective: To optimize (or minimize) speed (from 5-7mins to within 2 mins) of present Big (Flow) Data processing (2 ways: 1) csv file to DB & 2) binary file to DB) via MariaDB or MongoDB, or InfluxDB using JAVA or Python or C programming
During data processing for about 5 millions of flow data, we are seeking to optimizing (or minimizing) current speed (5~7 minutes) of data processing from Server and database. The current speed must be within 2 minutes. Please see attachment called ""[login to view URL]" for your understanding.
As you see the workflow and the project objective above, our project is to optimize current data processing between Server and Database into 2 mins or less. When the objective is accomplished, Next-core Technology must retrieve source codes and other applicable references for the objective. Next are information of sample data and detailed course instructions in our project.
1. Meta data (Field information) and sample data masked (1.5GB csv file) will be provided. We just attached data field information and examples of sample data first. Later, we will provide masked data. Please see the attached file for your development.
2. For this project, 5 millions of flow data will be standardized amounts. As you know, our objective is to make duration time of data processing between Server and Database into 2 minutes or less. Thus, we must make this 2 minutes or less with 5 million of flow data. To be specific, if you run 2.5 million data, your speed of data processing must be within 1 minute.
3. All masked data given will be flow data so that you should be familiar with their characteristics. For the 1st way (csv file to DB), you can refer to field information PDF file and csv file for your understanding. For the 2nd way (binary file to DB), we will discuss about which data to use after 1st way is figured out.
Please read carefully detailed courses of the development by opening the attached file called "Project [login to view URL]" before you approach to us. This guides you how to complete the project's objective.
If you have any other question, please let us know via chat @ Freelancer.
13 freelances font une offre moyenne de 2498 $ pour ce travail
Hi, I am interested, gone through the docs. Hire me /////////////////////////////////////////////////////////////
Hi I have experience in big data processing and so am very interested in your job. I'll complete your job PERFECTLY. I am ready to start right now. Best Regards.
Hi My name is Monil Patel and I am Senior Systems Programmer. I have experience creating script and automates processes like this for over 5 years. I have a very strong background in software development and console a Plus
Hi, I will customize application using the Oracle intelligent advisor(Oracle policy automation) and sort the data and offer the fast data processing for [login to view URL] let me know do i need to fill the form as per the repl Plus
Hi Sir, I can make your website so fast than many websites. I am full-stack developer and expert in Wordpress, HTML, CSS, Web design, Graphic design, MySQL and PHP I have more than 7 years of experience in develop Plus
I have read your requirements, i am an experienced Wordpress developer and expert Wordpress website speed optimization, I can get score of 85+ to 90 btw with average loading time of 3s to 2s on gtmetrix or google page Plus
Hello,your project is very motivated for me. As a senior developer 8+ years of experience python and Software Architecture.I have great experience to work on API,s and full stack development. It would be amazing experi Plus
Narinder Alliance Technologies LLC An IT Consulting and Software Development company. We have a team expert in Web Designing, Application Development and Databases. Major offerings include Digital Transformation, Appl Plus
Kindest regards from the viitorcloud!! Much appreciated for the great opportunity. We are innovation driven IT development and consulting services company. Inline notes about us: Serving Since: 10+ Years Team Size Plus
Hi, We are a team of data science and ML/AI experts who excel across multiple areas with more than twenty five years of combined experience. We hold expertise in Python, Backend Architecture (micro-services, Kubernete Plus