Find Jobs
Hire Freelancers

Software Engineer I

$35-50 USD / hour

Fermé
Publié il y a environ 4 ans

$35-50 USD / hour

Develop and validate solution architecture to support business requirements Develop Spark applications to read transaction data and process business rules to report errors and transaction summary Load and transform large sets of structured and semi structured. Read data from Kafka and process data using Spark. Adopt innovative architectural approaches to leverage in-house data integration capabilities. Analyze existing processes and prepare functional and requirements documents. Develop multiple spark streaming and core jobs with Kafka as a data pipe-line system. Hive external tables on HBase using Insert Overwrite with S3 as data storage. Troubleshoot developed Spark jobs. Manage and review Yarn Application Logs, Spark Event Logs and Metrics sink CSV files. Improve the performance and optimization of the existing jobs in Spark. Develop solutions by utilizing commercial and open source software including Minifi and Nifi to interface big data and relational solutions.
N° de projet : 23958059

Concernant le projet

16 propositions
Projet à distance
Actif à il y a 4 ans

Cherchez-vous à gagner de l'argent ?

Avantages de faire une offre sur Freelancer

Fixez votre budget et vos délais
Soyez payé pour votre travail
Surlignez votre proposition
Il est gratuit de s'inscrire et de faire des offres sur des travaux
16 freelances proposent en moyenne $45 USD/heure pour ce travail
Avatar de l'utilisateur
Hello. I have a masters degree in AI and have worked with Hadoop, Spark, YARN, ... on a lot of projects. I can share some demos if you are interested. Thanks, Helmot
$35 USD en 40 jours
4,8 (221 commentaires)
8,2
8,2
Avatar de l'utilisateur
Hi there, Being an expert in Python and matlab programming I can do this project for you. please leave a message on my chat so we can discuss the budget and deadline of the project. Also, have a look at my profile for my reviews and portfolios. Thanks Note: This is not an automated bid and i'm ready to discuss your project right now. .. .
$50 USD en 98 jours
5,0 (22 commentaires)
6,3
6,3
Avatar de l'utilisateur
Hi, there. I am a senior Python & Web developer and my main skills are Django and Flask. Fully experienced in each step of web development - design, development, deployment and maintenance, and mastered management of various kinds of DBs including MySQL, MongoDB, PostgreSQL, etc. Especially, I have experience in Big Data Processing using Spark and Hadoop. If you're interested, please have a discussion with me. Thanks.
$40 USD en 40 jours
4,8 (5 commentaires)
4,9
4,9
Avatar de l'utilisateur
Hi, I have 9 years of experience and working on hadoop, spark, Kafka, nosql, java, BI tools(tableau, powerbi), cloud(Amazon, Google, Microsoft Azure)... Done end to end data warehouse management projects on aws cloud with hadoop, hive, spark and presodb. Worked on multiple etl project like springboot, angular, node, PHP, Kafka, nifi, flume, mapreduce, spark with XML/JSON., Cassandra, mongodb, hbase, redis, oracle, sap hana, ASE.... Many more. Let's discuss the required things in detail. I am committed to work done and strong in issue resolving as well. Thanks
$50 USD en 30 jours
5,0 (7 commentaires)
4,0
4,0
Avatar de l'utilisateur
Hello! I'd like to optimize information aggregation. I'm familiar with python and theory of probabilities. I'll do the job blazingly fast. Please, give me a try!
$56 USD en 80 jours
5,0 (3 commentaires)
4,0
4,0
Avatar de l'utilisateur
Highly effective Data Engineer and Data Scientist with over 25 years of experience. I am specilaizing Big Data and have extensive Python / Spark experience both in PySpark and Scala also in MLLib. I developed a number of ML models including linear regression, logistic regression and multi classfication using various libraries. TECHNICAL SKILLS • Big Data (Hortonworks and Cloudera) – Spark(PySpark, Scala), Kafka, Hive,Impala, NiFi, HDFS, Sqoop, Ranger, Yarn, Solr, SAM, Schema Registry, SuperSet, Druid • Language: Python, Scala, R, JavaScript • Data Visualization – Tableau, PowerBI, OBIEE, DOMO • Plunk & ELK Stack – ElasticSearch, Logstash, Filebeat, Kibana • AWS – S3, RefShift, DynamoDB, Athena, Kinesis, EMR, Aurora, Glue • Azure – Data Warehouse, Polybase, SQL Server, HDInsight, SSIS, SSAS • ETL – Informatica Power Centre, Informatica Big Data Management (BDM), SSIS • Data Science & Engineering – R / RStudio / SparkR Packages: dplyr, ggplot2, stringr, plyr, carrat, SparkR, NLP, tibble, TensorFlow, curl, Python / PySpark, MLLib • Libraries: MLLib, NumPy, SciPy, Pandas, Matplotlib, Seaborn, SciKit-Learn • DBMS / OLAP – Oracle, SQL Server, TeraData, MySQL, Postgres, Essbase, SSAS • Data Modelling – Kimball, Vault • Machine Learning / Statistical Modelling - Linear Regression, ?Logistic Regression, Classification and Regression Trees, Naive Bayes, ?K-Nearest Neighbors
$55 USD en 40 jours
5,0 (2 commentaires)
4,0
4,0
Avatar de l'utilisateur
I am currently working on each item which you described. I delivered end to end solution including kafka, spark, hive, nifi and all other required hadoop eco system components. Please let connect or have a call to discuss more in your requirements and also if you like with skills then only we move forward. Incase if you need any demo/POC work, I am ready to take that as well. I hope we will be connecting soon, I am available on my mobile, so feel free to connect at any time.
$44 USD en 20 jours
5,0 (2 commentaires)
2,2
2,2
Avatar de l'utilisateur
Hi there, I have quite some experience in most of the big data tech out there (Hadoop, Spark, Kafka, Cassandra, ...), since I have been specializing on Data Management, Data Consolidation etc. for quite a long time, now. All in all, I am a true Full-Stack developer (leveraging mainly Node.JS, Python, Java, Ruby, etc.) and can thus maintain the full stack, build a GraphQL layer on top of the data, in order to allow easy access to it or build some frontend for it, as well. Additionally, I have quite some experience on the Business side of things, so I know how to manage a project (even if it is only myself working on it), communicate with key stakeholders and preparing documentation for technical, but also non-technical users. I would love to have a talk with you, since I am always searching for long-lasting partnerships, so it's got to be right, of course! P.S.: This seems to be a REALLY exciting project, so I just want to make sure to point out, that I think that I would be perfect to tackle this, even though they probably all say that, but I would really love a shot at this!
$50 USD en 40 jours
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
I have developed country wide log parsing application using Apache Spark on Hadoop. I have experienced memory issues deeply. Sqoop Metastore Oozie workflow pipelines were used. Machine Learning is another side of thmy Big Data jobs. I use spring boot and Vue js for presentation layer. As a Software engineer I am working in a CMMI level 4 organisation.
$43 USD en 40 jours
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
Have the experience of over 5 years in developing Big data pipelines over multiple technologies, hold certification in Hortonworks and Cloudera platforms. Currently employed full time as analytics application engineer with large corporation. Can contribute 20 hours per week
$40 USD en 20 jours
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
Hi. I have a strong background in big data and Hadoop. I have worked as a Senior Data Platform Engineer and on technologies like spark, AWS, mapreduce, hbase, hive, Google bigquery, Kafka, Google cloud platform. I would love to discuss more about the position.
$55 USD en 30 jours
0,0 (0 commentaires)
0,0
0,0

À propos du client

Drapeau de UNITED STATES
Los Angeles, United States
0,0
0
Membre depuis févr. 17, 2020

Vérification du client

Autres travaux de ce client

Integration Engineer
min $50 USD / hour
Merci ! Nous vous avons envoyé un lien par e-mail afin de réclamer votre crédit gratuit.
Une erreur a eu lieu lors de l'envoi de votre e-mail. Veuillez réessayer.
Utilisateurs enregistrés Total des travaux publiés
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Chargement de l'aperçu
Permission donnée pour la géolocalisation.
Votre session de connexion a expiré et vous avez été déconnecté. Veuillez vous connecter à nouveau.