Fermé

Spark code to generate test data

I need to generate test data using spark code in HDFS path. If storing in AWS will be also more useful

Requirments :

We need to give the column names that needs to be created

Number of rows to be generated

Output format can be csv,parquet,txt,json

For the columns created we need to provide the data from another file

Compétences : Hadoop, Spark

Concernant le client :
( 1 commentaire ) Chennai, India

Nº du projet : #33736989

3 freelances font une offre moyenne de 1117 ₹ pour ce travail

sarvan92

Hai, I am Bigdata engineer and I am having rich experience in data pipelines and data processing on Hadoop,Azure and AWS using pyspark and java I can build a simple script for your requirements and we can make a great Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% INR en 7 jours
(0 Commentaires)
0.0
developeratkiit

Hi, I am a certified Azure Solution Architect and Data Engineer with Vast experience on on-prem spark and Databricks on Azure. I have 10+ years of experinece working in Data and Analytics using ETL, SQL and Spark.

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% INR en 7 jours
(0 Commentaires)
0.0
magoacua26

I have 6 years of experience working with Spark, Hadoop, Cloudera, Impala, Hive. I also have experience in Java and Python.

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% INR en 7 jours
(0 Commentaires)
0.0