Breaking News: Grepper is joining You.com. Read the official announcement!

Convert PySpark RDD to DataFrame

Rakesh Meher answered on July 7, 2022 Popularity 9/10 Helpfulness 1/10

answer Convert PySpark RDD to DataFrame

Convert PySpark RDD to DataFrame

Comment

-1

# importing necessary libraries
from pyspark.sql import SparkSession
  
# function to create new SparkSession
def create_session():
    spk = SparkSession.builder \
        .appName("Corona_cases_statewise.com") \
        .getOrCreate()
    return spk
  
# function to create RDD
def create_RDD(sc_obj, data):
    df = sc.parallelize(data)
    return df
  
  
if __name__ == "__main__":
  
    input_data = [("Uttar Pradesh", 122000, 89600, 12238),
                  ("Maharashtra", 454000, 380000, 67985),
                  ("Tamil Nadu", 115000, 102000, 13933),
                  ("Karnataka", 147000, 111000, 15306),
                  ("Kerala", 153000, 124000, 5259)]
  
    # calling function to create SparkSession
    spark = create_session()
  
    # creating spark context object
    sc = spark.sparkContext
  
    # calling function to create RDD
    rd_df = create_RDD(sc, input_data)
  
    # printing the type
    print(type(rd_df))

Popularity 9/10 Helpfulness 1/10 Language python

Source: www.geeksforgeeks.org

Tags: dataframe pyspark python rdd

Link to this answer
Share Copy Link

Contributed on Jul 07 2022

Rakesh Meher

0 Answers Avg Quality 2/10

Convert PySpark RDD to DataFrame

Contents

More Related Answers

Convert PySpark RDD to DataFrame

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.