AttributeError: 'str' object has no attribute 'show' PySpark

Question

AttributeError: 'str' object has no attribute 'show'

I am trying to pass any test json file as part of the command line argument. When doing so it treats it as a string , which I dont want but I want it to be treated as a DataFrame so it can show the dataframe with df.show(). I get this error message on the terminal saying AttributeError: 'str' object has no attribute 'show'.

df = sqlContext.read.json(“tester.json")

def show_data(df):
      df.show()

parser = argparse.ArgumentParser()
parser.add_argument(‘-I’, ‘—inputfile', required=True , default =df)
args = parser.parse_args()
show_data(args.parameterfile)

Answer 1

The problem is that you are passing a string to the show_data function.

Working code:

import argparse

def show_data(input_filename):
    df = sqlContext.read.json(input_filename)
    df.show()


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("filename", help="path of the file of interest")
    args = parser.parse_args()
    show_data(args.filename)

Hope this helps