How to read .pkl file using python in hdfs

Question

I'm using Python 3.5.2 in IBM cloud. I need to run a pyspark code in cluster environment. For this work I need to upload pickle file in hdfs using python. I saw the comments here

I want to upload terrs.pkl. The file location is given in attached screen shot

I'm using following command as per the suggestion given in the link that I shared

pickle.load( open( '/user/clsadmin/terrs.pkl', "rb" ) )

But it's throwing error that

No such file or directory: '/user/clsadmin/terrs.pkl'

Please refer the attached screen shot. Can you suggest me how do I load this .pkl file using python?

Answer 1

open_file = open("/user/clsadmin/terrs.pkl", "rb")
data = pickle.load(open_file)
open_file.close()