从pyspark shell运行脚本

Question

I can run my python+pyspark script from the unix command line by typing

pyspark script.py

But how do I run script.py from within the pyspark shell? This seems like an elementary question but I can't find the answer anywhere. I tried

execfile('script.py')

But I get an error which includes:

ValueError: Cannot run multiple SparkContexts at once

Answer 1

Could the error come from script.py trying to create a new SparkContext variable?

When you launch the pyspark interactive client it usually says : SparkContext available as sc, HiveContext available as sqlContext.

If your script file contains sc = SparkContext() , maybe try commenting it.