如何制作 pyspark - 在 Windows 命令提示符下 - 运行 jupyter notebook

Question

I have anaconda python running Jupyter perfectly i have Hadoop, yarn and spark running on windows 10 cmd perfectly. I changed a lot of variables in the system of windows but now works fine

when running PySpark, it works

but I want to start Jupyter notebook when I run PySpark on cmd, and cannot

Answer 1

This should work, go on powershell change this env variables.

$env:PYSPARK_DRIVER_PYTHON=jupyter
$env:PYSPARK_DRIVER_PYTHON_OPTS='notebook'

or CMD

SET PYSPARK_DRIVER_PYTHON=jupyter
set PYSPARK_DRIVER_PYTHON_OPTS='notebook'

and then run pyspark, it will automatically execute your jupyter notebook.