堆栈内存溢出
  简体   繁体   English

运行pyspark时系统找不到路径指定错误

[英]The system cannot find the path specified error while running pyspark

 原文  2018-03-17 19:17:40  1   10    windows/ apache-spark/ pyspark

问题描述

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了 spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的步骤pyspark installation for windows 10 。我使用注释 bin\\pyspark 运行 spark 并收到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附上报错信息截图在此处输入图像描述

Attached is the screen shot of my spark bin folder附件是我的 spark bin 文件夹的屏幕截图在此处输入图像描述

Screen shot of my path variable looks like我的路径变量的屏幕截图看起来像

在此处输入图像描述 在此处输入图像描述 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的 Windows 10 系统中有 python 3.6 和 Java "1.8.0_151" 你能建议我如何解决这个问题吗？

10 个解决方案

解决方案1
 15  2019-03-31 00:39:43

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案2
 14  2019-02-04 09:22:52

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案3
 3  2018-03-27 05:38:47

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案4
 1  2018-08-20 14:48:50

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案5
 0  2018-03-17 19:29:58

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案6
 0  2018-03-18 07:25:23

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案7
 0  2020-12-15 13:01:52

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案8
 0  2020-12-15 21:02:34

I just downloaded spark-2.3.0-bin-hadoop2.7.tgz.我刚刚下载了spark-2.3.0-bin-hadoop2.7.tgz。 After downloading I followed the steps mentioned here pyspark installation for windows 10 .I used the comment bin\\pyspark to run the spark & got error message下载后，我按照此处提到的Windows 10 pyspark安装步骤进行操作。我使用了注释bin \\ pyspark来运行spark和得到错误消息

The system cannot find the path specified

Attached is the screen shot of error message附件是错误消息的屏幕截图在此处输入图片说明

Attached is the screen shot of my spark bin folder附件是我的Spark Bin文件夹的屏幕截图在此处输入图片说明

Screen shot of my path variable looks like我的path变量的屏幕截图看起来像

在此处输入图片说明 在此处输入图片说明 I have python 3.6 & Java "1.8.0_151" in my windows 10 system Can you suggest me how to resolve this issue?我的Windows 10系统中有python 3.6和Java“ 1.8.0_151”，您能建议我如何解决此问题吗？

解决方案9
 0  2021-12-28 17:47:42

对于那些使用 Windows 并仍在尝试的人，解决我的问题是将 Python（3.9）重新安装为本地用户（ c:\\Users\\<user>\\AppData\\Local\\Programs\\Python ）并定义了两个环境变量PYSPARK_PYTHON和PYSPARK_DRIVER_PYTHON c:\\Users\\<user>\\AppData\\Local\\Programs\\Python\\python.exe

解决方案10
 0  2022-01-17 19:23:37

Fixing problems installing Pyspark (Windows)修复安装 Pyspark (Windows) 的问题

Incorrect JAVA_HOME path JAVA_HOME 路径不正确

> pyspark  
The system cannot find the path specified.

Open System Environment variables:打开系统环境变量：

rundll32 sysdm.cpl,EditEnvironmentVariables

Set JAVA_HOME: System Variables > New:设置 JAVA_HOME：系统变量 > 新建：

Variable Name: JAVA_HOME
Variable Value: C:\Program Files\Java\jdk1.8.0_261

Also, check that SPARK_HOME and HADOOP_HOME are correctly set, eg:另外，检查 SPARK_HOME 和 HADOOP_HOME 是否设置正确，例如：

SPARK_HOME=C:\Spark\spark-3.2.0-bin-hadoop3.2
HADOOP_HOME=C:\Spark\spark-3.2.0-bin-hadoop3.2

Important: Double-check the following重要提示：仔细检查以下内容

  1. The path exists路径存在
  2. The path does not contain the bin folder路径不包含bin文件夹

Incorrect Java version Java 版本不正确

> pyspark
WARN SparkContext: Another SparkContext is being constructed 
UserWarning: Failed to initialize Spark session.
java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.storage.StorageUtils$

Ensure that JAVA_HOME is set to Java 8 (jdk1.8.0)确保 JAVA_HOME 设置为 Java 8 (jdk1.8.0)

winutils not installed winutils 未安装

> pyspark
WARN Shell: Did not find winutils.exe
java.io.FileNotFoundException: Could not locate Hadoop executable

Download winutils.exe and copy it to your spark home bin folder下载winutils.exe并将其复制到您的 spark home bin 文件夹中

 curl -OutFile C:\Spark\spark-3.2.0-bin-hadoop3.2\bin\winutils.exe -Uri https://github.com/steveloughran/winutils/raw/master/hadoop-3.0.0/bin/winutils.exe

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 报错：系统找不到指定的路径 - Make error: The system cannot find the path specified Python：[错误3]系统找不到指定的路径： - Python :[Error 3] The system cannot find the path specified: 在Windows 10上运行Kafka失败：系统找不到指定的路径 - Running Kafka on Windows 10 fails: The system cannot find the path specified 该系统找不到指定的路径 - The system cannot find the path specified Windows / Python错误WindowsError：[错误3]系统找不到指定的路径 - Windows/Python Error WindowsError: [Error 3] The system cannot find the path specified cx冻结“ [错误3]系统找不到指定的路径” - cx Freeze “[Error 3] The system cannot find the path specified” MinGW Make throw“系统无法找到指定的路径。”错误 - MinGW Make throwing “The system cannot find the path specified.” error Shuutil.copytree ----&gt; WindowsError: [错误 3] 系统找不到指定的路径： - Shutil.copytree ----> WindowsError: [Error 3] The system cannot find the path specified: Docker Windows 构建失败并显示错误：“系统找不到指定的路径” - Docker Windows build fails with error: “The system cannot find the path specified” 如何修复 Windows 10 上的“系统找不到指定的路径”错误？ - How to fix `The system cannot find the path specified` error on Windows 10?
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM