嘗試在亞馬遜 AWS 實例上安裝 keras & tensorflow

Question

嘗試在 Ubuntu EC2 實例上安裝 keras 和 tensorflow 時遇到一些問題。

我加載庫並嘗試安裝 keras。

library(keras)
library(tensorflow)
install_keras(tensorflow = "gpu")

我收到以下錯誤：

 Error: could not find a Python environment for /usr/bin/python

我還運行了以下命令以及一些其他終端命令來嘗試安裝 conda 和 python 等。

sudo apt install git libpython3.7-dev

我也跟着 keras.io 網站嘗試安裝它。 我在服務器上有一個名為/usr/bin/python3.6:的文件夾，但是當我嘗試訪問它時出現錯誤Directory /usr/bin/python3.6: no such file or directory 。 我在服務器的home目錄中有兩個用戶帳戶（以及兩個同名文件夾） ubuntu和myUsername_folder 。

在/home/myUsername_folder我有.rstudio文件。 ( /home/myUsername_folder/.rstudio ) 和一些數據集。 在home/ubuntu文件夾中，我安裝了 R /home/ubuntu/R/x86_64-pc-linux-gnu-library/3.6 。 在/根目錄中，我有usr 、 var 、 lib64 、 tmp 、 bin等文件夾。

我如何告訴 R Python 位於何處？

編輯：

我重新啟動 RStudio 並運行：

> library(keras)
> library(tensorflow)
> install_keras(tensorflow = "gpu")
Using virtual environment '~/.virtualenvs/r-reticulate' ...
/home/myUser/.virtualenvs/r-reticulate/bin/python: No module named pip
Error in strsplit(output, "\\s+")[[1]] : subscript out of bounds
In addition: Warning message:
In system2(python, c("-m", "pip", "--version"), stdout = TRUE) :
  running command ''/home/myUser/.virtualenvs/r-reticulate/bin/python' -m pip --version' had status 1

嘗試使用以下方法安裝pip ：

sudo apt-get install python3-pip

Reading package lists... Done
Building dependency tree
Reading state information... Done
python3-pip is already the newest version (9.0.1-2.3~ubuntu1.18.04.1).
0 upgraded, 0 newly installed, 0 to remove and 39 not upgraded.

我運行以下命令來安裝 Anaconda：

wget https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh bash Anaconda3-2019.10-Linux-x86_64.sh

然后跑：

> install_keras(method = c("conda"),
+               conda = "auto", version = "default", tensorflow = "gpu")
ERROR: The install method you used for conda--probably either `pip install conda`
or `easy_install conda`--is not compatible with using conda as an application.
If your intention is to install conda as a standalone application, currently
supported install methods include the Anaconda installer and the miniconda
installer.  You can download the miniconda installer from
https://conda.io/miniconda.html.

Error: Error 1 occurred creating conda environment r-reticulate

編輯：我試過：

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh bash Miniconda3-latest-Linux-x86_64.sh

然后在 RStudio 中

> install_keras(method = c("conda"),
+               conda = "/home/myUser/miniconda3/bin/",
+               version = "default", tensorflow = "gpu")
sh: 1: /home/myuser/miniconda3/bin/: Permission denied
Error: Error 126 occurred creating conda environment r-reticulate

我更改權限：

sudo chmod -R 777 /home/myUser/miniconda3/bin/

並在運行install_keras(...)后得到相同的錯誤。

編輯：

我在 RStudio 中運行

tf_config()

未找到 TensorFlow 的安裝。

Python environments searched for 'tensorflow' package:
 /usr/bin/python3.6
 /usr/bin/python3.6
 /usr/bin/python2.7
 /home/myuser/anaconda3/bin/python3.7
 /usr/bin/python3.6
 /home/myuser/miniconda3/bin/python3.7

您可以使用 install_tensorflow() 函數安裝 TensorFlow。

install_tensorflow()

錯誤：找不到 /usr/bin/python 的 Python 環境

我也試過use_python("/home/myuser/anaconda3/bin/python3.7")

編輯

上面的問題仍然存在，但我可以運行以下 R 代碼（“有效”）：

nn_dat = iris %>% as_tibble %>%
  mutate(sepal_l_feat = scale(Sepal.Length),
         sepal_w_feat = scale(Sepal.Width),
         petal_l_feat = scale(Petal.Length),
         petal_w_feat = scale(Petal.Width),          
         class_num    = as.numeric(Species) - 1, # factor, so = 0, 1, 2
         class_label  = Species) %>%
  select(contains("feat"), class_num, class_label)
nn_dat %>% head(3)

test_f = 0.20
nn_dat = nn_dat %>%
  mutate(partition = sample(c('train','test'), nrow(.), replace = TRUE, prob = c(1 - test_f, test_f)))

x_train = nn_dat %>% filter(partition == 'train') %>% select(contains("feat")) %>% as.matrix
y_train = nn_dat %>% filter(partition == 'train') %>% pull(class_num) %>% to_categorical(3)
x_test  = nn_dat %>% filter(partition == 'test')  %>% select(contains("feat")) %>% as.matrix
y_test  = nn_dat %>% filter(partition == 'test')  %>% pull(class_num) %>% to_categorical(3)

model = keras_model_sequential()
model %>% 
  layer_dense(units = 4, activation = 'relu', input_shape = 4) %>% 
  layer_dense(units = 3, activation = 'softmax')
model %>% summary

model %>% compile(
  loss      = 'categorical_crossentropy',
  optimizer = optimizer_rmsprop(),
  metrics   = c('accuracy')
)

history = model %>% fit(
  x = x_train, y = y_train,
  epochs           = 200,
  batch_size       = 20,
  validation_split = 0
)
plot(history)

在這里我收到這條消息：

/home/myuser/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/myuser/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

2020-01-21 21:49:45.344207: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-01-21 21:49:45.384211: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2400050000 Hz
2020-01-21 21:49:45.384567: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55fc4e3c43b0 executing computations on platform Host. Devices:
2020-01-21 21:49:45.384598: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2020-01-21 21:49:45.448620: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.

然后運行：

perf = model %>% evaluate(x_test, y_test)
print(perf)

$loss
[1] 0.2995828

$acc
[1] 0.84375

所以我可以使用keras & tensor flow，但我仍然收到如上所示的錯誤消息。

Answer 1

好吧，我以前沒有使用過 R，但是，在使用諸如 Tensorflow 之類的庫時，我遇到了類似的環境問題。 當我為深度學習工作安裝 Tensorflow 和 Keras 時，我遇到了同樣的問題。 就我而言，我發現漏洞問題是由於 Tensorflow 庫本身造成的。 我使用 Windows，但這也可能適用於 Linux，所以我所做的是：

1. 從您的計算機上卸載所有 Conda、Python 和 R（包括附加庫，例如 tensorflow）。
2. 我下載了“ANACONDA”的las版本，但只下載了ANACONDA，沒有別的。 這實際上就是消除錯誤所需的全部內容。
3. 繼續安裝，然后您將擁有 Anaconda 導航器。 默認情況下（在 anaconda 導航器中）包括 app 和 python，例如 Jupyter、Spyder 和 R。
4. 轉到環境部分，創建一個專用環境，您將在其中安裝 Tensorflow 庫。 您可以在 Anaconda 導航器內的導航器中鍵入名稱，以查找您正在搜索的庫。
5. 確保在剛剛創建的新環境中安裝了所需的孔庫。
6. 返回主頁部分，確保您已選擇“新創建的環境”，安裝您需要的應用程序，這些應用程序可能在下面。 其中R是可行的。

這應該有效，希望對您有所幫助。 這種方式比傳統的pip安裝過程更加清晰明了。 請注意，Tensorflow 不是一個簡單的庫，通過使用此方法，ANACONDA 導航器可以很好地組織所有內容，不會出現未來錯誤。

Answer 2

確保正確安裝了 conda 或 virtualenv。