我如何启动 tensorflow docker jupyter notebook

Question

I've installed the tensorflow docker container on an ubuntu machine. The tensorflow docker setup instructions specify:

docker run -it b.gcr.io/tensorflow/tensorflow

This puts me into the docker container terminal, and I can run python and execute the Hello World example. I can also manually run .\\run_jupyter.sh to start the jupyter notebook. However, I can't reach the notebook from host.

How do I start the jupyter notebook such that I can use the notebook from the host machine? Ideally I would like to use docker to launch the container and start jupyter in a single command.

Answer 1

For a Linux host Robert Graves answer will work, but for Mac OS X or Windows there is more to be done because docker runs in a virtual machine.

So to begin launch the docker shell (or any shell if you are using Linux) and run the following command to launch a new TensorFlow container:

docker run -p 8888:8888 -p 6006:6006 b.gcr.io/tensorflow/tensorflow ./run_jupyter.sh

Then for Mac OS X and Windows you need to do the following only once:

1. Open VirtualBox
2. Click on the docker vm (mine was automatically named "default")
3. Open the settings by clicking settings
4. In the network settings open the port forwarding dialog
5. Click the + symbol to add another port and connect a port from your mac to the VM by filling in the dialog as shown below. In this example I chose port 8810 because I run other notebooks using port 8888.
6. then open a browser and connect to http://localhost:8810 (or whichever port you set in the host port section
7. Make your fancy pants machine learning app!

Answer 2

My simple yet efficient workflow:

TL;DR version:

1. Open Docker Quickstart Terminal . If it is already open, run $ cd
2. Run this once : $ docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/tensorflow:/notebooks --name tf b.gcr.io/tensorflow/tensorflow
3. To start every time: $ docker start -i tf

If you are not on windows, you should probably change /$(pwd) to $(pwd)

You will get an empty folder named tensorflow in your home directory for use as a persistent storage of project files such as Ipython Notebooks and datasets.

Explanation:

1. cd for making sure you are in your home directory.
2. params:
   • -it stands for interactive, so you can interact with the container in the terminal environment.
   • -v host_folder:container_folder enables sharing a folder between the host and the container. The host folder should be inside your home directory. /$(pwd) translates to //c/Users/YOUR_USER_DIR in Windows 10. This folder is seen as notebooks directory in the container which is used by Ipython/Jupyter Notebook.
   • --name tf assigns the name tf to the container.
   • -p 8888:8888 -p 6006:6006 mapping ports of container to host, first pair for Jupyter notebook, the second one for Tensorboard
3. -i stands for interactive

Running TensorFlow on the cloud

Answer 3

After further reading of docker documentation I have a solution that works for me:

docker run -p 8888:8888 -p 6006:6006 b.gcr.io/tensorflow/tensorflow ./run_jupyter.sh

The -p 8888:8888 and -p 6006:6006 expose the container ports to the host on the same port number. If you just use -p 8888, a random port on the host will be assigned.

The ./run_jupyter.sh tells docker what to execute within the container.

With this command, I can use a browser on the host machine to connect to http://localhost:8888/ and access the jupyter notebook.

UPDATE: After wrestling with docker on windows I switched back to a Ubuntu machine with docker. My notebook was being erased between docker sessions which makes sense after reading more docker documentation. Here is an updated command which also mounts a host directory within the container and starts jupyter pointing to that mounted directory. Now my notebook is saved on the host and will be available next time start up tensorflow.

docker run -p 8888:8888 -p 6006:6006 -v /home/rob/notebook:/notebook b.gcr.io/tensorflow/tensorflow sh -c "jupyter notebook /notebook"

Answer 4

These steps worked for me if you are a total docker noob using a windows machine.

Versions: Windows 8.1, docker 1.10.3, tensorflow r0.7

1. Run Docker Quickstart Terminal
2. After it is loaded, note the ip address. If you can't find it use this docker-machine ip and make a note. Lets call it 'ip address'. Will look something like this: 192.168.99.104 (I made up this ip address)

3. Paste this command on the docker terminal:

   docker run -p 8888:8888 -p 6006:6006 b.gcr.io/tensorflow/tensorflow .

   If you are running this for the first time, it will download and install the image on this light weight vm. Then it should say 'The Jupyter notebook is running at ....' -> This is a good sign!

4. Open your browser at: <your ip address (see above)>:8888 . Eg. 192.168.99.104:8888/
5. Hopefully you can see your ipython files.

Answer 5

Jupyter 现在可以为 TensorFlow 运行Docker 镜像：

docker run -d -v $(pwd):/home/jovyan/work -p 8888:8888 jupyter/tensorflow-notebook

Answer 6

To get this to run under hyper-v. Perform the following steps:

1) Create a docker virtual machine using https://blogs.msdn.microsoft.com/scicoria/2014/10/09/getting-docker-running-on-hyper-v-8-1-2012-r2/ this will get you a working docker container. You can connect to it via the console or via ssh. I'd put at least 8gb of memory since I'm sure this will use a lot of memory.

2) run "ifconfig" to determine the IP address of the Docker VM

3) On the docker shell prompt type:

docker run -p 8888:8888 -p 6006:6006 -it b.gcr.io/tensorflow/tensorflow

4) Connect to the Jupyter Workbench using http:/[ifconfig address]:8888/

Answer 7

To tidy up the things a little bit, I want to give some additional explanations because I also suffered a lot setting up docker with tensorflow. For this I refer to this video which is unfortunately not selfexplanatory in all cases. I assume you already installed docker. The really interesting general part of the video starts at minute 0:44 where he finally started docker. Until there he only downloads the tensorflow repo into the folder, that he then mounts into the container. You can of course put anything else into the container and access it later in the docker VM.

---

1. First he runs the long docker command docker run –dit -v /c/Users/Jay/:/media/disk –p 8000 –p 8888 –p 6006 b.gcr.io/tensorflow/tensorflow . The “run” command starts containers. In this case it starts the container “b.gcr.io/tensorflow/tensorflow”, whose address is provided within the tensorflow docker installation tutorial . The container will be downloaded by docker if not already locally available. Then he gives two additional kinds of arguments: He mounts a folder of the hostsystem at the given path to the container. DO NOT forget to give the partition in the beginning (eg. "/c/"). Additionally he declares ports being available later from the host machine with the params -p. From all this command you get back the [CONTAINER_ID] of this container execution! You can always see the currently running containers by running “docker ps” in the docker console. Your container created above should appear in this list with the same id.

   ---

2. Next Step: With your container running, you now want to execute something in it. In our case jupyter notebook or tensorflow or whatever: To do this you make docker execute the bash on the newly created container: docker exec –ti [CONTAINER_ID] bash . This command now starts a bash shell on your container. You see this because the “$” now changed to root@[CONTAINER_ID]:. From here is no way back. If you want to get back to the docker terminal, you have to start another fresh docker console like he is doing in minute 1:10. Now with a bash shell running in the container you can do whatever you want and execute Jupiter or tensorflow or whatever. The folder of the host system, you gave in the run command, should be available now under “/media/disk”.

   ---

3. Last step accessing the VM output. It still did not want to work out for me and I could not access my notebook. You still have to find the correct IP and Port to access the launched notebook, tensorboard session or whatever. First find out the main IP by using docker-machine –ls . In this list you get the URL. (If it is your only container it is called default.) You can leave away the port given here. Then from docker ps you get the list of forwarded ports. When there is written 0.0.0.32776->6006/tcp in the list, you can access it from the hostmachine by using the port given in the first place (Awkyard). So in my case the executed tensorboard in the container said “launched on port 6006”. Then from my hostmachine I needed to enter http://192.168.99.100:32776/ to access it.

-> And that's it! It ran for me like this!

Answer 8

It gives you the terminal prompt:

FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd vdocker') DO %i
docker run -it tensorflow/tensorflow:r0.9-devel

or

FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd vdocker') DO %i
docker run -it b.gcr.io/tensorflow/tensorflow:latest-devel

You should have 'vdocker' or change vdocker to 'default'.

Answer 9

For some reason I ran into one additional problem that I needed to overcome beyond the examples provided, using the --ip flag:

nvidia-docker run --rm \
  -p 8888:8888 -p 6006:6006 \
  -v `pwd`:/root \
  -it tensorflow/tensorflow:latest-devel-gpu-py3 sh -c "jupyter notebook --ip 0.0.0.0 ."

And then I can access via http://localhost:8888 from my machine. In some ways this makes sense; within the container you bind to 0.0.0.0 which represents all available addresses. But whether I need to do this seems to vary (eg I've started notebooks using jupyter/scipy-notebook without having to do this).

In any case, the above command works for me, might be of use to others.

Answer 10

As an alternative to the official TensorFlow image, you can also use the ML Workspace Docker image. The ML Workspace is an open-source web IDE that combines Jupyter, VS Code, TensorFlow, and many other tools & libraries into one convenient Docker image. Deploying a single workspace instance is as simple as:

docker run -p 8080:8080 mltooling/ml-workspace:latest

All tools are accessible from the same port and integrated into the Jupyter UI. You can find the documentation here .

Answer 11

A more elegant approach without port mapping

Many answers above are really helpful and works, but in case that you just want to use docker on your host machine to build development environment (you are using jupyter afterall...), solutions above are more or less painful than you expected.

Some painful facts

1. If you are using port mapping: docker run -p 8888:8888 -p 6006:6006 ... , you will have to mannually map port everytime in advance. If you forgot to expose a port, you have to abandom current container and re-create a new container with updated -p xxxx:xxxx .
   
   Moverover, I am kind of guy who will open multiple projects with jupyter lab (in different containers) at the sametime. Doing port mapping mannual is really discouraging.

2. When you run jupyter lab (a better alternative to jupyter notebook), you will probably have to type jupyter lab --ip=0.0.0.0 --port=8888 in container (if you run container with bash) and eventually find out you cannot change jupyter lab theme to dark mode because of some permission issues. (light mode will make me blind XD)

Better Solution

TL;DR

Use --network="host" in your docker run

If you use the host network mode for a container, that container's network stack is not isolated from the Docker host (the container shares the host's networking namespace), and the container does not get its own IP-address allocated. For instance, if you run a container which binds to port 80 and you use host networking, the container's application is available on port 80 on the host's IP address.

ref: https://docs.docker.com/network/host/

(drop any space and comments after all '\\' before run it)

docker run -it -u $(id -u):$(id -g) \  # use same user as your host instead of using root user
               --rm \
               --network="host" \ # with this line, you don't have to map ports manually anymore
               tensorflow/tensorflow:latest-gpu-py3-jupyter \
               bash

when you are in container, you can launch jupyter labs without specifying ip/ports:

jupyter lab

Example of building a ML enviroment for a project

In this section, I share a detailed example, which is beyond what this question asked, but I think you will probably find it helpful.

Files :

folder structure:

.
├── Dockerfile
├── requirement.txt
└── run.sh

Dockerfile:

FROM tensorflow/tensorflow:latest-gpu-py3-jupyter AS pre

WORKDIR /repo
RUN apt-get -qqy update && apt-get install -qqy \
    curl \
    npm \
    nodejs

FROM pre AS dev
COPY ./* /repo/
RUN mkdir /.jupyter  # make a directory for jupyter lab settings
RUN pip install --upgrade pip && pip install -r requirement.txt
RUN jupyter labextension install @jupyterlab/toc  # install table of content for juptyer lab
RUN chmod -R 777 /.local /.jupyter  # authenticate these 2 folders for current users

requirement.txt

pandas
scipy
scikit-learn
matplotlib
seaborn
kaggle
jupyterlab

run.sh:

docker run -it -u $(id -u):$(id -g) \
               --rm \
               --network="host" \
               -v /$(pwd):/repo \
               -w /repo \
               --name $(basename $("pwd"))\
               $(basename $("pwd")):latest \
               bash

step:

Build docker image and tag it with name of current folder:

docker build -t $(basename $("pwd")) .

run docker container:

bash run.sh

run jupyter lab in container:

jupyter lab

Benefits of above example

1. Now you don't need to do port mapping in advance anymore. You can open jupyter, tensorboard as you wish anytime by just clicking the link.
2. You are using same user as your host machine in the container instead of root user
3. Your current project is mounted on the container.
4. Jupyter Lab have been given permissions for any configurations
5. All in all, this docker container is just as easy to use as a virtual environment