访问 AWS ec2 本地集群上的仪表板

Question

I am running a Dask script on a EC2 instance from AWS. I would like to connect and see the dashboard provided by Dask, but I can't figure out how.

I am creating a Local Cluster on my EC2 instance, the scripts runs fine and I am connecting to my AWS instance via Putty. However, I would like to see the available dashboard: on my PC it is enough to connect to the provided IP and port, but I am not able to do that on the AWS machine.

Once the script is running, this is my output for the "parameters" of the local cluster:

<Client: 'inproc://172.31.29.4/7475/1' processes=1 threads=8, memory=27.94 GiB>
LocalCluster(b8be08dd, 'inproc://172.31.29.4/7475/1', workers=1, threads=8, memory=27.94 GiB)
dashboard address: {'dashboard': 8787}

For example, I tried to write 172.32.29.4:8787/status on my browser, but I wasn't able to connect to the dashboard.

I already checked this question: How to view Dask dashboard when running on a virtual machine? However I am using a Local Cluster and I would like to connect to its dashboard from remote. Is it possible? If so, how?

Answer 1

The answer is in the comments, but I will type it out here, so that the original question looks "answered".

You need two things to connect to a port on an EC2 machine: the external IP, and access. The former is most easily found from the AWS console. For the latter, you typically need to edit the security group to add an inbound TCP rule for the port (either open to the world, or just your IP). There are other ways to do this part, depending on whether your machine is inside a VPC, has any custom gateways or routers... but if you don't know what that means, find the security group first. Both the public IP and the security group will be linked from the machine's row in the EC2 "running instances" list.

Answer 2

I've setup dask-labextension visualizations to provide this type of UI.

Create the client object:

from dask.distributed import Client
client = Client()

Then click the magnifying glass provided by the extension to automatically connect with the cluster.

The detailed instructions are in this post .