Dask Dashboard with Local Clusters

How to access the Dask Dashboard for local clusters

Local Clusters

Dask clusters on Saturn Cloud enable you to scale up to many worker nodes. It is also possible to run Dask on a single machine by passing in a LocalCluster, or leaving the cluster argument blank when initializing a Client:

from dask.distributed import Client
client = Client()

Dask then splits up work across the cores of the single machine. In Saturn Cloud, a local cluster can run in your Jupyter Server. You can also use Dask to take advantage of a machine with multiple GPUs by using a LocalCUDACluster from dask-cuda:

from dask_cuda import LocalCUDACluster
from dask.distributed import Client

cluster = LocalCUDACluster()
client = Client(cluster)

Accessing Dask Dashboard

Previewing the client object displays information about the Dask client along with a link to the Dask Dashboard:

client

Client

  • Scheduler: tcp://127.0.0.1:44603
  • Dashboard: http://127.0.0.1:8787/status

Cluster

  • Workers: 2
  • Cores: 2
  • Memory: 15.50 GB

The “Dashboard” URL links to the localhost, but since this is running inside of Saturn Cloud, that link will not work. Saturn utilizes a Jupyter proxy to be able to access interfaces hosted on the server. You can copy the URL of the Jupyter window from your browser and replace /lab/* with /proxy/8787/status. For example, your Jupyter URL might be:

https://j-aaron-proj.community.saturnenterprise.io/user/aaron/examples-cpu/lab/workspaces/examples-cpu

Then your dashboard URL would be:

https://j-aaron-proj.community.saturnenterprise.io/user/aaron/examples-cpu/proxy/8787/status

If you find yourself doing this often, or if you spin up multiple local clusters in one Jupyter Server, you can utilize the following function to display a clickable link to the dashboard:

import os
from IPython.display import display, HTML

def local_dashboard_link():
    link = client.dashboard_link.replace(
        'http://127.0.0.1:', 
        "https://{}/user/{}/{}/proxy/".format(
            os.environ['SATURN_JUPYTER_BASE_DOMAIN'],
            os.environ['SATURN_USERNAME'],
            os.environ['SATURN_PROJECT_NAME'],
        )
    )

    display(HTML(
        f'<a href="{link}" target="_blank" rel="noopener">{link}</a>'
    ))

local_dashboard_link()