Some DASH users will be granted additional privileges to be able to spin up clusters, those are denoted as BusAdmin users. They are there to facilitate their teams’ work as different analysis teams have different computing needs that potentially need bespoke configurations. Please remember that cluster creation is a privilege, not a right; if someone abuses it, it will be taken away.
Here is some general advice on how to manage a cluster:
Best practices on cluster configuration can be found here: Best practices: Cluster configuration - Azure Databricks. Please read them before continuing.
Best practices on spinning up personal compute clusters from Databricks can be found here: Create a Personal Compute resource - Azure Databricks, they complement the previous point.
Please select the smallest VM you can perform your work with. A rule of thumb: E-series VMs are good for R based analytics. Information about VM types can be found here: Virtual Machine series | Microsoft Azure.
You are strongly discouraged of using GPUs clusters unless you need them. They are very expensive.
DASH Support will be happy to discuss with you how to tailor your cluster configuration.
To make cluster creation simpler there is a selection of “T-shirt
size” cluster profiles to choose from. When selected these limit the
options available. For example, selecting an RStudio T-shirt size will
have the settings required for RStudio server set and not shown as
options.
If you outgrow the T-shirts available please contact DASH support, we
might be able to get you a new one with a better fit.
A separate GitHub repository contains init scripts that can be used
during DASH cluster creation.
DASH
Init scripts. General information about init scripts can be found in
What
are init scripts?. There is also an older user guide with some
useful background available here.