Hello. I am working through the Zingg deployment on Databricks tutorial and it appears that the Databricks UI has changed in terms of where you can modify jars, wheels and runtimes to include the zingg.jar, wheels, specific version of spark and scala. Can anyone provide me with the new Databricks UI navigation path to get to a screen that allows this level of configuration in Databricks? BTW: I am using a Databricks trial account for this tutorial. Here's the Zingg tutorial I am attempting to follow:https://www.zingg.ai/documentation-article/identity-resolution-on-databricks-for-customer-360
A good way to add jars to Databricks now is to first upload them to a Volume... then from the cluster configuration page, you can click the Libraries tab, then the Install new button, then the Volumes Library Source , then navigate to the jar location in the volume
Hi Luke. Thank you. Can you send me a snap of the cluster configuration page? I am not seeing that option in my DBX trial UI.
start with the "compute" page on the left, instead of the "SQL warehouse" , which is just for SQL workloads... zingg will require python... the "compute" page allows for creating more general compute clusters
from there, are you able to see the "all-purpose compute" tab?
Negative. There is not a all-purpose compute tab.
I have identified the cause of the problem. The 'All-purpose compute' tab is only available in paid DBX accounts.
I validated this against a commercial non-trial DBX account.
yea, sorry, meant to come back and tell you the same... the trial accounts are what we call "serverless", so you cannot spin up classic compute. this is a problem for zingg, b/c zingg installs as a jar, and you are not able to install custom jars on serverless compute
Yeah, the Zingg documentation is basically not usable for deployment on Databricks trial. Even with a paid DBX account its nearly impossible to follow the required steps in the DBX UI. Is anyone going to update the Zingg documentation? It would seem that DBX is a pretty important platform to be current with. Any pointers on how to translate to the new DBX UI? Even the above mention of a 'cluster configuration page' I see nothing with that label in the trial or paid version of Databricks. Is it just me?