Help needed with starting customer entity resolution PoC using Zingg on Databricks | Zingg Community

Zingg Community Icon

Zingg Community

Help needed with starting customer entity resolution PoC using Zingg on Databricks | Zingg Community

Luke B.
·
Hey Ryan, try this notebook... and reach out to me if I can help lucas.bilbro@databricks.com
👍1
Sonal G.
·
In addition, here is the Zingg example notebook. https://github.com/zinggAI/zingg/blob/main/examples/databricks/FebrlExample.ipynb
Ryan J.
·
Thank you both. Luke, I’ll take a look at that notebook this morning. Sonal, It’s likely my unfamiliarity with zingg (and likely a little spark as an overall framework), however I have looked at that notebook and still have questions. It feels what I’m missing is basic documentation around zingg’s functional concepts. Examples of questions I currently have:
Are pipes required to get data in and out of zingg? If so, it seems like interactions within databricks may be best using the InMemoryPipe utilizing spark data frames?
How do I ensure the zing python bindings are able to utilize the java core?
Is there a simple method to test the binding?
Ryan J.
·
Just popping back in momentarily to mention Luke’s notebook appears fairly detailed and I have high confidence that will answer some of my most basic questions. I’ll be spending some time with it today to validate and hopefully come back with some better questions (or a success story).
Ryan J.
·
I’m working with my Databricks SA on resolving Java class issues. Hosting the zingg jar in a volume, and have “installed” the library via the UI. Unfortunately the JVM can’t find the proper class. (Based on conversation this sounds like a common java issue in DBX) Troubleshooting so far:
The jar file is not currently found on the worker node local storage.
The java class path does not include the path where we’ve “installed” the jar.
Sonal G.
·
This is strange. Zingg jar as cluster library is what we tested with.
Luke B.
·
which DBR and zingg version are you using Ryan J.
Ryan J.
·
The issue did turn out to be spark compatibility. I was originally attempting to utilize the old notebooks, so tried to keep the versions consistent. I was using 0.3.4 earlier. Moving to 0.4.0 did resolve the class execution issues. I’m using the 15.4 DBR on a private compute cluster.
👍1