I executed match phase (seems like intermediate results cannot be stored on UC volumes and hence, needed to copy the trained model into a local path):
options = ClientOptions([ClientOptions.PHASE, "match"])
# Run match against the DBFS-staged model (see DBFS workaround section above)
with zingg_dbfs_context(args):
zingg = ZinggWithSpark(args, options)
zingg.initAndExecute()
And after that, this code which goes into except block:
try:
outputDF = spark.read.csv(ZINGG_OUTPUT_PATH)
colNames = ["z_minScore", "z_maxScore", "z_cluster", "rec_id", "fname", "lname", "stNo", "add1", "add2", "city", "state", "dob", "ssn"]
outputDF.toDF(*colNames).show(100)
except Exception as e:
if "PATH_NOT_FOUND" in str(e):
print(f"No match output found at: {ZINGG_OUTPUT_PATH}")
else:
raise
Not sure, what am I missing?
