Use this connector to have Foundational extract information from Unity catalog, widget configurations of notebooks and usage metrics.
This setup should take 5 minutes.
Step #1: Generate a Personal Access Token
Make sure that the user you are creating the personal access token for, has access to the relevant workspaces. You can do this by following this Databricks guide.
Follow the steps in Databricks guide to generate a personal access token
Note: Pleae make sure to not set a lifetime limit on the token, as if the token expires - Foundational will not be able to ingest the information from Databricks.
Save the generated token. We will use it later when we configure the Databricks connector in Foundational.
Step #2: Grant 'BROWSE' Privilege
Follow this Databricks Guide to grant the 'BROWSE' privilege to the user for whom you created the personal access token.
Step #3: Grant Permissions to Query History (and other tables)
Grant the user, for whom you created the personal access token, permissions for the query history tables:
GRANT SELECT ON SYSTEM.QUERY.HISTORY TO `<user-name>`; GRANT USE SCHEMA ON SCHEMA system.query TO `<user-name>`;
Step #4: Configure Databricks in Foundational
Open the Connectors page in Foundational and select the Databricks connector
Click on "Add new Databricks project"
Fill in the following information:
Workspace URL: The URL for your workspace. You can get it by:
Opening your databricks workspace
Copying the top level URL from the browser's address bar
Personal Access Token: The personal access token you created
Warehouse ID: the id of the warehouse that Foundational should use for running its ingestion
Click on "Save"