Skip to main content

How to Set up Hive

Updated over a week ago

Setup takes about 5 minutes.

Introduction

Use this connector to allow Foundational to scan Hive metadata and determine lineage and downstream impact.

To make the connection, you need to:

  • Create a database user and grant permissions on the Hive Metastore database

  • Connect Foundational to Hive using the metastore database connection details


Prerequisites

Ensure you have:

  • Hive administrator access

  • Access to the PostgreSQL database used by the Hive Metastore

  • A PostgreSQL client (for example, psql)

  • Added Foundational IP addresses to the allowlist, see the article Allowing IP access to Foundational.


Access permissions

Foundational accesses metadata only and does not read or query table data.


The permissions you assign to Foundational enable it to:

  • Read Hive Metastore metadata from the underlying PostgreSQL database

  • List databases, tables, and columns

  • Determine lineage and downstream impact


Create user and grant permissions to the Metastore database

You can access the PostgreSQL database through your preferred PostgreSQL client. The commands below use psql.

  1. Connect to your PostgreSQL database. By default, PostgreSQL runs on port 5432 on the Hive host.

    psql -h <host> -p 5432 -U <admin-user>

  2. Switch to the Hive Metastore database.

    metastore is the default database name. If your deployment uses a different name, use that name instead.

    \\connect metastore

  3. Create a user and role, and grant the required permissions.

    Replace <password> with a password of your choice.

    -- Create user and role
    CREATE USER foundational WITH PASSWORD '<password>';
    CREATE ROLE "Foundational-role";
    GRANT "Foundational-role" TO foundational;

    -- Grant database and schema access
    GRANT CONNECT ON DATABASE metastore TO "Foundational-role";
    GRANT USAGE ON SCHEMA public TO "Foundational-role";

    -- Grant SELECT on tables
    GRANT SELECT ON "COLUMNS_V2", "SDS", "TBLS", "DBS" TO "Foundational-role";

    -- Grant SELECT on specific columns
    GRANT SELECT ("COLUMN_NAME", "TYPE_NAME", "INTEGER_IDX", "CD_ID")
    ON "COLUMNS_V2"
    TO "Foundational-role";

    GRANT SELECT ("INPUT_FORMAT", "LOCATION", "SD_ID")
    ON "SDS"
    TO "Foundational-role";

    GRANT SELECT ("TBL_NAME", "TBL_TYPE", "SD_ID", "DB_ID")
    ON "TBLS"
    TO "Foundational-role";

    GRANT SELECT ("NAME", "DESC", "CTLG_NAME", "DB_ID")
    ON "DBS"
    TO "Foundational-role";

  4. Verify that the role and user exist. Run the following queries and confirm that they return results, the role name and the user name.

    SELECT rolname FROM pg_roles WHERE rolname = 'Foundational-role';
    SELECT usename FROM pg_user WHERE usename = 'foundational';

Connect Foundational to Hive

  1. In Foundational, open the Connectors & Integrations page.

  2. In Data Stores, select the Hive connector card and click Connect.

  3. Click Add Database.

  4. The initial setup screen opens. Click Start Setup.

  5. The Set Up New Connection screen opens.

    Enter the following details:

    • Hive Metastore DB Host: Host address of the Hive Metastore PostgreSQL database.

    • Hive Metastore DB Port: Port number of the Hive Metastore database.

    • Hive Metastore DB Name: Name of the Hive Metastore database. The default is metastore.

    • User Name: PostgreSQL username. If you used the script above, use foundational

    • Password: Password for the PostgreSQL user.

  6. Click Next.

  7. The final setup screen opens. To complete the connection, click Save.​

That’s it. Foundational is now connected to Hive.

Did this answer your question?