Metastore

In the Ultra Transcenders books

DP-750

Each book explains Metastore in context, with comparison tables and the common traps.

Terms in this definition

• Unity Catalog

  Azure Databricks' governance solution covering both data and AI in one place, with centralised permissions, auditing, data discovery and lineage.

• Container

  Something that groups data. Blob Storage containers sit inside a storage account and hold blobs much as folders hold files; Cosmos DB containers hold items and set the scope for partitioning and throughput.

• Metadata

  Describes data rather than being the data itself, for instance how a file is laid out or what rows each chunk contains, so tools can read things efficiently.

• AGDLP

  Nesting pattern: users go into global groups, which go into domain local groups, which receive the permissions. AGUDLP adds universal groups for forests with several domains.

• region

  A provider-defined grouping in VCF Automation of Supervisors that all share one NSX Local Manager; tenants consume its compute, storage and memory via quotas set per region.

• Workspace

  Teams in Power BI and Microsoft Fabric collaborate in this folder-style container, which groups items such as reports, semantic models and lakehouses, controls who can access them and is assigned a capacity.

Related terms

• ABAC policy

  Governed tags drive this kind of Unity Catalog rule: when a table or column has tags that satisfy its WHEN and MATCH COLUMNS conditions, a row filter or column mask UDF is put on it automatically. Policies are written with CREATE POLICY and can sit on a metastore, catalog, schema or table.

• Allowlist

  Held at metastore level, it records which JAR files, Maven libraries and init scripts may run on compute using the standard access mode. It begins with no entries, and only holders of MANAGE ALLOWLIST may change it.

• Azure Databricks workspace

  A deployed Azure Databricks environment where a group of people build and run notebooks, jobs and compute. Accounts often hold several, each attached to a Unity Catalog metastore within its own Azure region.

• DENY policy

  A Beta ABAC policy type that takes a privilege away rather than granting one; at the moment it can only block MANAGE ACCESS CONTROL. It applies wherever governed tags match and beats every grant, ownership too, though it never restricts metastore admins.

• External data access

  A metastore option, disabled unless an admin switches it on, that opens Unity Catalog data to reads and writes from other engines via REST APIs (Unity's own or the Iceberg catalog). Each principal also needs the external use privilege on the schema.

• Managed storage location

  A cloud storage path where Unity Catalog places managed tables and volumes, set with MANAGED LOCATION on a metastore, catalog or schema. The most specific level takes precedence, and Learn advises setting it per catalog to keep data isolated.

• Managed table

  Recommended by default, this table type keeps its Delta or Apache Iceberg files in storage Unity Catalog controls, namely the managed location of its schema, catalog or metastore. After a DROP, the files are deleted once the recovery period has passed.

• OpenSharing

  An open protocol and platform from Azure Databricks for sharing data and AI assets securely, whether with another metastore (Databricks-to-Databricks) or with recipients who don't use Databricks (open sharing).