Skip to main content

Use Fabric Lakehouse for Prepare instance storage (Public Preview)

Note: This article is intended for database administrators. Ensure the following tasks are completed by someone with the appropriate database administration privileges.

This article describes how to set up Prepare instances with Fabric Lakehouse storage.

Fabric Lakehouse Prepare instance storage is available as part of the standard, premium, or enterprise package.

For more information on standard, premium, or enterprise package, contact Product Support.

A public preview of this feature is currently available that supports the following functionality:

  • Data extraction from Ingest instances using Fabric Lakehouse storage.
  • All standard functionality except related records and hierarchy tables.
  • If you use the data in a Delivery instance, PowerBI endpoints can be used.

The following features are currently not supported in Fabric Lakehouse:

When a Prepare instance uses Fabric Lakehouse storage, the supported features work the same as on any other storage type, with the following exception:

  • nchar columns always have the predefined length, with spaces added to the value as padding. For example, a CustomerName column with the datatype nchar(20) would store Jet Analytics as "Jet Analytics. When you create selection rules on nchar columns, remember to include the trailing spaces in the rule.

Prerequisites

  • Your Ingest Instance must also use Fabric Lakehouse storage. Currently, Fabric Prepare Instances can only use data from Ingest instances with Fabric storage. Using Fabric Prepare instances in combination with non-Fabric Ingest Instances is not supported.
  • Create an App Registration in the Azure Portal. It is recommended to use a dedicated app registration to ensure this account is the only one with access to the client credentials.
  • In the Fabric/Power BI Admin Portal, enable the following settings and ensure these settings are enabled for the app registration if using specific security groups:
    • Users can create Fabric items

    • Service principals can create workspaces, connections and deployment pipelines
    • Service principals can call Fabric public APIs

Note:

If using workspace settings to override tenant admin settings for the Users can create Fabric items setting, ensure this setting is also enabled at the workspace level and that it is enabled for the app registration if using specific security groups.

  • Create a workspace, or navigate to an existing workspace, in the Fabric portal and select Manage access. Grant the App Registration account Member access to the Fabric workspace.

Add Prepare Instance with Fabric Lakehouse Storage

Note:

You can connect to an existing Lakehouse that has been created directly in the Fabric Portal, or you can choose to create the Lakehouse within Jet Analytics Data Integration.

  1. Add a Prepare Instance and select the storage type Microsoft Fabric Storage.

    For information on adding a Prepare instance, contact Product Support.

  2. Enter the workspace name for the existing Fabric workspace.
  3. Provide a name for the Lakehouse.

    Make sure to use different Lakehouses for your Ingest and Prepare instances to avoid table name clashes.

  4. Enter the Tenant ID for the tenant associated with Fabric.
  5. Enter the Application ID for the App Registration.
  6. Enter the Application Key (i.e. the client secret value) associated with the App Registration.
  7. Open Jet Analytics Data Integration and open the instance you just created.
  8. If you have not yet created the Lakehouse, you can do so now by right-clicking the instance in the Solution Explorer and clicking Edit Instance followed by Create Storage. Keep the Use Lakehouse schemas property checked — this is recommended as it allows for different schemas by organizing tables into sub-folders in the Lakehouse, rather than creating all tables in the dbo schema, which risks overwriting tables when multiple data areas contain tables with the same name. Click OK.

Objects deployed within Fabric Lakehouse

Upon deployment of a table in a Fabric Prepare instance, a Spark-based Fabric Notebook is created in the workspace and named using the following format: Jet Analytics_<Lakehouse Name>_<TABLE/VIEW>_<Data Area>_<Table Name>. A notebook is created for each table that is deployed. To view the notebooks, navigate to your Fabric Lakehouse, click Open notebook, and select Existing notebook. Search for the notebook and click the relevant result.

When a table is executed within a Fabric Prepare instance, the notebooks created during deployment are run. The execution of the notebook results in the creation of a Lakehouse table, populated with data from the Ingest instance Fabric storage.

Note:

Unlike other storage types, tables are not created in Fabric Lakehouse until the table is executed. In addition, empty tables are not created.

If a table is added to the data area without any transformations, selection rules, incremental rules, or other modifications, a shortcut to the table in the Ingest instance Fabric Lakehouse is created in the Prepare instance Fabric Lakehouse, rather than a delta parquet table.

Views and stored procedures are deployed to the SQL Analytics endpoint for the Lakehouse. To review the views and stored procedures deployed behind the scenes, navigate to your Fabric Lakehouse, click Settings, and select SQL analytics endpoint. Copy the connection string and paste it into SSMS, then connect using the Microsoft Entra MFA authentication option.

Was this article helpful?

We're sorry to hear that.