Jet Analytics Data Sources General Connection Settings

This article provides an overview of the connection settings that apply to the following Jet Analytics data sources:

General Connection Settings

These settings are consistent across all location types.

Location

The following location types are supported:

Local file or folder
Azure Blob Storage or Azure Data Lake Storage (ADLS)
AWS S3 Bucket
SharePoint or OneDrive
Google Cloud Storage
SFTP

Path

Specify the path to the file or files in the selected storage location. Each data connector supports only one path. For local files, this is the full file path. For files in Azure Blob Storage or ADLS, enter the file path as filename.filetype (e.g. PostedHourLine.parquet).

Include sub-folders

Specifies whether files within sub-directories of the configured path should be included. Enable this setting when extracting files in Azure Blob Storage or ADLS, even when the file is located in the main blob container.

Included file types

A comma-separated list of file extensions to include. Files with extensions not listed here will be ignored.

File aggregation pattern

A comma-separated list of file patterns to aggregate as a single table, using the wildcards * and ?. Files matching a pattern are treated as one table.

Note:

Files grouped by an aggregation pattern must share the same schema. A schema mismatch will cause the process to break during a full data load.

Metadata URI

(Optional) Specify a separate file to use for metadata parsing instead of the main data files. When set, the Metadata URI overrides the Path setting during synchronization but is ignored during full data loads.

This is useful when you have several files with the same structure and want to use a specific file to define the data types applied when ingesting the files specified in the Path property. It can reduce execution time for the import metadata task by limiting the rows scanned for data type inference to just the rows in the metadata file. The metadata file can be one of the files in the path, or a separate file with sample data.

Note:

Only one metadata file can be specified.

Location-Specific Connection Settings

Each storage location type has unique connection settings as described below.

Local file or folder

Use when the file is stored locally on a drive. To connect to an Azure File Share, use the following UNC path format:

\\<storageAccountName>.file.core.windows.net\<fileShareName>\<folder>\<filename>.<filetype>

Before connecting to an Azure File Share, the file share must be mounted as a network drive. To do this, locate the file share, click Connect, then Show Script, and copy the PowerShell script using the Copy code button.

[Insert image (General_Azure_File_Share_Connect_Button) here]

Run this script in PowerShell as the same user account that runs the Ingest Instance Service. This cannot be done when running as Local System. To launch PowerShell as a different user, search for PowerShell, right-click it, choose Open file location, then right-click the application and select Run as a different user.

Paste the copied script into PowerShell and press Enter. A message will confirm that the folder has been created.

Azure Blob Storage or ADLS settings

Connect using either a connection string or a service principal. When both are provided, the connection string takes precedence.

Azure Blob Connection String: The full connection string for the Azure Blob container. This can be found under Access keys in the Azure portal.
Azure Tenant ID: The tenant ID of the Azure service principal.
Azure Client ID: The client ID of the service principal.
Azure Client Secret: The client secret for the service principal.
Azure Storage Account Name: The name of the Azure storage account.
Azure Blob Container: The name of the blob container in Azure.

AWS S3 Bucket settings

This connector supports Access Key ID-based authentication. See the AWS documentation for more information.

AWS Region: The region of the S3 bucket (e.g. eu-west-2).
AWS Access Key ID: The access key ID for the S3 bucket.
AWS Secret Access Key: The secret access key for the S3 bucket.
AWS Bucket: The name of the S3 bucket.

SharePoint or OneDrive settings

These storage types are accessed using the Microsoft Graph API. Only OneDrive for Business accounts are supported — personal OneDrive accounts are not compatible.

SharePoint/OneDrive Client ID: The client ID of the service principal.
SharePoint/OneDrive Client Secret: The client secret of the service principal.
SharePoint/OneDrive Tenant ID: The tenant ID of the service principal.
SharePoint/OneDrive Drive ID: The drive ID of the file location. See Locate a SharePoint/OneDrive drive ID.

Google Cloud Storage settings

Supports GCM authentication via service account keys. See the Google Cloud IAM documentation for more information.

Google Credential File: Path to the Service Account private key file (JSON or P12 format).
Google Storage Bucket Name: The name of the Google Cloud storage bucket.

SFTP settings

Supports authentication via password or public key file. If both are provided, the public key file takes precedence.

SFTP Host: The hostname or IP address of the SFTP server.
SFTP Port: The port of the SFTP server.
SFTP Username: The username for SFTP authentication.
SFTP Password: The password associated with the SFTP username.
SFTP Key Path: The path to the public key file for the SFTP user.