Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Azure Databricks previews.
This page contains reference documentation for the GitHub connector in Lakeflow Connect.
Connection properties
When you create the Unity Catalog connection, you must specify the following properties for OAuth U2M authentication. These values come from the GitHub OAuth App you configured in Configure OAuth U2M for GitHub ingestion.
| Property | Required | Description |
|---|---|---|
| Client ID | Yes | The client ID from your GitHub OAuth App. |
| Client secret | Yes | The client secret from your GitHub OAuth App. This value is displayed only once when you generate it. |
| Enterprise domain | No | Your GitHub Enterprise Server (GHES) hostname, if applicable (for example, github.mycompany.com). Leave empty for github.com and GitHub Enterprise Cloud (GHEC). If not set, defaults to api.github.com. |
Connector options
Set the following options when you configure the ingestion pipeline. For an example, see Restrict ingestion to specific repositories.
Repository selection (repository_id_selections)
By default, the connector ingests data from all repositories in the organization. To restrict ingestion to specific repositories, set repository_id_selections to a list of GitHub repository IDs. These are the numeric, immutable repository IDs, which remain stable when a repository is renamed. When the list is empty, the connector ingests all repositories in the organization.
Repository selection applies to repository-scoped source tables: repositories, repo_contents, commits, issues, issue_comments, pull_request_review_comments, tags, workflows, branches, collaborators, deployments, discussions, labels, milestones, pull_requests, and releases. It has no effect on organization-scoped and nested tables. For the list of tables that ignore this option, see Repository selection.
Supported source tables
For information about which tables support incremental updates and which are fully refreshed on each pipeline run, see Supported data.
Repository contents (repo_contents table)
The repo_contents table ingests all entries in each repository's default branch, including files, directories, submodules, and symbolic links. The primary key is (repository_id, branch_name, path). The type column identifies each entry as a blob (file), tree (directory), or commit (submodule). Symbolic links are blob entries with a mode of 120000. Only file (blob) entries populate the content column, which holds the decoded text content. When a file is deleted from the source repository, its row is removed from the table (hard delete). For limitations, see Repository contents.