Identity Provider Integration
Last updated
Was this helpful?
Last updated
Was this helpful?
The Data Streamhouse supports integration with external identity providers using OpenID Connect or LDAP. This enables secure, centralized authentication and authorization aligned with your organization's existing identity and access management systems.
This guide covers how to configure Data Streamhouse to authenticate users via OpenID or LDAP and manage user groups automatically.
Before configuring identity provider integration (OpenID Connect or LDAP), the following conditions must be met:
You must activate your Data Streamhouse licenses before enabling identity provider authentication.
Why this matters: Without a valid license, Portal will not allow external authentication, and login will fail. Always ensure that the licenses are fully activated before proceeding with identity provider setup.
Refer to the section for details on activating your license.
Before enabling integration with an external identity provider (OpenID or LDAP), you must first create the required groups and roles inside the Data Streamhouse Portal.
Why this matters: Once identity provider integration is active, Portal will attempt to map authenticated users to internal groups based on external group claims or attributes. If no matching groups exist, users will not be assigned any roles and will be denied access.
You can create groups in two ways:
Portal UI: Navigate to Administration → Groups and manually create all required groups, setting the External Mapping field to match external group names or IDs (if required).
YAML Files: Define groups and roles as YAML configuration files and import them during deployment. (This method is recommended for production environments and is covered in the Advanced Setup section.)
Data Streamhouse can authenticate users via OpenID Connect compliant providers, such as Azure AD, Okta, or Entra ID.
To activate OpenID authentication:
Set the authentication method:
Configure the required environment variables prefixed with dsh_portal_authentication_openid_:
client_id
Client ID provided by your OpenID provider
client_secret
Client secret provided by your OpenID provider
redirect_url
Redirect URL after authentication (e.g., https://dsh.acme.org/api/auth/oauth/callback)
jwks_url
JWKS URL for verifying tokens
authorization_url
URL for initiating the OpenID flow
token_url
URL for exchanging codes for tokens.
The redirect_url must always end with the fixed path /api/auth/oauth/callback. You can configure the domain freely (e.g., https://portal.company.com), but the path itself is mandatory and cannot be changed. Identity providers must redirect users exactly to /api/auth/oauth/callback after authentication.
Ensure the container can access the specified URLs.
groups_claim
Claim name containing user groups
groups
scope
Requested OpenID scopes
openid profile email offline_access
debug
Enable debug logging for OpenID
false
group_management
Enable automatic group mapping
false
login_url
Login page URL
/#/login
sync_interval_mins
Interval for synchronizing group information
60 (minutes)
Database Preparation: Ensure an external database is configured. Portal requires persistence for user and group mappings.
Provider Registration: Register Data Streamhouse as a client application in your OpenID provider.
Environment Configuration:
Configure environment variables with your client ID, secret, and URLs.
Ensure the redirect_url matches the registered redirect URI.
Group Mapping (Optional):
Map OpenID groups to internal Data Streamhouse groups.
Use the "External Mapping" field inside the Portal administration interface.
Activation: Set the authentication method to OpenID and restart the Portal.
Login: Authenticate via your OpenID provider.
Note: To enable support for proxies terminating TLS, set:
This avoids CSRF errors when using HTTPS offloading.
To troubleshoot OpenID configuration issues, set the system-wide log level to DEBUG and enable the OpenID debug parameter by setting it to true.
Here's an example of a successful authentication log output:
Data Streamhouse also supports LDAP and LDAPS for authentication with Active Directory (AD) or OpenLDAP directories.
To activate LDAP authentication:
Set the authentication method:
Configure the required environment variables prefixed with dsh_portal_authentication_ldap_:
url
LDAP server URL (ldap:// or ldaps://)
user
Service user DN
password
Password for the service user
base
Base DN for user searches
filter
LDAP filter for retrieving users
user_id_attribute
Attribute used for login username
cn
user_fullname_attribute
Attribute used for full display name
user_id_attribute
group_attribute
Attribute containing groups (typically memberOf)
memberOf
group_regex
Regex to extract group names from DNs
(?i)cn=(\w+),ou=Groups.*
group_management
Enable automatic group assignment
false
sync_interval_mins
Interval for synchronizing user and group information
60 (minutes)
debug
Enable LDAP debug logging
false
Service Account: Configure a dedicated service account in LDAP with permissions to read user and group information.
Group Management (Optional): Enable automatic group mapping by setting:
Ensure the group names match or are mapped in the Portal administration section.
Synchronization: User and group data will synchronize at startup and at the configured interval.
Nested Groups (Advanced): If users are members of nested groups, activate the nested LDAP module:
Configure:
dsh_portal_authentication_ldap_group_search_filter
dsh_portal_authentication_ldap_group_search_user_filter
Use the extended matching rule (1.2.840.113556.1.4.1941) for Active Directory recursive search, e.g.: (&(objectClass=person)(memberOf:1.2.840.113556.1.4.1941:=%s))
Basic LDAP configuration snippet:
Both OpenID and LDAP integrations allow mapping external groups to internal Data Streamhouse groups.
To configure:
Navigate to Administration → Groups.
Edit or create a group.
Set the External Mapping field to match the external group name or ID.
Recommendation: Always map a group for administrative users (e.g., Superusers) before activating identity provider integration to ensure continuous access.
After configuration:
Restart the Portal.
Attempt login via the configured identity provider.
Verify successful authentication and correct group assignment.
If issues occur:
Review Portal logs for detailed authentication errors.
Ensure that all OpenID URLs (redirect, JWKS, token, authorization) are reachable.
Check LDAP filters and user attributes.
Verify client ID, client secret, and scopes for OpenID.
For SSL/TLS issues, verify certificates and proxy trust settings.
If your LDAP environment uses referrals (e.g., multiple LDAP servers for failover), configure:
Supported values depend on your LDAP server configuration.
By default, Portal searches recursively within the LDAP directory (subtree search scope). To adjust, set:
Supported scopes typically include subtree, onelevel, and base.
Instead of manually creating groups and assigning roles through the Portal user interface, Data Streamhouse also supports YAML-based configuration for groups and roles.
This approach allows you to:
Define all groups, roles, and permissions as code.
Version-control your access management.
Automate group and role provisioning during deployment.
Note: YAML-based group and role management is covered in the Automated Deployment section.
Using YAML definitions is highly recommended for environments requiring strict change control, automated deployments, or integration with GitOps workflows.
Authentication Source
OpenID Connect Provider (e.g., Okta, Azure AD)
LDAP Directory (e.g., Active Directory, OpenLDAP)
Group Management
Via token claims (e.g., groups claim)
Via LDAP memberOf attributes
Synchronization
Token-based on login + periodic sync
Database synchronization at startup + interval
Nested Groups Support
Depends on provider
Supported via LDAP-nested module
Debug Mode
Available (dsh_portal_authentication_openid_debug)
Available (dsh_portal_authentication_ldap_debug)
Typical Use Cases
Cloud-native identity providers
Enterprise directories, on-prem environments
Required External Access
Yes (to OpenID endpoints)
Yes (to LDAP server)
Group Mapping
Via External Mapping field in UI or YAML
Via External Mapping field in UI or YAML