Identity Provider Integration
The Data Streamhouse supports integration with external identity providers using OpenID Connect or LDAP. This enables secure, centralized authentication and authorization aligned with your organization's existing identity and access management systems.
This guide covers how to configure Data Streamhouse to authenticate users via OpenID or LDAP and manage user groups automatically.
Prerequisites
Before configuring identity provider integration (OpenID Connect or LDAP), the following conditions must be met:
License Activation
You must activate your Data Streamhouse licenses before enabling identity provider authentication.
Why this matters: Without a valid license, Portal will not allow external authentication, and login will fail. Always ensure that the licenses are fully activated before proceeding with identity provider setup.
Refer to the License Activation Scenarios section for details on activating your license.
Group Preparation
Before enabling integration with an external identity provider (OpenID or LDAP), you must first create the required groups and roles inside the Data Streamhouse Portal.
Why this matters: Once identity provider integration is active, Portal will attempt to map authenticated users to internal groups based on external group claims or attributes. If no matching groups exist, users will not be assigned any roles and will be denied access.
You can create groups in two ways:
Portal UI: Navigate to Administration → Groups and manually create all required groups, setting the External Mapping field to match external group names or IDs (if required).
YAML Files: Define groups and roles as YAML configuration files and import them during deployment. (This method is recommended for production environments and is covered in the Advanced Setup section.)
OpenID Integration
Data Streamhouse can authenticate users via OpenID Connect compliant providers, such as Azure AD, Okta, or Entra ID.
Configuration Overview
To activate OpenID authentication:
Set the authentication method:
DSH_PORTAL_AUTHENTICATION_METHOD=openid
Configure the required environment variables prefixed with dsh_portal_authentication_openid_:
client_id
Client ID provided by your OpenID provider
client_secret
Client secret provided by your OpenID provider
redirect_url
Redirect URL after authentication (e.g., https://dsh.acme.org/api/auth/oauth/callback
)
jwks_url
JWKS URL for verifying tokens
authorization_url
URL for initiating the OpenID flow
token_url
URL for exchanging codes for tokens.
The redirect_url must always end with the fixed path /api/auth/oauth/callback. You can configure the domain freely (e.g., https://portal.company.com), but the path itself is mandatory and cannot be changed. Identity providers must redirect users exactly to /api/auth/oauth/callback after authentication.
Ensure the container can access the specified URLs.
Optional Parameters
groups_claim
Claim name containing user groups
groups
scope
Requested OpenID scopes
openid profile email offline_access
debug
Enable debug logging for OpenID
false
group_management
Enable automatic group mapping
false
login_url
Login page URL
/#/login
sync_interval_mins
Interval for synchronizing group information
60
(minutes)
Setup Steps
Database Preparation: Ensure an external database is configured. Portal requires persistence for user and group mappings.
Provider Registration: Register Data Streamhouse as a client application in your OpenID provider.
Environment Configuration: Configure environment variables with your client ID, secret, and URLs. Ensure the
redirect_url
matches the registered redirect URI.Group Mapping (Optional):
Map OpenID groups to internal Data Streamhouse groups.
Use the "External Mapping" field inside the Portal administration interface.
Activation: Set the authentication method to OpenID and restart the Portal.
Login: Authenticate via your OpenID provider.
Note: To enable support for proxies terminating TLS, set:
DSH_PORTAL_TRUST_PROXY_SSL=true
This avoids CSRF errors when using HTTPS offloading.
Troubleshooting
To troubleshoot OpenID configuration issues, set the system-wide log level to DEBUG and enable the OpenID debug parameter by setting it to true.
Here's an example of a successful authentication log output:
2025-04-10 14:23:36,345 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] OpenIdAuthProvider: OpenIdAuthProvider license found. Proceeding with authentication.
2025-04-10 14:23:36,345 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] Retrieving token for authorization code: 1.ANON...XYZ1AA
2025-04-10 14:23:36,345 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] sendTokenRequest with request body: grant_type=authorization_code&client_id=00000000-0000-0000-0000-000000000000&client_secret=ANON...SECRET&code=1.ANON...XYZ1AA&redirect_uri=https://dsh.acme.org/api/auth/oauth/callback
2025-04-10 14:23:37,063 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Response status code: 200
2025-04-10 14:23:37,063 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Response body: {"token_type":"Bearer","scope":"openid profile email","expires_in":3716,"ext_expires_in":3716,"access_token":"eyJ0...ANON1","refresh_token":"1.ANON...REFRESH","id_token":"eyJ0...ANON2"}
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: aud = [00000000-0000-0000-0000-000000000000]
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: iss = https://login.microsoftonline.com/11111111-1111-1111-1111-111111111111/v2.0
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: iat = 1744319916
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: nbf = 1744319916
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: exp = 1744323816
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: email = [email protected]
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: groups = [22222222-2222-2222-2222-222222222222, 33333333-3333-3333-3333-333333333333, 44444444-4444-4444-4444-444444444444, 55555555-5555-5555-5555-555555555555, 66666666-6666-6666-6666-666666666666, 77777777-7777-7777-7777-777777777777, 88888888-8888-8888-8888-888888888888, 99999999-9999-9999-9999-999999999999, aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa, bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb]
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: name = Xeotek Dev
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: oid = cccccccc-cccc-cccc-cccc-cccccccccccc
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: preferred_username = [email protected]
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: rh = 1.ANON...XYZ1AA
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: sid = dddddddd-dddd-dddd-dddd-dddddddddddd
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: sub = ANON_SUB_ID
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: tid = 11111111-1111-1111-1111-111111111111
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: uti = ANON_UTI_ID
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: ver = 2.0
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: wids = [eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee, ffffffff-ffff-ffff-ffff-ffffffffffff]
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] UserInfo: Xeotek Dev, [22222222-2222-2222-2222-222222222222, 33333333-3333-3333-3333-333333333333, 44444444-4444-4444-4444-444444444444, 55555555-5555-5555-5555-555555555555, 66666666-6666-6666-6666-666666666666, 77777777-7777-7777-7777-777777777777, 88888888-8888-8888-8888-888888888888, 99999999-9999-9999-9999-999999999999, aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa, bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb], [email protected]
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] TokenResponse: GZON1p....
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] OpenIdAuthProvider: found 149 group(s) in Kadeck
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] LdapAuthProvider: Group 22222222-2222-2222-2222-222222222222 not found for user [email protected].
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] LdapAuthProvider: Group 33333333-3333-3333-3333-333333333333 not found for user [email protected].
...
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] LdapAuthProvider: Adding user [email protected] to 2 groups
LDAP Integration
Data Streamhouse also supports LDAP and LDAPS for authentication with Active Directory (AD) or OpenLDAP directories.
Configuration Overview
To activate LDAP authentication:
Set the authentication method:
DSH_PORTAL_AUTHENTICATION_METHOD=ldap
Configure the required environment variables prefixed with dsh_portal_authentication_ldap_:
url
LDAP server URL (ldap:// or ldaps://)
user
Service user DN
password
Password for the service user
base
Base DN for user searches
filter
LDAP filter for retrieving users
Optional Parameters
user_id_attribute
Attribute used for login username
cn
user_fullname_attribute
Attribute used for full display name
user_id_attribute
group_attribute
Attribute containing groups (typically memberOf
)
memberOf
group_regex
Regex to extract group names from DNs
(?i)cn=(\w+),ou=Groups.*
group_management
Enable automatic group assignment
false
sync_interval_mins
Interval for synchronizing user and group information
60
(minutes)
debug
Enable LDAP debug logging
false
Setup Steps
Service Account: Configure a dedicated service account in LDAP with permissions to read user and group information.
Group Management (Optional): Enable automatic group mapping by setting:
DSH_PORTAL_AUTHENTICATION_LDAP_GROUP_MANAGEMENT=true
Ensure the group names match or are mapped in the Portal administration section.
Synchronization: User and group data will synchronize at startup and at the configured interval.
Nested Groups (Advanced): If users are members of nested groups, activate the nested LDAP module:
DSH_PORTAL_AUTHENTICATION_METHOD=ldap-nested
Configure:
dsh_portal_authentication_ldap_group_search_filter
dsh_portal_authentication_ldap_group_search_user_filter
Use the extended matching rule (
1.2.840.113556.1.4.1941
) for Active Directory recursive search, e.g.:(&(objectClass=person)(memberOf:1.2.840.113556.1.4.1941:=%s))
Example
Basic LDAP configuration snippet:
DSH_PORTAL_AUTHENTICATION_METHOD=ldap
DSH_PORTAL_AUTHENTICATION_LDAP_URL=ldaps://ldap.company.com
DSH_PORTAL_AUTHENTICATION_LDAP_USER=cn=service_user,ou=users,dc=company,dc=com
DSH_PORTAL_AUTHENTICATION_LDAP_PASSWORD=secret
DSH_PORTAL_AUTHENTICATION_LDAP_BASE=ou=users,dc=company,dc=com
DSH_PORTAL_AUTHENTICATION_LDAP_FILTER=(objectClass=person)
Group Mapping
Both OpenID and LDAP integrations allow mapping external groups to internal Data Streamhouse groups.
To configure:
Navigate to Administration → Groups.
Edit or create a group.
Set the External Mapping field to match the external group name or ID.
Recommendation: Always map a group for administrative users (e.g., Superusers) before activating identity provider integration to ensure continuous access.
Verification and Troubleshooting
After configuration:
Restart the Portal.
Attempt login via the configured identity provider.
Verify successful authentication and correct group assignment.
If issues occur:
Review Portal logs for detailed authentication errors.
Ensure that all OpenID URLs (redirect, JWKS, token, authorization) are reachable.
Check LDAP filters and user attributes.
Verify client ID, client secret, and scopes for OpenID.
For SSL/TLS issues, verify certificates and proxy trust settings.
Additional Configuration
LDAP Referral Handling
If your LDAP environment uses referrals (e.g., multiple LDAP servers for failover), configure:
DSH_PORTAL_AUTHENTICATION_LDAP_REFERRAL=<referral-mode>
Supported values depend on your LDAP server configuration.
LDAP Search Scope
By default, Portal searches recursively within the LDAP directory (subtree
search scope). To adjust, set:
DSH_PORTAL_AUTHENTICATION_LDAP_SCOPE=subtree
Supported scopes typically include subtree
, onelevel
, and base
.
Creating Groups and Roles
Instead of manually creating groups and assigning roles through the Portal user interface, Data Streamhouse also supports YAML-based configuration for groups and roles.
This approach allows you to:
Define all groups, roles, and permissions as code.
Version-control your access management.
Automate group and role provisioning during deployment.
Note: YAML-based group and role management is covered in the Automated Deployment section.
Using YAML definitions is highly recommended for environments requiring strict change control, automated deployments, or integration with GitOps workflows.
Quick Reference: OpenID vs LDAP Integration
Authentication Source
OpenID Connect Provider (e.g., Okta, Azure AD)
LDAP Directory (e.g., Active Directory, OpenLDAP)
Group Management
Via token claims (e.g., groups claim)
Via LDAP memberOf
attributes
Synchronization
Token-based on login + periodic sync
Database synchronization at startup + interval
Nested Groups Support
Depends on provider
Supported via LDAP-nested module
Debug Mode
Available (dsh_portal_authentication_openid_debug
)
Available (dsh_portal_authentication_ldap_debug
)
Typical Use Cases
Cloud-native identity providers
Enterprise directories, on-prem environments
Required External Access
Yes (to OpenID endpoints)
Yes (to LDAP server)
Group Mapping
Via External Mapping field in UI or YAML
Via External Mapping field in UI or YAML
Last updated
Was this helpful?