Identity Provider Integration

The Data Streamhouse supports integration with external identity providers using OpenID Connect or LDAP. This enables secure, centralized authentication and authorization aligned with your organization's existing identity and access management systems.

This guide covers how to configure Data Streamhouse to authenticate users via OpenID or LDAP and manage user groups automatically.

Prerequisites

Before configuring identity provider integration (OpenID Connect or LDAP), the following conditions must be met:

License Activation

You must activate your Data Streamhouse licenses before enabling identity provider authentication.

Why this matters: Without a valid license, Portal will not allow external authentication, and login will fail. Always ensure that the licenses are fully activated before proceeding with identity provider setup.

Refer to the License Activation Scenarios section for details on activating your license.

Group Preparation

Before enabling integration with an external identity provider (OpenID or LDAP), you must first create the required groups and roles inside the Data Streamhouse Portal.

Why this matters: Once identity provider integration is active, Portal will attempt to map authenticated users to internal groups based on external group claims or attributes. If no matching groups exist, users will not be assigned any roles and will be denied access.

You can create groups in two ways:

Portal UI: Navigate to Administration → Groups and manually create all required groups, setting the External Mapping field to match external group names or IDs (if required).
YAML Files: Define groups and roles as YAML configuration files and import them during deployment. (This method is recommended for production environments and is covered in the Advanced Setup section.)

Recommendation: Always create and map the Superusers group first to ensure administrative access after activating identity provider integration.

OpenID Integration

Data Streamhouse can authenticate users via OpenID Connect compliant providers, such as Azure AD, Okta, or Entra ID.

Configuration Overview

To activate OpenID authentication:

Set the authentication method:

DSH_PORTAL_AUTHENTICATION_METHOD=openid

Configure the required environment variables prefixed with dsh_portal_authentication_openid_:

Variable

Description

client_id

Client ID provided by your OpenID provider

client_secret

Client secret provided by your OpenID provider

redirect_url

Redirect URL after authentication (e.g., https://dsh.acme.org/api/auth/oauth/callback)

jwks_url

JWKS URL for verifying tokens

authorization_url

URL for initiating the OpenID flow

token_url

URL for exchanging codes for tokens.

The redirect_url must always end with the fixed path /api/auth/oauth/callback. You can configure the domain freely (e.g., https://portal.company.com), but the path itself is mandatory and cannot be changed. Identity providers must redirect users exactly to /api/auth/oauth/callback after authentication.

Ensure the container can access the specified URLs.

Optional Parameters

Variable

Description

Default

groups_claim

Claim name containing user groups

groups

scope

Requested OpenID scopes

openid profile email offline_access

debug

Enable debug logging for OpenID

false

group_management

Enable automatic group mapping

false

login_url

/#/login

sync_interval_mins

Interval for synchronizing group information

60 (minutes)

Setup Steps

Database Preparation: Ensure an external database is configured. Portal requires persistence for user and group mappings.
Provider Registration: Register Data Streamhouse as a client application in your OpenID provider.
Environment Configuration: Configure environment variables with your client ID, secret, and URLs. Ensure the redirect_url matches the registered redirect URI.
Group Mapping (Optional):
- Map OpenID groups to internal Data Streamhouse groups.
- Use the "External Mapping" field inside the Portal administration interface.
Activation: Set the authentication method to OpenID and restart the Portal.
Login: Authenticate via your OpenID provider.

Note: To enable support for proxies terminating TLS, set:
DSH_PORTAL_TRUST_PROXY_SSL=true
This avoids CSRF errors when using HTTPS offloading.

Troubleshooting

To troubleshoot OpenID configuration issues, set the system-wide log level to DEBUG and enable the OpenID debug parameter by setting it to true.

Here's an example of a successful authentication log output:

2025-04-10 14:23:36,345 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] OpenIdAuthProvider: OpenIdAuthProvider license found. Proceeding with authentication.
2025-04-10 14:23:36,345 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] Retrieving token for authorization code: 1.ANON...XYZ1AA
2025-04-10 14:23:36,345 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] sendTokenRequest with request body: grant_type=authorization_code&client_id=00000000-0000-0000-0000-000000000000&client_secret=ANON...SECRET&code=1.ANON...XYZ1AA&redirect_uri=https://dsh.acme.org/api/auth/oauth/callback
2025-04-10 14:23:37,063 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Response status code: 200
2025-04-10 14:23:37,063 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Response body: {"token_type":"Bearer","scope":"openid profile email","expires_in":3716,"ext_expires_in":3716,"access_token":"eyJ0...ANON1","refresh_token":"1.ANON...REFRESH","id_token":"eyJ0...ANON2"}
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: aud = [00000000-0000-0000-0000-000000000000]
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: iss = https://login.microsoftonline.com/11111111-1111-1111-1111-111111111111/v2.0
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: iat = 1744319916
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: nbf = 1744319916
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: exp = 1744323816
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: email = [email protected]
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: groups = [22222222-2222-2222-2222-222222222222, 33333333-3333-3333-3333-333333333333, 44444444-4444-4444-4444-444444444444, 55555555-5555-5555-5555-555555555555, 66666666-6666-6666-6666-666666666666, 77777777-7777-7777-7777-777777777777, 88888888-8888-8888-8888-888888888888, 99999999-9999-9999-9999-999999999999, aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa, bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb]
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: name = Xeotek Dev
2025-04-10 14:23:37,306 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: oid = cccccccc-cccc-cccc-cccc-cccccccccccc
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: preferred_username = [email protected]
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: rh = 1.ANON...XYZ1AA
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: sid = dddddddd-dddd-dddd-dddd-dddddddddddd
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: sub = ANON_SUB_ID
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: tid = 11111111-1111-1111-1111-111111111111
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: uti = ANON_UTI_ID
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: ver = 2.0
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIDUserInfoFetcher [nioEventLoopGroup-3-3] Claim: wids = [eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee, ffffffff-ffff-ffff-ffff-ffffffffffff]
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] UserInfo: Xeotek Dev, [22222222-2222-2222-2222-222222222222, 33333333-3333-3333-3333-333333333333, 44444444-4444-4444-4444-444444444444, 55555555-5555-5555-5555-555555555555, 66666666-6666-6666-6666-666666666666, 77777777-7777-7777-7777-777777777777, 88888888-8888-8888-8888-888888888888, 99999999-9999-9999-9999-999999999999, aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa, bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb], [email protected]
2025-04-10 14:23:37,307 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] TokenResponse: GZON1p....
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] OpenIdAuthProvider: found 149 group(s) in Kadeck
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] LdapAuthProvider: Group 22222222-2222-2222-2222-222222222222 not found for user [email protected].
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] LdapAuthProvider: Group 33333333-3333-3333-3333-333333333333 not found for user [email protected].
...
2025-04-10 14:23:37,309 INFO c.x.k.a.p.OpenIdAuthProviderImpl [nioEventLoopGroup-3-3] LdapAuthProvider: Adding user [email protected] to 2 groups

LDAP Integration

Data Streamhouse also supports LDAP and LDAPS for authentication with Active Directory (AD) or OpenLDAP directories.

Configuration Overview

To activate LDAP authentication:

Set the authentication method:

DSH_PORTAL_AUTHENTICATION_METHOD=ldap

Configure the required environment variables prefixed with dsh_portal_authentication_ldap_:

Variable

Description

url

LDAP server URL (ldap:// or ldaps://)

user

Service user DN

password

Password for the service user

base

Base DN for user searches

filter

LDAP filter for retrieving users

Optional Parameters

Variable

Description

Default

user_id_attribute

Attribute used for login username

cn

user_fullname_attribute

Attribute used for full display name

user_id_attribute

group_attribute

Attribute containing groups (typically memberOf)

memberOf

group_regex

Regex to extract group names from DNs

(?i)cn=(\w+),ou=Groups.*

group_management

Enable automatic group assignment

false

sync_interval_mins

Interval for synchronizing user and group information

60 (minutes)

debug

Enable LDAP debug logging

false

Setup Steps

Service Account: Configure a dedicated service account in LDAP with permissions to read user and group information.
Group Management (Optional): Enable automatic group mapping by setting:
```
DSH_PORTAL_AUTHENTICATION_LDAP_GROUP_MANAGEMENT=true
```
Ensure the group names match or are mapped in the Portal administration section.
Synchronization: User and group data will synchronize at startup and at the configured interval.
Nested Groups (Advanced): If users are members of nested groups, activate the nested LDAP module:
```
DSH_PORTAL_AUTHENTICATION_METHOD=ldap-nested
```
Configure:
- dsh_portal_authentication_ldap_group_search_filter
- dsh_portal_authentication_ldap_group_search_user_filter
Use the extended matching rule (1.2.840.113556.1.4.1941) for Active Directory recursive search, e.g.: (&(objectClass=person)(memberOf:1.2.840.113556.1.4.1941:=%s))

Example

Basic LDAP configuration snippet:

DSH_PORTAL_AUTHENTICATION_METHOD=ldap
DSH_PORTAL_AUTHENTICATION_LDAP_URL=ldaps://ldap.company.com
DSH_PORTAL_AUTHENTICATION_LDAP_USER=cn=service_user,ou=users,dc=company,dc=com
DSH_PORTAL_AUTHENTICATION_LDAP_PASSWORD=secret
DSH_PORTAL_AUTHENTICATION_LDAP_BASE=ou=users,dc=company,dc=com
DSH_PORTAL_AUTHENTICATION_LDAP_FILTER=(objectClass=person)

Group Mapping

Both OpenID and LDAP integrations allow mapping external groups to internal Data Streamhouse groups.

To configure:

Navigate to Administration → Groups.
Edit or create a group.
Set the External Mapping field to match the external group name or ID.

Recommendation: Always map a group for administrative users (e.g., Superusers) before activating identity provider integration to ensure continuous access.

Verification and Troubleshooting

After configuration:

Restart the Portal.
Attempt login via the configured identity provider.
Verify successful authentication and correct group assignment.

If issues occur:

Review Portal logs for detailed authentication errors.
Ensure that all OpenID URLs (redirect, JWKS, token, authorization) are reachable.
Check LDAP filters and user attributes.
Verify client ID, client secret, and scopes for OpenID.
For SSL/TLS issues, verify certificates and proxy trust settings.

Additional Configuration

LDAP Referral Handling

If your LDAP environment uses referrals (e.g., multiple LDAP servers for failover), configure:

DSH_PORTAL_AUTHENTICATION_LDAP_REFERRAL=<referral-mode>

Supported values depend on your LDAP server configuration.

LDAP Search Scope

By default, Portal searches recursively within the LDAP directory (subtree search scope). To adjust, set:

DSH_PORTAL_AUTHENTICATION_LDAP_SCOPE=subtree

Supported scopes typically include subtree, onelevel, and base.

Creating Groups and Roles

Instead of manually creating groups and assigning roles through the Portal user interface, Data Streamhouse also supports YAML-based configuration for groups and roles.

This approach allows you to:

Define all groups, roles, and permissions as code.
Version-control your access management.
Automate group and role provisioning during deployment.

Note: YAML-based group and role management is covered in the Automated Deployment section.

Using YAML definitions is highly recommended for environments requiring strict change control, automated deployments, or integration with GitOps workflows.

Quick Reference: OpenID vs LDAP Integration

Feature

OpenID

LDAP

Authentication Source

OpenID Connect Provider (e.g., Okta, Azure AD)

LDAP Directory (e.g., Active Directory, OpenLDAP)

Group Management

Via token claims (e.g., groups claim)

Via LDAP memberOf attributes

Synchronization

Token-based on login + periodic sync

Database synchronization at startup + interval

Nested Groups Support

Depends on provider

Supported via LDAP-nested module

Debug Mode

Available (dsh_portal_authentication_openid_debug)

Available (dsh_portal_authentication_ldap_debug)

Typical Use Cases

Cloud-native identity providers

Enterprise directories, on-prem environments

Required External Access

Yes (to OpenID endpoints)

Yes (to LDAP server)

Group Mapping

Via External Mapping field in UI or YAML

PreviousAutomated Deployment NextLicense Activation Scenarios

Last updated 3 months ago

Was this helpful?