Interoperability & Network

Overview

A comprehensive network strategy is essential before deploying the Data Streamhouse (DSH) Portal. This chapter outlines the interoperability considerations, network requirements, and connectivity patterns necessary for a successful single deployment of the DSH Portal.

While this guide focuses on a single deployment architecture, we strongly recommend a High Availability (HA) setup for production environments to ensure system resilience and continuous operation.

Network Strategy Planning

Before deployment, develop a detailed network strategy that addresses:

1. Ingress configurations - How external clients will access the DSH Portal

2. Egress requirements - How the DSH Portal will connect to external systems

3. Internal communication - Database connections and service dependencies

4. Security boundaries - Network segmentation and access controls

5. Authentication services - LDAP/OpenID connectivity

Connectivity Requirements

Ingress

The DSH Portal requires the following ingress configurations:

• HTTP(S) port accessibility for web browser connections

• WebSocket support for real-time updates and streaming data

• Proper proxy configurations if operating behind a load balancer or gateway

Egress

The DSH Portal establishes outbound connections to:

• PostgreSQL database for configuration storage

• Authentication providers (LDAP/OpenID)

• Integration endpoints (Kafka, Kinesis, Schema Registry, etc.)

Port Requirements

Service	Default Port	Protocol	Notes
DSH Portal Web Interface	8080	HTTP	Configurable
DSH Portal Web Interface (TLS)	8443	HTTPS	With mounted certificate
PostgreSQL	5432	TCP	For configuration storage
LDAP	389	TCP	For authentication
LDAP (TLS)	636	TCP	Secure LDAP
OpenID Connect	443	HTTPS	For authentication
Kafka Brokers	9092	TCP	Plain connection
Kafka Brokers (TLS)	9093	TLS	Secure connection
Kafka Schema Registry	8081	HTTP	For schema management
Kafka Connect	8083	HTTP	For connector management
Amazon Kinesis	443	HTTPS	AWS service
JMX	9010-9020	Varies	Apache Kafka cluster metrics
ARGUS	8080/8443	HTTP(S)	Connection to one or multiple ARGUS instances

Replace the default sample ports with your designated port numbers.

Configuration

Environment Variables

The DSH Portal can be configured through environment variables, including:

• Web server ports and binding addresses

• Database connection details (JDBC URL, credentials)

• Proxy settings if required

• Log levels and output paths

The exact environment variable names are documented in the installation guide.

Integration Configuration

Connections to external systems such as:

• Apache Kafka clusters (+ optional JMX)

• Amazon Kinesis

• Schema Registry

• Kafka Connect

• Microsoft Teams

• Slack

• PagerDuty

• Mail (SMTP)

These can be configured through the DSH Portal UI after deployment.

Deployment Considerations

Container Image

The DSH Portal is available as a Docker image containing all required dependencies including:

• Java Runtime Environment

• Web server components

• Required libraries

No additional runtime dependencies need to be installed on the host.

TLS/SSL Configuration

For secure deployments, SSL certificates must be:

1. Generated or obtained from a trusted Certificate Authority

2. Mounted to the container or pod

3. Referenced in the DSH Portal configuration

The exact mounting path depends on the orchestration platform (Kubernetes, OpenShift, etc.).

Health Monitoring

For proper operation with container orchestrators, the DSH Portal exposes the following health endpoints:

• /health - Overall health status

• /live - Liveness probe endpoint

• /ready - Readiness probe endpoint

Configure your orchestrator to use these endpoints for proper lifecycle management.

Network Topology

As illustrated in the attached diagram, the DSH Portal deployment consists of:

1. Configuration Layer:

   • HTTP(S) Port with optional mounted certificate

   • Database connection (URL and Port)

   • Optional proxy configuration

2. Ingress Layer:

   • HTTP(S) connections from web browsers

   • WebSocket connections with appropriate proxy headers

3. Outgoing Connections:

   • PostgreSQL database for configuration storage

   • Authentication services (LDAP/OpenID)

   • Integration endpoints for various services

Security Recommendations

1. Use TLS for all external connections

2. Implement proper network segmentation

3. Apply the principle of least privilege for all connections

4. Configure proper authentication for all integration endpoints

5. Regularly review and update network security policies

Example Deployment Architecture

A typical deployment would include:

1. Ingress controller or load balancer handling external HTTP(S) traffic

2. DSH Portal container with mounted certificates

3. Dedicated PostgreSQL instance for configuration storage

4. Secure connections to authentication providers (optional)

5. Properly configured firewall rules for integration endpoints

By carefully planning your network strategy before deployment, you can ensure proper operation and minimize security risks in your DSH Portal implementation.