X-Road Installation in Sandbox

Installation is based on information found in https://github.com/nortal/GovStack-IM-BB-SandBox-Deployment. The procedure is defined as a GitLab CI pipeline, so some adaptation was needed.

Due to existing playground deployment, the custom images were already in ECR. The deployment uses a slightly modified X-Road, and building X-Road is needed without access to the images.

Deployment

git clone https://github.com/nortal/GovStack-IM-BB-SandBox-Deployment

Set up ENV variables:

K8S_NAMESPACE=im-xroad
X_ROAD_IMAGE_TAG="7.2.2-IAM"
K8S_CS_SS_DB_STORAGE_CLASS_NAME=gp3
K8S_TLD_NAME=im-xroad.playground.sandbox-playground.com
K8S_EXPOSE_SERVICES=false
AWS_ACCOUNT=`<account id>`
AWS_DEFAULT_REGION=eu-central-1
X_ROAD_METRICS_IMAGE_TAG=latest
PUBSUB_TAG=0.0.1-develop-c5e275ed
PUBSUB_MESSAGING_API_IMAGE_TAG=$PUBSUB_TAG
PUBSUB_MANAGEMENT_API_IMAGE_TAG=$PUBSUB_TAG
PUBSUB_MANAGEMENT_UI_IMAGE_TAG=$PUBSUB_TAG
PUBSUB_SUBSCRIBER_MOCK_IMAGE_TAG=$PUBSUB_TAG
PUBSUB_DB_SCHEMA_IMAGE_TAG=$PUBSUB_TAG
MANAGEMENT_API_XROAD_SECURITY_SERVER_BASE_URL=<https://sandbox-xroad-ss3.${K8S_NAMESPACE}.svc.cluster.local:4000/api/v1/>
MANAGEMENT_API_XROAD_PUBSUB_SUBSYSTEM_IDENTIFIER=SANDBOX:GOV:PROVIDER:TEST
K8S_SUBNET_ALLOW_LIST=""

Most deployments in the provided Helm charts do not specify resource requests and limits, which does not work well with Karpenter and autoscaling. One possibility is to add a default resource limit for the namespace ( 1Gi is probably too much for some services):

#limits.yml
apiVersion: v1
kind: LimitRange
metadata:
  name: default-mem-limit
spec:
  limits:
  - default:
      memory: 1Gi
    defaultRequest:
      memory: 1Gi
    type: Container
#
kubectl apply -f limits.yml -n $K8S_NAMESPACE

Core X-Road Deployment

Keycloak is part of PubSub, but it is shared with X-Road to provide authentication. For keycloak deployment, create new pubsub/keycloak/config/pubsub-realm-sandbox.json file based on existing template and change the URLs to be consistent with the selected external domain (K8S_TLD_NAME). Also update `pubsub/keycloak/values.yml` with

PubSub

Set up limited access to X-Road Admin UIs (see https://github.com/GovStackWorkingGroup/sandbox-infra/blob/SND-651/live/playground/kube/im-xroad.tf for an example).
Create MANAGEMENT_API_XROAD_ADMIN_API_KEY on security server SS3