MOSIP ID-BB Sandbox Installation Notes

Installation was done using the Updated Deployment Guide with some changes.

Changes to Ingress setup.

The Sanbox uses a shared ALB for all exposed services. Therefore it was necessary to adapt the ingress setup in k8s-infra/mosip/aws/istio

Add type: ClusterIP to both ingress gateway service in iop.yaml

Remove proxy-protocol envoy filter (ALB does not use that) from istio-addons

After the ingress setup, complete the load balancer configuration before continuing. It is important the the “api-internal” endpoint works inside the cluster, otherwise partner onboarding step will fail (the failure is not obvious). See sandbox-infra for an example setup.

Installation notes

In Updated Deployment Guide | IDBB Govstack External Dependencies setup, ignore recaptcha setup, it is not required.
config-server pulls configuration from an external source and provides it to the various id-bb configurations. For Sandbox deployment, the upstream config branch was forked to https://github.com/GovStackWorkingGroup/sandbox-bb-identity-mosip-config and config-server configuration upgraded accordingly
The Helm charts have an systematic problem regarding allocated resources. Due to autoscaling (Karpenter), it is important to have consistent resource requests and limits. Fixing the resource limits required quite lot of work since true resource requirements are unknown.
- As a rule of thumb, especially for Java apps. See also https://aws.github.io/aws-eks-best-practices/reliability/docs/dataplane/#configure-and-size-resource-requestslimits-for-all-workloads
  - resources.limits.memory = resources.requests.memory
  - resources.limits.cpu > 1 (e.g. 2-4, definitely not <<1 or service startup takes forever)
  - resource.requests.cpu < 1 (e.g. 100m to allow over-provisioning. Services are idle most of the time)
  - Java max heap size ~50% of the memory limit (can be more if memory is >>1GiB)
The “partner-onboarding” step can fail silently (helm deployment works, but results is failure), and it is difficult to recover from that. What the step does, is that it runs postman collections using newman inside the cluster and calls the various internal APIs to configure things. The results are written to the “S3 bucket” simulated by minio.
Installing regclient fails if the keys and certificates are created with Openssl v3+. Use 1.1.1 instead.

Configuration notes

After installation, it is necessary to add some UINs to the system and configure an oidc appliction for the the USCT Demo. Details TBD.