• Work in progress
  • Mifos Research Notes

    1. What we want to run as BB

    2. How they do it

    3. How we will do it.

      1. Sandbox setup

        1. Setup Details

      2. What are the differences

        • Private images from provided repository should be exported and then imported in our repository → Currently AWS ECR.

      3. What are the obstacles

        • Apply known issue fix

        • Elasticsearch ceritificates

        • Manage Domans in order to access the apps via Ingresses


    Resources:
    Payment Hub installation link ( ) or this deployment mode referencing component architecture link ( ) or examples ( ).

     

    Basic information :

    1. Payment hub EE - Mifos (there is a a documentation that I will follow but the repo is empty: )

      1. Questions:

        1. Is there a guidebook/doc that we can follow to run the environment?

        2. Will installation guideline execution will set up a full environment or we will need to install other components?

        3. Which are all the repositories that will be used

          1. https://mifos.gitbook.io/docs/payment-hub-ee/overview/source-code-repositories

        4. After we run the environment how do we access the APIs?

          1. Postman Collections

    2. Fineract - Mifos

     

    Information from MIFOS related with running payment BB in sandbox:

     

    Steps  to add/login to ecr from another cluster  :

    1. After AWS configure / login to our ECR cluster :AWS ecr get-login-password --region ap-south-1 > password

    2. After AWS configure / login to third party (Mifos) AWS account / EKS cluster : kubectl create secret docker-registry dockersecret --docker-server=419830066942.dkr.ecr.ap-south-1.amazonaws.com --docker-username=AWS --docker-password=“$(cat password)” --docker-email=somanath@fynarfin.io

    3.  Patching service account with docker imagePullSecret: kubectl patch service account default -p ‘{“imagePullSecrets”: [{“name”: “dockersecret”}]}’

    4. Verify the Service account patch with pod — kubectl get pod <pod name> _n <namespace> -o=jsonpath=‘{.spec.imagePullSecrets[0].name}{“\n”}’

    Paymenthub Setup:-

    1. Link to base PHEE base helm chart :- (for referencing values.yaml) https://github.com/openMF/ph-ee-env-template/blob/master/helm/ph-ee-engine/values.yaml

    2. Link to paymenthub G2Psandbox Helm chart ( git clone env-labs— )

    3. For G2Psandbox chart—>

    4. Please refer to Operations app latest version known issue work around in the G2Psandbox helm chart README file

    5. Postman SIT env for paymenthub : https://github.com/openMF/ph-ee-env-template/blob/master/PostmanCollections/Environment/SIT.json

    6. Postman collection for paymenthub : https://github.com/openMF/ph-ee-env-template/blob/master/PostmanCollections/Payment%20Hub.json

    7. Test paymenthub postman collection for channel —>GSMA API—> GSMA P2P API

    Question & Answers over Paymenthub Setup:

    1. What image tag should we use for ph_ee_connector_mojaloop (https://github.com/openMF/ph-ee-env-labs/blob/7ed115f79efd46a42e1c63cdbf845d3adfbfec4e/helm/g2p-sandbox/values.yaml#L215 )

      1. Answered by Somanath Hugar via Slack: “You can use the latest image tag for ph_ee_connector_mojaloop”

    2. we will need some help for starting the ElasticSerach with elastic-certificates

      1. Answered by Somanath Hugar via Slack: Please refer to the documentation for ElasticSearch certificates, secrets

    3. We redeply the chart, but now we have 1 pod restarting, can you help sorting out the error and run the pod?

      1. pod: ph-ee-connector-ams-mifos

      2. we successful run it with ams_local_enabled: true

    4. Identify the pods required to run the Payment Hub bare minimum configuration?

      1. SLCB (commercial bank payment connector) service is not necessarily required for most demos so you can go ahead and remove it.

      2. In barebones, you can remove Zeebe Operate & either one of Mojaloop or GSMA payment schema connectors but that would impede us from giving demos so I wouldn't recommend removing them.

    5. We have a blocker on how to proceed with "Deploying the BPMN flows" and do we need to deploy any BPMN flows. Could you share more information or any short written explanation/procedure? 

      1. That would be the Upload BPMN API. You can find it in Payment Hub APIs Postman Collection Zeebe Operations APIs folder in ph-ee-env-template Github repository.    

      2. BPMNS : BPMNS

     

    FSP Setup :-

    1. Link to fineract base helm chart : https://github.com/fynarfin/fineract-env/blob/master/helm/fineract/values.yaml

    2. Link to fineract g2p sandbox helm chart: https://github.com/fynarfin/fineract-env/blob/master/helm/g2p-Sandbox/values.yaml

    3.  Community app link (for seeing the debits and credits) : - https://communityapp.sandbox.fynarfin.io/?baseApiUrl=https://fynams.sandbox.fynarfin.io&tenantIdentifier=rhino

    4. [ You should always specify the backend via baseApiUrl and tenant via tenant identifier Eg : https://openmf.github.io/community-app?baseApiUrl=https://demo.fineract.dev&tenantIdentifier=default

    5. Note: in case of Self-signed certificates, use curl —insecure flag or open the FSP host name (fynams.sandbox.fynarfin.io)in a new tab and accept the self-signed certs.

    Initial Actions:

    Mifos Fineract (Initial tryouts not used in current solution!)

    Docker run

    Info:

    Docker Run:

    API Info:

    Local Swagger

    Legacy doc:

     

    Kubernetes:

    Guide Kubernetes :

    • was not able to run working instance in local minikube. The described run information was followed but it’s not working, Tested by @Vladislav Todorov on local minikube and by @Tsvetomir Krumov . Same problem for both error after starting fineract-server (was trying to connect to DB on localhost instead the one described in fineract/kubernetes/fineractmysql-deployment.yml).

    • After adding 3 new environment variables to fineract/kubernetes/fineract-server-deployment.yml fineract-server starts successfully

    - name: fineract_tenants_url value: jdbc:mariadb://fineractmysql:3306/fineract_tenants - name: fineract_tenants_uid valueFrom: secretKeyRef: name: fineract-tenants-db-secret key: username - name: fineract_tenants_pwd valueFrom: secretKeyRef: name: fineract-tenants-db-secret key: password

     

    Access Info:

    Mifos Community APP - UI

    Swagger API:

    Health Check: https://localhost:8443/fineract-provider/actuator/health to return {"status":"UP"}

     

    Paymenthub/Fineract Setup

    Source

    Both provided charts for Paymenthub(PaymentHub Setup) and Fineract (FSP setup) are used as a dependent charts in current setup.

    Mifos PaymentHub and Fineract setup for sandbox

    Images

    All images referenced from Mifos private repository are pulled and pushed in Our ECR

    ElasticSearch secrets

    Original source provided:

    // curent elasticsearch version 7.16.3 // Current chart namespace "paymenthub" // Change them is other namespace is used //. when chart is installed or different version //. of elasticsearch is used in the chart docker pull docker.elastic.co/elasticsearch/elasticsearch:7.16.3 docker run --name elastic-helm-charts-certs -i -w /app \ docker.elastic.co/elasticsearch/elasticsearch:7.16.3 \ /bin/sh -c " \ elasticsearch-certutil ca --out /app/elastic-stack-ca.p12 --pass '' && \ elasticsearch-certutil cert --name security-master --dns security-master --ca /app/elastic-stack-ca.p12 --pass '' --ca-pass '' --out /app/elastic-certificates.p12" docker cp elastic-helm-charts-certs:/app/elastic-certificates.p12 ./ openssl pkcs12 -nodes -passin pass:'' -in elastic-certificates.p12 -out elastic-certificate.pem openssl x509 -outform der -in elastic-certificate.pem -out elastic-certificate.crt kubectl create secret generic elastic-certificates --from-file=elastic-certificates.p12 --namespace paymenthub kubectl create secret generic elastic-certificate-pem --from-file=elastic-certificate.pem --namespace paymenthub kubectl create secret generic elastic-certificate-crt --from-file=elastic-certificate.crt --namespace paymenthub

     

    Install/Update chart commands:

    Repo: Sandbox Github Repo

    Installation instructions github: GitHub Repository

    Obtain dependency:

    helm dependency build helm/g2p-sandbox
    • rebuild the charts/ directory based on the Chart.lock file

      or use "helm dependency update" to update charts/ based on the contents of Chart.yaml

    Upgrade/Install chart:

    Known Issues

    Known Issue Payment hub EE

    Known Issue Fineract

    Uninstall chart

    Uninstall:

    Delete all related Persistent Volume Claims:

     

    Services PaymentHubEE

    NAMESPACE

    NAME

    TYPE

    CLUSTER-IP      

    EXTERNAL-IP

    PORTS  

    COMMENTS  

    Commands Port forwarding

    URLS / status check

    paymenthub        

    fineract-mysql                 

    ClusterIP     

    172.20.160.162  

                                                                                

    mysql:3306►0                 

                           

     

     

    paymenthub        

    fineract-server                

    LoadBalancer  

    172.20.214.30   

    a80156cc82d9d450b812122a2550d372-118577547.eu-central-1.elb.amazonaws.com   

    8443►31694                   

                           

     

     

    paymenthub        

    fineract-server-local          

    ClusterIP     

    172.20.40.248   

                                                                                

    443►0                        

    Connection OK

    kubectl -n paymenthub port-forward service/fineract-server-local 15200:443

    https://127.0.0.1:15200/fineract-provider/actuator/health

    paymenthub        

    kafka                          

    NodePort      

    172.20.171.198  

                                                                                

    9092►30092                   

                           

     

     

    paymenthub        

    message-gateway                

    ClusterIP     

    172.20.230.70   

                                                                                

    port:80►0                    

                           

     

     

    paymenthub        

    mifos-community                

    LoadBalancer  

    172.20.33.11    

    a106843e644274921a9536ef41647a5a-2118717487.eu-central-1.elb.amazonaws.com  

    9090►31659                   

                           

     

     

    paymenthub        

    mifos-community-local          

    ClusterIP     

    172.20.226.30   

                                                                                

    9090►0                       

    Connection OK

    kubectl -n paymenthub port-forward service/mifos-community-local 15201:9090

    http://127.0.0.1:15201/?tenantIdentifier=rhino&baseApiUrl=https://127.0.0.1:15200/fineract-provider#/

    paymenthub        

    operationsmysql                

    ClusterIP     

    172.20.74.236   

                                                                                

    mysql:3306►0                 

                           

     

     

    paymenthub        

    operationsmysql-headless       

    ClusterIP     

                    

                                                                                

    mysql:3306►0                 

                           

     

     

    paymenthub        

    ph-ee-connector-channel        

    ClusterIP     

    172.20.215.128  

                                                                                

    port:80►0 http:82►0          

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-connector-channel 15102:80

    Postman collection

    paymenthub        

    ph-ee-connector-mojaloop-java  

    ClusterIP     

    172.20.108.164  

                                                                                

    port:80►0                    

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-connector-mojaloop-java 15105:80

    curl http://localhost:15105 -verbose

    paymenthub        

    ph-ee-elasticsearch            

    ClusterIP     

    172.20.208.46   

                                                                                

    http:9200►0 transport:9300►0 

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-elasticsearch 15103:9200

    curl -X GET "localhost:15103/_cluster/health?pretty"

    paymenthub        

    ph-ee-elasticsearch-headless   

    ClusterIP     

                    

                                                                                

    http:9200►0 transport:9300►0 

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-elasticsearch-headless 15104:9200

    curl -X GET "localhost:15104/_cluster/health?pretty"

    paymenthub        

    ph-ee-kibana                   

    ClusterIP     

    172.20.75.153   

                                                                                

    http:5601►0                  

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-kibana 15105:5601

    curl -k -s https://localhost:15105/api/status | json_pp

    paymenthub        

    ph-ee-operations-app           

    ClusterIP     

    172.20.172.220  

                                                                                

    port:80►0                    

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-operations-app 15100:80

    Postman collection

    paymenthub        

    ph-ee-operations-web           

    ClusterIP     

    172.20.134.142  

                                                                                

    4200►0                       

    Connection ? 

    Auth request goes trough dns?!? We have CORS error when trying with port forward

    kubectl -n paymenthub port-forward service/ph-ee-operations-web 15101:4200

     

    paymenthub        

    ph-ee-zeebe-ops                

    ClusterIP     

    172.20.240.140  

                                                                                

    port:80►0                    

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-zeebe-ops 15103:80

    Check es and Upload bpmn used

    Postman collection

     

     

    paymenthub        

    zeebe-operate                  

    ClusterIP     

    172.20.64.104   

                                                                                

    http:80►0                    

    Connection OK                  

    kubectl -n paymenthub port-forward service/zeebe-operate 15104:80

    Accesible in browser

    paymenthub        

    zeebe-zeebe                    

    ClusterIP     

                    

                                                                                

    http:9600►0 internal:26502►0 

    command:26501►0

     

     

     

    paymenthub        

    zeebe-zeebe-gateway            

    ClusterIP     

    172.20.81.82    

                                                                                

    http:9600►0 gateway:26500►0  

                 

     

     

    paymenthub

    ph-ee-connector-bulk      

    ClusterIP

     172.20.84.216

     

    port:80►0

    Connection OK

    kubectl -n paymenthub port-forward service/ph-ee-connector-bulk 15106:80

     

     

     

     

     

     

     

     

     

     

     

    BPMN Deployment

    Upload BPMN API. You can find it in Payment Hub Postman Collection Zeebe Operations APIs folder in ph-ee-env-template Github repository.

    Initially provided bpmn-s: BPMNs

    API information provided by Mifos:

    For creating Savings products and Tenant Clients:

    Environment: https://github.com/openMF/ph-ee-env-template/blob/master/PostmanCollections/Environment/FineractCoDevelop_environment.json
    Postman Collection : https://github.com/openMF/ph-ee-env-template/blob/master/PostmanCollections/G2P%20Sandbox%20Demo%20Prep.json

    For transactions:

    Then Use "Bulk APIs" => "Batch Transactions" API endpoint from : https://github.com/openMF/ph-ee-env-template/blob/master/PostmanCollections/Payment%20Hub.json . Environment https://github.com/openMF/ph-ee-env-template/blob/master/PostmanCollections/Environment/SIT.json
    With csv file: https://github.com/openMF/ph-ee-env-template/blob/master/PostmanCollections/ph-ee-bulk-demo-6.csv