VERSION 1 from Nov 15, 2024

Table of contents

Project Overview

Husika is an early warning system prototype focusing on efficient communication between civil servants and communities in the Horn of Africa region. The system enables civil servants to validate raw threat information and create broadcasts to inform communities about potential hazards, while collecting structured feedback from recipients.

Project Goals (Govstack)

1. Create Blueprint and Foundation:

   • Build a working example solution for Husika

   • Serve as foundation for future GovStack cases

   • Build reusable technical components

   • Support similar initiatives in the future

   • Demonstrate practical implementation

   • Provide foundation for Husika 2.0

2. Enable ICPAC and Bunifu:

   • Improve positive impact through early warnings

   • Build reliable warning system

   • Reach target audience effectively (citizens in HoA)

System Goals (Husika Platform)

Core Objectives:

1. Enable civil servants to monitor threats effectively

2. Facilitate clear communication about potential hazards

3. Collect and organize community responses

4. Support multi-language broadcasts

Key Functionalities:

1. Threat monitoring and validation

2. Structured broadcast creation

3. Response collection

4. Jurisdiction-based access control

Timeline & Team Activity Resources

Timeline

Open

Architectural Workshop

On Aug 22, 2024 and Sep 8, 2024 we had two architectural workshops with the following goals:

• Introduce the team, meeting objectives, and set the stage for discussion.

• Review team structure, roles, and task organization.

• Provide a high-level overview of the existing system architecture and key components.

• Start to discuss and elaborate potential architectural changes, improvements, and their potential impact.

• Address questions and outline next steps for the project.

The workshop material and results can be found here:

Initial Challenges

1. Research & Documentation Gaps

• There's a need for foundational user research

• Initial documentation could use some improvement

• It would be great to have a clearly defined scope

2. User & System Structure

• User personas are still undefined

• Role definitions could be clearer

• Jurisdictional requirements are a bit complicated

3. Design & Interface

• We don't have established design patterns yet

• We're starting from the ground up

• It's important to handle complex processes intuitively

• Multi-language support is a requirement

4. Communication Flow

• A structured approach to broadcast creation would be helpful

• Systematic feedback collection is needed

• Implementing a clear threat monitoring system is essential

Methodology

With limited time and resources, our approach focused on quickly capturing essential functionalities and user needs, using an agile, iterative workflow.

Essential steps of GovStack methodology had to be skipped for this demo application due to resource and time constraints, i.e. As-Is User Journey, To-Be User Journey and Service Blueprint. What we managed to accomplish during research and design phase:

Market Research

To further inform our design decisions and contextualize our project within the broader industry landscape, we conducted a quick market analysis of multi-hazard early warning systems. This research provided valuable insights into current best practices, user expectations, and feature sets in similar solutions, which helped shape our approach.

Direct BPMN Model Creation

We initiated the project by developing a Business Process Model and Notation (BPMN) model to map out the system's core workflows. This decision stemmed from our need to quickly visualize the structure of the system, given the time constraints. Our BPMN model was based on initial customer meetings, assumptions, and available documents. Since we couldn't access the existing system directly, we leveraged screenshots from their presentations to understand the current flow and identify gaps.

Wireframe Development and Ideation

After establishing the BPMN model, we transitioned to the ideation phase to create initial wireframes. Throughout this phase, we worked closely with the development team to verify that our design ideas aligned with technical constraints and current capabilities. This frequent communication allowed us to iterate rapidly, ensuring that our designs were both practical and aligned with the project's technical requirements.

Prototype Creation and Iteration

After refining our wireframes based on feedback from stakeholders and technical input, we developed the first prototype. This prototype embodied the initial assumptions and evolved requirements, serving as the foundation for further testing and validation.

Design Solutions

Clear Information Structure:

• Dashboard with key metrics

• Recent threats overview

• Broadcast creation and tracking

• Response collection and organization

User-Focused Approach:

• Intuitive navigation

• Step-by-step broadcast creation

• Clear status indicators

• Easy access to critical information

Communication Tools:

• Template-based broadcast creation

• Multi-language support

• Structured response options

• Clear feedback organization

Key Design Decisions

Decision 1: Use of MUI (Material-UI) for Component Library

Context and Challenge

https://mui.com/

Given the project's tight timeline, we needed to streamline the design and development process as much as possible. Although a custom component library would allow for more tailored, case-specific design solutions, building this from scratch would require extensive time and resources.

Solution and Rationale

To expedite the design process without compromising functionality, we decided to use MUI (Material-UI) as our component library. MUI offers ready-made set of components that adhere to industry-standard Material Design principles. This choice provided a balance between speed and design quality, allowing us to focus more on flow and usability rather than component creation.

Impact on Flow and Users

Using MUI significantly reduced development time and ensured consistency in the UI. While custom components would offer greater design specificity, MUI allowed us to achieve a professional and polished interface that meets general user experience standards. This decision, however, may require customization or future adjustments to better align with case-specific needs as the project matures.

Decision 2: Limiting User Layers for Prototype

Context and Challenge

The original system had multiple user roles and complex hierarchies. For the prototype, we needed to simplify the user structure while maintaining core functionality.

Solution and Rationale

We limited the system to two primary user types:

• Administrators (civil servants managing the system and working based on the jurisdiction)

• Recipients (people receiving broadcast messages)
  This simplified approach allows us to focus on core functionalities and user flows without getting caught in complex role management. In addition, due to lack of documentation regarding various role this approach would be optimum solution within our limited time.

Impact on Flow and Users

This decision dramatically simplified the user experience and development process, allowing us to focus on the core interactions between administrators and recipients. While this may need expansion in future versions, it provides a solid foundation for testing and validating the main system concepts.

Decision 3: New Flow

Context and Challenge

Starting from scratch, we needed to create intuitive user flows that would make complex emergency communication processes manageable and effective.

Solution and Rationale

We developed a streamlined flow focusing on two main areas:

• Threat monitoring and validation

• Broadcast creation and distribution while collecting responses
  Each area was designed with clear steps and logical progression, minimizing confusion and user error.

Impact on Flow and Users

The new flow provides a clear path for users to follow, making complex tasks more manageable. It reduces cognitive load while ensuring all necessary steps are completed in the right order.

Decision 4: New Terminology

Context and Challenge

Clear communication required consistent, understandable terminology that accurately reflected system functions while avoiding confusion.

Solution and Rationale

We developed a simplified terminology framework. These terms were chosen for clarity and directness, avoiding technical jargon.

• "Threats" for potential hazards being monitored

  • Threat: Raw information regarding a warning or potential hazard. This is what admins (civil servants) see and use to create broadcasts. 

• "Broadcasts" for communications sent to recipients

  • Broadcast: Information that is shared with a group of end-users. These are created based on threats and distributed to affected people.

• "Feedbacks" for responses

  • Feedbacks: Messages received in response to broadcasts or about ongoing situations. We have two primary different Feedback.

    • Feedback to ICPAC: which is mainly responses conducted for validation purposes towards threat.

    • Feedback to Broadcast: which is responses that is collected through the broadcast.

Impact on Flow and Users

Consistent terminology helps users quickly understand system functions and reduces confusion. It provides a common language for all users while maintaining professional standards. Used terminologies should be clearly documented also for the future.

Decision 5: Quantitative and Qualitative Feedback Collection System

Context and Challenge

The system needed to collect both structured and unstructured feedback from recipients of broadcasts. We needed a way to gather quantifiable data for quick analysis and validations while also allowing for detailed, contextual information from the community.

Solution and Rationale

We implemented a two-tier response system:

1. Validate Status: Simple Yes/No responses for quick, quantifiable feedback

2. General Response: Open-text format for detailed information
   This approach allows for both statistical analysis and rich, contextual feedback while keeping the response mechanism simple for end-users.

Impact on Flow and Users

This decision simplified data collection and analysis while maintaining flexibility in feedback gathering. The validate status option provides immediate, measurable insights, while the general response option allows for more detailed community input when needed.

Decision 6: Jurisdiction-Based Access Control

Context and Challenge

Users needed to see threats across multiple locations for better situational awareness, but broadcast capabilities needed to be restricted to appropriate jurisdictional areas to maintain proper authority structures and prevent unauthorized communications.

Solution and Rationale

We implemented a hybrid approach where:

• Threats are visible across multiple locations

• Broadcast capabilities are restricted to user's jurisdiction

• Clear indication of jurisdictional limitations in the interface
  This balanced the need for widespread threat awareness with controlled communication channels.

Impact on Flow and Users

This approach provides users with comprehensive threat visibility while maintaining appropriate control over broadcast creation. The clear distinction between viewing and action capabilities helps users understand their roles and responsibilities within the system.

Decision 7: Template-Based Broadcast Creation

Context and Challenge

Creating effective emergency communications requires consistency in messaging while allowing for situation-specific details. We needed to balance standardization with flexibility in broadcast creation.

Solution and Rationale

We implemented a template system for broadcasts that:

• Provides pre-structured formats

• Allows for customization

• Supports multiple languages

• Includes specific response collection options
  This approach ensures consistency in communication while maintaining flexibility for different situations.

Impact on Flow and Users

Templates streamline the broadcast creation process while ensuring important information isn't overlooked. This makes the system more efficient for civil servants while maintaining message quality and effectiveness.

Key Technical Decisions

#1 (Micro)service-based architecture according to GovStack Building Blocks

The (micro)service-based architecture of GovStack is designed to enhance the modularity, scalability, and interoperability of digital government services. This architecture is structured around specific building blocks that facilitate efficient service delivery. The GovStack Building Block specifications can be found here.

Advantages of the Architecture

• Modularity: Each service can be developed, deployed, and scaled independently, allowing for rapid updates and maintenance.

• Interoperability: Services can easily communicate and share data, promoting collaboration across different government platforms.

• Scalability: The architecture supports scaling services based on demand, optimizing resource utilization and enhancing performance.

Implementation Considerations

• Governance: Establish clear guidelines for service development and integration to ensure compliance with standards.

• Monitoring and Maintenance: Implement tools for monitoring service performance and health to quickly address issues.

• User Feedback: Incorporate mechanisms for collecting user feedback to continuously improve services.

This architecture not only supports the immediate goals of the GovStack initiative but also lays the groundwork for future innovations in digital governance, ensuring that services remain relevant and effective in meeting the needs of citizens. For more detailed specifications, you can refer to the GovStack Building Blocks documentation.

#2 Software choice for Messaging Building Block: RapidPro

RapidPro is a robust messaging platform chosen as a software option for the Messaging Building Block within the GovStack framework. It is designed to facilitate effective communication between government services and citizens, enabling timely and relevant interactions.

RapidPro is an open-source platform developed by UNICEF that enables users to design, scale, and manage communication workflows using channels like SMS, Telegram, and WhatsApp. It provides a user-friendly, visual interface for building interactive messaging services, widely used in sectors like health, education, and disaster response. Key features include flow creation, contact management, real-time analytics, and multi-channel communication.

RapidPro is acting as a GovStack building block software, meaning it is a modular solution designed to be reused across different government services and digital infrastructure projects. It has the potential to be leveraged in various GovStack implementations to streamline communication, citizen engagement, and service delivery across digital platforms.

RapidPro SaaS offerings:

• RapidPro (UNICEF)

• TextIt - The Leading Bot Platform

RapidPro deployment experience:

We explored three different approaches:

1. Deploying the app on a bare virtual privet server or personal laptop: This involved manually installing all dependencies, but it failed due to dependency conflicts and difficulties of managing/cleaning dependencies.

2. Creating custom Kubernetes manifests: We attempted to build our own manifests based on the provided Dockerfiles. However, this approach also failed because some internal variables were hardcoded, making it impossible to override them externally. Potential work require good internal knowledge of the system.

3. Using the provided docker-compose file (intended for development): Initially, it seemed to work, but when we started integrating it with our system, it failed.

The RapidPro team suggested adjusting the database connection pool, but this solution is problematic.
It requires us to decompose the application, dive into the source code, and have a deep understanding of the internal workings of the system.

#3 Software choice for Information Mediator Building Block: Kafka

Kafka is a powerful distributed event streaming platform selected for the Information Mediator Building Block within the GovStack framework. It is designed to facilitate the seamless exchange of data between various services, ensuring interoperability and efficient data flow.

#4 Software choice for Registry Building Block: Postgres (over Baserow)

PostgreSQL, commonly known as Postgres, is the chosen database management system for the Registry Building Block within the GovStack framework. It is favored for its robustness, flexibility, and advanced features that support the complex data requirements of digital government services.

Further information on initial Baserow investigation, conducted by @Vuk Damjanovic (Deactivated)

Overall findings
Baserow as a standalone database/storage/registry solution is excellent. It provides a package of technologies combined in a way that allows non technical person to manage databases on their own in a, somewhat, intuitive way. The advantage of the Baserow over a traditional database is the UI, and package of technologies ready to cover various data storage scenarios out of the box using Docker or Helm charts.

Mentioning Docker and Helm charts, Docker is a preferred way for deployment. Both official instructions and hands on experience are heavily leaning toward Docker as a preferred deployment approach. Helm charts are an alternative, however, not an easy one. In order to up and run Baserow with Helm charts, official chart and instructions were not sufficient to achieve this. In order to make it up and running we had to resort to a chart made by a different user, on top of that the amount of time and people involved in this process was substantially higher compared to the Docker approach.

Going outside of the scope of the standalone approach makes Baserow quite complicated to work. If the goal of the Baserow is to be used as an alternative to the traditional open-source databases (like PostgreSQL/MySQL), it seems like an over-engineered approach and there's no clear benefit that would cover for the amount of work needed to make Baserow to do what a simple instance of a traditional DB would do. Moreover, mentioned traditional technologies combined with open-source DB UI client (DBeaver Community for example) covers 80% of the Baserow benefits. The only loss is that user isn’t really able to use it as easy as Baserow without having SQL knowledge.

One of technologies within the Baserow is a PostgreSQL instance.
Baserow is adding a lot of abstractions on top of it, and working with it it’s like working with a black box. These are additional hoops needed to be jumped trough in order to make Baserow do what a traditional DB is doing from a development point of view.
For instance, connecting Baserow with a SpringBoot app (or something even more complicated like X-Road) takes massive amount of time, it’s more verbose and it’s quite prone to errors and potential issues which are not the case with conventional DBs. When using Baserow trough UI there's many automated actions that are happening in the background, which are needed to be executed manually in sequence when working with Baserow via APIs.
To wrap up the overall view on the Baserow, it’s excellent as a standalone product, but not that good (over-engineered)as a traditional DB for the same purpose.

Suitable for production Husika?
Suitable for POC demo from GoFore?

I would say no. What Husika is currently using (MySQL) and what is expected from the production DB, traditional DBs are more suitable in this case. We could make Baserow to do what a conventional DB does and make Baserow to be a go-to solution for Husika, but with a massive investment of time and effort which seems unnecessary and could be a potential waste of time which we need in other areas. In terms of POC, still the traditional DB solves all our problems. Of course, if it’s absolutely a must to use Baserow to follow the philosophy of using the mentioned GovStack BBs then we can do it, but with the mentioned costs from above.

Additional note

There’s a possibility to run Baserow in a way that each technology inside of it can be run as a separate container. This opens potential capability of reusing them in a way it would suit us. For example, use the PostgreSQL and Redis directly as a standalone app within the Baserow. However, this is something that has to be tested and confirmed if it works this way.

investigate if running migration scripts is possible → No feasible due to too complex structure of the tables. In order to create one table with three columns and three values using conventional SQL language (which is used in migration files run with liquibase) it’s needed to cover over 15+ tables. Inserting one field in the table requires inserting data into more than 10 tables.

#5 Frontend technology of choice: React with Material UI

React, a popular JavaScript library for building user interfaces, is the chosen frontend technology for the demo. When paired with Material UI, a robust component library that implements Google’s Material Design, it provides a powerful framework for creating responsive and visually appealing applications.

Key Features

• Component-Based Architecture: React’s component-based structure allows developers to build encapsulated components that manage their own state, promoting reusability and maintainability.

• Declarative UI: React enables developers to describe how the UI should look based on the current application state, making it easier to understand and debug.

• Virtual DOM: This feature optimizes rendering by updating only the parts of the UI that have changed, enhancing performance and user experience.

Material UI Integration

• Pre-Built Components: Material UI offers a wide range of pre-designed components, such as buttons, forms, and navigation elements, which adhere to Material Design principles. This accelerates development and ensures consistency across the application.

• Customizability: While providing a solid design foundation, Material UI allows for extensive customization, enabling developers to tailor components to meet specific branding and functional requirements.

• Responsive Design: The library supports responsive layouts, ensuring that applications look great on various devices, from desktops to mobile phones.

#6 Backend technology of choice: Java Spring Boot

Java Spring Boot is the selected backend technology for the demo, chosen for its robustness, flexibility, and ability to streamline the development of microservices. It simplifies the process of building production-ready applications by providing a comprehensive framework that integrates various components seamlessly.

Java Spring Boot is an ideal choice for the backend of the GovStack architecture due to its ease of use, scalability, and strong community support. Its capabilities align well with the goals of the initiative, enabling the development of efficient, secure, and maintainable digital government services that can adapt to evolving needs.

Technical Documentation

Architecture diagram

Open

BPMN

Demo BPMN (v1.6, reflecting the final demo application):

Open

For reference, this was the originally envisioned BPMN (v1.1) for the demo:

GitHub repositories

The repositories can be accessed here:

How to up and run the demo locally

Prerequisites: node and yarn and docker

Backend services

(for each service)

1. up and run docker

2. create a new network in Docker using the command docker network create shared_network

3. up and run Information mediator using the docker file

4. clone/Download each service in a new folder (if downloaded, unzip the folder)

5. start each service by opening the service folder in a new terminal

6. run the command docker-compose up --build

Frontend

1. once backend services are running, clone the frontend repository

2. open a terminal and open on it the root folder of the frontend repository

3. copy the .env.example  to .env. This can be done with this command:

   • cp .env.example .env

4. verify that the values in .env are correct. They will be if they haven't modified anything while starting the microservices.

5. next step is to install the frontend dependencies. this can be done, again with the terminal and again in the root folder, by executing this command: yarn install

6. after all of this, they are ready to start the project. to do that they can, again in root folder and with the terminal, run this command: yarn dev

7. once they have ran that command, they can open their browser and open the url http://localhost:3000/

8. this will open the login page, default credentials are:

   • user: firstUser@test.com

   • password: asdf1234

Something similar is mentioned in frontend's readme: https://github.com/GovStackWorkingGroup/sandbox-usecase-earlywarning-frontend/blob/main/README.md

Port mapping

Database Schema

Threat Table

Broadcast Table