Skip to main content

Low Level Design

Introduction

About Drought-map Hub

The Eswatini DroughtMap Hub is a collaborative effort between the National Drought Management Center of Eswatini, the Ministry of Agriculture, and the Eswatini Meteorological Service. This platform is designed to provide a streamlined and user-friendly interface for validating, publishing, and exploring Combined Drought Indicator (CDI) products, enhancing the country's drought monitoring capabilities by enabling efficient data management and promoting public access to critical information.

The platform is structured into two key areas, catering to both administrative and public needs. The Admin Area is dedicated to authorized users who play a vital role in ensuring the accuracy and reliability of the CDI products. In this section, users can review and validate the outputs generated by CDI scripts, engage in collaborative discussions to assess data quality and address any inconsistencies, and approve and prepare validated datasets for public release. By maintaining robust data validation protocols, the Admin Area ensures that only high-quality, reliable information is made available to the public.

The Public Area serves as the platform’s primary interface for open access to CDI data. It is designed to be intuitive and engaging, allowing users of all backgrounds to explore comprehensive drought-related datasets, gain insights into drought trends and patterns, and utilize the data for research, planning, and decision-making purposes.

By bridging the gap between data generation and public accessibility, the Eswatini DroughtMap Hub empowers stakeholders, policymakers, and communities to make informed decisions for mitigating the impacts of droughts. This platform embodies a commitment to transparency, collaboration, and resilience in the face of environmental challenges.

The Purpose of Drought-map Hub

  • Automate CDI generation: Using external datasets, the platform automatically computes the CDI on a monthly basis, leveraging scripts from the National Drought Mitigation Center (NDMC).
  • Structure CDI processing and storage: It processes CDI data and stores the output, including GeoTIFF raster files and aggregated GeoJSON vector files, in a structured manner on a Geonode platform.
  • Create a web application for collaboration and publication: It offers a web application for authorized users to review, validate, and discuss the CDI map before publishing it for public access.

Functional Overview

This functional overview describes how the Drought-map hub will use GeoNode to manage and publish CDI Maps, focusing on preventing automatic publishing and access control. It also covers the CDI Map publication workflow.

Automation / Jobs

The system for managing the CDI Map dataset integrates automated processes with robust access controls to ensure accurate data generation and secure sharing. A cron job automates the monthly computation of CDI (Combined Drought Index) values by downloading external datasets, checking for updates, and running NDMC scripts to produce CDI raster files. These files are then aggregated to calculate CDI values for each inkhundla, generating a CDI vector map in GeoJSON format, which is uploaded to the GeoNode platform.

For flexibility, Drought-map Hub Admins can manually trigger the cron job as needed, providing oversight over the automated process while maintaining secure data handling and publication.

Home Page

The sources describe several functionalities offered by the Drought-Map Hub Public Area:

  • Visitors can browse published maps and select specific maps based on the date. The interface will display the map, associated narrative, and a link to the relevant bulletin (if available).
  • To compare changes over time, visitors can use the compare published maps functionality to select two points in time. The two chosen maps will be displayed with a slider to control the overlay, facilitating comparison of CDI values.
  • Map download options are available in PNG and SVG formats. The sources also suggest a bonus feature for shapefile (.shp) download. Users can select elements like titles, legends, and annotations to customize downloaded maps for easier integration into knowledge products.
  • A feedback form allows visitors to share feedback or report bugs, with submissions sent directly to the admins.
  • An About Page provides information about the drought monitor, data sources, drought severity levels, contact information, and resources for drought monitoring and planning.

Data Approval

The Drought-map hub will use GeoNode's API to retrieve the CDI Map dataset. This API allows the hub to interact with the GeoNode instance and retrieve the necessary data. More information on using the GeoNode API can be found in the GeoNode documentation. 

The sources provide details on the CDI Map publication process within the Drought-map Hub, encompassing three key steps:

1. Review Step

Reviewers, who are members of the Technical Working Group, are notified via email when a new CDI Map is available for review. They can access the map in the Drought-map hub Admin Area, review each inkhundla (administrative region), approve the computed CDI value, or suggest new values with justifications. Reviewers can also access the non-aggregated CDI Raster Map for additional context. Once the review is complete, reviewers finalize it, triggering an email notification to admins.

2. Validation Step

The validation of a CDI Map is done based on the reviews submitted by the reviewers. The admin does not have to wait for all reviewers to send their reviews to start the validation step. Admins click a button to change the status of the Publication Process object to “In Review”. For a CDI Map not yet published, the admins see a table that provides them with an overview of the suggestions of reviews for each inkhundla. The table has the following format:

Inkhungla Initial Value Reviewer 1 Reviewer 2 Reviewer X Validated Value


Value + rationale

Value + rationale

All columns but the “Validated Value” are read-only. The admins have an easy way to pick the value from a reviewer or to input their value. They can optionally provide a comment/rationale.

3. Publication Step

After validating the values, admins can publish the CDI Map, making it accessible to the public on the Drought-map hub Public Area. The publishing process involves adding a narrative to provide context to visitors, with the option to share the map on social media.

User Access Control

The platform has three user types:

  1. Visitors: Read-only access to published CDI maps and public area features.
  2. Reviewers: Access to review and provide feedback on computed CDI maps before publication.
  3. DroughtMap Hub Admins: Full access to manage the platform, users, and data publication processes.

Data Visualization and Maps

The Drought-map Hub offers various visualization and map-related features:

  • Interactive CDI Maps: Users can interact with published CDI maps, zoom in/out, and view CDI values for different inkhundlas.
  • Historical Map Comparison: Visitors can compare two published maps to observe changes in CDI values over time.
  • CDI Raster Map: Reviewers have access to the non-aggregated CDI Raster Map for additional context during the review process.
  • Map Download: Visitors can download published maps in PNG and SVG formats, with options for customization.

User Workflow

Account Creation

  1. Admin Action:

    • The admin logs into the Droughtmap Admin Panel and fills out a form to create a new user.
    • Fields include: email, role, and optionally a temporary password (auto-generated if not provided).

  2. Backend Actions:

    • Validate the input (e.g., check for duplicate email addresses).
    • Create a new user in the Droughtmap backend database.

  3. Email Notification:

    • Send an invitation email to the user with a link to set their password and log in.

Data Review Process

Create A New CDI Map Publication

External Workflow:

  1. The admin receives a success notification email for the new CDI Map.
  2. The admin follows a link from the email to create a new publication on the platform.
    1. Session valid: Redirect to the publication list page.
    2. Session invalid: Redirect to the login page. 
  3. Done

Internal Workflow:

  1. Go to the publication list page.
  2. Create a new CDI Map publication.
  3. Get the latest CDI Map from Geonode.
  4. Add desired reviewers.
  5. Set a due date.
  6. Submit a new CDI Map publication.
  7. Done

CDI Map In Review Process

External workflow:

  1. The reviewer receives a review request email notification
  2. The reviewer follows a link to start review on the platform
    1. Session valid: Redirect to the review list page.
    2. Session invalid: Redirect to the login page.
  3. Done.

Internal Workflow:

  1. Go to the Review list page.
  2. Select the pending review item
  3. Review all initial values in the table for each inkhudla.
    1. Click approve all if all initial values do not need to change.
    2. Add suggestion value if the initial value needs adjustment and add a comment if it's necessary.
  4. Submit review.
  5. Done: All suggestions values are locked/read-only.

CDI Map In Validation Process

External workflow

  1. The admin receives all reviewers have been submitted
  2. The admin follows a link to start validation on the platform
    1. Session is valid: Redirect to the publication list page.
    2. Session is invalid: Redirect to the login page.
  3. Done

Internal workflow

  1. Go to the publication list page.
  2.  Select the In-Validation item.
  3. Review all suggestions & comments values from all reviewers in the table.
  4. Add a validated value for each inkhudla and add a comment if needed.
  5. Submit validated values.
  6. Done

CDI Map Publication Process

  1. Go to publication list page.
  2. Select the "Ready to publish" item
  3. Write a narrative text and add bulletin URL if available.
  4. Click Publish
  5. Share to available channels: social media or generated shareable URL.
  6. Done

Architecture

Class Functions

Class Overview

Database Overview

Main Tables

  1. Users Table: Stores information about users, including their role (visitor, reviewer, admin), username, password, email address, and other relevant details.
  2. Administrations Table: Stores information about administrative regions in Eswatini, such as inkhundlas, districts, and regions, including their names, boundaries, and hierarchical relationships.
  3. Data Table: Stores the CDI data for each inkhundla and time period, including the computed CDI value, data source, and timestamp.
  4. Data Details Table: Stores additional details about the CDI data, such as the values of the input parameters used for the computation and any relevant metadata.
  5. Reviews Table: Stores comments and feedback provided by reviewers and admins during the data approval process, linked to specific inkhundlas and time periods.

Relationship Diagrams

EswatiniDroughtMapHub-4.png

Source: https://drive.google.com/file/d/1utQVWFxQchvs0_D2lfy1951gwnIg6RlL/view?usp=drive_link

Sequence Diagrams

Data Flow Diagrams

eswatini-dataflow-2024-12-12-0728.png

Source: https://drive.google.com/file/d/1vNs_SCCV6mztkzBSEVuLHUOo78RbnpJr/view?usp=drive_link

Dependencies

Software Dependencies

The DroughtMap Hub platform incorporates various dependencies and frameworks to enable its functionality and deliver a seamless user experience. The following dependencies are essential components used in the development of the platform:

  1. Django: The DroughtMap Hub utilizes Django, a high-level Python web framework, to build the back-end infrastructure. Django provides a solid foundation for handling data management, authentication, and implementing business logic.
  2. Pandas: The platform relies on Pandas, a powerful data manipulation and analysis library in Python, to handle data processing tasks efficiently. Pandas enable tasks such as data filtering, transformation, and aggregation, enhancing the platform's data management capabilities.
  3. NextJS: The front end of the DroughtMap Hub is developed using NextJS, a popular ReactJS framework for building modern web applications. NextJS provides server-side rendering (SSR) and static site generation (SSG), ensuring optimal performance, faster load times, and better SEO. Its built-in routing and API capabilities streamline development, while React enables the creation of dynamic and interactive UI components, ensuring a responsive and engaging user experience.
  4. Tailwind CSS: The styling of the DroughtMap Hub leverages Tailwind CSS, a utility-first CSS framework that simplifies the process of designing responsive and consistent interfaces. Tailwind's modular approach allows developers to efficiently create visually appealing and highly customizable layouts without writing extensive custom CSS.
  5. Ant Design (antd): The platform utilizes Ant Design, a comprehensive UI library based on React, to design and implement a consistent and visually appealing user interface. Ant Design provides a rich set of customizable and reusable UI components, streamlining the development process.
  6. Leaflet: The platform incorporates Leaflet, a JavaScript library for interactive maps, to handle geospatial data visualization. Leaflet enables the integration of maps, markers, and other geospatial features, enhancing the platform's ability to represent and analyze location-based information.
  7. Echarts: Echarts, a powerful charting library, is integrated into the DroughtMap Hub to generate various data visualizations. With Echarts, the platform can display charts, graphs, and other visual representations of data, enabling users to gain insights and make informed decisions.

In addition to the previously mentioned dependencies, the DroughtMap Hub relies on the following essential dependencies and libraries to support its functionality and development process:

  1. Django Rest Framework (DRF): The DroughtMap Hub utilises Django Rest Framework, a powerful and flexible toolkit for building Web APIs. DRF simplifies the development of APIs within the platform, providing features such as request/response handling, authentication, serialisation, and validation. It enables seamless integration of RESTful API endpoints, allowing for efficient communication between the frontend and backend components.
  2. PyJWT: PyJWT is a Python library that enables the implementation of JSON Web Tokens (JWT) for secure user authentication and authorisation. The DroughtMap Hub utilises PyJWT to generate, validate, and manage JWT tokens. JWT tokens play a crucial role in ensuring secure user sessions, granting authorised access to specific functionalities and data within the platform.
  3. Sphinx: Generate comprehensive and user-friendly documentation. Sphinx facilitates the creation of structured documentation, including API references, code examples, and user guides. It streamlines the documentation process, making it easier for developers and users to understand and utilize the platform's features and functionalities.

3rd-Party Services

Mailjet

    • Mailjet is utilized for seamless email communication within the DroughtMap Hub.
    • It provides features such as email delivery, tracking, and management, ensuring reliable and efficient communication between system users.
    • Mailjet enables the platform to send notifications, reports, and other email-based communications to users, enhancing user engagement and system responsiveness.

Rundeck

Rundeck is utilized within the DroughtMap Hub platform to trigger and manage background jobs for CDI Automation. It provides a centralized interface for scheduling, executing, and monitoring automated tasks. By leveraging Rundeck's job orchestration capabilities, the platform ensures efficient execution of complex workflows, enabling seamless integration and reliable automation of CDI processes.

Geonode

Geonode serves as the spatial data infrastructure for the DroughtMap Hub platform. It is used to store CDI raster data, providing an organized and accessible repository for spatial datasets. Additionally, Geonode hosts aggregation results, which compute aggregated CDI values for each inkhundla, facilitating spatial analysis and visualization. This integration enables efficient management and sharing of geospatial information within the platform.

High-Level Architecture Overview

1. Front-end (NextJS)

The front-end serves as the user-facing layer, providing an intuitive interface for both public users and authorized administrators. Built with Next.js (powered by React), it enables dynamic routing, and server-side rendering (SSR) for improved performance and SEO.

Components:

  • Homepage: Interactive content showcasing published CDI maps and includes an About page and feedback form.
  • Choropleth Map: Displays CDI data visually on an interactive map for exploration and analysis.
  • User Authentication and Authorization: Secure user login and role-based access control.
  • CDI Map Management: Enables authorized users to review, validate, and publish CDI maps.
  • Users Management: Provides tools for administrators to manage user accounts.
  • Communication: The front-end will communicate with the back-end via RESTful APIs, and data will be fetched or submitted in JSON format.

2. Back-end (Django)

The back-end powers the core functionality of the DroughtMap Hub, handling user authentication, data validation workflows, and publishing processes. It serves as the intermediary between the front-end and the GeoNode API for managing and retrieving datasets.

Components:

  • API Layer: RESTful APIs for handling all interactions between the front-end and back-end.
  • CDI Map Management: Handling data validation, publishing workflows, and API interactions for authenticated users with appropriate roles.
  • Authentication and Authorization: Secure login and user roles (reviewers and admins) management.
  • Email Service: Handles sending of verification emails, invitation links, and notifications.
  • Communication: The back-end will interact with the database using Django’s ORM, ensuring efficient data retrieval and manipulation.

3. Database (PostgreSQL)

To store and manage all data related to users, validation statuses, publishing workflows, and application logs. PostgreSQL, a reliable and feature-rich relational database, is used for efficient data storage and management. GeoNode's database remains external and is accessed through its API for geospatial data and user synchronization.

4. Container Orchestration

Kubernetes will be used to deploy and manage the containerized services (Next.js and Django applications). It ensures the platform remains scalable, fault-tolerant, and highly available.

Key Components:

  • Pods: Each service (frontend, backend) will run inside its own containerized environment.
  • Load Balancing: Kubernetes will automatically manage load balancing between instances of the frontend and backend to ensure smooth performance under high traffic.
  • Auto-scaling: The system can automatically scale resources up or down based on traffic or load to optimize performance.
  • Secrets Management: Sensitive information like API keys, passwords, and database credentials will be securely managed using Kubernetes secrets.

5. CD/CI Pipeline

Purpose: Continuous integration and deployment will be automated using GitHub Workflows to ensure smooth development and deployment processes.

Workflow:

  • Automated Testing: Each code push or pull request will trigger tests to ensure code quality.
  • Build and Deploy: Successful tests will automatically trigger build processes and deploy to the appropriate environment (staging, production).
  • Environment Management: Separate environments for test, and production will be maintained to ensure stability in releases.

6. Jobs

Security and Stability Considerations

  • Secure User Authentication: Implement robust authentication mechanisms like JWT (JSON Web Token) to protect user accounts and sensitive information.
  • Role-Based Access Control: Restrict access to features and data based on user roles (visitor, reviewer, admin).
  • Secure Data Storage: Encrypt sensitive data at rest and in transit to protect against unauthorized access.
  • Data Validation and Sanitization: Validate and sanitize user inputs to prevent security vulnerabilities like SQL injection and cross-site scripting (XSS) attacks.
  • Regular Security Audits: Conduct periodic security audits to identify and address potential vulnerabilities.
  • Error Handling and Logging: Implement comprehensive error handling and logging mechanisms to track and resolve issues, improving system stability.

Database Design

1. Users Table

pos column null dtype len default
1 id No int
system_user_id_seq
2 name No character varying 100
3 email No character varying (unique) 255
4 password No character varying 128
6 created_at No timestamp
current_datetime
7 updated_at Yes timestamp

8 deleted_at Yes timestamp

2. Roles

pos column null dtype len default
1 id No int
roles_id_eq
2 name No character varying (unique) 100
3 description Yes character varying 255
4 created_at No timestamp
current_date
5 updated_at Yes timestamp

3. User Roles

pos column null dtype len default
1 user_id No int

2 role_id No int

3 assigned_at No timestamp
current_date

4. Administrations Table

pos column null dtype len default
1 id No int
administration_id_seq
2 code No character varying 50
3 name No character varying 50
5 admin_level Yes int

6 parent_id Yes int

7 created_at No timestamp
current_datetime
8 updated_at Yes timestamp

5. CDI Map Table

pos column null dtype len default
1 id No int
cdi_map_id_seq
2 geonode_id No int

3 month No character varying 10
4 geotiff No character varying 255
5 geojson No character varying 255
6 published_geojson Yes character varying 255
7 published_at Yes timestamp

8 status No int 5
9 narrative  Yes text

10 bulletin_url Yes character varying 255
11 created_at No timestamp
current_datetime
12 updated_at Yes timestamp

6. CDI Map Values Table

pos column null dtype len default
1 id No int
cdi_map_id_seq
2 cdi_map_id No int

3
 administration_id No
 int

4
 initial_value No decimal

5
 validated_value No decimal

6
 created_at No
 timestamp
current_datetime
7
 updated_at Yes timestamp

7. Review Assignments Table

pos column null dtype len default
1 id No int
review_assignments_id_seq
2 cdi_map_id No int

3 due_date No date

4 message No text

5 created_at No timestamp
current_datetime
6 updated_at Yes timestamp

8. Reviewers Table

pos column null dtype len default
1 id No int
reviewers_id_seq
2 assignment_id No int

3 user_id No int

4 status No int

5 created_at No timestamp
current_datetime
6 updated_at Yes timestamp

7 approved_at Yes timestamp

9. Review  Table

pos column null dtype len default
1 id No int
reviewers_id_seq
2 cdi_map_id No int

3 user_id No int

4 administration_id No int

5 value Yes decimal

6
 comment Yes character varying 255
7
 approved Yes boolean

8
 created_at No timestamp
current_datetime
9
 updated_at Yes timestamp

10. Feedback Table

pos column null dtype len default
1 id No int
feedbacks_id_seq
2 user_id Yes int

3 message No text

4 name No character varying 100
5 email No character varying 100
6 tags Yes JSON

7 status No

ENUM

  • "New"
  • "Reviewed"
  • "Resolved"
New
8 submitted_at No timestamp
current_datetime

11. Jobs Table

pos column null dtype len default
1 id No int
jobs_id_seq
2 task_id No character varying

3 user_id No int

4 type No int 5
5 status No int 5
6 attempt Yes int 5
7 result No JSON

8 info Yes character varying 255
9 created_at No timestamp

10 updated_at Yes timestamp

API Design

A. Classes & Utilities

A.1. Custom Pagination Class

Response Structure

{
    "current": <current page number>,
    "total": <total items count>,
    "total_page": <total number of pages>,
    "data": <paginated data>
}

Class Structure

from rest_framework.response import Response
from rest_framework.pagination import PageNumberPagination


class Pagination(PageNumberPagination):
    page_size = 10
    page_size_query_param = "page_size"
    max_page_size = 100

    def get_paginated_response(self, data):
        return Response(
            {
                "current": self.page.number,
                "total": self.page.paginator.count,
                "total_page": self.page.paginator.num_pages,
                "data": data,
            }
        )

    def get_paginated_response_schema(self, schema):
        return {
            "type": "object",
            "properties": {
                "current": {
                    "type": "integer",
                    "example": 123,
                },
                "total": {
                    "type": "integer",
                    "example": 123,
                },
                "total_page": {
                    "type": "integer",
                    "example": 123,
                },
                "data": schema,
            },
        }

1. User API

1.1. Create User (Sign Up)

Register a new user on the platform.

Endpoint: POST /api/v1/users/register/

Request Body:

{
    "email": "john.doe@example.com",
    "password": "Test#123",
    "confirm_password": "Test#123",
}

Response: 201 Created on success. A verification email is sent.

{
    "id": 1,
    "email": "john.doe@example.com",
    "created_at": "2024-10-22T18:06:11Z"
}

1.2. Verify User

Verify the user's email using the verification link.

Endpoint: POST /api/v1/users/verify?code={code}

Response: 200 OK if successful.

{
    "message": "Email verified successfully."
}

1.3. User Login

This API endpoint allows users to log in to the platform using their email and password credentials. Upon successful authentication, a session is established for the user.

Endpoint: POST /api/v1/users/login/ 

Request Body:

{
    "email": "john.doe@example.com",
    "password": "password123"
}

Response: 200 OK on success, with JWT token for session authentication.

{
    "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
    "expiration_time": "2024-10-22T18:06:11Z",
    "user": {
        "id": 1,
        "first_name": "John",
        "last_name": "",
        "email": "john.doe@example.com",
    }
}

1.4. User Forgot password

This API endpoint allows users to request a password change. Users can initiate the process by providing their registered email address, and a password reset link or instructions will be sent to them.

Endpoint: POST /api/v1/users/forgot-password/ 

Request Body:

{
    "email": "john.doe@example.com"
}

Response: 200 OK on success

1.5. Verify password code

This API endpoint verifies the token provided during the password reset process. It validates that the token belongs to the correct user who requested the password reset, ensuring the authenticity of the request before proceeding.

Endpoint: GET /api/v1/users/verify-password-code?token=secret

Response: 200 OK on success

{
    "message": "OK"
}

1.6. User Reset Password

This API endpoint allows users to update their old password with a new one. A valid token is required to authorize the password reset, ensuring the request is tied to the correct user.

Endpoint: POST /api/v1/users/reset-password?token=secret 

Request Body:

{
    "password": "NewPass123",
    "confirm_password": "NewPass123"
}

Response: 200 OK on success

{
   "message": "Password reset successfully"
}

2. Public API

2.1 List Map API

Endpoint: GET /api/v1/maps/

Request Params:

Name Value Description
start_month YYYY-MM Select start month
end_month YYYY-MM Select end month
compare [] Array of Map ID

Response: 200 OK on success

[
  "current": 1,
  "total_page": 5,
  "total": 99,
  "data": [
      {
        "id": 99,
        "geonode_id": 112,
        "month": "2024-12",
        "geojson": "/cdi/maps/published/2024-12-12.geojson",
        "published_at": "2024-12-12 08:10:10TZ",
      },
      ...
      {
        "id": 1,
        "geonode_id": 3,
        "month": "2024-01",
        "geojson": "/cdi/maps/published/2024-01-31.geojson",
        "published_at": "2024-01-31 12:10:10TZ",
      }
  ]
]

2.2 Details Map API

Endpoint: GET /api/v1/map/{id}

Response: 200 OK on success

{
  "id": 99,
  "geonode_id": 112,
  "month": "2024-12",
  "geojson": "/cdi/maps/published/2024-12-12.geojson",
  "published_at": "2024-12-12 08:10:10TZ",
  "narrative": "On the basis of the range of indicators monitored above there are no Tinkhundla in the Lubombo region that are in the alert drought phase signalling improvement from the previous month’s index presentation",
  "bulletin_url": "https://example.com/eswatini-drought-map-2024-12.pdf"
}

2.3 Download Map API

Endpoint: GET /api/v1/map/{id}/download

Response: 200 OK on success

{
  "file": "https://example-geonode.com/datasets/geonode:eswatini_sample_cdi/dataset_download",
  "other_format": [
    {
      "extension": "jpg",
      "link_type": "image",
      "name": "JPEG",
      "mime": "image/jpeg",
      "url": "https://example-geonode.com/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Aeswatini_sample_cdi&format=image%2Fjpeg&height=550&width=550&srs=EPSG%3A4326&bbox=30.8000000%2C-27.3000000%2C32.1000000%2C-25.7500000"
    },
    {
      "extension": "pdf",
      "link_type": "image",
      "name": "PDF",
      "mime": "application/pdf",
      "url": "https://example-geonode.com/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Aeswatini_sample_cdi&format=application%2Fpdf&height=550&width=550&srs=EPSG%3A4326&bbox=30.8000000%2C-27.3000000%2C32.1000000%2C-25.7500000"
    },
    {
      "extension": "png",
      "link_type": "image",
      "name": "PNG",
      "mime": "image/png",
      "url": "https://example-geonode.com/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Aeswatini_sample_cdi&format=image%2Fpng&height=550&width=550&srs=EPSG%3A4326&bbox=30.8000000%2C-27.3000000%2C32.1000000%2C-25.7500000"
    },
  ]
}

2.4 Sumit Feedback API

Endpoint: POST /api/v1/feedback/ 

Request Body:

{
    "message": "The Drought Map Hub website provides an incredibly user-friendly interface, allowing me to easily track and understand drought conditions across different regions, which has been invaluable for my research.",
    "name": "Alice",
    "email": "alice@wonderland.com"
    "tags": ["UI", "good", "User experience"]
}

Response: 200 OK on success

{
   "message": "Your feedback has been sent successfully."
}

3. Admin API

3.1 List CDI Map

Endpoint: GET /api/v1/admin/cdi-maps 

JWT Access: Admin

Request params:

Name Value Description
start_month YYYY-MM Select start month
end_month YYYY-MM Select end month
status

enum:

1 - Pending

2 - In Review

3 - In Validation

4 - Published

 CDI Map Status

Response: 200 OK on success

[
  "current": 1,
  "total_page": 10,
  "total": 100,
  "data": [
      {
        "id": 100,
        "geonode_id": 150,
        "month": "2025-01",
        "geotiff": "/cdi/geotiff/cdi_2025-01.tif",
        "geojson": "/cdi/maps/2025-01.geojson",
        "published_geojson": null,
        "published_at": null,
        "status": 1,
        "progress_review": "2/8"
      },
      ...
      {
        "id": 1,
        "geonode_id": 3,
        "month": "2024-01",
        "geojson": "/cdi/maps/2024-01.geojson",
        "published_geojson": "/cdi/maps/published/2024-01-31.geojson",
        "published_at": "2024-01-31 12:10:10TZ",
        "status": 4,
        "progress_review": 8/8"
      }
  ]
]

3.2 Details CDI Map for Admin

Endpoint: GET /api/v1/admin/cdi-map/{id} 

JWT Access: Admin

Response: 200 OK on success

{
  "id": 100,
  "geonode_id": 150,
  "month": "2025-01",
  "geotiff": "/cdi/geotiff/cdi_2025-01.tif",
  "geojson": "/cdi/maps/2025-01.geojson",
  "published_geojson": null,
  "published_at": null,
  "status": 1,
  "narrative": "",
  "bulletin_url": null,
  "reviewers": [
    {
      "id": 8,
      "user_id": 28,
      "name": "John Doe",
      "email": "john.doe@mail.com",
      "status": 1,
    },
    ...
    {
      "id": 1,
      "user_id": 21,
      "name": "Jane Doe",
      "email": "jane.doe@mail.com",
      "status": 2,
    },
  ]
}

3.3 Details CDI Map Values for Admin

Endpoint: GET /api/v1/admin/cdi-map/{id}/values 

JWT Access: Admin

Response: 200 OK on success

[
  {
    "id": 711,
    "administration_id": 339,
    "value": 0.68,
    "comment": null,
    "suggestions": [
      {
        "reviewer_id": 8,
        "value": 0.7,
        "comment: "string"
      },
      {
        "reviewer_id": 7,
        "value": 0.71,
        "comment: "string"
      },
      {
        "reviewer_id": 6,
        "value": 0.54,
        "comment: "string"
      },
      ...
      {
        "reviewer_id": 1,
        "value": 0.67,
        "comment: "string"
      },      
    ]
  },
  ...
  {
    "id": 691,
    "administration_id": 330,
    "value": 1,
    "comment": null,
    "suggestions": [
      {
        "reviewer_id": 8,
        "value": 0.9,
        "comment: "string"
      },
      {
        "reviewer_id": 7,
        "value": 2,
        "comment: "string"
      },
      {
        "reviewer_id": 6,
        "value": 1,
        "comment: "string"
      },
      ...
      {
        "reviewer_id": 1,
        "value": 0.81,
        "comment: "string"
      },      
    ]
  },
]

3.4 Send CDI Map Review Request

Endpoint: POST /api/v1/admin/cdi-map-review/ 

JWT Access: Admin

Request Body:

{
    "reviewers": [1,5,7,8,21],
    "message": "Dear <reviewer name>,\nThe CDI Map for the month of <YYYY-MM> is available for review [here](link to map).\n\nPlease submit your review by <YYYY-MM–DD>.",
    "due_date": "2025-01-31",
    "cdi_map_id": 100,
}

Response: 200 OK on success

{
   "message": "Your review request has been sent successfully."
}

3.5 Submit validated value

Endpoint: POST /api/v1/admin/cdi-map/{id}/validated 

JWT Access: Admin

Request Body:

{
  "cdi_map_id": 100,
  "values": [
    {
      "administration_id": 339,
      "value": 0.68,
      "comment": "Severe Drought"
    },
    {
      "administration_id": 338,
      "value": 0.2,
      "comment": "Exceptional Drought"
    },
    ...
    {
      "administration_id": 330,
      "value": 1,
      "comment": "This area is Normal or wet conditions"
    },
  ]
}

Response: 200 OK on success

{
  "id": 100,
  "geojson": "/cdi/map/2025-01-24.geojson",
  "status": 4,
  "values": [
    {
      "id": 439,
      "administration_id": 339,
      "value": 0.68,
      "comment": "Severe Drought"
    },
    {
      "id": 438,
      "administration_id": 338,
      "value": 0.2,
      "comment": "Exceptional Drought"
    },
    ...
    {
      "id": 430,
      "administration_id": 330,
      "value": 1,
      "comment": "This area is Normal or wet conditions"
    },
  ]
}

3.6 Publish CDI Map

Endpoint: POST /api/v1/admin/cdi-map/{id}/publish 

JWT Access: Admin

Request Body:

{
  "narrative": "<p>The 3-month SPI reflects <b>positive short and medium-term</b> moisture conditions which has a positive impact on <u>vegetation and surface temperature</u> hence promoting vegetation growth and also lowering the the near-surface air temperature.</p>",
  "bulletin_url": "https://example.com/eswatini/cdi-2025-01.pdf",
}

Response: 200 OK on success

{
  "id": 100,
  "geonode_id": 150,
  "month": "2025-01",
  "geotiff": "/cdi/geotiff/cdi_2025-01.tif",
  "geojson": "/cdi/maps/2025-01.geojson",
  "published_geojson": "/cdi/map/published/2025-01-29.geojson",
  "published_at": "2025-01-29 08:00:30TZ",
  "status": 4,
  "narrative": "<p>The 3-month SPI reflects <b>positive short and medium-term</b> moisture conditions which has a positive impact on <u>vegetation and surface temperature</u> hence promoting vegetation growth and also lowering the the near-surface air temperature.</p>",
  "bulletin_url": "https://example.com/eswatini/cdi-2025-01.pdf",
}

4. Reviewer API

4.1 List CDI Map

Endpoint: GET /api/v1/reviewer/cdi-maps 

JWT Access: Reviewer

Request params:

Name Value Description
start_month YYYY-MM Select start month
end_month YYYY-MM Select end month
status

enum:

1 - Pending

2 - In Review

3 - In Validation

4 - Published

 CDI Map Status

Response: 200 OK on success

[
  "current": 1,
  "total_page": 10,
  "total": 100,
  "data": [
      {
        "id": 61, 
        "cdi_map_id": 100,
        "month": "2025-01",
        "geotiff": "/cdi/geotiff/cdi_2025-01.tif",
        "geojson": "/cdi/maps/2025-01.geojson",
        "published_geojson": null,
        "published_at": null,
        "status": 1,
        "progress_review": "2/8"
      },
      ...
      {
        "id": 1,
        "cdi_map_id": 10,
        "month": "2024-01",
        "geotiff": "/cdi/geotiff/cdi_2024-01.tif",
        "geojson": "/cdi/maps/2024-01.geojson",
        "published_geojson": "/cdi/maps/published/2024-01-31.geojson",
        "published_at": "2024-01-31 12:10:10TZ",
        "status": 4,
        "progress_review": 8/8"
      }
  ]
]

4.2 Details CDI Map for Reviewer

Endpoint: GET /api/v1/reviewer/cdi-map/{id} 

JWT Access: Reviewer

Response: 200 OK on success

{
  "id": 100,
  "geonode_id": 150,
  "month": "2025-01",
  "geotiff": "/cdi/geotiff/cdi_2025-01.tif",
  "geojson": "/cdi/maps/2025-01.geojson",
  "status": 1,
  "updated_at": "2025-01-19 18:22:00TZ",
  "approved_at": null,
}

4.3 Details CDI Map values for Reviewer

Endpoint: GET /api/v1/reviewer/cdi-map/{id}/values 

JWT Access: Reviewer

Response: 200 OK on success

[
  {
    "id": 811,
    "administration_id": 339,
    "value": 0.68,
    "value_suggestion": 0.61,
    "comment": null,
    "approved": false
  },
  {
    "id": 810,
    "administration_id": 338,
    "value": 0.68,
    "value_suggestion": null,
    "comment": null,
    "approved": true
  }
  ...
  {
    "id": 891,
    "administration_id": 330,
    "value": 1,
    "value_suggestion": 0.98,
    "comment": "Abnormally Dry actually",
    "approved": false
  }
]

4.3 Submit value suggestion

Endpoint: POST /api/v1/reviewer/cdi-map/{id}/values 

JWT Access: Reviewer

Request Body:

{
  "administration_id": 339,
  "value_suggestion": 0.61,
  "comment": "This area is moderate drought"
}

Response: 201 Created on success

{
  "id": 811,
  "administration_id": 339,
  "value": 0.68,
  "value_suggestion": 0.61,
  "comment": "This area is Moderate Drought",
  "approved": false
}

4.4 Update value suggestion and approved status

Endpoint: PUT /api/v1/reviewer/cdi-map/{id}/value/{value_id} 

JWT Access: Reviewer

Request Body:

{
  "administration_id": 339,
  "value_suggestion": 0.7,
  "comment": "This area is slightly moderate drought",
  "approved": true
}

Response: 200 OK on success

{
  "id": 811,
  "administration_id": 339,
  "value": 0.68,
  "value_suggestion": 0.7,
  "comment": "This area is slightly moderate drought",
  "approved": true
}

4.5 Submit review (Finalize the review)

Endpoint: PUT /api/v1/reviewer/cdi-map/{id} 

JWT Access: Reviewer

Response: 200 OK on success

{
  "id": 61,
  "cdi_map_id": 100,
  "month": "2025-01",
  "geotiff": "/cdi/geotiff/cdi_2025-01.tif",
  "geojson": "/cdi/maps/2025-01.geojson",
  "status": 2,
  "updated_at": "2025-01-19 18:22:00TZ",
  "approved_at": "2025-01-20 10:31:01TZ",
}

Data Storage

1. Static Maps

2. CDI Output

Error Handling

1. Application

1.1 Account Creation
  • Duplicate Email:
    • Check both the Droughtmap backend and GeoNode before creating the user.
    • Return a user-friendly error message if the email is already in use.
  • GeoNode API Failures:
    • If the GeoNode user creation fails, roll back the Droughtmap backend operation.
    • Log the error for further debugging.

2. CDI Jobs

In case of an error at any stage of the automation process, the cron job for that month is not executed anymore.

Message Types

Message Type Message Detail
Data is Missing

  • Recipient: DroughtHub Admins

  • Subject: CDI-E Automation - YYYY-MM - Mission data

  • Body: 

    • Datetime of execution

    • “The following datasets have not yet been updated. Another attempt will be done in XX days.”

      • List the datasets for which the data is missing
Error

  • Recipient: Technical Support Contacts

  • Subject: CDI-E Automation - YYYY-MM - Execution Error

  • Body:

    • Datetime of execution

    • “The CDI automation has failed for month YYYY-MM. See the attached log file for details.”

Attachment: Log file

YYYY-MM refers to the month for which the CDI computation was attempted.

Testing Strategy

Testing Framework and Tools

The DroughtMap Hub employs a comprehensive testing strategy to ensure the reliability, functionality, and quality of both its back-end and front-end components. The testing strategy encompasses different levels of testing, including back-end testing with Django Test, front-end testing with Jest, and container network testing with HTTP (bash). Here is an overview of the testing strategy for the DroughtMap Hub:

Back-end Testing with Django Test

  • The back-end testing of the DroughtMap Hub is conducted using Django Test, a testing framework provided by Django.
  • Django Test enables the creation of test cases and test suites to evaluate the functionality and behavior of the back-end components.
  • Back-end testing focuses on unit tests, integration tests, and API tests to validate individual modules, their interactions, and the API endpoints.
  • Test cases cover various scenarios, including positive and negative input cases, edge cases, and boundary conditions to ensure robustness and accuracy.

Front-end Testing with Jest

  • The DroughtMap Hub's front-end testing is performed using Jest, a JavaScript testing framework widely used for testing React applications.
  • Jest facilitates the creation of unit tests, integration tests, and component tests to assess the behavior and functionality of the front-end components.
  • Front-end testing focuses on validating the UI components, user interactions, and the correctness of the application's logic and state management.
  • Test cases cover various scenarios, including rendering components, user actions, and expected outcomes to ensure the desired user experience and functionality.

Deployment Strategy

Considerations

Design Considerations

  • Data Model: Carefully design the database schema to ensure efficient data storage and retrieval.
  • API Design: Create well-defined and documented API endpoints to enable seamless integration with other systems.
  • User Interface: Design a user-friendly and responsive interface that provides clear visualizations and intuitive interactions.

Performance Considerations

  • Optimize Data Processing: Use efficient algorithms and data structures to minimize processing time for CDI computation and data retrieval.
  • Caching: Implement caching mechanisms to reduce the load on the database and improve response times for frequently accessed data.
  • Load Balancing: Distribute traffic across multiple instances of the application to ensure high availability and responsiveness.

Swagger

Purpose: Generate and host interactive API documentation for the backend.
Deployment Strategy:

  • Generation: Use the drf-yasg library (or an equivalent tool) to generate Swagger documentation from the Django REST Framework APIs.
  • Hosting:
    • Deploy Swagger UI as a part of the DroughtMap backend application.
    • Make the Swagger documentation accessible at a dedicated route (e.g., /api/docs).
  • Automation: Integrate Swagger generation as a build step in the CI/CD pipeline to ensure updated API documentation with every deployment.

Read the Docs

Purpose: Host comprehensive technical documentation for developers and administrators.
Deployment Strategy:

  • Content: Write documentation using Sphinx (reStructuredText or Markdown) for tutorials, setup guides, and system architecture overviews.
  • Integration:
    • Configure the readthedocs.yml file for automated builds and hosting.
    • Link Read the Docs with the project's GitHub repository to trigger builds on documentation updates.
  • URL: Deploy under a consistent subdomain (e.g., docs.droughtmaphub.org).

DBDocs

Purpose: Visualize and document the database schema for improved collaboration.
Deployment Strategy:

  • Schema Generation:
    • Use tools like DBML (Database Markup Language) or SQL export to define and generate database schema documentation.
    • Automate the schema generation in the CI/CD pipeline to ensure it stays in sync with the database.
  • Hosting:
    • Host the schema documentation using DBDocs (e.g., https://dbdocs.io/eswatini-droughtmap-hub).
    • Link the schema documentation in the Read the Docs and Swagger UI for centralized access.
Back to top