Skip to main content

Low Level Design

Introduction

About Drought-map Hub

The Eswatini DroughtMap Hub is a collaborative effort between the National Drought Management Center of Eswatini, the Ministry of Agriculture, and the Eswatini Meteorological Service. This platform is designed to provide a streamlined and user-friendly interface for validating, publishing, and exploring Combined Drought Indicator (CDI) products, enhancing the country's drought monitoring capabilities by enabling efficient data management and promoting public access to critical information.

The platform is structured into two key areas, catering to both administrative and public needs. The Admin Area is dedicated to authorized users who play a vital role in ensuring the accuracy and reliability of the CDI products. In this section, users can review and validate the outputs generated by CDI scripts, engage in collaborative discussions to assess data quality and address any inconsistencies, and approve and prepare validated datasets for public release. By maintaining robust data validation protocols, the Admin Area ensures that only high-quality, reliable information is made available to the public.

The Public Area serves as the platform’s primary interface for open access to CDI data. It is designed to be intuitive and engaging, allowing users of all backgrounds to explore comprehensive drought-related datasets, gain insights into drought trends and patterns, and utilize the data for research, planning, and decision-making purposes.

By bridging the gap between data generation and public accessibility, the Eswatini DroughtMap Hub empowers stakeholders, policymakers, and communities to make informed decisions for mitigating the impacts of droughts. This platform embodies a commitment to transparency, collaboration, and resilience in the face of environmental challenges.

The Purpose of Drought-map Hub

  • Automate CDI generation: Using external datasets, the platform automatically computes the CDI on a monthly basis, leveraging scripts from the National Drought Mitigation Center (NDMC).
  • Structure CDI processing and storage: It processes CDI data and stores the output, including GeoTIFF raster files and aggregated GeoJSON vector files, in a structured manner on a Geonode platform.
  • Create a web application for collaboration and publication: It offers a web application for authorized users to review, validate, and discuss the CDI map before publishing it for public access.

Functional Overview

This functional overview describes how the Drought-map hub will use GeoNode to manage and publish CDI Maps, focusing on preventing automatic publishing and access control. It also covers the CDI Map publication workflow.

Automation / Jobs

The system for managing the CDI Map dataset integrates automated processes with robust access controls to ensure accurate data generation and secure sharing. A cron job automates the monthly computation of CDI (Combined Drought Index) values by downloading external datasets, checking for updates, and running NDMC scripts to produce CDI raster files. These files are then aggregated to calculate CDI values for each inkhundla, generating a CDI vector map in GeoJSON format, which is uploaded to the GeoNode platform.

To maintain data integrity and control, automatic publishing of the CDI Map is disabled in GeoNode by setting the RESOURCE_PUBLISHING configuration to True. This ensures that the dataset undergoes a review process before being made publicly available. A dedicated user group, such as "Drought-map Hub Users," comprising reviewers and admins, is created to restrict access to the CDI Map dataset. Access permissions and the GROUP_PRIVATE_RESOURCES setting are applied to prevent visibility to unauthorized users. Additionally, metadata tags are used to identify and safeguard the dataset, ensuring it is used solely for its intended purpose.

For flexibility, Drought-map Hub Admins can manually trigger the cron job as needed, providing oversight over the automated process while maintaining secure data handling and publication.

Home Page

The sources describe several functionalities offered by the Drought-Map Hub Public Area:

  • Visitors can browse published maps and select specific maps based on the date. The interface will display the map, associated narrative, and a link to the relevant bulletin (if available).
  • To compare changes over time, visitors can use the compare published maps functionality to select two points in time. The two chosen maps will be displayed with a slider to control the overlay, facilitating comparison of CDI values.
  • Map download options are available in PNG and SVG formats. The sources also suggest a bonus feature for shapefile (.shp) download. Users can select elements like titles, legends, and annotations to customize downloaded maps for easier integration into knowledge products.
  • A feedback form allows visitors to share feedback or report bugs, with submissions sent directly to the admins.
  • An About Page provides information about the drought monitor, data sources, drought severity levels, contact information, and resources for drought monitoring and planning.

Data Approval

The Drought-map hub will use GeoNode's API to retrieve the CDI Map dataset. This API allows the hub to interact with the GeoNode instance and retrieve the necessary data. More information on using the GeoNode API can be found in the GeoNode documentation. 

The sources provide details on the CDI Map publication process within the Drought-map Hub, encompassing three key steps:

  1. Review: Reviewers, who are members of the Technical Working Group, are notified via email when a new CDI Map is available for review. They can access the map in the Drought-map hub Admin Area, review each inkhundla (administrative region), approve the computed CDI value, or suggest new values with justifications. Reviewers can also access the non-aggregated CDI Raster Map for additional context. Once the review is complete, reviewers finalize it, triggering an email notification to admins.
  2. Validation: Admins can start the validation step even before all reviewers have submitted their reviews. Admins access a table that summarizes reviewer suggestions for each inkhundla, making it easier for them to compare and select final CDI values.
  3. Publication: After validating the values, admins can publish the CDI Map, making it accessible to the public on the Drought-map hub Public Area. The publishing process involves adding a narrative to provide context to visitors, with the option to share the map on social media.

User Access Control

The platform has three user types:

  1. Visitors: Read-only access to published CDI maps and public area features.
  2. Reviewers: Access to review and provide feedback on computed CDI maps before publication.
  3. DroughtMap Hub Admins: Full access to manage the platform, users, and data publication processes.

Data Visualization and Maps

The Drought-map Hub offers various visualization and map-related features:

  • Interactive CDI Maps: Users can interact with published CDI maps, zoom in/out, and view CDI values for different inkhundlas.
  • Historical Map Comparison: Visitors can compare two published maps to observe changes in CDI values over time.
  • CDI Raster Map: Reviewers have access to the non-aggregated CDI Raster Map for additional context during the review process.
  • Map Download: Visitors can download published maps in PNG and SVG formats, with options for customization.

User Workflow

Account Creation

This workflow handles two account creation features for the Droughtmap hub:

  1. Admin Invitation
  2. Self-Registration via a Register Page

For both scenarios, once the Droughtmap backend successfully creates a user in its database, it also creates a corresponding GeoNode account using GeoNode's user API.

1. Admin Invitation

  1. Admin Action:

    • The admin logs into the Droughtmap Admin Panel and fills out a form to create a new user.
    • Fields include: email, role, and optionally a temporary password (auto-generated if not provided).

  2. Backend Actions:

    • Validate the input (e.g., check for duplicate email addresses).
    • Create a new user in the Droughtmap backend database.
    • Call the GeoNode API to create a corresponding user account.

  3. Email Notification:

    • Send an invitation email to the user with a link to set their password and log in.

2. Self-Registration via a Register Page

  1. User Action:

    • The user navigates to the Droughtmap Register Page.
    • They fill out a form with their email, password, and confirm password.

  2. Frontend Validation:

    • Ensure the passwords match and the email is valid.
    • Send the data to the Droughtmap backend.

  3. Backend Actions:

    • Validate the input server-side.
    • Check if the email is already registered.
    • Create the user in the Droughtmap backend database.
    • Call the GeoNode API to create a corresponding GeoNode account.
    • Return a success message.

  4. Email Notification:

    • Send a welcome email to the user confirming their registration.

Data Monitoring

  • Log Management: The cron job generates log files for each execution, recording details of the CDI computation process.
  • Error Management: The system handles errors during the automation process, preventing further execution for the affected month and sending notifications to relevant contacts.
  • Message Notifications: The system sends email notifications to Drought-map Hub Admins and Technical Support Contacts in case of missing data, successful automation, or errors during the process.

Data Forecasting

Architecture

Class Functions

Class Overview

Database Overview

Main Tables

  1. Users Table: Stores information about users, including their role (visitor, reviewer, admin), username, password, email address, and other relevant details.
  2. Administrations Table: Stores information about administrative regions in Eswatini, such as inkhundlas, districts, and regions, including their names, boundaries, and hierarchical relationships.
  3. Data Table: Stores the CDI data for each inkhundla and time period, including the computed CDI value, data source, and timestamp.
  4. Data Details Table: Stores additional details about the CDI data, such as the values of the input parameters used for the computation and any relevant metadata.
  5. Data Comments Table: Stores comments and feedback provided by reviewers and admins during the data approval process, linked to specific inkhundlas and time periods.

Relationship Diagrams

Sequence Diagrams

Data Flow Diagrams

High-Level Architecture Overview

1. Front-end (NextJS)

2. Back-end (Django)

3. Database (PostgreSQL)

4. Container Orchestration

5. CD/CI Pipeline

6. Jobs

Security and Stability Considerations

  • Secure User Authentication: Implement robust authentication mechanisms like JWT (JSON Web Token) to protect user accounts and sensitive information.
  • Role-Based Access Control: Restrict access to features and data based on user roles (visitor, reviewer, admin).
  • Secure Data Storage: Encrypt sensitive data at rest and in transit to protect against unauthorized access.
  • Data Validation and Sanitization: Validate and sanitize user inputs to prevent security vulnerabilities like SQL injection and cross-site scripting (XSS) attacks.
  • Regular Security Audits: Conduct periodic security audits to identify and address potential vulnerabilities.
  • Error Handling and Logging: Implement comprehensive error handling and logging mechanisms to track and resolve issues, improving system stability.

Database Design

1. Users Table

pos column null dtype len default






2. Administrations Table

pos column null dtype len default






3. Data Table

pos column null dtype len default






4. Data Details Table

pos column null dtype len default






5. Data Comments Table

pos column null dtype len default






API Design

1. User API

2. Data API

3. Comments API

4. Data Approval API

Data Storage

1. Static Maps

2. CDI Output

Error Handling

1. Application

1.1 Account Creation
  • Duplicate Email:
    • Check both the Droughtmap backend and GeoNode before creating the user.
    • Return a user-friendly error message if the email is already in use.
  • GeoNode API Failures:
    • If the GeoNode user creation fails, roll back the Droughtmap backend operation.
    • Log the error for further debugging.

2. CDI Jobs

Data Flow Diagrams

Testing

Deployment Strategy

Considerations

Design Considerations

  • Data Model: Carefully design the database schema to ensure efficient data storage and retrieval.
  • API Design: Create well-defined and documented API endpoints to enable seamless integration with other systems.
  • User Interface: Design a user-friendly and responsive interface that provides clear visualizations and intuitive interactions.

Performance Considerations

  • Optimize Data Processing: Use efficient algorithms and data structures to minimize processing time for CDI computation and data retrieval.
  • Caching: Implement caching mechanisms to reduce the load on the database and improve response times for frequently accessed data.
  • Load Balancing: Distribute traffic across multiple instances of the application to ensure high availability and responsiveness.

Swagger

Read the Docs

DBDocs

Back to top