A decentralized distributed system

Summary

This project implements a fully decentralized distributed system where multiple types of nodes collaborate to collect, verify, and use data across different sites. The system models a real-world scenario where devices collect sensor data, verify its accuracy, and produce predictions, while ensuring proper coordination and consistency between geographically separated nodes.

Why this project?

This project was an excellent opportunity to learn about distributed systems, election algorithms, snapshots, collision detection, and how to use logical clocks to ensure consistency between replicas. The core idea is that sensors can provide erroneous data, disrupting users. Thus, verification systems are integrated to update, slowly, each data point one after another. The goal is to observe the impact of verification parameters on user behaviors.

Features

The system uses a decentralized architecture without a centralized database, where each node maintains and updates its own local replica. To ensure replica consistency, nodes coordinate via a distributed queue that employs logical clocks to serialize all updates. For shared data management, nodes focus on a sliding window of the last 15 days of temperature readings. Peer-to-peer communication enables direct messaging between nodes to propagate updates and coordinate actions efficiently.