📡🔗 iot-trace-chain

Hybrid architecture combining IoT, blockchain, and big data for secure traceability of waste data.

---

🧠 Overview

This project is part of a Master's Thesis and proposes a scalable architecture to ensure the traceability, integrity, and availability of data generated by IoT devices in waste management scenarios.

The solution combines:

•

🔧 IoT devices connected via MQTT

•

📦 Massive storage in Parquet format on a Data Lake (MinIO + Delta Lake)

•

🔐 Integrity verification using Merkle trees and blockchain (Ethereum/Hardhat)

•

📊 Optional visualization and analysis with DuckDB, Dremio, or other tools

---

🧱 Architecture

plaintext

1[IoT Device] --> MQTT --> Persistence Worker --> 
2    [Delta Lake (MinIO)] + [Blockchain (Merkle Root)]

---

🚀 Main Services (Docker Compose)

| Service | Description |

|----------------------|----------------------------------------------------|

| persistence-worker | Processes, validates, and stores IoT data |

| minio | S3-compatible storage backend for the Data Lake |

| hardhat-node | Local blockchain node (Ethereum) |

| mosquitto | MQTT broker receiving sensor messages |

| duckdb | Optional tools for data exploration and querying |

---

📦 Project Structure

text

1.
2├── persistence-worker/       # Main worker code (Python, Spark)
3├── blockchain/               # Smart contracts and Hardhat scripts
4├── mosquitto/                # MQTT broker configuration
5├── minio-mirror/             # Local copy for DuckDB querying
6├── docker-compose.yaml       # Service orchestration
7└── README.md                 # This file -> Project documentation

---

🛠 Technologies

•

Python 3.12

•

Apache Spark + PySpark

•

Delta Lake

•

MQTT (Eclipse Mosquitto)

•

Ethereum + Hardhat

•

MinIO (S3-compatible)

•

DuckDB / Dremio (optional)

•

Docker Compose

---

⚙️ Quick Start

🔧 Start environment:

bash

1docker compose up --build

📦 Run a manual service (e.g., MQTT Explorer):

bash

1docker compose --profile manual up mqtt-explorer

🐍 Run the worker manually:

bash

1cd persistence-worker
2poetry install
3poetry run poe start-local

If you want to run the worker with time of function execution, you can use the following command:

bash

1poetry run poe start-local-profiling

Additionally, if you want to send a test MQTT message, you can do so with the following command:

bash

1poetry run poe publish-message

---

📚 Documentation

TODO

📄 License

This project is licensed under the MIT License and developed as part of the Master's in Business Intelligence and Big Data in Secure Environments (UBu, ULe, UVa) in Spain.

---

👨‍🎓 Author

Javier Alonso-Núñez

LinkedIn • GitHub • ORCID