Skip to content

Commit 075855e

Browse files
JohnnyFFMJohnnyFFM
authored
Add PoCX Aggregator (#3)
POCX Aggregator
1 parent 99443d5 commit 075855e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+5081
-100
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,13 +199,15 @@ jobs:
199199
-p pocx_miner \
200200
-p pocx_plotter \
201201
-p pocx_verifier \
202+
-p pocx_aggregator \
202203
-p pocx_mockchain \
203204
$FEATURES_FLAG
204205
else
205206
cargo build --release --target ${{ matrix.target }} \
206207
-p pocx_miner \
207208
-p pocx_plotter \
208209
-p pocx_verifier \
210+
-p pocx_aggregator \
209211
-p pocx_mockchain \
210212
$FEATURES_FLAG
211213
fi
@@ -223,6 +225,7 @@ jobs:
223225
cp target/${{ matrix.target }}/release/pocx_miner artifacts/pocx_miner-${{ matrix.target }}${SUFFIX}
224226
cp target/${{ matrix.target }}/release/pocx_plotter artifacts/pocx_plotter-${{ matrix.target }}${SUFFIX}
225227
cp target/${{ matrix.target }}/release/pocx_verifier artifacts/pocx_verifier-${{ matrix.target }}${SUFFIX}
228+
cp target/${{ matrix.target }}/release/pocx_aggregator artifacts/pocx_aggregator-${{ matrix.target }}${SUFFIX}
226229
cp target/${{ matrix.target }}/release/pocx_mockchain artifacts/pocx_mockchain-${{ matrix.target }}${SUFFIX}
227230
chmod +x artifacts/*
228231
@@ -233,6 +236,7 @@ jobs:
233236
cp target/${{ matrix.target }}/release/pocx_miner.exe artifacts/pocx_miner-${{ matrix.target }}.exe
234237
cp target/${{ matrix.target }}/release/pocx_plotter.exe artifacts/pocx_plotter-${{ matrix.target }}.exe
235238
cp target/${{ matrix.target }}/release/pocx_verifier.exe artifacts/pocx_verifier-${{ matrix.target }}.exe
239+
cp target/${{ matrix.target }}/release/pocx_aggregator.exe artifacts/pocx_aggregator-${{ matrix.target }}.exe
236240
cp target/${{ matrix.target }}/release/pocx_mockchain.exe artifacts/pocx_mockchain-${{ matrix.target }}.exe
237241
shell: bash
238242

‎.github/workflows/release.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,13 +132,15 @@ jobs:
132132
-p pocx_miner \
133133
-p pocx_plotter \
134134
-p pocx_verifier \
135+
-p pocx_aggregator \
135136
-p pocx_mockchain \
136137
$FEATURES_FLAG
137138
else
138139
cargo build --release --target ${{ matrix.target }} \
139140
-p pocx_miner \
140141
-p pocx_plotter \
141142
-p pocx_verifier \
143+
-p pocx_aggregator \
142144
-p pocx_mockchain \
143145
$FEATURES_FLAG
144146
fi
@@ -168,9 +170,15 @@ jobs:
168170
cp target/${{ matrix.target }}/release/pocx_miner staging/
169171
cp target/${{ matrix.target }}/release/pocx_plotter staging/
170172
cp target/${{ matrix.target }}/release/pocx_verifier staging/
173+
cp target/${{ matrix.target }}/release/pocx_aggregator staging/
171174
cp target/${{ matrix.target }}/release/pocx_mockchain staging/
172175
chmod +x staging/*
173176
177+
# Copy config files
178+
cp pocx_miner/miner_config.yaml staging/
179+
cp pocx_aggregator/aggregator_config.yaml staging/
180+
cp pocx_mockchain/mockchain_config.toml staging/
181+
174182
# Create archive
175183
ARCHIVE_NAME="pocx-${VERSION}-${{ matrix.target }}${SUFFIX}.tar.gz"
176184
tar -czf "release-artifacts/${ARCHIVE_NAME}" -C staging .
@@ -194,8 +202,14 @@ jobs:
194202
Copy-Item target/${{ matrix.target }}/release/pocx_miner.exe staging/
195203
Copy-Item target/${{ matrix.target }}/release/pocx_plotter.exe staging/
196204
Copy-Item target/${{ matrix.target }}/release/pocx_verifier.exe staging/
205+
Copy-Item target/${{ matrix.target }}/release/pocx_aggregator.exe staging/
197206
Copy-Item target/${{ matrix.target }}/release/pocx_mockchain.exe staging/
198207
208+
# Copy config files
209+
Copy-Item pocx_miner/miner_config.yaml staging/
210+
Copy-Item pocx_aggregator/aggregator_config.yaml staging/
211+
Copy-Item pocx_mockchain/mockchain_config.toml staging/
212+
199213
# Create archive
200214
$ARCHIVE_NAME = "pocx-${VERSION}-${{ matrix.target }}.zip"
201215
Compress-Archive -Path staging/* -DestinationPath "release-artifacts/$ARCHIVE_NAME"
@@ -267,8 +281,14 @@ jobs:
267281
- `pocx_miner` - Mining client
268282
- `pocx_plotter` - Plot file generator
269283
- `pocx_verifier` - Plot verification tool
284+
- `pocx_aggregator` - Mining proxy/aggregator
270285
- `pocx_mockchain` - Mock blockchain for testing
271286
287+
**Configuration Files:**
288+
- `miner_config.yaml` - Miner configuration template
289+
- `aggregator_config.yaml` - Aggregator configuration template
290+
- `mockchain_config.toml` - Mockchain configuration template
291+
272292
**Installation:**
273293
```bash
274294
# Linux/macOS

‎Cargo.toml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
[workspace]
22
members = [
33
"pocx_address",
4-
"pocx_hashlib",
4+
"pocx_aggregator",
5+
"pocx_hashlib",
56
"pocx_miner",
67
"pocx_mockchain",
78
"pocx_plotfile",

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# PoCX - A neXt Generation Proof of Capacity Framework
22

3+
[![Release](https://img.shields.io/github/v/release/PoC-Consortium/pocx)](https://github.com/PoC-Consortium/pocx/releases)
34
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
45
[![Rust](https://img.shields.io/badge/rust-%3E%3D1.91.0--nightly-brightgreen.svg)](https://www.rust-lang.org)
56

@@ -55,6 +56,7 @@ cargo build --release
5556
| **pocx_plotfile** | Plot file I/O with memory-mapped operations |
5657
| **pocx_plotter** | High-performance plot file generator |
5758
| **pocx_miner** | Mining client supporting multiple chains |
59+
| **pocx_aggregator** | Mining proxy aggregating submissions from multiple miners |
5860
| **pocx_verifier** | Plot file integrity verification tool |
5961
| **pocx_protocol** | JSON-RPC 2.0 protocol implementation |
6062
| **pocx_mockchain** | Mock blockchain for testing |

‎pocx_aggregator/Cargo.toml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
[package]
2+
name = "pocx_aggregator"
3+
version = "1.0.0"
4+
license = "MIT"
5+
authors = ["Proof of Capacity Consortium"]
6+
description = "High-performance mining aggregator for PoCX protocol"
7+
keywords = ["poc", "cryptocurrency", "mining", "aggregator"]
8+
categories = ["cryptography", "network-programming"]
9+
readme = "README.md"
10+
edition = "2021"
11+
rust-version = "1.91.0"
12+
13+
[package.metadata.docs.rs]
14+
all-features = true
15+
rustdoc-args = ["--cfg", "docsrs"]
16+
17+
[[bin]]
18+
name = "pocx_aggregator"
19+
path = "src/main.rs"
20+
21+
[dependencies]
22+
# Protocol - workspace crate
23+
pocx_protocol = { path = "../pocx_protocol" }
24+
25+
# Async runtime
26+
tokio = { workspace = true }
27+
futures = { workspace = true }
28+
async-trait = { workspace = true }
29+
30+
# Web framework
31+
axum = { workspace = true }
32+
tower = { workspace = true }
33+
tower-http = { workspace = true }
34+
35+
# Serialization
36+
serde = { workspace = true }
37+
serde_json = { workspace = true }
38+
serde_yaml = { workspace = true }
39+
40+
# Logging
41+
log = { workspace = true }
42+
env_logger = "0.11"
43+
tracing = { workspace = true }
44+
45+
# Error handling
46+
thiserror = { workspace = true }
47+
48+
# CLI
49+
clap = { workspace = true }
50+
51+
# Database
52+
diesel = { workspace = true }
53+
diesel_migrations = { workspace = true }
54+
r2d2 = { workspace = true }
55+
56+
# Utilities
57+
uuid = { workspace = true }
58+
chrono = { workspace = true }
59+
url = { workspace = true }
60+
reqwest = { workspace = true }
61+
crossbeam-channel = { workspace = true }
62+
num-bigint = "0.4"
63+
num-traits = "0.2"
64+
bytesize = "2.1"
65+
66+
[dev-dependencies]
67+
tempfile = { workspace = true }
68+
mockito = { workspace = true }

‎pocx_aggregator/README.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# pocx_aggregator
2+
3+
High-performance mining aggregator for the PoCX protocol. Acts as a proxy between multiple miners and a single upstream pool or wallet, aggregating submissions and distributing mining information efficiently.
4+
5+
## Features
6+
7+
- **High Concurrency**: Async/await architecture using Tokio for handling thousands of simultaneous miner connections
8+
- **Protocol Compliant**: Full JSON-RPC 2.0 support via pocx_protocol
9+
- **Submission Filtering**: Intelligent per-account or global best submission filtering
10+
- **Statistics**: Real-time mining statistics and submission tracking
11+
- **Web Dashboard**: Optional HTTP dashboard for monitoring aggregator status
12+
- **Database**: SQLite-based submission tracking with configurable retention
13+
14+
## Architecture
15+
16+
The aggregator sits between miners and a single upstream pool/wallet:
17+
18+
```
19+
[Miners] <--JSON-RPC--> [Aggregator] <--JSON-RPC--> [Upstream Pool/Wallet]
20+
| |
21+
+-- miner1 / \
22+
+-- miner2 / \
23+
+-- miner3 / \
24+
+-- ... / \
25+
```
26+
27+
### Key Components
28+
29+
- **Miner Server**: Accepts connections from miners, serves mining info, accepts nonce submissions
30+
- **Upstream Client**: Connects to single upstream pool/wallet, fetches mining info, forwards filtered submissions
31+
- **Submission Queue**: Filters and queues submissions based on mode (Pool/Wallet)
32+
- **Cache**: Caches mining info to reduce upstream load
33+
- **Stats**: Tracks submissions, deadlines, and performance metrics
34+
- **Database**: Stores submission history for statistics and analysis
35+
36+
## Submission Modes
37+
38+
### Pool Mode (Default)
39+
- Tracks best submission **per account** for last 3 blocks
40+
- Forwards submissions with retry and exponential backoff
41+
- Ideal for aggregating submissions to mining pools
42+
43+
### Wallet Mode
44+
- Tracks **global best** submission across all accounts for last 3 blocks
45+
- No retry logic (fail fast)
46+
- Ideal for solo mining to local wallet
47+
48+
## Configuration
49+
50+
Create an `aggregator_config.yaml` file:
51+
52+
```yaml
53+
# Listen address for miner connections
54+
listen_address: "0.0.0.0:8080"
55+
56+
# Expected block time in seconds
57+
block_time_secs: 120
58+
59+
# Upstream pool or wallet
60+
upstream:
61+
name: "primary-pool"
62+
url: "http://pool.example.com:8080/pocx"
63+
# Optional authentication token
64+
# auth_token: "your_token_here"
65+
# Submission mode: pool (default) or wallet
66+
submission_mode: pool
67+
68+
# Cache settings
69+
cache:
70+
mining_info_ttl_secs: 5
71+
pool_timeout_secs: 30
72+
73+
# Database settings
74+
database:
75+
path: "aggregator.db"
76+
retention_days: 7 # 0 = keep forever
77+
78+
# Statistics dashboard (optional)
79+
dashboard:
80+
enabled: true
81+
listen_address: "0.0.0.0:8081"
82+
83+
# Logging
84+
logging:
85+
level: "info"
86+
file: "aggregator.log"
87+
```
88+
89+
## Build
90+
91+
```bash
92+
# Build from workspace root
93+
cargo build --release -p pocx_aggregator
94+
95+
# Or build in this directory
96+
cargo build --release
97+
```
98+
99+
## Run
100+
101+
```bash
102+
# From workspace root
103+
./target/release/pocx_aggregator -c aggregator_config.yaml
104+
105+
# Or with custom config path
106+
./target/release/pocx_aggregator --config /path/to/aggregator_config.yaml
107+
```
108+
109+
## Usage
110+
111+
Once running, point your miners to the aggregator:
112+
113+
```yaml
114+
# miner_config.yaml
115+
chains:
116+
- name: "aggregator"
117+
base_url: "http://localhost:8080"
118+
api_path: "/"
119+
accounts:
120+
- account: "your_account_id"
121+
```
122+
123+
## Development
124+
125+
```bash
126+
# Run tests
127+
cargo test -p pocx_aggregator
128+
129+
# Run with debug logging
130+
RUST_LOG=debug cargo run -p pocx_aggregator -- -c aggregator_config.yaml
131+
```
132+
133+
## License
134+
135+
MIT License - See [LICENSE](../LICENSE) for details.

‎pocx_aggregator/aggregator_config.yaml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# PoCX Aggregator Configuration
2+
3+
# Listen address for miner connections
4+
# Default: 0.0.0.0:8080
5+
listen_address: "0.0.0.0:8080"
6+
7+
# Expected block time in seconds (used for capacity estimation and database retention)
8+
# PoCX: 120, Burst: 240
9+
# Default: 120
10+
block_time_secs: 120
11+
12+
# Upstream pool or wallet configuration
13+
upstream:
14+
name: "primary_pool"
15+
url: "http://pool.example.com:8080/pocx"
16+
# Optional authentication token
17+
# auth_token: "your_secret_token"
18+
# Submission mode: "pool" (per-account best) or "wallet" (global best)
19+
# Default: pool
20+
submission_mode: pool
21+
22+
# Cache settings
23+
cache:
24+
# How long to cache mining info (in seconds)
25+
# Lower values = more upstream requests, higher values = stale data risk
26+
# Default: 5
27+
mining_info_ttl_secs: 5
28+
29+
# Upstream request timeout (in seconds)
30+
# Default: 30
31+
pool_timeout_secs: 30
32+
33+
# Database settings
34+
database:
35+
# Database file path
36+
# Default: aggregator.db
37+
path: "aggregator.db"
38+
39+
# Database retention period in days (0 = keep forever)
40+
# Submissions older than this will be automatically deleted
41+
# Default: 7 days
42+
# Example: 7 days at 120s blocks = 5,040 blocks retained
43+
retention_days: 7
44+
45+
# Dashboard settings (optional)
46+
dashboard:
47+
# Enable dashboard
48+
# Default: true
49+
enabled: true
50+
# Dashboard listen address
51+
# Default: 0.0.0.0:8081
52+
listen_address: "0.0.0.0:8081"
53+
54+
# Logging configuration
55+
logging:
56+
# Log level: trace, debug, info, warn, error
57+
# Default: info
58+
level: "info"
59+
# Log file path
60+
# Default: aggregator.log
61+
file: "aggregator.log"

‎pocx_aggregator/diesel.toml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[print_schema]
2+
file = "src/schema.rs"
3+
custom_type_derives = ["diesel::query_builder::QueryId"]
4+
5+
[migrations_directory]
6+
dir = "migrations"

‎pocx_aggregator/migrations/2025-01-01-000000_create_submissions/down.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
DROP TABLE IF EXISTS submissions;

0 commit comments

Comments
 (0)