Skip to content

Commit 4b5ec4b

Browse files
linconvidallinconvidal
authored
feat: implement stability testing with ApacheBench (#360)
- Added a new Python script for conducting stability tests on Cardano Rosetta API endpoints, measuring response times and validating against SLA thresholds. - Introduced a CSV file for test data input and detailed logging of test results. - Created comprehensive documentation in README.md outlining test setup, requirements, and output formats. - Generated markdown reports for detailed and summary results of the tests.
2 parents af8e313 + c56c2f6 commit 4b5ec4b

File tree

3 files changed

+786
-0
lines changed

3 files changed

+786
-0
lines changed

‎ab-tests/README.MD‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# Cardano Rosetta Stability Test
2+
3+
A tool for testing the stability and performance of Cardano Rosetta API endpoints under sustained load.
4+
5+
## Overview
6+
7+
This script performs stability testing of Cardano Rosetta API endpoints by:
8+
9+
- Running extended duration tests to verify API reliability under constant load
10+
- Monitoring endpoint stability across multiple concurrent connections
11+
- Measuring response times (p95, p99) and validating against SLA thresholds
12+
- Identifying potential degradation during sustained operation
13+
14+
## Requirements
15+
16+
- Python 3.6+
17+
- Apache Benchmark (`ab`) command-line tool
18+
- CSV file with test data in format: `address,block_index,block_hash,transaction_size,relative_ttl,transaction_hash`
19+
20+
## Hardware Specifications
21+
22+
The test records hardware specifications to provide context for performance results. To set your environment specs:
23+
24+
1. Edit the `MACHINE_SPECS` variable in `stability_test.py`:
25+
26+
```python
27+
MACHINE_SPECS = "16 cores, 16 threads, 125GB RAM, 3.9TB HDD, QEMU Virtual CPU v2.5+"
28+
```
29+
30+
2. Include relevant details that impact performance:
31+
- CPU (cores, threads, model)
32+
- RAM (total amount)
33+
- Storage type and size
34+
- Any virtualization environment
35+
36+
This information appears in test reports, allowing meaningful comparison between different hardware configurations.
37+
38+
### Getting Hardware Information in Ubuntu
39+
40+
To gather system specifications in Ubuntu, use these commands:
41+
42+
```bash
43+
# CPU info (cores, threads, model)
44+
lscpu
45+
46+
# RAM info
47+
free -h
48+
49+
# Storage info
50+
lsblk
51+
sudo hdparm -I /dev/sda | grep Model
52+
53+
# All-in-one detailed hardware report
54+
sudo apt install inxi
55+
inxi -Fxz
56+
```
57+
58+
For a GUI tool with comprehensive hardware information:
59+
60+
```bash
61+
sudo apt install hardinfo
62+
hardinfo
63+
```
64+
65+
## Running the Test
66+
67+
1. Configure the test parameters in the script header (or use defaults):
68+
69+
```python
70+
BASE_URL = "http://127.0.0.1:8082"
71+
CSV_FILE = "mainnet-data.csv"
72+
CONCURRENCIES = [1, 2, 4, 8, 12, 16, 24, 32]
73+
TEST_DURATION = 10 # seconds per concurrency level
74+
SLA_THRESHOLD = 1000 # ms
75+
...
76+
```
77+
78+
2. Execute the script:
79+
```bash
80+
./ab-tests/stability_test.py
81+
```
82+
83+
## Output
84+
85+
The script creates a timestamped directory containing:
86+
87+
1. **CSV Reports**:
88+
89+
- `details_results.csv`: Complete performance data for each endpoint/concurrency
90+
- `summary_results.csv`: Maximum concurrency achieved per endpoint
91+
92+
2. **Markdown Reports**:
93+
94+
- `details_results.md`: Formatted table of detailed results
95+
- `summary_results.md`: Formatted table of summary results
96+
97+
3. **Log Files**:
98+
- `ab_commands.log`: Record of all Apache Benchmark commands executed
99+
- JSON payload files for each endpoint
100+
101+
## Example Output
102+
103+
### Detailed Results (excerpt)
104+
105+
| Hardware | Machine Specs | Endpoint | Concurrency | p95 (ms) | p99 (ms) | Meets SLA | Complete Reqs | Reqs/sec | Mean Time (ms) |
106+
| ----------- | ------------------------------------------------ | ---------------- | ----------- | -------- | -------- | --------- | ------------- | -------- | -------------- |
107+
| entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /network/status | 1 | 1ms | 2ms | Yes | 9441 | 943.97 | 1.06ms |
108+
| entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /network/status | 2 | 1ms | 1ms | Yes | 20538 | 2053.68 | 0.49ms |
109+
| entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /network/status | 4 | 1ms | 1ms | Yes | 39249 | 3924.89 | 0.26ms |
110+
| entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /account/balance | 1 | 34ms | 40ms | Yes | 454 | 45.32 | 22.06ms |
111+
| entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /block | 4 | 120ms | 129ms | Yes | 361 | 36.02 | 27.76ms |
112+
113+
### Summary Results
114+
115+
| ID | Release | Hardware | Machine Specs | Endpoint | Max Concurrency | p95 (ms) | p99 (ms) |
116+
| --- | --------- | ----------- | ------------------------------------------------ | ---------------------- | --------------- | -------- | -------- |
117+
| 1 | 1.2.5-dev | entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /network/status | 4 | 1ms | 1ms |
118+
| 2 | 1.2.5-dev | entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /account/balance | 4 | 35ms | 39ms |
119+
| 3 | 1.2.5-dev | entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /account/coins | 4 | 33ms | 39ms |
120+
| 4 | 1.2.5-dev | entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /block | 4 | 120ms | 129ms |
121+
| 5 | 1.2.5-dev | entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /block/transaction | 4 | 84ms | 95ms |
122+
| 6 | 1.2.5-dev | entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /search/transactions | 4 | 4ms | 4ms |
123+
| 7 | 1.2.5-dev | entry_level | 4 cores, 8 threads, 32GB RAM, SSD, Intel Core i3 | /construction/metadata | 4 | 1ms | 1ms |

‎ab-tests/mainnet-data.csv‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
addr1qxw5ly68dml8ceg7eawa7we8pjw8j8hn74n2djt2upmnq9th42p6lrke4yj3e0xqg3sdqm6lzksa53wd2550vrpkedks4fttnm,11573705,8f4dcbb9633d885d7c9244222a7861cb745d89e486146a5209a08af1b0bbc1ef,296,1000,3a954835b69ca01ff9cf3b30ce385d5d9ef0cea502bd0f2ad156684dfbaf325a

0 commit comments

Comments
 (0)