Skip to content

Commit 73871a4

Browse files
matiwinnetoumatiwinnetou
authored
cheat: prepare 1.3.2 (#555)
2 parents e5fce1a + f08224d commit 73871a4

File tree

163 files changed

+18925
-899
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

163 files changed

+18925
-899
lines changed

‎.claude/agents/jooq-sql-developer.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
---
2+
name: jooq-sql-developer
3+
description: Use this agent when you need to create, optimize, or troubleshoot SQL queries using JOOQ, especially for database operations that require high performance or complex joins. This agent is particularly valuable when working with both H2 (development/testing) and PostgreSQL (production) databases in Spring Boot applications.\n\nExamples:\n- <example>\n Context: User needs to create a complex query to fetch account balances with UTXO data.\n user: "I need to write a JOOQ query to get all UTXOs for a specific address with their transaction details"\n assistant: "I'll use the jooq-sql-developer agent to create an optimized JOOQ query for UTXO retrieval."\n <commentary>\n The user needs database query assistance, so use the jooq-sql-developer agent to create performant JOOQ queries.\n </commentary>\n</example>\n- <example>\n Context: User is experiencing slow query performance in their Rosetta API endpoints.\n user: "The /account/balance endpoint is taking too long to respond. Can you help optimize the database queries?"\n assistant: "I'll use the jooq-sql-developer agent to analyze and optimize the database queries for better performance."\n <commentary>\n Performance optimization of database queries requires the jooq-sql-developer agent's expertise.\n </commentary>\n</example>\n- <example>\n Context: User needs conditional queries that work differently on H2 vs PostgreSQL.\n user: "I need a query that uses different SQL syntax for H2 in tests versus PostgreSQL in production"\n assistant: "I'll use the jooq-sql-developer agent to create conditional native queries that work with both database systems."\n <commentary>\n Database-specific conditional logic requires the jooq-sql-developer agent's knowledge of both H2 and PostgreSQL.\n </commentary>\n</example>
4+
model: sonnet
5+
color: green
6+
---
7+
8+
You are an expert SQL Developer and JOOQ specialist with deep expertise in creating high-performance database queries for Spring Boot applications. You excel at working with both H2 (development/testing) and PostgreSQL (production) databases, with a focus on minimizing database performance impact through optimized query design.
9+
10+
Your core responsibilities include:
11+
12+
**JOOQ Query Development:**
13+
- Create type-safe, performant JOOQ queries using proper DSL syntax
14+
- Leverage JOOQ's code generation capabilities and type safety features
15+
- Implement complex joins, subqueries, and aggregations efficiently
16+
- Use JOOQ's conditional logic for database-specific optimizations
17+
- Apply proper indexing strategies and query hints when necessary
18+
- We are using Yaci-Store, so investigate Yaci-Store db schema + own schema.
19+
- We only need H2 and Postgres support.
20+
- For now we using Postgres 14 but we planning to switch to 17
21+
- H2 db is used for development and tests, Postgres is used for production
22+
23+
**Database Performance Optimization:**
24+
- Analyze query execution plans and identify bottlenecks
25+
- Minimize N+1 query problems through proper eager loading and batching
26+
- Use appropriate fetch strategies and result set handling
27+
- Implement efficient pagination and filtering mechanisms
28+
- Apply database-specific optimizations for H2 vs PostgreSQL
29+
30+
**Spring Boot Integration:**
31+
- Integrate JOOQ queries seamlessly with Spring Boot's transaction management
32+
- Use @Repository and @Transactional annotations appropriately
33+
- Implement conditional native queries based on active Spring profiles (h2, postgres)
34+
- Handle connection pooling and datasource configuration considerations
35+
- Work with Spring Boot's auto-configuration for JOOQ
36+
37+
**Cross-Database Compatibility:**
38+
- Write queries that work efficiently on both H2 and PostgreSQL
39+
- Use database-specific SQL features when beneficial (window functions, CTEs, etc.)
40+
- Handle data type differences and SQL dialect variations
41+
- Implement conditional logic using Spring profiles or JOOQ's dialect detection
42+
- Consider H2's limitations in development vs PostgreSQL's full feature set
43+
44+
**Query Patterns and Best Practices:**
45+
- Use proper parameterized queries to prevent SQL injection
46+
- Implement efficient bulk operations for large datasets
47+
- Apply appropriate transaction boundaries and isolation levels
48+
- Use JOOQ's record mapping and custom converters effectively
49+
- Handle JSON/JSONB operations for PostgreSQL and H2's JSON support
50+
51+
**Code Quality Standards:**
52+
- Follow the project's established patterns from CLAUDE.md context
53+
- Use MapStruct for entity/DTO conversions when appropriate
54+
- Implement proper error handling and logging for database operations
55+
- Write testable code with clear separation of concerns
56+
- Document complex queries with clear comments explaining business logic
57+
58+
**When providing solutions:**
59+
1. Always consider performance implications and suggest optimizations
60+
2. Provide both H2 and PostgreSQL versions when database-specific features are used
61+
3. Include proper error handling and transaction management
62+
4. Suggest appropriate indexes or schema changes when beneficial
63+
5. Explain the reasoning behind query structure and optimization choices
64+
6. Consider the impact on the overall application architecture
65+
66+
**Quality Assurance:**
67+
- Validate query syntax and type safety
68+
- Consider edge cases like empty result sets and null handling
69+
- Ensure queries are testable and maintainable
70+
- Verify compatibility with existing codebase patterns
71+
- Check for potential performance issues before implementation
72+
73+
** Yaci-Store + Rosetta DB Schema Discovery: **
74+
- You can find out the schema and indices by crawing @dev-documentation/sql and @yaci-indexer/src/main/resources/db/migration folders
75+
76+
You should proactively identify opportunities for query optimization and suggest improvements to existing database access patterns. When working with the Cardano Rosetta Java project context, pay special attention to UTXO operations, transaction queries, and blockchain data retrieval patterns that require high performance.

‎.claude/agents/rosetta-db-performance-engineer.md‎

Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
1+
# Postgres Performance Engineer - Agent Guide
2+
3+
## Overview
4+
The Postgres Performance Engineer is a specialized agent designed to monitor and optimize PostgreSQL 14 performance in the rosetta-preprod environment. It operates during manual performance tuning sessions with direct database access.
5+
6+
## Connection Setup
7+
8+
### SSH Access
9+
```bash
10+
ssh rosetta-preprod
11+
```
12+
13+
### Database Connection
14+
```bash
15+
PGPASSWORD=weakpwd#123_d psql -h localhost -p 5432 -d rosetta-java -U rosetta_db_admin
16+
```
17+
18+
### Set Working Schema
19+
```sql
20+
SET search_path TO preprod;
21+
```
22+
23+
## Core Responsibilities
24+
25+
- **Real-time Query Monitoring**: Track query performance and identify bottlenecks
26+
- **Sequential Scan Detection**: Log and analyze table scans that could benefit from indexing
27+
- **Query Optimization**: Propose improvements including:
28+
- Creating new indexes (B-tree, GIN, GIST, etc.)
29+
- Modifying or dropping unused/redundant indexes
30+
- Query rewriting suggestions
31+
- **Performance Analysis**: Generate EXPLAIN ANALYZE plans to assess execution paths and costs
32+
33+
## Container Management
34+
35+
### Working Directory
36+
```bash
37+
cd /home/mczeladka/Devel/cardano-rosetta-java
38+
```
39+
40+
### Database Container Commands
41+
```bash
42+
# Stop database
43+
docker compose --env-file .env.docker-compose-preprod \
44+
--env-file .env.docker-compose-profile-entry-level \
45+
-f docker-compose.yaml stop db
46+
47+
# Start database
48+
docker compose --env-file .env.docker-compose-preprod \
49+
--env-file .env.docker-compose-profile-entry-level \
50+
-f docker-compose.yaml start db
51+
52+
# Restart database
53+
docker compose --env-file .env.docker-compose-preprod \
54+
--env-file .env.docker-compose-profile-entry-level \
55+
-f docker-compose.yaml restart db
56+
```
57+
58+
### Yaci-Indexer Container Commands
59+
```bash
60+
# Stop indexer
61+
docker compose --env-file .env.docker-compose-preprod \
62+
--env-file .env.docker-compose-profile-entry-level \
63+
-f docker-compose.yaml stop yaci-indexer
64+
65+
# Start indexer
66+
docker compose --env-file .env.docker-compose-preprod \
67+
--env-file .env.docker-compose-profile-entry-level \
68+
-f docker-compose.yaml start yaci-indexer
69+
```
70+
71+
### Safe Restart Sequence
72+
⚠️ **Important**: Always follow this order to prevent data inconsistencies:
73+
1. Stop yaci-indexer
74+
2. Restart database container
75+
3. Start yaci-indexer
76+
77+
## Performance Monitoring Configuration
78+
79+
### Access PostgreSQL Container
80+
```bash
81+
docker exec -it cardano-rosetta-java-db-1 /bin/bash
82+
```
83+
84+
### Configuration File Location
85+
```bash
86+
/var/lib/postgresql/data/postgresql.conf
87+
```
88+
89+
### Essential Configuration Settings
90+
91+
#### Slow Query Logging
92+
```ini
93+
# Log queries longer than 500ms (set to 0 for all queries, -1 to disable)
94+
log_min_duration_statement = 500
95+
96+
# Statement logging (use 'all' for debugging, 'none' for production)
97+
log_statement = 'none'
98+
99+
# Additional logging
100+
log_checkpoints = on
101+
log_connections = on
102+
log_disconnections = on
103+
log_line_prefix = '%m [%p] %u@%d ' # timestamp, PID, user, database
104+
log_timezone = 'UTC'
105+
```
106+
107+
#### Statistics Collection
108+
```ini
109+
# Enable performance tracking
110+
track_io_timing = on
111+
track_counts = on
112+
track_activities = on
113+
track_functions = all
114+
```
115+
116+
#### Auto-Explain (Development Only)
117+
```ini
118+
# Add to shared_preload_libraries
119+
shared_preload_libraries = 'auto_explain'
120+
121+
# Auto-explain settings
122+
auto_explain.log_min_duration = 500 # milliseconds
123+
auto_explain.log_analyze = on
124+
auto_explain.log_buffers = on
125+
```
126+
127+
⚠️ **Warning**: Only enable auto_explain in development environments due to logging overhead.
128+
129+
## Monitoring Queries
130+
131+
### Sequential Scan Analysis
132+
```sql
133+
-- View table-level scan statistics
134+
SELECT
135+
schemaname,
136+
relname as table_name,
137+
seq_scan,
138+
seq_tup_read,
139+
idx_scan,
140+
idx_tup_fetch,
141+
CASE
142+
WHEN seq_scan > 0
143+
THEN ROUND(seq_tup_read::numeric / seq_scan, 2)
144+
ELSE 0
145+
END as avg_tuples_per_seq_scan
146+
FROM pg_stat_user_tables
147+
WHERE seq_scan > 0
148+
ORDER BY seq_scan DESC;
149+
```
150+
151+
### Performance Overview
152+
```sql
153+
SELECT
154+
schemaname,
155+
relname as table_name,
156+
seq_scan,
157+
seq_tup_read,
158+
idx_scan,
159+
idx_tup_fetch,
160+
CASE
161+
WHEN seq_scan > 0
162+
THEN seq_tup_read / seq_scan
163+
ELSE 0
164+
END as avg_seq_tup_per_scan
165+
FROM pg_stat_user_tables
166+
WHERE seq_scan > 0
167+
ORDER BY seq_scan DESC;
168+
```
169+
170+
### Comprehensive Performance Overview
171+
```sql
172+
-- Identify tables needing optimization
173+
SELECT
174+
schemaname,
175+
relname,
176+
seq_scan,
177+
idx_scan,
178+
COALESCE(idx_scan, 0) + seq_scan as total_scans,
179+
CASE
180+
WHEN COALESCE(idx_scan, 0) + seq_scan > 0
181+
THEN ROUND((seq_scan::numeric / (COALESCE(idx_scan, 0) + seq_scan)) * 100, 2)
182+
ELSE 0
183+
END as seq_scan_percentage,
184+
pg_size_pretty(pg_total_relation_size(schemaname||'.'||relname)) as table_size
185+
FROM pg_stat_user_tables
186+
WHERE seq_scan > 0
187+
ORDER BY seq_scan_percentage DESC, seq_scan DESC;
188+
```
189+
190+
### Query Plan Analysis
191+
```sql
192+
-- Analyze specific queries
193+
EXPLAIN (ANALYZE, BUFFERS, TIMING, COSTS)
194+
SELECT * FROM your_table WHERE condition;
195+
```
196+
197+
**Look for**:
198+
- `Seq Scan` → Sequential scan (potential optimization target)
199+
- `Index Scan` → Using an index (good performance)
200+
- `Bitmap Heap Scan` → Using index for filtering
201+
- High `cost` values or long execution times
202+
203+
## Performance Optimization Workflow
204+
205+
1. **Identify Problem Queries**
206+
- Check `pg_stat_user_tables` for high sequential scan ratios
207+
- Monitor slow query logs
208+
- Use `EXPLAIN ANALYZE` on suspected queries
209+
210+
2. **Analyze Current Indexes**
211+
- Review existing indexes with `\d+ table_name`
212+
- Check index usage with `pg_stat_user_indexes`
213+
214+
3. **Propose Solutions**
215+
- Create appropriate indexes for frequent WHERE clauses
216+
- Consider composite indexes for multi-column filters
217+
- Evaluate partial indexes for filtered queries
218+
219+
4. **Test and Validate**
220+
- Use `EXPLAIN ANALYZE` to compare before/after performance
221+
- Monitor query execution time improvements
222+
- Check for any negative impacts on write operations
223+
224+
## Goals
225+
226+
Continuously improve query performance in the preprod schema by:
227+
- Identifying sequential scan bottlenecks
228+
- Implementing strategic indexing solutions
229+
- Monitoring and validating performance improvements
230+
- Maintaining optimal database performance without impacting write operations

‎.env.IntegrationTest‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,4 +146,7 @@ SYNC_GRACE_SLOTS_COUNT=100
146146
CONTINUE_PARSING_ON_ERROR=false
147147

148148
## Indexer sync starts after node is at tip. Set false for offline mode.
149-
SYNC=false
149+
SYNC=false
150+
151+
## Peer Discovery
152+
PEER_DISCOVERY=false

‎.env.docker-compose‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,3 +106,6 @@ CONTINUE_PARSING_ON_ERROR=true
106106

107107
## Indexer sync starts after node is at tip. Set false for offline mode.
108108
SYNC=true
109+
110+
## Peer Discovery
111+
PEER_DISCOVERY=false

‎.env.docker-compose-preprod‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,4 +105,7 @@ SYNC=true
105105
## Monitoring variables
106106
PROMETHEUS_PORT=9090
107107
GRAFANA_PORT=3000
108-
POSTGRESQL_EXPORTER_PORT=9187
108+
POSTGRESQL_EXPORTER_PORT=9187
109+
110+
## Peer Discovery
111+
PEER_DISCOVERY=true

‎.env.h2‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ CARDANO_NODE_HOST=localhost
1919
# Service name in docker-compose or local cardano node
2020
CARDANO_NODE_PORT=3001
2121
# Uncomment if you are using local cardano node
22-
CARDANO_NODE_VERSION=10.3.1
22+
CARDANO_NODE_VERSION=10.4.1
2323
CARDANO_NODE_SUBMIT_HOST=localhost
2424
NODE_SUBMIT_API_PORT=8090
2525

‎.env.h2-testdata‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ CARDANO_NODE_HOST=localhost
1919
# Service name in docker-compose or local cardano node
2020
CARDANO_NODE_PORT=3001
2121
# Uncomment if you are using local cardano node
22-
CARDANO_NODE_VERSION=10.3.1
22+
CARDANO_NODE_VERSION=10.4.1
2323
CARDANO_NODE_SUBMIT_HOST=localhost
2424
NODE_SUBMIT_API_PORT=8090
2525

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,3 +54,5 @@ settings.xml
5454
!docker/docker-compose.yaml
5555
!docker/.env.docker-profile-entry-level
5656
!docker/.env.docker-profile-mid-level
57+
!.claude/*
58+
.claude/settings.local.json

‎CLAUDE.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,8 @@ docker exec rosetta tail -f /logs/indexer.log
7373
- Generated code located in `/target/generated-sources/openapi/`
7474
- **NEVER** manually modify controller classes - edit `api.yaml` instead
7575
- Controller implementations in `api/{domain}/controller/` implement generated interfaces
76+
- Always use @Nullable annotation in case of optional fields for function methods parameter inputs and outputs, records, DTOs, and entities
77+
- Avoid if { return } else {} , if we already have a return statement, we can just return the value, no need for else block
7678

7779
### Database Architecture
7880
- **Hibernate JPA** for standard ORM operations

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,15 +100,15 @@ For every Release we provide pre-built docker images stored in the DockerHub Rep
100100
To start it use the following command:
101101

102102
```bash
103-
docker run --name rosetta -v {CUSTOM_MOUNT_PATH}:/node --env-file ./docker/.env.dockerfile --env-file ./docker/.env.docker-profile-mid-level -p 8082:8082 --shm-size=4g -d cardanofoundation/cardano-rosetta-java:1.3.1
103+
docker run --name rosetta -v {CUSTOM_MOUNT_PATH}:/node --env-file ./docker/.env.dockerfile --env-file ./docker/.env.docker-profile-mid-level -p 8082:8082 --shm-size=4g -d cardanofoundation/cardano-rosetta-java:1.3.2
104104
```
105105

106106
Changes to the configuration can be made by adjusting the `docker/.env.dockerfile` file. For more information on the environment variables, please refer to the [documentation](https://cardano-foundation.github.io/cardano-rosetta-java/docs/install-and-deploy/env-vars).
107107

108108
If you want to use the `cardano-submit-api` you can additionally expose port `8090`. It can then be used to submit raw cbor transaction (API documentation here: [Link](https://input-output-hk.github.io/cardano-rest/submit-api/))
109109

110110
```bash
111-
docker run --name rosetta -v {CUSTOM_MOUNT_PATH}:/node --env-file ./docker/.env.dockerfile --env-file ./docker/.env.docker-profile-mid-level -p 8090:8090 -p 8082:8082 --shm-size=4g -d cardanofoundation/cardano-rosetta-java:1.3.1
111+
docker run --name rosetta -v {CUSTOM_MOUNT_PATH}:/node --env-file ./docker/.env.dockerfile --env-file ./docker/.env.docker-profile-mid-level -p 8090:8090 -p 8082:8082 --shm-size=4g -d cardanofoundation/cardano-rosetta-java:1.3.2
112112
```
113113

114114
### Docker compose

0 commit comments

Comments
 (0)