Skip to content

Commit cc21483

Browse files
docs
1 parent 699da2c commit cc21483

File tree

3 files changed

+108
-0
lines changed

3 files changed

+108
-0
lines changed

apps/framework-docs/llm-docs/python/table-setup.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -396,6 +396,59 @@ Available replicated engines:
396396
- `ReplicatedAggregatingMergeTreeEngine` - Replicated with aggregation
397397
- `ReplicatedSummingMergeTreeEngine` - Replicated with summation
398398

399+
### Cluster-Aware Replicated Tables
400+
401+
For multi-node ClickHouse deployments, you can specify a cluster name to use `ON CLUSTER` DDL operations:
402+
403+
```python
404+
from moose_lib import OlapTable, OlapConfig
405+
from moose_lib.blocks import ReplicatedMergeTreeEngine
406+
from pydantic import BaseModel
407+
from datetime import datetime
408+
409+
class ReplicatedData(BaseModel):
410+
id: str
411+
data: str
412+
timestamp: datetime
413+
414+
# Replicated table on a cluster
415+
clustered_table = OlapTable[ReplicatedData](
416+
"ClusteredTable",
417+
OlapConfig(
418+
order_by_fields=["id"],
419+
engine=ReplicatedMergeTreeEngine(),
420+
cluster="default" # References cluster from moose.config.toml
421+
)
422+
)
423+
```
424+
425+
**Configuration in `moose.config.toml`:**
426+
```toml
427+
[[clickhouse_config.clusters]]
428+
name = "default"
429+
```
430+
431+
**When to omit all parameters (recommended):**
432+
-**ClickHouse Cloud** - Platform manages replication automatically
433+
-**Local development** - Moose auto-injects params: `/clickhouse/tables/{database}/{shard}/{table_name}`
434+
-**Most production deployments** - Works out of the box
435+
436+
**When to use `cluster`:**
437+
- ✅ Multi-node self-managed ClickHouse with cluster configuration
438+
- ✅ Need `ON CLUSTER` DDL for distributed operations
439+
- ✅ Works without explicit `keeper_path`/`replica_name` parameters
440+
441+
**When to use explicit `keeper_path`/`replica_name`:**
442+
- ✅ Custom replication topology required
443+
- ✅ Advanced ZooKeeper/Keeper configuration
444+
- ✅ Specific self-managed deployment requirements
445+
446+
**Important:** Cannot specify both `cluster` and explicit `keeper_path`/`replica_name` - choose one approach.
447+
448+
**Local Development:** Moose configures cluster names to point to your local ClickHouse instance, letting you develop with `ON CLUSTER` DDL without running multiple nodes.
449+
450+
**Production:** Cluster names must match your ClickHouse `remote_servers` configuration.
451+
399452
### S3Queue Engine Tables
400453

401454
The S3Queue engine enables automatic processing of files from S3 buckets as they arrive.

apps/framework-docs/llm-docs/typescript/table-setup.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -229,8 +229,55 @@ export const ReplicatedDedup = new OlapTable<ReplicatedSchema>("ReplicatedDedup"
229229
**Note**: The `keeperPath` and `replicaName` parameters are optional:
230230
- **Self-managed ClickHouse**: Both parameters are required for configuring ZooKeeper/ClickHouse Keeper paths
231231
- **ClickHouse Cloud / Boreal**: Omit both parameters - the platform manages replication automatically
232+
233+
### Cluster-Aware Replicated Tables
234+
235+
For multi-node ClickHouse deployments, you can specify a cluster name to use `ON CLUSTER` DDL operations:
236+
237+
```typescript
238+
import { OlapTable, ClickHouseEngines, Key } from '@514labs/moose-lib';
239+
240+
interface ReplicatedSchema {
241+
id: Key<string>;
242+
data: string;
243+
timestamp: Date;
244+
}
245+
246+
// Replicated table on a cluster
247+
export const ClusteredTable = new OlapTable<ReplicatedSchema>("ClusteredTable", {
248+
engine: ClickHouseEngines.ReplicatedMergeTree,
249+
orderByFields: ["id"],
250+
cluster: "default" // References cluster from moose.config.toml
251+
});
232252
```
233253

254+
**Configuration in `moose.config.toml`:**
255+
```toml
256+
[[clickhouse_config.clusters]]
257+
name = "default"
258+
```
259+
260+
**When to omit all parameters (recommended):**
261+
-**ClickHouse Cloud** - Platform manages replication automatically
262+
-**Local development** - Moose auto-injects params: `/clickhouse/tables/{database}/{shard}/{table_name}`
263+
-**Most production deployments** - Works out of the box
264+
265+
**When to use `cluster`:**
266+
- ✅ Multi-node self-managed ClickHouse with cluster configuration
267+
- ✅ Need `ON CLUSTER` DDL for distributed operations
268+
- ✅ Works without explicit `keeperPath`/`replicaName` parameters
269+
270+
**When to use explicit `keeperPath`/`replicaName`:**
271+
- ✅ Custom replication topology required
272+
- ✅ Advanced ZooKeeper/Keeper configuration
273+
- ✅ Specific self-managed deployment requirements
274+
275+
**Important:** Cannot specify both `cluster` and explicit `keeperPath`/`replicaName` - choose one approach.
276+
277+
**Local Development:** Moose configures cluster names to point to your local ClickHouse instance, letting you develop with `ON CLUSTER` DDL without running multiple nodes.
278+
279+
**Production:** Cluster names must match your ClickHouse `remote_servers` configuration.
280+
234281
### S3Queue Engine Tables
235282

236283
The S3Queue engine allows you to automatically process files from S3 buckets as they arrive.

apps/framework-docs/src/pages/moose/configuration.mdx

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,14 @@ native_port = 9000
243243
# Optional list of additional databases to create on startup (Default: [])
244244
# additional_databases = ["analytics", "staging"]
245245

246+
# ClickHouse cluster configuration for replicated tables (optional)
247+
# Define clusters for use with ON CLUSTER DDL operations and distributed tables
248+
# In local dev, Moose creates single-node clusters. In production, names must match your ClickHouse remote_servers config.
249+
# [[clickhouse_config.clusters]]
250+
# name = "default"
251+
# [[clickhouse_config.clusters]]
252+
# name = "my_cluster"
253+
246254
# HTTP server configuration for local development
247255
[http_server_config]
248256
# Host to bind the webserver to (Default: "localhost")

0 commit comments

Comments
 (0)