Skip to content

Commit fcf6012

Browse files
committed
Feature: Add multi-config
1 parent 0b189c3 commit fcf6012

16 files changed

+872
-27
lines changed

README.md

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1373,6 +1373,224 @@ Listed below are all configuration options.
13731373
corresponding table in DynamoDB at model persisting if the table
13741374
doesn't exist yet. Default is `true`
13751375

1376+
## Multi-Configuration Support
1377+
1378+
Dynamoid supports multiple configurations to connect to different DynamoDB instances
1379+
across multiple AWS accounts or regions. This is useful when you need to:
1380+
1381+
- Connect to DynamoDB tables in different AWS accounts
1382+
- Use different regions for different models
1383+
- Separate production/staging data across different AWS setups
1384+
- Implement cross-account data access patterns
1385+
1386+
### Setting up Multiple Configurations
1387+
1388+
Configure multiple DynamoDB connections in your application initializer:
1389+
1390+
```ruby
1391+
# config/initializers/dynamoid.rb
1392+
Dynamoid.multi_configure do |config|
1393+
# Primary configuration (e.g., main application data)
1394+
config.add_config(:primary) do |c|
1395+
c.access_key = ENV['PRIMARY_AWS_ACCESS_KEY']
1396+
c.secret_key = ENV['PRIMARY_AWS_SECRET_KEY']
1397+
c.region = 'us-east-1'
1398+
c.namespace = 'myapp_primary'
1399+
end
1400+
1401+
# Secondary configuration (e.g., analytics data)
1402+
config.add_config(:analytics) do |c|
1403+
c.access_key = ENV['ANALYTICS_AWS_ACCESS_KEY']
1404+
c.secret_key = ENV['ANALYTICS_AWS_SECRET_KEY']
1405+
c.region = 'us-west-2'
1406+
c.namespace = 'myapp_analytics'
1407+
end
1408+
1409+
# Cross-account configuration (e.g., partner data)
1410+
config.add_config(:partner) do |c|
1411+
c.credentials = Aws::AssumeRoleCredentials.new(
1412+
role_arn: ENV['PARTNER_ROLE_ARN'],
1413+
role_session_name: 'dynamoid-cross-account'
1414+
)
1415+
c.region = 'eu-west-1'
1416+
c.namespace = 'partner_shared'
1417+
end
1418+
end
1419+
```
1420+
1421+
### Using Multiple Configurations in Models
1422+
1423+
Specify which configuration a model should use with the `dynamoid_config` method:
1424+
1425+
```ruby
1426+
# Models using primary configuration
1427+
class User
1428+
include Dynamoid::Document
1429+
1430+
dynamoid_config :primary
1431+
1432+
field :name, :string
1433+
field :email, :string
1434+
1435+
has_many :orders
1436+
end
1437+
1438+
class Order
1439+
include Dynamoid::Document
1440+
1441+
dynamoid_config :primary
1442+
1443+
field :total, :number
1444+
field :status, :string
1445+
1446+
belongs_to :user
1447+
end
1448+
1449+
# Models using analytics configuration
1450+
class PageView
1451+
include Dynamoid::Document
1452+
1453+
dynamoid_config :analytics
1454+
1455+
field :url, :string
1456+
field :user_id, :string
1457+
field :timestamp, :datetime
1458+
1459+
global_secondary_index hash_key: :user_id, range_key: :timestamp
1460+
end
1461+
1462+
class Report
1463+
include Dynamoid::Document
1464+
1465+
dynamoid_config :analytics
1466+
1467+
field :name, :string
1468+
field :data, :serialized
1469+
field :generated_at, :datetime
1470+
end
1471+
1472+
# Models using partner configuration
1473+
class SharedData
1474+
include Dynamoid::Document
1475+
1476+
dynamoid_config :partner
1477+
1478+
field :partner_id, :string
1479+
field :content, :serialized
1480+
field :sync_status, :string
1481+
end
1482+
1483+
# Models using default configuration (fallback to main Dynamoid.configure)
1484+
class SystemLog
1485+
include Dynamoid::Document
1486+
1487+
# No dynamoid_config specified - uses default configuration
1488+
1489+
field :level, :string
1490+
field :message, :string
1491+
field :timestamp, :datetime
1492+
end
1493+
```
1494+
1495+
### Configuration Inheritance
1496+
1497+
Models automatically inherit the correct configuration for all operations:
1498+
1499+
- **Table operations**: `create_table`, `delete_table`
1500+
- **CRUD operations**: `create`, `save`, `update`, `delete`, `find`
1501+
- **Queries**: `where`, `all`, `first`, `last`
1502+
- **Batch operations**: `import`, `batch_write`
1503+
- **Scanning**: `scan`
1504+
1505+
```ruby
1506+
# Each model uses its own DynamoDB connection
1507+
User.create(name: "John", email: "john@example.com") # Uses :primary config
1508+
PageView.create(url: "/home", user_id: "123") # Uses :analytics config
1509+
SharedData.create(partner_id: "partner1", content: {}) # Uses :partner config
1510+
SystemLog.create(level: "info", message: "App started") # Uses default config
1511+
```
1512+
1513+
### Table Names and Namespaces
1514+
1515+
Each configuration can have its own namespace, resulting in different table prefixes:
1516+
1517+
```ruby
1518+
# With the configurations above:
1519+
User.table_name # => "myapp_primary_users"
1520+
PageView.table_name # => "myapp_analytics_page_views"
1521+
SharedData.table_name # => "partner_shared_shared_data"
1522+
SystemLog.table_name # => "dynamoid_system_logs" (uses default namespace)
1523+
```
1524+
1525+
### Configuration Management
1526+
1527+
```ruby
1528+
# List all configured names
1529+
Dynamoid::MultiConfig.configuration_names
1530+
# => [:primary, :analytics, :partner]
1531+
1532+
# Check if a configuration exists
1533+
Dynamoid::MultiConfig.configuration_exists?(:primary)
1534+
# => true
1535+
1536+
# Get a specific configuration
1537+
config = Dynamoid::MultiConfig.get_config(:primary)
1538+
config.region # => "us-east-1"
1539+
1540+
# Remove a configuration
1541+
Dynamoid::MultiConfig.remove_config(:analytics)
1542+
1543+
# Clear all configurations
1544+
Dynamoid::MultiConfig.clear_all
1545+
```
1546+
1547+
### Error Handling
1548+
1549+
If you specify a non-existent configuration, Dynamoid will raise an error:
1550+
1551+
```ruby
1552+
class InvalidModel
1553+
include Dynamoid::Document
1554+
1555+
dynamoid_config :nonexistent # This configuration doesn't exist
1556+
1557+
field :name, :string
1558+
end
1559+
1560+
InvalidModel.create(name: "test")
1561+
# => Dynamoid::Errors::UnknownConfiguration: Unknown configuration: nonexistent
1562+
```
1563+
1564+
### Best Practices
1565+
1566+
1. **Environment-based configuration**: Use environment variables for sensitive credentials
1567+
2. **Logical separation**: Group related models in the same configuration
1568+
3. **Namespace isolation**: Use distinct namespaces to avoid table name conflicts
1569+
4. **Role-based access**: Use IAM roles for cross-account access when possible
1570+
5. **Connection reuse**: Configurations create connection pools, so reuse them efficiently
1571+
1572+
```ruby
1573+
# Example: Environment-based setup
1574+
Dynamoid.multi_configure do |config|
1575+
# Production data
1576+
config.add_config(:production) do |c|
1577+
c.credentials = Aws::InstanceProfileCredentials.new
1578+
c.region = ENV.fetch('PRODUCTION_REGION', 'us-east-1')
1579+
c.namespace = "#{Rails.application.class.module_parent_name.downcase}_prod"
1580+
end
1581+
1582+
# Analytics warehouse
1583+
config.add_config(:warehouse) do |c|
1584+
c.credentials = Aws::AssumeRoleCredentials.new(
1585+
role_arn: ENV['WAREHOUSE_ROLE_ARN'],
1586+
role_session_name: "#{Rails.application.class.module_parent_name.downcase}-warehouse"
1587+
)
1588+
c.region = ENV.fetch('WAREHOUSE_REGION', 'us-west-2')
1589+
c.namespace = "warehouse_#{Rails.env}"
1590+
end
1591+
end
1592+
```
1593+
13761594

13771595
## Concurrency
13781596

lib/dynamoid.rb

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
require 'dynamoid/components'
3636
require 'dynamoid/document'
3737
require 'dynamoid/adapter'
38+
require 'dynamoid/multi_config'
3839
require 'dynamoid/transaction_write'
3940

4041
require 'dynamoid/tasks/database'
@@ -51,6 +52,10 @@ def configure
5152
end
5253
alias config configure
5354

55+
def multi_configure(&block)
56+
Dynamoid::MultiConfig.configure(&block)
57+
end
58+
5459
def logger
5560
Dynamoid::Config.logger
5661
end

lib/dynamoid/criteria/chain.rb

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -233,18 +233,18 @@ def delete_all
233233
ranges = []
234234

235235
if @key_fields_detector.key_present?
236-
Dynamoid.adapter.query(source.table_name, query_key_conditions, query_non_key_conditions, query_options).flat_map { |i| i }.collect do |hash|
236+
source.adapter.query(source.table_name, query_key_conditions, query_non_key_conditions, query_options).flat_map { |i| i }.collect do |hash|
237237
ids << hash[source.hash_key.to_sym]
238238
ranges << hash[source.range_key.to_sym] if source.range_key
239239
end
240240
else
241-
Dynamoid.adapter.scan(source.table_name, scan_conditions, scan_options).flat_map { |i| i }.collect do |hash|
241+
source.adapter.scan(source.table_name, scan_conditions, scan_options).flat_map { |i| i }.collect do |hash|
242242
ids << hash[source.hash_key.to_sym]
243243
ranges << hash[source.range_key.to_sym] if source.range_key
244244
end
245245
end
246246

247-
Dynamoid.adapter.delete(source.table_name, ids, range_key: ranges.presence)
247+
source.adapter.delete(source.table_name, ids, range_key: ranges.presence)
248248
end
249249
alias destroy_all delete_all
250250

@@ -575,7 +575,7 @@ def raw_pages
575575
# @since 3.1.0
576576
def raw_pages_via_query
577577
Enumerator.new do |y|
578-
Dynamoid.adapter.query(source.table_name, query_key_conditions, query_non_key_conditions, query_options).each do |items, metadata|
578+
source.adapter.query(source.table_name, query_key_conditions, query_non_key_conditions, query_options).each do |items, metadata|
579579
options = metadata.slice(:last_evaluated_key)
580580

581581
y.yield items, options
@@ -590,7 +590,7 @@ def raw_pages_via_query
590590
# @since 3.1.0
591591
def raw_pages_via_scan
592592
Enumerator.new do |y|
593-
Dynamoid.adapter.scan(source.table_name, scan_conditions, scan_options).each do |items, metadata|
593+
source.adapter.scan(source.table_name, scan_conditions, scan_options).each do |items, metadata|
594594
options = metadata.slice(:last_evaluated_key)
595595

596596
y.yield items, options
@@ -613,11 +613,11 @@ def issue_scan_warning
613613
end
614614

615615
def count_via_query
616-
Dynamoid.adapter.query_count(source.table_name, query_key_conditions, query_non_key_conditions, query_options)
616+
source.adapter.query_count(source.table_name, query_key_conditions, query_non_key_conditions, query_options)
617617
end
618618

619619
def count_via_scan
620-
Dynamoid.adapter.scan_count(source.table_name, scan_conditions, scan_options)
620+
source.adapter.scan_count(source.table_name, scan_conditions, scan_options)
621621
end
622622

623623
def field_condition(key, value_before_type_casting)

lib/dynamoid/document.rb

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,11 @@ module Document
88
include Dynamoid::Components
99

1010
included do
11-
class_attribute :options, :read_only_attributes, :base_class, instance_accessor: false
11+
class_attribute :options, :read_only_attributes, :base_class, :dynamoid_config_name, instance_accessor: false
1212
self.options = {}
1313
self.read_only_attributes = []
1414
self.base_class = self
15+
self.dynamoid_config_name = nil
1516

1617
Dynamoid.included_models << self unless Dynamoid.included_models.include? self
1718
end
@@ -21,6 +22,26 @@ def attr_readonly(*read_only_attributes)
2122
self.read_only_attributes.concat read_only_attributes.map(&:to_s)
2223
end
2324

25+
# Set the DynamoDB configuration to use for this model
26+
#
27+
# @param [Symbol] config_name the name of the configuration
28+
# @since 4.0.0
29+
def dynamoid_config(config_name)
30+
self.dynamoid_config_name = config_name.to_sym
31+
end
32+
33+
# Get the adapter for this model's configuration
34+
#
35+
# @return [Dynamoid::Adapter] the adapter instance
36+
# @since 4.0.0
37+
def adapter
38+
if dynamoid_config_name
39+
Dynamoid::MultiConfig.get_adapter(dynamoid_config_name)
40+
else
41+
Dynamoid.adapter
42+
end
43+
end
44+
2445
# Returns the read capacity for this table.
2546
#
2647
# @return [Integer] read capacity units
@@ -80,7 +101,7 @@ def hash_key
80101
# @return [Integer] items count in a table
81102
# @since 0.6.1
82103
def count
83-
Dynamoid.adapter.count(table_name)
104+
adapter.count(table_name)
84105
end
85106

86107
# Initialize a new object.

lib/dynamoid/errors.rb

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,5 +106,11 @@ def initialize(_msg = nil)
106106
super('Scan operations prohibited. Modify Dynamoid::Config.error_on_scan to change this behavior.')
107107
end
108108
end
109+
110+
class UnknownConfiguration < Error
111+
def initialize(config_name)
112+
super("Unknown configuration: #{config_name}")
113+
end
114+
end
109115
end
110116
end

0 commit comments

Comments
 (0)