Support plugin configurations within config.properties

[tdcmeehan]

Hello folks,

Wanted to get any inputs on supporting configuring plugins directly within config.properties.

Firstly, why do this at all? When configuring multiple plugins, it can be a bit of a pain to see the at a glance configuration of Presto, because they're scattered across several properties. I think it would be easier to provide a default instance of a configuration if we allow inline plugin configuration in the main config file.

The idea would be to add a special prefix for plugins, such as plugin., then followed by a key indicating the plugin itself (such as hive), then a key indicating the instance of the plugin if applicable (such as hive_interactive1). There's a bit of disorganization in how we configure plugins, and I think this would clean it up a bit. Keys which are not prefixed by plugin. will continue to be strictly validated, and to prevent an accident where a core property is added which has this prefix, we can add a checkstyle rule which would fail the build if it's added at any point in the future.

We could still support the old way of configuring plugins. I have a couple ideas

1. Preferred. We could fallback to the old way if no property of the matching plugin and plugin instance are found. Perhaps we can throw if both are present simultaneously.
2. We could treat the plugin properties in config.properties as a default, and override when a dedicated etc config present.

Appreciate any thoughts.

[ZacBlanco]

I think this would be nice feature to have. Separate configuration files is also nice for users who might have lots of configuration items or complex setups where navigating a single file can lead to misunderstandings or errors. Here are a few of my thoughts regarding the configuration of plugins:

1. I think plugins should always be configured explicitly in config.properties

• Currently plugins are determined by looking at the list of files under the etc/ folder, requiring filesystem inspection in order to determine the plugins which might be deployed This can lead to ambiguity or confusion as to which plugins are deployed. Even if we don't take the approach of plugins configured directly in config.properties I think that at minimum we should force users to declare the plugins they wish to use inside of config.properties. This might take a form similar to what was suggested above where users would add a line in the config.properties which looks like plugin.<user-provided-name>=<plugin name (e.g. iceberg, hive, ...)>,

2. I think plugin config location outside of config.properties should be defined explicitly.

• Following with the example previously, users would add a line in their config which has an enum with values INLINE or FILESYSTEM that tells Presto where to look for the plugin's configuration. e.g. plugin.<user-provided-name>.location=FILESYSTEM. INLINE could be the default and allow users to configure plugins within config.properties directly. FILESYSTEM would then either search in etc/ automatically or allow the users to specify a particular path to their configuration with something like plugin.<user-provided-name>.config-location=<path>.

3. I have a concern that for complex configurations, we might want to provide a new type of configuration file, such as TOML which has sections built-in to its format.

• I have seen some user configurations for hive which have lots of configuration values. Having to prefix everything in config.properties with plugin.<name> might be cumbersome. It could be useful to have config sections like in TOML which would prevent needing to prefix plugin configurations. Example TOML configuration below.
• TOML might also be an interesting choice because it allows specifying structs in addition to simple values which could help simplify configuration/prevent errors when configuring subsystems (e.g. related caching parameters/timeouts), security, etc

plugin.hive_interactive1 = "hive"
plugin.hive_batch = "hive"

[plugin.hive_batch]
location = "FILESYSTEM" # plugin.<name>.config-file must be specified when this is present
config-file = "/opt/my-custom-location/plugin.config" 

[plugin.hive_interactive1]
hive.metastore = "thrift"
hive.metastore.uri = "thrift://example.net:9083"
hive.metastore-cache = { ttl = "1m", refresh = { interval = "20m", max-threads = 4 } }

[tdcmeehan]

Using TOML is an interesting suggestion. I like how it allows inline tables as in your example, and it seems like an easy migration from more basic properties. I also find the syntax to be intuitive.

I wrestled a bit with the idea of making plugins an array of tables, but I think the example above is better since plugin configurations have set semantics (no duplicates), and using an array to specify them would mean we'd have to validate those set semantics.

[hantangwangd]

I think it would be nice to support plugin configurations directly in config.properties, and keep the old way as a fallback configuration. And agree with @ZacBlanco 's suggestion about the config sections, which do not need to use prefixes to distinguish the same configuration items used by different plugins. In this way, we can easily identify the common configuration items which may be used in multiple plugins, and move them to appropriate common modules as appropriate. For example, moving some configuration items from presto-hive to presto-hive-common or presto-hive-metastore, since they may be configured and used in some other lakehouse connectors like Iceberg, Hudi or Delta.

[lutongzero]

While this approach provides convenience, I've identified some issues:

Multi-node maintenance overhead:
When adjusting plugin parameters, synchronized modifications are required in both Coordinator/Worker's config.properties

Environment scalability constraints:
Tight coupling between plugin configurations and node settings makes modern operational requirements like cloud-based configuration synchronization challenging


Proposal: Abstract plugin configurations as independent modules
For example, define only the configuration source type in config.properties, while loading actual content via dedicated channels:


# Example: Load from local file
plugin.config.source=file
plugin.config.path=etc/catalog/*.properties  # plugin.toml

# Example: Extend to Git repository (optional)
plugin.config.source=git
plugin.config.path=git@github.com:prestodb/presto.git/etc/catalog


Expected benefits:

1. Reduce collaboration costs: Ops teams can manage plugin configurations independently without frequent node file modifications
2. Enhance environmental adaptability: Future data source integrations only require adding a new source type
3. Preserve compatibility: Default local file mode ensures no immediate changes to existing deployment workflows

[tdcmeehan]

Multi-node maintenance overhead:
When adjusting plugin parameters, synchronized modifications are required in both Coordinator/Worker's config.properties

This is already the case, right? It seems orthogonal to this issue.

Environment scalability constraints:
Tight coupling between plugin configurations and node settings makes modern operational requirements like cloud-based configuration synchronization challenging

Can you please elaborate on the challenges in synchronization?

The proposal also seems orthogonal to the underlying goal of simplifying configuration management.

Reduce collaboration costs: Ops teams can manage plugin configurations independently without frequent node file modifications

Can you elaborate on this benefit? All nodes require a restart when properties are changed, so it doesn't seem any less disruptive. You can already manage the configuration file independently through source control and external orchestration, which any team should be using.

[lutongzero]

Due to translation ambiguity, I confused 'plugin' with 'connector' in the technical context. My apologies for not clarifying this terminology earlier in the proposal.Please disregard the proposal as it was based on this misunderstanding.