Skip to content

Conversation

anishcheraku
Copy link

@anishcheraku anishcheraku commented Jun 20, 2025

Issue number: 4208

Closes #4208

Description of changes: Refactored the get logic for apiclient apply, implemented Structs for each of the resolver types, and added support for S3, SSM, Secrets Manager URIs as well as SSM and Secrets Manager ARNs to apiclient apply

Testing done: Manual Integration Testing for fetching and modifying configuration data from all sources (Stdin, File, HTTP/HTTPS, S3, SSM, Secrets Manager). Added Unit Tests to verify that the correct resolver is selected, positive and negative test cases to verify that URIs/ARNs are parsed correctly/throw an error, test cases to verify that S3 URIs with special characters and special URIs will be parsed correctly, and added a Unit Test to verify that files are parsed properly using Tempfile.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

use reqwest::Url;
use crate::apply::{Error, Result};
use crate::apply::error::{
FileReadSnafu, FileUriSnafu, ReqwestSnafu, StdinReadSnafu, S3UriMissingBucketSnafu, InvalidFileUriSnafu, InvalidHTTPUriSnafu, S3UriSchemeSnafu
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have both FileUriSnafu and InvalidFileUriSnafu
which is a bit confusing? Can we differentiate these error variants?

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Got rid of InvalidFileUriSnafu because it did the same thing as FileUriSnafu and to maintain consistency with other resolvers.

.await
.context(ReqwestSnafu {
uri: self.url.to_string(),
method: "GET".to_string(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need .to_string() here

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Addressing this in another comment: #554 (comment)

#[async_trait]
impl UriResolver for S3Uri {
async fn resolve(&self) -> Result<String> {
// still unimplemented
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally don't want to merge a PR with unimplemented methods like this, but since this is a draft I guess its alright for now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Jarrett, but as another tip, in rust you can use the unimplemented!() macro in functions to avoid having to "make stuff up" Like this:

async fn resolve(&self) -> Result<String> {
    unimplemented!()
}

Instead of having to make up an Err to return, unimplemented!() just does what it takes to make the compiler happy (it panics, lol)

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - I've now implemented the resolve() method completely, so I have removed the unimplemented comment.

@@ -0,0 +1,159 @@
// src/uri_resolver.rs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module is lacking alot of documentation, especially a module level devinition, maybe addint something like:

//! URI resolution for different protocols.
//!
//! This module provides a trait-based approach to resolving different types of URIs
//! including stdin (`-`), file paths (`file://`), HTTP(S) URLs, and S3 URIs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to imagine what someone reading this file for the first time would want.

The module-level docstring is your opportunity to explain how the abstractions/types in here relate to eachother.

If you prime programmers with the right information here, they will be able to read and comprehend your code much faster.

With more sparse information, you basically make the programmer guess at how the implementation works. Then they have to read the code to be absolutely certain (or what often times happen is that they don't do this, and then are surprised when their mental model doesn't totally match the actual code)

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Added a module-level docstring to the top of uri_resolver.rs

.context(error::StdinReadSnafu)
.await?;
return Ok(output);
pub async fn get(input: &str) -> Result<String> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason the visibility on this method needs to be bumped to pub?

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - the function is no longer pub. It was temporarily bumped to pub in the PR because one of the variables didn't have the correct scope.

let url = Url::parse(input).context(error::UriSnafu { input_source: input.to_string() })?;

// 3) file://
if let Ok(r) = FileUri::try_from(url.clone()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it might be better to use a prefix strip and then match instead of trying to directly form the Uri type directly and failing out, but its not a real blocker here.

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - fixed this issue in another comment: #554 (comment)

@anishcheraku
Copy link
Author

anishcheraku commented Jun 30, 2025

Pushed new Resolver for secretsmanager:// and added error handling for Secrets Manager. (reverted in next commit)

@anishcheraku
Copy link
Author

Reversed the changes I just made for secretsmanager://

Copy link
Contributor

@jmt-lab jmt-lab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like something went awry with the rebase packages/kubernetes-1.26 files are showing up here.

@@ -83,11 +84,16 @@ fn select_resolver(input: &str) -> Result<Box<dyn UriResolver>> {
return Ok(Box::new(r));
}

// 6) secretsmanager://
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These numberings look out of order, I used to do numbering too but now i've learned better to not do numbering as it prevents edits from creating inconsistencies.

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Removed the numberings and fixed the order.

Copy link
Contributor

@cbgbt cbgbt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

}

/// Choose which UriResolver applies to `input` (stdin, file://, http(s):// or s3://).
fn select_resolver(input: &str) -> Result<Box<dyn UriResolver>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is fine, but I somewhat prefer an implementation where we have a static list of resolvers, and the resolver selection iterates through the list, calling an interface that each resolver exposes to decide if that resolver is capable of handling this input.

This makes it a little easier to reason about how to add more resolvers, rather than having to keep the stateful "if/else" ladder in your mind.

trait ResolverSelector {
    fn try_create_resolver(input: &str) -> Option<Box<dyn UriResolver>>;
}

const RESOLVERS: &[&dyn ResolverSelector] = &[
    &StdioResolverSelector,
    &FileResolverSelector,
    // etc
];

struct FileResolverSelector;

impl ResolverSelector for FileResolverSelector {
    fn try_create_resolver(input: &str) -> Option<Box<dyn UriResolver>> {
        let input_uri = Url::parse(&input).ok()?; // This maps the "Result" response into an "Option"
        // try to make a FileResolver or return None if we can't
    }
}


use reqwest::Url;
use crate::apply::{Error, Result};
use crate::apply::error::{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't import Snafu's like this.

The "preferred" way is, in each function that needs these Snafu context selectors, do something like this:

fn something_that_can_fail() -> Result<(), ErrorType> {
    use crate::apply::error::*;
    // now the selectors are in scope just for this function
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One reason is that this import list basically becomes a chore to maintain.

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Used the preferred format for crate::uri_resolver as well as crate::apply.

method: "GET".to_string(),
})?
.error_for_status()
.context(ReqwestSnafu {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These shouldn't all be ReqwestSnafu. We want a different error variant to explain the failure mode more succinctly (e.g. an error variant for failing to get text() from a response body should be different than the error variant for failing to make a web request at all)

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - replaced ReqwestSnafus with HttpRequest, HttpStatus, and HttpBody.

}
}

/// s3://bucket/key URLs (stub; add aws-sdk-s3 later)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is useful information to include in a docstring, but you should start with a human-readable description first. Something like:

/// Uri Resolver that fetches content from S3
///
/// This resolver accepts input of the form s3://bucket/key and translates this into
/// authenticated AWS S3 get requests using the standard AWS credential resolution mechanism

Copy link
Author

@anishcheraku anishcheraku Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - added a description for each resolver.

#[async_trait]
impl UriResolver for S3Uri {
async fn resolve(&self) -> Result<String> {
// still unimplemented
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Jarrett, but as another tip, in rust you can use the unimplemented!() macro in functions to avoid having to "make stuff up" Like this:

async fn resolve(&self) -> Result<String> {
    unimplemented!()
}

Instead of having to make up an Err to return, unimplemented!() just does what it takes to make the compiler happy (it panics, lol)

@@ -0,0 +1,159 @@
// src/uri_resolver.rs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to imagine what someone reading this file for the first time would want.

The module-level docstring is your opportunity to explain how the abstractions/types in here relate to eachother.

If you prime programmers with the right information here, they will be able to read and comprehend your code much faster.

With more sparse information, you basically make the programmer guess at how the implementation works. Then they have to read the code to be absolutely certain (or what often times happen is that they don't do this, and then are surprised when their mental model doesn't totally match the actual code)

use aws_config::{self, BehaviorVersion, Region};
use aws_sdk_secretsmanager;

// 1) load AWS config (region/account via env)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, comments which exactly say what the code below does aren't a great idea. The newline separation you have in this function is good, and you can basically skim each newline-separated section to figure out the flow of the function.

I think a better approach would be to hoist a lot of this information into the docstring:

/// Resolves incoming source artifacts by fetching from AWS Secrets Manager
///
/// This function uses the typical AWS credentials resolution mechanism to authenticate.

Docstrings like this are useful at two levels:

  • People calling this function don't need to read the code to understand what's going to happen
  • People who need to change this code can just read the docstring and be primed to understand what they're about to read, much like how I mentioned with module-level docstrings.

So when is it more useful to have inner comments?

  • If the discrete actions of the function are hard to figure out by skimming each newline-separated section (this can also be a sign that you should refactor things, but like all things it's a tradeoff decision)
  • If the what of the code is clear, but not the why (e.g. Imagine a function much like this one, but it reads AWS configurations from a separate file. You would probably need to add a comment there explaining why you are doing that)

@anishcheraku anishcheraku changed the title Refactoring apiclient apply's get logic and apiclient apply support for S3 URIs Refactoring apiclient apply's get logic and apiclient apply support for S3, SSM, and Secrets Manager URIs Jul 2, 2025
Twoliter.toml Outdated
release-version = "9.1.0"
release-version = "8.2.0-1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to make sure we pull these changes out prior to merging.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Rebased my code on top of the bottlerocket-os:develop branch and stopped tracking changes to Twoliter.toml.

@@ -31,7 +37,7 @@ log.workspace = true
models.workspace = true
nix.workspace = true
rand = { workspace = true, features = ["default"] }
reqwest.workspace = true
reqwest.workspace = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
reqwest.workspace = true
reqwest.workspace = true

Copy link
Author

@anishcheraku anishcheraku Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Removed the extra space

@@ -44,6 +50,7 @@ tokio-tungstenite = { workspace = true, features = ["connect"] }
toml.workspace = true
unindent.workspace = true
url.workspace = true
aws-smithy-runtime-api.workspace = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we actually want to directly take a dependency here. Most SDK crates re-export the error types they create.

Copy link
Author

@anishcheraku anishcheraku Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - This dependency is no longer being used

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears to still be present in Cargo.toml

}


/// Choose which UriResolver applies to `input` (stdin, file://, http(s):// or s3://).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be out-of-date.

Copy link
Author

@anishcheraku anishcheraku Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Updated the comment to reflect the new resolvers that were added.

Comment on lines 270 to 282
#[snafu(display("Failed to read S3 object body '{bucket}/{key}': {}", source))]
S3Body {
bucket: String,
key: String,
source: aws_sdk_s3::primitives::ByteStreamError,
},

#[snafu(display("Failed to fetch S3 object '{bucket}/{key}': {}", source))]
S3Get {
bucket: String,
key: String,
source: SdkError<GetObjectError, SdkHttpResponse>,
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of expanding the top-level error type with all of these error variants, I think it would be better to instead make a ResolverError type in the uri_resolver module.

We could then add Resolver as a specific type here. This would make it easier to determine at what point in the program each error variant can occur.

Copy link
Author

@anishcheraku anishcheraku Jul 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Moved all error variants associated with the resolvers into uri_resolver.rs into a new ResolverError enum. Additionally, the functions now return a new type ResolverResult, so it is easier to associate errors with their sources.

use crate::apply::SdkHttpResponse;
use aws_smithy_runtime_api::client::result::SdkError;
use aws_sdk_s3::operation::get_object::GetObjectError;
//use aws_smithy_http::result::SdkHttpResponse;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove unused code.

Copy link
Author

@anishcheraku anishcheraku Jul 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - I have removed the unused code

let url = Url::parse(input).context(error::UriSnafu { input_source: input.to_string() })?;

// file://
if let Ok(r) = crate::uri_resolver::FileUri::try_from(&url) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might suggest adding use crate::uri_resolver so we don't have to fully specify crate:: each time here.

Copy link
Author

@anishcheraku anishcheraku Jul 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Imported the uri_resolver crate, so that I no longer have to specify crate:: each time



/// Choose which UriResolver applies to `input` (stdin, file://, http(s):// or s3://).
fn select_resolver(input: &str) -> Result<Box<dyn crate::uri_resolver::UriResolver>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add unit tests to assert that the correct resolver is selected?

I also think we really need tests to understand what happens when characters in the S3 keyspace are used which aren't supported by URL standard. We should add tests for these cases, as it seems like it's entirely up to our parser implementation to decide what to do given this current implementation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - I have added unit tests to assert that the correct resolver was selected.

Fixed - After trying out 4 different try_from() implementations for the S3Uri, I have found one that works on all sorts of URIs that have special characters, and ones that don't follow the URL standard. I have also added unit tests for testing different S3 URIs that have special characters and don't follow the URL standard.

parameter_name: String,
}

impl TryFrom<&str> for SsmUri {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're still supporting TryFrom on multiple types. Does it make sense to unify on each resolver accepting &str types and allowing them each to parse?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is technically less efficient than the current setup, but it would make it easier for us to test our Resolver selector.

Another option could be to do something like this:

struct SettingsInput {
    input: String,
    parsed_url: Option<Url>,
}

impl SettingsInput {
    pub(crate) fn new(input: impl Into<String>) -> Self {
        let input = input.into();
        let parsed_url = Url::parse(&input).ok();
        Self { input, parsed_url }
    }
}

Then you'd change select_resolver to be like this:

fn select_resolver(input: &SettingsInput) -> Result<Box<dyn crate::uri_resolver::UriResolver>> {
    // ...
}

And you could make it so that each resolver had a uniform TryFrom impl on &SettingsInput.

Doing this would let you parse the Url exactly once and standardize the TryFrom. Resolvers that require a Url could basically return an error if the parsed_url is None.

Copy link
Author

@anishcheraku anishcheraku Jul 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - TryFrom now takes &SettingsInput as its input, and determines what input type is required for each resolver. Also, only one type (&SettingsInput) is passed into all of the resolvers.

@cbgbt
Copy link
Contributor

cbgbt commented Jul 8, 2025

I think the build failed because the Cargo.lock file is not in-sync with your dependency closure?

You can fix this locally by going into the sources directory and doing cargo check.

SecretsManagerUriSnafu { input_source: input.input.as_str().to_string() }
);
// strip the prefix and ensure there's actually an ID
let id = &input.input.as_str()["secretsmanager://".len()..];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you could use .strip_prefix("secretsmanager://") here, which would be easier to read

Copy link
Author

@anishcheraku anishcheraku Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - replaced ["secretsmanager://".len()..] with .strip_prefix("secretsmanager://"). Also rewrote some of the code in the try_from() method to account for changing the type of id.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's update this in the TryFrom implementation for SsmUri as well. I think the S3 resolver's TryFrom implementation is the cleanest of the AWS resolvers as well, can you have all three of them align with that implementation?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - All TryFrom implementations are significantly cleaner and parsing logic for SSM and Secrets Manager align similarly with the parsing logic in S3Uri's TryFrom implementation.

impl UriResolver for HttpUri {
async fn resolve(&self) -> ResolverResult<String> {
use resolver_error::*;
const MAX_SIZE_BYTES: u64 = 100 * 1024 * 1024;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should share this value between the two resolvers that use it, so we can configure it in one place

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - MAX_SIZE_BYTE is now defined at the top of the uri_resolver crate, and has a comment explaining its purpose.

@@ -39,11 +45,14 @@ serde_json.workspace = true
signal-hook.workspace = true
simplelog.workspace = true
snafu = { workspace = true, features = ["futures"] }
test-case.workspace = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be listed under [dev-dependencies] instead of [dependencies]

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - moved test-case.workspace to [dev-dependencies].

@@ -44,6 +50,7 @@ tokio-tungstenite = { workspace = true, features = ["connect"] }
toml.workspace = true
unindent.workspace = true
url.workspace = true
aws-smithy-runtime-api.workspace = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears to still be present in Cargo.toml

Comment on lines 77 to 84
let parsed_url = Url::parse(&input).ok();
SettingsInput { input, parsed_url }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't be a bad idea to emit a debug!() message if the URL parse fails that includes the error message from that failure. That would help us debug cases where the URL parser fails when we would expect it to succeed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - a log::debug! message is now logged whenever the File or HTTP URI resolvers fails to parse a URL correctly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather see the debug message logged here where we still have access to the error message that would have been returned by Url::parse().

The error will only be meaningful in the File/HTTP URI cases, but the idea is that the message will be logged at debug, allowing it to be ignored in cases where it isn't relevant.

Comment on lines 92 to 99
/// Choose which UriResolver applies to `input` (stdin, file://, http(s)://, s3://, secretsmanager://, and ssm://).
fn select_resolver(input: &SettingsInput) -> Result<Box<dyn crate::uri_resolver::UriResolver>> {
use crate::uri_resolver;

// stdin ("-")
if let Ok(r) = uri_resolver::StdinUri::try_from(input) {
return Ok(Box::new(r));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The repetition in this function is still hard for me to thumbs up. I have a simple suggestion to improve this via macros:

/// Macro to try multiple settings resolver types in sequence, returning the first one that succeeds.
macro_rules! try_resolvers {
    ($input:expr, $($resolver_type:ty),+ $(,)?) => {
        $(
            if let Ok(r) = <$resolver_type>::try_from($input) {
                return Ok(Box::new(r));
            }
        )+
    };
}

/// Choose which UriResolver applies to `input` (stdin, file://, http(s)://, s3://, secretsmanager://, and ssm://).
fn select_resolver(input: &SettingsInput) -> Result<Box<dyn crate::uri_resolver::UriResolver>> {
    use crate::uri_resolver::*;

    try_resolvers!(
        input,
        StdinUri,
        FileUri,
        HttpUri,
        S3Uri,
        SecretsManagerArn,
        SecretsManagerUri,
        SsmArn,
        SsmUri,
    );

    error::NoResolverSnafu {
        input_source: input.input.clone(),
    }
    .fail()
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also probably emit a debug!-level log message when we select a resolver saying which one we are using.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Made both changes.

Comment on lines 279 to 287
#[test_case("-", TypeId::of::<crate::uri_resolver::StdinUri>(); "stdin")]
#[test_case("file:///tmp/folder", TypeId::of::<crate::uri_resolver::FileUri>(); "file")]
#[test_case("http://amazon.com", TypeId::of::<crate::uri_resolver::HttpUri>(); "http")]
#[test_case("https://amazon.com", TypeId::of::<crate::uri_resolver::HttpUri>(); "https")]
#[test_case("s3://mybucket/path", TypeId::of::<crate::uri_resolver::S3Uri>(); "s3")]
#[test_case("secretsmanager://sec", TypeId::of::<crate::uri_resolver::SecretsManagerUri>(); "secrets")]
#[test_case("ssm://param", TypeId::of::<crate::uri_resolver::SsmUri>(); "ssmUri")]
#[test_case("arn:aws:ssm:<region>:<account_id>:parameter/<name>", TypeId::of::<crate::uri_resolver::SsmArn>(); "ssmArn")]
#[test_case("arn:aws:secretsmanager:<region>:<account-id>:secret:<secret-id>", TypeId::of::<crate::uri_resolver::SecretsManagerArn>(); "secretsmanagerArn")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spacing on these is very odd. You should either consistently use one space after each comma, or modify these to look like this:

#[test_case(
    "-",
    TypeId::of::<crate::uri_resolver::StdinUri>()
    ; "stdin")]

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - The spacing is now consistent for all test cases.

// 3) Construct
Ok(SsmArn {
region: arn.region,
parameter_name: input.input.to_string(), // still use the full `/parameter/...` segment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment and parameter_name are both incorrect. This contains the full ARN.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Fixed both variable names for SecretsManagerArn and SsmArn to be more descriptive of what they actually contain (full_arn).

impl UriResolver for SsmArn {
async fn resolve(&self) -> ResolverResult<String> {
use resolver_error::*;
// 1) Load default config, then override region & credentials
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment is not correct, we aren't changing credentials.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - Forgot to update the comment. The original comment mentioned overwriting the credentials because that was my first approach, but then decided to use IAM permissions to access parameters and secrets instead (doesn't require overriding credentials).

Comment on lines 587 to 566
#[derive(Debug, Snafu)]
#[snafu(module)]
pub enum ResolverError {
#[snafu(display("Invalid ARN '{}': {}", input_source, reason))]
InvalidArnFormat {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer having this be composed of smaller Resolver-specific types like FileUriResolverError, S3ResolverError, SsmResolverError, etc would improve maintainability.

@cbgbt cbgbt marked this pull request as ready for review July 29, 2025 05:56
@cbgbt cbgbt marked this pull request as draft July 29, 2025 05:56
Copy link
Contributor

@cbgbt cbgbt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice work! I have a few comments about logging, but otherwise this LGTM.

Comment on lines 77 to 84
let parsed_url = Url::parse(&input).ok();
SettingsInput { input, parsed_url }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather see the debug message logged here where we still have access to the error message that would have been returned by Url::parse().

The error will only be meaningful in the File/HTTP URI cases, but the idea is that the message will be logged at debug, allowing it to be ignored in cases where it isn't relevant.

Comment on lines 114 to 112
let url = {
if let None = &input.parsed_url {
log::debug!("Failed to parse HTTP URI '{}'", input.input);
}
input.parsed_url.clone()
}.context(FileUriSnafu {
input_source: input.input.as_str(),
})?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug! is the wrong log level for a failure here. We expect the URI to have parsed properly, so an error here would be error! and not debug!.

I'd be OK with removing the logging in this case in favor of just always debug logging at the top level as mentioned in the other comment.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - updated the debug!() statement to be thrown whenever there is an error parsing the Url for all parsed Urls, not just in the File/HTTP URI cases.

@@ -44,6 +50,10 @@ tokio-tungstenite = { workspace = true, features = ["connect"] }
toml.workspace = true
unindent.workspace = true
url.workspace = true
tempfile.workspace = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this also be a dev dependency?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed - It should be under [dev-dependencies] since its only used to test the File parser.

@anishcheraku anishcheraku force-pushed the dev branch 2 times, most recently from 05ba52f to 0faab71 Compare August 1, 2025 18:20
@anishcheraku
Copy link
Author

^ Force pushed to fix commit messages and remove unnecessary changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support s3://, ssm://, and secretsmanager:// protocols in apiclient apply
4 participants