Skip to content
This repository was archived by the owner on Sep 29, 2025. It is now read-only.

Conversation

lalver1
Copy link
Member

@lalver1 lalver1 commented Jul 22, 2025

Closes #185

Add a policy (as a Copilot addon) for accessing the S3 marts bucket from the ECS streamlit task via its Task role.

Notes

Using a ManagedPolicy is the most straightforward way to add permissions to the Copilot-created ECS streamlit TaskRole.
To see how the CloudFormation templates created by Copilot will look like when adding the s3-marts-access-policy addon, run

copilot svc package --name streamlit --env dev --output-dir .

from the infra folder. The command will create 3 files (parameters, main stack template, and addon stack template) in infra. The addon stack template is essentially the template in this PR (with a few formatting changes) and the main stack template and parameters are what the Copilot CLI will produce to send to CloudFormation to create the stack. The key section of the main stack, where the link is made with the permissions defined in the addon, is shown below

...
  TaskRole:
    Metadata:
      'aws:copilot:description': 'An IAM role to control permissions for the containers in your tasks'
    Type: AWS::IAM::Role
    Properties:
      ManagedPolicyArns:
        - Fn::GetAtt: [AddonsStack, Outputs.S3martsBucketAccessPolicyArn]
 ...

Note that there is no mention of the environment in the addon stack because accessing the S3 marts bucket is not dependent on the environment, all environments will access the same S3 bucket with the same permissions.

@lalver1 lalver1 self-assigned this Jul 22, 2025
Copy link

Coverage report

This PR does not seem to contain any modification to coverable code.

@lalver1 lalver1 marked this pull request as ready for review July 22, 2025 19:31
@lalver1 lalver1 requested a review from a team as a code owner July 22, 2025 19:31
S3martsBucketAccessPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: s3martsBucketPolicy
Copy link
Member Author

@lalver1 lalver1 Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this to capital "S3"
Redundant since the policy was changed to a ManagedPolicy in 843eb60

PolicyName: s3martsBucketPolicy
# Attach this policy to the TaskRole provided by Copilot.
Roles:
- "pems-dev-streamlit-TaskRole-5Zuwi15thMkN"
Copy link
Member Author

@lalver1 lalver1 Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modify this so that the policy is also added to test and prod
Redundant since the policy was changed to a ManagedPolicy in 843eb60

@lalver1 lalver1 marked this pull request as draft July 23, 2025 13:48
@lalver1 lalver1 force-pushed the fix/s3-marts-data-permissions branch from 58fb003 to c4c3ea0 Compare July 24, 2025 15:22
lalver1 added 3 commits July 24, 2025 10:26
it turns out that the parameter "TaskRole" in the template is not
being set by Copilot, so it does not resolve. Because of this,
we need to hardcode the name of the Task role.
given how AWS Copilot works, it is more convenient to use a managed
policy since it is automatically attached to the Task role.
@lalver1 lalver1 force-pushed the fix/s3-marts-data-permissions branch from c4c3ea0 to 843eb60 Compare July 24, 2025 15:26
@lalver1
Copy link
Member Author

lalver1 commented Jul 24, 2025

@thekaveman I updated how we are giving permissions to the Task role that runs the streamlit ECS Task. Instead of using an inline policy (which requires assigning the policy explicitly to the dynamically named Task role), we now use a managed policy. The main advantage of a managed policy is that we don't need to manually keep track of the name of the Task role, Copilot automatically links the Task role with the managed policy we define in the addon. Copilot does not do this automatic linking with inline policies. I deployed the streamlit service manually to dev and this approach worked. It should also work with test and prod (the environment is passed as a parameter from the Copilot CLI) and because the policy is not dependent on the environment (we expect all 3 environments to be able to access the same S3 marts bucket) we don't need to make any changes to the template for test and prod deployments.

@lalver1 lalver1 marked this pull request as ready for review July 24, 2025 18:21
Copy link
Member

@thekaveman thekaveman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incredible! Using a Managed Policy seems like the best approach 👍

@lalver1 lalver1 merged commit 566c3b5 into main Jul 24, 2025
6 checks passed
@lalver1 lalver1 deleted the fix/s3-marts-data-permissions branch July 24, 2025 22:02
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix permissions to load 5-min data from S3 bucket
2 participants