Skip to content

Add support for AWS SigV4-authenticated OTLP Exporter in Python SDK (Implementation Ready) #3494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Stupidoodle opened this issue May 11, 2025 · 2 comments

Comments

@Stupidoodle
Copy link

What are you trying to achieve?

Enable first-class support for AWS SigV4-authenticated OTLP HTTP exports from the OpenTelemetry Python SDK — particularly to support containerized AWS Lambda apps sending data to AWS X-Ray.

Currently, AWS provides a Lambda Layer for OTLP export, but it is:
• Incompatible with containerized Lambda deployments
• Tightly scoped to environments with preconfigured agents
• Not usable from within standard OTLP Python

What did you expect to see?

A pluggable, officially supported way to send OTLP spans from Python to AWS X-Ray using AWS SigV4 signing (e.g., via an auth extension or built-in configuration). Ideally, this would:
• Respect the existing OTLP exporter structure
• Use botocore for credential resolution and SigV4 signing
• Support compression (gzip/deflate), retries, and timeout config
• Be compatible with OpenTelemetry SDK conventions (e.g., SpanExporter, BatchSpanProcessor)

Additional context

As a temporary solution, I built a custom SigV4OTLPSpanExporter from scratch, fully compatible with:
• xray.{region}.amazonaws.com
• All OTLP span export semantics
• AWS SDK credential chains
• CI pipelines (Docker + LocalStack integration tests)
• Observability best practices

GitHub Repo: https://github.com/Stupidoodle/opentelemetry-exporter-otlp-proto-http-sigv4

I’m happy to contribute this upstream (as core or contrib) or collaborate with others working on auth extension designs. Just let me know what path would be most useful.

@Stupidoodle
Copy link
Author

To enable OTLP HTTP trace exports to AWS X-Ray using this exporter, you must configure X-Ray to use CloudWatch Logs as its trace segment destination.

AWS returns a 400 error unless the following conditions are met:

✅ One-time setup (per account/region):

import boto3, json

# Allow X-Ray to write to CloudWatch Logs
policy_name = "AWSXrayCloudWatchAccess"
policy_document = {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AWSXrayCloudWatchAccess",
            "Effect": "Allow",
            "Principal": {
                "Service": "xray.amazonaws.com"
            },
            "Action": [
                "logs:PutLogEvents",
                "logs:CreateLogStream",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams"
            ],
            "Resource": "*"
        }
    ]
}

boto3.client("logs").put_resource_policy(
    policyName=policy_name,
    policyDocument=json.dumps(policy_document)
)

# Enable CloudWatch Logs as X-Ray destination
boto3.client("xray").update_trace_segment_destination(Destination="CloudWatchLogs")

If you skip this, you’ll get:

400 ERROR: The OTLP API is supported with CloudWatch Logs as a Trace Segment Destination.

@jpkrohling jpkrohling transferred this issue from open-telemetry/opentelemetry-specification May 12, 2025
@jpkrohling
Copy link
Member

I'm moving this to the python-contrib repo. @open-telemetry/python-approvers , let me know if this is not the right repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants