You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+31-31Lines changed: 31 additions & 31 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,14 @@
1
-
# Firetail Lambda Extension
1
+
# FireTail Lambda Extension
2
2
3
3
[](https://www.gnu.org/licenses/lgpl-3.0)[](https://github.com/FireTail-io/firetail-lambda-extension/actions/workflows/codecov.yml)[](https://codecov.io/gh/FireTail-io/firetail-lambda-extension)
4
4
5
5
6
6
7
7
## Overview
8
8
9
-
The Firetail Logging Extension receives AWS Lambda events & response payloads and sends them to the Firetail Logging API.
9
+
The FireTail Logging Extension receives AWS Lambda events and response payloads and sends them to the FireTail Logging API.
10
10
11
-
The extension receives these events and response payloads via a runtime-specific Firetail library which you will need to use in your Function code. The Firetail library outputs specifically formatted logs which the extension then receives via the [Lambda Logs API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-logs-api.html). You can find a table of Firetail function libraries which correspond with a Lambda runtime in the [Function Libraries](#function-libraries) section. Below is a diagram depicting how the Firetail extension, Extensions API, Logs API and Firetail API interact over the lifetime of a Lambda.
11
+
The extension receives these events and response payloads by a runtime-specific FireTail library which you will need to use in your Function code. The FireTail library outputs specifically formatted logs which the extension then receives from the [Lambda Logs API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-logs-api.html). You can find a table of FireTail function libraries that correspond with a Lambda runtime in the [Function Libraries](#function-libraries) section. Below is a diagram depicting how the FireTail extension, Extensions API, Logs API and FireTail API interact over the lifetime of a Lambda.
Automated testing is setup with the `testing` package, using [github.com/stretchr/testify](https://pkg.go.dev/github.com/stretchr/testify) for shorthand assertions. You can run them with `go test`, or use the provided [Makefile](./Makefile)'s `test` target, which is as simple as:
27
+
Automated testing is set up with the `testing` package, using [github.com/stretchr/testify](https://pkg.go.dev/github.com/stretchr/testify) for shorthand assertions. You can run them with `go test`, or use the provided [Makefile](./Makefile)'s `test` target, which is:
28
28
29
29
```bash
30
30
make test
31
31
```
32
32
33
-
This will output a coverage report (`coverage.out`) which you may view in your browser by using the [go tool cover](https://pkg.go.dev/cmd/cover) command:
33
+
This will output a coverage report (`coverage.out`) which you can view in your browser by using the [go tool cover](https://pkg.go.dev/cmd/cover) command:
34
34
35
35
```bash
36
36
go tool cover -html coverage.out
@@ -40,29 +40,29 @@ go tool cover -html coverage.out
40
40
41
41
## Deployment
42
42
43
-
The Firetail Logging Extension is an external Lambda extension, published as a Lambda Layer. Deploying it is a fivestep process. If you wish to use the publicly accessible Lambda Layer published by Firetail, you can skip to the final step. The full list of steps to build, package, publish and use the Firetail Lambda Extension are as follows:
43
+
The FireTail Logging Extension is an external Lambda extension, published as a Lambda Layer. Deploying it is a five-step process. If you want to use the publicly accessible Lambda Layer published by FireTail, you can skip to the final step. The full list of steps to build, package, publish and use the FireTail Lambda Extension are as follows:
44
44
45
-
- The first step is to [build the extension binary](#building-the-extension-binary).
46
-
- The second step is to [package the extension binary](#packaging-the-extension-binary).
47
-
- The third step is to [publish the package as a Lambda Layer](#publishing-the-package).
48
-
- An optional fourth step is to [make the layer public](#making-the-layer-public).
49
-
- The final step is to [add the layer to a Lambda Function](#adding-the-layer-to-a-lambda-function).
45
+
1.[Build the extension binary](#building-the-extension-binary).
46
+
2.[Package the extension binary](#packaging-the-extension-binary).
47
+
3.[Publish the package as a Lambda Layer](#publishing-the-package).
48
+
4. (Optional) [Make the layer public](#making-the-layer-public).
49
+
5.[Add the layer to a Lambda Function](#adding-the-layer-to-a-lambda-function).
50
50
51
-
This process has been partially automated in the provided [Makefile](./Makefile). In order to use this makefile you will need to install the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), [Golang](https://go.dev/doc/install) and [JQ](https://stedolan.github.io/jq/). You may observe how this Makefile is used by us in the Github action named "[build & publish](./.github/workflows/release.yaml)".
51
+
This process has been partially automated in the provided [Makefile](./Makefile). To use this makefile you need to install the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), [Golang](https://go.dev/doc/install) and [JQ](https://stedolan.github.io/jq/). You can observe how this Makefile is used by us in the Github action named "[build & publish](./.github/workflows/release.yaml)".
52
52
53
53
54
54
55
55
### Building The Extension Binary
56
56
57
-
The logging extension is a standard Go project and can be built by [installing Go](https://go.dev/doc/install) and using the [go build](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies) command from the root directory of this repository. You will need to set the `GOOS` and `GOARCH` environment variables appropriately for your target Lambda runtime's operating system and architecture. See the [Environment variables](https://pkg.go.dev/cmd/go#hdr-Environment_variables) section of the [go command docs](https://pkg.go.dev/cmd/go) for more information. An example may look like this:
57
+
The logging extension is a standard Go project and can be built by [installing Go](https://go.dev/doc/install) and using the [go build](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies) command from the root directory of this repository. You will need to set the `GOOS` and `GOARCH` environment variables appropriately for your target Lambda runtime's operating system and architecture. See the [Environment variables](https://pkg.go.dev/cmd/go#hdr-Environment_variables) section of the [go command docs](https://pkg.go.dev/cmd/go) for more information. An example is:
58
58
59
59
```bash
60
60
GOOS=linux GOARCH=amd64 go build
61
61
```
62
62
63
63
This will yield a binary with the same name as the root directory, which if you have just cloned this repository will be `firetail-lambda-extension`.
64
64
65
-
The target in the provided makefile that corresponds to this step is `build`. It requires a target architecture (`ARCH`) which defaults to `amd64`. For example, you may wish to do:
65
+
The target in the provided makefile that corresponds to this step is `build`. It requires a target architecture (`ARCH`) that defaults to `amd64`. For example, you can:
66
66
67
67
```bash
68
68
make build ARCH=arm64
@@ -80,7 +80,7 @@ To package the extension binary, it must be placed into a directory named `exten
The target in the provided makefile that corresponds to this step is `package`, and it depends upon the `build` step. It requries a target architecture (`ARCH`), and extension version (`VERSION`) which defaults to `latest`. For example, you may wish to do:
83
+
The target in the provided makefile that corresponds to this step is `package`, and it depends upon the `build` step. It requires a target architecture (`ARCH`), and extension version (`VERSION`) which defaults to `latest`. For example, you can do:
84
84
85
85
```bash
86
86
make package ARCH=arm64 VERSION=v1.0.0
@@ -92,11 +92,11 @@ This will yield a `.zip` file in the `build` directory named `firetail-extension
92
92
93
93
### Publishing The Package
94
94
95
-
To publish the package, you may use the AWS CLI's [publish-layer-version](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/lambda/publish-layer-version.html) command. You will need to repeat this process for every region in which you wish to use the layer. You will also need to specify the compatible architectures, and give the layer a name. The output of the command will provide you with the layer's ARN and layer version, which you may use to add it to your Lambdas.
95
+
To publish the package, you can use the AWS CLI's [publish-layer-version](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/lambda/publish-layer-version.html) command. You will need to repeat this process for every region in which you wish to use the layer. You will also need to specify the compatible architectures, and give the layer a name. The output of the command will provide you with the layer's ARN and layer version, which you may use to add it to your Lambdas.
96
96
97
97
If you reuse the same layer name multiple times, the layer version will be incremented. The approach taken in the provided makefile is to publish each extension version with a new layer name, so the layer version will almost always be `1`.
98
98
99
-
The target in the provided makefile that corresponds to this step is `publish`. You must make the `build` target before the `publish` target. The `publish` target requires a target architecture (`ARCH`) and extension version (`VERSION`), which match that used when you made the `package` target; and a region in which to publish the layer (`AWS_REGION`). For example, you may wish to do:
99
+
The target in the provided makefile that corresponds to this step is `publish`. You must make the `build` target before the `publish` target. The `publish` target requires a target architecture (`ARCH`) and extension version (`VERSION`), which match that used when you made the `package` target; and a region in which to publish the layer (`AWS_REGION`). For example, you can:
100
100
101
101
```bash
102
102
make publish ARCH=arm64 VERSION=v1.0.0 AWS_REGION=eu-west-1
@@ -111,7 +111,7 @@ make publish ARCH=arm64 VERSION=v1.0.0 AWS_REGION=eu-west-1
111
111
112
112
To make the layer public, you may use the AWS CLI's [add-layer-version-permission](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/lambda/add-layer-version-permission.html) command. You will need to repeat this process for every layer you publish in every region. You will need to provide the layer name & layer version, a statement ID and region; and to make the layer public an action of `lambda:GetLayerVersion` and principal of `*`.
113
113
114
-
The target in the provided makefile corresponding to this step is `public`. You must make the `publish` target before the `public` target. The `public` target requires a target architecture (`ARCH`), extension version (`VERSION`) and AWS region (`AWS_REGION`) which match that used when you made the `publish` target, as well as the layer version created when you made the `publish` target (`AWS_LAYER_VERSION`). For example, you may wish to do:
114
+
The target in the provided makefile corresponding to this step is `public`. You must make the `publish` target before the `public` target. The `public` target requires a target architecture (`ARCH`), extension version (`VERSION`) and AWS region (`AWS_REGION`) which match that used when you made the `publish` target, as well as the layer version created when you made the `publish` target (`AWS_LAYER_VERSION`). For example, you can:
115
115
116
116
```bash
117
117
make public ARCH=arm64 VERSION=v1.0.0 AWS_REGION=eu-west-1 AWS_LAYER_VERSION=1
@@ -121,15 +121,15 @@ make public ARCH=arm64 VERSION=v1.0.0 AWS_REGION=eu-west-1 AWS_LAYER_VERSION=1
121
121
122
122
### Adding The Layer To A Lambda Function
123
123
124
-
There are a number of ways to add the published layer to your Lambda Function:
124
+
There are several ways to add the published layer to your Lambda Function:
125
125
126
126
1.[Using the AWS CLI](#using-the-aws-cli).
127
127
2.[Using An AWS Lambda Docker Build](#using-an-aws-lambda-docker-build).
128
128
3.[Using Terraform](#using-terraform).
129
129
130
-
You will need to ascertain the layer ARN of the Lambda Layer containing the Firetail Lambda Extension that you wish to use. If you are not publishing your own Firetail Extension Lambda Layer, you may use the Lambda Layer published publicly by Firetail.
130
+
You will need to ascertain the layer ARN of the Lambda Layer containing the FireTail Lambda Extension that you wish to use. If you are not publishing your own FireTail Extension Lambda Layer, you may use the Lambda Layer published publicly by FireTail.
131
131
132
-
The latest extension version of the publically accessible Lambda Layer published by Firetail can be derived by taking the latest version tag in the [Github Releases](https://github.com/FireTail-io/firetail-lambda-extension/releases) of this repository, and replacing the `.` characters with `-` characters. For example, `v1.2.3` would become `v1-2-3`. You will also need to determine the architecture you need for your Lambda Runtime, which may be either `arm64` or `x86_64`. Once you have these two values, you may substitute them into `${VERSION}` and `${ARCH}` respectively in the following string:
132
+
The latest extension version of the publically accessible Lambda Layer published by FireTail can be derived by taking the latest version tag in the [Github Releases](https://github.com/FireTail-io/firetail-lambda-extension/releases) of this repository, and replacing the `.` characters with `-` characters. For example, `v1.2.3` would become `v1-2-3`. You will also need to determine the architecture you need for your Lambda Runtime, which may be either `arm64` or `x86_64`. Once you have these two values, you may substitute them into `${VERSION}` and `${ARCH}` respectively in the following string:
Regardless of how you add the Lambda Layer to your Lambda Function, you will also need to configure at least one environment variable: `FIRETAIL_API_TOKEN`. Find below a full list of the environment variables used by the Firetail Lambda Extension:
144
+
Regardless of how you add the Lambda Layer to your Lambda Function, you will also need to configure at least one environment variable: `FIRETAIL_API_TOKEN`. Find below a full list of the environment variables used by the FireTail Lambda Extension:
145
145
146
146
| Environment Variable | Default Value | Description |
| FIRETAIL_API_TOKEN | None | Your API token for the Firetail Logging API. If left unset, no logs will be sent to the Firetail Logging API |
149
-
| FIRETAIL_API_URL |`https://api.logging.eu-west-1.prod.firetail.app/logs/bulk`| The URL of the Firetail Logging API |
150
-
| FIRETAIL_EXTENSION_DEBUG |`false`| Enables debug logging from the extension if set to a value parsed as `true` by [strconv.ParseBool](https://pkg.go.dev/strconv#ParseBool)|
151
-
| FIRETAIL_LOG_BUFFER_SIZE |`1000`| The maximum amount of logs the extension will hold in its buffer from which logs are batched and sent to Firetail|
152
-
| FIRETAIL_MAX_BATCH_SIZE |`100`| The maximum size of a batch of logs to be sent to the Firetail logging API in one request |
148
+
| FIRETAIL_API_TOKEN | None | Your API token for the FireTail Logging API. If left unset, no logs will be sent to the Firetail Logging API.|
149
+
| FIRETAIL_API_URL |`https://api.logging.eu-west-1.prod.firetail.app/logs/bulk`| The URL of the FireTail Logging API.|
150
+
| FIRETAIL_EXTENSION_DEBUG |`false`| Enables debug logging from the extension if set to a value parsed as `true` by [strconv.ParseBool](https://pkg.go.dev/strconv#ParseBool).|
151
+
| FIRETAIL_LOG_BUFFER_SIZE |`1000`| The maximum amount of logs the extension will hold in its buffer from which logs are batched and sent to FireTail.|
152
+
| FIRETAIL_MAX_BATCH_SIZE |`100`| The maximum size of a batch of logs to be sent to the FireTail logging API in one request.|
153
153
154
154
155
155
156
156
#### Using The AWS CLI
157
157
158
158
To add the Lambda Layer to a Function, you may use the AWS CLI's [update-function-configuration](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/lambda/update-function-configuration.html) command. You will need to provide a region, the layer ARN and the name of the Function to which the layer is to be added.
159
159
160
-
The target in the provided makefile corresponding to this step is `add`. The `add` target requires the layer ARN (`LAYER_ARN`), the name of the Function to add the layer to (`FUNCTION_NAME`), and the AWS region in which both the layer and the Function must be found (`AWS_REGION`). For example, you may wish to do:
160
+
The target in the provided makefile corresponding to this step is `add`. The `add` target requires the layer ARN (`LAYER_ARN`), the name of the Function to add the layer to (`FUNCTION_NAME`), and the AWS region in which both the layer and the Function must be found (`AWS_REGION`). For example, you can do:
161
161
162
162
```bash
163
163
make add AWS_REGION=eu-west-1 LAYER_ARN=your-layer-arn FUNCTION_NAME=your-function-name
@@ -195,7 +195,7 @@ You will then need to add the following step into the final stage in your Docker
195
195
COPY --from=firetail-layer-copy /opt /opt
196
196
```
197
197
198
-
Once these steps are complete you should be able to run your build process as before, except with the addition of the `AWS_DEFAULT_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_LAYER_ARN` build arguments.
198
+
When these steps are complete you should be able to run your build process as before, except with the addition of the `AWS_DEFAULT_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_LAYER_ARN` build arguments.
199
199
200
200
⚠️ Ensure you securely use, and store these minimal access credentials ⚠️
0 commit comments