Add Prometheus Sink #6229
Add Prometheus Sink #6229
Conversation
Signed-off-by: Kondaka <krishkdk@amazon.com>
kkondaka
requested review from
KarstenSchnitter,
chenqi0805,
dinujoh,
dlvenable,
engechas,
graytaylor0,
oeyh,
san81,
sb2k16 and
srikanthjg
as code owners
Signed-off-by: Kondaka <krishkdk@amazon.com>
Signed-off-by: Kondaka <krishkdk@amazon.com>
Signed-off-by: Kondaka <krishkdk@amazon.com>
|
How much effort would be required to make this compatible with a standard Prometheus instance, e.g. |
|
@KarstenSchnitter I think it should work with any prometheus server once we support different auth mechanisms. May be the simplest one maybe username/password auth support |
...k/src/main/java/org/opensearch/dataprepper/plugins/sink/prometheus/PrometheusHttpSender.java
Show resolved
Hide resolved
...us-sink/src/main/java/org/opensearch/dataprepper/plugins/sink/prometheus/PrometheusSink.java
Show resolved
Hide resolved
| .with("statusCode", statusCode) | ||
| .with("pluginName", pluginSetting.getName()) | ||
| .with("pipelineName", pluginSetting.getPipelineName()); |
There was a problem hiding this comment.
Should this metadata be made into variables? Seems like common metadata all sinks may use
There was a problem hiding this comment.
We could. Is it OK to do this as a separate followup PR?
|
@kkondaka can you add basic auth support? Other authentications can be added on request. |
...pensearch/dataprepper/plugins/sink/prometheus/configuration/PrometheusSinkConfiguration.java
Show resolved
Hide resolved
...pensearch/dataprepper/plugins/sink/prometheus/configuration/PrometheusSinkConfiguration.java
Show resolved
Hide resolved
...r-plugins/common/src/main/java/org/opensearch/dataprepper/common/sink/DefaultSinkBuffer.java
Show resolved
Hide resolved
...k/src/main/java/org/opensearch/dataprepper/plugins/sink/prometheus/PrometheusHttpSender.java
Show resolved
Hide resolved
...va/org/opensearch/dataprepper/plugins/sink/prometheus/service/PrometheusSinkBufferEntry.java
Show resolved
Hide resolved
...va/org/opensearch/dataprepper/plugins/sink/prometheus/service/PrometheusSinkBufferEntry.java
Show resolved
Hide resolved
...in/java/org/opensearch/dataprepper/plugins/sink/prometheus/service/PrometheusTimeSeries.java
Show resolved
Hide resolved
| Region region = (awsConfig != null) ? awsConfig.getAwsRegion() : awsCredentialsSupplier.getDefaultRegion().get(); | ||
|
|
||
| sinkMetrics = new DefaultSinkMetrics(pluginMetrics, "metric"); | ||
| httpSender = new PrometheusHttpSender(awsCredentialsSupplier, prometheusSinkConfiguration, sinkMetrics, prometheusSinkConfiguration.getConnectionTimeoutMs(), prometheusSinkConfiguration.getIdleTimeoutMs()); |
There was a problem hiding this comment.
Let's start moving toward dependency injection. This is support now. You can see an example of this in PR #6190. I added DI to the DynamoDB source coordinator.
There was a problem hiding this comment.
I will do this in a fast-followup PR
| import static org.mockito.Mockito.mock; | ||
| import static org.mockito.Mockito.when; | ||
|
|
||
| public class PrometheusSinkTest { |
There was a problem hiding this comment.
We should have at least one test that uses the test plugin framework to verify behavior. See GrokProcessorIT as an example. At the very least it will get the automatic tests that verify that the plugin can load.
There was a problem hiding this comment.
I will do this in a fast-followup PR
4 participants
Description
Add Prometheus Sink.
Only supports Amazon Promethus Sink as the destination with AWS credentials.
Issues Resolved
Resolves #3028
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.