-
-
Notifications
You must be signed in to change notification settings - Fork 7k
Add Observability page to docs #9723
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
itssimon
wants to merge
7
commits into
encode:master
Choose a base branch
from
itssimon:docs-topics-observability
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
b3f6138
First draft
itssimon 5288b05
Tweaks
itssimon 4e073c1
Fix formatting
itssimon 45cfc66
Fix formatting again
itssimon 8ca33c6
Move Prometheus section below OpenTelemetry
itssimon 04a0871
Move Observability to bottom of Topics menu
itssimon 6574c6c
Remove django-silk
itssimon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
# Observability | ||
|
||
> Measurement is the first step that leads to control and eventually to improvement. If you can't measure something, you can't understand it. If you can't understand it, you can't control it. If you can't control it, you can't improve it. | ||
> | ||
> — H. James Harrington | ||
|
||
REST framework doesn’t include built-in observability features, but it works well with popular tools, standards, and third party packages. For simpler needs, you can also implement custom logging or metrics collection using Django middleware. | ||
|
||
## Custom request logging | ||
|
||
You can implement a custom middleware that logs relevant information about handled API requests and responses using Python's built-in `logging` module. | ||
|
||
```python | ||
import logging | ||
import time | ||
|
||
logger = logging.getLogger("your_app.requests") | ||
|
||
|
||
def request_logging_middleware(get_response): | ||
def middleware(request): | ||
start_time = time.time() | ||
response = get_response(request) | ||
duration = time.time() - start_time | ||
logger.info( | ||
f"{request.method} {request.path} - {response.status_code} {response.reason_phrase} - {int(duration*1000)}ms" | ||
) | ||
return response | ||
|
||
return middleware | ||
``` | ||
|
||
Then, add the middleware to your Django settings. | ||
|
||
```python | ||
MIDDLEWARE = [ | ||
"your_app.middleware.request_logging_middleware", | ||
# ... other middleware | ||
] | ||
``` | ||
|
||
## OpenTelemetry | ||
|
||
[OpenTelemetry](https://opentelemetry.io/) is an open-source framework for collecting distributed traces and metrics from applications. It provides a vendor-neutral standard for instrumenting code and exporting telemetry data. | ||
|
||
In Django applications, OpenTelemetry can be used to trace requests through views, middleware, and database operations. The [opentelemetry-instrumentation-django](https://pypi.org/project/opentelemetry-instrumentation-django/) package automatically instruments Django and integrates with the wider OpenTelemetry ecosystem. | ||
|
||
The collected data can be exported to any OpenTelemetry-compatible backend for visualization and analysis. Examples include: | ||
|
||
- Jaeger | ||
- Grafana Tempo | ||
- Datadog | ||
- Elastic APM | ||
|
||
## Prometheus | ||
|
||
[Prometheus](https://prometheus.io/) is an open-source monitoring system that collects metrics by scraping HTTP endpoints exposed by applications. It stores the data in a time series database and supports flexible querying and alerting. | ||
|
||
For a REST framework project, Prometheus can be used to track metrics such as request counts, error rates, and latency. The [django-prometheus](https://pypi.org/project/django-prometheus/) package adds the necessary instrumentation and exposes a `/metrics` endpoint that Prometheus can scrape. You can also add your own application-specific metrics. | ||
|
||
Prometheus can be paired with [Grafana](https://grafana.com/) to visualize metrics with interactive charts and dashboards. | ||
|
||
## Other third party packages | ||
|
||
### Apitally | ||
|
||
[Apitally](https://apitally.io/) is a simple API monitoring, analytics, and request logging tool with an integration for Django REST framework. It provides intuitive dashboards with metrics and insights into API requests, errors, performance and consumers. | ||
|
||
The [apitally](https://pypi.org/project/apitally/) package integrates with Django applications through middleware, which automatically captures metrics for API requests and responses, and synchronizes with Apitally in the background. See DRF-specific setup guide [here](https://docs.apitally.io/frameworks/django-rest-framework). | ||
|
||
### django-health-check | ||
|
||
[django-health-check](https://pypi.org/project/django-health-check/) provides a set of health check endpoints for Django applications. It allows you to monitor the status of various application components including database connections, cache backends, and custom services. | ||
|
||
The library is helpful for production deployments as it enables load balancers, orchestration systems, and monitoring tools to determine application health. It supports both simple health checks and detailed status reporting for different application components. | ||
|
||
### Sentry | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. They're sponsoring us (I think), so that might be a good enough criteria to add them here. While you can self-host, it's also primarily a paid (albeit relatively cheap) service |
||
|
||
[Sentry](https://sentry.io/) is an error monitoring platform that provides real-time error tracking and performance monitoring for applications. It automatically captures exceptions, tracks performance issues, and provides detailed context to help debug problems in production. | ||
|
||
For Django REST framework applications, Sentry can monitor API errors, slow endpoints, and database query performance. The [sentry-sdk](https://pypi.org/project/sentry-sdk/) package provides automatic instrumentation for Django views, middleware, and database operations. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I appreciate that this is your baby 😃 but I'm a bit reluctant to place some paid services into our docs. If we start to add this one, why not add others as well? Where do we draw the line and stop adding? I don't really fancy maintaining an awsome-observability list as part of the docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I completely understand the concern. That said, I'd argue that Apitally is highly relevant in this context, offers DRF-specific integration and docs, and aligns closely with the needs of the audience likely to visit this page. It's a paid service, but very affordable – similar to Sentry.
I completely understand the need to draw a line in terms of linking paid services though, especially as that's a benefit offered to organizations that sponsor DRF. To help with that, I'll sign Apitally up to a corporate funding plan for DRF. That way, inclusion can be based on clear criteria: