Skip to content

Conversation

@the-mikedavis
Copy link
Collaborator

The at-least-once dead lettering feature for QQs can cause a pile-up of dead-lettered messages in a QQ when the dead-letter routing topology leads to no viable dead-letter queues. When this happens the dead-letter worker logs at warning level. When you adjust the routing topology so that pending messages can be delivered we should log once at info level as well. This way you can confirm in the logs that you've successfully fixed your routing topology. Otherwise you would only be able to know by seeing the message stats change for dead-lettered messages in the source QQ and available messages in the destination dead-letter queue.

Discussed in #14494
There's more we could do to make unroutable messages more visible, but this is some nice low-hanging fruit.

The at-least-once dead lettering feature for QQs can cause a pile-up of
dead-lettered messages in a QQ when the dead-letter routing topology
leads to no viable dead-letter queues. When this happens the dead-letter
worker logs at warning level. When you adjust the routing topology so
that pending messages can be delivered we should log once at info level
as well. This way you can confirm in the logs that you've successfully
fixed your routing topology. Otherwise you would only be able to know by
seeing the message stats change for dead-lettered messages in the source
QQ and available messages in the destination dead-letter queue.
@the-mikedavis the-mikedavis requested a review from ansd November 15, 2025 16:40
@the-mikedavis the-mikedavis self-assigned this Nov 15, 2025
case RouteToQs of
[] ->
State1;
log_no_route_once(State1);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you delete your DLX and then try to forward a dead-letter message, you'll get the warning about the DLX not existing. If you then create the DLX but don't yet have a binding that makes the route to the DLQ viable, we previously weren't logging the no-route warning. I think the old code was assuming that, because we're redelivering in this function, we would've already warned about no-route.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants