Desktop Notifications stopped working, should have steps to debug and test option

[ziesemer]

I previously had the desktop notifications working well. Somewhere along running OS updates and a reboot, they have stopped working. I am still seeing new Events and Incidents under Notifications in the web UI, as well as getting emails generated by Notifications - but no further Desktop Notifications.

I know this is in "experimental" status, and am reporting this for the requested feedback.

systemctl status icinga-desktop-notifications shows as active (running), with no errors.

I am using Firefox. The developer console shows only an initial error on page load:

The connection to https://.../icingaweb2/notifications/v1/subscribe was interrupted while the page was loading.

... which also appears under the "Network" tab with "Transferred" shown as NS_BINDING_ABORDED - though then further calls to the URL appear as successful (200), with "Transferred" shown as service worker. Under the "Application" tab, I also see the service worker for notifications-worker.js shown as "Running".

The "Send notifications" permission is enabled in my browser. I also disabled and re-enabled the "Enable notifications" option under my account preferences.

Would be curious for any further troubleshooting steps that can be provided - either here and/or in further documentation.

It seems that there should also be an easier way to send a test notification, shy of forcing a service check to fail. Even when it was working, I had hoped that "Send Custom Notification" on a service would come through - but at least at the time, it appeared that custom notifications did not propagate through here.

Reference:

• https://icinga.com/docs/icinga-notifications-web/latest/doc/06-Desktop-Notifications/

[nilmerg]

Hi, sorry for the late reply.

Do you still have the issue?

Firefox has a notorious habit of being very picky about what is allowed and what not. Since desktop notifications are only supported by using https, are you using an officially certified certificate, an internally certified certificate or a self signed one?

If internally certified or self signed, the upgrade you mentioned may have reset the trust store Firefox relies on.

To investigate, you can open /icingaweb2/notifications/v1/subscribe in a tab manually and see what's going on. Hopefully Firefox shows you a more detailed error then.

[ziesemer]

Thank you. Again, I recognize this is still in an "experimental" status, and only looking to help further along what should be a great feature here.

It certainly has been hit-and-miss in terms of ongoing notifications. I'm not sure it is even limited to Firefox. And just when I've mentally written it off, I'm surprised to see some notifications come through... I've had the icinga-desktop-notifications.service go into a failed status at least once, and had to restart that. Though nothing in its logs for the past 5 days and appears to be running, but I haven't seen anything recently. I've been working through fine-tuning my service thresholds, but they're nowhere near perfect yet. According to /icingaweb2/notifications/events, there were critical events as recent as this morning that I did not receive as a notification, but did receive by email generated by the same notifications.

Yes, I'm on HTTPS using a certificate signed by an internal certificate authority and is fully trusted by the browser.

I'll keep an eye on /icingaweb2/notifications/v1/subscribe. At present, Firefox's developer tools show it returning a HTTP/2 200 OK, with a response content-type of a text/event-stream.

What I think would be most helpful would be help in determining the easiest way to send a test notification. Anything shy of forcing a service check to fail? It appears that "Send Custom Notification" on a service does not propagate here? Even if there is something I could generate or execute directly on the server to send a test and to assist with timely troubleshooting? Thanks!

[nilmerg]

The notifications daemon does not log by default, pass it the switch --verbose or --debug to see it. Not sure though, if error logging is also dependent on that. Normally it's not, so I guess you should take a look at it with journalctl the next time it fails.

You can setup another source, and process an event by using the HTTP API. It's not a push-a-single-button test, but at least something side-effect free. 😛