Incident closed alert with no summary or condition

[tc-rm]

Hi, we use GCP monitoring for alerting on various metrics and log based events in a kubernetes deployment. We typically alert into both a MS Teams channel and an email.

We have recently, since early November, been getting alerts into only the MS Teams channel for an incident with no details, stating "Incident closed for "-".", and with no condition and no link to view the incident.

We are struggling to work out what is creating these. Would anyone have any idea as to what might be generating these alerts or where we can look in GCP to see what might be triggering them?

There is no pattern to the time when they alert. Most of our alerts aren't configured to notify on closure.

Thanks in advance.

[lawrencenelson]

Hi @tc-rm,

Welcome to the Google Cloud Community!

Could you please check your Logs Explorer? Try monitoring your workload and wait for the same incident to trigger an alert in your channel, then look for a corresponding log in the same timeframe. Additionally, double-check your metrics and alert policies. Also, inspect your Kubernetes deployment at the time of the incident by using the command kubectl get events.

If you continue to receive empty notifications randomly, consider recreating the log-based alerts, as this may be due to a glitch on GCP's end.

You can always contact Google Cloud Support to further look into your case. Thank you!

[tc-rm]

Hi thanks for your response. It looks like this relates to notifications from Error reporting not alerting. These teams alerts with no detail correspond with new Error groups, and also correspond with emails received as an alternative notification channel.

Is there any thing we can do to get more detail relating to this into the MS Teams alert? The email does contain relevant detail.