Cart

Cloud Status Dashboard

This page provides status information on the services that are part of Cloud Platform. Check back here to view the current status of the services listed below.

Cloud Spanner Incident #16591718137333047668

us-central1, us-west1, us-west1, us-east1: Cloud Spanner missing stackdriver CPU, memory and disk metrics.

Incident began at 2021-11-12 18:45 and ended at 2021-11-13 20:39 (all times are US/Pacific).

Date Time Description
Nov 15, 2021 13:26

We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support. (All Times US/Pacific) **Incident Start:** 12 November 2021 18:45 **Incident End:** 13 November 2021 20:39 **Duration:** 1 day, 1 hours, 54 minutes **Affected Services and Features:** - Cloud Spanner - Missing/Incorrect Cloud Monitoring metrics for CPU, memory and disk **Regions/Zones:** - us-central1 - us-east1 - Global **Description:** Cloud Spanner experienced unavailable and incorrect monitoring metrics for instance resources including cpu, memory and disk for a duration of 1 day, 1 hours and 54 minutes. From preliminary analysis, the root cause of the issue is a rollout flag which incorrectly modified how metrics are exported. **Customer Impact:** - us-central1: Experienced missing metrics during the impact duration until mitigation at 13 November 2021 18:30 - us-east1: Experienced missing metrics during the impact duration until mitigation at 13 November 2021 17:45 - All other regions: Many customers globally experienced metrics that incorrectly under-reported actual values until full resolution at 13 November 2021 20:39

Nov 13, 2021 21:17

The issue with Cloud Spanner has been resolved for all affected users as of Saturday, 2021-11-13 21:16 US/Pacific. We thank you for your patience while we worked on resolving the issue.

Nov 13, 2021 16:47

Summary: us-central1, us-west1, us-west1, us-east1: Cloud Spanner missing stackdriver CPU, memory and disk metrics. Description: Mitigation work is currently underway by our engineering team. Current data indicates that customers in us-central1, us-west1, us-west1 and us-east1 are affected by this issue. We will provide more information by Saturday, 2021-11-13 23:00 US/Pacific. Diagnosis: Missing Stackdriver metrics on the dashboard. Workaround: None at this time.

  • All times are US/Pacific