Cart

Cloud Status Dashboard

This page provides status information on the services that are part of Cloud Platform. Check back here to view the current status of the services listed below.

Google BigQuery Incident #3605126490861261353

Multi-region/US: BigQuery queries may show latencies during execution.

Incident began at 2021-10-27 02:00 and ended at 2021-10-30 12:00 (all times are US/Pacific).

Date Time Description
Nov 01, 2021 12:27

We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support. (All Times US/Pacific) **Incident Start:** 27 Oct 2021 02:00 **Incident End:** 30 Oct 2021 12:00 **Duration:** 82 hours **Affected Services and Features:** Google BigQuery **Regions/Zones:** US Multi-region **Description:** Google BigQuery service performance experienced an increase in latency for queries in multiple regions in the US for 82 hours. From preliminary analysis, the root cause of the issue was capacity increase that caused rebalancing of workload and increased the scheduler processing time. This caused delays in query execution. **Customer Impact:** - Queries exhibited higher than normal latency. - Customers were unable to fully utilize their reserved slots.

Oct 28, 2021 17:58

The issue with BigQuery queries showing latencies, has been resolved for all affected projects as of Thursday, 2021-10-28 17:45 US/Pacific. If you have any further questions or are still impacted, please open a case with the Support Team and we will work with you until the issue is resolved. We thank you for your patience while we worked towards resolving the issue.

Oct 28, 2021 16:01

Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Mitigation work is progressing and our Engineering teams continue to monitor the system performance. We do not have an ETA for mitigation completion at this moment. We will provide more information by Thursday, 2021-10-28 18:00 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time.

Oct 28, 2021 14:49

Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Mitigation work is progressing and our Engineering teams continue to monitor the system performance. We do not have an ETA for mitigation completion at this moment. We will provide more information by Thursday, 2021-10-28 16:00 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time.

Oct 28, 2021 13:18

Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Engineering has confirmed that the mitigation work is progressing and we are continuing to monitor the system performance. We do not have an ETA for the mitigation of the impact. We will provide more information by Thursday, 2021-10-28 15:00 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time.

Oct 28, 2021 11:33

Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Mitigation work is still underway by our engineering team. The issue is affecting the US region and is currently under investigation. We do not have an ETA for the mitigation of the impact. We will provide more information by Thursday, 2021-10-28 13:30 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time.

  • All times are US/Pacific