Continuously receiving "internal error" issues when trying to create a BigQuery model

[shanif]

Hey everyone, first time poster so I apologize in advance for any issues with this posting!

My company has been using BigQuery to create and use ML models for about 10 months now. We train these models using transaction data on ecommerce stores.

Recently, we've onboarded a couple of very large stores that have a lot of data, and when trying to create a new model for one of these stores, we are continuously getting the message (screenshot below):

Error Running Query

An internal error occurred and the request could not be completed. This is usually caused by a transient issue. Retrying the job with back-off as described in the BigQuery SLA should solve the problem: https://cloud.google.com/bigquery/sla. If the error continues to occur please contact support at https://cloud.google.com/support. at [271:5]

Normally with BigQuery errors I can find the cause of the issue using the error message, but unfortunately that's not possible here. I've tried several things, like reducing the amount of data used to create the model, reducing the number of features in the model, but unfortunately I can't isolate the issue.

Is someone able to provide any additional details on what's going wrong?

Thanks so much!

[dikaur]

Hello Shanif,

This is an internal error [1] that can occur within BigQuery, for example in streaming ingestion service. It might be overloaded when you ran your query and therefore it could not serve the data that you were trying to ingest for your table.

According to the backoff requirements [2], you need to wait for a period of time before issuing another request. 

[1] https://cloud.google.com/bigquery/docs/error-messages#errortable

[2] https://cloud.google.com/bigquery/sla#:~:text=%22Back-off%20Requirements

[mohamedlaalays]

How long is the wait? I have been getting this error for 5 days

[paullear]

From my experience, the issue is due to BigQuery resource binding and availability. It is not as simple as waiting a minute to execute a query a second time, but rather the query itself attempts to access a resource too soon.  My "solution", short of seeing clear documentation from Google how to inject "wait for resource" SQL commands, was to create temporary tables for sections of the original query, then use those temporary tables once created, folding them back into the desired output. 

[asinghal]

I am also getting the same error but when creating a table from   a csv located in Google cloud storage. The csv is big ~143GB.

Did anybody resolve it in any systematic way?