I have created a BigQuery external table. It refers to a GCP bucket where I have csv files containing e-commerce product details. There are close to 1.5 million files; each file contains a single product detail in the form
<product_id>, <product_json>.
When I try to run a simple query, for example:
SELECT * FROM `<external-table>` WHERE product_id='0RCN5Z1Y9J64';
I get the error
Not found: Files gs://<bucket>/product_json/0V9QAEFA5VN0.csv (Version 1714969112548075)
What could be the possible reason? Each time, the file and version are different. Please note that data is constantly being added or files are getting overwritten while I run the query.
LinkedIn
Twitter
