I am currently working on a project that involves leveraging the BigQuery - External Connections feature to access data stored in an AWS S3 bucket. However, I have encountered a challenge and would appreciate any help or guidance from the community.
Here is the specific query I am trying to run in BigQuery:
SELECT * FROM EXTERNAL_QUERY("data-test.aws-us-east-1.sample_data_source", "SELECT * FROM INFORMATION_SCHEMA.TABLES;");
Unfortunately, when attempting to execute this query in BigQuery console, I encounter the following error:
Invalid table-valued function EXTERNAL_QUERY External database credentials not found for connection data-test.aws-us-east-1.sample_data_source at [1:15]
Despite my best efforts, I am facing difficulties in resolving this error and establishing a successful connection between BigQuery and the AWS S3 bucket. If anyone in the community has experience with BigQuery External Connections and connecting to AWS S3, I would greatly appreciate any insights, tips, or guidance you could provide.
Before posting this question, i went through this documentation : https://cloud.google.com/bigquery/docs/omni-aws-create-connection, but with no luck.
Specifically, if there are key considerations, best practices, or common pitfalls to be aware of, I would be grateful for that information. Additionally, given the error message mentioned above, if there are specific steps or configurations that need attention regarding external database credentials.
Thank you in advance for your time and assistance.
LinkedIn
Twitter
