This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Hi,I have a DQ job which essentially fetches data from cloud storage
bucket , checks the data quality and once success it persist the data to
BigQuery data set. I actually want to persist the success data to
another cloud storage bucket instead of Bi...
Hi ,i am having a scenario like only when Data quality check is success
i want to persist the data from pub/sub to GCS/Bigquery. I dont want the
data which fails the DQ validation into GCS/BigQuery. Are there any GCP
services available which will do ...
Hi,Need some info on the below points. Please suggest.1. Can data
catalog collect the actual data from Pub/ Sub topics ? Or it can be just
used to collect only the metadata of Pub/Sub topics.2. Data catalog
refers collecting two kinds of metadata nam...
Hi,Is it possible to stream data from pub/sub to cloud data quality ? I
have seen docs saying that we need to create a data lake to get it
consumed by cloud data quality. I want to directly consume it from
pub/sub topic without creating any data lake...
Hi,I am trying to collect Pub/Sub data to Cloud DQ for data quality
check. But i can see in the docs like in cloud DQ , there is option to
create only data lakes and associating data to that data lakes and doing
the data quality check with that data....
Hi,I had a look at these Dataplex templates but this dataplex wants data
to be structured like a GCS/Bigquery data as an input and not directly
from Pub/Sub. Actually i want to persist to GCS/Bigquery only if DQ
validation is success. Please suggest.
Hi, One query regarding the below one, may be it might be basic one .
Please clarify.Column Names and Data Types: Clarifies database table
attributes and formats.How data catalog collects this from a pub/sub
topic ? Is it like pub sub topic sends thi...
