Question

LookML examples to implement data tests

  • 25 November 2021
  • 0 replies
  • 7 views

New to looker and would like to implement some tests for ETL validation (examples below from @rebecca ‘s post on this topic) 

This document has some limited examples...https://docs.looker.com/reference/model-params/test
 

Does anyone have examples they can share of these? Especially interested in #5 and how to write a test with queries. I’d likely be counting, summing, taking the max of fields and comparing them to other time periods or current date. 

Tests for ETL validation:

  1. Data exists: count measure is greater than 0
  2. Data is sufficient: count measure is greater than or equal to…
  3. Data is not redundant: count of primary key field is equal to 1 [the Docs page shows an example]
  4. All data is available: count or sum measure is equal to…for some filters [picking a fixed timeframe in the past works best]
  5. Data is recent: max of some timestamp is greater than or equal to… [You can do something like DATE_ADD(current_date(), interval -1 day) so you are always getting “yesterday” without hard-coding a date here]

0 replies

Be the first to reply!

Reply