Got a question?
Ask your big data questions here.
- 83 Topics
- 195 Replies
Continuing the discussion from ETL Tool Recommendations: Hi @lloydtabb, thanks for the info! I saw this post as well: LookML For BigQuery (Deprecated) LookML Deprecated: This much of this post is about Google Legacy SQL, we recommend using Google Standard SQL. #LookML for Google’s BigQuery As of 3.34, Looker has improved support for Google’s BigQuery. BigQuery takes a higher degree of tuning in order to work with Looker. For example, you’ll need a little more awareness of data sizes in order to build your model. Query Size Estimator When running a query in a typical database, your query takes longer to return as the amount of data you query go… and the TABLE_DATE_RANGE feature makes me wonder what the best approach is. My first thought was just to append a current date column each day to the snapshot and keep appending to the same table, but your post here seems to suggest it is better to create a new table each day with the snapshot date in the t
I’ve begun exploration of creating a transformed database (data warehouse) for use with looker. Right now we just have Looker hooked up to a replica of our production DB, which has worked great in the short-term, but is showing some limitations. In order to not go into this too blindly I’ve been doing some reading on data warehouse best practices, including ‘The Data Warehouse Toolkit’, which from what I can gather is one of the must read books in this space. In this book they are quite adamant about the creation of a ‘Date’ table, instead of using date fields directly in your fact tables. Benefits include being able to assign dimensions to dates such as ‘holiday’, ‘quarter’, ‘weekend’, etc. that aren’t readily accessible in SQL. Of course, Looker does have some helpful date/time tools allowing easy grouping by month/week etc. So, as I begin down this road I wonder how important these classical warehouse approaches are when using a tool like looker, as it seems much of what looker does
At Looker we are often asked about best practices when it comes to designing a new data warehouse. Typically this happens just as when companies are moving to MPP, and maybe even, column-oriented, databases, so it is clear from the start that replicating the design from an operational database is not appropriate. It is also clear that there is not a single answer - search on Google turns up millions of results. That said, here are a few rules of thumb that you can apply as you focus on building your analytical data warehouse to work with Looker: simplicity (aka shortest path) performance single copy of data transparent EL process Shortest Path You should not need to use mapping tables or Entity–attribute–value tables to get to the value. The path to any two dimensions in one SELECT query should not involve more than a couple of joins. Typically “long path” designs arise from storing original data in NoSQL format. Because there is very little analytical value derived from performing s
Already have an account? Login
Login to the community
No account yet? Create an account
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.