Question

JOIN 2017 - Deep Dive - To Use or Not Use PDT's

  • 11 September 2017
  • 2 replies
  • 349 views

Why is a pragmatic balance between PDT’s and ETL important?



  • Makes your database resources more efficient and effective

  • Improves the end user experience

  • An optimized data pipeline provides stable infrastructure for scaleable analytics


1. When to EDT, PDT or ETL


EDT



  • Real-time data

  • Quick to query

  • Dynamic Builds


PDT



  • Data freshness

  • Available database resources

  • Prototyping


ETL



  • Powerful ETL tool - Discourse Discussion

    • To name a few: Matillion, Talend, Alooma, ETLeap, Keboola, DataVirtuality, Xplenty



  • Well understood PDT

  • Consistently used PDT

  • PDT used outside of Looker


2. Know your database



Discourse Discussion


3. Optimize your database to improve PDT development and ETL performance


Fabio’s Deep Dive - Redshift Optimization with Looker’s Redshift Block presentation provides specifics for Redshift as well as general database optimization concepts and techniques that apply to a wide range of databases



Looker Analytic Block



You can find my JOIN 2017 presentation [here](https://docs.google.com/presentation/d/e/2PACX-1vSsKcPJfT3kmWPY6WoGFTW2sBaHXAXGkKTAz5yUVdOi-mlDenU_BQbsmc9AWlatvJkrrBSffkyCPtC4/pub?start=true&loop=true&delayms=15000).


2 replies

Jonathon,


It was mentioned during this session at JOIN that you could provide your internal LookML guidelines for us to use/build on. How do we go about getting that?

Hey @remenaker! Thanks for the question and for reaching out. I hope you had an awesome time a JOIN 2017!


Here is a set of example development guidelines. We hope this will provide a good starting place.


Happy developing!

Reply