Question

JOIN 2017 - Deep Dive - To Use or Not Use PDT's

  • 11 September 2017
  • 2 replies
  • 428 views

Why is a pragmatic balance between PDT’s and ETL important?





  • Makes your database resources more efficient and effective


  • Improves the end user experience


  • An optimized data pipeline provides stable infrastructure for scaleable analytics




1. When to EDT, PDT or ETL



EDT





  • Real-time data


  • Quick to query


  • Dynamic Builds




PDT





  • Data freshness


  • Available database resources


  • Prototyping




ETL





  • Powerful ETL tool - Discourse Discussion



    • To name a few: Matillion, Talend, Alooma, ETLeap, Keboola, DataVirtuality, Xplenty






  • Well understood PDT


  • Consistently used PDT


  • PDT used outside of Looker




2. Know your database






Discourse Discussion



3. Optimize your database to improve PDT development and ETL performance



Fabio’s Deep Dive - Redshift Optimization with Looker’s Redshift Block presentation provides specifics for Redshift as well as general database optimization concepts and techniques that apply to a wide range of databases






Looker Analytic Block






You can find my JOIN 2017 presentation [here](https://docs.google.com/presentation/d/e/2PACX-1vSsKcPJfT3kmWPY6WoGFTW2sBaHXAXGkKTAz5yUVdOi-mlDenU_BQbsmc9AWlatvJkrrBSffkyCPtC4/pub?start=true&loop=true&delayms=15000).


2 replies

Jonathon,



It was mentioned during this session at JOIN that you could provide your internal LookML guidelines for us to use/build on. How do we go about getting that?

Hey @remenaker! Thanks for the question and for reaching out. I hope you had an awesome time a JOIN 2017!



Here is a set of example development guidelines. We hope this will provide a good starting place.



Happy developing!

Reply