Why is a pragmatic balance between PDT’s and ETL important?
- Makes your database resources more efficient and effective
- Improves the end user experience
- An optimized data pipeline provides stable infrastructure for scaleable analytics
1. When to EDT, PDT or ETL
- Real-time data
- Quick to query
- Dynamic Builds
- Data freshness
- Available database resources
- Powerful ETL tool - Discourse Discussion
- To name a few: Matillion, Talend, Alooma, ETLeap, Keboola, DataVirtuality, Xplenty
- Well understood PDT
- Consistently used PDT
- PDT used outside of Looker
2. Know your database
3. Optimize your database to improve PDT development and ETL performance
Fabio’s Deep Dive - Redshift Optimization with Looker’s Redshift Block presentation provides specifics for Redshift as well as general database optimization concepts and techniques that apply to a wide range of databases
Looker Analytic Block
You can find my JOIN 2017 presentation [here](https://docs.google.com/presentation/d/e/2PACX-1vSsKcPJfT3kmWPY6WoGFTW2sBaHXAXGkKTAz5yUVdOi-mlDenU_BQbsmc9AWlatvJkrrBSffkyCPtC4/pub?start=true&loop=true&delayms=15000).
It was mentioned during this session at JOIN that you could provide your internal LookML guidelines for us to use/build on. How do we go about getting that?
@remenaker! Thanks for the question and for reaching out. I hope you had an awesome time a JOIN 2017!
Here is a set of example development guidelines. We hope this will provide a good starting place.