Knowledge Drop

My LookML Validator is slow!

  • 5 April 2021
  • 2 replies
  • 239 views

Userlevel 4

Last tested: Jun 25, 2020

 

TL;DR:

The bigger the project is, the longer it takes to validate.

 

Quick wins:

  1. Be strategic about your "includes"
    1. Always including every other view may be easy for development but being strategic by using naming conventions to only include the necessary files will increase performance.
  2. Comment or delete unused explores (and joins, fields, etc)
  3. Consider splitting your project into multiple projects

Check out this article for LookML best practices.

 

Long answer:

LookML project size (total number of lines) and unnecessary includes both affect the performance of the LookML validator.
Validation is a two-step process; 1) the first pass of validation happens by loading every single file in the project, even the ones not referenced anywhere, into memory and does a full translation of the source code one model at a time looking for "compile" errors. 2) the second pass of validation starts at each model and then parses only the files referenced by that model. And then it builds a query based on that model and looks for query or "run time" errors. Also, it's important to note that, both of these passes happen for every validation.
To conclude, the first pass of validation is affected by the size of the LookML project and the second pass of validation by unnecessary include statements. So it's good to recommend more selective and strategic includes as well as keeping the size of the LookML project reasonable.


Bonus note:

The number of fields per explore is very important in terms of validation time. That often takes longer than parsing the model.

 

This content is subject to limited support.                

 


2 replies

Userlevel 4
Badge

Some other customers have found that clearing browser cache and using an incognito window has also helped alleviate LookML validation issues. Specifically, customers who noticed that this process never completes found that clearing cache resolved this issue for them. 

Userlevel 6
Badge

Will try it next time I need this thanks!

I can’t understand why there is not some kind of pre filter option to look at only content in a folder or content from a specific model or explore, it’s basically unusable without this.

Reply