Looker Default caching policy

[rsingh]

Hello Looker Community,

I am trying to understand how Looker default caching works. This is what I have understand so far

1. Default caching policy is 60 minutes
2. When user runs a query, if this query exist in cache, then looker will return the results from cache instead from the database directly
3. If query does not exist in cache, then results will be fetched from the database directly and results will be stored in cache as well
4. In parallel, Looker monitor each query timer in cache and invalidates (delete) cache for the queries every 60 minutes. It will ensure that cache storage is optimized

Same steps (above) applies if we use ‘persist_for’ 

Did I understand correctly?

Much appreciated! 🙂

Thank you

Raman

[Eric_Lyons · Accepted solution]

Hey @rsingh,

That summation of the caching policy sounds spot on. This article adds a few more details. Persist_for will let you override the default caching policy on the specific objects e.g. explore or models in which it is applied to. This article describes how we can apply a persist_for parameter. Model/explore persist_for control how long the query previously run in Looker is kept in memory.

I would also highly advise looking at our data groups and persist_with. These can be more precise and be reused on multiple objects. 

Please let me know if you have any questions!

Thanks,

Eric

[rsingh]

Hello @Eric_Lyons 

Thank you very much for the confirmation. I also tried to read about data groups with sql_trigger and this is what I understand

For explorers and models

• When user runs a query(Q1), if this query exist in cache, then looker will return the results from cache instead  from the database directly
• If the query does not exist in the cache (eg: new query), then results will be fetched from the database directly and results will be stored in the cache as well
• In parallel, Looker will constantly check in database for fresh results (‘sql_trigger’) and if values changes, it will reload cache with fresh results
• Also in parallel, max_cache_age (eg: 24 hours) perimeter will ensure that cache gets invalidated (deleted the query) to optimize cache storage. It means after 24 hours, the earlier user query (Q1) will be removed from cache and will be treated as ‘new query’ next time when user tries to run it

Did I summarized it correctly? 

Thank you

Raman