When downloading queries, some queries could require large amounts of memory.
When downloading all results for some types of queries, the amount of memory required could cause performance issues, even crashing an instance. The All Results option is disabled for users with download_with_limit permissions but other users could download all results even for queries that have memory-intensive characteristics.
Starting in Looker 4.14, you can prevent these queries by turning off the Allow unlimited downloads that might crash Looker legacy feature. When the legacy feature is off, the All Results option is disabled for queries returning more than 50,000 rows in situations where the rows need to be stored in memory. This includes queries returning more than 50,000 rows with one of these memory-intensive characteristics:
- involving row totals or table calculations
- using percent_of_total, percent_of_previous, or running_total measure types
- getting data from a database dialect that can’t stream results
- including pivoted columns that Looker calculates because the data comes from a database dialect that can’t calculate the pivots
If All Results is unavailable for a query, users can still select the Custom Limits option with a limit of up to 50,000 rows. Starting in Looker 4.14.13+, the download UI limit has been increased to 100,000 rows (which applies whether or not the legacy feature is enabled). If you know that your instance has been provisioned to be able to export more than 100,000 rows of unstreamed results, you can further increase that limit with the
--max-unstreamed-limit startup option.
You’ll want to make sure your instance is allocating enough memory to the Java process for very large downloads. This doc has some general guidelines on memory settings: https://docs.looker.com/setup-and-management/on-prem-mgmt/java-memory-settings
The Allowing Unlimited Downloads legacy feature was removed in version 5.4, so if you are on an earlier version, you can turn this legacy feature on to allow All Results for unstreamed downloads if you are not worried about the possibility of a very large download crashing your instance. After 5.4, there is no way to remove the limit for unstreamed downloads, so in this case, I would recommend setting the --max-unstreamed-limit startup option to be large enough to accommodate all of your downloads.
Hi, how do we determine if our “instance has been provisioned to be able to export more than 100,000 rows of unstreamed results”?
Also, is there a way to remove the limit?
Could you send more info to help.looker.com so we can look at your case in more detail. What we need in particular is a look/explore URL and a user that tries to download the data.
This doesn’t seem to work on our instance. We have this legacy feature enabled, but users are still prevented from downloading All Results in Looks that contain table calculations.
Tried turning it off, also doesn’t help.