Dimension to Handle Array Type data in Lookml

Hi,
I have a table >500B rows. DBA has put a rule that the table can only be queried with a where condition on date field. One of the column I'm interested in has line items separated by comma, and within this line, the data is separated by semi-colon
it looks something like below:
";PRD11942;;;226=29.70;118=19.32|122=922576 , ;PRD322567;;;226=29.70;118=10.21|122=97653 ,"
and there could be n number of line items.
It has values as amount, count and many more. I need to be able to create dimension for some of the values and measure for some. 

I have currently used derived_table, defined the logic to extract the values using un-nest and split to create dimensions and measure, but since the table is huge it takes quiet some time to process.
Moreover, the dimensions and measures that needs to be un-nested and split isn't used that frequently. Another problem is that I need to use liquid parameter to get user's date input.

To reduce processing time and over complicating the view, I am looking to use sql_table_name so the join against un-nest and split only happens when these dimension-measures are used in analysis, and the date filter can be easily applied as well.

Is there a way I can do that? Any ways to process array type data in looker or any suggestions to handle such large dataset?
Help is much appreciated.

0 REPLIES 0
Top Labels in this Space
Top Solution Authors