Question

Looker Data Dictionary


Userlevel 5

Let’s talk about the Data Dictionary!


It would be good to gather feedback and requests in one place. I think it would be great addition in the future.


My humble requests and observations:




  1. I really like the UI. It’s clean and yet has a lot of information. Would prefer the list of fields to be more compact (in general I use 80% zoom anywhere in Looker)




  2. Optional display of fields from JOINs. Otherwise each explore gets big with repetitive metrics. I have two explores that are joined to everything. No need to show their fields on each page, they have their own page.




  3. Showing the explore fields first then the JOINs’. If it happens that my explore starts with N but join starts with A, the join fields will be first. I think the order should go with the importance and then alphabet. If I have a join in each explore that starts with A, then it will look like it’s the same page everywhere (until people start scrolling a lot).




  4. Ability to add descriptions that only live in Data Dictionary not LookML. There are things I want to add but not to be visible in the UI.




  5. It would be amazing to have a different view that groups items by the group first. Again it’s all to make it more compact upon first entry and then allow people to find what they need.




  6. Show dimensions group as one field not each timeframe separately




  7. Filter for hidden fields




As you can see for me the most important part is that the list is as compact as possible. Good UI and search invite people to explore, which is great but they can get overwhelmed upon first visit.


16 replies

Userlevel 2
Badge

Adding to the thread, it would be useful to add admin controls to the Data Dictionary.

For example, as an admin, I would like to limit certain parameters in the UI, so that end-users are only presented with a simple and easy to understand list of dimensions/metrics.

I’m aware of the view options dropdown in the data dictionary, but I would like to take it a step forward and unilaterally decide which fields end-users get to see or choose from.

 

Also, +1 for extended definitions that live in the data dictionary, not necessarily in the tooltip description.

@Dillon_Morrison 

I have a similar concern to @Dawid_Nawrot in regards to #4, it goes one step further, we don’t want to show any descriptions at all as tooltips in our dashboards. We had started using descriptions because of the data dictionary, when we realized they were in tooltips everywhere we had to comment out our descriptions and stop using the data dictionary. 

It’d be nice if we could have a “data dictionary only description” available in the LookML. Or the ability to disable description tooltips in our dashboards.

Also, one additional feature that would be great to have is a way to export the Data Dictionary to PDF and/or to a spreadsheet.

Userlevel 5

More of the first one. I think Data Dictionary should read LookML as it does but I don’t want to keep extensive descriptions in the description parameter.


The way parameter is shown in the UI of the explore makes sense, kind of. It’s a small i icon that you need to make a conscious action on to show the description. I used them if there are intricacies about the field, for example, something is excluded etc.


It doesn’t mean that the field itself is self-explanatory but that’s not the place to put big descriptions. Imagine having to hover with your mouse over each i to get this.


Data Dictionary to me should have its own separate layer where I can put long descriptions, show calculations, even attach images. If I don’t have to use Confluence for such things or other software, it would be magnificent.


Basically I would like Data Dictionary to be more than just a UI to read LookML. I want the LookML to be the source but for the Data Dictionary to enrich it much more than just parameters in the LookML.


Hope it makes sense.

Userlevel 4

@Dawid_Nawrot I’d love to learn more about item number 4 on your list - “Ability to add descriptions that only live in Data Dictionary not LookML. There are things I want to add but not to be visible in the UI.” Can you elaborate on your use-case a bit more here? I’m wondering if it’s either of these use-cases. Please correct me if I’m wrong 🙂



  • Are you, as a data modeler/developer, wanting to add a second definition (in addition to the LookML description)? This definition would only live in the Data Dictionary. If so, why does the core LookML description not suffice? It seems like it would be better to overwrite the core LookML description.


OR



  • Are you hoping that your non-developers (viewers, explorers, and anyone without access to LookML) are given the ability to add a second definition to a field. A definition/comment that would only live in the Data Dictionary UI. A definition/comment that you would not want in the LookML description, and would largely benefit the non-developers.Would you want any non-developer to have this capability, or would it be a specific permission set in your mind?

Userlevel 4

@Gordon thanks so much, appreciate the clarification.




  1. This request makes total sense. I’ve added to our roadmap, and is something we’re actively thinking through now.




  2. I agree with you that this would definitely be helpful functionality, and it’s something we really want to tackle eventually. We certainly though about this for the Data Dictionary as well, but came to the conclusion that the Data Dictionary UI isn’t the “best” place to solve for this. If you’re editing fields, you’re almost always in the model, in your IDE. In our view, a better workflow would be to see how a change in one field affects any other, straight from within the IDE. We’re planning on adding the ability to insert Extensions into the IDE sometime next year. At that point, I can see a new “lineage” tool being introduced for the IDE for this use-case specifically…



Thanks for getting back to me @Dillon_Morrison,




  1. Exactly right, so in the first example if you looked up the definition of profit and saw ${revenue} - ${cost} you wouldn’t need to search for revenue and cost to find those definitions. Ideally this could be through unfolding each of the fields as those ‘staging’ fields are also good to know!




  2. Yes, so to use the example above, it would be great to be able to see what fields elsewhere would be changed by a redefinition of ${revenue} whether that only impacts profit, or would also impact net_profit and Profitability both in that view or at a model level.



Userlevel 4

@Gordon I’d love to dig into the data “lineage” use-case a bit more here. Do either of these capture your goals accurately? If not, would you mind providing an example?




  • When viewing fields in the Data Dictionary UI, more easily understand the definitions of derived fields. For example, if I had a derived dimension equal to profit, you currently see a LookML definition of ${revenue} - ${cost} and you’d prefer someway to see the core table/SQL definitions of the revenue and cost fields as well? Something like ${TABLE}.revenue - ${TABLE}.cost.




  • When developing a LookML model, quickly understand how a change in one field (e.g. revenue) will impact all the other fields that are derived off it (e.g. profit)?



Userlevel 4

Hey There!


Dillon, PM for the Data Dictionary, here. We’re thrilled to see the discussion In broad strokes, it seems like there are a few common requests here. Please chime in if I’ve misunderstood, or if you have additional feedback!


SQL vs LookML Field Displays - Provide the LookML parameters that help define logic outside of the sql: parameter. The Data Dictionary already provides the options to view the SQL and the LookML definitions, but this loses all context when additional LookML parameters, such as filters: are introduced. This makes complete sense, we’ll think through the best way to display that information.


Compactness/Configuration - The ability to see more information on the page with compact formatting, and options around the ordering/display of tables and field groupings, for the purpose of making browsing through fields quicker. This is great feedback and something we’re working towards. Expect iterative improvements on this.


Adding Descriptions - Adding descriptions that either only live in the Data Dictionary UI, or that get pushed back to the underlying LookML. This is an interesting request that we’ve thought a lot about as well. Some of this functionality will be unlocked with the ability to save data within a data store specific to the app. We’re actively working on this and hope to introduce something in the Q3 timeframe (July-Aug).


Global Search - The ability to search for a field without specifying a Model or Explore first. We agree, this is very high on our list. The current search functionality requires users to have an understanding of Looker’s Model and Explore structure to find a field. We’re aiming to eliminate this friction, and make it quick and easy for a user to find a field without any prior knowledge of Looker’s model structure. We’re working through various implementation models as we speak. We’re hoping to land these improvements in the Q3 timeframe as well. Eventually, we’d love to tie additional metadata such as popularity or query/user count to these fields too.


Again, very helpful feedback. Please keep it coming!

@Dawid_Nawrot thanks for starting this thread! my company likes the data dictionary too, our biggest asks would be:



  • clustering/hiding repetitive date dimensions (there is no need to have a new definition for a date, month, year variable)

  • a global text search feature

  • all dimension types are auto-highlighted as filters, or there is an easy way to select/deselect all dimension types

Userlevel 5

To me, that’s the best call. LookML can be approached from the dev stream or the business stream, so I feel being able to see side-by-side LookML and/or full SQL would allow for easier entrance from either world.

Userlevel 3

yes, I think so too

Userlevel 5

I think both ways should be presented. LookML version is easier to read even for people who are not LookML developers and just want to see what filters are applied. The SQL, however, could be helpful for developers (?)

Userlevel 3

Good point.

yes and no

If this should really help, filters (which could also be a case when condition) would have to be resolved.

Or the recommendation should clearly go to a case when solution instead of the filter solution.

Userlevel 5

Interesting! Meaning it should be ‘unfurled’ to its expanded SQL before presentation?

Userlevel 3

Overall I like the Data Dictionary application very much.

It is clear and well structured.

But unfortunately it is missing one, for me, decisive “feature”


If I define a measure that way:


 measure: value_usd{
type: sum
sql: case when currency.currency = 'USD' then ${value} else nul end;;
}

then everything’s great.

but if I use the standard Looker syntax:


 measure: value_usd{
type: sum
sql: ${value} ;;
filters: [currency.currency: "USD"]
}

the information is lost.

That should be changed.

I like the clean interface, I was initially doubtful if there would be a huge benefit when users can look at the LookML if they are unsure but the search capability across views in explores and less intimidating view are big pluses to me and I see a lot of potential use cases.

My ideal (and likely impossible) enhancements would include:



  1. A way to see where/how often a field is used ideally in all dashboards but if not in LookML ones at least so you can ensure that you are using the same version of the metric and what impact any changes could have

  2. A way to see the lineage of a field inside the explore. I.e. the field is used in the derivation of these metrics in the lookml layer this would allow you to see the impact of changes and also if that field is still required


Also, fully agree on Dawid’s points, particularly hidden fields, and descriptions in Data Dictionary only.

Reply