Looker provides the ability to create custom visualizations using almost anything you want. This is very helpful for times where you need something custom, but the development process is quite lacking. Basically, Looker wants you to provide a reference to your single js file in the admin page so it has the ability to load it up when it needs to. Once the reference has been registered in Looker, it can then be used in an Explore to build out your visualization.
This process presents some issues…
Versioning: There isn’t any versioning of the referencedjsfile unless you maintain the file in a repository and have some kind of continous integration process in place. Looker just trusts that the url reference to thejsfile is valid. Since you’re not uploading an actual file to Looker, there isn’t any history to be tracked.
Being able to debug: Since thejsfile is loaded up by the url reference at runtime, you aren’t going to debug it very easily. Even more so, if you’re optimizing the code (as you should be) before you upload it to wherever you’re storing it, it’s going to be minimized code which makes the debugging process even worse (yea, there’s alwaysconsole.log();anddebugger;if you need to actually step through something). It’s just not ideal when I should be able to set my own breakpoint within my IDE.
Speed of development: The process of the development is what I find to be the most lacking. Basically, you will need to make your changes locally, save the file, run whatever bundle process you have and then re-upload it to wherever you’re storing it. After all that, then you can go and refresh the page and your changes should be live. It’s quite tedious and could take all day to develop a simple feature.
In this discourse, I’ll explain how to leverage modern technologies such as Webpack, Webpack’s Dev Server, TypeScript, and SCSS. This addresses most of issues stated above with the exception of versioning (since that is more custom to how you manage your code and where you keep it).
I will say that internally we use Visual Studio Team Services along with Microsoft Azure to store the finalized files. The continuous intregration peice makes this entire process a breeze and feels like a standard software lifecycle development process.
For those that would rather just see an example, check out the this GitHub repo.
Project Setup
The very first thing you’re going to want to do is create an empty project with npm initialized so we can leverage npm packages.
After your project is setup, install the following npm packages (most of these packages are only important during development time so make sure you’re using the appropriate --save-dev flag so that these don’t get bundled up with your production build:
npm install webpack --save-dev
npm install webpack-cli --save-dev
npm install webpack-dev-server --save-dev
npm install typescript --save-dev
npm install ts-loader --save-dev
npm install style-loader --save-dev
npm install css-loader --save-dev
npm install sass-loader --save-dev
npm install node-sass --save-dev
npm install d3 --save
NOTE: d3 is needed because Looker references d3 in their utils.ts file. You could just do away with this if you weren’t using it and remove it from their utils file. But chances are, you’re going want either Highcharts, d3 or some other charting library.
After your packages are installed, create your config files and paste in the content (below):
webpack.config.js
tsconfig.json
Now, you are setup (for the most part) to have TypeScript transpile on change and webpack to watch for changed files so it continuously bundles and updates as you change code. The next step is to add the looker types and utils files (You can learn more about these files from their custom vis repo on GitHub ).
Project setup for source files
Example project setup (or reference this):
- my-project
- src
- common
- types.ts
- utils.ts
- visualizations
- my-custom-viz.ts
- common
- src
- package.json
- tsconfig.json
- webpack.config.js
After the project has been created, you can now begin the build process so we can get the file we will reference within Looker. Looker requires the reference to the script to be https and not http Luckily, The Web Pack Dev Server allows us to tell to run on https and it will automatically generate the certifcate and everything for you. In the webpack.config.js, you can see that the devServer property has https: true and is running on port 3443.
In the package.json file, you can add a new npm script so you can launch the web server with little effort:
"scripts": {
"start:dev": "webpack-dev-server"
}
If you’ve added the script to the package.json correctly, you should now be able to run the command npm run start:dev and your web server will start right up (thus making your js file available so that Looker can load it up).
Adding the local file reference to Looker
- Navigate to the Admin page in Looker
- Select Visualizations from the left nav
- Select the Add visualization button
- Fill out the form
Example
ID: my-custom-viz-dev
Label: My Custom Visualization - Development
Main: https://localhost:3443/myCustomViz.js
- Select Save
- Create or go to an Explore and you should see the custom visualization in the menu. select it.
- Add some data so your explore has some data
- Start your local webserver by running the
npm script start:dev
Now, you have a reference within looker to your localhost so the webpack-dev-server can serve up the files to the url. If you’re server isn’t running, then your custom viz won’t work either. You can now make changes to your my-custom-viz.ts file and as you make changes, webpack will keep the bundles up to date. You can also open dev tools and debug your typescript in the browser if you wish since source-maps are avaialable. Once you’re finished and ready to go live, you can then comit the changes to git or some other repo and have continuous integration take care of the rest.
Links:
Example repo
Looker Custom Viz Repo
Webpack
Webpack Dev Server
TypeScript
Code References
types.ts
// API Globals
export interface Looker {
plugins: {
visualizations: {
add: (visualization: VisualizationDefinition) => void
}
}
}
export interface LookerChartUtils {
Utils: {
openDrillMenu: (options: { links: Link[], event: object }) => void
openUrl: (url: string, event: object) => void
textForCell: (cell: Cell) => string
filterableValueForCell: (cell: Cell) => string
htmlForCell: (cell: Cell, context?: string, fieldDefinitionForCell?: any, customHtml?: string) => string
}
}
// Looker visualization types
export interface VisualizationDefinition {
id?: string
label?: string
options: VisOptions
addError?: (error: VisualizationError) => void
clearErrors?: (errorName?: string) => void
create: (element: HTMLElement, settings: VisConfig) => void
trigger?: (event: string, config: object[]) => void
update?: (data: VisData, element: HTMLElement, config: VisConfig, queryResponse: VisQueryResponse, details?: VisUpdateDetails) => void
updateAsync?: (data: VisData, element: HTMLElement, config: VisConfig, queryResponse: VisQueryResponse, details: VisUpdateDetails | undefined, updateComplete: () => void) => void
destroy?: () => void
}
export interface VisOptions { [optionName: string]: VisOption }
export interface VisOptionValue { [label: string]: string }
export interface VisQueryResponse {
[key: string]: any
data: VisData
fields: {
[key: string]: any[]
}
pivots: Pivot[]
}
export interface Pivot {
key: string
is_total: boolean
data: { [key: string]: string }
metadata: { [key: string]: { [key: string]: string } }
}
export interface Link {
label: string
type: string
type_label: string
url: string
}
export interface Cell {
[key: string]: any
value: any
rendered?: string
html?: string
links?: Link[]
}
export interface FilterData {
add: string
field: string
rendered: string
}
export interface PivotCell {
[pivotKey: string]: Cell
}
export interface Row {
[fieldName: string]: PivotCell | Cell
}
export type VisData = Row[]
export interface VisConfig {
[key: string]: VisConfigValue
}
export type VisConfigValue = any
export interface VisUpdateDetails {
changed: {
config?: string[]
data?: boolean
queryResponse?: boolean
size?: boolean
}
}
export interface VisOption {
type: string,
values?: VisOptionValue[],
display?: string,
default?: any,
label: string,
section?: string,
placeholder?: string,
display_size?: 'half' | 'third' | 'normal'
order?: number
min?: number
max?: number
step?: number
required?: boolean
}
export interface VisualizationError {
group?: string
message?: string
title?: string
retryable?: boolean
warning?: boolean
}
utils.ts
import * as d3 from 'd3'
import {
VisConfig,
VisQueryResponse,
VisualizationDefinition
} from './types'
export const formatType = (valueFormat: string) => {
if (!valueFormat) return undefined
let format = ''
switch (valueFormat.charAt(0)) {
case '$':
format += '$'; break
case '£':
format += '£'; break
case '€':
format += '€'; break
}
if (valueFormat.indexOf(',') > -1) {
format += ','
}
const splitValueFormat = valueFormat.split('.')
format += '.'
format += splitValueFormat.length > 1 ? splitValueFormat[1].length : 0
switch (valueFormat.slice(-1)) {
case '%':
format += '%'; break
case '0':
format += 'f'; break
}
return d3.format(format)
}
export const handleErrors = (vis: VisualizationDefinition, res: VisQueryResponse, options: VisConfig) => {
const check = (group: string, noun: string, count: number, min: number, max: number): boolean => {
if (!vis.addError || !vis.clearErrors) return false
if (count < min) {
vis.addError({
title: `Not Enough ${noun}s`,
message: `This visualization requires ${min === max ? 'exactly' : 'at least'} ${min} ${noun.toLowerCase()}${ min === 1 ? '' : 's' }.`,
group
})
return false
}
if (count > max) {
vis.addError({
title: `Too Many ${noun}s`,
message: `This visualization requires ${min === max ? 'exactly' : 'no more than'} ${max} ${noun.toLowerCase()}${ min === 1 ? '' : 's' }.`,
group
})
return false
}
vis.clearErrors(group)
return true
}
const { pivots, dimensions, measure_like: measures } = res.fields
return (check('pivot-req', 'Pivot', pivots.length, options.min_pivots, options.max_pivots)
&& check('dim-req', 'Dimension', dimensions.length, options.min_dimensions, options.max_dimensions)
&& check('mes-req', 'Measure', measures.length, options.min_measures, options.max_measures))
}
my-custom-viz.ts
import { Looker, VisualizationDefinition } from '../common/types';
import { handleErrors } from '../common/utils';
declare var looker: Looker;
interface WhateverNameYouWantVisualization extends VisualizationDefinition {
elementRef?: HTMLDivElement,
}
const vis: WhateverNameYouWantVisualization = {
id: 'something', // id/label not required, but nice for testing and keeping manifests in sync
label: 'Something',
options: {
title: {
type: 'string',
label: 'Title',
display: 'text',
default: 'Somethingt'
}
},
// Set up the initial state of the visualization
create(element, config) {
this.elementRef = element;
},
// Render in response to the data or settings changing
update(data, element, config, queryResponse) {
console.log( 'data', data );
console.log( 'element', element );
console.log( 'config', config );
console.log( 'queryResponse', queryResponse );
const errors = handleErrors(this, queryResponse, {
// min_pivots: 0,
// max_pivots: 0,
// min_dimensions: 1,
// max_dimensions: 1,
// min_measures: 1,
// max_measures: 1
});
if (errors) { // errors === true means no errors
element.innerHTML = 'Hello Looker!';
}
}
};
looker.plugins.visualizations.add(vis);
tsconfig.json
{
"compileOnSave": true,
"compilerOptions": {
"module": "commonjs",
"target": "es6",
"sourceMap": true
},
"exclude": [
"node_modules"
]
}
webpack.config.js
let path = require('path');
const UglifyJSPlugin = require('uglifyjs-webpack-plugin');
let webpackConfig = {
entry: {
myCustomViz: './src/visualizations/my-custom-viz.ts'
},
output: {
filename: '[name].js',
path: path.join(__dirname, 'dist'),
library: '[name]',
libraryTarget: 'umd'
},
resolve: {
extensions: ['.ts', '.js', '.scss', '.css']
},
plugins: [
new UglifyJSPlugin()
],
module: {
rules: [
{ test: /\.ts$/, loader: 'ts-loader' },
{ test: /\.css$/, loader: [ 'to-string-loader', 'css-loader' ] },
{ test: /\.scss$/,
use: [
'style-loader',
'css-loader',
'sass-loader',
]
}
]
},
devServer: {
contentBase: false,
compress: true,
port: 3443,
https: true
},
devtool: 'eval',
watch: true
};
module.exports = webpackConfig;
