Looker Connect Training

Help Center

Documentation

Community

Cloud Certifications

Question

Generating Client SDKs for the Looker API

  • 16 August 2016
  • 65 replies
  • 2743 views
danny
Rank
Userlevel 2

Update: This article is now out of date. For the latest info on how to download or generate client SDKs for the Looker API, please see our Looker SDK Codegen repo on Github




The Looker API is a collection of “RESTful” operations that enables you to tap into the power of the Looker data platform in your own applications. The Looker API can be invoked using plain old HTTP(S) requests. Any development tool, language, or application that can make HTTP requests and ingest JSON responses should be able to use the Looker API.


Web geeks who live and breathe HTTP and/or AJAX XHR will feel at home using just the “raw” Looker API HTTP URL endpoints. GET, POST, PUT, PATCH, DELETE, oh my!


Web APIs For The ‘REST’ of Us


If writing HTTP requests is not something you or your developers do every day, accessing a REST API for the first time can be a little intimidating and disorienting. Programming with web requests often requires different idioms and patterns of behavior than traditional app development. What do you do if you’re more interested in the data results than in the journey required to get them?


What non-web devs desire is something that makes the Looker API feel like a set of functions in your language of choice: a client SDK. A client SDK is a set of functions that resides in your application that encapsulates all the HTTP goop so you can just call functions and get results.


The Looker API provides a metadata document which describes all the URL endpoints, their parameter names and types, and their response types. From that metadata you can use Swagger tools to generate client SDK libraries for your programming language.


This allows us to make the Looker data platform accessible via convenient client libraries for a wide variety of programming languages, without having to be experts in all of those languages or having to commit resources to developing and supporting all those languages. It also helps ensure consistency across all manifestations of these client SDKs which would be very difficult to manage if each SDK were hand-written.


Enough back story. Let’s generate a Looker API client SDK!


Prerequisites




  1. Login to your Looker instance as an admin to enable and configure the Looker API: https://<your_looker_endpoint>/admin/api. Make note of your Looker API Host url displayed here if you have customized it from the default.




  2. Generate API3 credentials by navigating to https://<your_looker_endpoint>/admin/users, editing your user, and clicking “New API3 Key.” You’ll need the API3 clientId and clientSecret values later to authenticate your client application.




  3. With your API configured and credentials in hand, navigate to your Looker API documentation: https://<your_looker_endpoint>:19999/api-docs/index.html. (Substitute your custom Looker API Host domain and port if applicable)

    This page is our reference for available Looker API function/endpoints specific to our Looker instance. Since this doc page is generated from the Looker API metadata, this doc page may show beta or experimental API functions that are not yet officially documented elsewhere.




  4. Make sure Python 2.7 or 3.0 is installed on your computer. At a command line / shell prompt, run python --version to see what version of Python, if any, is installed.




  5. You’ll need a Java runtime distribution installed on your computer. We recommend at least Java 8 (version 1.8.*). Run java -version to verify version installed.




  6. Make sure Maven 3.3 or later is installed on your computer. At a command line / shell prompt, run mvn --version to see what version of Maven is installed.




  7. You’ll also need to have git client tools installed on your computer. Check version with git --version




Setup




  1. Make a directory to house your Looker SDK bits: cd ~/Desktop; mkdir looker_sdk; cd looker_sdk




  2. Download your Looker API swagger.json metadata.




  3. Login to your Looker instance as admin and navigate to the admin/api panel. (https://<your-looker-endpoint>:9999/admin/api).




  4. Set “Documentation Access” to “Allow anyone to see API docs”




  5. Click the Save button.




  6. Open a new browser window and navigate to this URL: https://<your-looker-endpoint>:19999/api/3.0/swagger.json Subsitute your custom Looker API Host domain name and port if necessary.




  7. Save the contents of that swagger.json page to looker_sdk/lookerapi.json.




  8. Take a peek inside the saved file to make sure the browser saved it as JSON text, not transmogrified HTML.




  9. Return to your Looker instance Admin/API page and set the “Documentation Access” setting back to what it was before and click Save. “Require API login to see API docs” is the default setting and is required if you want to use the “Try it!” button to call API functions directly from the API doc page. (You do want this, it’s cool!)




  10. Download the Swagger-Codegen tools from github.

    Note: Swagger-Codegen is in active development. To avoid the headaches of “bleeding edge” development, it’s a good idea to pull from a stable release. At the time of this writing, we recommend using Swagger-Codegen release 2.1.6. (If you’re using Python 3.0 or later, you’ll want Swagger-Codegen release 2.2.2 instead)

    Use these steps to checkout the swagger-codegen tools into your looker_sdk/swagger-codegen directory:




  11. cd looker_sdk




  12. git clone git@github.com:swagger-api/swagger-codegen.git ./swagger-codegen




  13. cd swagger-codegen




  14. git checkout tags/v2.1.6




  15. Build the Swagger-Codegen tools




  16. cd looker_sdk/swagger-codegen




  17. mvn package




  18. Make yourself a cup of coffee. Fry up some donuts while you’re at it - this will take awhile!




  19. When the dust settles, you want to see Build Success near the end of the Maven output. If Maven or one of its subprocesses exits with an error, you need to recheck your machine configuration and recheck that you have all the prerequisites installed and operational.




Generate a Looker API Client SDK


To generate a Looker API client SDK for Python, do this:



  1. cd looker_sdk

  2. java -jar ./swagger-codegen/modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate -i lookerapi.json -l python -o python_sdk


This will generate a Looker API client SDK for Python in the looker_sdk/python_sdk directory


You can use the same command to generate Looker API client SDKs for Java, .NET, and a variety of other languages and platforms supported by the Swagger-Codegen tools. Change the language target with the -l parameter and the output directory with the -o parameter. See https://github.com/swagger-api/swagger-codegen/tree/master/modules/swagger-codegen/src/main/resources and the Swagger-Codegen documentation for the list of supported languages and platforms.


Versioning


The generated client SDK is a snapshot of the “shape” of the Looker API at a single point in time. As new stuff is added in subsequent releases of the Looker product, your existing client SDK will continue to work just fine, but your client SDK will only be aware of API stuff that existed in the Looker product release that the client SDK was generated from.


If a new Looker API feature turns up in a future Looker product release that you want to use in your app, you will need to download a new lookerapi.json file from the new Looker product instance and use that to regenerate the client SDK files.


Use the Looker API Client SDK (for Python)


Among the files generated by Swagger-Codegen for Python is a subdirectory python_sdk/swagger_client which contains all the Python class definitions we need to make Python function calls to the Looker API.


For quick-and-dirty work, you can just copy the swagger_client directory (and its subdirectories) into a subdirectory of your Python project. Note that we’re changing the directory name to looker as part of the move (so that our Python code can refer to the client SDK module as “looker” instead of as “swagger_client”)

mv looker_sdk/python_sdk/swagger_client ~/mypythonapp/looker

cd ~/mypythonapp


Here’s a quick Python test app to kickstart your Looker API development.

Fill in your Looker API Host name and port, your API3 clientId and clientSecret values:


~/mypythonapp/test_app.py


import looker



# replace with your custom Looker API Host domain and port, if applicable.

base_url = 'https://your.looker.instance:19999/api/3.0/'

client_id = 'your-API3-client-id'

client_secret = 'your-API3-client-secret'



# instantiate Auth API

unauthenticated_client = looker.ApiClient(base_url)

unauthenticated_authApi = looker.ApiAuthApi(unauthenticated_client)



# authenticate client

token = unauthenticated_authApi.login(client_id=client_id, client_secret=client_secret)

client = looker.ApiClient(base_url, 'Authorization', 'token ' + token.access_token)



# instantiate User API client

userApi = looker.UserApi(client)

me = userApi.me();



print me



# instantiate Look API client

lookApi = looker.LookApi(client)



print lookApi.all_looks()

This test app imports the Looker API Python client SDK from the looker subdirectory of the test_app.py file’s home directory. The Python code can then access the Looker API classes via the looker module symbol.


This test app authenticates to the Looker API using the clientId and clientSecret values you generated earlier on the Looker Admin/User panel. It fetches the user info for the current logged-in user using the userApi.me() API call, and fetches a list of all looks accessible to the current user using the lookApi.all_looks() API call.


Audience Participation


The Swagger-Codegen generated Python client SDK code looks like it is intended to be installed as a Python package/module. Not being a Python expert, I haven’t had much success with naive attempts to install the client SDK files as a Python module.


SO… If any of you Python alpha geeks out there would like to chime in with tips on how to install the Python client SDK as a module (and successfully reference it from a Python app) the rest of us Python n00bs would be grateful!


Proceed Carefully


This test app is safe to use because it doesn’t use any Looker APIs that change the state of the Looker instance. Many Looker APIs, however, can change or delete settings, users, models, and more from your Looker instance. Be careful! You can limit what a program can do with the Looker API by using API3 client credentials associated with a limited-rights user account, and only enable roles or permissions on that user account as needed. It’s always a good idea to do development work using a non-production Looker instance.


Disclaimers


Looker provides full support for the use and execution of the HTTP Looker API operations themselves. Officially, Looker cannot provide support for bugs or issues in Swagger generated client SDKs.


If you find something wonky when making a Looker API call through a client SDK, try performing the same Looker API call using the equivalent raw HTTP/JSON request. If the raw HTTP request also behaves oddly, that’s on us. Please contact Looker support to let us know!


Pro Tip:


Use the Looker API doc page to experiment with raw HTTP requests. Auth is taken care of for you!


If the HTTP request behaves correctly but the equivalent client SDK call is broken, that may be a bug in the generated client SDK. Let us know if you find something like this so we can check for workarounds and make sure our JSON metadata is correct. Be aware that the extent of our support for client SDK issues will be limited to recommending that you try a different release of the Swagger-Codegen tools, and suggesting you post a cry for help from fellow Looker devs here in the Looker Discourse community. We can’t help you write your application code or your homework assignments.

65 replies

David_Stocker

This problem is also found when building for Java.

izzy
Userlevel 7
Badge +1

David, which problem are you referring to? This is a long thread with a bunch of examples of things people have bumped into 😄.

David_Stocker

I’m trying to use the swagger code-gen SDK for Looker API version 3.1 built in Java. It’s unable to deserialize [‘vis_config’][‘series_types’]. The SDK expects a string for the value, but series_types is a hashmap.

izzy
Userlevel 7
Badge +1

Replied over here:


LStanevich
Userlevel 2

I know one of our devs was able to successfully generate a Python SDK for 3.1 recently, but it was definitely a struggle.

rassack

May I suggest : https://github.com/looker-open-source/sdk-codegen/tree/master/python it’s managed by Looker

LStanevich
Userlevel 2

That’s what we used – my main point was that it still took a fair amount of futzing around to get everything for the 3.1 SDK working correctly, even after starting with the official Looker tool. Nothing deeply technical, at the end of the day, but just chasing around various config issues that needed to be cleaned up, etc.


(Also, to be clear, we were using the code-gen for 3.1 very soon after it was released for any public consumption at all, and clearly labeled as “not-yet-official”. A lot of those the little issues we ran into have hopefully been addressed in the meantime.)

LStanevich
Userlevel 2

Aaaand…Looker 6.22 Release Notes

gsvitak

Thanks for releasing the updated python SDK officially.


Do you have a sample that uses environment variables rather than the python.ini. I do not see how to setup the client.


Do I simply do


sdk = client.setup()

izzy
Userlevel 7
Badge +1

I think that would be a question for @jax. After a quick test it looks like after setting env variables (make sure to name them properly like LOOKERSDK_BASE_URL, for example), you can just run it with no args and it will capture the environment variables.

gsvitak

Will the new SDK only work with python 3.7? I have python 3.6 and receiving the following.


Any suggestions


venv/lib/python3.6/site-packages/looker_sdk/rtl/serialize.py", line 31, in <module>

    from typing import (  # type: ignore

ImportError: cannot import name 'ForwardRef'
jax
Rank

Hi Greg,


Yes, the SDK requires python 3.7. We suggest using pyenv if you want to manage multiple python versions on your system. See https://github.com/looker-open-source/sdk-codegen/tree/master/python for more details.

Marc_Wilson
Userlevel 2

A new API (3.0) is now released, but the swagger file still doesn’t appear to work without heavy modification: https://github.com/looker-open-source/sdk-examples/issues/45

izzy
Userlevel 7
Badge +1

Mirroring @john.kaster’s comment from github:



If you are on Looker 7.2 or later, I highly recommend using the API 4.0 version of the swagger file. API 4.0 was explicitly started to resolve these issues for strongly typed languages like C#, Kotlin, and Swift. I also recommend using the legacy option for the SDK Code generator project, and point the looker.ini to the API 4.0 version.




I haven’t tested the legacy option in a few merges, so feel free to log an issue in sdk-codegen if you have a problem with it and I’ll look into it.


tomcribb

Hi, any chance this could be done for JAVA language?

Reply


Terms of Service