Got a question?
Ask your big data questions here.
- 117 Topics
- 203 Replies
Hi there, Could you suggest the best and easiest way to load one small dataset from DynamoDb into Looker? The dataset represented with one table of time-series data, up to 20 fields, not more than 100k rows in total. The data needs to be updated one or two times a day. And this doesn’t need to be a production-ready integration, we just need a quick and dirty way to get the data and play with it. If we decide that this data looks good in looker we will consider implementing a more robust integration.
Blog on Scalable R-based Web Server
@Brett_Sauve suggested this community might be interested in some of our work at PERTS. This blog post describes and gives sample code for a containerized web server in R for ETL work (or whatever else you like). Thanks for your interest and feedback 🙂 Medium – 11 Jul 18 A Scalable Pure-R Web Server – PERTStech – Medium At PERTS we’d like to run custom ETL jobs in R, and enterprise products like Tableau and RStudio Connect are heavy, expensive, and (for… Reading time: 2 min read
Segment v's Snowplow
Suggestions for moving Looker data to Salesforce?
Does anyone have recommendations for a tool we can use to sync Looker data to Salesforce? Note that we’d like to actually sync data from Looker to Salesforce rather than embedding a Looker dashboard within Salesforce which I know is also possible. We’re hoping there is a tool out there that can handle this without too much engineering effort using the Looker API or the soon to be released Looker webhooks.
Oracle Responsys - Looker; automating exports
Hello, Is there a place where I can find more information about how to go about sending file exports from Oracle Responsys to a Looker customer’s instance? The use case is to allow for marketing data exports to flow from Responsys Interact into customer’s Looker DB where it can be developed into their reporting model. Or specifically to know if it’s possible to have an sftp folder associated directly to the Looker instance. Or more generally - what are the suggested ways to build 3rd party connections in Looker? Are there specific partners or solutions that one would use for integrating Looker with their ESP (email service provider) or marketing platform? Thanks in advance, any advice much appreciated! -Nicole H
Your company's data team organization
I had asked a question earlier this year surveying everyone here about your company’s data stack and thought it was a fruitful conversation. As my company Payoff has matured, I’ve realized more and more that something that is as or possibly more important is how the data team is structured across the company. And when I say data team I’m referring to the full data stack and the role people play in building out the infrastructure, building data integrations and of course analyzing the data itself. So how is your data team structured? Does it have a clear delineation between data engineers (generate the data in the application aka website), DBAs (those that work on the data pipes or ETL) and data analysts/scientists (those that derive business value from the data)? Or do you have an organization where the line is blurred between DBAs and data scientists kind of like Stichfix does it: their motto is engineers shouldn’t write ETL. To kick things off I thought I’d share how my company is st
Data from Salesforce
Hello, I was wondering how often does the data from Salesforce update in Looker. I know it is 24 hours but when I pull the data and compare it to Looker it is off by more than 48 hours of data update. Can anyone help me to understand if there is something I missing in the refresh that is supposed to occur every night (or every 24 hours)? Thanks, Parijat
I’ve begun exploration of creating a transformed database (data warehouse) for use with looker. Right now we just have Looker hooked up to a replica of our production DB, which has worked great in the short-term, but is showing some limitations. In order to not go into this too blindly I’ve been doing some reading on data warehouse best practices, including ‘The Data Warehouse Toolkit’, which from what I can gather is one of the must read books in this space. In this book they are quite adamant about the creation of a ‘Date’ table, instead of using date fields directly in your fact tables. Benefits include being able to assign dimensions to dates such as ‘holiday’, ‘quarter’, ‘weekend’, etc. that aren’t readily accessible in SQL. Of course, Looker does have some helpful date/time tools allowing easy grouping by month/week etc. So, as I begin down this road I wonder how important these classical warehouse approaches are when using a tool like looker, as it seems much of what looker does
Designing a Data Warehouse - what would a BI solution recommend?
At Looker we are often asked about best practices when it comes to designing a new data warehouse. Typically this happens just as when companies are moving to MPP, and maybe even, column-oriented, databases, so it is clear from the start that replicating the design from an operational database is not appropriate. It is also clear that there is not a single answer - search on Google turns up millions of results. That said, here are a few rules of thumb that you can apply as you focus on building your analytical data warehouse to work with Looker: simplicity (aka shortest path) performance single copy of data transparent EL process Shortest Path You should not need to use mapping tables or Entity–attribute–value tables to get to the value. The path to any two dimensions in one SELECT query should not involve more than a couple of joins. Typically “long path” designs arise from storing original data in NoSQL format. Because there is very little analytical value derived from performing s
Already have an account? Login
Login to the community
No account yet? Create an account
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.