Looker Incremental PDT creates and duplicate records

Hi, 

Recently I have implemented incremental PDT and after few weeks I realize its creating duplicate records in table. After some research I got the reason for this, its because looker use only:

DETETE

INSERT

statement while incrementing table, update statement not being used. So, Looker delete the records based on incremental offset and then insert records for that offset. This process end up with duplicate records because it loads already inserted old records if those got updated in current incremental pdt cycle.

Anyone know how to solve this?

0 0 289
0 REPLIES 0
Top Labels in this Space
Top Solution Authors