Shopify Connector: Loop over multiple tables to fetch data

[ianala]

Hi, 

I want to fetch the data from multiple tables using the Shopify connector. While creating a flow in the Application Integration, I noticed that the table needs to be selected when configuring the connector there. For instance, for the testing purposes I picked the table called "Countries". However, there are also other tables such as "order", "gift cards" and etc. How can I fetch the data at the same time from multiple  OR all available tables? 

[Meenchou]

you can have multiple connector tasks for those respective tables and connect all of these tasks to API_TRIGGER directly for parallel execution

[ianala]

@Meenchou In this case, one could create 20 multiple connector tasks for different 20 tables and connect them to one sub-integration for parallel execution?

For instance, in the sub-integration part, the table has a name "test" and at the moment it has values from one table. Is it possible that the data from those desired 20 tables can be added to 20 different tables in BQ? Or does it require to have 20 different BigQuery connectors to store to different tables?

[Meenchou · Accepted solution]

@ianala 

You don't need to use subintegration for this use case 

We are planning to  support dynamic value assignment in connectors task in future

you can create 20 different Shopify connectors task and 20 different BigQuery connectors task and connect them depicted in the screenshot attached