Friday, 13 September 2013

Talend - tFlowToIterate

About tFlowToIterate:

It is used to convert Main flow to iterate flow. It iterates input and convert into global variables. We can use these global variable at any situation in the job

Difference between Main and Iterate connections

Main Flow: It will process the records as batch wise. 
Iterate Flow: It will process the records as record wise 


If  there are 1000 records to process, batch size has been fixed to 200 then Main connection will process 200 records at a time as a batch, but iterate will process one by one record only.


Sample Job

We will see one sample job about this, In database  "SALES" table which contains month wise sales details. This data need to move to CSV files but One CSV file should be generated per one month. If table contains 12 months data then 12 files should be generated.

SALES Table Data like below table


Job process will be like below image


Step by Step Process;

Step 1: Define one context variable "filename" to store the name of the file.




Step 2: Read the distinct month from database using tOracleInput component
 Query:
                  "Select distinct month from Sales"





Step 3:  Set the file name with month  using with "tJavaRow" code given below
Code:
             output_row.MONTH = input_row.MONTH;
             context.filename=input_row.MONTH+".csv";





Step 4: Convert main flow to Iterate flow

Step 5:  Read the data from SALES table for each month using "tOracleInput" Component
Query:
          "SELECT S.CUST_ID,S.CUST_NAME,S.TIME_ID,S.AMOUNT_SOLD FROM SALES S       
           WHERE MONTH='"+((String)globalMap.get("row4.MONTH"))+"'"




Step 6: Load data in CSV file using with "tFileOutputDelimited" component using the following in File Path

Code:

                   "E:/Notes/"+context.filename




No comments:

Post a Comment