Thursday, March 21, 2013

Informatica Session properties

There are three major properties, defined in Informatica Workflow Manager for each session, which impact Informatica mappings performance.

1. Commit Interval: The target-based commit interval determines the commit points at which the Integration Service commits data writes in the target database. The larger the commit interval, the better the overall mapping’s performance. However too large commit interval may cause database logs to fill and result in session failure.The recommended range for commit intervals is from 10,000 up to 200,000 for Oracle database.    

2. DTM Buffer Size: The DTM Buffer Size specifies the amount of memory the Integration Service uses for DTM buffer memory. Informatica uses DTM buffer memory to create the internal data structures and buffer blocks used to bring data into and out of the Integration Service.

3. Additional Concurrent Pipelines for Lookup Cache Creation: Additional Concurrent Pipelines for Lookup Cache Creation parameter defines the concurrency for lookup cache creation.To reduce lookup cache build time by enabling parallel lookup cache creation set value larger than one.

4. Default Buffer Block Size: The buffer block size specifies the amount of buffer memory used to move a block of data from the source to the target. For better performance for both Initial and Incremental ETL with Default Buffer Block Size set to 512,000 (512K). You can run the following SQL to update the Buffer Block Size to 512K for all mappings in your Informatica repository:

SQL> update opb_cfg_attr set attr_value='512000' where attr_value='128000' and attr_id = 5;
SQL> commit;

Informatica Parallel Sessions Load on ETL tier:
Informatica mappings with complex transformations and heavy lookups typically consume larger amounts of memory during ETL execution. While processing large data volumes and executing in parallel, such mappings may easily overload the ETL server and cause very heavy memory swapping and paging. As the result, the overall ETL execution would take much longer time to complete. To avoid such potential bottlenecks:
  1. Consider implementing Informatica 64-bit version on your ETL tier.
  2. Ensure you have enough physical memory on your ETL tier server.
  3. Keep in mind that too many Informatica sessions, running in parallel, may overload either source or target database.
  4. Set smaller number of connections to Informatica Integration Service in DAC. Navigate to DAC's Setup screen ->Informatica Servers tab -> Maximum Sessions in the lower pane for both Informatica and Repository connections.The recommended range is from 5 to 10 sessions.

No comments:

Post a Comment