Informatica编码

Question

I am currently working on a scenario in Informatica Powercenter designer where the situation is as follows:

SQ1: I am pulling employee records according to the criteria of having a layer of employees based on their hierarchy (Client relation directors) which is the first source qualifier and in which i am doing a sql override to extract data from 3 tables, and for those selected employees I have to pull some other information for example:

SQ2: what client relations they are handling which is in a separate source qualifier and

SQ3 some of the personal information from their profile which is in a third source qualifier.

I have single mapping in which there are three source qualifiers as described above and in all of them I am using SQL override. My question is that the data that i have pulled in first qualifier brings a subset of the total employee records, but in Source qualifier 2 and source qualifier 3 I have to pull all employee data and then do a join on employee_id in two joiners to finally collect data for the layer of employees that are coming from source qualifier 1. What i want is that if somehow I save the employee ids from and SQ1 and use them in SQ2 and SQ3 so that i pull data for only a subset of employees, the problem is I cant split the mapping and cannot add the code for selecting the subset from SQ1 bc it will be repitition of code and taking a long time to run, also the number of records are about one million. I cant find a way to perform the above that is why i am asking for help here.

I am pulling data from db2, and working in powercenter designer 9.5.1. I will be thankful if i can get guidance regarding the above issue

Answer 1

What you can do is if all the table is in database,you can pull the source tables in one source qualifier and then override the SQL and create a join. So the point is instead of 3 different source qualifier you can have one source qualifier.

Answer 2

I assume you are having three separate source qualifiers because the data is present in different databases. If not, doing an application join from three different source qualifiers( you will have to use 2 joiners) is very expensive. There are a couple of ways you can do this:

1. split the mapping to stage the data first and then use this staging layer as source to perform more complex operation

2. Identify your driving table. Since the record count in the SQ2 and SQ3 are bigger, I am assuming they can be the driving table. Use a lookup for SQ1 (Since its a smaller table size, the cache time would not be very big)

I would still suggest you use a staging layer to extract and stage the data, then transform it. Try to perform database joins(or lookups) as much as you can instead of joining at the application layer.

Answer 3

Consider using a pipeline lookup as a query for your SQ1 and use it in the pipeline that joins SQ2 and SQ3 .

Usage for pipeline lookup can be found at :

https://marketplace.informatica.com/solutions/performance_tuning_pipeline_lookup

Let me know if it helps .