Building Maps
When you build maps try to use the following steps. Sometimes this can help clarify "how" to architect the database. Before beginning ask the following questions:
- How much data is there passing through this map?
- What is the source, target, aggregator, and lookup sizes (in terms of Rows and Megabytes of space)?
- What is the time frame this map is expected to run in?
- Are we loading data over the network?
- Are we waiting on a flat file? More than one?
- Is there a way to balance the size of the source / target and maybe eliminate the lookups by utilizing Database Routines?
- Is the speed such a problem that we must eliminate aggregators?
- Can we tune the SQL on the source qualifier?
- Do we need to generate new sequence ID numbers for each row?
Guidelines Utilized to Divide Maps:
The designer tool is highly capable of depicting, graphically, complex mappings. It is also fairly easy to develop these complex mappings. However, it can quickly become difficult to maintain them. In an effort to offer the best of the designer, we suggest breaking up the complex mappings in to more manageable pieces. The maps should only be broken up as a last resort when attempting to make more operations parallel, or in an attempt to speed up the whole process (end-to-end). Start with development using the Designer. As pieces get too difficult, or problems present themselves (due to speed, mobility, or cross-functionality), take a look at the architecture of the maps to see if they wouldn't benefit from being sub-divided.
Does the map have multiple targets?
If so, could each target be loaded in parallel?
What's the affect of breaking up the logic?
Can the logic be placed in to maplets?
Are the targets setup with Constraints?
Does a parent need to be loaded first?
- ONE MAP PER TARGET TABLE. This rule forces you to think about where and how the logic will be applied. However, it allows us to control the load order by constraints, as well as which maps and sessions can be run concurrently. It also forces you to think about how to apply common logic in the most efficient manner. Another benefit is that all of the data that flows in to that target goes through the same set of business rules designed for the target fields.
- On some occasions (due to lack of speed) we need to take this further, one map per target per action. Where the action is Insert, Update, or Delete.
- In breaking the maps out like this, it keeps maps simple and maintainable. Each map is therefore responsible for a single set of business rules, as well as a single task.
