INTEGRATION SERVICE PROCESS
The Integration Service starts an Integration Service process to run and monitor workflows. The Integration Service process accepts requests from the Power Center Client and from pmcmd. It performs the following tasks:
- Manages workflow scheduling.
- Locks and reads the workflow.
- Reads the parameter file.
- Creates the workflow log.
- Runs workflow tasks and evaluates the conditional links connecting tasks.
- Starts the DTM process or processes to run the session.
- Writes historical run information to the repository.
- Sends post-session email in the event of a DTM failure.
LOAD BALANCER
The Load Balancer is a component of the Integration Service that dispatches tasks to achieve optimal performance and scalability. When we run a workflow, the Load Balancer dispatches the Session, Command, and predefined Event-Wait tasks within the workflow.
The Load Balancer dispatches tasks in the order it receives them. When the Load Balancer needs to dispatch more Session and Command tasks than the Integration Service can run, it places the tasks it cannot run in a queue. When nodes become available, the Load Balancer dispatches tasks from the queue in the order determined by the workflow service level.
DTM PROCESS
When the workflow reaches a session, the Integration Service process starts the DTM process. The DTM is the process associated with the session task. The DTM process performs the following tasks:
- Retrieves and validates session information from the repository.
- Performs pushdown optimization when the session is configured for pushdown optimization.
- Adds partitions to the session when the session is configured for dynamic partitioning.
- Expands the service process variables, session parameters, and mapping variables and parameters.
- Creates the session log.
- Validates source and target code pages.
- Verifies connection object permissions.
- Runs pre-session shell commands, stored procedures, and SQL.
- Sends a request to start worker DTM processes on other nodes when the session is configured to run on a grid.
- Creates and run mapping, reader, writer, and transformation threads to extract, transform, and load data.
- Runs post-session stored procedures, SQL, and shell commands.
- Sends post-session email.
PROCESSING THREADS
The DTM allocates process memory for the session and divides it into buffers. This is also known as buffer memory. The default memory allocation is 12,000,000 bytes.
The DTM uses multiple threads to process data in a session. The main DTM thread is called the master thread.
The master thread can create the following types of threads:
- Mapping Threads: One mapping thread for each session.
- Pre- and Post-Session Threads: One thread created.
- Reader Threads: One thread for each partition
- Transformation Threads: One thread for each partition
- Writer Threads: One thread for each partition
CODE PAGES and DATA MOVEMENT
A code page contains the encoding to specify characters in a set of one or more languages. An encoding is the assignment of a number to a character in the character set.
The Integration Service can move data in either ASCII or Unicode data movement mode. These modes determine how the Integration Service handles character data.
We choose the data movement mode in the Integration Service configuration settings. If we want to move multi byte data, choose Unicode data movement mode.
ASCII Data Movement Mode: In ASCII mode, the Integration Service recognizes 7-bit ASCII and EBCDIC characters and stores each character in a single byte.
Unicode Data Movement Mode: Use Unicode data movement mode when sources or targets use 8-bit or multi byte character sets and contain character data.
No comments:
Post a Comment