Ab Initio Component | DEDUP SORTED : Part 2

 ...Continue from Part 1...

check-sort (boolean, optional)

• Defines whether you want processing to abort on the first record that is out of sorted order. True causes processing to abort on the first record out of order.

    Default is True.

logging (boolean, optional)

• Defines whether the component logs certain events.

    Default is False.

log_input (choice, optional)

•  Defines how often the component sends an input record to its log port. Available only when the logging parameter is set to True.

log_output     (choice, optional)

•  Defines how  often the component sends an output record to its log port. Available only when the logging parameter is set to True.

log_reject (choice, optional)

• Defines how  often the component sends a reject record to its log port. Available only when the logging parameter is set to True.

    

Runtime behavior of DEDUP SORTED

    
DEDUP SORTED performs following operations:

   1.  Reads a grouped flow of records from the in port.

    2.  Does one of the following if a select expression is specified:

 

•  If the expression  values evaluates to 0 for a particular record then it does not process the record.

• Produces NULL for a particular record then it writes the record to the reject port and writes a descriptive error message to the error port.

• Evaluates to anything other than 0 or NULL for a particular record    Processes the record.

• If you do not supply an expression for the select parameter, DEDUP SORTED processes all records on the in port.

    3. Processes groups of records as follows:

• It considers any consecutive records with the same key value to be in the same group.

• If a group consists of one record, writes that record to the out port.

•  If a group consists of more than one record, uses the value of the keep parameter to determine which record — if any — to write to the out port, and which record or records to write to the dup port.

•  If you have chosen unique-only for the keep parameter, does not write records to the out port from any groups consisting of more than one record.

Ab Initio Component | DEDUP SORTED: Part 1

 Purpose of DEDUP SORTED

• DEDUP SORTED is used to separate one specified record in each group of records from the rest of the records in the group.

 

Parameters for DEDUP SORTED 

 

key (key specifier, required)

•  Name of the key/(s) field you want the component to use when determining groups of data records.

select (expression, optional)

• Provide the expresion to filters/select only those records accordingly before the component separates duplicates. 

 

Keep (choice, required)

• It specifies which records the component keeps to write to the out port. You have to set one of the following options:

        first — Keeps the first record of a group

        last — Keeps the last record of a group

        unique-only — Keeps only records with unique key values

    The component writes the remaining records of each group to the dup port.

    Default is first.
    
package (transform, optional)

•  Allows you to define this component’s log- and error-handling functions.

    
error_group  (string, optional)

• Defines name of the error group to which this component belongs. The component sends its error output to the HANDLE ERRORS component with a matching error_group value.

log_group  (string, optional)

•  Defines name of the log group to which this component belongs. The component sends its log output to the HANDLE LOGS component with a matching log_group value.

reject-threshold (choice, optional)

• The component’s tolerance for reject events.

limit (integer, optional)

•  A number representing reject events.

• When the reject-threshold parameter is set to Use limit/ramp, the component uses the values of the ramp and limit parameters in a formula to determine the component’s tolerance for reject events.

    Default is 0.

ramp (real, optional)

•  Rate of tolerance for reject events in the number of records processed.

• When the reject-threshold parameter is set to Use limit/ramp, the component uses the values of the ramp and limit parameters in a formula to determine the component’s tolerance for reject events.

    Default is 0.0.

Note:

• When you set the reject-threshold parameter of a component to Use limit/ramp, the limit and ramp parameters become available. The component then uses the limit and ramp parameters together in a formula to control the component’s tolerance for reject events:

• The component stops execution of the graph if the number of reject events exceeds the result of the following formula:

limit + (ramp *  number_of_records_processed_so_far)