Ab Initio Component | DEDUP SORTED: Part 1

 Purpose of DEDUP SORTED

• DEDUP SORTED is used to separate one specified record in each group of records from the rest of the records in the group.

 

Parameters for DEDUP SORTED 

 

key (key specifier, required)

•  Name of the key/(s) field you want the component to use when determining groups of data records.

select (expression, optional)

• Provide the expresion to filters/select only those records accordingly before the component separates duplicates. 

 

Keep (choice, required)

• It specifies which records the component keeps to write to the out port. You have to set one of the following options:

        first — Keeps the first record of a group

        last — Keeps the last record of a group

        unique-only — Keeps only records with unique key values

    The component writes the remaining records of each group to the dup port.

    Default is first.
    
package (transform, optional)

•  Allows you to define this component’s log- and error-handling functions.

    
error_group  (string, optional)

• Defines name of the error group to which this component belongs. The component sends its error output to the HANDLE ERRORS component with a matching error_group value.

log_group  (string, optional)

•  Defines name of the log group to which this component belongs. The component sends its log output to the HANDLE LOGS component with a matching log_group value.

reject-threshold (choice, optional)

• The component’s tolerance for reject events.

limit (integer, optional)

•  A number representing reject events.

• When the reject-threshold parameter is set to Use limit/ramp, the component uses the values of the ramp and limit parameters in a formula to determine the component’s tolerance for reject events.

    Default is 0.

ramp (real, optional)

•  Rate of tolerance for reject events in the number of records processed.

• When the reject-threshold parameter is set to Use limit/ramp, the component uses the values of the ramp and limit parameters in a formula to determine the component’s tolerance for reject events.

    Default is 0.0.

Note:

• When you set the reject-threshold parameter of a component to Use limit/ramp, the limit and ramp parameters become available. The component then uses the limit and ramp parameters together in a formula to control the component’s tolerance for reject events:

• The component stops execution of the graph if the number of reject events exceeds the result of the following formula:

limit + (ramp *  number_of_records_processed_so_far)