Ab Initio Component | REFORMAT: Part 3

  ..Continue from Part 2..

 

 Runtime behavior of REFORMAT

• First it read the port count(n) from count parameter of component the n in outn gives each out port a unique number. Each outn port has a corresponding rejectn and errorn port. REFORMAT does the following:
  

1. REFORMAT reads a record from the in port of component. 
2.  If the select parameter has an expression specified, It uses the expression to evaluate the input record: 

•  if the expression evaluates to false (0), then REFORMAT discards the input record and starts over with step 1.

•  If the expression produces NULL, then REFORMAT writes a descriptive error message and stops execution of the graph.

• If the expression evaluates to true (anything other than 0 or NULL), then REFORMAT begins processing the input record.

3. If the select parameter does not have a value, REFORMAT begins processing the input record.

 

4.  REFORMAT determines whether a transform function is specified in either the output-index or output-indexes parameter:

• If neither output-index nor output-indexes has a value (the usual case when there is only one out port), REFORMAT sends the input record to every transform-out port pair, beginning with out0 and progressing sequentially.

• If output-index or output-indexes has a value, REFORMAT evaluates the specified index transform. If output-indexes is defined, it should return a vector of port index values. If output-index is defined, it should return a single port index value.
  

•  REFORMAT uses one or more values from the index transform to determine the appropriate transform-output port pair or pairs for the input record. If the index transform returns more than one value, REFORMAT sends the record to each of the appropriate ports, starting with the lowest numbered port and progressing to the other ports sequentially.
  

 5. REFORMAT determines whether each outn port has a transform function.

•  If an out port does not have a transform function, REFORMAT uses implicit reformat to process the input record. For more information, see “Implicit reformat”.

• If the input record is sent to more than one port, the order of the transform evaluation is sequential: it calls the transform function on each port in order, starting with the lowest numbered port. For example, if the record is to be sent to port0 and port2, it is sent to port0 first, and then to port2. The evaluation of the second transform can depend on the side-effects of the first transform, which means you could make successive calls to a function like next_in_sequence from sequential transforms for the same input record
  

•  If a transform function results in an error or returns NULL, REFORMAT writes the following: 

• An error message to the corresponding error port
• The current input record to the corresponding reject port

•  The component stops execution of the graph when the number of reject events exceeds the reject threshold. 

• if the reject or error ports do not have flows attached to them, REFORMAT discards the record.   

6. REFORMAT writes the record to the out port of each successful transform,and then begins processing the next input record.

 

 

 

 

 

 

Ab Initio Component | REFORMAT: Part 2

 ..Continue from Part 1..

 

output-index(filename or string, optional)

• It specifies either the name of a file containing a transform function, or a transform string. This  component calls the specified transform function for each input record. The transform function should return an index value between 0 and the highest-numbered output port. REFORMAT uses this value to direct the input record to the output port that has the same number as the value, and executes the transform function, if any, associated with that port.

• When you specify a value for this parameter, each input record goes to exactly one transform-output port pair. For example, suppose there are 100 input records and two output ports. Each output port receives between 0 and 100 records. So according to transform function you can specify the output-index split to 50/50, 60/40, 0/100, 99/1 or any other combination which can add up to 100
  

• If an index is out of range (less than zero or greater than the highest-numbered port), the component discards the input record.

    

    NOTE: If you specify a value for the output-index parameter, you cannot also specify the output‑indexes parameter and vice-versa

• If you do not specify a value for either output-index or output-indexes, the component sends every input record to every transform-output port pair.  

 

output-indexes (filename or string, optional)

• It specifies either the name of a file containing a transform function, or a transform string. The component calls the specified transform function for each input record. The transform function uses the value of the input record to direct that input record to particular transform-output ports. 

•  The expected output of the transform function is a vector of numeric values. The component considers each element of this vector as an index into the output transforms and ports. The component directs the input record to the identified output ports and executes the transform functions, if any, associated with those ports. 

• If an index is out of range (less than zero or greater than the highest-numbered port), the component discards the input record.

• If you do not specify a value for either output-indexes or output-index, the component sends every input record to every transform-output port pair.

• As an example of how the component uses the transform function specified in the output-indexes parameter, consider the following:

    out :: output_indexes(in) =
    begin
      out :1: if (in.kind == "x") [vector 0, 1, 2];
      out :2: if (in.kind == "y") [vector 2, 3, 4];
      out : : [vector 5];
    end; 

 

•  When you specify this transform function for the output-indexes parameter, it directs the input record to the transform functions for:

        Ports 0, 1, and 2 if the field in.kind is “x”

        Ports 2, 3, and 4 if the field in.kind is “y”

        Port 5 if otherwise

 logging(boolean, optional)

•     Specifies whether the component logs certain events.

 

 log_input (choice, optional)

• Specifies how often the component sends an input record to its log port. The logging parameter must be set to True for this parameter to be available.

•  For example, if you select 100, the component sends every 100th input record to its log port. 

 

log_output (choice, optional) 

• Specifies how often the component sends an output record to its log port. The logging parameter must be set to True for this parameter to be available.

• For example, if you select 100, the component sends every 100th output record to its log port. 

 

log_reject(choice, optional)

• Specifies how often the component sends a reject record to its log port. The logging parameter must be set to True for this parameter to be available.

• For example, if you select 100, the component sends every 100th reject record to its log port. 

 

 

 

Ab Initio Component | REFORMAT: Part 1

 Purpose of Reformat:

• Reformat Component is used to change the format of records by dropping fields, or by using DML expressions to add fields, combine fields, or transform the data in the records.
•  REFORMAT performs an implicit reformat when you do not define a reformat function or transformation for the fields

 

Parameters for REFORMAT

 

 count (integer, required)

     It is used to sets the number of:

•         out ports
•         reject ports
•         error ports
•         transform parameters

    Default is 1. 

 

select (expression, optional)

•  It is used to filter the records before reformatting 

 

error_group (string, optional)

•  It is name of the error group to which this component belongs. It sends its error output to the HANDLE ERRORS component with a matching error_group value.

 

log_group (string, optional)

•  It is name of the log group to which this component belongs. It sends its log output to the HANDLE LOGS component with a matching log_group value.

reject-threshold (choice, required)

• It is used to specifies the component’s tolerance for reject events.

•  The reject-threshold parameter specifies the component’s tolerance for reject events. Choose one of the following       
  

1. Abort on first reject — The component stops execution of the graph at the first reject event it generates.
2. Never abort — The component does not stop execution of the graph no matter how many reject events it generates.
3. Use limit/ramp — The component uses the settings in the limit and ramp parameters to determine how many reject events to allow before it stops execution of the graph.

 limit(integer, required)

• A number representing reject events.
• When the reject-threshold parameter is set to Use limit/ramp, the component uses the values of the ramp and limit parameters in a formula to determine the component’s tolerance for reject events. 

       Default is 0.  

ramp(real, required)

• Rate of toleration of reject events in the number of records processed.
• When the reject-threshold parameter is set to Use limit/ramp, the component uses the values of the ramp and limit parameters in a formula to determine the component’s tolerance for reject events.
  

    Default is 0.0.

Note:

When you set the reject-threshold parameter of a component to Use limit/ramp, the limit and ramp parameters become available. The component then uses the limit and ramp parameters together in a formula to control the component’s tolerance for reject events:

The component stops execution of the graph if the number of reject events exceeds the result of the following formula:

limit + (ramp *  number_of_records_processed_so_far)