The file having Header and Trailer needs to be sorted but standard sort parameters will sort all records including header and trailer. So either exclude these from sorting or rearranges them after sorting. To avoid these additional steps we can use ICETOOL to sort detail records while keeping header(s) and/or trailer(s) in place. Let’s go with keeping one header (the first record) and one trailer (the last record) in place.
SORT Example with Header and Trailer
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1), // NOTIFY=&SYSUID //************************************************************ //STEP01 EXEC PGM=ICETOOL //TOOLMSG DD SYSOUT=* //DFSMSG DD SYSOUT=* //IN DD * 20080413 342008041300000010 00000000034 10191610022005030119890717FANDERSON 00000000034 21328400022005031019850705FLYONS 00000000034 C1817600022005030119000101FMITCHELL 00000000034 D1965890052005030120031203MBENTON 00000000034 B2529180062005030119940122MGONZALEZ 00000000034 42667570102005030119000101MMATHEW 00000000034 B2868070052005033019951005MSOLORIO 00000000034 C3548350072005030119660930FMACARIO 00000000034 F4078320032005032220050318FWILSON 0000000000034200804130000001000294138 /* //OUT DD DSN=... output file //TOOLIN DD * DATASORT FROM(IN) TO(OUT) HEADER TRAILER USING(CTL1) //CTL1CNTL DD * SORT FIELDS=(14,10,CH,A) /*
The output records in OUT will look as follows:
20080413 342008041300000010 00000000034 B2529180062005030119940122MGONZALEZ 00000000034 B2868070052005033019951005MSOLORIO 00000000034 C1817600022005030119000101FMITCHELL 00000000034 C3548350072005030119660930FMACARIO 00000000034 D1965890052005030120031203MBENTON 00000000034 F4078320032005032220050318FWILSON 00000000034 10191610022005030119890717FANDERSON 00000000034 21328400022005031019850705FLYONS 00000000034 42667570102005030119000101MMATHEW 0000000000034200804130000001000294138
Explanation:
- IN refers to input files for sorting
- OUT refers output file for sorting
- CTL1CNTL refers to sorting criteria.
- Here only CNTL is a defined word and that should be used.
- CTL1 can be replaced by any word of your preference.
- TOOLIN indicates how the sorting can be performed by using the defined criteria in CTL1CNTL.To exclude both needs to be mentioned in TOOLIN as seen above in the sample job.
With DATASORT, you specify the number of header records using HEADER, HEADER(n), FIRST or FIRST(n), and/or the number of trailer records using TRAILER, TRAILER(n), LAST, or LAST(n). Below job that shows how to keep two headers (the first two records) and three trailers (the last three records) in place.
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1), // NOTIFY=&SYSUID //************************************************************ //STEP01 EXEC PGM=ICETOOL //TOOLMSG DD SYSOUT=* //DFSMSG DD SYSOUT=* //IN1 DD DSN=... input file //OUT1 DD DSN=... output file //TOOLIN DD * DATASORT FROM(IN1) TO(OUT1) HEADER(2) TRAILER(3) USING(CTL1) //CTL1CNTL DD * SORT FIELDS=(21,8,ZD,D) /*
Examples
Scenario -1: Let’s say we want to SORT an input file by a section that is divided by a constant keyword or symbol or some unique identifier. We can use the WHEN=GROUP function, we can add group numbers after the end of the records. Then we can SORT by the group number and by the key. Finally, we can remove the group numbers.
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1), // NOTIFY=&SYSUID //************************************************************ //STEP01 EXEC PGM=ICEMAN //SYSOUT DD SYSOUT=* //SORTIN DD DSN=... input file (FB/10) //SORTOUT DD DSN=... output file (FB/10) //SYSIN DD * INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,5,CH,EQ,C'*****'), PUSH=(11:ID=8)) SORT FIELDS=(11,8,ZD,A,1,5,CH,A) OUTREC BUILD=(1,10) /*
Please note the above example is for the FB file so we were able to place the temporary group number after the end of the records and remove it later. But if you want to sort on the VB file then we don’t want to add it at the end as it would pad out every variable-length record to the same length. In this scenario, we need to add the group number between the RDW and the first data byte, and remove it later, so no padding would occur.
//SYSIN DD * INREC IFTHEN=(WHEN=INIT,BUILD=(1,4,5:8X,13:5)), IFTHEN=(WHEN=GROUP,BEGIN=(13,5,CH,EQ,C'*****'), PUSH=(5:ID=8)) SORT FIELDS=(5,8,ZD,A,13,5,CH,A) OUTREC BUILD=(1,4,5:13) /*
Let’s say our input data is having multiple subgroups divided by different unique identifiers. For example header record is identified by ‘HDR’ and has the date in ‘yyyy/mm/dd’ format. The trailer record is identified by ‘TRL’.
Input: HDR 2021/04/10 001 23.05- 002 5213.75+ 003 861.51+ 004 753.90- TRL T862143 HDR 2021/04/09 001 282.15+ 002 8.00- 003 1496.28+ TRL T201576 HDR 2021/03/11 001 123.86+ 002 98.07- 003 61.16+ TRL T031893 HDR 2021/04/09 001 106.27- 002 2308.00+ 003 96.72+ 004 206.99- 005 208.82- TRL T201573
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1), // NOTIFY=&SYSUID //************************************************************ //STEP01 EXEC PGM=ICEMAN //SYSOUT DD SYSOUT=* //SORTIN DD DSN=... input file //SORTOUT DD DSN=... output file //SYSIN DD * IFTHEN=(WHEN=GROUP,BEGIN=(1,3,CH,EQ,C'HDR'), PUSH=(31:6,10,41:ID=8)), IFTHEN=(WHEN=(1,3,CH,EQ,C'HDR'),OVERLAY=(49:C'A')), IFTHEN=(WHEN=(1,3,CH,EQ,C'TRL'),OVERLAY=(49:C'C')), IFTHEN=(WHEN=NONE,OVERLAY=(49:C'B')) SORT FIELDS=(31,10,CH,A, - date 41,8,ZD,A, - group number 49,1,CH,A, - A header,B data,C trailer 6,9,SFF,D) - amount BUILD=(1,30) /*
Output: HDR 2008/03/11 001 123.86+ 003 61.16+ 002 98.07- TRL T031893 HDR 2008/04/09 003 1496.28+ 001 282.15+ 002 8.00- TRL T201576 HDR 2008/04/09 002 2308.00+ 003 96.72+ 001 106.27- 004 206.99- 005 208.82- TRL T201573 HDR 2008/04/10 002 5213.75+ 003 861.51+ 001 23.05- 004 753.90- TRL T862143
Scenario -2: This example illustrates how you can SORT and INCLUDE groups of FB records depending on a value in the first record of each group. We propagate the value in the first record of the group to every record of the group, SORT and INCLUDE on the value, and then remove it.
Input File
1RPT.SRIHARI LINE 1 FOR REPORT 1 LINE 2 FOR REPORT 1 ... 1RPT.VICKY LINE 1 FOR REPORT 2 LINE 2 FOR REPORT 2 ... 1RPT.FRANK LINE 1 FOR REPORT 3 LINE 2 FOR REPORT 3 ... 1RPT.DAVID LINE 1 FOR REPORT 4 LINE 2 FOR REPORT 4 ...
INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,4,CH,EQ,C’RPT.’),
PUSH=(31:6,8))
OPTION EQUALS
SORT FIELDS=(31,8,CH,A)
OUTFIL INCLUDE=(31,8,CH,EQ,C’FRANK’,OR,
31,8,CH,EQ,C’SRIHARI’),BUILD=(1,30)
Each report starts with ‘RPT.reptname’ in positions 2-13. In the output dataset, we only want to include records for reports with specific reptname values, and the reptname values we want can change from run to run. We also want to sort by the reptname values in ascending order. For this example, let’s say we just want the SRIHARI and FRANK reports.
We use an IFTHEN WHEN=GROUP clause to propagate the reptname value to each record of the group. BEGIN indicates a group starts with ‘RPT.’ in positions 2-5. PUSH overlays the reptname value from the first record of the group (the ‘RPT.reptname’ record) at positions 31-38 (after the end of the record) in each record of the group including the first. After the IFTHEN GROUP clause is executed, the intermediate records look like this:
1RPT.SRIHARI LINE 1 FOR REPORT 1 SRIHARI LINE 2 FOR REPORT 1 SRIHARI … SRIHARI 1RPT.VICKY VICKY LINE 1 FOR REPORT 2 VICKY LINE 2 FOR REPORT 2 VICKY … VICKY 1RPT.FRANK FRANK LINE 1 FOR REPORT 3 FRANK LINE 2 FOR REPORT 3 FRANK … FRANK 1RPT.DAVID DAVID LINE 1 FOR REPORT 4 DAVID LINE 2 FOR REPORT 4 DAVID … DAVID
Note that the records of each group have the reptname value from the first record of that group in positions 31-38.
We use a SORT statement to sort ascending on the reptname in positions 31-38. We use the EQUALS option to ensure that records in the same group (that is, with the same reptname value) are kept in their original order. After the SORT statement is executed, the intermediate records look like this:
1RPT.DAVID DAVID LINE 1 FOR REPORT 4 DAVID LINE 2 FOR REPORT 4 DAVID … DAVID 1RPT.FRANK FRANK LINE 1 FOR REPORT 3 FRANK LINE 2 FOR REPORT 3 FRANK … FRANK 1RPT.SRIHARI SRIHARI LINE 1 FOR REPORT 1 SRIHARI LINE 2 FOR REPORT 1 SRIHARI … SRIHARI 1RPT.VICKY VICKY LINE 1 FOR REPORT 2 VICKY LINE 2 FOR REPORT 2 VICKY …. VICKY
We use an OUTFIL statement to only INCLUDE the records with a reptname of FRANK or SRIHARI in positions 31-38, and to remove the reptname from positions 31-38 so the included output records will be identical to the input records. After the OUTFIL statement is executed, the final output records look like this:
1RPT.FRANK LINE 1 FOR REPORT 3 LINE 2 FOR REPORT 3 ... 1RPT.SRIHARI LINE 1 FOR REPORT 1 LINE 2 FOR REPORT 1