Posted: Sun Nov 18, 2007 8:26 pm Post subject: Break program and Parallel processing to reduce the run-time
Parallel processing: breaks a report to pieces and processes it parallel to reduce the run-time
Code:
REPORT ZPARPROC.
************************************************************************
* This program implements parallel processing. Many time consuming
* report can be broken into equal pieces and run parallel, reducing
* the runtime (in this case searching for a string in the source code
* of a set of abaps). The main program (mother) creates CHILD_NU
* number of child programs(1-9),gives each of them a separate task, runs
* them parallel and then collects the end-result. The child program is
* actually embedded in the mother program, so the mother virtually
* gives birth of her children. A child that finished it's part, commits
* suicide: deletes itself at the end.
* Here is the result of a performonce test with 1-9 children processes
* for a light and a heavy task:
*
* children# light(Z* programs)(sec) heavy(R* programs)(sec)
* 1 51 664 100%
* 2 28 357
* 3 22 250
* 4 20 232
* 5 18 207 31%
* 6 19 209
* 7 18 219
* 8 21 202
* 9 20 211
*
* Conclusions:
* The performance starts to deteriorate when the children# exceeds a
* certain, task specific number (the children start to compete
* against each other for the same resources)
* The heavier the task - the more significant the performance gain.
************************************************************************
* The parent's data declaration
TABLES: TRDIR, TBTCO, INDX.
PARAMETERS: CHILD_NU TYPE I DEFAULT '3',
PATTERN(2) DEFAULT 'Z%'.
DATA: C,
CHILD(32) VALUE 'ZCHILD#',
JOBNAME(32),
ABAP_NAME(8).
DATA: BEGIN OF IV_TABLE OCCURS 10, "Table of intervals
LOW LIKE TRDIR-NAME,
HIGH LIKE TRDIR-NAME,
END OF IV_TABLE.
DATA: BEGIN OF ITAB OCCURS 500, "Table of program names
LINE(72),
END OF ITAB.
DATA: BEGIN OF CHILD_NAME_TAB OCCURS 10, "Table of the children's names
L(7), LL(23),
END OF CHILD_NAME_TAB.
DATA: BEGIN OF RESULT_TAB OCCURS 100, "The result of a single child
NAME LIKE TRDIR-NAME,
END OF RESULT_TAB.
DATA: BEGIN OF END_RESULT_TAB OCCURS 100, "The table of the end-results
NAME LIKE TRDIR-NAME,
END OF END_RESULT_TAB.
* Evenly distribute the work between the children
* With EXEC SQL it would be nicer, but less generic
DATA: INDEX TYPE I.
DATA: LINE_NUM TYPE I.
DATA: INTERVAL TYPE I.
DATA: BEGIN OF RANGE OCCURS 1000,
NAME LIKE TRDIR-NAME,
END OF RANGE.
* Timestamp
GET TIME. WRITE:/ SY-UZEIT. SKIP.
* Timestamp
SELECT * FROM TRDIR WHERE NAME LIKE PATTERN. "Fill up the range table
IF NOT TRDIR-NAME IS INITIAL.
RANGE = TRDIR-NAME.
APPEND RANGE.
ENDIF.
ENDSELECT.
SORT RANGE BY NAME.
DESCRIBE TABLE RANGE LINES LINE_NUM.
INTERVAL = LINE_NUM / CHILD_NU. "This will be the length of the
"working range of a single child
INDEX = 0.
DO. "Create the table of distinct ranges: 1 for each child
INDEX = INDEX + 1.
READ TABLE RANGE INDEX INDEX.
IF SY-SUBRC <> 0.
EXIT.
ENDIF.
IV_TABLE-LOW = RANGE-NAME.
INDEX = INDEX + INTERVAL.
READ TABLE RANGE INDEX INDEX.
IF SY-SUBRC <> 0.
INDEX = LINE_NUM.
READ TABLE RANGE INDEX INDEX.
IV_TABLE-HIGH = RANGE-NAME.
APPEND IV_TABLE.
EXIT.
ENDIF.
IV_TABLE-HIGH = RANGE-NAME.
APPEND IV_TABLE.
ENDDO.
* Create the children
DATA: NUM.
DO CHILD_NU TIMES. "Loop on the children
NUM = SY-INDEX. "The index of the child
CLEAR ITAB. REFRESH ITAB.
READ REPORT SY-REPID INTO ITAB.
LOOP AT ITAB.
C = ITAB+1(1).
IF C <> '@'. "Give birth of a child:
DELETE ITAB. "Filter it out from mamy's belly
ELSE.
SHIFT ITAB LEFT BY 2 PLACES.
MODIFY ITAB.
ENDIF.
ENDLOOP.
*
LOOP AT ITAB. "Give the child identity and a task to work on
REPLACE '#' WITH NUM INTO ITAB.
READ TABLE IV_TABLE INDEX NUM.
REPLACE '&' WITH IV_TABLE-LOW INTO ITAB.
IF SY-SUBRC = 0.MODIFY ITAB.CONTINUE.ENDIF.
REPLACE '$' WITH IV_TABLE-HIGH INTO ITAB.
MODIFY ITAB.
ENDLOOP.
*
CHILD+6(1) = NUM. "At last: create the child
INSERT REPORT CHILD FROM ITAB.
*
JOBNAME = CHILD. "Create a distinct job name to run the child in bg
JOBNAME+8(8) = SY-DATUM.
JOBNAME+16(8) = SY-UZEIT.
*
CHILD_NAME_TAB = JOBNAME. "Save the name of my wonderful child
APPEND CHILD_NAME_TAB.
*
PERFORM CREATE_A_CHILD_JOB. "Start the child
ENDDO.
* Wait for my children to complete their task and get the result
DO.
LOOP AT CHILD_NAME_TAB.
SELECT * FROM TBTCO WHERE JOBNAME = CHILD_NAME_TAB.
ENDSELECT.
IF TBTCO-STATUS = 'F' OR TBTCO-STATUS = 'A'.
REFRESH RESULT_TAB.
IMPORT RESULT_TAB FROM SHARED BUFFER INDX(ST) ID CHILD_NAME_TAB-L.
LOOP AT RESULT_TAB. "Merge the results in one single table
END_RESULT_TAB = RESULT_TAB.
APPEND END_RESULT_TAB.
ENDLOOP.
DELETE CHILD_NAME_TAB.
ENDIF.
ENDLOOP.
IF SY-SUBRC <> 0. EXIT. ENDIF.
ENDDO.
* Hooray, all the children are finished
LOOP AT END_RESULT_TAB.
WRITE: END_RESULT_TAB.
ENDLOOP.
* Timestamp
GET TIME. SKIP. WRITE:/ SY-UZEIT.
* Timestamp
* Schedule the child to run in background
FORM CREATE_A_CHILD_JOB.
DATA: JOBCOUNT LIKE TBTCJOB-JOBCOUNT.
ABAP_NAME = CHILD.
CALL FUNCTION 'JOB_OPEN'
EXPORTING
JOBNAME = JOBNAME
IMPORTING
JOBCOUNT = JOBCOUNT.
CALL FUNCTION 'JOB_SUBMIT'
EXPORTING
JOBNAME = JOBNAME
JOBCOUNT = JOBCOUNT
AUTHCKNAM = SY-UNAME
REPORT = ABAP_NAME.
CALL FUNCTION 'JOB_CLOSE'
EXPORTING
JOBNAME = JOBNAME
JOBCOUNT = JOBCOUNT
STRTIMMED = 'X'.
ENDFORM.
* Here is the child template in the mothers tummy:
*@program zchild#.
*@tables: trdir, indx.
*@data: begin of itab occurs 500,
*@ line(72),
*@end of itab.
*@data: begin of result_tab occurs 100,
*@ name like trdir-name,
*@end of result_tab.
*@data: string(20) value 'loadi'.
*@
*@select * from trdir where name between
*@'&'
*@and
*@'$'.
*@ read report trdir-name into itab.
*@ loop at itab.
*@ if itab-line cs string.
*@ write: trdir-name.
*@ result_tab = trdir-name.
*@ append result_tab.
*@ exit.
*@ endif.
*@ endloop.
*@endselect.
*@* Export the result to shared buffer
*@export result_tab to shared buffer INDX(ST) ID 'ZCHILD#'.
*@* The child commits a suicide after it has finished it's task
*@delete report 'ZCHILD#'.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
All product names are trademarks of their respective companies. SAPNET.RU websites are in no way affiliated with SAP AG. SAP, SAP R/3, R/3 software, mySAP, ABAP, BAPI, xApps, SAP NetWeaver and any other are registered trademarks of SAP AG. Every effort is made to ensure content integrity. Use information on this site at your own risk.
