SAP R/3 форум ABAP консультантов
Russian ABAP Developer's Club

Home - FAQ - Search - Memberlist - Usergroups - Profile - Log in to check your private messages - Register - Log in - English
Blogs - Weblogs News

Break program and Parallel processing to reduce the run-time



 
Post new topic   Reply to topic    Russian ABAP Developer's Club Forum Index -> Programming Techniques | Приемы программирования
View previous topic :: View next topic  
Author Message
admin
Администратор
Администратор



Joined: 01 Sep 2007
Posts: 1639

PostPosted: Sun Nov 18, 2007 8:26 pm    Post subject: Break program and Parallel processing to reduce the run-time Reply with quote

Parallel processing: breaks a report to pieces and processes it parallel to reduce the run-time

Code:
REPORT ZPARPROC.
************************************************************************
* This program implements parallel processing. Many time consuming
* report can be broken into equal pieces and run parallel, reducing
* the runtime (in this case searching for a string in the source code
* of a set of abaps). The main program (mother) creates CHILD_NU
* number of child programs(1-9),gives each of them a separate task, runs
* them parallel and then collects the end-result. The child program is
* actually embedded in the mother program, so the mother virtually
* gives birth of her children. A child that finished it's part, commits
* suicide: deletes itself at the end.
* Here is the result of a performonce test with 1-9 children processes
* for a light and a heavy task:
*
* children#      light(Z* programs)(sec)      heavy(R* programs)(sec)
* 1              51                           664  100%
* 2              28                           357
* 3              22                           250
* 4              20                           232
* 5              18                           207  31%
* 6              19                           209
* 7              18                           219
* 8              21                           202 
* 9              20                           211
*
* Conclusions:
* The performance starts to deteriorate when the children# exceeds a
*     certain, task specific number (the children start to compete
*     against each other for the same resources)
* The heavier the task - the more significant the performance gain.
************************************************************************
* The parent's data declaration
TABLES: TRDIR, TBTCO, INDX.
PARAMETERS: CHILD_NU TYPE I DEFAULT '3',
            PATTERN(2) DEFAULT 'Z%'.
DATA: C,
      CHILD(32) VALUE 'ZCHILD#',
      JOBNAME(32),
      ABAP_NAME(8).
DATA: BEGIN OF IV_TABLE OCCURS 10,        "Table of intervals
      LOW LIKE TRDIR-NAME,
      HIGH LIKE TRDIR-NAME,
END OF IV_TABLE.
DATA: BEGIN OF ITAB OCCURS 500,           "Table of program names
      LINE(72),
END OF ITAB.
DATA: BEGIN OF CHILD_NAME_TAB OCCURS 10,  "Table of the children's names
      L(7), LL(23),
END OF CHILD_NAME_TAB.
DATA: BEGIN OF RESULT_TAB OCCURS 100,     "The result of a single child
      NAME LIKE TRDIR-NAME,
END OF RESULT_TAB.
DATA: BEGIN OF END_RESULT_TAB OCCURS 100, "The table of the end-results
      NAME LIKE TRDIR-NAME,
END OF END_RESULT_TAB.

* Evenly distribute the work between the children
* With EXEC SQL it would be nicer, but less generic
DATA: INDEX TYPE I.
DATA: LINE_NUM TYPE I.
DATA: INTERVAL TYPE I.
DATA: BEGIN OF RANGE OCCURS 1000,
      NAME LIKE TRDIR-NAME,
END OF RANGE.
* Timestamp
GET TIME. WRITE:/ SY-UZEIT. SKIP.
* Timestamp
SELECT * FROM TRDIR WHERE NAME LIKE PATTERN.    "Fill up the range table
  IF NOT TRDIR-NAME IS INITIAL.
    RANGE = TRDIR-NAME.
    APPEND RANGE.
  ENDIF.
ENDSELECT.
SORT RANGE BY NAME.
DESCRIBE TABLE RANGE LINES LINE_NUM.
INTERVAL = LINE_NUM / CHILD_NU.       "This will be the length of the
                                      "working range of a single child
INDEX = 0.
DO.               "Create the table of distinct ranges: 1 for each child
  INDEX = INDEX + 1.
  READ TABLE RANGE INDEX INDEX.
  IF SY-SUBRC <> 0.
    EXIT.
  ENDIF.
  IV_TABLE-LOW = RANGE-NAME.
  INDEX = INDEX + INTERVAL.
  READ TABLE RANGE INDEX INDEX.
  IF SY-SUBRC <> 0.
    INDEX = LINE_NUM.
    READ TABLE RANGE INDEX INDEX.
    IV_TABLE-HIGH = RANGE-NAME.
    APPEND IV_TABLE.
    EXIT.
  ENDIF.
  IV_TABLE-HIGH = RANGE-NAME.
  APPEND IV_TABLE.
ENDDO.

* Create the children
DATA: NUM.
DO CHILD_NU TIMES.                     "Loop on the children
  NUM = SY-INDEX.                      "The index of the child
  CLEAR ITAB. REFRESH ITAB.
  READ REPORT SY-REPID INTO ITAB.
  LOOP AT ITAB.
    C = ITAB+1(1).
    IF C <> '@'.                     "Give birth of a child:
      DELETE ITAB.                   "Filter it out from mamy's belly
    ELSE.
      SHIFT ITAB LEFT BY 2 PLACES.
      MODIFY ITAB.
    ENDIF.
  ENDLOOP.
*
  LOOP AT ITAB.           "Give the child identity and a task to work on
    REPLACE '#' WITH NUM INTO ITAB.
    READ TABLE IV_TABLE INDEX NUM.
    REPLACE '&' WITH IV_TABLE-LOW INTO ITAB.
    IF SY-SUBRC = 0.MODIFY ITAB.CONTINUE.ENDIF.
    REPLACE '$' WITH IV_TABLE-HIGH INTO ITAB.
    MODIFY ITAB.
  ENDLOOP.
*
  CHILD+6(1) = NUM.      "At last: create the child
  INSERT REPORT CHILD FROM ITAB.
*
  JOBNAME = CHILD.    "Create a distinct job name to run the child in bg
  JOBNAME+8(8) = SY-DATUM.
  JOBNAME+16(8) = SY-UZEIT.
*
  CHILD_NAME_TAB = JOBNAME.   "Save the name of my wonderful child
  APPEND CHILD_NAME_TAB.
*
  PERFORM CREATE_A_CHILD_JOB.  "Start the child
ENDDO.

* Wait for my children to complete their task and get the result
DO.
  LOOP AT CHILD_NAME_TAB.
    SELECT * FROM TBTCO WHERE JOBNAME = CHILD_NAME_TAB.
    ENDSELECT.
    IF TBTCO-STATUS = 'F' OR TBTCO-STATUS = 'A'.
      REFRESH RESULT_TAB.
      IMPORT RESULT_TAB FROM SHARED BUFFER INDX(ST) ID CHILD_NAME_TAB-L.
      LOOP AT RESULT_TAB.         "Merge the results in one single table
        END_RESULT_TAB = RESULT_TAB.
        APPEND END_RESULT_TAB.
      ENDLOOP.
      DELETE CHILD_NAME_TAB.
    ENDIF.
  ENDLOOP.
  IF SY-SUBRC <> 0. EXIT. ENDIF.
ENDDO.

* Hooray, all the children are finished
LOOP AT END_RESULT_TAB.
  WRITE: END_RESULT_TAB.
ENDLOOP.

* Timestamp
GET TIME. SKIP. WRITE:/ SY-UZEIT.
* Timestamp

* Schedule the child to run in background
FORM CREATE_A_CHILD_JOB.
  DATA: JOBCOUNT LIKE TBTCJOB-JOBCOUNT.
  ABAP_NAME = CHILD.
  CALL FUNCTION 'JOB_OPEN'
       EXPORTING
            JOBNAME  = JOBNAME
       IMPORTING
            JOBCOUNT = JOBCOUNT.
  CALL FUNCTION 'JOB_SUBMIT'
       EXPORTING
            JOBNAME   = JOBNAME
            JOBCOUNT  = JOBCOUNT
            AUTHCKNAM = SY-UNAME
            REPORT    = ABAP_NAME.
  CALL FUNCTION 'JOB_CLOSE'
       EXPORTING
            JOBNAME   = JOBNAME
            JOBCOUNT  = JOBCOUNT
            STRTIMMED = 'X'.
ENDFORM.

* Here is the child template in the mothers tummy:

*@program zchild#.
*@tables: trdir, indx.
*@data: begin of itab occurs 500,
*@      line(72),
*@end of itab.
*@data: begin of result_tab occurs 100,
*@      name like trdir-name,
*@end of result_tab.
*@data: string(20) value 'loadi'.
*@
*@select * from trdir where name between
*@'&'
*@and
*@'$'.
*@  read report trdir-name into itab.
*@  loop at itab.
*@    if itab-line cs string.
*@      write: trdir-name.
*@      result_tab = trdir-name.
*@      append result_tab.
*@      exit.
*@    endif.
*@  endloop.
*@endselect.
*@* Export the result to shared buffer
*@export result_tab to shared buffer INDX(ST) ID 'ZCHILD#'.
*@* The child commits a suicide after it has finished it's task
*@delete report 'ZCHILD#'.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Russian ABAP Developer's Club Forum Index -> Programming Techniques | Приемы программирования All times are GMT + 4 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


All product names are trademarks of their respective companies. SAPNET.RU websites are in no way affiliated with SAP AG.
SAP, SAP R/3, R/3 software, mySAP, ABAP, BAPI, xApps, SAP NetWeaver and any other are registered trademarks of SAP AG.
Every effort is made to ensure content integrity. Use information on this site at your own risk.