climateprediction.net home page
Task 22445989

Task 22445989

Name oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0
Workunit 12292383
Created 13 Jun 2024, 1:53:59 UTC
Sent 14 Jun 2024, 2:04:20 UTC
Report deadline 13 Aug 2024, 2:04:20 UTC
Received 15 Jun 2024, 12:10:16 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1549270
Run time 15 hours 34 min 41 sec
CPU time 15 hours 14 min 54 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 3.80 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,014.97 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,352.18 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
026
  20:48:16 STEP 1484 H= 371:00 +CPU= 29.640
  20:48:46 STEP 1485 H= 371:15 +CPU= 28.802
  20:49:16 STEP 1486 H= 371:30 +CPU= 29.227
  20:49:44 STEP 1487 H= 371:45 +CPU= 28.180
  20:50:26 STEP 1488 H= 372:00 +CPU= 40.070
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001488
  20:50:55 STEP 1489 H= 372:15 +CPU= 28.756
  20:51:25 STEP 1490 H= 372:30 +CPU= 28.904
  20:51:55 STEP 1491 H= 372:45 +CPU= 28.940
  20:52:25 STEP 1492 H= 373:00 +CPU= 29.802
  20:52:55 STEP 1493 H= 373:15 +CPU= 28.786
  20:53:25 STEP 1494 H= 373:30 +CPU= 29.080
  20:53:54 STEP 1495 H= 373:45 +CPU= 28.800
  20:54:27 STEP 1496 H= 374:00 +CPU= 30.653
  20:54:59 STEP 1497 H= 374:15 +CPU= 29.860
  20:55:30 STEP 1498 H= 374:30 +CPU= 29.221
  20:56:02 STEP 1499 H= 374:45 +CPU= 30.009
  20:56:40 STEP 1500 H= 375:00 +CPU= 36.639
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001500
  20:57:10 STEP 1501 H= 375:15 +CPU= 28.929
  20:57:40 STEP 1502 H= 375:30 +CPU= 28.721
  20:58:10 STEP 1503 H= 375:45 +CPU= 29.146
  20:58:40 STEP 1504 H= 376:00 +CPU= 29.309
  20:59:10 STEP 1505 H= 376:15 +CPU= 28.918
  20:59:40 STEP 1506 H= 376:30 +CPU= 28.804
  21:00:09 STEP 1507 H= 376:45 +CPU= 28.872
  21:00:40 STEP 1508 H= 377:00 +CPU= 29.708
  21:01:10 STEP 1509 H= 377:15 +CPU= 29.169
  21:01:42 STEP 1510 H= 377:30 +CPU= 30.169
  21:02:12 STEP 1511 H= 377:45 +CPU= 29.051
  21:02:49 STEP 1512 H= 378:00 +CPU= 36.252
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001512
  21:03:18 STEP 1513 H= 378:15 +CPU= 27.938
  21:03:48 STEP 1514 H= 378:30 +CPU= 28.922
  21:04:18 STEP 1515 H= 378:45 +CPU= 28.831
  21:04:46 STEP 1516 H= 379:00 +CPU= 28.218
  21:05:14 STEP 1517 H= 379:15 +CPU= 27.269
  21:05:41 STEP 1518 H= 379:30 +CPU= 27.393
  21:06:09 STEP 1519 H= 379:45 +CPU= 27.601
  21:06:39 STEP 1520 H= 380:00 +CPU= 29.308
  21:07:08 STEP 1521 H= 380:15 +CPU= 28.289
  21:07:37 STEP 1522 H= 380:30 +CPU= 28.098
  21:08:07 STEP 1523 H= 380:45 +CPU= 28.966
  21:08:43 STEP 1524 H= 381:00 +CPU= 35.572
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001524
  21:09:12 STEP 1525 H= 381:15 +CPU= 28.014
  21:09:41 STEP 1526 H= 381:30 +CPU= 28.930
  21:10:10 STEP 1527 H= 381:45 +CPU= 28.486
  21:10:40 STEP 1528 H= 382:00 +CPU= 29.020
  21:11:09 STEP 1529 H= 382:15 +CPU= 28.354
  21:11:38 STEP 1530 H= 382:30 +CPU= 28.475
  21:12:07 STEP 1531 H= 382:45 +CPU= 28.420
  21:12:36 STEP 1532 H= 383:00 +CPU= 28.818
  21:13:06 STEP 1533 H= 383:15 +CPU= 28.647
  21:13:36 STEP 1534 H= 383:30 +CPU= 29.060
  21:14:08 STEP 1535 H= 383:45 +CPU= 29.999
  21:14:51 STEP 1536 H= 384:00 +CPU= 41.352
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0_r2085640283_15.zip
Uploading the intermediate file: upload_file_15.zip
  21:15:23 STEP 1537 H= 384:15 +CPU= 29.731
  21:15:53 STEP 1538 H= 384:30 +CPU= 29.443
  21:16:23 STEP 1539 H= 384:45 +CPU= 28.789
  21:16:55 STEP 1540 H= 385:00 +CPU= 30.370
  21:17:28 STEP 1541 H= 385:15 +CPU= 30.685
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.668] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4412MB (heap), 5025MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.670] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.670] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.670] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.670] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.670] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4412 MB (maxheap), 5025 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212632:1718418392:45360.671] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212633:1718418393:45361.671] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Funk2:1:1:2515:2515] [20240614:212633:1718418393:45361.671] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000F4E401  ec_phys_drv_              637  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a34o
(argv4) batchid: 1018
(argv5) wuid: 12292383
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6dabb51f3091870617c2b91b499df7a4 to: /var/lib/boinc-client/slots/5/oifs_43r3_bl_a34o_2016092300_20_1018_12292383.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_bl_a34o_2016092300_20_1018_12292383.zip
ic_ancil_file: ic_ancil_12292383
ifsdata_file: ifsdata_12292383
climate_data_file: clim_data_12292383
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/5/ic_ancil_12292383.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12292383.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3a8f0ebee2af12992282bae9af8ee0c1 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12292383.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12292383.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12292383.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12292383.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12292383.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12292383.xml
-- Model is restarting --
Adjusting last_iter, 1542, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12292383.xml
last_cpu_time: 43905
upload_file_number: 16
last_iter: 1537
last_upload: 1382400
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0_r2085640283
The child process has been launched with process id: 2435
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Funk2:1:1:2435:2435] [20240615:030051:1718438451:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  03:01:28 STEP 1536 H= 384:00 +CPU= 29.887
  03:01:56 STEP 1537 H= 384:15 +CPU= 28.013
  03:02:24 STEP 1538 H= 384:30 +CPU= 27.711
  03:02:55 STEP 1539 H= 384:45 +CPU= 29.449
  03:03:24 STEP 1540 H= 385:00 +CPU= 28.713
  03:03:53 STEP 1541 H= 385:15 +CPU= 28.035
  03:04:21 STEP 1542 H= 385:30 +CPU= 27.863
  03:04:48 STEP 1543 H= 385:45 +CPU= 27.236
  03:05:17 STEP 1544 H= 386:00 +CPU= 28.715
  03:05:47 STEP 1545 H= 386:15 +CPU= 28.449
  03:06:18 STEP 1546 H= 386:30 +CPU= 29.366
  03:06:45 STEP 1547 H= 386:45 +CPU= 27.200
  03:07:20 STEP 1548 H= 387:00 +CPU= 34.182
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001548
  03:07:48 STEP 1549 H= 387:15 +CPU= 27.883
  03:08:15 STEP 1550 H= 387:30 +CPU= 27.259
  03:08:43 STEP 1551 H= 387:45 +CPU= 27.280
  03:09:11 STEP 1552 H= 388:00 +CPU= 27.978
  03:09:39 STEP 1553 H= 388:15 +CPU= 27.616
  03:10:07 STEP 1554 H= 388:30 +CPU= 28.352
  03:10:36 STEP 1555 H= 388:45 +CPU= 28.743
  03:11:05 STEP 1556 H= 389:00 +CPU= 28.622
  03:11:33 STEP 1557 H= 389:15 +CPU= 27.392
  03:12:01 STEP 1558 H= 389:30 +CPU= 27.881
  03:12:30 STEP 1559 H= 389:45 +CPU= 27.888
  03:13:04 STEP 1560 H= 390:00 +CPU= 34.043
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001560
  03:13:32 STEP 1561 H= 390:15 +CPU= 27.316
  03:14:00 STEP 1562 H= 390:30 +CPU= 28.099
  03:14:30 STEP 1563 H= 390:45 +CPU= 28.143
  03:14:59 STEP 1564 H= 391:00 +CPU= 28.853
  03:15:28 STEP 1565 H= 391:15 +CPU= 28.077
  03:15:57 STEP 1566 H= 391:30 +CPU= 28.395
  03:16:26 STEP 1567 H= 391:45 +CPU= 28.317
  03:16:56 STEP 1568 H= 392:00 +CPU= 28.972
  03:17:25 STEP 1569 H= 392:15 +CPU= 28.296
  03:17:53 STEP 1570 H= 392:30 +CPU= 28.110
  03:18:22 STEP 1571 H= 392:45 +CPU= 28.440
  03:18:58 STEP 1572 H= 393:00 +CPU= 34.880
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001572
  03:19:27 STEP 1573 H= 393:15 +CPU= 28.508
  03:19:56 STEP 1574 H= 393:30 +CPU= 28.088
  03:20:25 STEP 1575 H= 393:45 +CPU= 28.527
  03:20:55 STEP 1576 H= 394:00 +CPU= 28.825
  03:21:23 STEP 1577 H= 394:15 +CPU= 27.805
  03:21:52 STEP 1578 H= 394:30 +CPU= 28.399
  03:22:20 STEP 1579 H= 394:45 +CPU= 27.362
  03:22:49 STEP 1580 H= 395:00 +CPU= 28.698
  03:23:19 STEP 1581 H= 395:15 +CPU= 28.450
  03:23:59 STEP 1582 H= 395:30 +CPU= 34.036
  03:24:39 STEP 1583 H= 395:45 +CPU= 33.326
  03:25:25 STEP 1584 H= 396:00 +CPU= 42.025
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001584
  03:25:54 STEP 1585 H= 396:15 +CPU= 28.303
  03:26:22 STEP 1586 H= 396:30 +CPU= 27.593
  03:26:51 STEP 1587 H= 396:45 +CPU= 28.220
  03:27:21 STEP 1588 H= 397:00 +CPU= 28.674
  03:27:50 STEP 1589 H= 397:15 +CPU= 28.631
  03:28:18 STEP 1590 H= 397:30 +CPU= 27.633
  03:28:47 STEP 1591 H= 397:45 +CPU= 28.014
  03:29:16 STEP 1592 H= 398:00 +CPU= 28.623
  03:29:44 STEP 1593 H= 398:15 +CPU= 28.201
  03:30:12 STEP 1594 H= 398:30 +CPU= 27.671
  03:30:40 STEP 1595 H= 398:45 +CPU= 27.204
  03:31:15 STEP 1596 H= 399:00 +CPU= 35.221
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001596
  03:31:45 STEP 1597 H= 399:15 +CPU= 28.700
  03:32:14 STEP 1598 H= 399:30 +CPU= 28.342
  03:32:42 STEP 1599 H= 399:45 +CPU= 27.977
  03:33:12 STEP 1600 H= 400:00 +CPU= 28.605
  03:33:40 STEP 1601 H= 400:15 +CPU= 27.996
  03:34:09 STEP 1602 H= 400:30 +CPU= 28.283
  03:34:39 STEP 1603 H= 400:45 +CPU= 28.827
  03:35:08 STEP 1604 H= 401:00 +CPU= 28.975
  03:35:36 STEP 1605 H= 401:15 +CPU= 27.609
  03:36:06 STEP 1606 H= 401:30 +CPU= 28.596
  03:36:35 STEP 1607 H= 401:45 +CPU= 28.756
  03:37:15 STEP 1608 H= 402:00 +CPU= 36.880
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001608
  03:37:49 STEP 1609 H= 402:15 +CPU= 31.019
  03:38:16 STEP 1610 H= 402:30 +CPU= 27.177
  03:38:44 STEP 1611 H= 402:45 +CPU= 27.820
  03:39:13 STEP 1612 H= 403:00 +CPU= 28.752
  03:39:42 STEP 1613 H= 403:15 +CPU= 28.257
  03:40:11 STEP 1614 H= 403:30 +CPU= 27.794
  03:40:38 STEP 1615 H= 403:45 +CPU= 27.224
  03:41:11 STEP 1616 H= 404:00 +CPU= 30.627
  03:41:45 STEP 1617 H= 404:15 +CPU= 30.685
  03:42:20 STEP 1618 H= 404:30 +CPU= 31.694
  03:42:53 STEP 1619 H= 404:45 +CPU= 30.535
  03:43:35 STEP 1620 H= 405:00 +CPU= 38.693
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001620
  03:44:06 STEP 1621 H= 405:15 +CPU= 30.147
  03:44:35 STEP 1622 H= 405:30 +CPU= 28.205
  03:45:03 STEP 1623 H= 405:45 +CPU= 27.638
  03:45:31 STEP 1624 H= 406:00 +CPU= 28.004
  03:45:59 STEP 1625 H= 406:15 +CPU= 27.331
  03:46:26 STEP 1626 H= 406:30 +CPU= 27.169
  03:46:53 STEP 1627 H= 406:45 +CPU= 27.279
  03:47:21 STEP 1628 H= 407:00 +CPU= 27.422
  03:47:48 STEP 1629 H= 407:15 +CPU= 27.208
  03:48:15 STEP 1630 H= 407:30 +CPU= 26.979
  03:48:42 STEP 1631 H= 407:45 +CPU= 27.037
  03:49:20 STEP 1632 H= 408:00 +CPU= 37.640
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0_r2085640283_16.zip
Uploading the intermediate file: upload_file_16.zip
  03:49:48 STEP 1633 H= 408:15 +CPU= 27.489
  03:50:15 STEP 1634 H= 408:30 +CPU= 27.338
  03:50:42 STEP 1635 H= 408:45 +CPU= 27.036
  03:51:10 STEP 1636 H= 409:00 +CPU= 27.665
  03:51:37 STEP 1637 H= 409:15 +CPU= 27.041
  03:52:05 STEP 1638 H= 409:30 +CPU= 27.263
  03:52:32 STEP 1639 H= 409:45 +CPU= 27.140
  03:53:02 STEP 1640 H= 410:00 +CPU= 29.085
  03:53:34 STEP 1641 H= 410:15 +CPU= 29.941
  03:54:06 STEP 1642 H= 410:30 +CPU= 29.902
  03:54:37 STEP 1643 H= 410:45 +CPU= 28.969
  03:55:18 STEP 1644 H= 411:00 +CPU= 37.800
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001644
  03:55:46 STEP 1645 H= 411:15 +CPU= 27.627
  03:56:14 STEP 1646 H= 411:30 +CPU= 28.006
  03:56:41 STEP 1647 H= 411:45 +CPU= 27.408
  03:57:09 STEP 1648 H= 412:00 +CPU= 27.919
  03:57:37 STEP 1649 H= 412:15 +CPU= 27.516
  03:58:05 STEP 1650 H= 412:30 +CPU= 27.538
  03:58:34 STEP 1651 H= 412:45 +CPU= 28.131
  03:59:02 STEP 1652 H= 413:00 +CPU= 28.090
  03:59:31 STEP 1653 H= 413:15 +CPU= 27.859
  03:59:59 STEP 1654 H= 413:30 +CPU= 27.519
  04:00:28 STEP 1655 H= 413:45 +CPU= 28.343
  04:01:04 STEP 1656 H= 414:00 +CPU= 35.377
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001656
  04:01:33 STEP 1657 H= 414:15 +CPU= 28.771
  04:02:02 STEP 1658 H= 414:30 +CPU= 27.736
  04:02:30 STEP 1659 H= 414:45 +CPU= 27.809
  04:02:59 STEP 1660 H= 415:00 +CPU= 28.443
  04:03:26 STEP 1661 H= 415:15 +CPU= 27.123
  04:03:54 STEP 1662 H= 415:30 +CPU= 27.676
  04:04:23 STEP 1663 H= 415:45 +CPU= 28.062
  04:04:52 STEP 1664 H= 416:00 +CPU= 28.275
  04:05:20 STEP 1665 H= 416:15 +CPU= 27.781
  04:05:48 STEP 1666 H= 416:30 +CPU= 27.602
  04:06:15 STEP 1667 H= 416:45 +CPU= 27.273
  04:06:52 STEP 1668 H= 417:00 +CPU= 35.490
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001668
  04:07:21 STEP 1669 H= 417:15 +CPU= 28.212
  04:07:50 STEP 1670 H= 417:30 +CPU= 28.177
  04:08:19 STEP 1671 H= 417:45 +CPU= 28.328
  04:08:48 STEP 1672 H= 418:00 +CPU= 28.706
  04:09:17 STEP 1673 H= 418:15 +CPU= 27.904
  04:09:46 STEP 1674 H= 418:30 +CPU= 28.388
  04:10:15 STEP 1675 H= 418:45 +CPU= 27.872
  04:10:44 STEP 1676 H= 419:00 +CPU= 28.600
  04:11:13 STEP 1677 H= 419:15 +CPU= 28.359
  04:11:42 STEP 1678 H= 419:30 +CPU= 27.985
  04:12:10 STEP 1679 H= 419:45 +CPU= 27.873
  04:12:50 STEP 1680 H= 420:00 +CPU= 38.931
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001680
  04:13:18 STEP 1681 H= 420:15 +CPU= 27.685
  04:13:46 STEP 1682 H= 420:30 +CPU= 27.988
  04:14:15 STEP 1683 H= 420:45 +CPU= 28.075
  04:14:43 STEP 1684 H= 421:00 +CPU= 27.844
  04:15:11 STEP 1685 H= 421:15 +CPU= 27.719
  04:15:40 STEP 1686 H= 421:30 +CPU= 28.051
  04:16:09 STEP 1687 H= 421:45 +CPU= 28.399
  04:16:38 STEP 1688 H= 422:00 +CPU= 28.815
  04:17:09 STEP 1689 H= 422:15 +CPU= 29.562
  04:17:39 STEP 1690 H= 422:30 +CPU= 28.870
  04:18:08 STEP 1691 H= 422:45 +CPU= 28.121
  04:18:44 STEP 1692 H= 423:00 +CPU= 35.064
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001692
  04:19:12 STEP 1693 H= 423:15 +CPU= 28.051
  04:19:41 STEP 1694 H= 423:30 +CPU= 27.991
  04:20:11 STEP 1695 H= 423:45 +CPU= 29.080
  04:20:41 STEP 1696 H= 424:00 +CPU= 29.073
  04:21:09 STEP 1697 H= 424:15 +CPU= 27.904
  04:21:38 STEP 1698 H= 424:30 +CPU= 28.552
  04:22:07 STEP 1699 H= 424:45 +CPU= 28.064
  04:22:35 STEP 1700 H= 425:00 +CPU= 27.997
  04:23:03 STEP 1701 H= 425:15 +CPU= 27.344
  04:23:30 STEP 1702 H= 425:30 +CPU= 27.567
  04:23:59 STEP 1703 H= 425:45 +CPU= 28.265
  04:24:35 STEP 1704 H= 426:00 +CPU= 35.339
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001704
  04:25:04 STEP 1705 H= 426:15 +CPU= 28.059
  04:25:32 STEP 1706 H= 426:30 +CPU= 27.801
  04:26:01 STEP 1707 H= 426:45 +CPU= 28.230
  04:26:32 STEP 1708 H= 427:00 +CPU= 29.869
  04:27:01 STEP 1709 H= 427:15 +CPU= 28.188
  04:27:32 STEP 1710 H= 427:30 +CPU= 29.375
  04:28:00 STEP 1711 H= 427:45 +CPU= 27.431
  04:28:27 STEP 1712 H= 428:00 +CPU= 27.714
  04:28:54 STEP 1713 H= 428:15 +CPU= 26.959
  04:29:21 STEP 1714 H= 428:30 +CPU= 26.914
  04:29:48 STEP 1715 H= 428:45 +CPU= 26.890
  04:30:22 STEP 1716 H= 429:00 +CPU= 33.307
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001716
  04:30:49 STEP 1717 H= 429:15 +CPU= 26.859
  04:31:15 STEP 1718 H= 429:30 +CPU= 26.899
  04:31:42 STEP 1719 H= 429:45 +CPU= 26.898
  04:32:10 STEP 1720 H= 430:00 +CPU= 27.459
  04:32:39 STEP 1721 H= 430:15 +CPU= 28.732
  04:33:08 STEP 1722 H= 430:30 +CPU= 28.232
  04:33:36 STEP 1723 H= 430:45 +CPU= 27.427
  04:34:04 STEP 1724 H= 431:00 +CPU= 27.785
  04:34:32 STEP 1725 H= 431:15 +CPU= 27.673
  04:34:59 STEP 1726 H= 431:30 +CPU= 27.354
  04:35:27 STEP 1727 H= 431:45 +CPU= 27.246
  04:36:04 STEP 1728 H= 432:00 +CPU= 37.267
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0_r2085640283_17.zip
Uploading the intermediate file: upload_file_17.zip
  04:36:32 STEP 1729 H= 432:15 +CPU= 27.614
  04:37:01 STEP 1730 H= 432:30 +CPU= 28.495
  04:37:30 STEP 1731 H= 432:45 +CPU= 28.379
  04:37:58 STEP 1732 H= 433:00 +CPU= 28.244
  04:38:26 STEP 1733 H= 433:15 +CPU= 27.509
  04:38:54 STEP 1734 H= 433:30 +CPU= 27.623
  04:39:23 STEP 1735 H= 433:45 +CPU= 27.968
  04:39:53 STEP 1736 H= 434:00 +CPU= 29.231
  04:40:21 STEP 1737 H= 434:15 +CPU= 27.727
  04:40:49 STEP 1738 H= 434:30 +CPU= 27.558
  04:41:16 STEP 1739 H= 434:45 +CPU= 27.183
  04:41:50 STEP 1740 H= 435:00 +CPU= 33.677
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001740
  04:42:18 STEP 1741 H= 435:15 +CPU= 27.393
  04:42:46 STEP 1742 H= 435:30 +CPU= 27.483
  04:43:15 STEP 1743 H= 435:45 +CPU= 28.431
  04:43:46 STEP 1744 H= 436:00 +CPU= 29.707
  04:44:18 STEP 1745 H= 436:15 +CPU= 29.997
  04:44:49 STEP 1746 H= 436:30 +CPU= 29.305
  04:45:19 STEP 1747 H= 436:45 +CPU= 29.166
  04:45:50 STEP 1748 H= 437:00 +CPU= 29.531
  04:46:18 STEP 1749 H= 437:15 +CPU= 27.855
  04:46:49 STEP 1750 H= 437:30 +CPU= 29.106
  04:47:21 STEP 1751 H= 437:45 +CPU= 30.101
  04:48:00 STEP 1752 H= 438:00 +CPU= 37.454
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001752
  04:48:29 STEP 1753 H= 438:15 +CPU= 27.887
  04:48:56 STEP 1754 H= 438:30 +CPU= 27.458
  04:49:24 STEP 1755 H= 438:45 +CPU= 27.681
  04:49:54 STEP 1756 H= 439:00 +CPU= 29.251
  04:50:23 STEP 1757 H= 439:15 +CPU= 28.228
  04:50:52 STEP 1758 H= 439:30 +CPU= 27.757
  04:51:21 STEP 1759 H= 439:45 +CPU= 28.681
  04:51:50 STEP 1760 H= 440:00 +CPU= 28.333
  04:52:19 STEP 1761 H= 440:15 +CPU= 27.936
  04:52:47 STEP 1762 H= 440:30 +CPU= 28.032
  04:53:16 STEP 1763 H= 440:45 +CPU= 28.024
  04:53:55 STEP 1764 H= 441:00 +CPU= 37.117
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001764
  04:54:23 STEP 1765 H= 441:15 +CPU= 27.300
  04:54:57 STEP 1766 H= 441:30 +CPU= 30.619
  04:55:34 STEP 1767 H= 441:45 +CPU= 32.598
  04:56:09 STEP 1768 H= 442:00 +CPU= 32.027
  04:56:43 STEP 1769 H= 442:15 +CPU= 30.847
  04:57:17 STEP 1770 H= 442:30 +CPU= 31.536
  04:57:53 STEP 1771 H= 442:45 +CPU= 32.264
  04:58:27 STEP 1772 H= 443:00 +CPU= 32.071
  04:58:59 STEP 1773 H= 443:15 +CPU= 29.883
  04:59:27 STEP 1774 H= 443:30 +CPU= 27.890
  04:59:56 STEP 1775 H= 443:45 +CPU= 28.193
  05:00:38 STEP 1776 H= 444:00 +CPU= 40.064
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001776
  05:01:07 STEP 1777 H= 444:15 +CPU= 28.511
  05:01:36 STEP 1778 H= 444:30 +CPU= 28.066
  05:02:05 STEP 1779 H= 444:45 +CPU= 28.484
  05:02:34 STEP 1780 H= 445:00 +CPU= 28.695
  05:03:04 STEP 1781 H= 445:15 +CPU= 28.882
  05:03:32 STEP 1782 H= 445:30 +CPU= 28.203
  05:04:01 STEP 1783 H= 445:45 +CPU= 27.948
  05:04:31 STEP 1784 H= 446:00 +CPU= 29.484
  05:05:00 STEP 1785 H= 446:15 +CPU= 27.945
  05:05:28 STEP 1786 H= 446:30 +CPU= 27.571
  05:05:56 STEP 1787 H= 446:45 +CPU= 27.942
  05:06:32 STEP 1788 H= 447:00 +CPU= 35.095
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001788
  05:07:00 STEP 1789 H= 447:15 +CPU= 27.767
  05:07:28 STEP 1790 H= 447:30 +CPU= 27.950
  05:07:56 STEP 1791 H= 447:45 +CPU= 28.099
  05:08:25 STEP 1792 H= 448:00 +CPU= 28.402
  05:08:53 STEP 1793 H= 448:15 +CPU= 27.797
  05:09:22 STEP 1794 H= 448:30 +CPU= 28.242
  05:09:50 STEP 1795 H= 448:45 +CPU= 28.040
  05:10:20 STEP 1796 H= 449:00 +CPU= 28.900
  05:10:48 STEP 1797 H= 449:15 +CPU= 27.810
  05:11:15 STEP 1798 H= 449:30 +CPU= 26.914
  05:11:42 STEP 1799 H= 449:45 +CPU= 26.848
  05:12:16 STEP 1800 H= 450:00 +CPU= 33.946
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001800
  05:12:42 STEP 1801 H= 450:15 +CPU= 26.526
  05:13:09 STEP 1802 H= 450:30 +CPU= 26.794
  05:13:36 STEP 1803 H= 450:45 +CPU= 26.823
  05:14:03 STEP 1804 H= 451:00 +CPU= 27.304
  05:14:30 STEP 1805 H= 451:15 +CPU= 26.688
  05:14:57 STEP 1806 H= 451:30 +CPU= 26.760
  05:15:24 STEP 1807 H= 451:45 +CPU= 26.950
  05:15:51 STEP 1808 H= 452:00 +CPU= 27.304
  05:16:18 STEP 1809 H= 452:15 +CPU= 26.755
  05:16:45 STEP 1810 H= 452:30 +CPU= 26.757
  05:17:12 STEP 1811 H= 452:45 +CPU= 26.892
  05:17:46 STEP 1812 H= 453:00 +CPU= 33.802
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001812
  05:18:12 STEP 1813 H= 453:15 +CPU= 26.548
  05:18:39 STEP 1814 H= 453:30 +CPU= 26.823
  05:19:06 STEP 1815 H= 453:45 +CPU= 26.826
  05:19:34 STEP 1816 H= 454:00 +CPU= 27.339
  05:20:00 STEP 1817 H= 454:15 +CPU= 26.796
  05:20:27 STEP 1818 H= 454:30 +CPU= 26.974
  05:20:54 STEP 1819 H= 454:45 +CPU= 26.842
  05:21:22 STEP 1820 H= 455:00 +CPU= 27.366
  05:21:49 STEP 1821 H= 455:15 +CPU= 26.876
  05:22:15 STEP 1822 H= 455:30 +CPU= 26.768
  05:22:42 STEP 1823 H= 455:45 +CPU= 26.709
  05:23:20 STEP 1824 H= 456:00 +CPU= 37.095
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0_r2085640283_18.zip
Uploading the intermediate file: upload_file_18.zip
  05:23:47 STEP 1825 H= 456:15 +CPU= 26.965
  05:24:13 STEP 1826 H= 456:30 +CPU= 26.885
  05:24:41 STEP 1827 H= 456:45 +CPU= 27.189
  05:25:08 STEP 1828 H= 457:00 +CPU= 27.392
  05:25:35 STEP 1829 H= 457:15 +CPU= 26.784
  05:26:02 STEP 1830 H= 457:30 +CPU= 26.747
  05:26:29 STEP 1831 H= 457:45 +CPU= 26.804
  05:26:56 STEP 1832 H= 458:00 +CPU= 27.460
  05:27:23 STEP 1833 H= 458:15 +CPU= 26.868
  05:27:50 STEP 1834 H= 458:30 +CPU= 26.831
  05:28:17 STEP 1835 H= 458:45 +CPU= 26.853
  05:28:51 STEP 1836 H= 459:00 +CPU= 33.829
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001836
  05:29:17 STEP 1837 H= 459:15 +CPU= 26.708
  05:29:44 STEP 1838 H= 459:30 +CPU= 26.768
  05:30:11 STEP 1839 H= 459:45 +CPU= 26.721
  05:30:38 STEP 1840 H= 460:00 +CPU= 27.311
  05:31:05 STEP 1841 H= 460:15 +CPU= 26.841
  05:31:32 STEP 1842 H= 460:30 +CPU= 26.742
  05:31:59 STEP 1843 H= 460:45 +CPU= 26.802
  05:32:26 STEP 1844 H= 461:00 +CPU= 27.367
  05:32:53 STEP 1845 H= 461:15 +CPU= 26.705
  05:33:20 STEP 1846 H= 461:30 +CPU= 26.838
  05:33:47 STEP 1847 H= 461:45 +CPU= 26.702
  05:34:20 STEP 1848 H= 462:00 +CPU= 33.649
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001848
  05:34:47 STEP 1849 H= 462:15 +CPU= 26.582
  05:35:14 STEP 1850 H= 462:30 +CPU= 26.756
  05:35:41 STEP 1851 H= 462:45 +CPU= 26.888
  05:36:08 STEP 1852 H= 463:00 +CPU= 27.431
  05:36:35 STEP 1853 H= 463:15 +CPU= 26.851
  05:37:02 STEP 1854 H= 463:30 +CPU= 26.893
  05:37:29 STEP 1855 H= 463:45 +CPU= 26.856
  05:37:56 STEP 1856 H= 464:00 +CPU= 27.521
  05:38:24 STEP 1857 H= 464:15 +CPU= 27.074
  05:38:51 STEP 1858 H= 464:30 +CPU= 26.856
  05:39:17 STEP 1859 H= 464:45 +CPU= 26.901
  05:39:51 STEP 1860 H= 465:00 +CPU= 33.677
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001860
  05:40:18 STEP 1861 H= 465:15 +CPU= 26.646
  05:40:45 STEP 1862 H= 465:30 +CPU= 26.705
  05:41:11 STEP 1863 H= 465:45 +CPU= 26.613
  05:41:39 STEP 1864 H= 466:00 +CPU= 27.280
  05:42:05 STEP 1865 H= 466:15 +CPU= 26.650
  05:42:32 STEP 1866 H= 466:30 +CPU= 26.677
  05:42:59 STEP 1867 H= 466:45 +CPU= 26.637
  05:43:26 STEP 1868 H= 467:00 +CPU= 27.240
  05:43:53 STEP 1869 H= 467:15 +CPU= 26.681
  05:44:19 STEP 1870 H= 467:30 +CPU= 26.594
  05:44:46 STEP 1871 H= 467:45 +CPU= 26.731
  05:45:24 STEP 1872 H= 468:00 +CPU= 37.472
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001872
  05:45:51 STEP 1873 H= 468:15 +CPU= 26.854
  05:46:17 STEP 1874 H= 468:30 +CPU= 26.723
  05:46:44 STEP 1875 H= 468:45 +CPU= 26.768
  05:47:12 STEP 1876 H= 469:00 +CPU= 27.396
  05:47:39 STEP 1877 H= 469:15 +CPU= 26.919
  05:48:05 STEP 1878 H= 469:30 +CPU= 26.832
  05:48:32 STEP 1879 H= 469:45 +CPU= 26.785
  05:49:00 STEP 1880 H= 470:00 +CPU= 27.323
  05:49:27 STEP 1881 H= 470:15 +CPU= 26.825
  05:49:53 STEP 1882 H= 470:30 +CPU= 26.885
  05:50:20 STEP 1883 H= 470:45 +CPU= 26.856
  05:50:54 STEP 1884 H= 471:00 +CPU= 33.678
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001884
  05:51:21 STEP 1885 H= 471:15 +CPU= 26.768
  05:51:48 STEP 1886 H= 471:30 +CPU= 26.817
  05:52:15 STEP 1887 H= 471:45 +CPU= 26.801
  05:52:42 STEP 1888 H= 472:00 +CPU= 27.334
  05:53:09 STEP 1889 H= 472:15 +CPU= 26.731
  05:53:36 STEP 1890 H= 472:30 +CPU= 26.778
  05:54:03 STEP 1891 H= 472:45 +CPU= 26.932
  05:54:30 STEP 1892 H= 473:00 +CPU= 27.454
  05:54:57 STEP 1893 H= 473:15 +CPU= 26.866
  05:55:24 STEP 1894 H= 473:30 +CPU= 26.773
  05:55:51 STEP 1895 H= 473:45 +CPU= 26.888
  05:56:24 STEP 1896 H= 474:00 +CPU= 33.611
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001896
  05:56:51 STEP 1897 H= 474:15 +CPU= 26.852
  05:57:18 STEP 1898 H= 474:30 +CPU= 26.823
  05:57:45 STEP 1899 H= 474:45 +CPU= 26.806
  05:58:13 STEP 1900 H= 475:00 +CPU= 27.442
  05:58:40 STEP 1901 H= 475:15 +CPU= 26.891
  05:59:06 STEP 1902 H= 475:30 +CPU= 26.754
  05:59:33 STEP 1903 H= 475:45 +CPU= 26.809
  06:00:01 STEP 1904 H= 476:00 +CPU= 27.456
  06:00:27 STEP 1905 H= 476:15 +CPU= 26.737
  06:00:54 STEP 1906 H= 476:30 +CPU= 26.756
  06:01:21 STEP 1907 H= 476:45 +CPU= 26.803
  06:01:55 STEP 1908 H= 477:00 +CPU= 33.525
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001908
  06:02:22 STEP 1909 H= 477:15 +CPU= 26.950
  06:02:49 STEP 1910 H= 477:30 +CPU= 26.938
  06:03:16 STEP 1911 H= 477:45 +CPU= 26.893
  06:03:43 STEP 1912 H= 478:00 +CPU= 27.675
  06:04:10 STEP 1913 H= 478:15 +CPU= 26.968
  06:04:37 STEP 1914 H= 478:30 +CPU= 26.934
  06:05:04 STEP 1915 H= 478:45 +CPU= 26.928
  06:05:32 STEP 1916 H= 479:00 +CPU= 27.589
  06:05:59 STEP 1917 H= 479:15 +CPU= 26.781
  06:06:26 STEP 1918 H= 479:30 +CPU= 26.830
  06:06:53 STEP 1919 H= 479:45 +CPU= 27.032
  06:07:28 STEP 1920 H= 480:00 +CPU= 34.928
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1816738981E-21      I       0.3434979663E-06      I
ICloud fractionI      0.0000000000E+00      I       0.1321101000E-01      I
I  Relat. Hum. I      0.4251164698E+00      I       0.4537942202E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2601255656E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    298447492 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS596895.0 Mbytes/s, TOTAL TIME=    0.0096( 5.2%)
 06:07:28 STEP 1920 H= 480:00 +CPU= 34.928
 END CNT3
 NSTEP =  1920 CNT0   000000000  25.313  25.313  25.353   0.998               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        11440/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    203    0       46      1044068    11440 ************    25.3     3.8    21.5     4.6    43.6
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.95      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  417    4804.1    4804.1     17.89      0.00
   9 SPCM     - SPECTRAL COMP.                     384     970.2     970.2      3.33      0.00
  10 SCAN2M - PHYSICS                              385   14179.9   14179.9     48.76      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 32     906.2     906.2      0.26      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 17      46.3      46.3      0.01      0.00
  17 GRIDFPOS IN CNT4                               32      26.7      26.7      0.01      0.00
  27 RESTART FILES - WRITING                         8    3624.7    3624.7      0.26      0.00
  28 RESTART FILES - READING                         1    5516.1    5516.1      0.05      0.00
  29 SU4FPOS IN CNT4                                32       0.2       0.2      0.00      0.00
  30 DYNFPOS IN CNT4                                32    6413.4    6413.4      1.83      0.00
  31 POSDDH IN STEPO                                 2       6.4       6.4      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    385    5544.3    5544.3     19.06      0.00
  39 SU0YOMB                                         1     398.3     398.3      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    385      20.1      20.1      0.07      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              384     293.5     293.5      1.01      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              384      75.5      75.5      0.26      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 17       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      836     408.0     408.0      3.05      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       800     252.1     252.1      1.80      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        800      83.0      83.0      0.59      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       836     131.0     131.0      0.98      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      42.8      42.8      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            836      60.6      60.6      0.45      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            800      33.4      33.4      0.24      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            836      68.3      68.3      0.51      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            800      28.6      28.6      0.20      0.00
 400 GSTATS                                      45508       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 42288       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    11196.9 CPU TIME   10989.6 VECTOR TIME    10989.6


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      154.4
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/5/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/5/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292383/ICMGGh7zg+001848
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0_r2085640283_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
06:10:56 (2425): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 11:09:44 1549270 22445989 oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0 1,727,100 54,894 0.0318
14 Jun 2024 21:41:06 1549270 22445989 oifs_43r3_bl_a34o_2016092300_20_1018_12292383_0 864,000 27,461 0.0318


©2024 cpdn.org