climateprediction.net home page
Task 22282138

Task 22282138

Name oifs_43r3_ps_0535_2010050100_123_979_12196179_0
Workunit 12196179
Created 23 Dec 2022, 23:55:19 UTC
Sent 28 Dec 2022, 8:44:13 UTC
Report deadline 27 Jan 2023, 8:44:13 UTC
Received 14 Jan 2023, 16:46:54 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1529705
Run time 11 hours 37 min 17 sec
CPU time 11 hours 14 min 25 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 9.20 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,408.88 MB
Peak swap size 4,978.07 MB
Peak disk usage 4,579.63 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
TEP 2451 H=2451:00 +CPU= 18.039
  16:36:52 STEP 2452 H=2452:00 +CPU= 11.740
  16:37:04 STEP 2453 H=2453:00 +CPU= 11.307
  16:37:22 STEP 2454 H=2454:00 +CPU= 18.206
  16:37:34 STEP 2455 H=2455:00 +CPU= 11.612
  16:37:46 STEP 2456 H=2456:00 +CPU= 11.764
  16:38:04 STEP 2457 H=2457:00 +CPU= 17.984
  16:38:16 STEP 2458 H=2458:00 +CPU= 11.742
  16:38:28 STEP 2459 H=2459:00 +CPU= 11.510
  16:38:48 STEP 2460 H=2460:00 +CPU= 19.766
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002460
  16:38:59 STEP 2461 H=2461:00 +CPU= 11.333
  16:39:11 STEP 2462 H=2462:00 +CPU= 11.658
  16:39:29 STEP 2463 H=2463:00 +CPU= 17.591
  16:39:41 STEP 2464 H=2464:00 +CPU= 11.659
  16:39:52 STEP 2465 H=2465:00 +CPU= 11.435
  16:40:11 STEP 2466 H=2466:00 +CPU= 18.403
  16:40:23 STEP 2467 H=2467:00 +CPU= 11.725
  16:40:35 STEP 2468 H=2468:00 +CPU= 11.806
  16:40:53 STEP 2469 H=2469:00 +CPU= 18.054
  16:41:05 STEP 2470 H=2470:00 +CPU= 11.789
  16:41:18 STEP 2471 H=2471:00 +CPU= 11.912
  16:41:40 STEP 2472 H=2472:00 +CPU= 21.706
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_102.zip
Uploading the intermediate file: upload_file_102.zip
  16:41:53 STEP 2473 H=2473:00 +CPU= 12.140
  16:42:05 STEP 2474 H=2474:00 +CPU= 12.297
  16:42:24 STEP 2475 H=2475:00 +CPU= 18.112
  16:42:36 STEP 2476 H=2476:00 +CPU= 11.951
  16:42:48 STEP 2477 H=2477:00 +CPU= 11.853
  16:43:07 STEP 2478 H=2478:00 +CPU= 19.014
  16:43:20 STEP 2479 H=2479:00 +CPU= 11.876
  16:43:31 STEP 2480 H=2480:00 +CPU= 11.821
  16:43:50 STEP 2481 H=2481:00 +CPU= 17.756
  16:44:02 STEP 2482 H=2482:00 +CPU= 12.159
  16:44:14 STEP 2483 H=2483:00 +CPU= 11.766
  16:44:34 STEP 2484 H=2484:00 +CPU= 19.394
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002484
  16:44:45 STEP 2485 H=2485:00 +CPU= 11.352
  16:44:57 STEP 2486 H=2486:00 +CPU= 11.664
  16:45:15 STEP 2487 H=2487:00 +CPU= 18.031
  16:45:28 STEP 2488 H=2488:00 +CPU= 12.119
  16:45:40 STEP 2489 H=2489:00 +CPU= 11.960
  16:45:59 STEP 2490 H=2490:00 +CPU= 18.717
  16:46:11 STEP 2491 H=2491:00 +CPU= 11.682
  16:46:23 STEP 2492 H=2492:00 +CPU= 11.927
  16:46:42 STEP 2493 H=2493:00 +CPU= 18.455
  16:46:54 STEP 2494 H=2494:00 +CPU= 11.931
  16:47:07 STEP 2495 H=2495:00 +CPU= 12.149
  16:47:30 STEP 2496 H=2496:00 +CPU= 22.047
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_103.zip
Uploading the intermediate file: upload_file_103.zip
  16:47:42 STEP 2497 H=2497:00 +CPU= 11.826
  16:47:54 STEP 2498 H=2498:00 +CPU= 12.094
  16:48:12 STEP 2499 H=2499:00 +CPU= 17.966
  16:48:25 STEP 2500 H=2500:00 +CPU= 11.970
  16:48:37 STEP 2501 H=2501:00 +CPU= 11.692
  16:48:55 STEP 2502 H=2502:00 +CPU= 18.086
  16:49:07 STEP 2503 H=2503:00 +CPU= 11.568
  16:49:18 STEP 2504 H=2504:00 +CPU= 11.643
  16:49:37 STEP 2505 H=2505:00 +CPU= 17.905
  16:49:49 STEP 2506 H=2506:00 +CPU= 12.451
  16:50:02 STEP 2507 H=2507:00 +CPU= 12.357
  16:50:22 STEP 2508 H=2508:00 +CPU= 19.329
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002508
  16:50:33 STEP 2509 H=2509:00 +CPU= 11.648
  16:50:45 STEP 2510 H=2510:00 +CPU= 11.878
  16:51:03 STEP 2511 H=2511:00 +CPU= 17.794
  16:51:15 STEP 2512 H=2512:00 +CPU= 11.796
  16:51:27 STEP 2513 H=2513:00 +CPU= 11.420
  16:51:45 STEP 2514 H=2514:00 +CPU= 17.994
  16:51:57 STEP 2515 H=2515:00 +CPU= 11.450
  16:52:09 STEP 2516 H=2516:00 +CPU= 11.608
  16:52:26 STEP 2517 H=2517:00 +CPU= 17.704
  16:52:38 STEP 2518 H=2518:00 +CPU= 11.441
  16:52:50 STEP 2519 H=2519:00 +CPU= 11.698
  16:53:12 STEP 2520 H=2520:00 +CPU= 21.366
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_104.zip
Uploading the intermediate file: upload_file_104.zip
  16:53:24 STEP 2521 H=2521:00 +CPU= 11.564
  16:53:35 STEP 2522 H=2522:00 +CPU= 11.573
  16:53:53 STEP 2523 H=2523:00 +CPU= 17.668
  16:54:05 STEP 2524 H=2524:00 +CPU= 11.666
  16:54:17 STEP 2525 H=2525:00 +CPU= 11.511
  16:54:35 STEP 2526 H=2526:00 +CPU= 17.812
  16:54:46 STEP 2527 H=2527:00 +CPU= 11.391
  16:54:58 STEP 2528 H=2528:00 +CPU= 11.641
  16:55:16 STEP 2529 H=2529:00 +CPU= 17.684
  16:55:28 STEP 2530 H=2530:00 +CPU= 11.825
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 2550MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 2598 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        WVCOUPLE 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         WAVEMDL 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          WAMODEL 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           >OMP-WAMODEL 2            (1431) 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            IMPLSCH 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             SINPUT 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165539:1672934139:3244.133] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              SINPUT_JAN 
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165540:1672934140:3245.134] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Lynx.local:1:1:2437:2437] [20230105:165540:1672934140:3245.134] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC9F89  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001A81DC2  sinput_jan_               252  sinput_jan.F
oifs_43r3_model.e  0000000001A7B775  sinput_                    68  sinput.F90
oifs_43r3_model.e  0000000001A3329E  implsch_                  441  implsch.F
oifs_43r3_model.e  000000000168CBD7  wamodel_                 1008  wamodel.F
oifs_43r3_model.e  0000000000872295  wavemdl_                  584  wavemdl.F
oifs_43r3_model.e  00000000007E0165  wvcouple_                 260  wvcouple.F90
oifs_43r3_model.e  000000000055E4AD  cnt4_                    1283  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2010050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0535
(argv4) batchid: 979
(argv5) wuid: 12196179
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/4
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_da9ca43e8477e02cece7fc11838be978 to: /var/lib/boinc/slots/4/oifs_43r3_ps_0535_2010050100_123_979_12196179.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/4/oifs_43r3_ps_0535_2010050100_123_979_12196179.zip
ic_ancil_file: ic_ancil_12196179
ifsdata_file: ifsdata_12196179
climate_data_file: clim_data_12196179
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_fd58a85545fb4b47cac539fdef98dbb1 to: /var/lib/boinc/slots/4/ic_ancil_12196179.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/4/ic_ancil_12196179.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_231627462e4952ac498eb54b289c0ff5 to: /var/lib/boinc/slots/4/ifsdata/ifsdata_12196179.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/4/ifsdata/ifsdata_12196179.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/4/159l_2/clim_data_12196179.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/4/159l_2/clim_data_12196179.zip
Checking for progress XML file: /var/lib/boinc/slots/4/progress_file_12196179.xml
Opened progress file ok : /var/lib/boinc/slots/4/progress_file_12196179.xml
-- Model is restarting --
Adjusting last_iter, 2531, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/4/progress_file_12196179.xml
last_cpu_time: 33937
upload_file_number: 105
last_iter: 2519
last_upload: 9072000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108
The child process has been launched with process id: 2570
Executing the command: /var/lib/boinc/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lynx.local:1:1:2570:2570] [20230105:183235:1672939955:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  18:33:01 STEP 2520 H=2520:00 +CPU= 21.217
  18:33:13 STEP 2521 H=2521:00 +CPU= 11.807
  18:33:25 STEP 2522 H=2522:00 +CPU= 11.851
  18:33:42 STEP 2523 H=2523:00 +CPU= 17.560
  18:33:54 STEP 2524 H=2524:00 +CPU= 11.168
  18:34:05 STEP 2525 H=2525:00 +CPU= 11.039
  18:34:23 STEP 2526 H=2526:00 +CPU= 17.531
  18:34:34 STEP 2527 H=2527:00 +CPU= 11.474
  18:34:49 STEP 2528 H=2528:00 +CPU= 14.328
  18:35:09 STEP 2529 H=2529:00 +CPU= 19.298
  18:35:23 STEP 2530 H=2530:00 +CPU= 12.818
  18:35:35 STEP 2531 H=2531:00 +CPU= 12.539
  18:35:57 STEP 2532 H=2532:00 +CPU= 21.002
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002532
  18:36:10 STEP 2533 H=2533:00 +CPU= 12.352
  18:36:23 STEP 2534 H=2534:00 +CPU= 12.539
  18:36:42 STEP 2535 H=2535:00 +CPU= 18.926
  18:36:55 STEP 2536 H=2536:00 +CPU= 12.189
  18:37:08 STEP 2537 H=2537:00 +CPU= 12.720
  18:37:28 STEP 2538 H=2538:00 +CPU= 19.238
  18:37:41 STEP 2539 H=2539:00 +CPU= 12.380
  18:37:54 STEP 2540 H=2540:00 +CPU= 12.822
  18:38:14 STEP 2541 H=2541:00 +CPU= 19.084
  18:38:27 STEP 2542 H=2542:00 +CPU= 12.560
  18:38:40 STEP 2543 H=2543:00 +CPU= 12.667
  18:39:04 STEP 2544 H=2544:00 +CPU= 22.822
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_105.zip
Uploading the intermediate file: upload_file_105.zip
  18:39:17 STEP 2545 H=2545:00 +CPU= 12.476
  18:39:30 STEP 2546 H=2546:00 +CPU= 12.756
  18:39:49 STEP 2547 H=2547:00 +CPU= 18.888
  18:40:02 STEP 2548 H=2548:00 +CPU= 12.500
  18:40:15 STEP 2549 H=2549:00 +CPU= 12.367
  18:40:34 STEP 2550 H=2550:00 +CPU= 19.097
  18:40:47 STEP 2551 H=2551:00 +CPU= 12.043
  18:41:00 STEP 2552 H=2552:00 +CPU= 12.633
  18:41:20 STEP 2553 H=2553:00 +CPU= 19.614
  18:41:33 STEP 2554 H=2554:00 +CPU= 12.852
  18:41:46 STEP 2555 H=2555:00 +CPU= 12.243
  18:42:09 STEP 2556 H=2556:00 +CPU= 22.099
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002556
  18:42:23 STEP 2557 H=2557:00 +CPU= 13.054
  18:42:37 STEP 2558 H=2558:00 +CPU= 13.399
  18:42:57 STEP 2559 H=2559:00 +CPU= 19.927
  18:43:11 STEP 2560 H=2560:00 +CPU= 13.332
  18:43:25 STEP 2561 H=2561:00 +CPU= 13.131
  18:43:46 STEP 2562 H=2562:00 +CPU= 20.152
  18:43:59 STEP 2563 H=2563:00 +CPU= 12.734
  18:44:13 STEP 2564 H=2564:00 +CPU= 13.159
  18:44:33 STEP 2565 H=2565:00 +CPU= 19.980
  18:44:47 STEP 2566 H=2566:00 +CPU= 13.200
  18:45:01 STEP 2567 H=2567:00 +CPU= 13.420
  18:45:27 STEP 2568 H=2568:00 +CPU= 23.920
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_106.zip
Uploading the intermediate file: upload_file_106.zip
  18:45:40 STEP 2569 H=2569:00 +CPU= 13.179
  18:45:54 STEP 2570 H=2570:00 +CPU= 13.492
  18:46:14 STEP 2571 H=2571:00 +CPU= 19.341
  18:46:27 STEP 2572 H=2572:00 +CPU= 12.323
  18:46:40 STEP 2573 H=2573:00 +CPU= 12.471
  18:46:59 STEP 2574 H=2574:00 +CPU= 18.608
  18:47:12 STEP 2575 H=2575:00 +CPU= 12.824
  18:47:25 STEP 2576 H=2576:00 +CPU= 12.658
  18:47:45 STEP 2577 H=2577:00 +CPU= 19.440
  18:47:59 STEP 2578 H=2578:00 +CPU= 12.798
  18:48:12 STEP 2579 H=2579:00 +CPU= 12.424
  18:48:33 STEP 2580 H=2580:00 +CPU= 20.639
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002580
  18:48:46 STEP 2581 H=2581:00 +CPU= 12.521
  18:48:59 STEP 2582 H=2582:00 +CPU= 12.782
  18:49:19 STEP 2583 H=2583:00 +CPU= 19.319
  18:49:33 STEP 2584 H=2584:00 +CPU= 13.240
  18:49:46 STEP 2585 H=2585:00 +CPU= 12.918
  18:50:07 STEP 2586 H=2586:00 +CPU= 19.887
  18:50:20 STEP 2587 H=2587:00 +CPU= 12.813
  18:50:34 STEP 2588 H=2588:00 +CPU= 13.050
  18:50:54 STEP 2589 H=2589:00 +CPU= 19.495
  18:51:07 STEP 2590 H=2590:00 +CPU= 13.064
  18:51:21 STEP 2591 H=2591:00 +CPU= 13.294
  18:51:46 STEP 2592 H=2592:00 +CPU= 23.484
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_107.zip
Uploading the intermediate file: upload_file_107.zip
  18:52:00 STEP 2593 H=2593:00 +CPU= 12.915
  18:52:14 STEP 2594 H=2594:00 +CPU= 13.233
  18:52:34 STEP 2595 H=2595:00 +CPU= 19.462
  18:52:47 STEP 2596 H=2596:00 +CPU= 12.867
  18:53:00 STEP 2597 H=2597:00 +CPU= 12.855
  18:53:20 STEP 2598 H=2598:00 +CPU= 19.007
  18:53:32 STEP 2599 H=2599:00 +CPU= 11.802
  18:53:44 STEP 2600 H=2600:00 +CPU= 12.011
  18:54:03 STEP 2601 H=2601:00 +CPU= 18.768
  18:54:17 STEP 2602 H=2602:00 +CPU= 13.151
  18:54:30 STEP 2603 H=2603:00 +CPU= 12.621
  18:54:52 STEP 2604 H=2604:00 +CPU= 21.093
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002604
  18:55:05 STEP 2605 H=2605:00 +CPU= 12.738
  18:55:19 STEP 2606 H=2606:00 +CPU= 12.942
  18:55:39 STEP 2607 H=2607:00 +CPU= 19.465
  18:55:52 STEP 2608 H=2608:00 +CPU= 13.041
  18:56:06 STEP 2609 H=2609:00 +CPU= 12.680
  18:56:26 STEP 2610 H=2610:00 +CPU= 19.664
  18:56:39 STEP 2611 H=2611:00 +CPU= 12.686
  18:56:53 STEP 2612 H=2612:00 +CPU= 12.913
  18:57:13 STEP 2613 H=2613:00 +CPU= 19.616
  18:57:26 STEP 2614 H=2614:00 +CPU= 12.969
  18:57:40 STEP 2615 H=2615:00 +CPU= 13.133
  18:58:05 STEP 2616 H=2616:00 +CPU= 23.703
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_108.zip
Uploading the intermediate file: upload_file_108.zip
  18:58:19 STEP 2617 H=2617:00 +CPU= 12.958
  18:58:33 STEP 2618 H=2618:00 +CPU= 13.039
  18:58:53 STEP 2619 H=2619:00 +CPU= 19.731
  18:59:07 STEP 2620 H=2620:00 +CPU= 13.254
  18:59:20 STEP 2621 H=2621:00 +CPU= 12.808
  18:59:41 STEP 2622 H=2622:00 +CPU= 19.875
  18:59:54 STEP 2623 H=2623:00 +CPU= 12.862
  19:00:08 STEP 2624 H=2624:00 +CPU= 12.947
  19:00:27 STEP 2625 H=2625:00 +CPU= 18.718
  19:00:39 STEP 2626 H=2626:00 +CPU= 12.033
  19:00:51 STEP 2627 H=2627:00 +CPU= 11.771
  19:01:11 STEP 2628 H=2628:00 +CPU= 19.836
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002628
  19:01:24 STEP 2629 H=2629:00 +CPU= 11.850
  19:01:36 STEP 2630 H=2630:00 +CPU= 12.058
  19:01:55 STEP 2631 H=2631:00 +CPU= 18.287
  19:02:07 STEP 2632 H=2632:00 +CPU= 12.025
  19:02:19 STEP 2633 H=2633:00 +CPU= 11.725
  19:02:37 STEP 2634 H=2634:00 +CPU= 18.425
  19:02:50 STEP 2635 H=2635:00 +CPU= 11.913
  19:03:02 STEP 2636 H=2636:00 +CPU= 12.027
  19:03:21 STEP 2637 H=2637:00 +CPU= 18.935
  19:03:35 STEP 2638 H=2638:00 +CPU= 13.148
  19:03:48 STEP 2639 H=2639:00 +CPU= 12.567
  19:04:11 STEP 2640 H=2640:00 +CPU= 22.152
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_109.zip
Uploading the intermediate file: upload_file_109.zip
  19:04:23 STEP 2641 H=2641:00 +CPU= 11.810
  19:04:36 STEP 2642 H=2642:00 +CPU= 12.154
  19:04:54 STEP 2643 H=2643:00 +CPU= 18.414
  19:05:07 STEP 2644 H=2644:00 +CPU= 11.943
  19:05:19 STEP 2645 H=2645:00 +CPU= 11.802
  19:05:38 STEP 2646 H=2646:00 +CPU= 18.408
  19:05:50 STEP 2647 H=2647:00 +CPU= 11.872
  19:06:02 STEP 2648 H=2648:00 +CPU= 12.109
  19:06:21 STEP 2649 H=2649:00 +CPU= 18.242
  19:06:33 STEP 2650 H=2650:00 +CPU= 12.055
  19:06:45 STEP 2651 H=2651:00 +CPU= 11.759
  19:07:06 STEP 2652 H=2652:00 +CPU= 20.170
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002652
  19:07:18 STEP 2653 H=2653:00 +CPU= 11.773
  19:07:30 STEP 2654 H=2654:00 +CPU= 12.038
  19:07:49 STEP 2655 H=2655:00 +CPU= 18.284
  19:08:01 STEP 2656 H=2656:00 +CPU= 12.081
  19:08:13 STEP 2657 H=2657:00 +CPU= 11.810
  19:08:32 STEP 2658 H=2658:00 +CPU= 18.451
  19:08:44 STEP 2659 H=2659:00 +CPU= 11.853
  19:08:56 STEP 2660 H=2660:00 +CPU= 12.005
  19:09:14 STEP 2661 H=2661:00 +CPU= 18.040
  19:09:27 STEP 2662 H=2662:00 +CPU= 12.090
  19:09:39 STEP 2663 H=2663:00 +CPU= 12.182
  19:10:03 STEP 2664 H=2664:00 +CPU= 22.328
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_110.zip
Uploading the intermediate file: upload_file_110.zip
  19:10:15 STEP 2665 H=2665:00 +CPU= 11.813
  19:10:27 STEP 2666 H=2666:00 +CPU= 12.013
  19:10:46 STEP 2667 H=2667:00 +CPU= 18.114
  19:10:58 STEP 2668 H=2668:00 +CPU= 12.057
  19:11:10 STEP 2669 H=2669:00 +CPU= 11.746
  19:11:29 STEP 2670 H=2670:00 +CPU= 18.390
  19:11:41 STEP 2671 H=2671:00 +CPU= 11.807
  19:11:53 STEP 2672 H=2672:00 +CPU= 11.978
  19:12:14 STEP 2673 H=2673:00 +CPU= 19.860
  19:12:27 STEP 2674 H=2674:00 +CPU= 12.991
  19:12:40 STEP 2675 H=2675:00 +CPU= 12.724
  19:13:03 STEP 2676 H=2676:00 +CPU= 21.506
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002676
  19:13:16 STEP 2677 H=2677:00 +CPU= 12.847
  19:13:29 STEP 2678 H=2678:00 +CPU= 13.120
  19:13:50 STEP 2679 H=2679:00 +CPU= 19.664
  19:14:04 STEP 2680 H=2680:00 +CPU= 13.242
  19:14:17 STEP 2681 H=2681:00 +CPU= 13.033
  19:14:38 STEP 2682 H=2682:00 +CPU= 19.848
  19:14:51 STEP 2683 H=2683:00 +CPU= 12.856
  19:15:05 STEP 2684 H=2684:00 +CPU= 13.194
  19:15:25 STEP 2685 H=2685:00 +CPU= 19.551
  19:15:39 STEP 2686 H=2686:00 +CPU= 13.314
  19:15:52 STEP 2687 H=2687:00 +CPU= 13.076
  19:16:18 STEP 2688 H=2688:00 +CPU= 24.019
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_111.zip
Uploading the intermediate file: upload_file_111.zip
  19:16:31 STEP 2689 H=2689:00 +CPU= 12.924
  19:16:45 STEP 2690 H=2690:00 +CPU= 12.879
  19:17:05 STEP 2691 H=2691:00 +CPU= 19.809
  19:17:19 STEP 2692 H=2692:00 +CPU= 13.149
  19:17:32 STEP 2693 H=2693:00 +CPU= 12.731
  19:17:53 STEP 2694 H=2694:00 +CPU= 19.868
  19:18:06 STEP 2695 H=2695:00 +CPU= 12.828
  19:18:20 STEP 2696 H=2696:00 +CPU= 13.022
  19:18:41 STEP 2697 H=2697:00 +CPU= 20.068
  19:18:55 STEP 2698 H=2698:00 +CPU= 13.290
  19:19:08 STEP 2699 H=2699:00 +CPU= 12.772
  19:19:31 STEP 2700 H=2700:00 +CPU= 22.419
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002700
  19:19:46 STEP 2701 H=2701:00 +CPU= 13.734
  19:20:00 STEP 2702 H=2702:00 +CPU= 14.049
  19:20:22 STEP 2703 H=2703:00 +CPU= 20.980
  19:20:37 STEP 2704 H=2704:00 +CPU= 14.114
  19:20:52 STEP 2705 H=2705:00 +CPU= 13.847
  19:21:14 STEP 2706 H=2706:00 +CPU= 21.012
  19:21:28 STEP 2707 H=2707:00 +CPU= 13.714
  19:21:43 STEP 2708 H=2708:00 +CPU= 13.981
  19:22:05 STEP 2709 H=2709:00 +CPU= 21.215
  19:22:20 STEP 2710 H=2710:00 +CPU= 14.072
  19:22:35 STEP 2711 H=2711:00 +CPU= 14.017
  19:23:02 STEP 2712 H=2712:00 +CPU= 25.591
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_112.zip
Uploading the intermediate file: upload_file_112.zip
  19:23:17 STEP 2713 H=2713:00 +CPU= 13.998
  19:23:31 STEP 2714 H=2714:00 +CPU= 13.844
  19:23:53 STEP 2715 H=2715:00 +CPU= 20.968
  19:24:08 STEP 2716 H=2716:00 +CPU= 13.998
  19:24:22 STEP 2717 H=2717:00 +CPU= 13.944
  19:24:45 STEP 2718 H=2718:00 +CPU= 21.537
  19:24:59 STEP 2719 H=2719:00 +CPU= 13.812
  19:25:14 STEP 2720 H=2720:00 +CPU= 14.029
  19:25:36 STEP 2721 H=2721:00 +CPU= 21.088
  19:25:51 STEP 2722 H=2722:00 +CPU= 14.178
  19:26:06 STEP 2723 H=2723:00 +CPU= 14.093
  19:26:29 STEP 2724 H=2724:00 +CPU= 22.393
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002724
  19:26:43 STEP 2725 H=2725:00 +CPU= 13.168
  19:26:58 STEP 2726 H=2726:00 +CPU= 13.687
  19:27:20 STEP 2727 H=2727:00 +CPU= 20.555
  19:27:34 STEP 2728 H=2728:00 +CPU= 13.482
  19:27:48 STEP 2729 H=2729:00 +CPU= 13.211
  19:28:10 STEP 2730 H=2730:00 +CPU= 20.530
  19:28:24 STEP 2731 H=2731:00 +CPU= 13.388
  19:28:38 STEP 2732 H=2732:00 +CPU= 13.568
  19:29:00 STEP 2733 H=2733:00 +CPU= 20.364
  19:29:14 STEP 2734 H=2734:00 +CPU= 13.506
  19:29:28 STEP 2735 H=2735:00 +CPU= 13.598
  19:29:55 STEP 2736 H=2736:00 +CPU= 24.722
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_113.zip
Uploading the intermediate file: upload_file_113.zip
  19:30:09 STEP 2737 H=2737:00 +CPU= 12.957
  19:30:23 STEP 2738 H=2738:00 +CPU= 13.391
  19:30:43 STEP 2739 H=2739:00 +CPU= 19.248
  19:30:56 STEP 2740 H=2740:00 +CPU= 12.491
  19:31:10 STEP 2741 H=2741:00 +CPU= 13.271
  19:31:32 STEP 2742 H=2742:00 +CPU= 20.267
  19:31:46 STEP 2743 H=2743:00 +CPU= 13.257
  19:32:00 STEP 2744 H=2744:00 +CPU= 13.794
  19:32:21 STEP 2745 H=2745:00 +CPU= 19.783
  19:32:34 STEP 2746 H=2746:00 +CPU= 12.759
  19:32:47 STEP 2747 H=2747:00 +CPU= 12.471
  19:33:09 STEP 2748 H=2748:00 +CPU= 21.294
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002748
  19:33:22 STEP 2749 H=2749:00 +CPU= 12.252
  19:33:35 STEP 2750 H=2750:00 +CPU= 12.441
  19:33:53 STEP 2751 H=2751:00 +CPU= 18.373
  19:34:06 STEP 2752 H=2752:00 +CPU= 12.105
  19:34:18 STEP 2753 H=2753:00 +CPU= 11.828
  19:34:38 STEP 2754 H=2754:00 +CPU= 18.970
  19:34:50 STEP 2755 H=2755:00 +CPU= 12.376
  19:35:04 STEP 2756 H=2756:00 +CPU= 12.834
  19:35:23 STEP 2757 H=2757:00 +CPU= 18.430
  19:35:36 STEP 2758 H=2758:00 +CPU= 13.056
  19:35:50 STEP 2759 H=2759:00 +CPU= 13.319
  19:36:15 STEP 2760 H=2760:00 +CPU= 23.451
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_114.zip
Uploading the intermediate file: upload_file_114.zip
  19:36:28 STEP 2761 H=2761:00 +CPU= 12.431
  19:36:41 STEP 2762 H=2762:00 +CPU= 12.812
Uploading trickle at timestep: 9936000
  19:37:02 STEP 2763 H=2763:00 +CPU= 19.748
  19:37:16 STEP 2764 H=2764:00 +CPU= 12.879
  19:37:29 STEP 2765 H=2765:00 +CPU= 12.790
  19:37:50 STEP 2766 H=2766:00 +CPU= 20.046
  19:38:03 STEP 2767 H=2767:00 +CPU= 12.645
  19:38:16 STEP 2768 H=2768:00 +CPU= 12.655
  19:38:36 STEP 2769 H=2769:00 +CPU= 19.382
  19:38:49 STEP 2770 H=2770:00 +CPU= 12.623
  19:39:02 STEP 2771 H=2771:00 +CPU= 12.128
  19:39:23 STEP 2772 H=2772:00 +CPU= 20.429
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002772
  19:39:35 STEP 2773 H=2773:00 +CPU= 12.345
  19:39:48 STEP 2774 H=2774:00 +CPU= 12.482
  19:40:08 STEP 2775 H=2775:00 +CPU= 18.732
  19:40:20 STEP 2776 H=2776:00 +CPU= 12.269
  19:40:32 STEP 2777 H=2777:00 +CPU= 11.890
  19:40:52 STEP 2778 H=2778:00 +CPU= 19.083
  19:41:05 STEP 2779 H=2779:00 +CPU= 12.173
  19:41:17 STEP 2780 H=2780:00 +CPU= 12.284
  19:41:36 STEP 2781 H=2781:00 +CPU= 18.627
  19:41:48 STEP 2782 H=2782:00 +CPU= 12.002
  19:42:01 STEP 2783 H=2783:00 +CPU= 12.318
  19:42:25 STEP 2784 H=2784:00 +CPU= 22.674
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_115.zip
Uploading the intermediate file: upload_file_115.zip
  19:42:37 STEP 2785 H=2785:00 +CPU= 12.033
  19:42:50 STEP 2786 H=2786:00 +CPU= 12.130
  19:43:09 STEP 2787 H=2787:00 +CPU= 18.448
  19:43:21 STEP 2788 H=2788:00 +CPU= 12.065
  19:43:33 STEP 2789 H=2789:00 +CPU= 11.941
  19:43:53 STEP 2790 H=2790:00 +CPU= 19.206
  19:44:06 STEP 2791 H=2791:00 +CPU= 12.580
  19:44:19 STEP 2792 H=2792:00 +CPU= 12.852
  19:44:39 STEP 2793 H=2793:00 +CPU= 19.135
  19:44:52 STEP 2794 H=2794:00 +CPU= 12.523
  19:45:05 STEP 2795 H=2795:00 +CPU= 12.591
  19:45:26 STEP 2796 H=2796:00 +CPU= 20.750
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002796
  19:45:39 STEP 2797 H=2797:00 +CPU= 12.346
  19:45:52 STEP 2798 H=2798:00 +CPU= 12.654
  19:46:12 STEP 2799 H=2799:00 +CPU= 19.088
  19:46:25 STEP 2800 H=2800:00 +CPU= 12.555
  19:46:37 STEP 2801 H=2801:00 +CPU= 12.337
  19:46:57 STEP 2802 H=2802:00 +CPU= 19.180
  19:47:10 STEP 2803 H=2803:00 +CPU= 12.236
  19:47:23 STEP 2804 H=2804:00 +CPU= 12.699
  19:47:42 STEP 2805 H=2805:00 +CPU= 19.009
  19:47:55 STEP 2806 H=2806:00 +CPU= 12.685
  19:48:09 STEP 2807 H=2807:00 +CPU= 12.888
  19:48:33 STEP 2808 H=2808:00 +CPU= 23.430
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_116.zip
Uploading the intermediate file: upload_file_116.zip
  19:48:47 STEP 2809 H=2809:00 +CPU= 12.704
  19:49:00 STEP 2810 H=2810:00 +CPU= 12.813
  19:49:20 STEP 2811 H=2811:00 +CPU= 19.242
  19:49:33 STEP 2812 H=2812:00 +CPU= 12.616
  19:49:45 STEP 2813 H=2813:00 +CPU= 12.354
  19:50:05 STEP 2814 H=2814:00 +CPU= 19.383
  19:50:18 STEP 2815 H=2815:00 +CPU= 12.376
  19:50:31 STEP 2816 H=2816:00 +CPU= 12.440
  19:50:50 STEP 2817 H=2817:00 +CPU= 18.837
  19:51:03 STEP 2818 H=2818:00 +CPU= 12.605
  19:51:16 STEP 2819 H=2819:00 +CPU= 12.420
  19:51:38 STEP 2820 H=2820:00 +CPU= 20.902
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002820
  19:51:50 STEP 2821 H=2821:00 +CPU= 12.392
  19:52:03 STEP 2822 H=2822:00 +CPU= 12.305
  19:52:22 STEP 2823 H=2823:00 +CPU= 18.284
  19:52:34 STEP 2824 H=2824:00 +CPU= 12.287
  19:52:46 STEP 2825 H=2825:00 +CPU= 12.028
  19:53:05 STEP 2826 H=2826:00 +CPU= 18.607
  19:53:18 STEP 2827 H=2827:00 +CPU= 11.845
  19:53:31 STEP 2828 H=2828:00 +CPU= 12.807
  19:53:50 STEP 2829 H=2829:00 +CPU= 19.135
  19:54:03 STEP 2830 H=2830:00 +CPU= 12.545
  19:54:17 STEP 2831 H=2831:00 +CPU= 12.839
  19:54:41 STEP 2832 H=2832:00 +CPU= 23.020
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_117.zip
Uploading the intermediate file: upload_file_117.zip
  19:54:54 STEP 2833 H=2833:00 +CPU= 12.300
  19:55:07 STEP 2834 H=2834:00 +CPU= 12.609
  19:55:26 STEP 2835 H=2835:00 +CPU= 18.968
  19:55:39 STEP 2836 H=2836:00 +CPU= 12.636
  19:55:52 STEP 2837 H=2837:00 +CPU= 12.599
  19:56:13 STEP 2838 H=2838:00 +CPU= 20.183
  19:56:27 STEP 2839 H=2839:00 +CPU= 13.127
  19:56:40 STEP 2840 H=2840:00 +CPU= 12.895
  19:57:01 STEP 2841 H=2841:00 +CPU= 19.584
  19:57:15 STEP 2842 H=2842:00 +CPU= 13.291
  19:57:29 STEP 2843 H=2843:00 +CPU= 13.584
  19:57:52 STEP 2844 H=2844:00 +CPU= 22.465
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002844
  19:58:06 STEP 2845 H=2845:00 +CPU= 13.570
  19:58:20 STEP 2846 H=2846:00 +CPU= 13.251
  19:58:41 STEP 2847 H=2847:00 +CPU= 20.268
  19:58:55 STEP 2848 H=2848:00 +CPU= 13.411
  19:59:07 STEP 2849 H=2849:00 +CPU= 12.139
  19:59:27 STEP 2850 H=2850:00 +CPU= 18.783
  19:59:39 STEP 2851 H=2851:00 +CPU= 12.038
  19:59:52 STEP 2852 H=2852:00 +CPU= 12.514
  20:00:11 STEP 2853 H=2853:00 +CPU= 18.806
  20:00:24 STEP 2854 H=2854:00 +CPU= 12.240
  20:00:36 STEP 2855 H=2855:00 +CPU= 12.441
  20:01:00 STEP 2856 H=2856:00 +CPU= 22.855
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_118.zip
Uploading the intermediate file: upload_file_118.zip
  20:01:13 STEP 2857 H=2857:00 +CPU= 12.247
  20:01:26 STEP 2858 H=2858:00 +CPU= 12.377
  20:01:45 STEP 2859 H=2859:00 +CPU= 18.900
  20:01:58 STEP 2860 H=2860:00 +CPU= 12.278
  20:02:10 STEP 2861 H=2861:00 +CPU= 11.971
  20:02:29 STEP 2862 H=2862:00 +CPU= 18.960
  20:02:42 STEP 2863 H=2863:00 +CPU= 12.329
  20:02:55 STEP 2864 H=2864:00 +CPU= 12.389
  20:03:14 STEP 2865 H=2865:00 +CPU= 18.916
  20:03:27 STEP 2866 H=2866:00 +CPU= 12.487
  20:03:39 STEP 2867 H=2867:00 +CPU= 12.124
  20:04:00 STEP 2868 H=2868:00 +CPU= 20.402
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002868
  20:04:13 STEP 2869 H=2869:00 +CPU= 12.173
  20:04:25 STEP 2870 H=2870:00 +CPU= 12.297
  20:04:44 STEP 2871 H=2871:00 +CPU= 18.419
  20:04:57 STEP 2872 H=2872:00 +CPU= 12.500
  20:05:09 STEP 2873 H=2873:00 +CPU= 12.064
  20:05:28 STEP 2874 H=2874:00 +CPU= 18.852
  20:05:41 STEP 2875 H=2875:00 +CPU= 12.190
  20:05:54 STEP 2876 H=2876:00 +CPU= 12.223
  20:06:13 STEP 2877 H=2877:00 +CPU= 18.720
  20:06:25 STEP 2878 H=2878:00 +CPU= 12.245
  20:06:38 STEP 2879 H=2879:00 +CPU= 12.387
  20:07:02 STEP 2880 H=2880:00 +CPU= 22.889
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_119.zip
Uploading the intermediate file: upload_file_119.zip
  20:07:14 STEP 2881 H=2881:00 +CPU= 12.123
  20:07:27 STEP 2882 H=2882:00 +CPU= 12.261
  20:07:46 STEP 2883 H=2883:00 +CPU= 18.860
  20:07:59 STEP 2884 H=2884:00 +CPU= 12.508
  20:08:12 STEP 2885 H=2885:00 +CPU= 12.126
  20:08:31 STEP 2886 H=2886:00 +CPU= 18.886
  20:08:43 STEP 2887 H=2887:00 +CPU= 11.890
  20:08:56 STEP 2888 H=2888:00 +CPU= 12.474
  20:09:15 STEP 2889 H=2889:00 +CPU= 18.536
  20:09:28 STEP 2890 H=2890:00 +CPU= 12.455
  20:09:40 STEP 2891 H=2891:00 +CPU= 12.124
  20:10:01 STEP 2892 H=2892:00 +CPU= 20.481
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002892
  20:10:13 STEP 2893 H=2893:00 +CPU= 12.245
  20:10:26 STEP 2894 H=2894:00 +CPU= 12.256
  20:10:45 STEP 2895 H=2895:00 +CPU= 18.735
  20:10:58 STEP 2896 H=2896:00 +CPU= 12.382
  20:11:10 STEP 2897 H=2897:00 +CPU= 12.035
  20:11:30 STEP 2898 H=2898:00 +CPU= 19.007
  20:11:42 STEP 2899 H=2899:00 +CPU= 12.257
  20:11:55 STEP 2900 H=2900:00 +CPU= 12.194
  20:12:14 STEP 2901 H=2901:00 +CPU= 18.617
  20:12:26 STEP 2902 H=2902:00 +CPU= 12.284
  20:12:39 STEP 2903 H=2903:00 +CPU= 12.325
  20:13:04 STEP 2904 H=2904:00 +CPU= 23.321
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_120.zip
Uploading the intermediate file: upload_file_120.zip
  20:13:16 STEP 2905 H=2905:00 +CPU= 12.473
  20:13:29 STEP 2906 H=2906:00 +CPU= 12.243
  20:13:48 STEP 2907 H=2907:00 +CPU= 18.726
  20:14:01 STEP 2908 H=2908:00 +CPU= 12.267
  20:14:13 STEP 2909 H=2909:00 +CPU= 12.161
  20:14:33 STEP 2910 H=2910:00 +CPU= 18.884
  20:14:45 STEP 2911 H=2911:00 +CPU= 12.044
  20:14:58 STEP 2912 H=2912:00 +CPU= 12.336
  20:15:17 STEP 2913 H=2913:00 +CPU= 18.829
  20:15:29 STEP 2914 H=2914:00 +CPU= 12.254
  20:15:42 STEP 2915 H=2915:00 +CPU= 12.196
  20:16:03 STEP 2916 H=2916:00 +CPU= 20.598
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002916
  20:16:15 STEP 2917 H=2917:00 +CPU= 12.234
  20:16:28 STEP 2918 H=2918:00 +CPU= 12.331
  20:16:47 STEP 2919 H=2919:00 +CPU= 18.667
  20:17:00 STEP 2920 H=2920:00 +CPU= 12.467
  20:17:12 STEP 2921 H=2921:00 +CPU= 12.203
  20:17:32 STEP 2922 H=2922:00 +CPU= 19.111
  20:17:44 STEP 2923 H=2923:00 +CPU= 12.149
  20:17:57 STEP 2924 H=2924:00 +CPU= 12.446
  20:18:16 STEP 2925 H=2925:00 +CPU= 18.755
  20:18:29 STEP 2926 H=2926:00 +CPU= 12.436
  20:18:42 STEP 2927 H=2927:00 +CPU= 12.348
  20:19:06 STEP 2928 H=2928:00 +CPU= 22.902
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_121.zip
Uploading the intermediate file: upload_file_121.zip
  20:19:18 STEP 2929 H=2929:00 +CPU= 12.159
  20:19:31 STEP 2930 H=2930:00 +CPU= 12.374
  20:19:51 STEP 2931 H=2931:00 +CPU= 18.952
  20:20:03 STEP 2932 H=2932:00 +CPU= 12.302
  20:20:16 STEP 2933 H=2933:00 +CPU= 12.050
  20:20:35 STEP 2934 H=2934:00 +CPU= 18.904
  20:20:47 STEP 2935 H=2935:00 +CPU= 11.982
  20:21:00 STEP 2936 H=2936:00 +CPU= 12.403
  20:21:19 STEP 2937 H=2937:00 +CPU= 18.693
  20:21:32 STEP 2938 H=2938:00 +CPU= 12.418
  20:21:44 STEP 2939 H=2939:00 +CPU= 12.185
  20:22:05 STEP 2940 H=2940:00 +CPU= 20.396
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002940
  20:22:18 STEP 2941 H=2941:00 +CPU= 12.194
  20:22:30 STEP 2942 H=2942:00 +CPU= 12.385
  20:22:49 STEP 2943 H=2943:00 +CPU= 18.665
  20:23:02 STEP 2944 H=2944:00 +CPU= 12.486
  20:23:15 STEP 2945 H=2945:00 +CPU= 12.123
  20:23:34 STEP 2946 H=2946:00 +CPU= 19.044
  20:23:47 STEP 2947 H=2947:00 +CPU= 12.214
  20:23:59 STEP 2948 H=2948:00 +CPU= 12.483
  20:24:19 STEP 2949 H=2949:00 +CPU= 18.941
  20:24:32 STEP 2950 H=2950:00 +CPU= 12.821
  20:24:45 STEP 2951 H=2951:00 +CPU= 12.261
  20:25:08 STEP 2952 H=2952:00 +CPU= 22.128
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12196179/ICMUAhq0f+002952
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0535_2010050100_123_979_12196179_0_r671043108_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
20:28:30 (2478): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:10:09 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 10,623,600 40,465 0.0038
09 May 2023 16:10:09 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 9,936,000 37,625 0.0038
09 May 2023 16:00:33 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 7,776,000 28,355 0.0036
09 May 2023 16:00:29 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 6,912,000 25,046 0.0036
09 May 2023 16:00:26 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 6,048,000 21,618 0.0036
09 May 2023 16:00:23 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 5,184,000 18,375 0.0035
09 May 2023 16:00:20 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 4,320,000 15,232 0.0035
09 May 2023 16:00:15 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 3,456,000 12,077 0.0035
09 May 2023 16:00:11 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 2,592,000 8,915 0.0034
09 May 2023 16:00:08 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 1,728,000 5,923 0.0034
09 May 2023 16:00:04 1529705 22282138 oifs_43r3_ps_0535_2010050100_123_979_12196179_0 864,000 3,015 0.0035


©2024 climateprediction.net