climateprediction.net home page
Task 22305852

Task 22305852

Name oifs_43r3_ps_0844_1998050100_123_967_12184488_1
Workunit 12184488
Created 5 Feb 2023, 17:15:56 UTC
Sent 5 Feb 2023, 17:16:16 UTC
Report deadline 6 Apr 2023, 17:16:16 UTC
Received 10 Feb 2023, 10:17:42 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1512045
Run time 18 hours 27 min 50 sec
CPU time 17 hours 56 min 39 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 5.38 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.41 MB
Peak swap size 4,978.07 MB
Peak disk usage 2,017.85 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>

Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002484
  05:26:15 STEP 2485 H=2485:00 +CPU= 16.175
  05:26:31 STEP 2486 H=2486:00 +CPU= 16.486
  05:26:57 STEP 2487 H=2487:00 +CPU= 25.797
  05:27:13 STEP 2488 H=2488:00 +CPU= 16.515
  05:27:30 STEP 2489 H=2489:00 +CPU= 16.328
  05:27:56 STEP 2490 H=2490:00 +CPU= 26.151
  05:28:12 STEP 2491 H=2491:00 +CPU= 15.989
  05:28:29 STEP 2492 H=2492:00 +CPU= 16.657
  05:28:57 STEP 2493 H=2493:00 +CPU= 28.030
  05:29:15 STEP 2494 H=2494:00 +CPU= 17.951
  05:29:34 STEP 2495 H=2495:00 +CPU= 18.467
  05:30:10 STEP 2496 H=2496:00 +CPU= 34.840
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_103.zip
Uploading the intermediate file: upload_file_103.zip
  05:30:32 STEP 2497 H=2497:00 +CPU= 17.920
  05:30:47 STEP 2498 H=2498:00 +CPU= 15.453
  05:31:11 STEP 2499 H=2499:00 +CPU= 24.242
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3992MB (heap), 4415MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3992 MB (maxheap), 4415 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                CLOUD_LAYER 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053122:1676007082:56018.653] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 CLOUDSC 
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053123:1676007083:56019.658] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:ih2-VirtualBox:1:1:11001:11001] [20230210:053123:1676007083:56019.658] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC0A8C  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000101CA7F  cloudsc_                  848  cloudsc.F90
oifs_43r3_model.e  0000000001014D57  cloud_layer_              131  cloud_layer.F90
oifs_43r3_model.e  0000000000F10C1B  callpar_                 1011  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1998050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0844
(argv4) batchid: 967
(argv5) wuid: 12184488
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_adae17f924e4b976e70906461a103fc6 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0844_1998050100_123_967_12184488.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0844_1998050100_123_967_12184488.zip
ic_ancil_file: ic_ancil_12184488
ifsdata_file: ifsdata_12184488
climate_data_file: clim_data_12184488
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e0eb135090ce4565a4b610c6ea9d1808 to: /var/lib/boinc-client/slots/2/ic_ancil_12184488.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12184488.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_49e6a57ab263d4d940c5cb5d93767db3 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12184488.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12184488.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12184488.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12184488.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12184488.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12184488.xml
-- Model is restarting --
Adjusting last_iter, 2500, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12184488.xml
last_cpu_time: 54460
upload_file_number: 104
last_iter: 2495
last_upload: 8985600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481
The child process has been launched with process id: 2475
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ih2-VirtualBox:1:1:2475:2475] [20230210:071330:1676013210:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  07:14:10 STEP 2496 H=2496:00 +CPU= 32.647
  07:14:29 STEP 2497 H=2497:00 +CPU= 18.573
  07:14:47 STEP 2498 H=2498:00 +CPU= 18.051
  07:15:16 STEP 2499 H=2499:00 +CPU= 28.215
  07:15:37 STEP 2500 H=2500:00 +CPU= 20.200
  07:15:55 STEP 2501 H=2501:00 +CPU= 18.227
  07:16:26 STEP 2502 H=2502:00 +CPU= 30.623
  07:16:44 STEP 2503 H=2503:00 +CPU= 18.046
  07:17:03 STEP 2504 H=2504:00 +CPU= 18.397
  07:17:32 STEP 2505 H=2505:00 +CPU= 29.306
  07:17:52 STEP 2506 H=2506:00 +CPU= 18.779
  07:18:13 STEP 2507 H=2507:00 +CPU= 20.373
  07:18:46 STEP 2508 H=2508:00 +CPU= 32.616
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002508
  07:19:04 STEP 2509 H=2509:00 +CPU= 17.961
  07:19:23 STEP 2510 H=2510:00 +CPU= 18.422
  07:19:52 STEP 2511 H=2511:00 +CPU= 28.406
  07:20:11 STEP 2512 H=2512:00 +CPU= 18.711
  07:20:30 STEP 2513 H=2513:00 +CPU= 18.789
  07:21:00 STEP 2514 H=2514:00 +CPU= 29.776
  07:21:19 STEP 2515 H=2515:00 +CPU= 18.978
  07:21:38 STEP 2516 H=2516:00 +CPU= 18.549
  07:22:08 STEP 2517 H=2517:00 +CPU= 29.582
  07:22:26 STEP 2518 H=2518:00 +CPU= 18.140
  07:22:45 STEP 2519 H=2519:00 +CPU= 18.798
  07:23:22 STEP 2520 H=2520:00 +CPU= 33.393
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_104.zip
Uploading the intermediate file: upload_file_104.zip
  07:23:41 STEP 2521 H=2521:00 +CPU= 17.895
  07:23:59 STEP 2522 H=2522:00 +CPU= 17.942
  07:24:29 STEP 2523 H=2523:00 +CPU= 29.173
  07:24:47 STEP 2524 H=2524:00 +CPU= 18.224
  07:25:05 STEP 2525 H=2525:00 +CPU= 18.272
  07:25:35 STEP 2526 H=2526:00 +CPU= 29.706
  07:25:53 STEP 2527 H=2527:00 +CPU= 17.899
  07:26:12 STEP 2528 H=2528:00 +CPU= 18.876
  07:26:41 STEP 2529 H=2529:00 +CPU= 29.189
  07:27:00 STEP 2530 H=2530:00 +CPU= 18.469
  07:27:19 STEP 2531 H=2531:00 +CPU= 18.776
  07:27:51 STEP 2532 H=2532:00 +CPU= 32.581
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002532
  07:28:10 STEP 2533 H=2533:00 +CPU= 18.485
  07:28:28 STEP 2534 H=2534:00 +CPU= 18.352
  07:28:57 STEP 2535 H=2535:00 +CPU= 27.988
  07:29:15 STEP 2536 H=2536:00 +CPU= 18.203
  07:29:33 STEP 2537 H=2537:00 +CPU= 17.882
  07:30:01 STEP 2538 H=2538:00 +CPU= 28.534
  07:30:19 STEP 2539 H=2539:00 +CPU= 17.688
  07:30:37 STEP 2540 H=2540:00 +CPU= 18.146
  07:31:06 STEP 2541 H=2541:00 +CPU= 28.228
  07:31:24 STEP 2542 H=2542:00 +CPU= 18.166
  07:31:42 STEP 2543 H=2543:00 +CPU= 18.427
  07:32:17 STEP 2544 H=2544:00 +CPU= 33.376
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_105.zip
Uploading the intermediate file: upload_file_105.zip
  07:32:35 STEP 2545 H=2545:00 +CPU= 17.823
  07:32:54 STEP 2546 H=2546:00 +CPU= 18.579
  07:33:24 STEP 2547 H=2547:00 +CPU= 29.061
  07:33:42 STEP 2548 H=2548:00 +CPU= 18.297
  07:34:00 STEP 2549 H=2549:00 +CPU= 17.703
  07:34:28 STEP 2550 H=2550:00 +CPU= 28.299
  07:34:46 STEP 2551 H=2551:00 +CPU= 17.671
  07:35:04 STEP 2552 H=2552:00 +CPU= 18.016
  07:35:32 STEP 2553 H=2553:00 +CPU= 27.732
  07:35:50 STEP 2554 H=2554:00 +CPU= 17.815
  07:36:07 STEP 2555 H=2555:00 +CPU= 17.372
  07:36:39 STEP 2556 H=2556:00 +CPU= 31.610
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002556
  07:36:58 STEP 2557 H=2557:00 +CPU= 18.480
  07:37:17 STEP 2558 H=2558:00 +CPU= 19.040
  07:37:46 STEP 2559 H=2559:00 +CPU= 28.834
  07:38:04 STEP 2560 H=2560:00 +CPU= 18.247
  07:38:22 STEP 2561 H=2561:00 +CPU= 17.652
  07:38:51 STEP 2562 H=2562:00 +CPU= 28.513
  07:39:08 STEP 2563 H=2563:00 +CPU= 17.538
  07:39:27 STEP 2564 H=2564:00 +CPU= 18.310
  07:39:55 STEP 2565 H=2565:00 +CPU= 28.158
  07:40:13 STEP 2566 H=2566:00 +CPU= 18.000
  07:40:31 STEP 2567 H=2567:00 +CPU= 18.235
  07:41:06 STEP 2568 H=2568:00 +CPU= 33.343
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_106.zip
Uploading the intermediate file: upload_file_106.zip
  07:41:24 STEP 2569 H=2569:00 +CPU= 17.722
  07:41:43 STEP 2570 H=2570:00 +CPU= 18.598
  07:42:13 STEP 2571 H=2571:00 +CPU= 29.075
  07:42:31 STEP 2572 H=2572:00 +CPU= 18.279
  07:42:49 STEP 2573 H=2573:00 +CPU= 17.549
  07:43:17 STEP 2574 H=2574:00 +CPU= 28.403
  07:43:35 STEP 2575 H=2575:00 +CPU= 17.284
  07:43:53 STEP 2576 H=2576:00 +CPU= 17.895
  07:44:21 STEP 2577 H=2577:00 +CPU= 27.759
  07:44:39 STEP 2578 H=2578:00 +CPU= 18.172
  07:44:57 STEP 2579 H=2579:00 +CPU= 17.782
  07:45:28 STEP 2580 H=2580:00 +CPU= 31.077
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002580
  07:45:46 STEP 2581 H=2581:00 +CPU= 17.869
  07:46:04 STEP 2582 H=2582:00 +CPU= 18.137
  07:46:32 STEP 2583 H=2583:00 +CPU= 27.801
  07:46:50 STEP 2584 H=2584:00 +CPU= 18.115
  07:47:08 STEP 2585 H=2585:00 +CPU= 17.667
  07:47:37 STEP 2586 H=2586:00 +CPU= 28.325
  07:47:55 STEP 2587 H=2587:00 +CPU= 17.827
  07:48:13 STEP 2588 H=2588:00 +CPU= 17.946
  07:48:41 STEP 2589 H=2589:00 +CPU= 27.653
  07:49:00 STEP 2590 H=2590:00 +CPU= 18.236
  07:49:18 STEP 2591 H=2591:00 +CPU= 18.284
  07:49:52 STEP 2592 H=2592:00 +CPU= 33.058
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_107.zip
Uploading the intermediate file: upload_file_107.zip
  07:50:11 STEP 2593 H=2593:00 +CPU= 17.781
  07:50:29 STEP 2594 H=2594:00 +CPU= 17.958
  07:50:58 STEP 2595 H=2595:00 +CPU= 28.222
  07:51:16 STEP 2596 H=2596:00 +CPU= 18.234
  07:51:34 STEP 2597 H=2597:00 +CPU= 17.946
  07:52:02 STEP 2598 H=2598:00 +CPU= 28.309
  07:52:20 STEP 2599 H=2599:00 +CPU= 17.672
  07:52:38 STEP 2600 H=2600:00 +CPU= 18.021
  07:53:07 STEP 2601 H=2601:00 +CPU= 27.898
  07:53:25 STEP 2602 H=2602:00 +CPU= 18.209
  07:53:43 STEP 2603 H=2603:00 +CPU= 17.917
  07:54:14 STEP 2604 H=2604:00 +CPU= 30.789
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002604
  07:54:32 STEP 2605 H=2605:00 +CPU= 17.684
  07:54:50 STEP 2606 H=2606:00 +CPU= 17.918
  07:55:17 STEP 2607 H=2607:00 +CPU= 27.390
  07:55:35 STEP 2608 H=2608:00 +CPU= 18.203
  07:55:53 STEP 2609 H=2609:00 +CPU= 17.818
  07:56:22 STEP 2610 H=2610:00 +CPU= 28.212
  07:56:40 STEP 2611 H=2611:00 +CPU= 17.857
  07:56:59 STEP 2612 H=2612:00 +CPU= 18.473
  07:57:30 STEP 2613 H=2613:00 +CPU= 28.794
  07:57:53 STEP 2614 H=2614:00 +CPU= 18.555
  07:58:15 STEP 2615 H=2615:00 +CPU= 17.923
  07:58:54 STEP 2616 H=2616:00 +CPU= 32.535
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_108.zip
Uploading the intermediate file: upload_file_108.zip
  07:59:13 STEP 2617 H=2617:00 +CPU= 17.497
  07:59:32 STEP 2618 H=2618:00 +CPU= 18.130
  08:00:00 STEP 2619 H=2619:00 +CPU= 28.449
  08:00:19 STEP 2620 H=2620:00 +CPU= 18.240
  08:00:37 STEP 2621 H=2621:00 +CPU= 17.839
  08:01:06 STEP 2622 H=2622:00 +CPU= 28.299
  08:01:23 STEP 2623 H=2623:00 +CPU= 17.861
  08:01:42 STEP 2624 H=2624:00 +CPU= 18.161
  08:02:10 STEP 2625 H=2625:00 +CPU= 27.794
  08:02:28 STEP 2626 H=2626:00 +CPU= 17.857
  08:02:46 STEP 2627 H=2627:00 +CPU= 17.595
  08:03:17 STEP 2628 H=2628:00 +CPU= 30.811
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002628
  08:03:35 STEP 2629 H=2629:00 +CPU= 17.564
  08:03:53 STEP 2630 H=2630:00 +CPU= 18.175
  08:04:21 STEP 2631 H=2631:00 +CPU= 27.927
  08:04:39 STEP 2632 H=2632:00 +CPU= 18.001
  08:04:57 STEP 2633 H=2633:00 +CPU= 17.573
  08:05:25 STEP 2634 H=2634:00 +CPU= 28.203
  08:05:43 STEP 2635 H=2635:00 +CPU= 17.467
  08:06:02 STEP 2636 H=2636:00 +CPU= 17.933
  08:06:30 STEP 2637 H=2637:00 +CPU= 27.869
  08:06:48 STEP 2638 H=2638:00 +CPU= 18.028
  08:07:06 STEP 2639 H=2639:00 +CPU= 18.258
  08:07:39 STEP 2640 H=2640:00 +CPU= 32.195
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_109.zip
Uploading the intermediate file: upload_file_109.zip
  08:07:57 STEP 2641 H=2641:00 +CPU= 17.242
  08:08:15 STEP 2642 H=2642:00 +CPU= 18.159
  08:08:44 STEP 2643 H=2643:00 +CPU= 28.566
  08:09:02 STEP 2644 H=2644:00 +CPU= 18.281
  08:09:20 STEP 2645 H=2645:00 +CPU= 17.729
  08:09:48 STEP 2646 H=2646:00 +CPU= 28.141
  08:10:06 STEP 2647 H=2647:00 +CPU= 17.676
  08:10:25 STEP 2648 H=2648:00 +CPU= 18.208
  08:10:53 STEP 2649 H=2649:00 +CPU= 28.125
  08:11:11 STEP 2650 H=2650:00 +CPU= 17.821
  08:11:29 STEP 2651 H=2651:00 +CPU= 18.153
  08:12:02 STEP 2652 H=2652:00 +CPU= 32.038
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002652
  08:12:21 STEP 2653 H=2653:00 +CPU= 18.905
  08:12:40 STEP 2654 H=2654:00 +CPU= 19.108
  08:13:10 STEP 2655 H=2655:00 +CPU= 29.632
  08:13:29 STEP 2656 H=2656:00 +CPU= 19.276
  08:13:48 STEP 2657 H=2657:00 +CPU= 18.318
  08:14:18 STEP 2658 H=2658:00 +CPU= 29.985
  08:14:36 STEP 2659 H=2659:00 +CPU= 18.283
  08:14:55 STEP 2660 H=2660:00 +CPU= 18.481
  08:15:23 STEP 2661 H=2661:00 +CPU= 28.431
  08:15:42 STEP 2662 H=2662:00 +CPU= 18.573
  08:16:01 STEP 2663 H=2663:00 +CPU= 18.820
  08:16:36 STEP 2664 H=2664:00 +CPU= 34.195
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_110.zip
Uploading the intermediate file: upload_file_110.zip
  08:16:54 STEP 2665 H=2665:00 +CPU= 17.950
  08:17:14 STEP 2666 H=2666:00 +CPU= 18.649
  08:17:43 STEP 2667 H=2667:00 +CPU= 29.267
  08:18:03 STEP 2668 H=2668:00 +CPU= 19.845
  08:18:21 STEP 2669 H=2669:00 +CPU= 17.866
  08:18:51 STEP 2670 H=2670:00 +CPU= 29.348
  08:19:09 STEP 2671 H=2671:00 +CPU= 17.885
  08:19:27 STEP 2672 H=2672:00 +CPU= 18.476
  08:19:56 STEP 2673 H=2673:00 +CPU= 28.356
  08:20:14 STEP 2674 H=2674:00 +CPU= 18.231
  08:20:32 STEP 2675 H=2675:00 +CPU= 18.112
  08:21:03 STEP 2676 H=2676:00 +CPU= 31.074
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002676
  08:21:21 STEP 2677 H=2677:00 +CPU= 17.983
  08:21:40 STEP 2678 H=2678:00 +CPU= 18.287
  08:22:08 STEP 2679 H=2679:00 +CPU= 27.985
  08:22:27 STEP 2680 H=2680:00 +CPU= 18.622
  08:22:45 STEP 2681 H=2681:00 +CPU= 18.455
  08:23:14 STEP 2682 H=2682:00 +CPU= 28.672
  08:23:32 STEP 2683 H=2683:00 +CPU= 18.005
  08:23:50 STEP 2684 H=2684:00 +CPU= 18.359
  08:24:19 STEP 2685 H=2685:00 +CPU= 28.391
  08:24:37 STEP 2686 H=2686:00 +CPU= 17.949
  08:24:55 STEP 2687 H=2687:00 +CPU= 17.810
  08:25:30 STEP 2688 H=2688:00 +CPU= 33.176
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_111.zip
Uploading the intermediate file: upload_file_111.zip
  08:25:47 STEP 2689 H=2689:00 +CPU= 17.440
  08:26:07 STEP 2690 H=2690:00 +CPU= 18.631
  08:26:36 STEP 2691 H=2691:00 +CPU= 28.959
  08:26:55 STEP 2692 H=2692:00 +CPU= 18.609
  08:27:13 STEP 2693 H=2693:00 +CPU= 18.327
  08:27:42 STEP 2694 H=2694:00 +CPU= 28.857
  08:28:01 STEP 2695 H=2695:00 +CPU= 18.256
  08:28:19 STEP 2696 H=2696:00 +CPU= 18.228
  08:28:48 STEP 2697 H=2697:00 +CPU= 28.527
  08:29:06 STEP 2698 H=2698:00 +CPU= 18.310
  08:29:25 STEP 2699 H=2699:00 +CPU= 18.385
  08:29:56 STEP 2700 H=2700:00 +CPU= 31.528
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002700
  08:30:14 STEP 2701 H=2701:00 +CPU= 17.672
  08:30:32 STEP 2702 H=2702:00 +CPU= 18.067
  08:31:01 STEP 2703 H=2703:00 +CPU= 28.480
  08:31:20 STEP 2704 H=2704:00 +CPU= 18.961
  08:31:39 STEP 2705 H=2705:00 +CPU= 19.091
  08:32:08 STEP 2706 H=2706:00 +CPU= 28.993
  08:32:26 STEP 2707 H=2707:00 +CPU= 17.636
  08:32:44 STEP 2708 H=2708:00 +CPU= 18.161
  08:33:13 STEP 2709 H=2709:00 +CPU= 28.681
  08:33:33 STEP 2710 H=2710:00 +CPU= 19.435
  08:33:52 STEP 2711 H=2711:00 +CPU= 19.346
  08:34:29 STEP 2712 H=2712:00 +CPU= 35.008
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_112.zip
Uploading the intermediate file: upload_file_112.zip
  08:34:47 STEP 2713 H=2713:00 +CPU= 17.616
  08:35:06 STEP 2714 H=2714:00 +CPU= 18.439
  08:35:35 STEP 2715 H=2715:00 +CPU= 29.486
  08:35:54 STEP 2716 H=2716:00 +CPU= 19.032
  08:36:18 STEP 2717 H=2717:00 +CPU= 23.227
  08:36:58 STEP 2718 H=2718:00 +CPU= 39.737
  08:37:20 STEP 2719 H=2719:00 +CPU= 21.959
  08:37:42 STEP 2720 H=2720:00 +CPU= 22.342
  08:38:14 STEP 2721 H=2721:00 +CPU= 31.660
  08:38:33 STEP 2722 H=2722:00 +CPU= 18.981
  08:38:54 STEP 2723 H=2723:00 +CPU= 20.662
  08:39:30 STEP 2724 H=2724:00 +CPU= 35.301
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002724
  08:39:49 STEP 2725 H=2725:00 +CPU= 19.237
  08:40:08 STEP 2726 H=2726:00 +CPU= 18.885
  08:40:37 STEP 2727 H=2727:00 +CPU= 28.651
  08:40:55 STEP 2728 H=2728:00 +CPU= 18.320
  08:41:14 STEP 2729 H=2729:00 +CPU= 18.348
  08:41:43 STEP 2730 H=2730:00 +CPU= 29.336
  08:42:01 STEP 2731 H=2731:00 +CPU= 18.012
  08:42:20 STEP 2732 H=2732:00 +CPU= 18.604
  08:42:49 STEP 2733 H=2733:00 +CPU= 29.101
  08:43:07 STEP 2734 H=2734:00 +CPU= 17.745
  08:43:25 STEP 2735 H=2735:00 +CPU= 18.388
  08:44:01 STEP 2736 H=2736:00 +CPU= 34.633
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_113.zip
Uploading the intermediate file: upload_file_113.zip
  08:44:19 STEP 2737 H=2737:00 +CPU= 17.855
Uploading trickle at timestep: 9849600
  08:44:39 STEP 2738 H=2738:00 +CPU= 18.622
  08:45:08 STEP 2739 H=2739:00 +CPU= 28.984
  08:45:27 STEP 2740 H=2740:00 +CPU= 18.950
  08:45:46 STEP 2741 H=2741:00 +CPU= 18.227
  08:46:15 STEP 2742 H=2742:00 +CPU= 29.065
  08:46:33 STEP 2743 H=2743:00 +CPU= 18.184
  08:46:52 STEP 2744 H=2744:00 +CPU= 18.641
  08:47:20 STEP 2745 H=2745:00 +CPU= 28.532
  08:47:38 STEP 2746 H=2746:00 +CPU= 17.906
  08:47:56 STEP 2747 H=2747:00 +CPU= 17.565
  08:48:30 STEP 2748 H=2748:00 +CPU= 32.088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002748
  08:48:47 STEP 2749 H=2749:00 +CPU= 17.743
  08:49:06 STEP 2750 H=2750:00 +CPU= 19.018
  08:49:37 STEP 2751 H=2751:00 +CPU= 30.262
  08:49:57 STEP 2752 H=2752:00 +CPU= 19.716
  08:50:15 STEP 2753 H=2753:00 +CPU= 18.049
  08:50:44 STEP 2754 H=2754:00 +CPU= 29.000
  08:51:02 STEP 2755 H=2755:00 +CPU= 17.946
  08:51:21 STEP 2756 H=2756:00 +CPU= 18.274
  08:51:49 STEP 2757 H=2757:00 +CPU= 27.775
  08:52:07 STEP 2758 H=2758:00 +CPU= 17.946
  08:52:25 STEP 2759 H=2759:00 +CPU= 18.460
  08:52:59 STEP 2760 H=2760:00 +CPU= 33.040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_114.zip
Uploading the intermediate file: upload_file_114.zip
  08:53:17 STEP 2761 H=2761:00 +CPU= 17.473
  08:53:37 STEP 2762 H=2762:00 +CPU= 18.783
  08:54:06 STEP 2763 H=2763:00 +CPU= 28.359
  08:54:24 STEP 2764 H=2764:00 +CPU= 18.195
  08:54:42 STEP 2765 H=2765:00 +CPU= 18.325
  08:55:11 STEP 2766 H=2766:00 +CPU= 28.765
  08:55:29 STEP 2767 H=2767:00 +CPU= 17.751
  08:55:48 STEP 2768 H=2768:00 +CPU= 18.287
  08:56:16 STEP 2769 H=2769:00 +CPU= 28.066
  08:56:34 STEP 2770 H=2770:00 +CPU= 17.867
  08:56:52 STEP 2771 H=2771:00 +CPU= 17.734
  08:57:23 STEP 2772 H=2772:00 +CPU= 31.339
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002772
  08:57:41 STEP 2773 H=2773:00 +CPU= 17.557
  08:58:00 STEP 2774 H=2774:00 +CPU= 18.976
  08:58:29 STEP 2775 H=2775:00 +CPU= 28.909
  08:58:48 STEP 2776 H=2776:00 +CPU= 18.386
  08:59:06 STEP 2777 H=2777:00 +CPU= 18.072
  08:59:36 STEP 2778 H=2778:00 +CPU= 29.321
  08:59:54 STEP 2779 H=2779:00 +CPU= 17.845
  09:00:13 STEP 2780 H=2780:00 +CPU= 18.746
  09:00:42 STEP 2781 H=2781:00 +CPU= 28.726
  09:01:00 STEP 2782 H=2782:00 +CPU= 18.228
  09:01:20 STEP 2783 H=2783:00 +CPU= 19.275
  09:01:55 STEP 2784 H=2784:00 +CPU= 33.174
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_115.zip
Uploading the intermediate file: upload_file_115.zip
  09:02:13 STEP 2785 H=2785:00 +CPU= 17.818
  09:02:33 STEP 2786 H=2786:00 +CPU= 18.398
  09:03:03 STEP 2787 H=2787:00 +CPU= 29.110
  09:03:21 STEP 2788 H=2788:00 +CPU= 18.285
  09:03:40 STEP 2789 H=2789:00 +CPU= 18.274
  09:04:10 STEP 2790 H=2790:00 +CPU= 29.414
  09:04:28 STEP 2791 H=2791:00 +CPU= 17.677
  09:04:46 STEP 2792 H=2792:00 +CPU= 18.755
  09:05:16 STEP 2793 H=2793:00 +CPU= 29.320
  09:05:35 STEP 2794 H=2794:00 +CPU= 18.663
  09:05:53 STEP 2795 H=2795:00 +CPU= 17.943
  09:06:25 STEP 2796 H=2796:00 +CPU= 31.974
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002796
  09:06:43 STEP 2797 H=2797:00 +CPU= 17.687
  09:07:02 STEP 2798 H=2798:00 +CPU= 18.406
  09:07:31 STEP 2799 H=2799:00 +CPU= 29.014
  09:07:50 STEP 2800 H=2800:00 +CPU= 18.143
  09:08:08 STEP 2801 H=2801:00 +CPU= 17.998
  09:08:38 STEP 2802 H=2802:00 +CPU= 29.565
  09:08:56 STEP 2803 H=2803:00 +CPU= 17.826
  09:09:15 STEP 2804 H=2804:00 +CPU= 18.354
  09:09:43 STEP 2805 H=2805:00 +CPU= 27.763
  09:10:01 STEP 2806 H=2806:00 +CPU= 18.284
  09:10:20 STEP 2807 H=2807:00 +CPU= 18.315
  09:10:55 STEP 2808 H=2808:00 +CPU= 33.617
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_116.zip
Uploading the intermediate file: upload_file_116.zip
  09:11:13 STEP 2809 H=2809:00 +CPU= 18.131
  09:11:33 STEP 2810 H=2810:00 +CPU= 18.847
  09:12:03 STEP 2811 H=2811:00 +CPU= 29.603
  09:12:22 STEP 2812 H=2812:00 +CPU= 18.874
  09:12:40 STEP 2813 H=2813:00 +CPU= 18.008
  09:13:09 STEP 2814 H=2814:00 +CPU= 29.149
  09:13:28 STEP 2815 H=2815:00 +CPU= 18.053
  09:13:47 STEP 2816 H=2816:00 +CPU= 18.529
  09:14:15 STEP 2817 H=2817:00 +CPU= 28.690
  09:14:34 STEP 2818 H=2818:00 +CPU= 18.924
  09:14:53 STEP 2819 H=2819:00 +CPU= 18.233
  09:15:25 STEP 2820 H=2820:00 +CPU= 31.633
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002820
  09:15:43 STEP 2821 H=2821:00 +CPU= 18.510
  09:16:03 STEP 2822 H=2822:00 +CPU= 18.874
  09:16:31 STEP 2823 H=2823:00 +CPU= 28.410
  09:16:50 STEP 2824 H=2824:00 +CPU= 18.518
  09:17:08 STEP 2825 H=2825:00 +CPU= 18.003
  09:17:38 STEP 2826 H=2826:00 +CPU= 29.225
  09:17:56 STEP 2827 H=2827:00 +CPU= 18.162
  09:18:15 STEP 2828 H=2828:00 +CPU= 18.346
  09:18:44 STEP 2829 H=2829:00 +CPU= 28.632
  09:19:02 STEP 2830 H=2830:00 +CPU= 18.139
  09:19:21 STEP 2831 H=2831:00 +CPU= 18.728
  09:19:56 STEP 2832 H=2832:00 +CPU= 33.706
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_117.zip
Uploading the intermediate file: upload_file_117.zip
  09:20:14 STEP 2833 H=2833:00 +CPU= 17.753
  09:20:33 STEP 2834 H=2834:00 +CPU= 18.448
  09:21:02 STEP 2835 H=2835:00 +CPU= 28.983
  09:21:21 STEP 2836 H=2836:00 +CPU= 18.444
  09:21:39 STEP 2837 H=2837:00 +CPU= 18.171
  09:22:09 STEP 2838 H=2838:00 +CPU= 29.587
  09:22:27 STEP 2839 H=2839:00 +CPU= 17.865
  09:22:46 STEP 2840 H=2840:00 +CPU= 18.359
  09:23:15 STEP 2841 H=2841:00 +CPU= 28.353
  09:23:33 STEP 2842 H=2842:00 +CPU= 18.093
  09:23:51 STEP 2843 H=2843:00 +CPU= 17.770
  09:24:23 STEP 2844 H=2844:00 +CPU= 32.203
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002844
  09:24:42 STEP 2845 H=2845:00 +CPU= 18.220
  09:25:00 STEP 2846 H=2846:00 +CPU= 18.366
  09:25:29 STEP 2847 H=2847:00 +CPU= 28.370
  09:25:47 STEP 2848 H=2848:00 +CPU= 18.381
  09:26:05 STEP 2849 H=2849:00 +CPU= 17.929
  09:26:35 STEP 2850 H=2850:00 +CPU= 29.826
  09:26:54 STEP 2851 H=2851:00 +CPU= 18.162
  09:27:13 STEP 2852 H=2852:00 +CPU= 19.059
  09:27:44 STEP 2853 H=2853:00 +CPU= 29.899
  09:28:02 STEP 2854 H=2854:00 +CPU= 18.330
  09:28:22 STEP 2855 H=2855:00 +CPU= 19.525
  09:28:58 STEP 2856 H=2856:00 +CPU= 34.408
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_118.zip
Uploading the intermediate file: upload_file_118.zip
  09:29:16 STEP 2857 H=2857:00 +CPU= 17.818
  09:29:35 STEP 2858 H=2858:00 +CPU= 18.289
  09:30:05 STEP 2859 H=2859:00 +CPU= 29.276
  09:30:23 STEP 2860 H=2860:00 +CPU= 18.466
  09:30:42 STEP 2861 H=2861:00 +CPU= 18.776
  09:31:12 STEP 2862 H=2862:00 +CPU= 29.462
  09:31:31 STEP 2863 H=2863:00 +CPU= 18.370
  09:31:50 STEP 2864 H=2864:00 +CPU= 18.910
  09:32:19 STEP 2865 H=2865:00 +CPU= 28.768
  09:32:38 STEP 2866 H=2866:00 +CPU= 18.518
  09:32:56 STEP 2867 H=2867:00 +CPU= 18.261
  09:33:29 STEP 2868 H=2868:00 +CPU= 32.781
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002868
  09:33:47 STEP 2869 H=2869:00 +CPU= 17.944
  09:34:06 STEP 2870 H=2870:00 +CPU= 18.707
  09:34:35 STEP 2871 H=2871:00 +CPU= 29.160
  09:34:54 STEP 2872 H=2872:00 +CPU= 18.078
  09:35:12 STEP 2873 H=2873:00 +CPU= 18.039
  09:35:41 STEP 2874 H=2874:00 +CPU= 28.762
  09:35:59 STEP 2875 H=2875:00 +CPU= 17.685
  09:36:17 STEP 2876 H=2876:00 +CPU= 18.287
  09:36:45 STEP 2877 H=2877:00 +CPU= 27.822
  09:37:03 STEP 2878 H=2878:00 +CPU= 18.235
  09:37:22 STEP 2879 H=2879:00 +CPU= 18.972
  09:37:57 STEP 2880 H=2880:00 +CPU= 33.602
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_119.zip
Uploading the intermediate file: upload_file_119.zip
  09:38:15 STEP 2881 H=2881:00 +CPU= 17.365
  09:38:35 STEP 2882 H=2882:00 +CPU= 18.916
  09:39:06 STEP 2883 H=2883:00 +CPU= 28.238
  09:39:26 STEP 2884 H=2884:00 +CPU= 18.824
  09:39:44 STEP 2885 H=2885:00 +CPU= 18.241
  09:40:14 STEP 2886 H=2886:00 +CPU= 28.993
  09:40:32 STEP 2887 H=2887:00 +CPU= 18.018
  09:40:51 STEP 2888 H=2888:00 +CPU= 18.800
  09:41:20 STEP 2889 H=2889:00 +CPU= 28.702
  09:41:39 STEP 2890 H=2890:00 +CPU= 18.059
  09:41:58 STEP 2891 H=2891:00 +CPU= 18.418
  09:42:30 STEP 2892 H=2892:00 +CPU= 31.767
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002892
  09:42:48 STEP 2893 H=2893:00 +CPU= 17.521
  09:43:07 STEP 2894 H=2894:00 +CPU= 18.285
  09:43:36 STEP 2895 H=2895:00 +CPU= 28.406
  09:43:54 STEP 2896 H=2896:00 +CPU= 17.914
  09:44:13 STEP 2897 H=2897:00 +CPU= 18.088
  09:44:42 STEP 2898 H=2898:00 +CPU= 28.841
  09:45:00 STEP 2899 H=2899:00 +CPU= 17.844
  09:45:19 STEP 2900 H=2900:00 +CPU= 18.748
  09:45:48 STEP 2901 H=2901:00 +CPU= 28.957
  09:46:07 STEP 2902 H=2902:00 +CPU= 18.635
  09:46:26 STEP 2903 H=2903:00 +CPU= 18.694
  09:47:01 STEP 2904 H=2904:00 +CPU= 33.142
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_120.zip
Uploading the intermediate file: upload_file_120.zip
  09:47:20 STEP 2905 H=2905:00 +CPU= 17.939
  09:47:39 STEP 2906 H=2906:00 +CPU= 18.648
  09:48:09 STEP 2907 H=2907:00 +CPU= 29.932
  09:48:28 STEP 2908 H=2908:00 +CPU= 18.565
  09:48:47 STEP 2909 H=2909:00 +CPU= 18.547
  09:49:16 STEP 2910 H=2910:00 +CPU= 29.410
  09:49:34 STEP 2911 H=2911:00 +CPU= 18.020
  09:49:53 STEP 2912 H=2912:00 +CPU= 18.710
  09:50:23 STEP 2913 H=2913:00 +CPU= 29.677
  09:50:43 STEP 2914 H=2914:00 +CPU= 19.018
  09:51:02 STEP 2915 H=2915:00 +CPU= 19.324
  09:51:34 STEP 2916 H=2916:00 +CPU= 31.361
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002916
  09:51:52 STEP 2917 H=2917:00 +CPU= 17.839
  09:52:10 STEP 2918 H=2918:00 +CPU= 18.463
  09:52:40 STEP 2919 H=2919:00 +CPU= 29.369
  09:52:59 STEP 2920 H=2920:00 +CPU= 19.330
  09:53:19 STEP 2921 H=2921:00 +CPU= 19.028
  09:53:49 STEP 2922 H=2922:00 +CPU= 29.076
  09:54:08 STEP 2923 H=2923:00 +CPU= 18.018
  09:54:27 STEP 2924 H=2924:00 +CPU= 18.489
  09:54:56 STEP 2925 H=2925:00 +CPU= 28.569
  09:55:14 STEP 2926 H=2926:00 +CPU= 18.361
  09:55:34 STEP 2927 H=2927:00 +CPU= 18.829
  09:56:10 STEP 2928 H=2928:00 +CPU= 33.682
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_121.zip
Uploading the intermediate file: upload_file_121.zip
  09:56:28 STEP 2929 H=2929:00 +CPU= 17.790
  09:56:48 STEP 2930 H=2930:00 +CPU= 18.522
  09:57:18 STEP 2931 H=2931:00 +CPU= 28.913
  09:57:38 STEP 2932 H=2932:00 +CPU= 17.758
  09:57:56 STEP 2933 H=2933:00 +CPU= 18.371
  09:58:26 STEP 2934 H=2934:00 +CPU= 29.043
  09:58:44 STEP 2935 H=2935:00 +CPU= 17.803
  09:59:03 STEP 2936 H=2936:00 +CPU= 18.346
  09:59:32 STEP 2937 H=2937:00 +CPU= 29.176
  09:59:50 STEP 2938 H=2938:00 +CPU= 18.151
  10:00:09 STEP 2939 H=2939:00 +CPU= 18.091
  10:00:41 STEP 2940 H=2940:00 +CPU= 31.780
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002940
  10:00:59 STEP 2941 H=2941:00 +CPU= 17.657
  10:01:19 STEP 2942 H=2942:00 +CPU= 19.426
  10:01:48 STEP 2943 H=2943:00 +CPU= 28.963
  10:02:07 STEP 2944 H=2944:00 +CPU= 18.329
  10:02:25 STEP 2945 H=2945:00 +CPU= 18.102
  10:02:54 STEP 2946 H=2946:00 +CPU= 28.450
  10:03:12 STEP 2947 H=2947:00 +CPU= 17.421
  10:03:31 STEP 2948 H=2948:00 +CPU= 18.760
  10:04:00 STEP 2949 H=2949:00 +CPU= 28.392
  10:04:18 STEP 2950 H=2950:00 +CPU= 17.910
  10:04:37 STEP 2951 H=2951:00 +CPU= 18.276
  10:05:12 STEP 2952 H=2952:00 +CPU= 32.890
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184488/ICMUAhq0f+002952
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0844_1998050100_123_967_12184488_1_r1510994481_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
10:08:35 (2459): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:50:26 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 10,623,600 64,599 0.0061
09 May 2023 16:50:25 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 9,849,600 59,817 0.0061
09 May 2023 16:50:18 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 5,184,000 31,625 0.0061
09 May 2023 16:50:17 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 4,320,000 26,480 0.0061
09 May 2023 16:50:15 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 3,456,000 21,074 0.0061
09 May 2023 16:50:14 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 2,592,000 15,690 0.0061
09 May 2023 16:50:12 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 1,728,000 10,368 0.0060
09 May 2023 16:50:11 1512045 22305852 oifs_43r3_ps_0844_1998050100_123_967_12184488_1 864,000 5,194 0.0060


©2024 climateprediction.net