<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
--
Adjusting last_iter, 354, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12172211.xml
last_cpu_time: 16744
upload_file_number: 14
last_iter: 335
last_upload: 1209600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004
The child process has been launched with process id: 2197779
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197779:2197779] [20230115:074426:1673736266:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:46:57 STEP 336 H= 336:00 +CPU= 37.634
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:50:09 STEP 337 H= 337:00 +CPU= 20.581
07:50:31 STEP 338 H= 338:00 +CPU= 21.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:55:07 STEP 339 H= 339:00 +CPU= 32.910
07:55:28 STEP 340 H= 340:00 +CPU= 21.146
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:59:50 STEP 341 H= 341:00 +CPU= 20.488
08:00:25 STEP 342 H= 342:00 +CPU= 34.835
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:04:29 STEP 343 H= 343:00 +CPU= 21.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:07:41 STEP 344 H= 344:00 +CPU= 21.079
08:08:18 STEP 345 H= 345:00 +CPU= 36.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:13:12 STEP 346 H= 346:00 +CPU= 22.569
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:16:55 STEP 347 H= 347:00 +CPU= 21.627
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:21:22 STEP 348 H= 348:00 +CPU= 35.612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000348
08:21:43 STEP 349 H= 349:00 +CPU= 21.252
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:25:06 STEP 350 H= 350:00 +CPU= 21.247
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:29:22 STEP 351 H= 351:00 +CPU= 33.741
08:29:45 STEP 352 H= 352:00 +CPU= 22.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:41:31 STEP 353 H= 353:00 +CPU= 23.674
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:44:56 STEP 354 H= 354:00 +CPU= 33.391
08:45:16 STEP 355 H= 355:00 +CPU= 20.313
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:47:38 STEP 356 H= 356:00 +CPU= 20.841
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:50:02 STEP 357 H= 357:00 +CPU= 32.516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:52:06 STEP 358 H= 358:00 +CPU= 21.503
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:54:08 STEP 359 H= 359:00 +CPU= 20.835
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:56:35 STEP 360 H= 360:00 +CPU= 37.266
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004_14.zip
Uploading the intermediate file: upload_file_14.zip
08:56:56 STEP 361 H= 361:00 +CPU= 20.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:59:05 STEP 362 H= 362:00 +CPU= 20.404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:01:39 STEP 363 H= 363:00 +CPU= 31.772
09:02:00 STEP 364 H= 364:00 +CPU= 21.052
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:06:52 STEP 365 H= 365:00 +CPU= 20.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:09:16 STEP 366 H= 366:00 +CPU= 33.292
09:09:38 STEP 367 H= 367:00 +CPU= 21.703
09:09:59 STEP 368 H= 368:00 +CPU= 21.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:13:24 STEP 369 H= 369:00 +CPU= 32.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:16:15 STEP 370 H= 370:00 +CPU= 21.462
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:29:09 STEP 371 H= 371:00 +CPU= 20.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:32:46 STEP 372 H= 372:00 +CPU= 35.928
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000372
09:33:07 STEP 373 H= 373:00 +CPU= 20.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:36:00 STEP 374 H= 374:00 +CPU= 21.045
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:38:54 STEP 375 H= 375:00 +CPU= 32.617
09:39:16 STEP 376 H= 376:00 +CPU= 21.541
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:41:47 STEP 377 H= 377:00 +CPU= 20.261
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:44:12 STEP 378 H= 378:00 +CPU= 33.002
09:44:32 STEP 379 H= 379:00 +CPU= 20.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:47:25 STEP 380 H= 380:00 +CPU= 21.698
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0567
(argv4) batchid: 955
(argv5) wuid: 12172211
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8d435f0802ae2dafb3aa1c963d2f53f8 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0567_1986050100_123_955_12172211.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0567_1986050100_123_955_12172211.zip
ic_ancil_file: ic_ancil_12172211
ifsdata_file: ifsdata_12172211
climate_data_file: clim_data_12172211
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_9b5290b3437a83b5238da6ec6c004793 to: /var/lib/boinc-client/slots/0/ic_ancil_12172211.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12172211.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ac073620bffec7afe0cbc0bc3cba2c81 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12172211.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12172211.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12172211.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12172211.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12172211.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12172211.xml
-- Model is restarting --
Adjusting last_iter, 380, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12172211.xml
last_cpu_time: 17887
upload_file_number: 15
last_iter: 359
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004
The child process has been launched with process id: 2221683
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221683:2221683] [20230115:094920:1673743760:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:52:12 STEP 360 H= 360:00 +CPU= 36.609
09:52:32 STEP 361 H= 361:00 +CPU= 20.734
09:52:53 STEP 362 H= 362:00 +CPU= 20.506
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:55:47 STEP 363 H= 363:00 +CPU= 33.214
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:58:10 STEP 364 H= 364:00 +CPU= 21.489
09:58:31 STEP 365 H= 365:00 +CPU= 20.363
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:02:46 STEP 366 H= 366:00 +CPU= 33.226
10:03:08 STEP 367 H= 367:00 +CPU= 21.376
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:05:30 STEP 368 H= 368:00 +CPU= 20.931
10:06:02 STEP 369 H= 369:00 +CPU= 32.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:08:14 STEP 370 H= 370:00 +CPU= 20.992
10:08:35 STEP 371 H= 371:00 +CPU= 20.170
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:12:43 STEP 372 H= 372:00 +CPU= 35.917
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000372
10:13:04 STEP 373 H= 373:00 +CPU= 21.023
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:15:06 STEP 374 H= 374:00 +CPU= 21.506
10:15:38 STEP 375 H= 375:00 +CPU= 32.365
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:18:31 STEP 376 H= 376:00 +CPU= 21.016
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:20:42 STEP 377 H= 377:00 +CPU= 20.545
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:22:57 STEP 378 H= 378:00 +CPU= 33.524
10:23:18 STEP 379 H= 379:00 +CPU= 20.911
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:25:40 STEP 380 H= 380:00 +CPU= 20.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:28:24 STEP 381 H= 381:00 +CPU= 32.669
10:28:46 STEP 382 H= 382:00 +CPU= 21.922
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:31:07 STEP 383 H= 383:00 +CPU= 20.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:33:37 STEP 384 H= 384:00 +CPU= 38.033
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004_15.zip
Uploading the intermediate file: upload_file_15.zip
10:33:58 STEP 385 H= 385:00 +CPU= 21.174
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:36:31 STEP 386 H= 386:00 +CPU= 21.227
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:38:55 STEP 387 H= 387:00 +CPU= 32.274
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:40:29 STEP 388 H= 388:00 +CPU= 21.674
10:40:50 STEP 389 H= 389:00 +CPU= 20.460
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:43:24 STEP 390 H= 390:00 +CPU= 33.356
10:43:45 STEP 391 H= 391:00 +CPU= 20.449
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:46:07 STEP 392 H= 392:00 +CPU= 21.355
10:46:39 STEP 393 H= 393:00 +CPU= 31.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:49:43 STEP 394 H= 394:00 +CPU= 21.303
10:50:03 STEP 395 H= 395:00 +CPU= 19.873
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:53:01 STEP 396 H= 396:00 +CPU= 36.217
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000396
10:55:13 STEP 397 H= 397:00 +CPU= 20.469
10:55:34 STEP 398 H= 398:00 +CPU= 21.265
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:58:59 STEP 399 H= 399:00 +CPU= 32.725
10:59:20 STEP 400 H= 400:00 +CPU= 20.649
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:02:33 STEP 401 H= 401:00 +CPU= 20.596
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:05:07 STEP 402 H= 402:00 +CPU= 33.057
11:05:28 STEP 403 H= 403:00 +CPU= 20.663
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:08:20 STEP 404 H= 404:00 +CPU= 20.810
11:08:53 STEP 405 H= 405:00 +CPU= 32.896
11:09:14 STEP 406 H= 406:00 +CPU= 20.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:12:17 STEP 407 H= 407:00 +CPU= 21.642
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:15:26 STEP 408 H= 408:00 +CPU= 38.794
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004_16.zip
Uploading the intermediate file: upload_file_16.zip
11:15:45 STEP 409 H= 409:00 +CPU= 19.342
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:17:52 STEP 410 H= 410:00 +CPU= 20.611
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:21:26 STEP 411 H= 411:00 +CPU= 32.380
11:21:47 STEP 412 H= 412:00 +CPU= 21.146
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:24:58 STEP 413 H= 413:00 +CPU= 20.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:28:03 STEP 414 H= 414:00 +CPU= 33.341
11:28:23 STEP 415 H= 415:00 +CPU= 20.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:30:56 STEP 416 H= 416:00 +CPU= 21.212
11:31:27 STEP 417 H= 417:00 +CPU= 31.459
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:35:11 STEP 418 H= 418:00 +CPU= 21.880
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:37:53 STEP 419 H= 419:00 +CPU= 20.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:40:10 STEP 420 H= 420:00 +CPU= 36.022
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000420
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:42:33 STEP 421 H= 421:00 +CPU= 20.655
11:42:54 STEP 422 H= 422:00 +CPU= 20.803
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:46:38 STEP 423 H= 423:00 +CPU= 33.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:49:31 STEP 424 H= 424:00 +CPU= 21.045
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0567
(argv4) batchid: 955
(argv5) wuid: 12172211
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8d435f0802ae2dafb3aa1c963d2f53f8 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0567_1986050100_123_955_12172211.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0567_1986050100_123_955_12172211.zip
ic_ancil_file: ic_ancil_12172211
ifsdata_file: ifsdata_12172211
climate_data_file: clim_data_12172211
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_9b5290b3437a83b5238da6ec6c004793 to: /var/lib/boinc-client/slots/0/ic_ancil_12172211.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12172211.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ac073620bffec7afe0cbc0bc3cba2c81 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12172211.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12172211.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12172211.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12172211.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12172211.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12172211.xml
-- Model is restarting --
Adjusting last_iter, 425, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12172211.xml
last_cpu_time: 19550
upload_file_number: 17
last_iter: 407
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004
The child process has been launched with process id: 2240380
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240380:2240380] [20230115:115504:1673751304:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:57:36 STEP 408 H= 408:00 +CPU= 36.707
11:57:56 STEP 409 H= 409:00 +CPU= 19.987
11:58:17 STEP 410 H= 410:00 +CPU= 20.639
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:01:21 STEP 411 H= 411:00 +CPU= 32.601
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:05:23 STEP 412 H= 412:00 +CPU= 21.017
12:05:45 STEP 413 H= 413:00 +CPU= 21.447
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:09:28 STEP 414 H= 414:00 +CPU= 33.266
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:13:02 STEP 415 H= 415:00 +CPU= 20.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:16:15 STEP 416 H= 416:00 +CPU= 21.852
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:19:31 STEP 417 H= 417:00 +CPU= 33.715
12:19:52 STEP 418 H= 418:00 +CPU= 21.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:23:14 STEP 419 H= 419:00 +CPU= 20.330
12:23:50 STEP 420 H= 420:00 +CPU= 35.842
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000420
12:27:22 STEP 421 H= 421:00 +CPU= 20.205
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:30:25 STEP 422 H= 422:00 +CPU= 21.356
12:30:58 STEP 423 H= 423:00 +CPU= 32.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:34:20 STEP 424 H= 424:00 +CPU= 21.546
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:36:21 STEP 425 H= 425:00 +CPU= 19.912
12:36:53 STEP 426 H= 426:00 +CPU= 32.056
12:37:13 STEP 427 H= 427:00 +CPU= 20.110
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:22:54 STEP 428 H= 428:00 +CPU= 20.867
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:26:47 STEP 429 H= 429:00 +CPU= 32.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:29:19 STEP 430 H= 430:00 +CPU= 20.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:32:02 STEP 431 H= 431:00 +CPU= 21.153
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:02 STEP 432 H= 432:00 +CPU= 37.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004_17.zip
Uploading the intermediate file: upload_file_17.zip
22:37:24 STEP 433 H= 433:00 +CPU= 20.568
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0567
(argv4) batchid: 955
(argv5) wuid: 12172211
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8d435f0802ae2dafb3aa1c963d2f53f8 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0567_1986050100_123_955_12172211.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0567_1986050100_123_955_12172211.zip
ic_ancil_file: ic_ancil_12172211
ifsdata_file: ifsdata_12172211
climate_data_file: clim_data_12172211
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_9b5290b3437a83b5238da6ec6c004793 to: /var/lib/boinc-client/slots/0/ic_ancil_12172211.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12172211.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ac073620bffec7afe0cbc0bc3cba2c81 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12172211.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12172211.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12172211.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12172211.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12172211.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12172211.xml
-- Model is restarting --
Adjusting last_iter, 433, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12172211.xml
last_cpu_time: 20195
upload_file_number: 18
last_iter: 431
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004
The child process has been launched with process id: 2361727
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361727:2361727] [20230115:223939:1673789979:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:41:48 STEP 432 H= 432:00 +CPU= 36.244
22:42:08 STEP 433 H= 433:00 +CPU= 19.539
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:49 STEP 434 H= 434:00 +CPU= 20.316
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:47:02 STEP 435 H= 435:00 +CPU= 32.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:49:04 STEP 436 H= 436:00 +CPU= 20.905
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:04 STEP 437 H= 437:00 +CPU= 19.343
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:39 STEP 438 H= 438:00 +CPU= 32.519
22:54:00 STEP 439 H= 439:00 +CPU= 20.657
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:56:11 STEP 440 H= 440:00 +CPU= 20.329
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:58:46 STEP 441 H= 441:00 +CPU= 33.507
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:00:49 STEP 442 H= 442:00 +CPU= 21.816
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:02:49 STEP 443 H= 443:00 +CPU= 19.842
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:05:15 STEP 444 H= 444:00 +CPU= 34.777
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:07:07 STEP 445 H= 445:00 +CPU= 20.409
23:07:28 STEP 446 H= 446:00 +CPU= 20.216
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:11:23 STEP 447 H= 447:00 +CPU= 32.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:13:27 STEP 448 H= 448:00 +CPU= 21.839
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:15:20 STEP 449 H= 449:00 +CPU= 20.129
23:15:53 STEP 450 H= 450:00 +CPU= 32.826
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:18:24 STEP 451 H= 451:00 +CPU= 20.442
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:20:26 STEP 452 H= 452:00 +CPU= 20.801
23:20:58 STEP 453 H= 453:00 +CPU= 32.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:23:30 STEP 454 H= 454:00 +CPU= 20.709
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:25:51 STEP 455 H= 455:00 +CPU= 20.539
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:28:40 STEP 456 H= 456:00 +CPU= 38.040
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172211/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0567_1986050100_123_955_12172211_0_r212816004_18.zip
Uploading the intermediate file: upload_file_18.zip
23:28:58 STEP 457 H= 457:00 +CPU= 17.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:31:13 STEP 458 H= 458:00 +CPU= 19.154
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:33:27 STEP 459 H= 459:00 +CPU= 32.652
23:33:48 STEP 460 H= 460:00 +CPU= 21.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:36:10 STEP 461 H= 461:00 +CPU= 20.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:38:34 STEP 462 H= 462:00 +CPU= 33.560
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:40:46 STEP 463 H= 463:00 +CPU= 20.600
23:41:07 STEP 464 H= 464:00 +CPU= 20.884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:43:30 STEP 465 H= 465:00 +CPU= 32.285
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:45:21 STEP 466 H= 466:00 +CPU= 20.774
23:45:41 STEP 467 H= 467:00 +CPU= 19.916
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:59 STEP 468 H= 468:00 +CPU= 35.399
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000468
23:50:20 STEP 469 H= 469:00 +CPU= 20.537
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:21 STEP 470 H= 470:00 +CPU= 20.390
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:26 STEP 471 H= 471:00 +CPU= 32.612
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:56:39 STEP 472 H= 472:00 +CPU= 21.500
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
