<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ient/slots/2/progress_file_12185413.xml
last_cpu_time: 9468
upload_file_number: 13
last_iter: 311
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345
The child process has been launched with process id: 2197278
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2197278:2197278] [20230115:074117:1673736077:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:44:37 STEP 312 H= 312:00 +CPU= 36.957
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:46:49 STEP 313 H= 313:00 +CPU= 20.184
07:47:10 STEP 314 H= 314:00 +CPU= 21.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:50:37 STEP 315 H= 315:00 +CPU= 34.062
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:55:00 STEP 316 H= 316:00 +CPU= 21.172
07:55:22 STEP 317 H= 317:00 +CPU= 21.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
07:59:57 STEP 318 H= 318:00 +CPU= 34.077
08:00:19 STEP 319 H= 319:00 +CPU= 21.566
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:04:23 STEP 320 H= 320:00 +CPU= 21.522
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:07:47 STEP 321 H= 321:00 +CPU= 33.020
08:08:10 STEP 322 H= 322:00 +CPU= 23.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:13:05 STEP 323 H= 323:00 +CPU= 22.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:17:02 STEP 324 H= 324:00 +CPU= 36.808
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000324
08:17:22 STEP 325 H= 325:00 +CPU= 19.991
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:21:34 STEP 326 H= 326:00 +CPU= 21.779
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:25:08 STEP 327 H= 327:00 +CPU= 32.508
08:25:29 STEP 328 H= 328:00 +CPU= 21.333
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:29:34 STEP 329 H= 329:00 +CPU= 21.811
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:41:34 STEP 330 H= 330:00 +CPU= 37.499
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:44:46 STEP 331 H= 331:00 +CPU= 20.117
08:45:07 STEP 332 H= 332:00 +CPU= 21.315
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:47:41 STEP 333 H= 333:00 +CPU= 33.086
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:49:52 STEP 334 H= 334:00 +CPU= 20.577
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:51:55 STEP 335 H= 335:00 +CPU= 20.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:54:15 STEP 336 H= 336:00 +CPU= 39.145
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345_13.zip
Uploading the intermediate file: upload_file_13.zip
08:54:35 STEP 337 H= 337:00 +CPU= 19.847
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:56:45 STEP 338 H= 338:00 +CPU= 20.664
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:59:10 STEP 339 H= 339:00 +CPU= 33.225
08:59:30 STEP 340 H= 340:00 +CPU= 20.152
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:01:53 STEP 341 H= 341:00 +CPU= 20.792
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:06:57 STEP 342 H= 342:00 +CPU= 33.211
09:07:17 STEP 343 H= 343:00 +CPU= 19.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:09:30 STEP 344 H= 344:00 +CPU= 21.563
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:12:56 STEP 345 H= 345:00 +CPU= 33.194
09:13:18 STEP 346 H= 346:00 +CPU= 21.845
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:16:08 STEP 347 H= 347:00 +CPU= 20.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:29:18 STEP 348 H= 348:00 +CPU= 36.786
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000348
09:29:38 STEP 349 H= 349:00 +CPU= 20.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:33:02 STEP 350 H= 350:00 +CPU= 21.317
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:36:06 STEP 351 H= 351:00 +CPU= 32.810
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:38:48 STEP 352 H= 352:00 +CPU= 20.904
09:39:10 STEP 353 H= 353:00 +CPU= 21.750
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:41:55 STEP 354 H= 354:00 +CPU= 33.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0769
(argv4) batchid: 968
(argv5) wuid: 12185413
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9640ea6f4ab22626b74f75aba07c6258 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0769_1999050100_123_968_12185413.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0769_1999050100_123_968_12185413.zip
ic_ancil_file: ic_ancil_12185413
ifsdata_file: ifsdata_12185413
climate_data_file: clim_data_12185413
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6a225ec94cbfa70f94e23bab6efd20c4 to: /var/lib/boinc-client/slots/2/ic_ancil_12185413.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12185413.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_891823936fdbe8ce5d490b475ffc770b to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12185413.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12185413.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12185413.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12185413.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12185413.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12185413.xml
-- Model is restarting --
Adjusting last_iter, 355, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12185413.xml
last_cpu_time: 10582
upload_file_number: 14
last_iter: 335
last_upload: 1209600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345
The child process has been launched with process id: 2221393
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2221393:2221393] [20230115:094710:1673743630:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:49:32 STEP 336 H= 336:00 +CPU= 36.705
09:49:53 STEP 337 H= 337:00 +CPU= 20.647
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:52:27 STEP 338 H= 338:00 +CPU= 21.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:55:21 STEP 339 H= 339:00 +CPU= 32.317
09:55:43 STEP 340 H= 340:00 +CPU= 21.443
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:58:04 STEP 341 H= 341:00 +CPU= 20.072
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:00:49 STEP 342 H= 342:00 +CPU= 33.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:02:40 STEP 343 H= 343:00 +CPU= 20.944
10:03:02 STEP 344 H= 344:00 +CPU= 21.881
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:05:36 STEP 345 H= 345:00 +CPU= 32.626
10:05:57 STEP 346 H= 346:00 +CPU= 20.717
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:08:08 STEP 347 H= 347:00 +CPU= 20.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:10:45 STEP 348 H= 348:00 +CPU= 35.522
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000348
10:11:06 STEP 349 H= 349:00 +CPU= 20.814
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:12:58 STEP 350 H= 350:00 +CPU= 21.286
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:15:12 STEP 351 H= 351:00 +CPU= 32.534
10:15:33 STEP 352 H= 352:00 +CPU= 21.271
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:18:25 STEP 353 H= 353:00 +CPU= 20.487
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:20:50 STEP 354 H= 354:00 +CPU= 33.418
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:22:52 STEP 355 H= 355:00 +CPU= 20.896
10:23:14 STEP 356 H= 356:00 +CPU= 21.573
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:25:48 STEP 357 H= 357:00 +CPU= 32.771
10:26:09 STEP 358 H= 358:00 +CPU= 21.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:28:43 STEP 359 H= 359:00 +CPU= 21.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:31:22 STEP 360 H= 360:00 +CPU= 38.049
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345_14.zip
Uploading the intermediate file: upload_file_14.zip
10:31:42 STEP 361 H= 361:00 +CPU= 19.562
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:33:42 STEP 362 H= 362:00 +CPU= 20.065
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:36:28 STEP 363 H= 363:00 +CPU= 33.389
10:36:49 STEP 364 H= 364:00 +CPU= 21.249
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:39:02 STEP 365 H= 365:00 +CPU= 20.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:40:47 STEP 366 H= 366:00 +CPU= 33.495
10:41:08 STEP 367 H= 367:00 +CPU= 20.413
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:43:31 STEP 368 H= 368:00 +CPU= 21.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:46:05 STEP 369 H= 369:00 +CPU= 32.547
10:46:26 STEP 370 H= 370:00 +CPU= 21.031
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:49:29 STEP 371 H= 371:00 +CPU= 20.030
10:50:05 STEP 372 H= 372:00 +CPU= 36.476
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:52:46 STEP 373 H= 373:00 +CPU= 20.018
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:55:00 STEP 374 H= 374:00 +CPU= 21.920
10:55:32 STEP 375 H= 375:00 +CPU= 32.379
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:58:45 STEP 376 H= 376:00 +CPU= 21.512
10:59:05 STEP 377 H= 377:00 +CPU= 20.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:02:31 STEP 378 H= 378:00 +CPU= 33.178
11:02:51 STEP 379 H= 379:00 +CPU= 20.009
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:05:14 STEP 380 H= 380:00 +CPU= 21.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:08:18 STEP 381 H= 381:00 +CPU= 32.442
11:08:40 STEP 382 H= 382:00 +CPU= 21.925
11:09:01 STEP 383 H= 383:00 +CPU= 20.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:12:20 STEP 384 H= 384:00 +CPU= 37.829
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345_15.zip
Uploading the intermediate file: upload_file_15.zip
11:12:39 STEP 385 H= 385:00 +CPU= 18.361
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:15:10 STEP 386 H= 386:00 +CPU= 21.025
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:17:44 STEP 387 H= 387:00 +CPU= 32.684
11:18:05 STEP 388 H= 388:00 +CPU= 21.447
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:21:27 STEP 389 H= 389:00 +CPU= 20.658
11:22:01 STEP 390 H= 390:00 +CPU= 33.396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:25:13 STEP 391 H= 391:00 +CPU= 20.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:28:05 STEP 392 H= 392:00 +CPU= 21.071
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:30:49 STEP 393 H= 393:00 +CPU= 33.068
11:31:11 STEP 394 H= 394:00 +CPU= 21.400
11:31:31 STEP 395 H= 395:00 +CPU= 19.633
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:37:50 STEP 396 H= 396:00 +CPU= 36.264
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:39:53 STEP 397 H= 397:00 +CPU= 20.597
11:40:14 STEP 398 H= 398:00 +CPU= 21.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:42:49 STEP 399 H= 399:00 +CPU= 32.846
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:46:22 STEP 400 H= 400:00 +CPU= 21.192
11:46:43 STEP 401 H= 401:00 +CPU= 20.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0769
(argv4) batchid: 968
(argv5) wuid: 12185413
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9640ea6f4ab22626b74f75aba07c6258 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0769_1999050100_123_968_12185413.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0769_1999050100_123_968_12185413.zip
ic_ancil_file: ic_ancil_12185413
ifsdata_file: ifsdata_12185413
climate_data_file: clim_data_12185413
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6a225ec94cbfa70f94e23bab6efd20c4 to: /var/lib/boinc-client/slots/2/ic_ancil_12185413.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12185413.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_891823936fdbe8ce5d490b475ffc770b to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12185413.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12185413.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12185413.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12185413.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12185413.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12185413.xml
-- Model is restarting --
Adjusting last_iter, 402, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12185413.xml
last_cpu_time: 12255
upload_file_number: 16
last_iter: 383
last_upload: 1382400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345
The child process has been launched with process id: 2240037
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2240037:2240037] [20230115:115233:1673751153:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:55:14 STEP 384 H= 384:00 +CPU= 37.307
11:55:35 STEP 385 H= 385:00 +CPU= 20.717
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:57:47 STEP 386 H= 386:00 +CPU= 20.879
11:58:19 STEP 387 H= 387:00 +CPU= 32.219
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:01:11 STEP 388 H= 388:00 +CPU= 21.143
12:01:32 STEP 389 H= 389:00 +CPU= 20.355
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:05:48 STEP 390 H= 390:00 +CPU= 34.492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:09:20 STEP 391 H= 391:00 +CPU= 20.876
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:12:53 STEP 392 H= 392:00 +CPU= 20.115
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:16:19 STEP 393 H= 393:00 +CPU= 34.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:19:22 STEP 394 H= 394:00 +CPU= 21.868
12:19:43 STEP 395 H= 395:00 +CPU= 20.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:23:21 STEP 396 H= 396:00 +CPU= 36.020
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000396
12:23:41 STEP 397 H= 397:00 +CPU= 19.973
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:27:12 STEP 398 H= 398:00 +CPU= 20.580
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:30:27 STEP 399 H= 399:00 +CPU= 33.047
12:30:48 STEP 400 H= 400:00 +CPU= 20.758
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:34:08 STEP 401 H= 401:00 +CPU= 20.412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:36:23 STEP 402 H= 402:00 +CPU= 33.084
12:36:42 STEP 403 H= 403:00 +CPU= 19.609
12:37:02 STEP 404 H= 404:00 +CPU= 20.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:22:54 STEP 405 H= 405:00 +CPU= 32.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:26:36 STEP 406 H= 406:00 +CPU= 20.605
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:29:08 STEP 407 H= 407:00 +CPU= 20.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:32:06 STEP 408 H= 408:00 +CPU= 37.855
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345_16.zip
Uploading the intermediate file: upload_file_16.zip
22:32:25 STEP 409 H= 409:00 +CPU= 18.779
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0769
(argv4) batchid: 968
(argv5) wuid: 12185413
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9640ea6f4ab22626b74f75aba07c6258 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0769_1999050100_123_968_12185413.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0769_1999050100_123_968_12185413.zip
ic_ancil_file: ic_ancil_12185413
ifsdata_file: ifsdata_12185413
climate_data_file: clim_data_12185413
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6a225ec94cbfa70f94e23bab6efd20c4 to: /var/lib/boinc-client/slots/2/ic_ancil_12185413.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12185413.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_891823936fdbe8ce5d490b475ffc770b to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12185413.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12185413.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12185413.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12185413.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12185413.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12185413.xml
-- Model is restarting --
Adjusting last_iter, 409, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12185413.xml
last_cpu_time: 12901
upload_file_number: 17
last_iter: 407
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345
The child process has been launched with process id: 2361335
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2361335:2361335] [20230115:223708:1673789828:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:39:49 STEP 408 H= 408:00 +CPU= 37.180
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:41:41 STEP 409 H= 409:00 +CPU= 20.395
22:42:01 STEP 410 H= 410:00 +CPU= 19.954
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:53 STEP 411 H= 411:00 +CPU= 31.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:46:55 STEP 412 H= 412:00 +CPU= 21.119
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:48:55 STEP 413 H= 413:00 +CPU= 20.329
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:09 STEP 414 H= 414:00 +CPU= 32.564
22:51:29 STEP 415 H= 415:00 +CPU= 19.870
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:52 STEP 416 H= 416:00 +CPU= 21.109
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:56:15 STEP 417 H= 417:00 +CPU= 31.929
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:58:38 STEP 418 H= 418:00 +CPU= 22.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:00:40 STEP 419 H= 419:00 +CPU= 20.234
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:02:57 STEP 420 H= 420:00 +CPU= 35.618
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000420
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:05:06 STEP 421 H= 421:00 +CPU= 19.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:06:59 STEP 422 H= 422:00 +CPU= 20.842
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:09:23 STEP 423 H= 423:00 +CPU= 32.652
23:09:44 STEP 424 H= 424:00 +CPU= 21.198
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:11:37 STEP 425 H= 425:00 +CPU= 20.777
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:15:23 STEP 426 H= 426:00 +CPU= 31.896
23:15:43 STEP 427 H= 427:00 +CPU= 19.957
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:18:15 STEP 428 H= 428:00 +CPU= 20.089
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:20:28 STEP 429 H= 429:00 +CPU= 32.957
23:20:49 STEP 430 H= 430:00 +CPU= 21.239
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:23:22 STEP 431 H= 431:00 +CPU= 21.779
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:26:00 STEP 432 H= 432:00 +CPU= 37.497
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185413/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0769_1999050100_123_968_12185413_0_r701166345_17.zip
Uploading the intermediate file: upload_file_17.zip
23:26:18 STEP 433 H= 433:00 +CPU= 18.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:28:37 STEP 434 H= 434:00 +CPU= 20.356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:31:21 STEP 435 H= 435:00 +CPU= 31.958
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:33:23 STEP 436 H= 436:00 +CPU= 20.916
23:33:44 STEP 437 H= 437:00 +CPU= 20.651
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:36:18 STEP 438 H= 438:00 +CPU= 33.153
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:38:30 STEP 439 H= 439:00 +CPU= 20.457
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:40:43 STEP 440 H= 440:00 +CPU= 21.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:43:07 STEP 441 H= 441:00 +CPU= 33.119
23:43:28 STEP 442 H= 442:00 +CPU= 20.699
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:45:18 STEP 443 H= 443:00 +CPU= 19.764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:48:14 STEP 444 H= 444:00 +CPU= 35.064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:56 STEP 445 H= 445:00 +CPU= 19.779
23:50:17 STEP 446 H= 446:00 +CPU= 21.016
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:31 STEP 447 H= 447:00 +CPU= 32.002
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:24 STEP 448 H= 448:00 +CPU= 22.014
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:56:36 STEP 449 H= 449:00 +CPU= 20.790
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
