<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
HOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2219639:2219639] [20230115:093549:1673742949:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:38:51 STEP 288 H= 288:00 +CPU= 37.197
09:39:13 STEP 289 H= 289:00 +CPU= 21.704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:41:45 STEP 290 H= 290:00 +CPU= 20.821
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:44:09 STEP 291 H= 291:00 +CPU= 33.115
09:44:31 STEP 292 H= 292:00 +CPU= 21.384
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:47:23 STEP 293 H= 293:00 +CPU= 20.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:49:37 STEP 294 H= 294:00 +CPU= 33.660
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:52:08 STEP 295 H= 295:00 +CPU= 20.316
09:52:30 STEP 296 H= 296:00 +CPU= 21.579
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:55:24 STEP 297 H= 297:00 +CPU= 32.313
09:55:45 STEP 298 H= 298:00 +CPU= 21.813
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:58:07 STEP 299 H= 299:00 +CPU= 20.607
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:00:55 STEP 300 H= 300:00 +CPU= 36.210
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:02:47 STEP 301 H= 301:00 +CPU= 20.728
10:03:08 STEP 302 H= 302:00 +CPU= 21.186
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:05:42 STEP 303 H= 303:00 +CPU= 33.017
10:06:03 STEP 304 H= 304:00 +CPU= 20.853
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:08:15 STEP 305 H= 305:00 +CPU= 20.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:10:49 STEP 306 H= 306:00 +CPU= 33.164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:12:40 STEP 307 H= 307:00 +CPU= 20.553
10:13:02 STEP 308 H= 308:00 +CPU= 21.827
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:15:16 STEP 309 H= 309:00 +CPU= 32.671
10:15:37 STEP 310 H= 310:00 +CPU= 20.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:18:29 STEP 311 H= 311:00 +CPU= 20.984
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:20:59 STEP 312 H= 312:00 +CPU= 38.760
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258_12.zip
Uploading the intermediate file: upload_file_12.zip
10:21:18 STEP 313 H= 313:00 +CPU= 18.898
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:23:01 STEP 314 H= 314:00 +CPU= 20.370
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:25:35 STEP 315 H= 315:00 +CPU= 32.944
10:25:57 STEP 316 H= 316:00 +CPU= 21.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:28:29 STEP 317 H= 317:00 +CPU= 20.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:31:03 STEP 318 H= 318:00 +CPU= 34.093
10:31:24 STEP 319 H= 319:00 +CPU= 20.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:33:35 STEP 320 H= 320:00 +CPU= 20.736
10:34:09 STEP 321 H= 321:00 +CPU= 33.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:36:42 STEP 322 H= 322:00 +CPU= 21.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:38:54 STEP 323 H= 323:00 +CPU= 20.319
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:40:43 STEP 324 H= 324:00 +CPU= 36.682
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000324
10:41:03 STEP 325 H= 325:00 +CPU= 20.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:43:26 STEP 326 H= 326:00 +CPU= 21.130
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:46:00 STEP 327 H= 327:00 +CPU= 32.484
10:46:21 STEP 328 H= 328:00 +CPU= 21.443
10:46:41 STEP 329 H= 329:00 +CPU= 20.099
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:49:57 STEP 330 H= 330:00 +CPU= 33.237
10:50:17 STEP 331 H= 331:00 +CPU= 19.995
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:52:59 STEP 332 H= 332:00 +CPU= 21.118
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:55:24 STEP 333 H= 333:00 +CPU= 33.022
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
10:58:37 STEP 334 H= 334:00 +CPU= 20.747
10:58:58 STEP 335 H= 335:00 +CPU= 21.135
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:02:28 STEP 336 H= 336:00 +CPU= 37.592
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258_13.zip
Uploading the intermediate file: upload_file_13.zip
11:02:48 STEP 337 H= 337:00 +CPU= 20.509
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:05:11 STEP 338 H= 338:00 +CPU= 21.145
11:05:43 STEP 339 H= 339:00 +CPU= 31.891
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:08:35 STEP 340 H= 340:00 +CPU= 21.298
11:08:56 STEP 341 H= 341:00 +CPU= 20.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:12:10 STEP 342 H= 342:00 +CPU= 32.906
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:15:01 STEP 343 H= 343:00 +CPU= 20.481
11:15:22 STEP 344 H= 344:00 +CPU= 21.390
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:17:56 STEP 345 H= 345:00 +CPU= 32.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:21:19 STEP 346 H= 346:00 +CPU= 21.210
11:21:40 STEP 347 H= 347:00 +CPU= 20.807
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:25:07 STEP 348 H= 348:00 +CPU= 36.347
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000348
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:27:58 STEP 349 H= 349:00 +CPU= 20.095
11:28:20 STEP 350 H= 350:00 +CPU= 21.431
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:31:04 STEP 351 H= 351:00 +CPU= 32.554
11:31:24 STEP 352 H= 352:00 +CPU= 20.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:35:07 STEP 353 H= 353:00 +CPU= 20.708
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:38:02 STEP 354 H= 354:00 +CPU= 33.198
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2011050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0408
(argv4) batchid: 980
(argv5) wuid: 12197052
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/7
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ead6c1438350aebc10df2054866b5454 to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0408_2011050100_123_980_12197052.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0408_2011050100_123_980_12197052.zip
ic_ancil_file: ic_ancil_12197052
ifsdata_file: ifsdata_12197052
climate_data_file: clim_data_12197052
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b142143593a3f9d33a29a6f2007f3389 to: /var/lib/boinc-client/slots/7/ic_ancil_12197052.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/7/ic_ancil_12197052.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_17fffcd6a542cf85591bcecfdc46639f to: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12197052.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12197052.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/7/159l_2/clim_data_12197052.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/7/159l_2/clim_data_12197052.zip
Checking for progress XML file: /var/lib/boinc-client/slots/7/progress_file_12197052.xml
Opened progress file ok : /var/lib/boinc-client/slots/7/progress_file_12197052.xml
-- Model is restarting --
Adjusting last_iter, 355, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/7/progress_file_12197052.xml
last_cpu_time: 10792
upload_file_number: 14
last_iter: 335
last_upload: 1209600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258
The child process has been launched with process id: 2238277
Executing the command: /var/lib/boinc-client/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2238277:2238277] [20230115:114222:1673750542:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:46:15 STEP 336 H= 336:00 +CPU= 37.058
11:46:36 STEP 337 H= 337:00 +CPU= 20.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:49:27 STEP 338 H= 338:00 +CPU= 20.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:52:42 STEP 339 H= 339:00 +CPU= 32.685
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:55:05 STEP 340 H= 340:00 +CPU= 21.329
11:55:25 STEP 341 H= 341:00 +CPU= 20.274
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:57:50 STEP 342 H= 342:00 +CPU= 33.512
11:58:10 STEP 343 H= 343:00 +CPU= 20.031
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:01:02 STEP 344 H= 344:00 +CPU= 21.093
12:01:36 STEP 345 H= 345:00 +CPU= 33.129
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:05:39 STEP 346 H= 346:00 +CPU= 21.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:09:09 STEP 347 H= 347:00 +CPU= 20.529
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:12:57 STEP 348 H= 348:00 +CPU= 36.455
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000348
12:13:19 STEP 349 H= 349:00 +CPU= 21.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:16:32 STEP 350 H= 350:00 +CPU= 21.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:19:47 STEP 351 H= 351:00 +CPU= 32.801
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:23:09 STEP 352 H= 352:00 +CPU= 20.910
12:23:30 STEP 353 H= 353:00 +CPU= 20.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:27:15 STEP 354 H= 354:00 +CPU= 32.855
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:30:17 STEP 355 H= 355:00 +CPU= 20.536
12:30:39 STEP 356 H= 356:00 +CPU= 21.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:34:13 STEP 357 H= 357:00 +CPU= 33.485
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:36:15 STEP 358 H= 358:00 +CPU= 21.401
12:36:36 STEP 359 H= 359:00 +CPU= 20.669
12:37:13 STEP 360 H= 360:00 +CPU= 37.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258_14.zip
Uploading the intermediate file: upload_file_14.zip
22:22:54 STEP 361 H= 361:00 +CPU= 20.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:26:36 STEP 362 H= 362:00 +CPU= 20.555
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2011050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0408
(argv4) batchid: 980
(argv5) wuid: 12197052
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/7
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ead6c1438350aebc10df2054866b5454 to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0408_2011050100_123_980_12197052.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0408_2011050100_123_980_12197052.zip
ic_ancil_file: ic_ancil_12197052
ifsdata_file: ifsdata_12197052
climate_data_file: clim_data_12197052
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b142143593a3f9d33a29a6f2007f3389 to: /var/lib/boinc-client/slots/7/ic_ancil_12197052.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/7/ic_ancil_12197052.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_17fffcd6a542cf85591bcecfdc46639f to: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12197052.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12197052.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/7/159l_2/clim_data_12197052.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/7/159l_2/clim_data_12197052.zip
Checking for progress XML file: /var/lib/boinc-client/slots/7/progress_file_12197052.xml
Opened progress file ok : /var/lib/boinc-client/slots/7/progress_file_12197052.xml
-- Model is restarting --
Adjusting last_iter, 363, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/7/progress_file_12197052.xml
last_cpu_time: 11497
upload_file_number: 15
last_iter: 359
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258
The child process has been launched with process id: 2360638
Executing the command: /var/lib/boinc-client/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2360638:2360638] [20230115:223158:1673789518:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:01 STEP 360 H= 360:00 +CPU= 36.654
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:37:22 STEP 361 H= 361:00 +CPU= 20.192
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:39:44 STEP 362 H= 362:00 +CPU= 20.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:41:47 STEP 363 H= 363:00 +CPU= 32.011
22:42:07 STEP 364 H= 364:00 +CPU= 20.000
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:48 STEP 365 H= 365:00 +CPU= 19.395
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:47:01 STEP 366 H= 366:00 +CPU= 32.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:49:03 STEP 367 H= 367:00 +CPU= 20.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:05 STEP 368 H= 368:00 +CPU= 20.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:40 STEP 369 H= 369:00 +CPU= 32.757
22:54:01 STEP 370 H= 370:00 +CPU= 21.190
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:56:12 STEP 371 H= 371:00 +CPU= 19.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:58:50 STEP 372 H= 372:00 +CPU= 37.005
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:00:52 STEP 373 H= 373:00 +CPU= 20.434
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:02:53 STEP 374 H= 374:00 +CPU= 20.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:05:17 STEP 375 H= 375:00 +CPU= 32.310
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:07:10 STEP 376 H= 376:00 +CPU= 20.936
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:09:20 STEP 377 H= 377:00 +CPU= 19.663
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:11:26 STEP 378 H= 378:00 +CPU= 34.008
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:13:29 STEP 379 H= 379:00 +CPU= 21.174
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:15:22 STEP 380 H= 380:00 +CPU= 20.767
23:15:55 STEP 381 H= 381:00 +CPU= 32.323
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:18:28 STEP 382 H= 382:00 +CPU= 21.180
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:20:30 STEP 383 H= 383:00 +CPU= 20.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:23:20 STEP 384 H= 384:00 +CPU= 37.945
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258_15.zip
Uploading the intermediate file: upload_file_15.zip
23:23:40 STEP 385 H= 385:00 +CPU= 19.856
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:25:48 STEP 386 H= 386:00 +CPU= 18.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:28:33 STEP 387 H= 387:00 +CPU= 32.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:31:05 STEP 388 H= 388:00 +CPU= 21.071
23:31:25 STEP 389 H= 389:00 +CPU= 19.822
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:33:38 STEP 390 H= 390:00 +CPU= 33.640
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:36:01 STEP 391 H= 391:00 +CPU= 20.965
23:36:22 STEP 392 H= 392:00 +CPU= 21.179
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:38:46 STEP 393 H= 393:00 +CPU= 32.766
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:40:58 STEP 394 H= 394:00 +CPU= 20.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:43:10 STEP 395 H= 395:00 +CPU= 20.580
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:45:16 STEP 396 H= 396:00 +CPU= 35.180
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000396
23:45:36 STEP 397 H= 397:00 +CPU= 20.164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:48:19 STEP 398 H= 398:00 +CPU= 20.754
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:50:13 STEP 399 H= 399:00 +CPU= 32.408
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:17 STEP 400 H= 400:00 +CPU= 20.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:09 STEP 401 H= 401:00 +CPU= 20.009
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:56:33 STEP 402 H= 402:00 +CPU= 32.977
23:56:53 STEP 403 H= 403:00 +CPU= 20.274
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:59:04 STEP 404 H= 404:00 +CPU= 19.983
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:01:38 STEP 405 H= 405:00 +CPU= 32.282
00:01:59 STEP 406 H= 406:00 +CPU= 20.829
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:04:50 STEP 407 H= 407:00 +CPU= 20.127
00:05:27 STEP 408 H= 408:00 +CPU= 36.541
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258_16.zip
Uploading the intermediate file: upload_file_16.zip
00:05:44 STEP 409 H= 409:00 +CPU= 17.306
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:07:54 STEP 410 H= 410:00 +CPU= 18.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:10:49 STEP 411 H= 411:00 +CPU= 32.040
00:11:09 STEP 412 H= 412:00 +CPU= 20.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:13:41 STEP 413 H= 413:00 +CPU= 19.419
00:14:13 STEP 414 H= 414:00 +CPU= 32.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:16:34 STEP 415 H= 415:00 +CPU= 19.745
00:16:54 STEP 416 H= 416:00 +CPU= 20.385
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:19:37 STEP 417 H= 417:00 +CPU= 31.560
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:21:39 STEP 418 H= 418:00 +CPU= 20.394
00:21:59 STEP 419 H= 419:00 +CPU= 19.904
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:24:17 STEP 420 H= 420:00 +CPU= 35.446
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000420
00:24:36 STEP 421 H= 421:00 +CPU= 19.393
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:27:08 STEP 422 H= 422:00 +CPU= 20.278
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:29:10 STEP 423 H= 423:00 +CPU= 31.820
00:29:30 STEP 424 H= 424:00 +CPU= 20.166
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:31:31 STEP 425 H= 425:00 +CPU= 19.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:33:54 STEP 426 H= 426:00 +CPU= 32.022
00:34:14 STEP 427 H= 427:00 +CPU= 19.772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:36:06 STEP 428 H= 428:00 +CPU= 20.708
00:36:37 STEP 429 H= 429:00 +CPU= 31.315
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:39:28 STEP 430 H= 430:00 +CPU= 19.804
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:41:50 STEP 431 H= 431:00 +CPU= 20.814
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:43:59 STEP 432 H= 432:00 +CPU= 37.266
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258_17.zip
Uploading the intermediate file: upload_file_17.zip
00:44:19 STEP 433 H= 433:00 +CPU= 19.807
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2011050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0408
(argv4) batchid: 980
(argv5) wuid: 12197052
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/7
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ead6c1438350aebc10df2054866b5454 to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0408_2011050100_123_980_12197052.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0408_2011050100_123_980_12197052.zip
ic_ancil_file: ic_ancil_12197052
ifsdata_file: ifsdata_12197052
climate_data_file: clim_data_12197052
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b142143593a3f9d33a29a6f2007f3389 to: /var/lib/boinc-client/slots/7/ic_ancil_12197052.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/7/ic_ancil_12197052.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_17fffcd6a542cf85591bcecfdc46639f to: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12197052.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12197052.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/7/159l_2/clim_data_12197052.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/7/159l_2/clim_data_12197052.zip
Checking for progress XML file: /var/lib/boinc-client/slots/7/progress_file_12197052.xml
Opened progress file ok : /var/lib/boinc-client/slots/7/progress_file_12197052.xml
-- Model is restarting --
Adjusting last_iter, 433, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/7/progress_file_12197052.xml
last_cpu_time: 13314
upload_file_number: 18
last_iter: 431
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258
The child process has been launched with process id: 2379633
Executing the command: /var/lib/boinc-client/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:2379633:2379633] [20230116:004823:1673797703:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:50:52 STEP 432 H= 432:00 +CPU= 35.652
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:52:23 STEP 433 H= 433:00 +CPU= 19.476
00:52:44 STEP 434 H= 434:00 +CPU= 21.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:55:17 STEP 435 H= 435:00 +CPU= 31.814
00:55:37 STEP 436 H= 436:00 +CPU= 20.298
00:55:57 STEP 437 H= 437:00 +CPU= 20.217
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:00:23 STEP 438 H= 438:00 +CPU= 32.413
01:00:43 STEP 439 H= 439:00 +CPU= 19.729
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:03:25 STEP 440 H= 440:00 +CPU= 20.388
01:03:57 STEP 441 H= 441:00 +CPU= 31.027
01:04:17 STEP 442 H= 442:00 +CPU= 20.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:06:38 STEP 443 H= 443:00 +CPU= 19.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:09:53 STEP 444 H= 444:00 +CPU= 34.602
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000444
01:10:12 STEP 445 H= 445:00 +CPU= 19.733
01:10:33 STEP 446 H= 446:00 +CPU= 20.288
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:13:15 STEP 447 H= 447:00 +CPU= 31.640
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:14:47 STEP 448 H= 448:00 +CPU= 20.214
01:15:06 STEP 449 H= 449:00 +CPU= 19.453
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:18:20 STEP 450 H= 450:00 +CPU= 32.300
01:18:40 STEP 451 H= 451:00 +CPU= 19.884
01:19:00 STEP 452 H= 452:00 +CPU= 20.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:21:44 STEP 453 H= 453:00 +CPU= 31.195
01:22:04 STEP 454 H= 454:00 +CPU= 20.404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:24:44 STEP 455 H= 455:00 +CPU= 19.991
01:25:21 STEP 456 H= 456:00 +CPU= 36.770
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197052/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0408_2011050100_123_980_12197052_1_r1526951258_18.zip
Uploading the intermediate file: upload_file_18.zip
01:25:41 STEP 457 H= 457:00 +CPU= 19.590
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:27:53 STEP 458 H= 458:00 +CPU= 18.587
01:28:25 STEP 459 H= 459:00 +CPU= 32.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:31:37 STEP 460 H= 460:00 +CPU= 20.425
01:31:56 STEP 461 H= 461:00 +CPU= 19.318
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
