<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
13:27:01 STEP 460 H= 460:00 +CPU= 20.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:28:33 STEP 461 H= 461:00 +CPU= 20.572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:30:27 STEP 462 H= 462:00 +CPU= 33.337
13:30:48 STEP 463 H= 463:00 +CPU= 20.783
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:32:11 STEP 464 H= 464:00 +CPU= 20.957
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:33:56 STEP 465 H= 465:00 +CPU= 33.028
13:34:17 STEP 466 H= 466:00 +CPU= 21.152
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:35:48 STEP 467 H= 467:00 +CPU= 20.382
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:37:25 STEP 468 H= 468:00 +CPU= 36.231
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000468
13:37:45 STEP 469 H= 469:00 +CPU= 20.198
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:39:17 STEP 470 H= 470:00 +CPU= 21.426
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:41:01 STEP 471 H= 471:00 +CPU= 32.656
13:41:22 STEP 472 H= 472:00 +CPU= 20.844
13:41:42 STEP 473 H= 473:00 +CPU= 20.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:43:37 STEP 474 H= 474:00 +CPU= 33.489
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:45:47 STEP 475 H= 475:00 +CPU= 19.586
13:46:09 STEP 476 H= 476:00 +CPU= 21.740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:48:03 STEP 477 H= 477:00 +CPU= 32.791
13:48:24 STEP 478 H= 478:00 +CPU= 20.828
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:49:57 STEP 479 H= 479:00 +CPU= 21.150
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:52:07 STEP 480 H= 480:00 +CPU= 39.336
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212_19.zip
Uploading the intermediate file: upload_file_19.zip
13:52:28 STEP 481 H= 481:00 +CPU= 20.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:54:08 STEP 482 H= 482:00 +CPU= 21.430
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:55:54 STEP 483 H= 483:00 +CPU= 33.804
13:56:16 STEP 484 H= 484:00 +CPU= 21.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:57:48 STEP 485 H= 485:00 +CPU= 20.712
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:59:42 STEP 486 H= 486:00 +CPU= 33.505
14:00:03 STEP 487 H= 487:00 +CPU= 20.197
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:01:25 STEP 488 H= 488:00 +CPU= 21.123
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:03:08 STEP 489 H= 489:00 +CPU= 32.967
14:03:29 STEP 490 H= 490:00 +CPU= 20.920
14:03:50 STEP 491 H= 491:00 +CPU= 20.726
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.584] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4035MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4035 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : RADDRV
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : RADINTG
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153853:1674110333:8234.587] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-RADINTG-INPUT LOOP5 (1209)
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153854:1674110334:8235.587] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:p500:1:1:3285950:3285950] [20230119:153854:1674110334:8235.587] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 00000000012973C6 radintg_ 1305 radintg.F90
oifs_43r3_model.e 0000000001270124 raddrv_ 537 raddrv.F90
oifs_43r3_model.e 0000000000BFD839 gp_model_ 482 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0211
(argv4) batchid: 984
(argv5) wuid: 12200855
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4aa1cc1d1744a75323ae874788fc5132 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0211_2015050100_123_984_12200855.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0211_2015050100_123_984_12200855.zip
ic_ancil_file: ic_ancil_12200855
ifsdata_file: ifsdata_12200855
climate_data_file: clim_data_12200855
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_08e5ec787aa5a70a924ddc2961dca023 to: /var/lib/boinc-client/slots/4/ic_ancil_12200855.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12200855.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e9dbc99b147e4d55460f980200cf3fd3 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12200855.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12200855.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12200855.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12200855.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12200855.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12200855.xml
-- Model is restarting --
Adjusting last_iter, 492, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12200855.xml
last_cpu_time: 13925
upload_file_number: 20
last_iter: 479
last_upload: 1728000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212
The child process has been launched with process id: 52424
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52424:52424] [20230119:195022:1674125422:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:28:22 STEP 480 H= 480:00 +CPU= 39.663
20:28:46 STEP 481 H= 481:00 +CPU= 22.991
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:31:31 STEP 482 H= 482:00 +CPU= 23.300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:47:59 STEP 483 H= 483:00 +CPU= 35.505
20:48:21 STEP 484 H= 484:00 +CPU= 21.781
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:50:33 STEP 485 H= 485:00 +CPU= 20.749
20:51:10 STEP 486 H= 486:00 +CPU= 36.158
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:12:24 STEP 487 H= 487:00 +CPU= 21.356
21:12:44 STEP 488 H= 488:00 +CPU= 20.363
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:14:49 STEP 489 H= 489:00 +CPU= 31.933
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:16:32 STEP 490 H= 490:00 +CPU= 21.142
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:18:04 STEP 491 H= 491:00 +CPU= 19.935
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:19:40 STEP 492 H= 492:00 +CPU= 34.939
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:21:03 STEP 493 H= 493:00 +CPU= 20.913
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:22:46 STEP 494 H= 494:00 +CPU= 21.146
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:24:20 STEP 495 H= 495:00 +CPU= 32.238
21:24:40 STEP 496 H= 496:00 +CPU= 20.690
21:25:00 STEP 497 H= 497:00 +CPU= 19.845
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:26:53 STEP 498 H= 498:00 +CPU= 32.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:28:45 STEP 499 H= 499:00 +CPU= 20.580
21:29:06 STEP 500 H= 500:00 +CPU= 20.756
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:30:58 STEP 501 H= 501:00 +CPU= 31.621
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:32:30 STEP 502 H= 502:00 +CPU= 20.696
21:32:51 STEP 503 H= 503:00 +CPU= 21.125
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:35:51 STEP 504 H= 504:00 +CPU= 38.046
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212_20.zip
Uploading the intermediate file: upload_file_20.zip
21:36:12 STEP 505 H= 505:00 +CPU= 20.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:37:42 STEP 506 H= 506:00 +CPU= 20.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:39:44 STEP 507 H= 507:00 +CPU= 31.320
21:40:05 STEP 508 H= 508:00 +CPU= 21.323
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:41:38 STEP 509 H= 509:00 +CPU= 20.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:43:31 STEP 510 H= 510:00 +CPU= 33.449
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:44:52 STEP 511 H= 511:00 +CPU= 21.082
21:45:13 STEP 512 H= 512:00 +CPU= 20.907
21:45:44 STEP 513 H= 513:00 +CPU= 30.975
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:47:36 STEP 514 H= 514:00 +CPU= 21.234
21:47:56 STEP 515 H= 515:00 +CPU= 19.787
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:49:41 STEP 516 H= 516:00 +CPU= 34.536
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:51:14 STEP 517 H= 517:00 +CPU= 20.437
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:52:35 STEP 518 H= 518:00 +CPU= 21.104
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0211
(argv4) batchid: 984
(argv5) wuid: 12200855
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4aa1cc1d1744a75323ae874788fc5132 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0211_2015050100_123_984_12200855.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0211_2015050100_123_984_12200855.zip
ic_ancil_file: ic_ancil_12200855
ifsdata_file: ifsdata_12200855
climate_data_file: clim_data_12200855
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_08e5ec787aa5a70a924ddc2961dca023 to: /var/lib/boinc-client/slots/4/ic_ancil_12200855.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12200855.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e9dbc99b147e4d55460f980200cf3fd3 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12200855.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12200855.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12200855.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12200855.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12200855.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12200855.xml
-- Model is restarting --
Adjusting last_iter, 519, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12200855.xml
last_cpu_time: 14937
upload_file_number: 21
last_iter: 503
last_upload: 1814400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212
The child process has been launched with process id: 73679
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73679:73679] [20230119:215417:1674132857:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:56:06 STEP 504 H= 504:00 +CPU= 36.790
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:57:18 STEP 505 H= 505:00 +CPU= 19.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:58:41 STEP 506 H= 506:00 +CPU= 20.921
21:59:13 STEP 507 H= 507:00 +CPU= 31.683
21:59:33 STEP 508 H= 508:00 +CPU= 19.479
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:01:14 STEP 509 H= 509:00 +CPU= 20.054
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:03:08 STEP 510 H= 510:00 +CPU= 33.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:04:30 STEP 511 H= 511:00 +CPU= 20.191
22:04:51 STEP 512 H= 512:00 +CPU= 20.344
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:06:34 STEP 513 H= 513:00 +CPU= 31.639
22:06:55 STEP 514 H= 514:00 +CPU= 21.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:08:46 STEP 515 H= 515:00 +CPU= 20.209
22:09:22 STEP 516 H= 516:00 +CPU= 35.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000516
22:11:13 STEP 517 H= 517:00 +CPU= 20.316
22:11:34 STEP 518 H= 518:00 +CPU= 20.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:13:58 STEP 519 H= 519:00 +CPU= 32.833
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:15:29 STEP 520 H= 520:00 +CPU= 21.130
22:15:50 STEP 521 H= 521:00 +CPU= 20.517
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:17:23 STEP 522 H= 522:00 +CPU= 33.194
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:18:56 STEP 523 H= 523:00 +CPU= 20.606
22:19:17 STEP 524 H= 524:00 +CPU= 21.202
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:21:20 STEP 525 H= 525:00 +CPU= 32.662
22:21:41 STEP 526 H= 526:00 +CPU= 21.123
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:23:13 STEP 527 H= 527:00 +CPU= 20.817
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:25:02 STEP 528 H= 528:00 +CPU= 37.964
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212_21.zip
Uploading the intermediate file: upload_file_21.zip
22:25:22 STEP 529 H= 529:00 +CPU= 20.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:27:24 STEP 530 H= 530:00 +CPU= 20.949
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:28:58 STEP 531 H= 531:00 +CPU= 32.278
22:29:18 STEP 532 H= 532:00 +CPU= 20.377
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:31:00 STEP 533 H= 533:00 +CPU= 20.390
22:31:32 STEP 534 H= 534:00 +CPU= 32.258
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:33:05 STEP 535 H= 535:00 +CPU= 20.705
22:33:26 STEP 536 H= 536:00 +CPU= 20.689
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:30 STEP 537 H= 537:00 +CPU= 32.995
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:37:21 STEP 538 H= 538:00 +CPU= 20.383
22:37:42 STEP 539 H= 539:00 +CPU= 21.008
22:38:17 STEP 540 H= 540:00 +CPU= 34.988
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000540
22:40:09 STEP 541 H= 541:00 +CPU= 20.931
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:41:42 STEP 542 H= 542:00 +CPU= 20.940
22:42:14 STEP 543 H= 543:00 +CPU= 31.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:43:54 STEP 544 H= 544:00 +CPU= 20.477
22:44:14 STEP 545 H= 545:00 +CPU= 19.695
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:46:29 STEP 546 H= 546:00 +CPU= 33.724
22:46:49 STEP 547 H= 547:00 +CPU= 19.779
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:48:22 STEP 548 H= 548:00 +CPU= 20.875
22:48:54 STEP 549 H= 549:00 +CPU= 31.571
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:50:55 STEP 550 H= 550:00 +CPU= 20.233
22:51:15 STEP 551 H= 551:00 +CPU= 20.477
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:14 STEP 552 H= 552:00 +CPU= 37.371
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212_22.zip
Uploading the intermediate file: upload_file_22.zip
22:53:35 STEP 553 H= 553:00 +CPU= 20.857
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:55:07 STEP 554 H= 554:00 +CPU= 20.791
22:55:39 STEP 555 H= 555:00 +CPU= 32.350
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:57:31 STEP 556 H= 556:00 +CPU= 20.958
22:57:51 STEP 557 H= 557:00 +CPU= 19.933
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:59:54 STEP 558 H= 558:00 +CPU= 31.893
23:00:15 STEP 559 H= 559:00 +CPU= 20.501
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:01:48 STEP 560 H= 560:00 +CPU= 20.697
23:02:20 STEP 561 H= 561:00 +CPU= 31.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:04:32 STEP 562 H= 562:00 +CPU= 20.969
23:04:52 STEP 563 H= 563:00 +CPU= 20.391
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:06:39 STEP 564 H= 564:00 +CPU= 36.291
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000564
23:06:59 STEP 565 H= 565:00 +CPU= 20.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:08:40 STEP 566 H= 566:00 +CPU= 20.174
23:09:12 STEP 567 H= 567:00 +CPU= 32.176
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:10:54 STEP 568 H= 568:00 +CPU= 20.274
23:11:14 STEP 569 H= 569:00 +CPU= 20.232
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:13:09 STEP 570 H= 570:00 +CPU= 34.178
23:13:29 STEP 571 H= 571:00 +CPU= 19.770
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:15:01 STEP 572 H= 572:00 +CPU= 20.609
23:15:33 STEP 573 H= 573:00 +CPU= 32.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:17:25 STEP 574 H= 574:00 +CPU= 20.726
23:17:45 STEP 575 H= 575:00 +CPU= 19.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:19:32 STEP 576 H= 576:00 +CPU= 37.295
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212_23.zip
Uploading the intermediate file: upload_file_23.zip
23:19:53 STEP 577 H= 577:00 +CPU= 20.593
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:21:36 STEP 578 H= 578:00 +CPU= 20.962
23:22:08 STEP 579 H= 579:00 +CPU= 31.972
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:23:50 STEP 580 H= 580:00 +CPU= 20.845
23:24:10 STEP 581 H= 581:00 +CPU= 19.928
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:25:44 STEP 582 H= 582:00 +CPU= 33.097
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:27:25 STEP 583 H= 583:00 +CPU= 20.078
23:27:46 STEP 584 H= 584:00 +CPU= 20.859
23:28:18 STEP 585 H= 585:00 +CPU= 31.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:30:09 STEP 586 H= 586:00 +CPU= 20.271
23:30:29 STEP 587 H= 587:00 +CPU= 19.984
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:32:07 STEP 588 H= 588:00 +CPU= 35.913
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000588
23:33:29 STEP 589 H= 589:00 +CPU= 20.817
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:34:52 STEP 590 H= 590:00 +CPU= 21.029
23:35:23 STEP 591 H= 591:00 +CPU= 31.073
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:37:04 STEP 592 H= 592:00 +CPU= 20.259
23:37:24 STEP 593 H= 593:00 +CPU= 20.246
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:39:08 STEP 594 H= 594:00 +CPU= 32.376
23:39:28 STEP 595 H= 595:00 +CPU= 19.877
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:41:10 STEP 596 H= 596:00 +CPU= 20.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:42:43 STEP 597 H= 597:00 +CPU= 31.922
23:43:05 STEP 598 H= 598:00 +CPU= 21.519
23:43:25 STEP 599 H= 599:00 +CPU= 20.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:45:24 STEP 600 H= 600:00 +CPU= 38.542
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212_24.zip
Uploading the intermediate file: upload_file_24.zip
23:46:35 STEP 601 H= 601:00 +CPU= 20.118
23:46:55 STEP 602 H= 602:00 +CPU= 20.346
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:19 STEP 603 H= 603:00 +CPU= 32.330
23:49:40 STEP 604 H= 604:00 +CPU= 20.774
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:51:02 STEP 605 H= 605:00 +CPU= 19.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:17 STEP 606 H= 606:00 +CPU= 33.852
23:52:36 STEP 607 H= 607:00 +CPU= 19.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:09 STEP 608 H= 608:00 +CPU= 21.880
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0211
(argv4) batchid: 984
(argv5) wuid: 12200855
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200855
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4aa1cc1d1744a75323ae874788fc5132 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0211_2015050100_123_984_12200855.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0211_2015050100_123_984_12200855.zip
ic_ancil_file: ic_ancil_12200855
ifsdata_file: ifsdata_12200855
climate_data_file: clim_data_12200855
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_08e5ec787aa5a70a924ddc2961dca023 to: /var/lib/boinc-client/slots/4/ic_ancil_12200855.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12200855.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e9dbc99b147e4d55460f980200cf3fd3 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12200855.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12200855.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12200855.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12200855.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12200855.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12200855.xml
-- Model is restarting --
Adjusting last_iter, 609, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12200855.xml
last_cpu_time: 17556
upload_file_number: 25
last_iter: 599
last_upload: 2160000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0211_2015050100_123_984_12200855_0_r2105648212
The child process has been launched with process id: 91365
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91365:91365] [20230119:235539:1674140139:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:57:19 STEP 600 H= 600:00 +CPU= 35.826
23:57:40 STEP 601 H= 601:00 +CPU= 20.793
23:58:00 STEP 602 H= 602:00 +CPU= 20.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:59:53 STEP 603 H= 603:00 +CPU= 32.267
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:01:16 STEP 604 H= 604:00 +CPU= 21.822
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:02:38 STEP 605 H= 605:00 +CPU= 20.898
00:03:11 STEP 606 H= 606:00 +CPU= 32.092
00:03:31 STEP 607 H= 607:00 +CPU= 19.907
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:05:23 STEP 608 H= 608:00 +CPU= 20.907
00:05:53 STEP 609 H= 609:00 +CPU= 30.684
00:06:13 STEP 610 H= 610:00 +CPU= 19.938
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:08:06 STEP 611 H= 611:00 +CPU= 20.804
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:09:52 STEP 612 H= 612:00 +CPU= 34.781
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000612
00:10:14 STEP 613 H= 613:00 +CPU= 21.413
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:11:46 STEP 614 H= 614:00 +CPU= 21.337
00:12:19 STEP 615 H= 615:00 +CPU= 32.897
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:14:02 STEP 616 H= 616:00 +CPU= 21.731
00:14:23 STEP 617 H= 617:00 +CPU= 20.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:16:17 STEP 618 H= 618:00 +CPU= 32.311
00:16:37 STEP 619 H= 619:00 +CPU= 19.708
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:18:07 STEP 620 H= 620:00 +CPU= 20.259
00:18:40 STEP 621 H= 621:00 +CPU= 32.286
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:20:22 STEP 622 H= 622:00 +CPU= 20.937
00:20:44 STEP 623 H= 623:00 +CPU= 21.960
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process
</stderr_txt>
]]>
