<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_075e9f90bca79e48305de72f7272600a to: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
ic_ancil_file: ic_ancil_12170667
ifsdata_file: ifsdata_12170667
climate_data_file: clim_data_12170667
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_58bde664c39c1b869542abb4827bddc4 to: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecf601fa1042c4873537e6e50f79cf64 to: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Checking for progress XML file: /var/lib/boinc/slots/26/progress_file_12170667.xml
Creating progress file: /var/lib/boinc/slots/26/progress_file_12170667.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0023_1985050100_123_954_12170667_0_r290137270
The child process has been launched with process id: 22222
Executing the command: /var/lib/boinc/slots/26/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22222:22222] [20230103:234715:1672764435:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
23:47:46 STEP 0 H= 0:00 +CPU= 28.666
Moving to projects directory: /var/lib/boinc/slots/26/ICMGGhq0f+000000
Moving to projects directory: /var/lib/boinc/slots/26/ICMSHhq0f+000000
Moving to projects directory: /var/lib/boinc/slots/26/ICMUAhq0f+000000
23:48:04 STEP 1 H= 1:00 +CPU= 17.789
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:48:22 STEP 2 H= 2:00 +CPU= 15.591
23:48:47 STEP 3 H= 3:00 +CPU= 25.242
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:05 STEP 4 H= 4:00 +CPU= 15.637
23:49:20 STEP 5 H= 5:00 +CPU= 15.267
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:47 STEP 6 H= 6:00 +CPU= 24.922
23:50:01 STEP 7 H= 7:00 +CPU= 14.183
23:50:16 STEP 8 H= 8:00 +CPU= 14.855
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:50:42 STEP 9 H= 9:00 +CPU= 23.504
23:50:57 STEP 10 H= 10:00 +CPU= 14.816
23:51:11 STEP 11 H= 11:00 +CPU= 14.093
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:51:39 STEP 12 H= 12:00 +CPU= 25.203
Moving to projects directory: /var/lib/boinc/slots/26/ICMGGhq0f+000012
Moving to projects directory: /var/lib/boinc/slots/26/ICMSHhq0f+000012
Moving to projects directory: /var/lib/boinc/slots/26/ICMUAhq0f+000012
23:51:54 STEP 13 H= 13:00 +CPU= 15.519
23:52:10 STEP 14 H= 14:00 +CPU= 15.396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:37 STEP 15 H= 15:00 +CPU= 25.557
23:52:52 STEP 16 H= 16:00 +CPU= 14.662
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:53:10 STEP 17 H= 17:00 +CPU= 15.669
23:53:33 STEP 18 H= 18:00 +CPU= 22.994
23:53:48 STEP 19 H= 19:00 +CPU= 15.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:05 STEP 20 H= 20:00 +CPU= 15.094
23:54:30 STEP 21 H= 21:00 +CPU= 24.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:48 STEP 22 H= 22:00 +CPU= 16.521
23:55:04 STEP 23 H= 23:00 +CPU= 15.799
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:55:37 STEP 24 H= 24:00 +CPU= 30.760
Moving to projects directory: /var/lib/boinc/slots/26/ICMGGhq0f+000024
Moving to projects directory: /var/lib/boinc/slots/26/ICMSHhq0f+000024
Moving to projects directory: /var/lib/boinc/slots/26/ICMUAhq0f+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMGGhq0f+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMSHhq0f+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMUAhq0f+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMGGhq0f+000012
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMSHhq0f+000012
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMUAhq0f+000012
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0023_1985050100_123_954_12170667_0_r290137270_0.zip
Uploading the intermediate file: upload_file_0.zip
23:55:52 STEP 25 H= 25:00 +CPU= 15.251
23:56:07 STEP 26 H= 26:00 +CPU= 14.816
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:56:33 STEP 27 H= 27:00 +CPU= 23.855
23:56:47 STEP 28 H= 28:00 +CPU= 13.784
23:57:02 STEP 29 H= 29:00 +CPU= 14.627
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:57:30 STEP 30 H= 30:00 +CPU= 26.473
23:57:45 STEP 31 H= 31:00 +CPU= 15.073
23:58:04 STEP 32 H= 32:00 +CPU= 17.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:58:32 STEP 33 H= 33:00 +CPU= 26.392
23:58:52 STEP 34 H= 34:00 +CPU= 17.426
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:59:10 STEP 35 H= 35:00 +CPU= 15.796
23:59:38 STEP 36 H= 36:00 +CPU= 28.281
Moving to projects directory: /var/lib/boinc/slots/26/ICMGGhq0f+000036
Moving to projects directory: /var/lib/boinc/slots/26/ICMSHhq0f+000036
Moving to projects directory: /var/lib/boinc/slots/26/ICMUAhq0f+000036
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:59:56 STEP 37 H= 37:00 +CPU= 15.604
00:00:11 STEP 38 H= 38:00 +CPU= 14.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:00:37 STEP 39 H= 39:00 +CPU= 23.891
00:00:53 STEP 40 H= 40:00 +CPU= 15.629
00:01:07 STEP 41 H= 41:00 +CPU= 14.366
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_075e9f90bca79e48305de72f7272600a to: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
ic_ancil_file: ic_ancil_12170667
ifsdata_file: ifsdata_12170667
climate_data_file: clim_data_12170667
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_58bde664c39c1b869542abb4827bddc4 to: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecf601fa1042c4873537e6e50f79cf64 to: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
00:12:01 STEP 42 H= 42:00 +CPU= 28.921
Checking for progress XML file: /var/lib/boinc/slots/26/progress_file_12170667.xml
Opened progress file ok : /var/lib/boinc/slots/26/progress_file_12170667.xml
-- Model is restarting --
Adjusting last_iter, 42, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/26/progress_file_12170667.xml
last_cpu_time: 806
upload_file_number: 1
last_iter: 23
last_upload: 86400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0023_1985050100_123_954_12170667_0_r290137270
The child process has been launched with process id: 22379
Executing the command: /var/lib/boinc/slots/26/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22379:22379] [20230104:001202:1672765922:0.004] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:17:40 STEP 43 H= 43:00 +CPU=136.883
Invalid characters in stoi string: STEPO
Unable to update iter, resetting to last_iter.
ifs_line =
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Invalid characters in stoi string: STEPO
Unable to update iter, resetting to last_iter.
ifs_line =
00:42:07 STEP 44 H= 44:00 +CPU=129.444
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
01:05:44 STEP 24 H= 24:00 +CPU=205.748
01:06:00 STEP 45 H= 45:00 +CPU= 88.924
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
02:28:37 STEP 25 H= 25:00 +CPU= 94.601
02:29:35 STEP 46 H= 46:00 +CPU=120.196
02:29:42 STEP 26 H= 26:00 +CPU= 39.327
Copying the namelist files from: ../../projects/climateprediction.net/jf_075e9f90bca79e48305de72f7272600a to: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
ic_ancil_file: ic_ancil_12170667
ifsdata_file: ifsdata_12170667
climate_data_file: clim_data_12170667
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_58bde664c39c1b869542abb4827bddc4 to: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecf601fa1042c4873537e6e50f79cf64 to: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
02:29:57 STEP 47 H= 47:00 +CPU= 20.289
Checking for progress XML file: /var/lib/boinc/slots/26/progress_file_12170667.xml
Opened progress file ok : /var/lib/boinc/slots/26/progress_file_12170667.xml
-- Model is restarting --
Adjusting last_iter, -999, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/26/progress_file_12170667.xml
last_cpu_time: 961
upload_file_number: 1
last_iter: -985
last_upload: 86400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0023_1985050100_123_954_12170667_0_r290137270
The child process has been launched with process id: 22834
Executing the command: /var/lib/boinc/slots/26/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.837] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.837] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.837] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:22834:22834] [20230104:023000:1672774200:0.859] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
02:30:16 STEP 27 H= 27:00 +CPU= 26.586
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:55:26 STEP 28 H= 28:00 +CPU=262.437
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
06:07:49 STEP 48 H= 48:00 +CPU=480.497
06:07:58 STEP 29 H= 29:00 +CPU=211.526
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
06:47:56 STEP 48 H= 48:00 +CPU=313.907
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_075e9f90bca79e48305de72f7272600a to: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
ic_ancil_file: ic_ancil_12170667
ifsdata_file: ifsdata_12170667
climate_data_file: clim_data_12170667
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_58bde664c39c1b869542abb4827bddc4 to: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecf601fa1042c4873537e6e50f79cf64 to: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Checking for progress XML file: /var/lib/boinc/slots/26/progress_file_12170667.xml
Opened progress file ok : /var/lib/boinc/slots/26/progress_file_12170667.xml
-- Model is restarting --
Adjusting last_iter, 48, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/26/progress_file_12170667.xml
last_cpu_time: 1133
upload_file_number: 1
last_iter: 47
last_upload: 86400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0023_1985050100_123_954_12170667_0_r290137270
The child process has been launched with process id: 30441
Executing the command: /var/lib/boinc/slots/26/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30441:30441] [20230104:085113:1672797073:0.003] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0023
(argv4) batchid: 954
(argv5) wuid: 12170667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/26
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_075e9f90bca79e48305de72f7272600a to: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/26/oifs_43r3_ps_0023_1985050100_123_954_12170667.zip
ic_ancil_file: ic_ancil_12170667
ifsdata_file: ifsdata_12170667
climate_data_file: clim_data_12170667
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_58bde664c39c1b869542abb4827bddc4 to: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/26/ic_ancil_12170667.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecf601fa1042c4873537e6e50f79cf64 to: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/26/ifsdata/ifsdata_12170667.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/26/159l_2/clim_data_12170667.zip
Checking for progress XML file: /var/lib/boinc/slots/26/progress_file_12170667.xml
Opened progress file ok : /var/lib/boinc/slots/26/progress_file_12170667.xml
-- Model is restarting --
Adjusting last_iter, -999, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/26/progress_file_12170667.xml
last_cpu_time: 1242
upload_file_number: 1
last_iter: -985
last_upload: 86400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0023_1985050100_123_954_12170667_0_r290137270
The child process has been launched with process id: 30612
Executing the command: /var/lib/boinc/slots/26/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:30612:30612] [20230104:095320:1672800800:0.003] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:38:44 STEP 48 H= 48:00 +CPU=*******
Moving to projects directory: /var/lib/boinc/slots/26/ICMGGhq0f+000048
Moving to projects directory: /var/lib/boinc/slots/26/ICMSHhq0f+000048
Moving to projects directory: /var/lib/boinc/slots/26/ICMUAhq0f+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMGGhq0f+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMSHhq0f+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMUAhq0f+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMGGhq0f+000036
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMSHhq0f+000036
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12170667/ICMUAhq0f+000036
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0023_1985050100_123_954_12170667_0_r290137270_1.zip
Uploading the intermediate file: upload_file_1.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
The child process terminated with status: 0
..Failed, model did not complete successfully
</stderr_txt>
]]>
