<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ar/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000084
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a022_2016092300_15_1016_12281810_2_r1131481910_0.zip
20:19:10 STEP 97 H= 24:15 +CPU= 9.971
Uploading the intermediate file: upload_file_0.zip
20:19:29 STEP 98 H= 24:30 +CPU= 16.243
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:37:22 STEP 99 H= 24:45 +CPU= 19.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4411MB (heap), 5023MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4411 MB (maxheap), 5023 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001)
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : TURBULENCE_LAYER
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : VDFOUTER
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : VDFMAIN
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223500:1722890100:8971.516] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : VDFDIFM
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223501:1722890101:8972.516] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:linux-x12:1:1:10115:10115] [20240805:223501:1722890101:8972.516] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CF935B Unknown Unknown Unknown
oifs_43r3_model.e 00000000008214C1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DD5490 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001241690 vdfdifm_ 206 vdfdifm.F90
oifs_43r3_model.e 0000000001223411 vdfmain_ 1162 vdfmain.F90
oifs_43r3_model.e 0000000001207E29 vdfouter_ 739 vdfouter.F90
oifs_43r3_model.e 00000000011FDCB2 turbulence_layer_ 158 turbulence_layer.F90
oifs_43r3_model.e 0000000000FAC003 callpar_ 869 callpar.F90
oifs_43r3_model.e 0000000000F5FA6B ec_phys_ 670 ec_phys.F90
oifs_43r3_model.e 0000000000F4DBC4 ec_phys_drv_ 599 ec_phys_drv.F90
oifs_43r3_model.e 0000000000BD7131 gp_model_ 613 gp_model.F90
oifs_43r3_model.e 00000000012C5002 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BACBAE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 0000000000688153 stepo_ 327 stepo.F90
oifs_43r3_model.e 00000000004CD1DC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000004B5D85 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 00000000004B4F52 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 000000000049D638 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 0000000000406FBD cnt0_ 146 cnt0.F90
oifs_43r3_model.e 0000000000401E7F MAIN__ 96 master.F90
oifs_43r3_model.e 0000000000401E12 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCC34A Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCDBE7 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000401CD5 Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a022
(argv4) batchid: 1016
(argv5) wuid: 12281810
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_08155eea8bc87deecab7c9d77808c133 to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a022_2016092300_15_1016_12281810.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a022_2016092300_15_1016_12281810.zip
ic_ancil_file: ic_ancil_12281810
ifsdata_file: ifsdata_12281810
climate_data_file: clim_data_12281810
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/1/ic_ancil_12281810.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12281810.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_dfda4a4da7d8205e2491b68c28cf1ea8 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12281810.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12281810.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12281810.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12281810.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12281810.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12281810.xml
-- Model is restarting --
Adjusting last_iter, 100, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12281810.xml
last_cpu_time: 859
upload_file_number: 1
last_iter: 97
last_upload: 86400
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a022_2016092300_15_1016_12281810_2_r1131481910
The child process has been launched with process id: 5634
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:173033:1722958233:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
17:30:44 STEP 96 H= 24:00 +CPU= 8.271
17:30:51 STEP 97 H= 24:15 +CPU= 7.571
17:30:59 STEP 98 H= 24:30 +CPU= 7.978
17:31:06 STEP 99 H= 24:45 +CPU= 7.534
17:31:14 STEP 100 H= 25:00 +CPU= 7.941
17:31:22 STEP 101 H= 25:15 +CPU= 7.983
17:31:30 STEP 102 H= 25:30 +CPU= 8.061
17:31:38 STEP 103 H= 25:45 +CPU= 7.930
17:31:46 STEP 104 H= 26:00 +CPU= 8.088
17:31:54 STEP 105 H= 26:15 +CPU= 7.825
17:32:02 STEP 106 H= 26:30 +CPU= 7.601
17:32:10 STEP 107 H= 26:45 +CPU= 7.743
17:32:20 STEP 108 H= 27:00 +CPU= 10.265
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000108
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000108
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000108
17:32:28 STEP 109 H= 27:15 +CPU= 7.837
17:32:35 STEP 110 H= 27:30 +CPU= 7.598
17:32:43 STEP 111 H= 27:45 +CPU= 7.869
17:32:51 STEP 112 H= 28:00 +CPU= 8.015
17:32:59 STEP 113 H= 28:15 +CPU= 7.698
17:33:06 STEP 114 H= 28:30 +CPU= 7.648
17:33:14 STEP 115 H= 28:45 +CPU= 7.827
17:33:22 STEP 116 H= 29:00 +CPU= 8.168
17:33:30 STEP 117 H= 29:15 +CPU= 7.871
17:33:38 STEP 118 H= 29:30 +CPU= 7.984
17:33:46 STEP 119 H= 29:45 +CPU= 7.751
17:33:56 STEP 120 H= 30:00 +CPU= 10.275
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000120
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000120
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000120
17:34:04 STEP 121 H= 30:15 +CPU= 7.768
17:34:12 STEP 122 H= 30:30 +CPU= 7.833
17:34:20 STEP 123 H= 30:45 +CPU= 7.783
17:34:28 STEP 124 H= 31:00 +CPU= 7.958
17:34:36 STEP 125 H= 31:15 +CPU= 7.749
17:34:43 STEP 126 H= 31:30 +CPU= 7.775
17:34:51 STEP 127 H= 31:45 +CPU= 7.804
17:34:59 STEP 128 H= 32:00 +CPU= 8.019
17:35:07 STEP 129 H= 32:15 +CPU= 7.753
17:35:15 STEP 130 H= 32:30 +CPU= 7.709
17:35:22 STEP 131 H= 32:45 +CPU= 7.672
17:35:33 STEP 132 H= 33:00 +CPU= 10.373
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000132
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000132
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000132
17:35:40 STEP 133 H= 33:15 +CPU= 7.642
17:35:48 STEP 134 H= 33:30 +CPU= 7.813
17:35:56 STEP 135 H= 33:45 +CPU= 7.680
17:36:04 STEP 136 H= 34:00 +CPU= 8.107
17:36:12 STEP 137 H= 34:15 +CPU= 7.611
17:36:19 STEP 138 H= 34:30 +CPU= 7.734
17:36:27 STEP 139 H= 34:45 +CPU= 7.626
17:36:35 STEP 140 H= 35:00 +CPU= 8.025
17:36:43 STEP 141 H= 35:15 +CPU= 7.723
17:36:50 STEP 142 H= 35:30 +CPU= 7.561
17:36:58 STEP 143 H= 35:45 +CPU= 7.741
17:37:10 STEP 144 H= 36:00 +CPU= 12.003
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000144
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000144
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000144
17:37:18 STEP 145 H= 36:15 +CPU= 7.603
17:37:25 STEP 146 H= 36:30 +CPU= 7.664
17:37:33 STEP 147 H= 36:45 +CPU= 7.728
17:37:41 STEP 148 H= 37:00 +CPU= 7.972
17:37:49 STEP 149 H= 37:15 +CPU= 7.758
17:37:57 STEP 150 H= 37:30 +CPU= 7.784
17:38:04 STEP 151 H= 37:45 +CPU= 7.727
17:38:12 STEP 152 H= 38:00 +CPU= 8.031
17:38:20 STEP 153 H= 38:15 +CPU= 7.606
17:38:28 STEP 154 H= 38:30 +CPU= 7.640
17:38:35 STEP 155 H= 38:45 +CPU= 7.631
17:38:46 STEP 156 H= 39:00 +CPU= 10.312
17:38:53 STEP 157 H= 39:15 +CPU= 7.746
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000156
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000156
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000156
17:39:01 STEP 158 H= 39:30 +CPU= 7.837
17:39:09 STEP 159 H= 39:45 +CPU= 7.761
17:39:17 STEP 160 H= 40:00 +CPU= 7.952
17:39:25 STEP 161 H= 40:15 +CPU= 7.641
17:39:32 STEP 162 H= 40:30 +CPU= 7.609
17:39:40 STEP 163 H= 40:45 +CPU= 7.751
17:39:48 STEP 164 H= 41:00 +CPU= 7.959
17:39:56 STEP 165 H= 41:15 +CPU= 7.633
17:40:03 STEP 166 H= 41:30 +CPU= 7.745
17:40:11 STEP 167 H= 41:45 +CPU= 7.654
17:40:21 STEP 168 H= 42:00 +CPU= 10.287
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000168
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000168
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000168
17:40:29 STEP 169 H= 42:15 +CPU= 7.663
17:40:37 STEP 170 H= 42:30 +CPU= 7.683
17:40:44 STEP 171 H= 42:45 +CPU= 7.726
17:40:52 STEP 172 H= 43:00 +CPU= 7.998
17:41:00 STEP 173 H= 43:15 +CPU= 7.758
17:41:08 STEP 174 H= 43:30 +CPU= 7.783
17:41:16 STEP 175 H= 43:45 +CPU= 7.645
17:41:23 STEP 176 H= 44:00 +CPU= 7.901
17:41:31 STEP 177 H= 44:15 +CPU= 7.512
17:41:39 STEP 178 H= 44:30 +CPU= 7.712
17:41:46 STEP 179 H= 44:45 +CPU= 7.730
17:41:57 STEP 180 H= 45:00 +CPU= 10.265
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000180
17:42:05 STEP 181 H= 45:15 +CPU= 7.790
17:42:12 STEP 182 H= 45:30 +CPU= 7.794
17:42:20 STEP 183 H= 45:45 +CPU= 7.629
17:42:28 STEP 184 H= 46:00 +CPU= 7.993
17:42:36 STEP 185 H= 46:15 +CPU= 7.692
17:42:43 STEP 186 H= 46:30 +CPU= 7.758
17:42:51 STEP 187 H= 46:45 +CPU= 7.635
17:42:59 STEP 188 H= 47:00 +CPU= 7.987
17:43:07 STEP 189 H= 47:15 +CPU= 7.720
17:43:15 STEP 190 H= 47:30 +CPU= 7.698
17:43:22 STEP 191 H= 47:45 +CPU= 7.617
17:43:34 STEP 192 H= 48:00 +CPU= 12.044
17:43:42 STEP 193 H= 48:15 +CPU= 7.755
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000192
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000192
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000180
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a022_2016092300_15_1016_12281810_2_r1131481910_1.zip
Uploading the intermediate file: upload_file_1.zip
17:43:50 STEP 194 H= 48:30 +CPU= 7.725
17:43:57 STEP 195 H= 48:45 +CPU= 7.632
17:44:05 STEP 196 H= 49:00 +CPU= 8.103
17:44:13 STEP 197 H= 49:15 +CPU= 7.851
17:44:21 STEP 198 H= 49:30 +CPU= 7.904
17:44:29 STEP 199 H= 49:45 +CPU= 7.742
17:44:37 STEP 200 H= 50:00 +CPU= 8.082
17:44:45 STEP 201 H= 50:15 +CPU= 7.861
17:44:53 STEP 202 H= 50:30 +CPU= 7.875
17:45:01 STEP 203 H= 50:45 +CPU= 7.828
17:45:11 STEP 204 H= 51:00 +CPU= 10.333
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000204
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000204
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000204
17:45:19 STEP 205 H= 51:15 +CPU= 7.671
17:45:27 STEP 206 H= 51:30 +CPU= 7.798
17:45:34 STEP 207 H= 51:45 +CPU= 7.905
17:45:42 STEP 208 H= 52:00 +CPU= 8.078
17:45:50 STEP 209 H= 52:15 +CPU= 7.645
17:45:58 STEP 210 H= 52:30 +CPU= 7.698
17:46:06 STEP 211 H= 52:45 +CPU= 7.845
17:46:14 STEP 212 H= 53:00 +CPU= 7.893
17:46:21 STEP 213 H= 53:15 +CPU= 7.810
17:46:29 STEP 214 H= 53:30 +CPU= 7.630
17:46:37 STEP 215 H= 53:45 +CPU= 7.788
17:46:47 STEP 216 H= 54:00 +CPU= 10.373
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000216
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000216
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000216
17:46:55 STEP 217 H= 54:15 +CPU= 7.709
17:47:03 STEP 218 H= 54:30 +CPU= 7.866
17:47:11 STEP 219 H= 54:45 +CPU= 7.863
17:47:19 STEP 220 H= 55:00 +CPU= 8.192
17:47:27 STEP 221 H= 55:15 +CPU= 7.808
17:47:34 STEP 222 H= 55:30 +CPU= 7.719
17:47:42 STEP 223 H= 55:45 +CPU= 7.715
17:47:50 STEP 224 H= 56:00 +CPU= 8.050
17:47:58 STEP 225 H= 56:15 +CPU= 7.711
17:48:06 STEP 226 H= 56:30 +CPU= 7.873
17:48:14 STEP 227 H= 56:45 +CPU= 7.752
17:48:24 STEP 228 H= 57:00 +CPU= 10.252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000228
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000228
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000228
17:48:32 STEP 229 H= 57:15 +CPU= 7.680
17:48:39 STEP 230 H= 57:30 +CPU= 7.896
17:48:47 STEP 231 H= 57:45 +CPU= 7.717
17:48:55 STEP 232 H= 58:00 +CPU= 8.069
17:49:03 STEP 233 H= 58:15 +CPU= 7.847
17:49:11 STEP 234 H= 58:30 +CPU= 7.629
17:49:18 STEP 235 H= 58:45 +CPU= 7.708
17:49:26 STEP 236 H= 59:00 +CPU= 7.905
17:49:34 STEP 237 H= 59:15 +CPU= 7.818
17:49:42 STEP 238 H= 59:30 +CPU= 7.656
17:49:49 STEP 239 H= 59:45 +CPU= 7.660
17:50:02 STEP 240 H= 60:00 +CPU= 12.183
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000240
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000240
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000240
17:50:09 STEP 241 H= 60:15 +CPU= 7.758
17:50:17 STEP 242 H= 60:30 +CPU= 7.670
17:50:25 STEP 243 H= 60:45 +CPU= 7.743
17:50:33 STEP 244 H= 61:00 +CPU= 8.044
17:50:41 STEP 245 H= 61:15 +CPU= 7.925
17:50:49 STEP 246 H= 61:30 +CPU= 7.746
17:50:57 STEP 247 H= 61:45 +CPU= 7.932
17:51:05 STEP 248 H= 62:00 +CPU= 8.136
17:51:13 STEP 249 H= 62:15 +CPU= 7.806
17:51:20 STEP 250 H= 62:30 +CPU= 7.752
17:51:28 STEP 251 H= 62:45 +CPU= 7.731
17:51:39 STEP 252 H= 63:00 +CPU= 10.399
17:51:46 STEP 253 H= 63:15 +CPU= 7.753
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000252
17:51:54 STEP 254 H= 63:30 +CPU= 7.806
17:52:02 STEP 255 H= 63:45 +CPU= 7.814
17:52:10 STEP 256 H= 64:00 +CPU= 7.937
17:52:17 STEP 257 H= 64:15 +CPU= 7.627
17:52:25 STEP 258 H= 64:30 +CPU= 7.736
17:52:33 STEP 259 H= 64:45 +CPU= 7.826
17:52:41 STEP 260 H= 65:00 +CPU= 7.957
17:52:49 STEP 261 H= 65:15 +CPU= 7.598
17:52:56 STEP 262 H= 65:30 +CPU= 7.886
17:53:04 STEP 263 H= 65:45 +CPU= 7.797
17:53:15 STEP 264 H= 66:00 +CPU= 10.379
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000264
17:53:22 STEP 265 H= 66:15 +CPU= 7.636
17:53:30 STEP 266 H= 66:30 +CPU= 7.824
17:53:38 STEP 267 H= 66:45 +CPU= 7.751
17:53:46 STEP 268 H= 67:00 +CPU= 8.028
17:53:54 STEP 269 H= 67:15 +CPU= 7.868
17:54:02 STEP 270 H= 67:30 +CPU= 7.870
17:54:09 STEP 271 H= 67:45 +CPU= 7.777
17:54:18 STEP 272 H= 68:00 +CPU= 8.121
17:54:25 STEP 273 H= 68:15 +CPU= 7.573
17:54:33 STEP 274 H= 68:30 +CPU= 7.768
17:54:41 STEP 275 H= 68:45 +CPU= 7.740
17:54:51 STEP 276 H= 69:00 +CPU= 10.332
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000276
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000276
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000276
17:54:59 STEP 277 H= 69:15 +CPU= 7.843
17:55:07 STEP 278 H= 69:30 +CPU= 7.830
17:55:15 STEP 279 H= 69:45 +CPU= 7.771
17:55:22 STEP 280 H= 70:00 +CPU= 7.841
17:55:30 STEP 281 H= 70:15 +CPU= 7.700
17:55:38 STEP 282 H= 70:30 +CPU= 7.794
17:55:46 STEP 283 H= 70:45 +CPU= 7.937
17:55:54 STEP 284 H= 71:00 +CPU= 8.067
17:56:02 STEP 285 H= 71:15 +CPU= 7.925
17:56:10 STEP 286 H= 71:30 +CPU= 7.766
17:56:17 STEP 287 H= 71:45 +CPU= 7.880
17:56:30 STEP 288 H= 72:00 +CPU= 12.314
17:56:38 STEP 289 H= 72:15 +CPU= 7.844
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000288
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000288
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMGGh7zg+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMSHh7zg+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12281810/ICMUAh7zg+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a022_2016092300_15_1016_12281810_2_r1131481910_2.zip
Uploading the intermediate file: upload_file_2.zip
17:56:46 STEP 290 H= 72:30 +CPU= 7.948
17:56:53 STEP 291 H= 72:45 +CPU= 7.787
17:57:02 STEP 292 H= 73:00 +CPU= 8.211
17:57:09 STEP 293 H= 73:15 +CPU= 7.872
17:57:17 STEP 294 H= 73:30 +CPU= 7.740
17:57:25 STEP 295 H= 73:45 +CPU= 7.802
17:57:33 STEP 296 H= 74:00 +CPU= 8.043
17:57:41 STEP 297 H= 74:15 +CPU= 7.694
17:57:49 STEP 298 H= 74:30 +CPU= 7.791
17:57:56 STEP 299 H= 74:45 +CPU= 7.658
17:58:07 STEP 300 H= 75:00 +CPU= 10.423
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000300
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000300
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000300
17:58:15 STEP 301 H= 75:15 +CPU= 7.924
17:58:22 STEP 302 H= 75:30 +CPU= 7.705
17:58:30 STEP 303 H= 75:45 +CPU= 7.838
17:58:38 STEP 304 H= 76:00 +CPU= 7.965
17:58:46 STEP 305 H= 76:15 +CPU= 7.671
17:58:54 STEP 306 H= 76:30 +CPU= 7.769
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 3536MB (heap), 4795MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 3536 MB (maxheap), 4795 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CPG_DRV
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : >OMP-CPG (1025)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CPG
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CPG_GP
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175854:1722959934:1701.091] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_SPV
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/1/oifs_43r3_model.exe' : sigcontextptr=0x7ffd834a9dd0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 22) with addr2line-cmd
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/1/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0xd27426 0xd0d679 0xc2432e 0xc22238 0xbcc914 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0xd27426] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0xd0d679] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0xc2432e] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xc22238] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xbcc914] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x688153] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.091] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
Process 1 thread 1 calling linux_trbk from intel_trbk()
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/1/oifs_43r3_model.exe' : sigcontextptr=0x7ffd834a9ce0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 23) with addr2line-cmd
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/1/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0xd27426 0xd0d679 0xc2432e 0xc22238 0xbcc914 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0xd27426] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0xd0d679] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xc2432e] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xc22238] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xbcc914] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0x12c5002] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbacbae] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x688153] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x4cd1dc] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4b5d85] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b4f52] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x49d638] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x406fbd] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x401e7f] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e12] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x1dcc34a] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcdbe7] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x401cd5] : ??() at ??:0
[LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.104] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:linux-x12:1:1:5634:5634] [20240806:175855:1722959935:1702.104] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CF935B Unknown Unknown Unknown
oifs_43r3_model.e 00000000008214C1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DD5490 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001CC6692 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000D27426 gp_spv_ 122 gp_spv.F90
oifs_43r3_model.e 0000000000D0D679 cpg_gp_ 601 cpg_gp.F90
oifs_43r3_model.e 0000000000C2432E cpg_ 627 cpg.F90
oifs_43r3_model.e 0000000000C22238 cpg_drv_ 351 cpg_drv.F90
oifs_43r3_model.e 0000000000BCC914 gp_model_ 419 gp_model.F90
oifs_43r3_model.e 00000000012C5002 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BACBAE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 0000000000688153 stepo_ 327 stepo.F90
oifs_43r3_model.e 00000000004CD1DC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000004B5D85 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 00000000004B4F52 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 000000000049D638 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 0000000000406FBD cnt0_ 146 cnt0.F90
oifs_43r3_model.e 0000000000401E7F MAIN__ 96 master.F90
oifs_43r3_model.e 0000000000401E12 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCC34A Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCDBE7 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000401CD5 Unknown Unknown Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01
AVE 0.193183768845474E+00 0.000000000000000E+00 0.100000000000000E+01
GPNORM CLW AVERAGE MINIMUM MAXIMUM
AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
GPNORM CIW AVERAGE MINIMUM MAXIMUM
AVE 0.346465201574718E-07 0.000000000000000E+00 0.186569501204956E-06
GPNORM CC AVERAGE MINIMUM MAXIMUM
AVE 0.193141071772872E+00 0.000000000000000E+00 0.100000000000000E+01
GPNORM RFL AVERAGE MINIMUM MAXIMUM
AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
GPNORM SFL AVERAGE MINIMUM MAXIMUM
AVE 0.282477608084721E-09 0.000000000000000E+00 0.254147420025131E-08
GPNORM CRFL AVERAGE MINIMUM MAXIMUM
AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
GPNORM CSFL AVERAGE MINIMUM MAXIMUM
AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
GPNORM PFRC AVERAGE MINIMUM MAXIMUM
AVE 0.259888976724575E-01 0.000000000000000E+00 0.100000000000000E+01
NSTEP = 300 STEPO A00000000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 300 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+000300 MODE=w
IO-STREAM CLOSED - ICMGGh7zg+000300
MPI-TASK: 1 - 323952740 BYTES IN 6 RECORDS TRANSFERRED IN 0.0003 SECONDS******** Mbytes/s, TOTAL TIME= 0.0045( 6.7%)
NSTEP = 300 STEPO 0AA000000
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMSHh7zg+000300 MODE=w
IO-STREAM CLOSED - ./ICMSHh7zg+000300
MPI-TASK: 1 - 707399250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0076 SECONDS 93078.8 Mbytes/s, TOTAL TIME= 0.0854( 8.9%)
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+000300 MODE=w
IO-STREAM CLOSED - ./ICMUAh7zg+000300
MPI-TASK: 1 - 5183243840 BYTES IN 125 RECORDS TRANSFERRED IN 0.0032 SECONDS******** Mbytes/s, TOTAL TIME= 0.0426( 7.5%)
STEPO_FPOS WAS : B00000F00E
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+000300 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+000300
MPI-TASK: 1 - 10 BYTES IN 2 RECORDS TRANSFERRED IN 0.0002 SECONDS 0.1 Mbytes/s, TOTAL TIME= 0.0015(13.3%)
STEPO_FPOS WAS : V00000000Y
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+000300 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+000300
MPI-TASK: 1 - 315373011 BYTES IN 4 RECORDS TRANSFERRED IN 0.0002 SECONDS******** Mbytes/s, TOTAL TIME= 0.0021( 9.5%)
STEPO_FPOS WAS : T00000000M
NSTEP = 300 STEPO 0AAA00AAA
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 300 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+000300 MODE=a
IO-STREAM CLOSED - ICMGGh7zg+000300
MPI-TASK: 1 - 323952740 BYTES IN 7 RECORDS TRANSFERRED IN 0.0004 SECONDS809881.8 Mbytes/s, TOTAL TIME= 0.0037(10.8%)
17:58:07 STEP 300 H= 75:00 +CPU= 10.423
NSTEP = 301 STEPO 0AAA00AAA
17:58:15 STEP 301 H= 75:15 +CPU= 7.924
NSTEP = 302 STEPO 0AAA00AAA
17:58:22 STEP 302 H= 75:30 +CPU= 7.705
NSTEP = 303 STEPO 0AAA00AAA
17:58:30 STEP 303 H= 75:45 +CPU= 7.838
NSTEP = 304 STEPO 0AAA00AAA
17:58:38 STEP 304 H= 76:00 +CPU= 7.965
ISTEP= 305 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP= 305 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
NSTEP = 305 STEPO 0AAA00AAA
17:58:46 STEP 305 H= 76:15 +CPU= 7.671
NSTEP = 306 STEPO 0AAA00AAA
17:58:54 STEP 306 H= 76:30 +CPU= 7.769
NSTEP = 307 STEPO 0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat
17:58:07 0AAA00AAA STEPO 301 8.170 8.170 8.182 27:32 27:33 0.15873181443931E-07 2GB 0MB
17:58:15 0AAA00AAA STEPO 302 7.924 7.924 7.925 27:40 27:41 0.15873181443931E-07 2GB 0MB
17:58:22 0AAA00AAA STEPO 303 7.705 7.705 7.709 27:47 27:49 0.15873181443931E-07 2GB 0MB
17:58:30 0AAA00AAA STEPO 304 7.838 7.838 7.846 27:55 27:57 0.15873181443931E-07 2GB 0MB
17:58:38 0AAA00AAA STEPO 305 7.965 7.965 7.965 28:03 28:05 0.15873181443931E-07 2GB 0MB
17:58:46 0AAA00AAA STEPO 306 7.671 7.671 7.675 28:11 28:12 0.15873181443931E-07 2GB 0MB
17:58:54 0AAA00AAA STEPO 307 7.770 7.770 7.773 28:18 28:20 0.15873181443931E-07 2GB 0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf
CSTEP = " 288",
CTIME = "00030000 ",
NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 276, 2*0,
IPRGPNSRES = 1,
IPRGPEWRES = 1,
IPRTRWRES = 1,
IPRTRVRES = 1,
GMASS0 = 0.000000000000000E+000,
GMASSI = 0.000000000000000E+000
/
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/1/progress_file_12281810.xml
<running_values>
<last_cpu_time>2557.060000</last_cpu_time>
<upload_file_number>3</upload_file_number>
<last_iter>307</last_iter>
<last_upload>259200</last_upload>
<model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
</stderr_txt>
]]>
