climateprediction.net home page
Task 22309880

Task 22309880

Name oifs_43r3_bl_a28d_2016092300_15_991_12209680_0
Workunit 12209680
Created 13 Feb 2023, 17:20:10 UTC
Sent 13 Feb 2023, 17:49:11 UTC
Report deadline 14 Apr 2023, 17:49:11 UTC
Received 18 Feb 2023, 12:56:56 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1536834
Run time 3 hours 11 min 46 sec
CPU time 3 hours 11 min 46 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 7.78 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,489.80 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,334.74 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
CS              (1001) 
[EC_DRHOOK:YatagarasuDrive:1:1:308486:308486] [20230215:210733:1676491653:7884.651] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:YatagarasuDrive:1:1:308486:308486] [20230215:210733:1676491653:7884.651] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:YatagarasuDrive:1:1:308486:308486] [20230215:210733:1676491653:7884.651] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                STATE_UPDATE 
[EC_DRHOOK:YatagarasuDrive:1:1:308486:308486] [20230215:210734:1676491654:7885.651] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:308486:308486] [20230215:210734:1676491654:7885.651] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000013B2E20  state_update_              44  state_update.F90
oifs_43r3_model.e  000000000117991C  callpar_                 1404  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a28d
(argv4) batchid: 991
(argv5) wuid: 12209680
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/4
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/4/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/4/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ab28bc74756bb7d67a94ddf5dd6bd1f4 to: /var/lib/boinc/slots/4/oifs_43r3_bl_a28d_2016092300_15_991_12209680.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/4/oifs_43r3_bl_a28d_2016092300_15_991_12209680.zip
ic_ancil_file: ic_ancil_12209680
ifsdata_file: ifsdata_12209680
climate_data_file: clim_data_12209680
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/4/ic_ancil_12209680.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/4/ic_ancil_12209680.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_324de320f73a2bae903313194a428133 to: /var/lib/boinc/slots/4/ifsdata/ifsdata_12209680.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/4/ifsdata/ifsdata_12209680.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/4/159l_2/clim_data_12209680.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/4/159l_2/clim_data_12209680.zip
Checking for progress XML file: /var/lib/boinc/slots/4/progress_file_12209680.xml
Opened progress file ok : /var/lib/boinc/slots/4/progress_file_12209680.xml
-- Model is restarting --
Adjusting last_iter, 308, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/4/progress_file_12209680.xml
last_cpu_time: 7951
upload_file_number: 3
last_iter: 289
last_upload: 260100
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a28d_2016092300_15_991_12209680_0_r1022650133
The child process has been launched with process id: 16716
Executing the command: /var/lib/boinc/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:173713:1676565433:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  17:37:32 STEP  288 H=  72:00 +CPU= 15.121
  17:37:47 STEP  289 H=  72:15 +CPU= 14.280
  17:38:01 STEP  290 H=  72:30 +CPU= 14.400
  17:38:15 STEP  291 H=  72:45 +CPU= 13.640
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:38:52 STEP  292 H=  73:00 +CPU= 15.469
  17:39:06 STEP  293 H=  73:15 +CPU= 13.484
  17:39:19 STEP  294 H=  73:30 +CPU= 13.168
  17:39:32 STEP  295 H=  73:45 +CPU= 13.089
  17:39:46 STEP  296 H=  74:00 +CPU= 13.958
  17:40:03 STEP  297 H=  74:15 +CPU= 16.815
  17:40:20 STEP  298 H=  74:30 +CPU= 16.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:40:56 STEP  299 H=  74:45 +CPU= 14.627
  17:41:13 STEP  300 H=  75:00 +CPU= 17.439
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000300
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000300
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:41:52 STEP  301 H=  75:15 +CPU= 17.222
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:42:30 STEP  302 H=  75:30 +CPU= 16.499
  17:42:47 STEP  303 H=  75:45 +CPU= 16.999
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:43:45 STEP  304 H=  76:00 +CPU= 15.362
  17:44:01 STEP  305 H=  76:15 +CPU= 16.011
  17:44:15 STEP  306 H=  76:30 +CPU= 13.564
  17:44:32 STEP  307 H=  76:45 +CPU= 16.990
  17:44:47 STEP  308 H=  77:00 +CPU= 14.943
  17:45:02 STEP  309 H=  77:15 +CPU= 15.056
  17:45:17 STEP  310 H=  77:30 +CPU= 14.058
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:45:55 STEP  311 H=  77:45 +CPU= 16.657
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:46:42 STEP  312 H=  78:00 +CPU= 24.417
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000312
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000312
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:47:23 STEP  313 H=  78:15 +CPU= 19.501
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:05 STEP  314 H=  78:30 +CPU= 19.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:45 STEP  315 H=  78:45 +CPU= 18.639
  17:49:05 STEP  316 H=  79:00 +CPU= 18.280
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:49:47 STEP  317 H=  79:15 +CPU= 20.266
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:50:29 STEP  318 H=  79:30 +CPU= 18.723
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:11 STEP  319 H=  79:45 +CPU= 19.754
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:54 STEP  320 H=  80:00 +CPU= 21.187
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:52:35 STEP  321 H=  80:15 +CPU= 18.656
  17:52:55 STEP  322 H=  80:30 +CPU= 18.720
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:36 STEP  323 H=  80:45 +CPU= 18.562
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:54:23 STEP  324 H=  81:00 +CPU= 24.666
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000324
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000324
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000324
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:55:04 STEP  325 H=  81:15 +CPU= 19.468
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:55:48 STEP  326 H=  81:30 +CPU= 20.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:32 STEP  327 H=  81:45 +CPU= 20.972
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:57:09 STEP  328 H=  82:00 +CPU= 20.182
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:58:16 STEP  329 H=  82:15 +CPU= 21.370
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:59:00 STEP  330 H=  82:30 +CPU= 21.453
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:59:41 STEP  331 H=  82:45 +CPU= 19.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:01:09 STEP  332 H=  83:00 +CPU= 23.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:01:52 STEP  333 H=  83:15 +CPU= 20.434
  18:02:14 STEP  334 H=  83:30 +CPU= 21.063
  18:02:35 STEP  335 H=  83:45 +CPU= 19.608
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:03:45 STEP  336 H=  84:00 +CPU= 26.233
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000336
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000336
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000336
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:04:48 STEP  337 H=  84:15 +CPU= 19.659
  18:05:07 STEP  338 H=  84:30 +CPU= 18.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:05:51 STEP  339 H=  84:45 +CPU= 21.202
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:07:16 STEP  340 H=  85:00 +CPU= 22.020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:08:20 STEP  341 H=  85:15 +CPU= 20.360
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:15:40 STEP  342 H=  85:30 +CPU= 24.805
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:16:28 STEP  343 H=  85:45 +CPU= 22.382
  18:16:50 STEP  344 H=  86:00 +CPU= 20.816
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:17:31 STEP  345 H=  86:15 +CPU= 19.071
  18:17:53 STEP  346 H=  86:30 +CPU= 20.794
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:18:34 STEP  347 H=  86:45 +CPU= 18.787
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:19:18 STEP  348 H=  87:00 +CPU= 22.023
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000348
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000348
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000348
  18:19:36 STEP  349 H=  87:15 +CPU= 17.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:20:18 STEP  350 H=  87:30 +CPU= 20.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:21:02 STEP  351 H=  87:45 +CPU= 21.135
  18:21:22 STEP  352 H=  88:00 +CPU= 19.064
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:22:21 STEP  353 H=  88:15 +CPU= 17.197
  18:22:43 STEP  354 H=  88:30 +CPU= 21.135
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:23:21 STEP  355 H=  88:45 +CPU= 16.433
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:24:05 STEP  356 H=  89:00 +CPU= 21.725
  18:24:24 STEP  357 H=  89:15 +CPU= 17.320
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:25:02 STEP  358 H=  89:30 +CPU= 16.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:25:45 STEP  359 H=  89:45 +CPU= 20.822
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:26:31 STEP  360 H=  90:00 +CPU= 22.503
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000360
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000360
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000360
  18:27:12 STEP  361 H=  90:15 +CPU= 19.026
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:27:53 STEP  362 H=  90:30 +CPU= 18.894
  18:28:12 STEP  363 H=  90:45 +CPU= 17.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:28:53 STEP  364 H=  91:00 +CPU= 18.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:29:55 STEP  365 H=  91:15 +CPU= 18.506
  18:30:11 STEP  366 H=  91:30 +CPU= 14.928
  18:30:28 STEP  367 H=  91:45 +CPU= 16.697
  18:30:49 STEP  368 H=  92:00 +CPU= 19.940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:31:56 STEP  369 H=  92:15 +CPU= 21.728
  18:32:18 STEP  370 H=  92:30 +CPU= 21.176
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:33:04 STEP  371 H=  92:45 +CPU= 22.434
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:10:41 STEP  372 H=  93:00 +CPU= 26.799
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000372
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000372
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000372
  21:10:59 STEP  373 H=  93:15 +CPU= 17.744
  21:11:16 STEP  374 H=  93:30 +CPU= 16.312
  21:11:34 STEP  375 H=  93:45 +CPU= 16.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:12:18 STEP  376 H=  94:00 +CPU= 21.565
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:12:58 STEP  377 H=  94:15 +CPU= 18.382
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:13:38 STEP  378 H=  94:30 +CPU= 18.262
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:14:19 STEP  379 H=  94:45 +CPU= 18.944
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:15:21 STEP  380 H=  95:00 +CPU= 18.835
  21:15:40 STEP  381 H=  95:15 +CPU= 18.027
  21:15:59 STEP  382 H=  95:30 +CPU= 18.133
  21:16:14 STEP  383 H=  95:45 +CPU= 14.783
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4057MB (heap), 5425MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 4057 MB (maxheap), 5425 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS CLDPP T/S    (1002) 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211628:1676578588:13154.607] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                CUANCAPE2 
double free or corruption (top)
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211629:1676578589:13155.607] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:16716:16716] [20230216:211629:1676578589:13155.607] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CCEF51  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001398010  cuancape2_                148  cuancape2.F90
oifs_43r3_model.e  000000000116D1B1  callpar_                 1102  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a28d
(argv4) batchid: 991
(argv5) wuid: 12209680
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/4
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/4/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/4/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ab28bc74756bb7d67a94ddf5dd6bd1f4 to: /var/lib/boinc/slots/4/oifs_43r3_bl_a28d_2016092300_15_991_12209680.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/4/oifs_43r3_bl_a28d_2016092300_15_991_12209680.zip
ic_ancil_file: ic_ancil_12209680
ifsdata_file: ifsdata_12209680
climate_data_file: clim_data_12209680
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/4/ic_ancil_12209680.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/4/ic_ancil_12209680.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_324de320f73a2bae903313194a428133 to: /var/lib/boinc/slots/4/ifsdata/ifsdata_12209680.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/4/ifsdata/ifsdata_12209680.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/4/159l_2/clim_data_12209680.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/4/159l_2/clim_data_12209680.zip
Checking for progress XML file: /var/lib/boinc/slots/4/progress_file_12209680.xml
Opened progress file ok : /var/lib/boinc/slots/4/progress_file_12209680.xml
-- Model is restarting --
Adjusting last_iter, 384, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/4/progress_file_12209680.xml
last_cpu_time: 9755
upload_file_number: 3
last_iter: 385
last_upload: 260100
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a28d_2016092300_15_991_12209680_0_r1022650133
The child process has been launched with process id: 24337
Executing the command: /var/lib/boinc/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230217:175121:1676652681:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000300
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000300
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000300
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000312
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000312
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000312
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000324
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000324
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000324
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000336
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000336
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000336
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000348
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000348
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000348
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000360
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000360
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000360
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000372
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000372
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000372
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a28d_2016092300_15_991_12209680_0_r1022650133_3.zip
Uploading the intermediate file: upload_file_3.zip
  17:51:48 STEP  384 H=  96:00 +CPU= 22.354
  17:52:09 STEP  385 H=  96:15 +CPU= 19.411
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000384
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000384
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000384
  17:52:29 STEP  386 H=  96:30 +CPU= 18.554
  17:52:46 STEP  387 H=  96:45 +CPU= 16.951
  17:53:05 STEP  388 H=  97:00 +CPU= 17.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:43 STEP  389 H=  97:15 +CPU= 17.069
  17:54:01 STEP  390 H=  97:30 +CPU= 16.625
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:54:39 STEP  391 H=  97:45 +CPU= 16.870
  17:54:58 STEP  392 H=  98:00 +CPU= 17.625
  17:55:15 STEP  393 H=  98:15 +CPU= 16.737
  17:55:33 STEP  394 H=  98:30 +CPU= 16.850
  17:55:50 STEP  395 H=  98:45 +CPU= 16.847
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:33 STEP  396 H=  99:00 +CPU= 20.818
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000396
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000396
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000396
  17:56:50 STEP  397 H=  99:15 +CPU= 16.876
  17:57:08 STEP  398 H=  99:30 +CPU= 17.121
  17:57:26 STEP  399 H=  99:45 +CPU= 16.913
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:58:05 STEP  400 H= 100:00 +CPU= 16.886
  17:58:22 STEP  401 H= 100:15 +CPU= 16.772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:59:21 STEP  402 H= 100:30 +CPU= 16.424
  17:59:39 STEP  403 H= 100:45 +CPU= 16.792
  17:59:57 STEP  404 H= 101:00 +CPU= 17.270
  18:00:14 STEP  405 H= 101:15 +CPU= 17.000
  18:00:32 STEP  406 H= 101:30 +CPU= 17.193
  18:00:50 STEP  407 H= 101:45 +CPU= 16.790
  18:01:12 STEP  408 H= 102:00 +CPU= 21.413
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000408
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000408
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000408
  18:01:31 STEP  409 H= 102:15 +CPU= 17.500
  18:01:49 STEP  410 H= 102:30 +CPU= 17.119
  18:02:06 STEP  411 H= 102:45 +CPU= 16.658
  18:02:24 STEP  412 H= 103:00 +CPU= 17.353
  18:02:42 STEP  413 H= 103:15 +CPU= 16.942
  18:02:59 STEP  414 H= 103:30 +CPU= 16.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:03:38 STEP  415 H= 103:45 +CPU= 16.879
  18:03:56 STEP  416 H= 104:00 +CPU= 17.089
  18:04:14 STEP  417 H= 104:15 +CPU= 16.816
  18:04:31 STEP  418 H= 104:30 +CPU= 16.907
  18:04:49 STEP  419 H= 104:45 +CPU= 16.520
  18:05:11 STEP  420 H= 105:00 +CPU= 21.442
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000420
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000420
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000420
  18:05:28 STEP  421 H= 105:15 +CPU= 16.545
  18:05:46 STEP  422 H= 105:30 +CPU= 16.698
  18:06:03 STEP  423 H= 105:45 +CPU= 16.684
  18:06:21 STEP  424 H= 106:00 +CPU= 16.705
  18:06:38 STEP  425 H= 106:15 +CPU= 16.309
  18:07:01 STEP  426 H= 106:30 +CPU= 17.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:10:17 STEP  427 H= 106:45 +CPU= 19.359
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:10:54 STEP  428 H= 107:00 +CPU= 15.833
  18:11:09 STEP  429 H= 107:15 +CPU= 14.101
  18:11:23 STEP  430 H= 107:30 +CPU= 13.991
  18:11:38 STEP  431 H= 107:45 +CPU= 14.983
  18:11:59 STEP  432 H= 108:00 +CPU= 20.384
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000432
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000432
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000432
  18:12:15 STEP  433 H= 108:15 +CPU= 15.158
  18:12:30 STEP  434 H= 108:30 +CPU= 14.364
  18:12:44 STEP  435 H= 108:45 +CPU= 14.311
  18:12:59 STEP  436 H= 109:00 +CPU= 14.737
  18:13:13 STEP  437 H= 109:15 +CPU= 14.021
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:13:48 STEP  438 H= 109:30 +CPU= 13.810
  18:14:02 STEP  439 H= 109:45 +CPU= 12.962
  18:14:15 STEP  440 H= 110:00 +CPU= 13.090
  18:14:28 STEP  441 H= 110:15 +CPU= 13.108
  18:14:42 STEP  442 H= 110:30 +CPU= 13.583
  18:14:56 STEP  443 H= 110:45 +CPU= 13.965
  18:15:15 STEP  444 H= 111:00 +CPU= 17.634
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000444
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000444
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000444
  18:15:30 STEP  445 H= 111:15 +CPU= 14.770
  18:15:45 STEP  446 H= 111:30 +CPU= 14.706
  18:15:59 STEP  447 H= 111:45 +CPU= 13.397
  18:16:13 STEP  448 H= 112:00 +CPU= 14.186
  18:16:29 STEP  449 H= 112:15 +CPU= 14.861
  18:16:48 STEP  450 H= 112:30 +CPU= 18.232
  18:17:08 STEP  451 H= 112:45 +CPU= 18.816
  18:17:27 STEP  452 H= 113:00 +CPU= 18.487
  18:17:46 STEP  453 H= 113:15 +CPU= 17.775
  18:18:05 STEP  454 H= 113:30 +CPU= 18.449
  18:18:24 STEP  455 H= 113:45 +CPU= 17.886
  18:18:49 STEP  456 H= 114:00 +CPU= 23.179
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000456
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000456
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000456
  18:19:08 STEP  457 H= 114:15 +CPU= 18.531
  18:19:28 STEP  458 H= 114:30 +CPU= 18.825
  18:19:48 STEP  459 H= 114:45 +CPU= 19.231
  18:20:10 STEP  460 H= 115:00 +CPU= 19.946
  18:20:30 STEP  461 H= 115:15 +CPU= 19.166
  18:20:47 STEP  462 H= 115:30 +CPU= 16.879
  18:21:03 STEP  463 H= 115:45 +CPU= 15.040
  18:21:19 STEP  464 H= 116:00 +CPU= 15.719
  18:21:34 STEP  465 H= 116:15 +CPU= 15.066
  18:21:50 STEP  466 H= 116:30 +CPU= 15.297
  18:22:06 STEP  467 H= 116:45 +CPU= 15.300
  18:22:25 STEP  468 H= 117:00 +CPU= 19.073
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000468
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000468
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000468
  18:22:41 STEP  469 H= 117:15 +CPU= 15.296
  18:22:57 STEP  470 H= 117:30 +CPU= 15.231
  18:23:12 STEP  471 H= 117:45 +CPU= 15.413
  18:23:29 STEP  472 H= 118:00 +CPU= 15.895
  18:23:44 STEP  473 H= 118:15 +CPU= 15.241
  18:24:00 STEP  474 H= 118:30 +CPU= 15.653
  18:24:16 STEP  475 H= 118:45 +CPU= 15.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:24:55 STEP  476 H= 119:00 +CPU= 17.656
  18:25:11 STEP  477 H= 119:15 +CPU= 15.564
  18:25:27 STEP  478 H= 119:30 +CPU= 14.925
  18:25:41 STEP  479 H= 119:45 +CPU= 13.889
  18:26:02 STEP  480 H= 120:00 +CPU= 20.114
Moving to projects directory: /var/lib/boinc/slots/4/ICMGGh7zg+000480
Moving to projects directory: /var/lib/boinc/slots/4/ICMSHh7zg+000480
Moving to projects directory: /var/lib/boinc/slots/4/ICMUAh7zg+000480
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:26:38 STEP  481 H= 120:15 +CPU= 14.662
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000396
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000396
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000396
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000408
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000408
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000408
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000420
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000420
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000420
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000432
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000432
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000432
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000444
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000444
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000444
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000456
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000456
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000456
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000468
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000468
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000468
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMGGh7zg+000480
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMSHh7zg+000480
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12209680/ICMUAh7zg+000480
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a28d_2016092300_15_991_12209680_0_r1022650133_4.zip
  18:26:52 STEP  482 H= 120:30 +CPU= 13.547
Uploading the intermediate file: upload_file_4.zip
  18:27:06 STEP  483 H= 120:45 +CPU= 13.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:46:15 STEP  484 H= 121:00 +CPU= 17.729
  00:46:35 STEP  485 H= 121:15 +CPU= 19.348
  00:46:53 STEP  486 H= 121:30 +CPU= 17.217
  00:47:10 STEP  487 H= 121:45 +CPU= 16.046
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 2249MB (heap), 5413MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 2324 MB (maxheap), 5413 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         TRANSDIRH 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          TRANSDIR_MDL 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           DIR_TRANS 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            >TRS-LTDIR_CTL   - DIRECT ( 103) 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-LTDIR_CTL   - DIRECT (1645) 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              LTDIR_MOD 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010109:1676678469:25788.709] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               LE_DGEMM 
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010110:1676678470:25789.709] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:YatagarasuDrive:1:1:24337:24337] [20230218:010110:1676678470:25789.709] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001781D15  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000B38108  ledir_mod_mp_ledi         153  ledir_mod.F90
oifs_43r3_model.e  0000000000B2CC73  ltdir_mod_mp_ltdi         151  ltdir_mod.F90
oifs_43r3_model.e  0000000000B2C0F7  ltdir_ctl_mod_mp_          84  ltdir_ctl_mod.F90
oifs_43r3_model.e  0000000000B2BC68  dir_trans_ctl_mod         176  dir_trans_ctl_mod.F90
oifs_43r3_model.e  0000000000856971  dir_trans_                488  dir_trans.F90
oifs_43r3_model.e  000000000155BFF1  transdir_mdl_             148  transdir_mdl.F90
oifs_43r3_model.e  000000000155A450  transdirh_                 56  transdirh.F90
oifs_43r3_model.e  0000000000662213  stepo_                    375  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org