<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 2 (1005)
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAPINEB
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LARCINB
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAITRE_GMV
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230318:1673992998:2010.553] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAITRI
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230319:1673992999:2011.553] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:8489:8489] [20230117:230319:1673992999:2011.553] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000DD225E laitri_ 336 laitri.F90
oifs_43r3_model.e 0000000000DDBEE4 laitre_gmv_ 143 laitre_gmv.F90
oifs_43r3_model.e 0000000000DBEC9A larcinb_ 415 larcinb.F90
oifs_43r3_model.e 0000000000D7EBA8 lapineb_ 451 lapineb.F90
oifs_43r3_model.e 0000000000D7B935 call_sl_ 363 call_sl.F90
oifs_43r3_model.e 0000000000D784FD call_sl_heap_ 94 call_sl_heap.F90
oifs_43r3_model.e 0000000000BF9958 gp_model_ 533 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0916
(argv4) batchid: 954
(argv5) wuid: 12171560
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7bf0be468c0c1550d9ac062baac18b76 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0916_1985050100_123_954_12171560.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0916_1985050100_123_954_12171560.zip
ic_ancil_file: ic_ancil_12171560
ifsdata_file: ifsdata_12171560
climate_data_file: clim_data_12171560
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e1c417e049fcfa603a5281f825cf0831 to: /var/lib/boinc-client/slots/0/ic_ancil_12171560.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12171560.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1cd85c8b971f2cb9b200ffb8affd1433 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12171560.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12171560.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12171560.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12171560.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12171560.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12171560.xml
-- Model is restarting --
Adjusting last_iter, 978, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12171560.xml
last_cpu_time: 22725
upload_file_number: 40
last_iter: 959
last_upload: 3456000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0916_1985050100_123_954_12171560_0_r1928519026
The child process has been launched with process id: 2164
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:133454:1674045294:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:36:16 STEP 960 H= 960:00 +CPU= 31.939
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:36:46 STEP 961 H= 961:00 +CPU= 17.689
13:37:04 STEP 962 H= 962:00 +CPU= 17.566
13:37:33 STEP 963 H= 963:00 +CPU= 27.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:38:04 STEP 964 H= 964:00 +CPU= 19.093
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:38:33 STEP 965 H= 965:00 +CPU= 17.247
13:39:01 STEP 966 H= 966:00 +CPU= 27.455
13:39:19 STEP 967 H= 967:00 +CPU= 17.194
13:39:37 STEP 968 H= 968:00 +CPU= 17.683
13:40:05 STEP 969 H= 969:00 +CPU= 27.085
13:40:26 STEP 970 H= 970:00 +CPU= 19.610
13:40:44 STEP 971 H= 971:00 +CPU= 17.362
13:41:16 STEP 972 H= 972:00 +CPU= 30.430
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000972
13:41:34 STEP 973 H= 973:00 +CPU= 17.542
13:41:53 STEP 974 H= 974:00 +CPU= 18.001
13:42:22 STEP 975 H= 975:00 +CPU= 28.077
13:42:41 STEP 976 H= 976:00 +CPU= 18.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:43:12 STEP 977 H= 977:00 +CPU= 19.154
13:43:42 STEP 978 H= 978:00 +CPU= 28.123
13:44:00 STEP 979 H= 979:00 +CPU= 17.731
13:44:19 STEP 980 H= 980:00 +CPU= 18.374
13:44:48 STEP 981 H= 981:00 +CPU= 27.809
13:45:08 STEP 982 H= 982:00 +CPU= 18.430
13:45:28 STEP 983 H= 983:00 +CPU= 18.599
13:46:02 STEP 984 H= 984:00 +CPU= 32.671
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0916_1985050100_123_954_12171560_0_r1928519026_40.zip
Uploading the intermediate file: upload_file_40.zip
13:46:20 STEP 985 H= 985:00 +CPU= 17.599
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001)
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TURBULENCE_LAYER
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFOUTER
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFMAIN
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SURFEXCDRIVER
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134626:1674045986:692.054] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SURFEXCDRIVER_CTL_MOD:SURFEXCDRIVER_CTL
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134627:1674045987:693.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2164:2164] [20230118:134627:1674045987:693.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001CC9F7D Unknown Unknown Unknown
oifs_43r3_model.e 00000000017B8750 surfexcdriver_ctl 547 surfexcdriver_ctl_mod.F90
oifs_43r3_model.e 00000000015C619E surfexcdriver_ 847 surfexcdriver.F90
oifs_43r3_model.e 0000000000FA9554 vdfmain_ 738 vdfmain.F90
oifs_43r3_model.e 0000000000F92699 vdfouter_ 739 vdfouter.F90
oifs_43r3_model.e 0000000000F88512 turbulence_layer_ 158 turbulence_layer.F90
oifs_43r3_model.e 0000000000F0B683 callpar_ 869 callpar.F90
oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90
oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90
oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0916
(argv4) batchid: 954
(argv5) wuid: 12171560
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7bf0be468c0c1550d9ac062baac18b76 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0916_1985050100_123_954_12171560.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0916_1985050100_123_954_12171560.zip
ic_ancil_file: ic_ancil_12171560
ifsdata_file: ifsdata_12171560
climate_data_file: clim_data_12171560
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e1c417e049fcfa603a5281f825cf0831 to: /var/lib/boinc-client/slots/0/ic_ancil_12171560.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12171560.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1cd85c8b971f2cb9b200ffb8affd1433 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12171560.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12171560.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12171560.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12171560.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12171560.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12171560.xml
-- Model is restarting --
Adjusting last_iter, 984, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12171560.xml
last_cpu_time: 23275
upload_file_number: 40
last_iter: 983
last_upload: 3456000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0916_1985050100_123_954_12171560_0_r1928519026
The child process has been launched with process id: 2139
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:222803:1674077283:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:32:08 STEP 984 H= 984:00 +CPU= 37.163
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:32:41 STEP 985 H= 985:00 +CPU= 20.038
22:33:03 STEP 986 H= 986:00 +CPU= 19.767
22:33:35 STEP 987 H= 987:00 +CPU= 29.813
22:33:56 STEP 988 H= 988:00 +CPU= 19.833
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:34:28 STEP 989 H= 989:00 +CPU= 18.988
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:13 STEP 990 H= 990:00 +CPU= 31.148
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:54 STEP 991 H= 991:00 +CPU= 18.986
22:36:16 STEP 992 H= 992:00 +CPU= 19.953
22:36:48 STEP 993 H= 993:00 +CPU= 29.680
22:37:09 STEP 994 H= 994:00 +CPU= 19.424
22:37:30 STEP 995 H= 995:00 +CPU= 18.958
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:38:39 STEP 996 H= 996:00 +CPU= 33.792
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:39:12 STEP 997 H= 997:00 +CPU= 19.762
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:40:07 STEP 998 H= 998:00 +CPU= 21.327
22:40:39 STEP 999 H= 999:00 +CPU= 29.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:42:04 STEP 1000 H=1000:00 +CPU= 20.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:42:37 STEP 1001 H=1001:00 +CPU= 19.944
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:43:31 STEP 1002 H=1002:00 +CPU= 30.747
22:43:51 STEP 1003 H=1003:00 +CPU= 18.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:24 STEP 1004 H=1004:00 +CPU= 20.312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:45:07 STEP 1005 H=1005:00 +CPU= 29.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:45:51 STEP 1006 H=1006:00 +CPU= 19.974
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:46:23 STEP 1007 H=1007:00 +CPU= 19.752
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:47:12 STEP 1008 H=1008:00 +CPU= 34.869
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+000996
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0916_1985050100_123_954_12171560_0_r1928519026_41.zip
Uploading the intermediate file: upload_file_41.zip
22:47:34 STEP 1009 H=1009:00 +CPU= 20.350
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:48:16 STEP 1010 H=1010:00 +CPU= 20.247
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:50:04 STEP 1011 H=1011:00 +CPU= 31.371
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:50:36 STEP 1012 H=1012:00 +CPU= 19.808
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:20 STEP 1013 H=1013:00 +CPU= 20.389
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:38 STEP 1014 H=1014:00 +CPU= 31.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:54:22 STEP 1015 H=1015:00 +CPU= 19.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:55:57 STEP 1016 H=1016:00 +CPU= 20.520
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:04:49 STEP 1017 H=1017:00 +CPU= 32.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:05:22 STEP 1018 H=1018:00 +CPU= 20.601
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:05:54 STEP 1019 H=1019:00 +CPU= 19.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:06:51 STEP 1020 H=1020:00 +CPU= 33.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001020
23:07:23 STEP 1021 H=1021:00 +CPU= 19.144
23:07:44 STEP 1022 H=1022:00 +CPU= 19.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:09:10 STEP 1023 H=1023:00 +CPU= 30.625
23:09:31 STEP 1024 H=1024:00 +CPU= 19.453
23:09:52 STEP 1025 H=1025:00 +CPU= 19.056
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:10:36 STEP 1026 H=1026:00 +CPU= 31.142
23:10:57 STEP 1027 H=1027:00 +CPU= 19.215
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:11:59 STEP 1028 H=1028:00 +CPU= 19.739
23:12:30 STEP 1029 H=1029:00 +CPU= 29.511
23:12:51 STEP 1030 H=1030:00 +CPU= 19.224
23:13:12 STEP 1031 H=1031:00 +CPU= 19.336
23:13:49 STEP 1032 H=1032:00 +CPU= 34.546
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0916_1985050100_123_954_12171560_0_r1928519026_42.zip
Uploading the intermediate file: upload_file_42.zip
23:14:09 STEP 1033 H=1033:00 +CPU= 18.792
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:15:03 STEP 1034 H=1034:00 +CPU= 20.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:15:47 STEP 1035 H=1035:00 +CPU= 30.752
23:16:08 STEP 1036 H=1036:00 +CPU= 19.202
23:16:29 STEP 1037 H=1037:00 +CPU= 19.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:24:49 STEP 1038 H=1038:00 +CPU= 33.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:26:34 STEP 1039 H=1039:00 +CPU= 20.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:27:08 STEP 1040 H=1040:00 +CPU= 20.729
23:27:40 STEP 1041 H=1041:00 +CPU= 29.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:29:25 STEP 1042 H=1042:00 +CPU= 20.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:30:00 STEP 1043 H=1043:00 +CPU= 19.664
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233004:1674081004:3721.341] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 2553MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.355] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.355] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.355] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.355] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 2598 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TRANSINVH
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TRANSINV_MDL
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : INV_TRANS
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >TRS-FTINV_CTL - L TO G ( 157)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TRLTOG
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233005:1674081005:3721.356] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-TRLTOG LOCAL (1604)
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233006:1674081006:3722.357] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2139:2139] [20230118:233006:1674081006:3722.357] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000AD1BAC trltog_mod_mp_trl 639 trltog_mod.F90
oifs_43r3_model.e 0000000000ACF035 trltog_mod_mp_trl 88 trltog_mod.F90
oifs_43r3_model.e 0000000000ACB5A5 ftinv_ctl_mod_mp_ 258 ftinv_ctl_mod.F90
oifs_43r3_model.e 0000000000ACA619 inv_trans_ctl_mod 279 inv_trans_ctl_mod.F90
oifs_43r3_model.e 000000000085346D inv_trans_ 600 inv_trans.F90
oifs_43r3_model.e 000000000133D94C transinv_mdl_ 286 transinv_mdl.F90
oifs_43r3_model.e 0000000001334D43 transinvh_ 104 transinvh.F90
oifs_43r3_model.e 0000000000579C15 stepo_ 265 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0916
(argv4) batchid: 954
(argv5) wuid: 12171560
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7bf0be468c0c1550d9ac062baac18b76 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0916_1985050100_123_954_12171560.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0916_1985050100_123_954_12171560.zip
ic_ancil_file: ic_ancil_12171560
ifsdata_file: ifsdata_12171560
climate_data_file: clim_data_12171560
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e1c417e049fcfa603a5281f825cf0831 to: /var/lib/boinc-client/slots/0/ic_ancil_12171560.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12171560.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1cd85c8b971f2cb9b200ffb8affd1433 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12171560.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12171560.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12171560.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12171560.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12171560.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12171560.xml
-- Model is restarting --
Adjusting last_iter, 1044, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12171560.xml
last_cpu_time: 24705
upload_file_number: 43
last_iter: 1031
last_upload: 3715200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0916_1985050100_123_954_12171560_0_r1928519026
The child process has been launched with process id: 5009
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:5009:5009] [20230120:190331:1674237811:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
19:04:07 STEP 1032 H=1032:00 +CPU= 31.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:05:31 STEP 1033 H=1033:00 +CPU= 19.686
19:05:50 STEP 1034 H=1034:00 +CPU= 17.947
19:06:19 STEP 1035 H=1035:00 +CPU= 28.186
19:06:39 STEP 1036 H=1036:00 +CPU= 18.556
19:06:58 STEP 1037 H=1037:00 +CPU= 17.894
19:07:28 STEP 1038 H=1038:00 +CPU= 29.042
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:08:12 STEP 1039 H=1039:00 +CPU= 19.622
19:08:31 STEP 1040 H=1040:00 +CPU= 18.490
19:09:01 STEP 1041 H=1041:00 +CPU= 28.070
19:09:19 STEP 1042 H=1042:00 +CPU= 18.018
19:09:38 STEP 1043 H=1043:00 +CPU= 17.538
19:10:09 STEP 1044 H=1044:00 +CPU= 30.451
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001044
19:10:28 STEP 1045 H=1045:00 +CPU= 17.585
19:10:47 STEP 1046 H=1046:00 +CPU= 18.216
19:11:15 STEP 1047 H=1047:00 +CPU= 27.166
19:11:34 STEP 1048 H=1048:00 +CPU= 17.947
19:11:52 STEP 1049 H=1049:00 +CPU= 17.294
19:12:20 STEP 1050 H=1050:00 +CPU= 27.824
19:12:38 STEP 1051 H=1051:00 +CPU= 17.155
19:12:57 STEP 1052 H=1052:00 +CPU= 18.209
19:13:27 STEP 1053 H=1053:00 +CPU= 28.107
19:13:46 STEP 1054 H=1054:00 +CPU= 18.030
19:14:05 STEP 1055 H=1055:00 +CPU= 18.104
19:14:40 STEP 1056 H=1056:00 +CPU= 33.927
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171560/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0916_1985050100_123_954_12171560_0_r1928519026_43.zip
Uploading the intermediate file: upload_file_43.zip
19:15:22 STEP 1057 H=1057:00 +CPU= 18.636
19:15:43 STEP 1058 H=1058:00 +CPU= 19.178
19:16:15 STEP 1059 H=1059:00 +CPU= 28.783
19:16:35 STEP 1060 H=1060:00 +CPU= 18.783
19:16:55 STEP 1061 H=1061:00 +CPU= 18.438
Abort request received from BOINC client, ending the child process
</stderr_txt>
]]>
