<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ome/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231258:1669669978:2816.075] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231258:1669669978:2816.075] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231258:1669669978:2816.075] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 1 (1004)
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231258:1669669978:2816.075] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAPINEA
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231258:1669669978:2816.075] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LARCINA
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231258:1669669978:2816.075] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LASCAW
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231259:1669669979:2817.077] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:14330:14330] [20221128:231259:1669669979:2817.077] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
master.exe 0000000001CE916B Unknown Unknown Unknown
master.exe 000000000081FFF1 Unknown Unknown Unknown
master.exe 0000000001DC9090 Unknown Unknown Unknown
master.exe 0000000001DBAAFB Unknown Unknown Unknown
master.exe 0000000000DF1B71 lascaw_ 584 lascaw.F90
master.exe 0000000000DED525 larcina_ 319 larcina.F90
master.exe 0000000000DE60CD lapinea_ 325 lapinea.F90
master.exe 0000000000D7A4EC call_sl_ 300 call_sl.F90
master.exe 0000000000D784FD call_sl_heap_ 94 call_sl_heap.F90
master.exe 0000000000BF9958 gp_model_ 533 gp_model.F90
master.exe 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
master.exe 0000000000BBC1CE scan2m_ 535 scan2m.F90
master.exe 000000000057A664 stepo_ 327 stepo.F90
master.exe 000000000055DEEC cnt4_ 1133 cnt4.F90
master.exe 00000000005471C9 cnt3_ 267 cnt3.F90
master.exe 0000000000546412 cnt2_ 88 cnt2.F90
master.exe 0000000000545E28 cnt1_ 92 cnt1.F90
master.exe 000000000040708D cnt0_ 146 cnt0.F90
master.exe 000000000040220F MAIN__ 96 master.F90
master.exe 00000000004021A2 Unknown Unknown Unknown
master.exe 0000000001DCA390 Unknown Unknown Unknown
master.exe 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 0197
(argv4) batchid: 945
(argv5) wuid: 12163286
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_a00aa0e53e6d377d5cd4e9c7da3d9a80 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0197_2021050100_123_945_12163286.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0197_2021050100_123_945_12163286.zip
ic_ancil_file: ic_ancil_12163286
ifsdata_file: ifsdata_12163286
climate_data_file: clim_data_12163286
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_39ca5829c66ff679bedf1e3d1f9a9fb2 to: /var/lib/boinc-client/slots/4/ic_ancil_12163286.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12163286.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_4d8fe857d94c3a1e3826f7726a748bac to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163286.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163286.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163286.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163286.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435
The child process has been launched with process id: 2078
Executing the command: /var/lib/boinc-client/slots/4/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:060639:1669694799:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
06:07:09 STEP 96 H= 96:00 +CPU= 21.945
06:07:21 STEP 97 H= 97:00 +CPU= 11.965
06:07:34 STEP 98 H= 98:00 +CPU= 12.177
06:07:53 STEP 99 H= 99:00 +CPU= 19.904
06:08:05 STEP 100 H= 100:00 +CPU= 12.031
06:08:17 STEP 101 H= 101:00 +CPU= 11.549
06:08:43 STEP 102 H= 102:00 +CPU= 25.279
06:08:56 STEP 103 H= 103:00 +CPU= 13.077
06:09:08 STEP 104 H= 104:00 +CPU= 12.603
06:09:27 STEP 105 H= 105:00 +CPU= 19.132
06:09:40 STEP 106 H= 106:00 +CPU= 12.685
06:09:52 STEP 107 H= 107:00 +CPU= 11.309
06:10:14 STEP 108 H= 108:00 +CPU= 22.337
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000108
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000108
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000108
06:10:25 STEP 109 H= 109:00 +CPU= 11.325
06:10:38 STEP 110 H= 110:00 +CPU= 12.360
06:10:58 STEP 111 H= 111:00 +CPU= 20.185
06:11:11 STEP 112 H= 112:00 +CPU= 13.285
06:11:23 STEP 113 H= 113:00 +CPU= 11.866
06:11:44 STEP 114 H= 114:00 +CPU= 20.801
06:11:56 STEP 115 H= 115:00 +CPU= 12.003
06:12:08 STEP 116 H= 116:00 +CPU= 11.569
06:12:26 STEP 117 H= 117:00 +CPU= 18.368
06:12:38 STEP 118 H= 118:00 +CPU= 12.168
06:12:50 STEP 119 H= 119:00 +CPU= 11.549
06:13:11 STEP 120 H= 120:00 +CPU= 21.567
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000120
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000120
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000108
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_1.zip
Uploading the intermediate file: upload_file_1.zip
06:13:26 STEP 121 H= 121:00 +CPU= 14.539
06:13:40 STEP 122 H= 122:00 +CPU= 13.900
Uploading trickle at timestep: 432000
06:13:58 STEP 123 H= 123:00 +CPU= 18.500
06:14:17 STEP 124 H= 124:00 +CPU= 18.636
06:14:39 STEP 125 H= 125:00 +CPU= 21.603
06:15:12 STEP 126 H= 126:00 +CPU= 33.118
06:15:34 STEP 127 H= 127:00 +CPU= 21.957
06:15:57 STEP 128 H= 128:00 +CPU= 22.984
06:16:44 STEP 129 H= 129:00 +CPU= 47.400
06:17:15 STEP 130 H= 130:00 +CPU= 30.851
06:17:40 STEP 131 H= 131:00 +CPU= 24.899
06:18:22 STEP 132 H= 132:00 +CPU= 41.242
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000132
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000132
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000132
06:18:46 STEP 133 H= 133:00 +CPU= 24.184
06:19:10 STEP 134 H= 134:00 +CPU= 24.218
06:19:51 STEP 135 H= 135:00 +CPU= 40.779
06:20:19 STEP 136 H= 136:00 +CPU= 27.519
06:20:47 STEP 137 H= 137:00 +CPU= 28.690
06:21:31 STEP 138 H= 138:00 +CPU= 43.179
06:21:58 STEP 139 H= 139:00 +CPU= 27.699
06:22:27 STEP 140 H= 140:00 +CPU= 28.418
06:23:07 STEP 141 H= 141:00 +CPU= 39.739
06:23:35 STEP 142 H= 142:00 +CPU= 27.859
06:24:04 STEP 143 H= 143:00 +CPU= 28.783
06:24:53 STEP 144 H= 144:00 +CPU= 48.991
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000144
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000144
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000132
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_2.zip
Uploading the intermediate file: upload_file_2.zip
06:25:20 STEP 145 H= 145:00 +CPU= 27.144
Uploading trickle at timestep: 518400
06:25:49 STEP 146 H= 146:00 +CPU= 28.538
06:26:29 STEP 147 H= 147:00 +CPU= 40.837
06:26:57 STEP 148 H= 148:00 +CPU= 27.725
06:27:24 STEP 149 H= 149:00 +CPU= 26.994
06:28:07 STEP 150 H= 150:00 +CPU= 42.291
06:28:34 STEP 151 H= 151:00 +CPU= 27.388
06:29:02 STEP 152 H= 152:00 +CPU= 27.851
06:29:43 STEP 153 H= 153:00 +CPU= 40.588
06:30:11 STEP 154 H= 154:00 +CPU= 27.983
06:30:38 STEP 155 H= 155:00 +CPU= 27.263
06:31:23 STEP 156 H= 156:00 +CPU= 44.912
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000156
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000156
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000156
06:31:51 STEP 157 H= 157:00 +CPU= 27.718
06:32:19 STEP 158 H= 158:00 +CPU= 28.544
06:33:00 STEP 159 H= 159:00 +CPU= 40.956
06:33:28 STEP 160 H= 160:00 +CPU= 27.845
06:33:56 STEP 161 H= 161:00 +CPU= 27.296
06:34:38 STEP 162 H= 162:00 +CPU= 42.020
06:35:05 STEP 163 H= 163:00 +CPU= 27.330
06:35:33 STEP 164 H= 164:00 +CPU= 28.123
06:36:14 STEP 165 H= 165:00 +CPU= 41.201
06:36:43 STEP 166 H= 166:00 +CPU= 28.383
06:37:12 STEP 167 H= 167:00 +CPU= 28.733
06:38:00 STEP 168 H= 168:00 +CPU= 48.203
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000168
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000168
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000156
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_3.zip
Uploading the intermediate file: upload_file_3.zip
Uploading trickle at timestep: 604800
06:38:28 STEP 169 H= 169:00 +CPU= 27.673
06:38:56 STEP 170 H= 170:00 +CPU= 28.052
06:39:36 STEP 171 H= 171:00 +CPU= 40.081
06:40:04 STEP 172 H= 172:00 +CPU= 28.113
06:40:32 STEP 173 H= 173:00 +CPU= 28.150
06:41:15 STEP 174 H= 174:00 +CPU= 42.417
06:41:43 STEP 175 H= 175:00 +CPU= 27.686
06:42:11 STEP 176 H= 176:00 +CPU= 28.093
06:42:52 STEP 177 H= 177:00 +CPU= 40.649
06:43:22 STEP 178 H= 178:00 +CPU= 29.948
06:43:51 STEP 179 H= 179:00 +CPU= 29.113
06:44:38 STEP 180 H= 180:00 +CPU= 47.193
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000180
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000180
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000180
06:45:06 STEP 181 H= 181:00 +CPU= 28.280
06:45:35 STEP 182 H= 182:00 +CPU= 28.697
06:46:16 STEP 183 H= 183:00 +CPU= 41.038
06:46:44 STEP 184 H= 184:00 +CPU= 28.011
06:47:13 STEP 185 H= 185:00 +CPU= 28.200
06:47:55 STEP 186 H= 186:00 +CPU= 42.612
06:48:23 STEP 187 H= 187:00 +CPU= 27.500
06:48:52 STEP 188 H= 188:00 +CPU= 29.143
06:49:35 STEP 189 H= 189:00 +CPU= 42.785
06:50:03 STEP 190 H= 190:00 +CPU= 28.302
06:50:32 STEP 191 H= 191:00 +CPU= 28.531
06:51:22 STEP 192 H= 192:00 +CPU= 49.887
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000192
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000192
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000180
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_4.zip
Uploading the intermediate file: upload_file_4.zip
Uploading trickle at timestep: 691200
06:51:50 STEP 193 H= 193:00 +CPU= 27.597
06:52:20 STEP 194 H= 194:00 +CPU= 30.004
06:53:04 STEP 195 H= 195:00 +CPU= 44.148
06:53:33 STEP 196 H= 196:00 +CPU= 28.770
06:54:00 STEP 197 H= 197:00 +CPU= 27.427
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.581] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4035MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.582] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.582] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.582] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.582] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.582] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4035 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.582] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.583] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : RADDRV
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : RADINTG
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-RADINTG-RADLSW (1210)
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : RADIATION_SCHEME
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : radiation_interface:radiation
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065411:1669697651:2851.584] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : radiation_mcica_sw:solver_mcica_sw
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065412:1669697652:2852.585] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:2078:2078] [20221129:065412:1669697652:2852.585] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
master.exe 0000000001CE916B Unknown Unknown Unknown
master.exe 000000000081FFF1 Unknown Unknown Unknown
master.exe 0000000001DC9090 Unknown Unknown Unknown
master.exe 0000000001CC0C1C Unknown Unknown Unknown
master.exe 000000000188C13A radiation_two_str 382 radiation_two_stream.F90
master.exe 000000000196A573 radiation_mcica_s 259 radiation_mcica_sw.F90
master.exe 0000000001862DC8 radiation_interfa 419 radiation_interface.F90
master.exe 00000000012D30E5 radiation_scheme_ 484 radiation_scheme.F90
master.exe 000000000129A415 radintg_ 1455 radintg.F90
master.exe 0000000001270124 raddrv_ 537 raddrv.F90
master.exe 0000000000BFD839 gp_model_ 482 gp_model.F90
master.exe 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
master.exe 0000000000BBC1CE scan2m_ 535 scan2m.F90
master.exe 000000000057A664 stepo_ 327 stepo.F90
master.exe 000000000055DEEC cnt4_ 1133 cnt4.F90
master.exe 00000000005471C9 cnt3_ 267 cnt3.F90
master.exe 0000000000546412 cnt2_ 88 cnt2.F90
master.exe 0000000000545E28 cnt1_ 92 cnt1.F90
master.exe 000000000040708D cnt0_ 146 cnt0.F90
master.exe 000000000040220F MAIN__ 96 master.F90
master.exe 00000000004021A2 Unknown Unknown Unknown
master.exe 0000000001DCA390 Unknown Unknown Unknown
master.exe 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 0197
(argv4) batchid: 945
(argv5) wuid: 12163286
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_a00aa0e53e6d377d5cd4e9c7da3d9a80 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0197_2021050100_123_945_12163286.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0197_2021050100_123_945_12163286.zip
ic_ancil_file: ic_ancil_12163286
ifsdata_file: ifsdata_12163286
climate_data_file: clim_data_12163286
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_39ca5829c66ff679bedf1e3d1f9a9fb2 to: /var/lib/boinc-client/slots/4/ic_ancil_12163286.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12163286.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_4d8fe857d94c3a1e3826f7726a748bac to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163286.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163286.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163286.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163286.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435
The child process has been launched with process id: 1909
Executing the command: /var/lib/boinc-client/slots/4/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1909:1909] [20221129:184522:1669740322:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
18:46:12 STEP 192 H= 192:00 +CPU= 41.966
18:46:30 STEP 193 H= 193:00 +CPU= 17.948
18:46:43 STEP 194 H= 194:00 +CPU= 12.533
18:47:04 STEP 195 H= 195:00 +CPU= 21.206
18:47:18 STEP 196 H= 196:00 +CPU= 13.420
18:47:31 STEP 197 H= 197:00 +CPU= 13.196
18:47:52 STEP 198 H= 198:00 +CPU= 21.123
18:48:03 STEP 199 H= 199:00 +CPU= 11.290
18:48:16 STEP 200 H= 200:00 +CPU= 12.556
18:48:35 STEP 201 H= 201:00 +CPU= 19.308
18:48:48 STEP 202 H= 202:00 +CPU= 12.638
18:48:59 STEP 203 H= 203:00 +CPU= 11.270
18:49:20 STEP 204 H= 204:00 +CPU= 21.135
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000204
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000204
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000204
18:49:32 STEP 205 H= 205:00 +CPU= 12.165
18:49:45 STEP 206 H= 206:00 +CPU= 12.555
18:50:04 STEP 207 H= 207:00 +CPU= 19.377
18:50:16 STEP 208 H= 208:00 +CPU= 11.996
18:50:28 STEP 209 H= 209:00 +CPU= 11.731
18:50:47 STEP 210 H= 210:00 +CPU= 18.929
18:50:59 STEP 211 H= 211:00 +CPU= 12.396
18:51:11 STEP 212 H= 212:00 +CPU= 11.596
18:51:31 STEP 213 H= 213:00 +CPU= 19.664
18:51:43 STEP 214 H= 214:00 +CPU= 12.008
18:51:55 STEP 215 H= 215:00 +CPU= 12.132
18:52:18 STEP 216 H= 216:00 +CPU= 23.090
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000216
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000216
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000204
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_1.zip
Uploading the intermediate file: upload_file_1.zip
18:52:29 STEP 217 H= 217:00 +CPU= 11.449
18:52:41 STEP 218 H= 218:00 +CPU= 11.992
Uploading trickle at timestep: 777600
18:53:00 STEP 219 H= 219:00 +CPU= 18.254
18:53:12 STEP 220 H= 220:00 +CPU= 12.182
18:53:25 STEP 221 H= 221:00 +CPU= 13.181
18:53:51 STEP 222 H= 222:00 +CPU= 25.328
18:54:03 STEP 223 H= 223:00 +CPU= 12.612
18:54:18 STEP 224 H= 224:00 +CPU= 14.792
18:54:43 STEP 225 H= 225:00 +CPU= 24.839
18:54:59 STEP 226 H= 226:00 +CPU= 15.893
18:55:14 STEP 227 H= 227:00 +CPU= 15.657
18:55:42 STEP 228 H= 228:00 +CPU= 27.351
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000228
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000228
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000228
18:55:57 STEP 229 H= 229:00 +CPU= 15.495
18:56:13 STEP 230 H= 230:00 +CPU= 15.692
18:56:37 STEP 231 H= 231:00 +CPU= 24.446
18:56:53 STEP 232 H= 232:00 +CPU= 15.704
18:57:09 STEP 233 H= 233:00 +CPU= 15.827
18:57:34 STEP 234 H= 234:00 +CPU= 24.995
18:57:50 STEP 235 H= 235:00 +CPU= 15.708
18:58:06 STEP 236 H= 236:00 +CPU= 16.595
18:58:33 STEP 237 H= 237:00 +CPU= 26.847
18:58:50 STEP 238 H= 238:00 +CPU= 16.928
18:59:07 STEP 239 H= 239:00 +CPU= 17.133
18:59:36 STEP 240 H= 240:00 +CPU= 28.742
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000240
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000240
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000228
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_2.zip
Uploading the intermediate file: upload_file_2.zip
18:59:53 STEP 241 H= 241:00 +CPU= 16.573
Uploading trickle at timestep: 864000
19:00:10 STEP 242 H= 242:00 +CPU= 16.823
19:00:38 STEP 243 H= 243:00 +CPU= 28.103
19:00:54 STEP 244 H= 244:00 +CPU= 16.045
19:01:10 STEP 245 H= 245:00 +CPU= 15.667
19:01:35 STEP 246 H= 246:00 +CPU= 25.115
19:01:51 STEP 247 H= 247:00 +CPU= 16.099
19:02:07 STEP 248 H= 248:00 +CPU= 16.198
19:02:32 STEP 249 H= 249:00 +CPU= 24.453
19:02:48 STEP 250 H= 250:00 +CPU= 16.112
19:03:03 STEP 251 H= 251:00 +CPU= 15.246
19:03:31 STEP 252 H= 252:00 +CPU= 28.103
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000252
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000252
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000252
19:03:46 STEP 253 H= 253:00 +CPU= 15.419
19:04:02 STEP 254 H= 254:00 +CPU= 16.029
19:04:29 STEP 255 H= 255:00 +CPU= 26.229
19:04:46 STEP 256 H= 256:00 +CPU= 17.372
19:05:02 STEP 257 H= 257:00 +CPU= 15.895
19:05:28 STEP 258 H= 258:00 +CPU= 25.829
19:05:43 STEP 259 H= 259:00 +CPU= 15.551
19:05:59 STEP 260 H= 260:00 +CPU= 15.843
19:06:26 STEP 261 H= 261:00 +CPU= 26.905
19:06:44 STEP 262 H= 262:00 +CPU= 17.715
19:07:01 STEP 263 H= 263:00 +CPU= 17.328
19:07:33 STEP 264 H= 264:00 +CPU= 31.984
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000264
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000264
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000252
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_3.zip
Uploading the intermediate file: upload_file_3.zip
Uploading trickle at timestep: 950400
19:07:58 STEP 265 H= 265:00 +CPU= 24.674
19:08:33 STEP 266 H= 266:00 +CPU= 34.436
19:09:19 STEP 267 H= 267:00 +CPU= 46.439
19:09:52 STEP 268 H= 268:00 +CPU= 32.526
19:10:23 STEP 269 H= 269:00 +CPU= 30.731
19:11:10 STEP 270 H= 270:00 +CPU= 47.006
19:11:40 STEP 271 H= 271:00 +CPU= 30.079
19:12:11 STEP 272 H= 272:00 +CPU= 30.915
19:12:58 STEP 273 H= 273:00 +CPU= 46.856
19:13:32 STEP 274 H= 274:00 +CPU= 34.010
19:14:03 STEP 275 H= 275:00 +CPU= 30.695
19:14:56 STEP 276 H= 276:00 +CPU= 53.476
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000276
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000276
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000276
19:15:29 STEP 277 H= 277:00 +CPU= 32.746
19:16:03 STEP 278 H= 278:00 +CPU= 33.504
19:16:52 STEP 279 H= 279:00 +CPU= 49.120
19:17:26 STEP 280 H= 280:00 +CPU= 33.242
19:17:58 STEP 281 H= 281:00 +CPU= 32.059
19:18:47 STEP 282 H= 282:00 +CPU= 49.632
19:19:21 STEP 283 H= 283:00 +CPU= 33.092
19:19:53 STEP 284 H= 284:00 +CPU= 32.592
19:20:43 STEP 285 H= 285:00 +CPU= 49.943
19:21:17 STEP 286 H= 286:00 +CPU= 33.235
19:21:50 STEP 287 H= 287:00 +CPU= 33.090
19:22:47 STEP 288 H= 288:00 +CPU= 57.302
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000288
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000288
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_4.zip
Uploading the intermediate file: upload_file_4.zip
Uploading trickle at timestep: 1036800
19:23:21 STEP 289 H= 289:00 +CPU= 33.227
19:23:53 STEP 290 H= 290:00 +CPU= 32.341
19:24:42 STEP 291 H= 291:00 +CPU= 48.603
19:25:15 STEP 292 H= 292:00 +CPU= 33.229
19:25:47 STEP 293 H= 293:00 +CPU= 32.131
19:26:37 STEP 294 H= 294:00 +CPU= 49.106
19:27:10 STEP 295 H= 295:00 +CPU= 33.016
19:27:44 STEP 296 H= 296:00 +CPU= 33.702
19:28:34 STEP 297 H= 297:00 +CPU= 49.840
19:29:07 STEP 298 H= 298:00 +CPU= 33.287
19:29:39 STEP 299 H= 299:00 +CPU= 32.366
19:30:33 STEP 300 H= 300:00 +CPU= 53.923
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000300
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000300
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000300
19:31:06 STEP 301 H= 301:00 +CPU= 32.351
19:31:38 STEP 302 H= 302:00 +CPU= 32.398
19:32:27 STEP 303 H= 303:00 +CPU= 48.442
19:33:00 STEP 304 H= 304:00 +CPU= 32.991
19:33:32 STEP 305 H= 305:00 +CPU= 32.341
19:34:23 STEP 306 H= 306:00 +CPU= 50.681
19:34:56 STEP 307 H= 307:00 +CPU= 32.758
19:35:29 STEP 308 H= 308:00 +CPU= 33.131
19:36:18 STEP 309 H= 309:00 +CPU= 49.334
19:36:52 STEP 310 H= 310:00 +CPU= 33.768
19:37:26 STEP 311 H= 311:00 +CPU= 33.508
19:38:24 STEP 312 H= 312:00 +CPU= 57.902
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000312
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000312
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_5.zip
Uploading the intermediate file: upload_file_5.zip
Uploading trickle at timestep: 1123200
19:38:57 STEP 313 H= 313:00 +CPU= 32.823
19:39:31 STEP 314 H= 314:00 +CPU= 33.769
19:40:20 STEP 315 H= 315:00 +CPU= 49.171
19:40:54 STEP 316 H= 316:00 +CPU= 33.645
19:41:26 STEP 317 H= 317:00 +CPU= 32.237
19:42:16 STEP 318 H= 318:00 +CPU= 49.683
19:42:49 STEP 319 H= 319:00 +CPU= 32.857
19:43:22 STEP 320 H= 320:00 +CPU= 33.037
19:44:12 STEP 321 H= 321:00 +CPU= 49.795
19:44:46 STEP 322 H= 322:00 +CPU= 33.973
19:45:18 STEP 323 H= 323:00 +CPU= 32.240
19:46:13 STEP 324 H= 324:00 +CPU= 55.075
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000324
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000324
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000324
19:46:51 STEP 325 H= 325:00 +CPU= 37.433
19:47:27 STEP 326 H= 326:00 +CPU= 35.346
19:48:17 STEP 327 H= 327:00 +CPU= 49.508
19:48:51 STEP 328 H= 328:00 +CPU= 34.332
19:49:26 STEP 329 H= 329:00 +CPU= 34.358
19:50:15 STEP 330 H= 330:00 +CPU= 48.374
19:50:47 STEP 331 H= 331:00 +CPU= 32.292
19:51:22 STEP 332 H= 332:00 +CPU= 34.811
19:52:15 STEP 333 H= 333:00 +CPU= 51.122
19:52:47 STEP 334 H= 334:00 +CPU= 32.157
19:53:20 STEP 335 H= 335:00 +CPU= 32.905
19:54:16 STEP 336 H= 336:00 +CPU= 55.196
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000336
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000336
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_6.zip
Uploading the intermediate file: upload_file_6.zip
19:54:33 STEP 337 H= 337:00 +CPU= 17.452
Uploading trickle at timestep: 1209600
19:54:53 STEP 338 H= 338:00 +CPU= 18.416
19:55:18 STEP 339 H= 339:00 +CPU= 25.431
19:55:34 STEP 340 H= 340:00 +CPU= 15.822
19:55:51 STEP 341 H= 341:00 +CPU= 17.172
19:56:20 STEP 342 H= 342:00 +CPU= 28.114
19:56:38 STEP 343 H= 343:00 +CPU= 18.222
19:56:56 STEP 344 H= 344:00 +CPU= 17.559
19:57:23 STEP 345 H= 345:00 +CPU= 26.989
19:57:40 STEP 346 H= 346:00 +CPU= 17.222
19:57:57 STEP 347 H= 347:00 +CPU= 16.343
19:58:29 STEP 348 H= 348:00 +CPU= 32.800
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000348
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000348
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000348
19:58:47 STEP 349 H= 349:00 +CPU= 17.589
19:59:04 STEP 350 H= 350:00 +CPU= 16.796
19:59:30 STEP 351 H= 351:00 +CPU= 25.598
19:59:46 STEP 352 H= 352:00 +CPU= 16.278
20:00:03 STEP 353 H= 353:00 +CPU= 17.352
20:00:34 STEP 354 H= 354:00 +CPU= 30.145
20:00:50 STEP 355 H= 355:00 +CPU= 15.938
20:01:08 STEP 356 H= 356:00 +CPU= 18.355
20:01:36 STEP 357 H= 357:00 +CPU= 27.700
20:01:55 STEP 358 H= 358:00 +CPU= 18.985
20:02:13 STEP 359 H= 359:00 +CPU= 18.212
20:02:47 STEP 360 H= 360:00 +CPU= 33.847
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000360
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000360
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_7.zip
Uploading the intermediate file: upload_file_7.zip
20:03:04 STEP 361 H= 361:00 +CPU= 16.429
Uploading trickle at timestep: 1296000
20:03:21 STEP 362 H= 362:00 +CPU= 16.805
20:03:45 STEP 363 H= 363:00 +CPU= 24.838
20:04:03 STEP 364 H= 364:00 +CPU= 17.354
20:04:20 STEP 365 H= 365:00 +CPU= 17.433
20:04:48 STEP 366 H= 366:00 +CPU= 27.542
20:05:04 STEP 367 H= 367:00 +CPU= 16.557
20:05:21 STEP 368 H= 368:00 +CPU= 16.177
20:05:46 STEP 369 H= 369:00 +CPU= 25.048
20:06:03 STEP 370 H= 370:00 +CPU= 16.986
20:06:22 STEP 371 H= 371:00 +CPU= 19.069
20:06:55 STEP 372 H= 372:00 +CPU= 33.083
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000372
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000372
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000372
20:07:14 STEP 373 H= 373:00 +CPU= 19.202
20:07:32 STEP 374 H= 374:00 +CPU= 17.525
20:07:58 STEP 375 H= 375:00 +CPU= 25.812
20:08:15 STEP 376 H= 376:00 +CPU= 17.103
20:08:32 STEP 377 H= 377:00 +CPU= 16.835
20:08:59 STEP 378 H= 378:00 +CPU= 27.147
20:09:16 STEP 379 H= 379:00 +CPU= 17.504
20:09:34 STEP 380 H= 380:00 +CPU= 17.907
20:10:00 STEP 381 H= 381:00 +CPU= 25.965
20:10:17 STEP 382 H= 382:00 +CPU= 17.175
20:10:34 STEP 383 H= 383:00 +CPU= 16.756
20:11:08 STEP 384 H= 384:00 +CPU= 33.969
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000384
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000384
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_8.zip
Uploading the intermediate file: upload_file_8.zip
20:11:25 STEP 385 H= 385:00 +CPU= 16.554
Uploading trickle at timestep: 1382400
20:11:43 STEP 386 H= 386:00 +CPU= 17.823
20:12:11 STEP 387 H= 387:00 +CPU= 27.794
20:12:30 STEP 388 H= 388:00 +CPU= 18.872
20:12:46 STEP 389 H= 389:00 +CPU= 16.762
20:13:13 STEP 390 H= 390:00 +CPU= 26.311
20:13:29 STEP 391 H= 391:00 +CPU= 16.411
20:13:46 STEP 392 H= 392:00 +CPU= 16.578
20:14:14 STEP 393 H= 393:00 +CPU= 27.617
20:14:32 STEP 394 H= 394:00 +CPU= 17.842
20:14:47 STEP 395 H= 395:00 +CPU= 15.852
20:15:16 STEP 396 H= 396:00 +CPU= 28.991
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000396
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000396
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000396
20:15:33 STEP 397 H= 397:00 +CPU= 16.721
20:15:50 STEP 398 H= 398:00 +CPU= 16.536
20:16:16 STEP 399 H= 399:00 +CPU= 26.181
20:16:34 STEP 400 H= 400:00 +CPU= 18.179
20:17:03 STEP 401 H= 401:00 +CPU= 28.586
20:17:53 STEP 402 H= 402:00 +CPU= 49.570
20:18:25 STEP 403 H= 403:00 +CPU= 32.301
20:18:59 STEP 404 H= 404:00 +CPU= 33.769
20:19:49 STEP 405 H= 405:00 +CPU= 50.098
20:20:23 STEP 406 H= 406:00 +CPU= 33.886
20:20:55 STEP 407 H= 407:00 +CPU= 31.900
20:21:52 STEP 408 H= 408:00 +CPU= 56.006
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000408
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000408
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_9.zip
Uploading the intermediate file: upload_file_9.zip
Uploading trickle at timestep: 1468800
20:22:26 STEP 409 H= 409:00 +CPU= 34.140
20:23:00 STEP 410 H= 410:00 +CPU= 34.044
20:23:51 STEP 411 H= 411:00 +CPU= 50.186
20:24:23 STEP 412 H= 412:00 +CPU= 32.019
20:24:55 STEP 413 H= 413:00 +CPU= 32.148
20:25:49 STEP 414 H= 414:00 +CPU= 53.229
20:26:24 STEP 415 H= 415:00 +CPU= 34.504
20:26:58 STEP 416 H= 416:00 +CPU= 33.663
20:27:44 STEP 417 H= 417:00 +CPU= 46.771
20:28:18 STEP 418 H= 418:00 +CPU= 33.461
20:28:51 STEP 419 H= 419:00 +CPU= 32.392
20:29:48 STEP 420 H= 420:00 +CPU= 56.667
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000420
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000420
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000420
20:30:23 STEP 421 H= 421:00 +CPU= 34.955
20:31:00 STEP 422 H= 422:00 +CPU= 37.285
20:31:51 STEP 423 H= 423:00 +CPU= 50.926
20:32:27 STEP 424 H= 424:00 +CPU= 35.748
20:33:02 STEP 425 H= 425:00 +CPU= 34.581
20:33:53 STEP 426 H= 426:00 +CPU= 50.382
20:34:23 STEP 427 H= 427:00 +CPU= 30.725
20:34:56 STEP 428 H= 428:00 +CPU= 32.124
20:35:48 STEP 429 H= 429:00 +CPU= 51.769
20:36:22 STEP 430 H= 430:00 +CPU= 34.510
20:37:01 STEP 431 H= 431:00 +CPU= 38.048
20:38:12 STEP 432 H= 432:00 +CPU= 71.039
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000432
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000432
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMGGhpi1+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMSHhpi1+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163286/ICMUAhpi1+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0197_2021050100_123_945_12163286_0_r1556521435_10.zip
Uploading the intermediate file: upload_file_10.zip
Uploading trickle at timestep: 1555200
20:38:54 STEP 433 H= 433:00 +CPU= 41.140
20:39:34 STEP 434 H= 434:00 +CPU= 40.016
20:40:35 STEP 435 H= 435:00 +CPU= 60.186
20:41:16 STEP 436 H= 436:00 +CPU= 40.726
20:41:52 STEP 437 H= 437:00 +CPU= 36.426
20:42:53 STEP 438 H= 438:00 +CPU= 60.575
20:43:33 STEP 439 H= 439:00 +CPU= 39.655
20:44:15 STEP 440 H= 440:00 +CPU= 42.205
20:45:15 STEP 441 H= 441:00 +CPU= 59.526
20:45:54 STEP 442 H= 442:00 +CPU= 38.460
20:46:28 STEP 443 H= 443:00 +CPU= 33.919
20:47:25 STEP 444 H= 444:00 +CPU= 56.275
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000444
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000444
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000444
20:47:59 STEP 445 H= 445:00 +CPU= 33.842
20:48:32 STEP 446 H= 446:00 +CPU= 33.224
20:49:18 STEP 447 H= 447:00 +CPU= 45.648
20:49:53 STEP 448 H= 448:00 +CPU= 34.749
20:50:27 STEP 449 H= 449:00 +CPU= 33.359
The child process terminated with status: 0
..Failed, model did not complete successfully
</stderr_txt>
]]>
