climateprediction.net home page
Task 22449371

Task 22449371

Name oifs_43r3_bl_a29h_2016092300_20_1018_12291260_1
Workunit 12291260
Created 16 Jun 2024, 5:41:03 UTC
Sent 16 Jun 2024, 5:43:26 UTC
Report deadline 15 Aug 2024, 5:43:26 UTC
Received 16 Jun 2024, 13:48:54 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1539256
Run time 6 hours 39 min 32 sec
CPU time 5 hours 55 min 26 sec
Validate state Invalid
Credit 1,318.46
Device peak FLOPS 7.36 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,025.08 MB
Peak swap size 5,981.06 MB
Peak disk usage 1,900.46 MB
Stderr
<core_client_version>7.16.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001092
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001104
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001104
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001104
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001116
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001116
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001116
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001128
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001128
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001128
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001140
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001140
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001140
Zipping up the intermediate file: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_a29h_2016092300_20_1018_12291260_1_r999332426_11.zip
Uploading the intermediate file: upload_file_11.zip
  08:26:34 STEP 1153 H= 288:15 +CPU= 16.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:26:52 STEP 1154 H= 288:30 +CPU= 16.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:27:13 STEP 1155 H= 288:45 +CPU= 16.377
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:27:35 STEP 1156 H= 289:00 +CPU= 17.553
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:27:55 STEP 1157 H= 289:15 +CPU= 16.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:28:17 STEP 1158 H= 289:30 +CPU= 17.247
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:28:40 STEP 1159 H= 289:45 +CPU= 17.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:29:03 STEP 1160 H= 290:00 +CPU= 18.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:29:24 STEP 1161 H= 290:15 +CPU= 17.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:29:45 STEP 1162 H= 290:30 +CPU= 17.446
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:30:05 STEP 1163 H= 290:45 +CPU= 17.254
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:30:33 STEP 1164 H= 291:00 +CPU= 22.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001164
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001164
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:30:55 STEP 1165 H= 291:15 +CPU= 17.437
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:31:16 STEP 1166 H= 291:30 +CPU= 17.354
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:31:38 STEP 1167 H= 291:45 +CPU= 17.312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:32:01 STEP 1168 H= 292:00 +CPU= 17.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:32:22 STEP 1169 H= 292:15 +CPU= 17.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:32:44 STEP 1170 H= 292:30 +CPU= 17.139
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:33:04 STEP 1171 H= 292:45 +CPU= 16.733
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:33:26 STEP 1172 H= 293:00 +CPU= 17.334
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:33:47 STEP 1173 H= 293:15 +CPU= 16.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:34:07 STEP 1174 H= 293:30 +CPU= 16.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:34:28 STEP 1175 H= 293:45 +CPU= 16.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:34:55 STEP 1176 H= 294:00 +CPU= 21.332
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001176
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001176
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001176
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:35:16 STEP 1177 H= 294:15 +CPU= 16.585
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:35:37 STEP 1178 H= 294:30 +CPU= 16.442
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:35:57 STEP 1179 H= 294:45 +CPU= 16.534
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:36:22 STEP 1180 H= 295:00 +CPU= 18.590
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:36:44 STEP 1181 H= 295:15 +CPU= 17.928
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:37:06 STEP 1182 H= 295:30 +CPU= 17.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:37:27 STEP 1183 H= 295:45 +CPU= 17.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:37:51 STEP 1184 H= 296:00 +CPU= 17.967
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:38:13 STEP 1185 H= 296:15 +CPU= 17.846
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:38:35 STEP 1186 H= 296:30 +CPU= 17.284
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:38:56 STEP 1187 H= 296:45 +CPU= 17.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:39:25 STEP 1188 H= 297:00 +CPU= 22.414
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001188
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001188
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001188
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:39:46 STEP 1189 H= 297:15 +CPU= 17.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:40:07 STEP 1190 H= 297:30 +CPU= 17.278
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:40:30 STEP 1191 H= 297:45 +CPU= 16.700
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:40:52 STEP 1192 H= 298:00 +CPU= 17.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:41:14 STEP 1193 H= 298:15 +CPU= 17.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:41:35 STEP 1194 H= 298:30 +CPU= 17.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:41:56 STEP 1195 H= 298:45 +CPU= 16.604
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:42:17 STEP 1196 H= 299:00 +CPU= 17.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:42:37 STEP 1197 H= 299:15 +CPU= 16.625
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:42:58 STEP 1198 H= 299:30 +CPU= 16.433
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:43:21 STEP 1199 H= 299:45 +CPU= 17.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:43:52 STEP 1200 H= 300:00 +CPU= 24.563
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001200
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001200
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:44:14 STEP 1201 H= 300:15 +CPU= 17.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:44:36 STEP 1202 H= 300:30 +CPU= 17.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:44:57 STEP 1203 H= 300:45 +CPU= 17.525
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:45:21 STEP 1204 H= 301:00 +CPU= 18.190
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:45:43 STEP 1205 H= 301:15 +CPU= 17.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:46:04 STEP 1206 H= 301:30 +CPU= 17.290
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:46:26 STEP 1207 H= 301:45 +CPU= 17.532
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:46:48 STEP 1208 H= 302:00 +CPU= 17.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:47:11 STEP 1209 H= 302:15 +CPU= 17.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:47:32 STEP 1210 H= 302:30 +CPU= 17.200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:47:53 STEP 1211 H= 302:45 +CPU= 16.808
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:48:21 STEP 1212 H= 303:00 +CPU= 22.151
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001212
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001212
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001212
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:48:42 STEP 1213 H= 303:15 +CPU= 17.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:49:03 STEP 1214 H= 303:30 +CPU= 16.874
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:49:24 STEP 1215 H= 303:45 +CPU= 16.878
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:49:45 STEP 1216 H= 304:00 +CPU= 17.093
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:50:06 STEP 1217 H= 304:15 +CPU= 16.509
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:50:26 STEP 1218 H= 304:30 +CPU= 16.407
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:50:47 STEP 1219 H= 304:45 +CPU= 16.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:51:12 STEP 1220 H= 305:00 +CPU= 18.673
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:51:34 STEP 1221 H= 305:15 +CPU= 18.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:51:56 STEP 1222 H= 305:30 +CPU= 17.748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:52:17 STEP 1223 H= 305:45 +CPU= 17.597
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:52:46 STEP 1224 H= 306:00 +CPU= 22.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001224
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001224
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:53:07 STEP 1225 H= 306:15 +CPU= 17.635
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:53:31 STEP 1226 H= 306:30 +CPU= 17.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:53:52 STEP 1227 H= 306:45 +CPU= 17.419
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:54:14 STEP 1228 H= 307:00 +CPU= 18.088
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:54:36 STEP 1229 H= 307:15 +CPU= 17.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:54:57 STEP 1230 H= 307:30 +CPU= 17.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:55:20 STEP 1231 H= 307:45 +CPU= 16.791
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:55:41 STEP 1232 H= 308:00 +CPU= 17.384
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:56:03 STEP 1233 H= 308:15 +CPU= 17.042
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:56:23 STEP 1234 H= 308:30 +CPU= 16.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:56:45 STEP 1235 H= 308:45 +CPU= 17.456
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:12 STEP 1236 H= 309:00 +CPU= 21.469
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001236
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001236
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001236
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:33 STEP 1237 H= 309:15 +CPU= 16.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:54 STEP 1238 H= 309:30 +CPU= 16.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:58:15 STEP 1239 H= 309:45 +CPU= 16.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:58:36 STEP 1240 H= 310:00 +CPU= 17.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:58:59 STEP 1241 H= 310:15 +CPU= 17.099
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:59:21 STEP 1242 H= 310:30 +CPU= 17.966
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:59:43 STEP 1243 H= 310:45 +CPU= 17.905
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:00:06 STEP 1244 H= 311:00 +CPU= 18.433
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:00:28 STEP 1245 H= 311:15 +CPU= 17.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:00:51 STEP 1246 H= 311:30 +CPU= 17.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:01:13 STEP 1247 H= 311:45 +CPU= 17.879
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:01:44 STEP 1248 H= 312:00 +CPU= 24.896
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001248
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001248
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001248
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001152
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001152
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001152
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001164
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001164
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001164
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001176
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001176
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001176
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001188
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001188
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001188
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001200
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001200
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001200
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001212
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001212
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001212
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001224
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001224
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001224
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMGGh7zg+001236
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMSHh7zg+001236
Adding to the zip: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_12291260/ICMUAh7zg+001236
Zipping up the intermediate file: /home/mike/BOINC/projects/climateprediction.net/oifs_43r3_bl_a29h_2016092300_20_1018_12291260_1_r999332426_12.zip
Uploading the intermediate file: upload_file_12.zip
  09:02:03 STEP 1249 H= 312:15 +CPU= 17.323
  09:02:20 STEP 1250 H= 312:30 +CPU= 16.840
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:02:42 STEP 1251 H= 312:45 +CPU= 17.557
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:03:04 STEP 1252 H= 313:00 +CPU= 17.826
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:03:25 STEP 1253 H= 313:15 +CPU= 17.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:03:46 STEP 1254 H= 313:30 +CPU= 17.029
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:04:08 STEP 1255 H= 313:45 +CPU= 17.385
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:04:31 STEP 1256 H= 314:00 +CPU= 17.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:04:52 STEP 1257 H= 314:15 +CPU= 16.966
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:05:13 STEP 1258 H= 314:30 +CPU= 16.591
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:05:34 STEP 1259 H= 314:45 +CPU= 16.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:06:01 STEP 1260 H= 315:00 +CPU= 21.407
Moving to projects directory: /home/mike/BOINC/slots/18/ICMGGh7zg+001260
Moving to projects directory: /home/mike/BOINC/slots/18/ICMSHh7zg+001260
Moving to projects directory: /home/mike/BOINC/slots/18/ICMUAh7zg+001260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:06:21 STEP 1261 H= 315:15 +CPU= 16.433
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:06:42 STEP 1262 H= 315:30 +CPU= 16.760
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:07:04 STEP 1263 H= 315:45 +CPU= 17.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:07:26 STEP 1264 H= 316:00 +CPU= 18.238
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:07:50 STEP 1265 H= 316:15 +CPU= 17.877
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:08:12 STEP 1266 H= 316:30 +CPU= 17.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:08:34 STEP 1267 H= 316:45 +CPU= 17.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:08:56 STEP 1268 H= 317:00 +CPU= 18.145
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:09:20 STEP 1269 H= 317:15 +CPU= 17.757
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:09:42 STEP 1270 H= 317:30 +CPU= 17.506
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:10:03 STEP 1271 H= 317:45 +CPU= 17.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4412MB (heap), 5025MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4412 MB (maxheap), 5025 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS CLDPP T/S    (1002) 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091024:1718543424:26567.869] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/home/mike/BOINC/slots/18/oifs_43r3_model.exe' : sigcontextptr=0x7ffd5a11aad0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/home/mike/BOINC/slots/18/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.873] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/home/mike/BOINC/slots/18/oifs_43r3_model.exe' : sigcontextptr=0x7ffd5a11a9e0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/home/mike/BOINC/slots/18/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.895] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:Test101:1:1:171292:171292] [20240616:091025:1718543425:26568.895] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC6692  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000121FD8B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  0000000001207E29  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000011FDCB2  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000FAC003  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

ISTEP=    1272: Reset RSOLINC= 1361.4482 RIP0= 1361.4482
 SU_GHGCLIM, RCARDI,RCH4,RN2O,RCFC11,RCFC12,RNO2:   6.175516699499999E-004
  1.087291259320000E-006  4.989444016862500E-007  3.778881179906250E-009
  2.099153872040000E-009  7.940283000000001E-011
 suecaec NAERMACC =  1
 UPDT RAESHDU =  2000.0
 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT)     20160923
         450  9.999999999999998E-005
 NORMS AT NSTEP CNT4 1272
 SPECTRAL NORMS -   LOG(PREHYDS)    0.115261410439046E+02
 LEV     VORTICITY             DIVERGENCE           TEMPERATURE           KINETIC ENERGY    
 AVE 0.305895985342915E-04 0.296453008234809E-04 0.205985346169569E+03 0.201552735330908E+03
 GPNORM  HUMIDITY            AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.301601904677835E-02 0.999999999849727E-08 0.367427809924798E-01
 GPNORM  LIQUID WATER        AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.265369096761682E-05 0.000000000000000E+00 0.543294445296626E-02
 GPNORM  ICE WATER           AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.106192970431868E-04 0.000000000000000E+00 0.838327734720097E-02
 GPNORM SNOW                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.366531445977500E-04 0.000000000000000E+00 0.159449839454278E-01
 GPNORM RAIN                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.435380111703975E-05 0.000000000000000E+00 0.566032900795453E-02
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.234601119215561E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.265272113671923E-05 0.000000000000000E+00 0.514380368780614E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.106432526877981E-04 0.000000000000000E+00 0.795641471675806E-02
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.234191664929923E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.163319557376178E-04 0.000000000000000E+00 0.199778093325935E-01
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.123726296478180E-04 0.000000000000000E+00 0.614577994967247E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.399050185265547E-05 0.000000000000000E+00 0.957131714076222E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.158302408019670E-05 0.000000000000000E+00 0.723278691463359E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.140716028409376E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =  1272 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1272 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001272 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+001272
 MPI-TASK:   1 -    318325236 BYTES IN      6 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0080( 3.7%)
 NSTEP =  1272 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+001272 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+001272
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0156 SECONDS 45346.1 Mbytes/s, TOTAL TIME=    0.2154( 7.2%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001272 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+001272
 MPI-TASK:   1 -   5093203776 BYTES IN    125 RECORDS TRANSFERRED IN    0.0050 SECONDS******** Mbytes/s, TOTAL TIME=    0.1732( 2.9%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001272 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001272
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS     0.1 Mbytes/s, TOTAL TIME=    0.0031( 3.2%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001272 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001272
 MPI-TASK:   1 -    309745507 BYTES IN      4 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0030( 3.3%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =  1272 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 09:09:42 0AAA00AAA STEPO     1271    17.506   17.506   21.557    354:50    442:05 0.28740667313981E-04       2GB       0MB
 09:10:03 A00000000 STEPO     1272    17.486   17.486   21.530    355:08    442:27 0.29645300823481E-04       2GB       0MB
 09:10:03 0AA000000 STEPO     1272     0.020    0.020    0.020    355:08    442:27 0.29645300823481E-04       2GB       0MB
 09:10:04 FULLPOS-B DYNFPOS   1272     1.044    1.044    1.049    355:09    442:28 0.29645300823481E-04       2GB       0MB
 09:10:07 FULLPOS-V DYNFPOS   1272     2.562    2.562    2.574    355:11    442:31 0.29645300823481E-04       2GB       0MB
 09:10:07 FULLPOS-T DYNFPOS   1272     0.465    0.465    0.468    355:12    442:31 0.29645300823481E-04       2GB       0MB
 09:10:08 0AAA00AAA STEPO     1272     0.370    0.370    0.370    355:12    442:31 0.29645300823481E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    1248",
 CTIME   = "00130000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,        1236, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /home/mike/BOINC/slots/18/progress_file_12291260.xml

<running_values>
  <last_cpu_time>21324.750000</last_cpu_time>
  <upload_file_number>13</upload_file_number>
  <last_iter>1272</last_iter>
  <last_upload>1123200</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
16 Jun 2024 11:48:28 1539256 22449371 oifs_43r3_bl_a29h_2016092300_20_1018_12291260_1 864,000 15,807 0.0183


©2024 climateprediction.net