<core_client_version>7.16.16</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
cess
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:45:56 STEP 202 H= 202:00 +CPU= 22.704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:46:39 STEP 203 H= 203:00 +CPU= 21.492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:48:17 STEP 204 H= 204:00 +CPU= 36.887
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:49:11 STEP 205 H= 205:00 +CPU= 21.112
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:49:55 STEP 206 H= 206:00 +CPU= 22.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:50:55 STEP 207 H= 207:00 +CPU= 33.830
12:51:26 STEP 208 H= 208:00 +CPU= 22.387
12:51:56 STEP 209 H= 209:00 +CPU= 21.317
12:52:46 STEP 210 H= 210:00 +CPU= 34.570
12:53:16 STEP 211 H= 211:00 +CPU= 21.622
12:53:48 STEP 212 H= 212:00 +CPU= 22.562
12:54:36 STEP 213 H= 213:00 +CPU= 33.196
12:55:08 STEP 214 H= 214:00 +CPU= 22.419
12:55:39 STEP 215 H= 215:00 +CPU= 22.338
12:56:35 STEP 216 H= 216:00 +CPU= 39.053
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000204
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876_8.zip
Uploading the intermediate file: upload_file_8.zip
12:57:06 STEP 217 H= 217:00 +CPU= 21.994
12:57:38 STEP 218 H= 218:00 +CPU= 22.284
12:58:26 STEP 219 H= 219:00 +CPU= 33.699
12:58:57 STEP 220 H= 220:00 +CPU= 21.726
12:59:27 STEP 221 H= 221:00 +CPU= 21.804
13:00:15 STEP 222 H= 222:00 +CPU= 33.502
13:00:46 STEP 223 H= 223:00 +CPU= 21.153
13:01:17 STEP 224 H= 224:00 +CPU= 22.266
13:02:05 STEP 225 H= 225:00 +CPU= 33.635
13:02:37 STEP 226 H= 226:00 +CPU= 22.330
13:03:09 STEP 227 H= 227:00 +CPU= 21.978
13:04:02 STEP 228 H= 228:00 +CPU= 37.327
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000228
13:04:34 STEP 229 H= 229:00 +CPU= 22.224
13:05:05 STEP 230 H= 230:00 +CPU= 22.219
13:05:53 STEP 231 H= 231:00 +CPU= 34.055
13:06:25 STEP 232 H= 232:00 +CPU= 22.190
13:06:56 STEP 233 H= 233:00 +CPU= 22.062
13:07:46 STEP 234 H= 234:00 +CPU= 34.316
13:08:18 STEP 235 H= 235:00 +CPU= 21.877
13:08:49 STEP 236 H= 236:00 +CPU= 22.075
13:09:37 STEP 237 H= 237:00 +CPU= 33.712
13:10:09 STEP 238 H= 238:00 +CPU= 22.414
13:10:42 STEP 239 H= 239:00 +CPU= 22.748
13:11:38 STEP 240 H= 240:00 +CPU= 39.145
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000228
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876_9.zip
Uploading the intermediate file: upload_file_9.zip
Uploading trickle at timestep: 864000
13:12:09 STEP 241 H= 241:00 +CPU= 21.695
13:12:42 STEP 242 H= 242:00 +CPU= 22.602
13:13:30 STEP 243 H= 243:00 +CPU= 33.377
13:14:01 STEP 244 H= 244:00 +CPU= 21.826
13:14:32 STEP 245 H= 245:00 +CPU= 21.867
13:15:22 STEP 246 H= 246:00 +CPU= 34.501
13:15:53 STEP 247 H= 247:00 +CPU= 21.835
13:16:25 STEP 248 H= 248:00 +CPU= 22.298
13:17:13 STEP 249 H= 249:00 +CPU= 33.628
13:17:45 STEP 250 H= 250:00 +CPU= 22.553
13:18:16 STEP 251 H= 251:00 +CPU= 21.705
13:19:09 STEP 252 H= 252:00 +CPU= 37.085
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000252
13:19:40 STEP 253 H= 253:00 +CPU= 21.838
13:20:13 STEP 254 H= 254:00 +CPU= 22.775
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:21:11 STEP 255 H= 255:00 +CPU= 32.552
13:21:44 STEP 256 H= 256:00 +CPU= 22.522
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:22:36 STEP 257 H= 257:00 +CPU= 21.285
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:23:57 STEP 258 H= 258:00 +CPU= 32.934
13:24:28 STEP 259 H= 259:00 +CPU= 21.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:25:13 STEP 260 H= 260:00 +CPU= 22.369
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:26:23 STEP 261 H= 261:00 +CPU= 32.680
13:26:56 STEP 262 H= 262:00 +CPU= 22.561
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:27:38 STEP 263 H= 263:00 +CPU= 21.536
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:28:46 STEP 264 H= 264:00 +CPU= 39.175
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000252
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876_10.zip
Uploading the intermediate file: upload_file_10.zip
13:29:17 STEP 265 H= 265:00 +CPU= 21.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:30:00 STEP 266 H= 266:00 +CPU= 21.903
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:31:21 STEP 267 H= 267:00 +CPU= 32.905
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:32:16 STEP 268 H= 268:00 +CPU= 21.995
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:32:59 STEP 269 H= 269:00 +CPU= 21.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:34:35 STEP 270 H= 270:00 +CPU= 34.156
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:35:18 STEP 271 H= 271:00 +CPU= 21.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:36:12 STEP 272 H= 272:00 +CPU= 22.027
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:37:23 STEP 273 H= 273:00 +CPU= 34.005
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:38:17 STEP 274 H= 274:00 +CPU= 21.844
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:39:11 STEP 275 H= 275:00 +CPU= 21.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:40:37 STEP 276 H= 276:00 +CPU= 36.721
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000276
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:41:32 STEP 277 H= 277:00 +CPU= 21.971
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:42:15 STEP 278 H= 278:00 +CPU= 21.774
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:43:36 STEP 279 H= 279:00 +CPU= 32.830
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:44:42 STEP 280 H= 280:00 +CPU= 21.175
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:45:24 STEP 281 H= 281:00 +CPU= 21.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:46:35 STEP 282 H= 282:00 +CPU= 33.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:47:29 STEP 283 H= 283:00 +CPU= 20.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:48:34 STEP 284 H= 284:00 +CPU= 22.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:49:56 STEP 285 H= 285:00 +CPU= 32.756
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:50:49 STEP 286 H= 286:00 +CPU= 21.284
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:51:33 STEP 287 H= 287:00 +CPU= 22.196
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:53:28 STEP 288 H= 288:00 +CPU= 39.717
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876_11.zip
Uploading the intermediate file: upload_file_11.zip
13:53:52 STEP 289 H= 289:00 +CPU= 17.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:54:50 STEP 290 H= 290:00 +CPU= 21.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:56:13 STEP 291 H= 291:00 +CPU= 34.114
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:57:08 STEP 292 H= 292:00 +CPU= 22.485
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:57:52 STEP 293 H= 293:00 +CPU= 22.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:59:16 STEP 294 H= 294:00 +CPU= 34.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:00:20 STEP 295 H= 295:00 +CPU= 21.350
14:00:53 STEP 296 H= 296:00 +CPU= 22.896
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:01:52 STEP 297 H= 297:00 +CPU= 33.501
14:02:25 STEP 298 H= 298:00 +CPU= 22.750
14:02:57 STEP 299 H= 299:00 +CPU= 22.311
14:03:51 STEP 300 H= 300:00 +CPU= 37.778
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000300
14:04:22 STEP 301 H= 301:00 +CPU= 21.764
14:04:54 STEP 302 H= 302:00 +CPU= 22.528
14:05:44 STEP 303 H= 303:00 +CPU= 34.155
14:06:16 STEP 304 H= 304:00 +CPU= 22.752
14:06:47 STEP 305 H= 305:00 +CPU= 21.693
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:07:48 STEP 306 H= 306:00 +CPU= 34.319
14:08:19 STEP 307 H= 307:00 +CPU= 21.956
14:08:52 STEP 308 H= 308:00 +CPU= 22.883
14:09:40 STEP 309 H= 309:00 +CPU= 33.771
14:10:13 STEP 310 H= 310:00 +CPU= 22.417
14:10:45 STEP 311 H= 311:00 +CPU= 22.543
14:11:42 STEP 312 H= 312:00 +CPU= 39.422
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876_12.zip
Uploading the intermediate file: upload_file_12.zip
14:12:13 STEP 313 H= 313:00 +CPU= 21.498
14:12:44 STEP 314 H= 314:00 +CPU= 22.086
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:13:45 STEP 315 H= 315:00 +CPU= 33.141
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:14:39 STEP 316 H= 316:00 +CPU= 21.877
14:15:11 STEP 317 H= 317:00 +CPU= 21.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:16:22 STEP 318 H= 318:00 +CPU= 33.888
14:16:54 STEP 319 H= 319:00 +CPU= 21.977
14:17:26 STEP 320 H= 320:00 +CPU= 22.571
14:18:15 STEP 321 H= 321:00 +CPU= 33.258
14:18:48 STEP 322 H= 322:00 +CPU= 21.862
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:19:42 STEP 323 H= 323:00 +CPU= 21.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:21:20 STEP 324 H= 324:00 +CPU= 36.131
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000324
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:22:04 STEP 325 H= 325:00 +CPU= 21.212
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:23:10 STEP 326 H= 326:00 +CPU= 21.922
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:24:24 STEP 327 H= 327:00 +CPU= 33.542
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:25:29 STEP 328 H= 328:00 +CPU= 21.615
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:26:26 STEP 329 H= 329:00 +CPU= 21.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:28:00 STEP 330 H= 330:00 +CPU= 33.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:28:55 STEP 331 H= 331:00 +CPU= 21.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:30:01 STEP 332 H= 332:00 +CPU= 21.499
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:31:14 STEP 333 H= 333:00 +CPU= 33.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:32:09 STEP 334 H= 334:00 +CPU= 22.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:33:16 STEP 335 H= 335:00 +CPU= 22.531
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:35:01 STEP 336 H= 336:00 +CPU= 39.064
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876_13.zip
Uploading the intermediate file: upload_file_13.zip
14:35:32 STEP 337 H= 337:00 +CPU= 15.437
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:36:27 STEP 338 H= 338:00 +CPU= 22.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:37:48 STEP 339 H= 339:00 +CPU= 32.028
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:38:55 STEP 340 H= 340:00 +CPU= 22.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:39:48 STEP 341 H= 341:00 +CPU= 20.867
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:41:12 STEP 342 H= 342:00 +CPU= 34.020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:42:17 STEP 343 H= 343:00 +CPU= 20.582
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:43:12 STEP 344 H= 344:00 +CPU= 22.043
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:44:35 STEP 345 H= 345:00 +CPU= 32.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:45:41 STEP 346 H= 346:00 +CPU= 22.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:46:37 STEP 347 H= 347:00 +CPU= 22.047
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:48:17 STEP 348 H= 348:00 +CPU= 37.160
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000348
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:49:13 STEP 349 H= 349:00 +CPU= 22.046
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:50:08 STEP 350 H= 350:00 +CPU= 22.022
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:51:43 STEP 351 H= 351:00 +CPU= 33.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:52:39 STEP 352 H= 352:00 +CPU= 22.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:53:44 STEP 353 H= 353:00 +CPU= 21.547
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:55:18 STEP 354 H= 354:00 +CPU= 33.738
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:56:16 STEP 355 H= 355:00 +CPU= 22.526
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:57:22 STEP 356 H= 356:00 +CPU= 22.363
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:58:58 STEP 357 H= 357:00 +CPU= 33.523
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:59:54 STEP 358 H= 358:00 +CPU= 22.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:01:04 STEP 359 H= 359:00 +CPU= 23.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:03:00 STEP 360 H= 360:00 +CPU= 38.956
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876_14.zip
Uploading the intermediate file: upload_file_14.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:03:36 STEP 361 H= 361:00 +CPU= 19.957
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:04:46 STEP 362 H= 362:00 +CPU= 23.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:06:24 STEP 363 H= 363:00 +CPU= 33.890
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:07:33 STEP 364 H= 364:00 +CPU= 22.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:08:30 STEP 365 H= 365:00 +CPU= 21.924
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:10:19 STEP 366 H= 366:00 +CPU= 34.739
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:11:16 STEP 367 H= 367:00 +CPU= 22.739
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:12:12 STEP 368 H= 368:00 +CPU= 22.643
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:13:59 STEP 369 H= 369:00 +CPU= 34.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:15:07 STEP 370 H= 370:00 +CPU= 22.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:16:03 STEP 371 H= 371:00 +CPU= 22.380
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:17:44 STEP 372 H= 372:00 +CPU= 37.763
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:18:50 STEP 373 H= 373:00 +CPU= 22.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:19:58 STEP 374 H= 374:00 +CPU= 22.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:21:12 STEP 375 H= 375:00 +CPU= 34.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:22:20 STEP 376 H= 376:00 +CPU= 22.884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:23:06 STEP 377 H= 377:00 +CPU= 22.459
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:24:43 STEP 378 H= 378:00 +CPU= 33.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:25:38 STEP 379 H= 379:00 +CPU= 21.961
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:26:45 STEP 380 H= 380:00 +CPU= 22.772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:28:10 STEP 381 H= 381:00 +CPU= 34.165
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:29:04 STEP 382 H= 382:00 +CPU= 21.991
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:30:01 STEP 383 H= 383:00 +CPU= 23.008
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
ECCODES ERROR : IO ERROR: No space left on device (No space left on device)
GRIB_WRITE_BYTES_INT 50003 FAILED -11
GRIB_API ERROR MSG: Input output problem
MPL_ABORT: CALLED FROM PROCESSOR 1 THRD 1
MPL_ABORT: THRD 1 GRIB_WRITE_BYTES_REAL8 FAILED
[EC_DRHOOK:jesus:1:1:432774:432774] [20230113:153016:1673620216:17767.197] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3042 MB (maxheap), 4774 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:jesus:1:1:432774:432774] [20230113:153016:1673620216:17767.197] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:jesus:1:1:432774:432774] [20230113:153016:1673620216:17767.197] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:jesus:1:1:432774:432774] [20230113:153016:1673620216:17767.197](argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1991050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0284
(argv4) batchid: 960
(argv5) wuid: 12176928
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/7
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176928
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8b2aca672dded7171ad234992a4523d2 to: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0284_1991050100_123_960_12176928.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/7/oifs_43r3_ps_0284_1991050100_123_960_12176928.zip
ic_ancil_file: ic_ancil_12176928
ifsdata_file: ifsdata_12176928
climate_data_file: clim_data_12176928
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_46590604edb90055941dad89130ec82d to: /var/lib/boinc-client/slots/7/ic_ancil_12176928.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/7/ic_ancil_12176928.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_67d9ed5d60d33a9ea4c54425cf6d85e6 to: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12176928.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12176928.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/7/159l_2/clim_data_12176928.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/7/159l_2/clim_data_12176928.zip
Checking for progress XML file: /var/lib/boinc-client/slots/7/progress_file_12176928.xml
Opened progress file ok : /var/lib/boinc-client/slots/7/progress_file_12176928.xml
-- Model is restarting --
Adjusting last_iter, 383, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/7/progress_file_12176928.xml
last_cpu_time: 9724
upload_file_number: 15
last_iter: 359
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0284_1991050100_123_960_12176928_1_r2113239876
The child process has been launched with process id: 9078
Executing the command: /var/lib/boinc-client/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060051:1673672451:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
ABORT! 1 !! *** WAVE MODEL HAS ABORTED
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 1721 MB (maxheap), 3900 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WVCOUPLE
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WAVEMDL
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : INITMDL
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.420] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : USERIN
SDL_TRACEBACK: Calling INTEL_TRBK, THRD = 1
Process 1 thread 1 calling linux_trbk from intel_trbk()
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' : sigcontextptr=0x7ffdca2161a0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 19) with addr2line-cmd
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' 0x839273 0x839879 0x8c2957 0x81d89e 0x15f8aca 0x15d51c8 0x166b07f 0x15d7ff0 0x8711e0 0x7e0165 0x55e4ad 0x5471c9 0x546412 0x545e28 0x40708d 0x40220f 0x4021a2 0x1dca390 0x40206e
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x839273] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x839879] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x8c2957] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x81d89e] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x15f8aca] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x15d51c8] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x166b07f] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x15d7ff0] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0x8711e0] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0x7e0165] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0x55e4ad] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x5471c9] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x546412] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x545e28] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x40708d] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x40220f] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4021a2] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x1dca390] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x40206e] : ??() at ??:0
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
SDL_TRACEBACK: Done INTEL_TRBK, THRD = 1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.458] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#6 (SIGABRT) :: 1721MB (heap), 3900MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.458] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.458] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.458] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#6 was caused by unrecognized si_code [memaddr=0x8400002376], nsigs = 1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060251:1673672571:120.458] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#6, nsigs = 1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060252:1673672572:121.459] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#6, nsigs = 1
[EC_DRHOOK:jesus:1:1:9078:9078] [20230114:060252:1673672572:121.459] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#6, nsigs = 1
forrtl: error (76): Abort trap signal
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC903B Unknown Unknown Unknown
oifs_43r3_model.e 00000000008C2880 sdl_mod_mp_sdl_sr 100 sdl_mod.F90
oifs_43r3_model.e 000000000081D8AF abor1_ 41 abor1.F90
oifs_43r3_model.e 00000000015F8ACA mpabort_ 50 mpabort.F
oifs_43r3_model.e 00000000015D51C8 abort1_ 44 abort1.F
oifs_43r3_model.e 000000000166B07F userin_ 553 userin.F
oifs_43r3_model.e 00000000015D7FF0 initmdl_ 452 initmdl.F
oifs_43r3_model.e 00000000008711E0 wavemdl_ 393 wavemdl.F
oifs_43r3_model.e 00000000007E0165 wvcouple_ 260 wvcouple.F90
oifs_43r3_model.e 000000000055E4AD cnt4_ 1283 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
The child process terminated with status: 0
..Failed, model did not complete successfully
</stderr_txt>
]]>
