climateprediction.net home page
Task 22313113

Task 22313113

Name oifs_43r3_bl_a3y9_2016092300_15_991_12211908_1
Workunit 12211908
Created 13 Feb 2023, 23:08:20 UTC
Sent 14 Feb 2023, 8:39:05 UTC
Report deadline 15 Apr 2023, 8:39:05 UTC
Received 15 Feb 2023, 4:34:23 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1499094
Run time 5 hours 43 min 15 sec
CPU time 5 hours 37 min 54 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 5.80 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,047.25 MB
Peak swap size 5,930.79 MB
Peak disk usage 2,031.77 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000420
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000432
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000432
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000432
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000444
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000444
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000444
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000456
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000456
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000456
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000468
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000468
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000468
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3y9_2016092300_15_991_12211908_1_r1016158472_4.zip
Uploading the intermediate file: upload_file_4.zip
  21:08:22 STEP  481 H= 120:15 +CPU= 16.907
  21:08:39 STEP  482 H= 120:30 +CPU= 16.835
  21:08:56 STEP  483 H= 120:45 +CPU= 16.882
  21:09:13 STEP  484 H= 121:00 +CPU= 17.587
  21:09:31 STEP  485 H= 121:15 +CPU= 17.056
  21:09:48 STEP  486 H= 121:30 +CPU= 17.027
  21:10:05 STEP  487 H= 121:45 +CPU= 16.961
  21:10:23 STEP  488 H= 122:00 +CPU= 17.435
  21:10:40 STEP  489 H= 122:15 +CPU= 16.809
  21:10:57 STEP  490 H= 122:30 +CPU= 16.969
  21:11:14 STEP  491 H= 122:45 +CPU= 16.985
  21:11:35 STEP  492 H= 123:00 +CPU= 21.243
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000492
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000492
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000492
  21:11:52 STEP  493 H= 123:15 +CPU= 16.951
  21:12:09 STEP  494 H= 123:30 +CPU= 16.948
  21:12:26 STEP  495 H= 123:45 +CPU= 17.021
  21:12:44 STEP  496 H= 124:00 +CPU= 17.500
  21:13:01 STEP  497 H= 124:15 +CPU= 17.054
  21:13:19 STEP  498 H= 124:30 +CPU= 17.131
  21:13:36 STEP  499 H= 124:45 +CPU= 17.111
  21:13:54 STEP  500 H= 125:00 +CPU= 17.601
  21:14:11 STEP  501 H= 125:15 +CPU= 17.104
  21:14:28 STEP  502 H= 125:30 +CPU= 17.171
  21:14:45 STEP  503 H= 125:45 +CPU= 17.141
  21:15:07 STEP  504 H= 126:00 +CPU= 21.664
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000504
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000504
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000504
  21:15:24 STEP  505 H= 126:15 +CPU= 17.111
  21:15:42 STEP  506 H= 126:30 +CPU= 17.106
  21:15:59 STEP  507 H= 126:45 +CPU= 17.064
  21:16:17 STEP  508 H= 127:00 +CPU= 17.505
  21:16:34 STEP  509 H= 127:15 +CPU= 17.087
  21:16:51 STEP  510 H= 127:30 +CPU= 17.046
  21:17:08 STEP  511 H= 127:45 +CPU= 17.057
  21:17:26 STEP  512 H= 128:00 +CPU= 17.563
  21:17:43 STEP  513 H= 128:15 +CPU= 17.026
  21:18:00 STEP  514 H= 128:30 +CPU= 16.984
  21:18:17 STEP  515 H= 128:45 +CPU= 17.064
  21:18:39 STEP  516 H= 129:00 +CPU= 21.563
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000516
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000516
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000516
  21:18:56 STEP  517 H= 129:15 +CPU= 17.176
  21:19:14 STEP  518 H= 129:30 +CPU= 17.176
  21:19:31 STEP  519 H= 129:45 +CPU= 17.119
  21:19:49 STEP  520 H= 130:00 +CPU= 17.541
  21:20:06 STEP  521 H= 130:15 +CPU= 17.184
  21:20:23 STEP  522 H= 130:30 +CPU= 17.177
  21:20:40 STEP  523 H= 130:45 +CPU= 17.034
  21:20:58 STEP  524 H= 131:00 +CPU= 17.459
  21:21:15 STEP  525 H= 131:15 +CPU= 17.106
  21:21:32 STEP  526 H= 131:30 +CPU= 17.169
  21:21:50 STEP  527 H= 131:45 +CPU= 17.166
  21:22:14 STEP  528 H= 132:00 +CPU= 24.116
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000528
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000528
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000528
  21:22:31 STEP  529 H= 132:15 +CPU= 17.160
  21:22:49 STEP  530 H= 132:30 +CPU= 17.117
  21:23:06 STEP  531 H= 132:45 +CPU= 17.137
  21:23:24 STEP  532 H= 133:00 +CPU= 17.603
  21:23:41 STEP  533 H= 133:15 +CPU= 17.153
  21:23:58 STEP  534 H= 133:30 +CPU= 17.059
  21:24:15 STEP  535 H= 133:45 +CPU= 17.145
  21:24:33 STEP  536 H= 134:00 +CPU= 17.613
  21:24:50 STEP  537 H= 134:15 +CPU= 17.143
  21:25:08 STEP  538 H= 134:30 +CPU= 17.123
  21:25:25 STEP  539 H= 134:45 +CPU= 17.235
  21:25:47 STEP  540 H= 135:00 +CPU= 21.662
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000540
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000540
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000540
  21:26:04 STEP  541 H= 135:15 +CPU= 16.972
  21:26:21 STEP  542 H= 135:30 +CPU= 17.079
  21:26:38 STEP  543 H= 135:45 +CPU= 16.996
  21:26:56 STEP  544 H= 136:00 +CPU= 17.549
  21:27:13 STEP  545 H= 136:15 +CPU= 17.254
  21:27:31 STEP  546 H= 136:30 +CPU= 17.134
  21:27:48 STEP  547 H= 136:45 +CPU= 17.094
  21:28:06 STEP  548 H= 137:00 +CPU= 17.573
  21:28:23 STEP  549 H= 137:15 +CPU= 17.253
  21:28:40 STEP  550 H= 137:30 +CPU= 17.188
  21:28:58 STEP  551 H= 137:45 +CPU= 17.229
  21:29:19 STEP  552 H= 138:00 +CPU= 21.708
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000552
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000552
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000552
  21:29:37 STEP  553 H= 138:15 +CPU= 17.286
  21:29:54 STEP  554 H= 138:30 +CPU= 17.203
  21:30:12 STEP  555 H= 138:45 +CPU= 17.268
  21:30:29 STEP  556 H= 139:00 +CPU= 17.485
  21:30:46 STEP  557 H= 139:15 +CPU= 17.045
  21:31:04 STEP  558 H= 139:30 +CPU= 17.195
  21:31:21 STEP  559 H= 139:45 +CPU= 17.192
  21:31:39 STEP  560 H= 140:00 +CPU= 17.491
  21:31:56 STEP  561 H= 140:15 +CPU= 17.084
  21:32:13 STEP  562 H= 140:30 +CPU= 17.156
  21:32:30 STEP  563 H= 140:45 +CPU= 17.139
  21:32:52 STEP  564 H= 141:00 +CPU= 21.773
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000564
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000564
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000564
  21:33:10 STEP  565 H= 141:15 +CPU= 17.224
  21:33:27 STEP  566 H= 141:30 +CPU= 17.217
  21:33:44 STEP  567 H= 141:45 +CPU= 17.266
  21:34:02 STEP  568 H= 142:00 +CPU= 17.664
  21:34:20 STEP  569 H= 142:15 +CPU= 17.136
  21:34:37 STEP  570 H= 142:30 +CPU= 17.372
  21:34:54 STEP  571 H= 142:45 +CPU= 17.265
  21:35:12 STEP  572 H= 143:00 +CPU= 17.772
  21:35:30 STEP  573 H= 143:15 +CPU= 17.313
  21:35:47 STEP  574 H= 143:30 +CPU= 17.310
  21:36:05 STEP  575 H= 143:45 +CPU= 17.238
  21:36:29 STEP  576 H= 144:00 +CPU= 24.255
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000576
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000576
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000480
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000480
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000480
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000492
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000492
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000492
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000504
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000504
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000504
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000516
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000516
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000516
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000528
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000528
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000528
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000540
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000540
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000540
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000552
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000552
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000552
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000564
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000564
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000564
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3y9_2016092300_15_991_12211908_1_r1016158472_5.zip
Uploading the intermediate file: upload_file_5.zip
  21:36:47 STEP  577 H= 144:15 +CPU= 17.271
  21:37:04 STEP  578 H= 144:30 +CPU= 17.382
  21:37:21 STEP  579 H= 144:45 +CPU= 17.257
  21:37:39 STEP  580 H= 145:00 +CPU= 17.754
  21:37:57 STEP  581 H= 145:15 +CPU= 17.366
  21:38:14 STEP  582 H= 145:30 +CPU= 17.315
  21:38:32 STEP  583 H= 145:45 +CPU= 17.234
  21:38:50 STEP  584 H= 146:00 +CPU= 17.751
  21:39:07 STEP  585 H= 146:15 +CPU= 17.459
  21:39:25 STEP  586 H= 146:30 +CPU= 17.274
  21:39:42 STEP  587 H= 146:45 +CPU= 17.317
  21:40:04 STEP  588 H= 147:00 +CPU= 21.460
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000588
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000588
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000588
  21:40:21 STEP  589 H= 147:15 +CPU= 17.258
  21:40:39 STEP  590 H= 147:30 +CPU= 17.293
  21:40:56 STEP  591 H= 147:45 +CPU= 17.335
  21:41:14 STEP  592 H= 148:00 +CPU= 17.652
  21:41:31 STEP  593 H= 148:15 +CPU= 17.309
  21:41:49 STEP  594 H= 148:30 +CPU= 17.370
  21:42:06 STEP  595 H= 148:45 +CPU= 17.364
  21:42:24 STEP  596 H= 149:00 +CPU= 17.788
  21:42:42 STEP  597 H= 149:15 +CPU= 17.281
  21:42:59 STEP  598 H= 149:30 +CPU= 17.313
  21:43:16 STEP  599 H= 149:45 +CPU= 17.223
  21:43:38 STEP  600 H= 150:00 +CPU= 21.640
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000600
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000600
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000600
  21:43:56 STEP  601 H= 150:15 +CPU= 17.243
  21:44:13 STEP  602 H= 150:30 +CPU= 17.398
  21:44:31 STEP  603 H= 150:45 +CPU= 17.145
  21:44:48 STEP  604 H= 151:00 +CPU= 17.588
  21:45:06 STEP  605 H= 151:15 +CPU= 17.297
  21:45:23 STEP  606 H= 151:30 +CPU= 17.251
  21:45:41 STEP  607 H= 151:45 +CPU= 17.334
  21:45:59 STEP  608 H= 152:00 +CPU= 17.852
  21:46:16 STEP  609 H= 152:15 +CPU= 17.404
  21:46:34 STEP  610 H= 152:30 +CPU= 17.334
  21:46:51 STEP  611 H= 152:45 +CPU= 17.292
  21:47:13 STEP  612 H= 153:00 +CPU= 21.939
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000612
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000612
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000612
  21:47:30 STEP  613 H= 153:15 +CPU= 17.046
  21:47:48 STEP  614 H= 153:30 +CPU= 17.186
  21:48:05 STEP  615 H= 153:45 +CPU= 17.439
  21:48:23 STEP  616 H= 154:00 +CPU= 17.776
  21:48:41 STEP  617 H= 154:15 +CPU= 17.409
  21:48:58 STEP  618 H= 154:30 +CPU= 17.366
  21:49:16 STEP  619 H= 154:45 +CPU= 17.230
  21:49:33 STEP  620 H= 155:00 +CPU= 17.583
  21:49:51 STEP  621 H= 155:15 +CPU= 17.233
  21:50:08 STEP  622 H= 155:30 +CPU= 17.284
  21:50:26 STEP  623 H= 155:45 +CPU= 17.332
  21:50:50 STEP  624 H= 156:00 +CPU= 24.496
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000624
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000624
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000624
  21:51:08 STEP  625 H= 156:15 +CPU= 17.213
  21:51:25 STEP  626 H= 156:30 +CPU= 17.311
  21:51:43 STEP  627 H= 156:45 +CPU= 17.329
  21:52:01 STEP  628 H= 157:00 +CPU= 17.811
  21:52:18 STEP  629 H= 157:15 +CPU= 17.316
  21:52:35 STEP  630 H= 157:30 +CPU= 17.299
  21:52:53 STEP  631 H= 157:45 +CPU= 17.147
  21:53:10 STEP  632 H= 158:00 +CPU= 17.600
  21:53:28 STEP  633 H= 158:15 +CPU= 17.250
  21:53:45 STEP  634 H= 158:30 +CPU= 17.332
  21:54:03 STEP  635 H= 158:45 +CPU= 17.423
  21:54:25 STEP  636 H= 159:00 +CPU= 21.798
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000636
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000636
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000636
  21:54:42 STEP  637 H= 159:15 +CPU= 17.392
  21:55:00 STEP  638 H= 159:30 +CPU= 17.399
  21:55:17 STEP  639 H= 159:45 +CPU= 17.335
  21:55:35 STEP  640 H= 160:00 +CPU= 17.859
  21:55:53 STEP  641 H= 160:15 +CPU= 17.363
  21:56:10 STEP  642 H= 160:30 +CPU= 17.360
  21:56:28 STEP  643 H= 160:45 +CPU= 17.337
  21:56:46 STEP  644 H= 161:00 +CPU= 17.589
  21:57:03 STEP  645 H= 161:15 +CPU= 17.196
  21:57:20 STEP  646 H= 161:30 +CPU= 17.253
  21:57:38 STEP  647 H= 161:45 +CPU= 17.487
  21:58:00 STEP  648 H= 162:00 +CPU= 21.774
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000648
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000648
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000648
  21:58:18 STEP  649 H= 162:15 +CPU= 17.412
  21:58:35 STEP  650 H= 162:30 +CPU= 17.422
  21:58:53 STEP  651 H= 162:45 +CPU= 17.400
  21:59:11 STEP  652 H= 163:00 +CPU= 17.877
  21:59:28 STEP  653 H= 163:15 +CPU= 17.449
  21:59:46 STEP  654 H= 163:30 +CPU= 17.483
  22:00:03 STEP  655 H= 163:45 +CPU= 17.402
  22:00:21 STEP  656 H= 164:00 +CPU= 17.790
  22:00:39 STEP  657 H= 164:15 +CPU= 17.235
  22:00:56 STEP  658 H= 164:30 +CPU= 17.229
  22:01:14 STEP  659 H= 164:45 +CPU= 17.424
  22:01:35 STEP  660 H= 165:00 +CPU= 21.748
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000660
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000660
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000660
  22:01:53 STEP  661 H= 165:15 +CPU= 17.400
  22:02:11 STEP  662 H= 165:30 +CPU= 17.361
  22:02:28 STEP  663 H= 165:45 +CPU= 17.481
  22:02:46 STEP  664 H= 166:00 +CPU= 18.016
  22:03:04 STEP  665 H= 166:15 +CPU= 17.503
  22:03:22 STEP  666 H= 166:30 +CPU= 17.462
  22:03:39 STEP  667 H= 166:45 +CPU= 17.558
  22:03:57 STEP  668 H= 167:00 +CPU= 17.937
  22:04:15 STEP  669 H= 167:15 +CPU= 17.440
  22:04:32 STEP  670 H= 167:30 +CPU= 17.347
  22:04:50 STEP  671 H= 167:45 +CPU= 17.464
  22:05:15 STEP  672 H= 168:00 +CPU= 24.486
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000672
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000672
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000576
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000576
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000588
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000588
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000588
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000600
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000600
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000600
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000612
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000612
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000612
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000624
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000624
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000624
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000636
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000636
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000636
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000648
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000648
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000648
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000660
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000660
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000660
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3y9_2016092300_15_991_12211908_1_r1016158472_6.zip
Uploading the intermediate file: upload_file_6.zip
  22:05:33 STEP  673 H= 168:15 +CPU= 17.651
  22:05:50 STEP  674 H= 168:30 +CPU= 17.490
  22:06:08 STEP  675 H= 168:45 +CPU= 17.651
  22:06:26 STEP  676 H= 169:00 +CPU= 18.064
  22:06:44 STEP  677 H= 169:15 +CPU= 17.553
  22:07:02 STEP  678 H= 169:30 +CPU= 17.548
  22:07:19 STEP  679 H= 169:45 +CPU= 17.514
  22:07:37 STEP  680 H= 170:00 +CPU= 17.827
  22:07:55 STEP  681 H= 170:15 +CPU= 17.455
  22:08:13 STEP  682 H= 170:30 +CPU= 17.481
  22:08:30 STEP  683 H= 170:45 +CPU= 17.445
  22:08:52 STEP  684 H= 171:00 +CPU= 21.698
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000684
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000684
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000684
  22:09:10 STEP  685 H= 171:15 +CPU= 17.462
  22:09:27 STEP  686 H= 171:30 +CPU= 17.542
  22:09:45 STEP  687 H= 171:45 +CPU= 17.517
  22:10:03 STEP  688 H= 172:00 +CPU= 17.942
  22:10:21 STEP  689 H= 172:15 +CPU= 17.542
  22:10:38 STEP  690 H= 172:30 +CPU= 17.466
  22:10:56 STEP  691 H= 172:45 +CPU= 17.438
  22:11:14 STEP  692 H= 173:00 +CPU= 17.821
  22:11:31 STEP  693 H= 173:15 +CPU= 17.545
  22:11:49 STEP  694 H= 173:30 +CPU= 17.512
  22:12:07 STEP  695 H= 173:45 +CPU= 17.519
  22:12:29 STEP  696 H= 174:00 +CPU= 21.774
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000696
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000696
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000696
  22:12:46 STEP  697 H= 174:15 +CPU= 17.577
  22:13:04 STEP  698 H= 174:30 +CPU= 17.449
  22:13:22 STEP  699 H= 174:45 +CPU= 17.428
  22:13:40 STEP  700 H= 175:00 +CPU= 17.860
  22:13:57 STEP  701 H= 175:15 +CPU= 17.489
  22:14:15 STEP  702 H= 175:30 +CPU= 17.429
  22:14:32 STEP  703 H= 175:45 +CPU= 17.443
  22:14:50 STEP  704 H= 176:00 +CPU= 17.950
  22:15:08 STEP  705 H= 176:15 +CPU= 17.535
  22:15:26 STEP  706 H= 176:30 +CPU= 17.493
  22:15:43 STEP  707 H= 176:45 +CPU= 17.576
  22:16:05 STEP  708 H= 177:00 +CPU= 21.732
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000708
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000708
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000708
  22:16:23 STEP  709 H= 177:15 +CPU= 17.446
  22:16:40 STEP  710 H= 177:30 +CPU= 17.417
  22:16:58 STEP  711 H= 177:45 +CPU= 17.468
  22:17:16 STEP  712 H= 178:00 +CPU= 17.972
  22:17:34 STEP  713 H= 178:15 +CPU= 17.564
  22:17:51 STEP  714 H= 178:30 +CPU= 17.506
  22:18:09 STEP  715 H= 178:45 +CPU= 17.516
  22:18:27 STEP  716 H= 179:00 +CPU= 17.910
  22:18:45 STEP  717 H= 179:15 +CPU= 17.407
  22:19:02 STEP  718 H= 179:30 +CPU= 17.457
  22:19:20 STEP  719 H= 179:45 +CPU= 17.562
  22:19:45 STEP  720 H= 180:00 +CPU= 24.578
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000720
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000720
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000720
  22:20:03 STEP  721 H= 180:15 +CPU= 17.665
  22:20:20 STEP  722 H= 180:30 +CPU= 17.595
  22:20:38 STEP  723 H= 180:45 +CPU= 17.636
  22:20:56 STEP  724 H= 181:00 +CPU= 18.034
  22:21:14 STEP  725 H= 181:15 +CPU= 17.408
  22:21:31 STEP  726 H= 181:30 +CPU= 17.522
  22:21:49 STEP  727 H= 181:45 +CPU= 17.549
  22:22:07 STEP  728 H= 182:00 +CPU= 18.105
  22:22:25 STEP  729 H= 182:15 +CPU= 17.512
  22:22:43 STEP  730 H= 182:30 +CPU= 17.498
  22:23:00 STEP  731 H= 182:45 +CPU= 17.436
  22:23:22 STEP  732 H= 183:00 +CPU= 22.004
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000732
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000732
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000732
  22:23:40 STEP  733 H= 183:15 +CPU= 17.548
  22:23:58 STEP  734 H= 183:30 +CPU= 17.533
  22:24:15 STEP  735 H= 183:45 +CPU= 17.551
  22:24:33 STEP  736 H= 184:00 +CPU= 17.920
  22:24:51 STEP  737 H= 184:15 +CPU= 17.584
  22:25:09 STEP  738 H= 184:30 +CPU= 17.624
  22:25:27 STEP  739 H= 184:45 +CPU= 17.520
  22:25:45 STEP  740 H= 185:00 +CPU= 18.007
  22:26:02 STEP  741 H= 185:15 +CPU= 17.530
  22:26:20 STEP  742 H= 185:30 +CPU= 17.364
  22:26:37 STEP  743 H= 185:45 +CPU= 17.414
  22:27:00 STEP  744 H= 186:00 +CPU= 22.049
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000744
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000744
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000744
  22:27:17 STEP  745 H= 186:15 +CPU= 17.583
  22:27:35 STEP  746 H= 186:30 +CPU= 17.588
  22:27:53 STEP  747 H= 186:45 +CPU= 17.495
  22:28:11 STEP  748 H= 187:00 +CPU= 18.168
  22:28:29 STEP  749 H= 187:15 +CPU= 17.683
  22:28:47 STEP  750 H= 187:30 +CPU= 17.623
  22:29:04 STEP  751 H= 187:45 +CPU= 17.632
  22:29:23 STEP  752 H= 188:00 +CPU= 18.117
  22:29:41 STEP  753 H= 188:15 +CPU= 17.759
  22:29:58 STEP  754 H= 188:30 +CPU= 17.543
  22:30:16 STEP  755 H= 188:45 +CPU= 17.403
  22:30:38 STEP  756 H= 189:00 +CPU= 21.975
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000756
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000756
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000756
  22:30:56 STEP  757 H= 189:15 +CPU= 17.658
  22:31:14 STEP  758 H= 189:30 +CPU= 17.681
  22:31:31 STEP  759 H= 189:45 +CPU= 17.619
  22:31:50 STEP  760 H= 190:00 +CPU= 18.136
  22:32:08 STEP  761 H= 190:15 +CPU= 17.716
  22:32:25 STEP  762 H= 190:30 +CPU= 17.687
  22:32:43 STEP  763 H= 190:45 +CPU= 17.758
  22:33:02 STEP  764 H= 191:00 +CPU= 18.137
  22:33:19 STEP  765 H= 191:15 +CPU= 17.525
  22:33:37 STEP  766 H= 191:30 +CPU= 17.491
  22:33:55 STEP  767 H= 191:45 +CPU= 17.542
  22:34:20 STEP  768 H= 192:00 +CPU= 24.800
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000768
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000768
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000672
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000672
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000684
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000684
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000684
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000696
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000696
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000696
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000708
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000708
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000708
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000720
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000720
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000720
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000732
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000732
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000732
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000744
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000744
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000744
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000756
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000756
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000756
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3y9_2016092300_15_991_12211908_1_r1016158472_7.zip
Uploading the intermediate file: upload_file_7.zip
  22:34:37 STEP  769 H= 192:15 +CPU= 17.776
  22:34:55 STEP  770 H= 192:30 +CPU= 17.664
  22:35:13 STEP  771 H= 192:45 +CPU= 17.904
  22:35:32 STEP  772 H= 193:00 +CPU= 18.164
  22:35:50 STEP  773 H= 193:15 +CPU= 17.705
  22:36:07 STEP  774 H= 193:30 +CPU= 17.662
  22:36:25 STEP  775 H= 193:45 +CPU= 17.518
  22:36:43 STEP  776 H= 194:00 +CPU= 18.047
  22:37:01 STEP  777 H= 194:15 +CPU= 17.642
  22:37:19 STEP  778 H= 194:30 +CPU= 17.767
  22:37:37 STEP  779 H= 194:45 +CPU= 17.772
  22:37:59 STEP  780 H= 195:00 +CPU= 22.210
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000780
  22:38:17 STEP  781 H= 195:15 +CPU= 17.702
  22:38:35 STEP  782 H= 195:30 +CPU= 17.815
  22:38:53 STEP  783 H= 195:45 +CPU= 17.801
  22:39:11 STEP  784 H= 196:00 +CPU= 18.160
  22:39:29 STEP  785 H= 196:15 +CPU= 17.547
  22:39:47 STEP  786 H= 196:30 +CPU= 17.598
  22:40:04 STEP  787 H= 196:45 +CPU= 17.667
  22:40:23 STEP  788 H= 197:00 +CPU= 18.104
  22:40:41 STEP  789 H= 197:15 +CPU= 17.788
  22:40:59 STEP  790 H= 197:30 +CPU= 17.799
  22:41:16 STEP  791 H= 197:45 +CPU= 17.735
  22:41:39 STEP  792 H= 198:00 +CPU= 22.278
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000792
  22:41:57 STEP  793 H= 198:15 +CPU= 17.760
  22:42:15 STEP  794 H= 198:30 +CPU= 17.741
  22:42:33 STEP  795 H= 198:45 +CPU= 17.730
  22:42:51 STEP  796 H= 199:00 +CPU= 18.185
  22:43:09 STEP  797 H= 199:15 +CPU= 17.595
  22:43:26 STEP  798 H= 199:30 +CPU= 17.681
  22:43:44 STEP  799 H= 199:45 +CPU= 17.626
  22:44:02 STEP  800 H= 200:00 +CPU= 18.131
  22:44:20 STEP  801 H= 200:15 +CPU= 17.771
  22:44:38 STEP  802 H= 200:30 +CPU= 17.659
  22:44:56 STEP  803 H= 200:45 +CPU= 17.758
  22:45:18 STEP  804 H= 201:00 +CPU= 22.186
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000804
  22:45:36 STEP  805 H= 201:15 +CPU= 17.660
  22:45:54 STEP  806 H= 201:30 +CPU= 17.614
  22:46:12 STEP  807 H= 201:45 +CPU= 17.482
  22:46:30 STEP  808 H= 202:00 +CPU= 17.899
  22:46:47 STEP  809 H= 202:15 +CPU= 17.590
  22:47:05 STEP  810 H= 202:30 +CPU= 17.582
  22:47:23 STEP  811 H= 202:45 +CPU= 17.748
  22:47:41 STEP  812 H= 203:00 +CPU= 18.178
  22:47:59 STEP  813 H= 203:15 +CPU= 17.586
  22:48:17 STEP  814 H= 203:30 +CPU= 17.787
  22:48:35 STEP  815 H= 203:45 +CPU= 17.754
  22:49:00 STEP  816 H= 204:00 +CPU= 24.771
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000816
  22:49:17 STEP  817 H= 204:15 +CPU= 17.492
  22:49:35 STEP  818 H= 204:30 +CPU= 17.565
  22:49:53 STEP  819 H= 204:45 +CPU= 17.606
  22:50:11 STEP  820 H= 205:00 +CPU= 18.168
  22:50:29 STEP  821 H= 205:15 +CPU= 17.703
  22:50:47 STEP  822 H= 205:30 +CPU= 17.699
  22:51:05 STEP  823 H= 205:45 +CPU= 17.711
  22:51:23 STEP  824 H= 206:00 +CPU= 18.063
  22:51:41 STEP  825 H= 206:15 +CPU= 17.715
  22:51:59 STEP  826 H= 206:30 +CPU= 17.740
  22:52:16 STEP  827 H= 206:45 +CPU= 17.753
  22:52:39 STEP  828 H= 207:00 +CPU= 21.994
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000828
  22:52:56 STEP  829 H= 207:15 +CPU= 17.687
  22:53:14 STEP  830 H= 207:30 +CPU= 17.777
  22:53:32 STEP  831 H= 207:45 +CPU= 17.755
  22:53:51 STEP  832 H= 208:00 +CPU= 18.179
  22:54:08 STEP  833 H= 208:15 +CPU= 17.757
  22:54:26 STEP  834 H= 208:30 +CPU= 17.694
  22:54:44 STEP  835 H= 208:45 +CPU= 17.682
  22:55:02 STEP  836 H= 209:00 +CPU= 18.281
  22:55:20 STEP  837 H= 209:15 +CPU= 17.765
  22:55:38 STEP  838 H= 209:30 +CPU= 17.754
  22:55:56 STEP  839 H= 209:45 +CPU= 17.749
  22:56:18 STEP  840 H= 210:00 +CPU= 22.088
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000840
  22:56:36 STEP  841 H= 210:15 +CPU= 17.761
  22:56:54 STEP  842 H= 210:30 +CPU= 17.769
  22:57:12 STEP  843 H= 210:45 +CPU= 17.799
  22:57:31 STEP  844 H= 211:00 +CPU= 18.327
  22:57:49 STEP  845 H= 211:15 +CPU= 17.823
  22:58:06 STEP  846 H= 211:30 +CPU= 17.627
  22:58:24 STEP  847 H= 211:45 +CPU= 17.756
  22:58:43 STEP  848 H= 212:00 +CPU= 18.264
  22:59:01 STEP  849 H= 212:15 +CPU= 17.698
  22:59:18 STEP  850 H= 212:30 +CPU= 17.493
  22:59:36 STEP  851 H= 212:45 +CPU= 17.552
  22:59:58 STEP  852 H= 213:00 +CPU= 22.311
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000852
  23:00:16 STEP  853 H= 213:15 +CPU= 17.771
  23:00:34 STEP  854 H= 213:30 +CPU= 17.822
  23:00:52 STEP  855 H= 213:45 +CPU= 17.786
  23:01:10 STEP  856 H= 214:00 +CPU= 18.086
  23:01:28 STEP  857 H= 214:15 +CPU= 17.591
  23:01:46 STEP  858 H= 214:30 +CPU= 17.745
  23:02:04 STEP  859 H= 214:45 +CPU= 17.781
  23:02:22 STEP  860 H= 215:00 +CPU= 17.957
  23:02:40 STEP  861 H= 215:15 +CPU= 17.532
  23:02:58 STEP  862 H= 215:30 +CPU= 17.670
  23:03:15 STEP  863 H= 215:45 +CPU= 17.655
  23:03:40 STEP  864 H= 216:00 +CPU= 24.893
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000768
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000768
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211908/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3y9_2016092300_15_991_12211908_1_r1016158472_8.zip
Uploading the intermediate file: upload_file_8.zip
  23:03:58 STEP  865 H= 216:15 +CPU= 17.792
  23:04:16 STEP  866 H= 216:30 +CPU= 17.661
  23:04:34 STEP  867 H= 216:45 +CPU= 17.661
  23:04:52 STEP  868 H= 217:00 +CPU= 18.130
  23:05:10 STEP  869 H= 217:15 +CPU= 17.584
  23:05:28 STEP  870 H= 217:30 +CPU= 17.484
  23:05:45 STEP  871 H= 217:45 +CPU= 17.534
  23:06:04 STEP  872 H= 218:00 +CPU= 18.114
  23:06:21 STEP  873 H= 218:15 +CPU= 17.666
  23:06:39 STEP  874 H= 218:30 +CPU= 17.726
  23:06:57 STEP  875 H= 218:45 +CPU= 17.623
  23:07:20 STEP  876 H= 219:00 +CPU= 22.249
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000876
  23:07:37 STEP  877 H= 219:15 +CPU= 17.471
  23:07:55 STEP  878 H= 219:30 +CPU= 17.356
  23:08:12 STEP  879 H= 219:45 +CPU= 17.542
  23:08:31 STEP  880 H= 220:00 +CPU= 18.076
  23:08:48 STEP  881 H= 220:15 +CPU= 17.581
  23:09:06 STEP  882 H= 220:30 +CPU= 17.445
  23:09:23 STEP  883 H= 220:45 +CPU= 17.311
  23:09:41 STEP  884 H= 221:00 +CPU= 17.855
  23:09:59 STEP  885 H= 221:15 +CPU= 17.511
  23:10:17 STEP  886 H= 221:30 +CPU= 17.496
  23:10:34 STEP  887 H= 221:45 +CPU= 17.554
  23:10:56 STEP  888 H= 222:00 +CPU= 22.103
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000888
  23:11:14 STEP  889 H= 222:15 +CPU= 17.527
  23:11:32 STEP  890 H= 222:30 +CPU= 17.366
  23:11:49 STEP  891 H= 222:45 +CPU= 17.596
  23:12:07 STEP  892 H= 223:00 +CPU= 17.948
  23:12:25 STEP  893 H= 223:15 +CPU= 17.538
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.190] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4362MB (heap), 5055MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.190] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.190] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc4a8f], nsigs = 1
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 4362 MB (maxheap), 5055 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231236:1676434356:20531.191] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc/slots/1/oifs_43r3_model.exe' : sigcontextptr=0x7ffdb645c8d0
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc/slots/1/oifs_43r3_model.exe' 0x838803 0x838e09 0x1d8bb20 0x1cc4a8f 0x12d283b 0x12ba8d9 0x12b0762 0x1163fc3 0x10b682f 0x10a915c 0xe680c3 0x15c1e62 0x15ac40e 0x6620b4 0x4a0e7c 0x49dc29 0x49ce72 0x49cd68 0x4078ad 0x40276f 0x40270d 0x1d8287a 0x1d840d7 0x4025e5
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x838803] : LinuxTraceBack() at linuxtrbk.c:264
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x838e09] : LinuxTraceBack() at linuxtrbk.c:236
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1d8bb20] : __restore_rt() at libc_sigaction.o:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc4a8f] : __libm_exp_e7() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x12d283b] : vdfmain_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x12ba8d9] : vdfouter_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x12b0762] : turbulence_layer_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x1163fc3] : callpar_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0x10b682f] : ec_phys_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0x10a915c] : ec_phys_drv_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xe680c3] : gp_model_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x15c1e62] : gp_model_heap_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x15ac40e] : scan2m_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x6620b4] : stepo_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4a0e7c] : cnt4_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x49dc29] : cnt3_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x49ce72] : cnt2_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49cd68] : cnt1_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x4078ad] : cnt0_() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x40276f] : MAIN__() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x40270d] : main() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1d8287a] : __libc_start_call_main() at libc-start.o:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1d840d7] : __libc_start_main_impl() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x4025e5] : _start() at ??:?
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.191] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc/slots/1/oifs_43r3_model.exe' : sigcontextptr=0x7ffdb645c7e0
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc/slots/1/oifs_43r3_model.exe' 0x838803 0x838e09 0x820d70 0x1d8bb20 0x1cc4a8f 0x12d283b 0x12ba8d9 0x12b0762 0x1163fc3 0x10b682f 0x10a915c 0xe680c3 0x15c1e62 0x15ac40e 0x6620b4 0x4a0e7c 0x49dc29 0x49ce72 0x49cd68 0x4078ad 0x40276f 0x40270d 0x1d8287a 0x1d840d7 0x4025e5
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x838803] : LinuxTraceBack() at linuxtrbk.c:264
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x838e09] : LinuxTraceBack() at linuxtrbk.c:236
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x820d70] : signal_drhook() at drhook.c:1734
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1d8bb20] : __restore_rt() at libc_sigaction.o:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc4a8f] : __libm_exp_e7() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x12d283b] : vdfmain_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x12ba8d9] : vdfouter_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x12b0762] : turbulence_layer_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0x1163fc3] : callpar_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0x10b682f] : ec_phys_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0x10a915c] : ec_phys_drv_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xe680c3] : gp_model_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x15c1e62] : gp_model_heap_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x15ac40e] : scan2m_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x6620b4] : stepo_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4a0e7c] : cnt4_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x49dc29] : cnt3_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49ce72] : cnt2_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49cd68] : cnt1_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x4078ad] : cnt0_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x40276f] : MAIN__() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x40270d] : main() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1d8287a] : __libc_start_call_main() at libc-start.o:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1d840d7] : __libc_start_main_impl() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x4025e5] : _start() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.311] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:quietneshie.regency:1:1:47483:47483] [20230214:231237:1676434357:20532.311] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC4A8F  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000012D283B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  00000000012BA8D9  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000012B0762  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000001163FC3  callpar_                  869  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.166916467280377E-04 0.000000000000000E+00 0.104042466724241E-01
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.206380086911166E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.423930592272779E-05 0.000000000000000E+00 0.629702769861004E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.101478021612361E-04 0.000000000000000E+00 0.867712002728859E-02
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.207009886958651E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.539084233666024E-04 0.000000000000000E+00 0.300024485214976E-01
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.131674238883199E-04 0.000000000000000E+00 0.550058071588089E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.507047982178768E-05 0.000000000000000E+00 0.807570075375586E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.207842968009357E-06 0.000000000000000E+00 0.304857714043739E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.189894440369164E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   888 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   888 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000888 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000888
 MPI-TASK:   1 -    315564234 BYTES IN      6 RECORDS TRANSFERRED IN    0.0004 SECONDS788910.6 Mbytes/s, TOTAL TIME=    0.0065( 6.2%)
 NSTEP =   888 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000888 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000888
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0082 SECONDS 86312.8 Mbytes/s, TOTAL TIME=    0.1771( 4.6%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000888 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000888
 MPI-TASK:   1 -   5049027735 BYTES IN    125 RECORDS TRANSFERRED IN    0.0056 SECONDS901612.1 Mbytes/s, TOTAL TIME=    0.1510( 3.7%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000888 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000888
 MPI-TASK:   1 -    306990601 BYTES IN      2 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0027( 7.4%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000888 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000888
 MPI-TASK:   1 -    315564234 BYTES IN      4 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0041( 4.9%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   888 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   888 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000888 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000888
 MPI-TASK:   1 -    315564225 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS788910.6 Mbytes/s, TOTAL TIME=    0.0078( 5.1%)
 23:10:56 STEP  888 H= 222:00 +CPU= 22.103
 NSTEP =   889 STEPO   0AAA00AAA
 23:11:14 STEP  889 H= 222:15 +CPU= 17.527
ISTEP=     890 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     890 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   890 STEPO   0AAA00AAA
 23:11:32 STEP  890 H= 222:30 +CPU= 17.366
 NSTEP =   891 STEPO   0AAA00AAA
 23:11:49 STEP  891 H= 222:45 +CPU= 17.596
 NSTEP =   892 STEPO   0AAA00AAA
 23:12:07 STEP  892 H= 223:00 +CPU= 17.948
 NSTEP =   893 STEPO   0AAA00AAA
 23:12:25 STEP  893 H= 223:15 +CPU= 17.538
 NSTEP =   894 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 23:10:38 0AAA00AAA STEPO      888     0.434    0.434    0.437    335:57    340:13 0.41958363320686E-04       2GB       0MB
 23:10:56 0AAA00AAA STEPO      889    18.030   18.030   18.168    336:15    340:31 0.41958363320686E-04       2GB       0MB
 23:11:14 0AAA00AAA STEPO      890    17.527   17.527   17.659    336:32    340:49 0.41958363320686E-04       2GB       0MB
 23:11:32 0AAA00AAA STEPO      891    17.366   17.366   17.500    336:50    341:06 0.41958363320686E-04       2GB       0MB
 23:11:49 0AAA00AAA STEPO      892    17.596   17.596   17.738    337:07    341:24 0.41958363320686E-04       2GB       0MB
 23:12:07 0AAA00AAA STEPO      893    17.948   17.948   18.090    337:25    341:42 0.41958363320686E-04       2GB       0MB
 23:12:25 0AAA00AAA STEPO      894    17.538   17.538   17.673    337:43    342:00 0.41958363320686E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     864",
 CTIME   = "00090000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         852, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc/slots/1/progress_file_12211908.xml

<running_values>
  <last_cpu_time>20269.000000</last_cpu_time>
  <upload_file_number>9</upload_file_number>
  <last_iter>894</last_iter>
  <last_upload>777600</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org