climateprediction.net home page
Task 22251028

Task 22251028

Name oifs_43r3_ps_0173_1981050100_123_950_12166817_0
Workunit 12166817
Created 20 Dec 2022, 18:49:44 UTC
Sent 20 Dec 2022, 18:55:12 UTC
Report deadline 19 Jan 2023, 18:55:12 UTC
Received 26 Dec 2022, 8:07:10 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1533691
Run time 10 hours 37 min 22 sec
CPU time 10 hours 31 min
Validate state Invalid
Credit 4,838.40
Device peak FLOPS 3.91 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,403.00 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,190.17 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
= 536:00 +CPU= 27.081
  11:12:30 STEP  537 H= 537:00 +CPU= 41.837
  11:12:57 STEP  538 H= 538:00 +CPU= 26.855
  11:13:23 STEP  539 H= 539:00 +CPU= 25.854
  11:14:10 STEP  540 H= 540:00 +CPU= 46.628
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000540
  11:14:37 STEP  541 H= 541:00 +CPU= 26.822
  11:15:04 STEP  542 H= 542:00 +CPU= 26.972
  11:15:46 STEP  543 H= 543:00 +CPU= 42.404
  11:16:14 STEP  544 H= 544:00 +CPU= 27.618
  11:16:40 STEP  545 H= 545:00 +CPU= 26.392
  11:17:24 STEP  546 H= 546:00 +CPU= 43.270
  11:17:50 STEP  547 H= 547:00 +CPU= 26.566
  11:18:18 STEP  548 H= 548:00 +CPU= 27.503
  11:19:00 STEP  549 H= 549:00 +CPU= 41.897
  11:19:26 STEP  550 H= 550:00 +CPU= 26.406
  11:19:53 STEP  551 H= 551:00 +CPU= 27.009
  11:20:43 STEP  552 H= 552:00 +CPU= 48.980
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_22.zip
Uploading the intermediate file: upload_file_22.zip
  11:21:10 STEP  553 H= 553:00 +CPU= 26.887
  11:21:37 STEP  554 H= 554:00 +CPU= 27.219
  11:22:20 STEP  555 H= 555:00 +CPU= 42.314
  11:22:46 STEP  556 H= 556:00 +CPU= 26.599
  11:23:13 STEP  557 H= 557:00 +CPU= 26.369
  11:23:57 STEP  558 H= 558:00 +CPU= 44.163
  11:24:23 STEP  559 H= 559:00 +CPU= 26.296
  11:24:51 STEP  560 H= 560:00 +CPU= 27.194
  11:25:33 STEP  561 H= 561:00 +CPU= 42.042
  11:26:00 STEP  562 H= 562:00 +CPU= 26.833
  11:26:26 STEP  563 H= 563:00 +CPU= 26.048
  11:27:13 STEP  564 H= 564:00 +CPU= 47.139
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000564
  11:27:40 STEP  565 H= 565:00 +CPU= 26.556
  11:28:07 STEP  566 H= 566:00 +CPU= 26.805
  11:28:49 STEP  567 H= 567:00 +CPU= 42.392
  11:29:16 STEP  568 H= 568:00 +CPU= 26.410
  11:29:41 STEP  569 H= 569:00 +CPU= 25.798
  11:30:25 STEP  570 H= 570:00 +CPU= 43.406
  11:30:51 STEP  571 H= 571:00 +CPU= 25.635
  11:31:17 STEP  572 H= 572:00 +CPU= 26.477
  11:32:01 STEP  573 H= 573:00 +CPU= 43.766
  11:32:29 STEP  574 H= 574:00 +CPU= 27.323
  11:32:56 STEP  575 H= 575:00 +CPU= 27.250
  11:33:45 STEP  576 H= 576:00 +CPU= 49.159
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_23.zip
Uploading the intermediate file: upload_file_23.zip
  11:34:12 STEP  577 H= 577:00 +CPU= 26.664
  11:34:39 STEP  578 H= 578:00 +CPU= 26.525
  11:35:21 STEP  579 H= 579:00 +CPU= 41.803
  11:35:47 STEP  580 H= 580:00 +CPU= 26.741
  11:36:15 STEP  581 H= 581:00 +CPU= 27.134
  11:36:58 STEP  582 H= 582:00 +CPU= 43.433
  11:37:24 STEP  583 H= 583:00 +CPU= 25.954
  11:37:52 STEP  584 H= 584:00 +CPU= 27.289
  11:38:34 STEP  585 H= 585:00 +CPU= 42.176
  11:39:01 STEP  586 H= 586:00 +CPU= 27.367
  11:39:28 STEP  587 H= 587:00 +CPU= 26.968
  11:40:15 STEP  588 H= 588:00 +CPU= 46.402
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000588
  11:40:41 STEP  589 H= 589:00 +CPU= 25.818
  11:41:08 STEP  590 H= 590:00 +CPU= 27.455
  11:41:51 STEP  591 H= 591:00 +CPU= 42.361
  11:42:18 STEP  592 H= 592:00 +CPU= 27.398
  11:42:43 STEP  593 H= 593:00 +CPU= 25.050
  11:43:27 STEP  594 H= 594:00 +CPU= 43.581
  11:43:53 STEP  595 H= 595:00 +CPU= 25.625
  11:44:20 STEP  596 H= 596:00 +CPU= 26.863
  11:45:02 STEP  597 H= 597:00 +CPU= 42.467
  11:45:29 STEP  598 H= 598:00 +CPU= 26.440
  11:45:56 STEP  599 H= 599:00 +CPU= 27.249
  11:46:47 STEP  600 H= 600:00 +CPU= 49.843
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_24.zip
Uploading the intermediate file: upload_file_24.zip
  11:47:12 STEP  601 H= 601:00 +CPU= 25.671
  11:47:40 STEP  602 H= 602:00 +CPU= 27.275
  11:48:22 STEP  603 H= 603:00 +CPU= 42.014
  11:48:48 STEP  604 H= 604:00 +CPU= 26.612
  11:49:14 STEP  605 H= 605:00 +CPU= 25.810
  11:49:58 STEP  606 H= 606:00 +CPU= 43.368
  11:50:24 STEP  607 H= 607:00 +CPU= 26.388
  11:50:52 STEP  608 H= 608:00 +CPU= 27.261
  11:51:35 STEP  609 H= 609:00 +CPU= 42.867
  11:52:02 STEP  610 H= 610:00 +CPU= 26.956
  11:52:27 STEP  611 H= 611:00 +CPU= 25.326
  11:53:14 STEP  612 H= 612:00 +CPU= 47.039
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000612
  11:53:41 STEP  613 H= 613:00 +CPU= 26.582
  11:54:08 STEP  614 H= 614:00 +CPU= 27.302
  11:54:52 STEP  615 H= 615:00 +CPU= 43.472
  11:55:20 STEP  616 H= 616:00 +CPU= 27.738
  11:55:46 STEP  617 H= 617:00 +CPU= 26.152
  11:56:30 STEP  618 H= 618:00 +CPU= 43.623
  11:56:56 STEP  619 H= 619:00 +CPU= 26.297
  11:57:24 STEP  620 H= 620:00 +CPU= 27.416
  11:58:07 STEP  621 H= 621:00 +CPU= 43.269
  11:58:34 STEP  622 H= 622:00 +CPU= 26.611
  11:59:01 STEP  623 H= 623:00 +CPU= 27.445
  11:59:51 STEP  624 H= 624:00 +CPU= 49.987
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_25.zip
Uploading the intermediate file: upload_file_25.zip
  12:00:18 STEP  625 H= 625:00 +CPU= 26.377
  12:00:45 STEP  626 H= 626:00 +CPU= 26.956
  12:01:28 STEP  627 H= 627:00 +CPU= 42.804
  12:01:55 STEP  628 H= 628:00 +CPU= 27.040
  12:02:21 STEP  629 H= 629:00 +CPU= 26.313
  12:03:05 STEP  630 H= 630:00 +CPU= 42.981
  12:03:30 STEP  631 H= 631:00 +CPU= 25.073
  12:03:57 STEP  632 H= 632:00 +CPU= 27.511
  12:04:39 STEP  633 H= 633:00 +CPU= 41.636
  12:05:05 STEP  634 H= 634:00 +CPU= 26.280
  12:05:32 STEP  635 H= 635:00 +CPU= 26.676
  12:06:18 STEP  636 H= 636:00 +CPU= 45.940
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000636
  12:06:45 STEP  637 H= 637:00 +CPU= 26.350
  12:07:11 STEP  638 H= 638:00 +CPU= 26.412
  12:07:52 STEP  639 H= 639:00 +CPU= 40.934
  12:08:19 STEP  640 H= 640:00 +CPU= 27.132
  12:08:45 STEP  641 H= 641:00 +CPU= 25.910
  12:09:28 STEP  642 H= 642:00 +CPU= 42.990
  12:09:55 STEP  643 H= 643:00 +CPU= 26.289
  12:10:22 STEP  644 H= 644:00 +CPU= 27.115
  12:11:03 STEP  645 H= 645:00 +CPU= 41.278
  12:11:30 STEP  646 H= 646:00 +CPU= 26.891
  12:11:57 STEP  647 H= 647:00 +CPU= 26.659
  12:12:46 STEP  648 H= 648:00 +CPU= 48.723
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_26.zip
Uploading the intermediate file: upload_file_26.zip
  12:13:12 STEP  649 H= 649:00 +CPU= 25.480
  12:13:39 STEP  650 H= 650:00 +CPU= 26.960
  12:14:21 STEP  651 H= 651:00 +CPU= 42.203
  12:14:48 STEP  652 H= 652:00 +CPU= 26.986
  12:15:14 STEP  653 H= 653:00 +CPU= 25.662
  12:15:56 STEP  654 H= 654:00 +CPU= 42.238
  12:16:22 STEP  655 H= 655:00 +CPU= 25.869
  12:16:49 STEP  656 H= 656:00 +CPU= 26.502
  12:17:30 STEP  657 H= 657:00 +CPU= 41.448
  12:17:57 STEP  658 H= 658:00 +CPU= 26.912
  12:18:23 STEP  659 H= 659:00 +CPU= 25.702
  12:19:09 STEP  660 H= 660:00 +CPU= 46.010
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000660
  12:19:35 STEP  661 H= 661:00 +CPU= 26.254
  12:20:02 STEP  662 H= 662:00 +CPU= 26.389
  12:20:43 STEP  663 H= 663:00 +CPU= 41.590
  12:21:10 STEP  664 H= 664:00 +CPU= 26.312
  12:21:36 STEP  665 H= 665:00 +CPU= 25.838
  12:22:18 STEP  666 H= 666:00 +CPU= 42.669
  12:22:45 STEP  667 H= 667:00 +CPU= 25.962
  12:23:12 STEP  668 H= 668:00 +CPU= 26.988
  12:23:54 STEP  669 H= 669:00 +CPU= 42.644
  12:24:21 STEP  670 H= 670:00 +CPU= 26.602
  12:24:48 STEP  671 H= 671:00 +CPU= 27.128
  12:25:39 STEP  672 H= 672:00 +CPU= 50.057
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_27.zip
Uploading the intermediate file: upload_file_27.zip
  12:26:05 STEP  673 H= 673:00 +CPU= 25.698
  12:26:31 STEP  674 H= 674:00 +CPU= 26.557
  12:27:14 STEP  675 H= 675:00 +CPU= 42.046
  12:27:41 STEP  676 H= 676:00 +CPU= 27.084
  12:28:07 STEP  677 H= 677:00 +CPU= 26.284
  12:28:50 STEP  678 H= 678:00 +CPU= 42.733
  12:29:16 STEP  679 H= 679:00 +CPU= 25.566
  12:29:43 STEP  680 H= 680:00 +CPU= 26.954
  12:30:24 STEP  681 H= 681:00 +CPU= 41.305
  12:30:51 STEP  682 H= 682:00 +CPU= 27.141
  12:31:17 STEP  683 H= 683:00 +CPU= 26.128
  12:32:04 STEP  684 H= 684:00 +CPU= 46.965
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000684
  12:32:30 STEP  685 H= 685:00 +CPU= 25.166
  12:32:56 STEP  686 H= 686:00 +CPU= 26.648
  12:33:39 STEP  687 H= 687:00 +CPU= 42.681
  12:34:07 STEP  688 H= 688:00 +CPU= 27.422
  12:34:33 STEP  689 H= 689:00 +CPU= 26.160
  12:35:16 STEP  690 H= 690:00 +CPU= 42.630
  12:35:41 STEP  691 H= 691:00 +CPU= 25.525
  12:36:08 STEP  692 H= 692:00 +CPU= 26.557
  12:36:49 STEP  693 H= 693:00 +CPU= 41.436
  12:37:16 STEP  694 H= 694:00 +CPU= 26.860
  12:37:43 STEP  695 H= 695:00 +CPU= 26.508
  12:38:31 STEP  696 H= 696:00 +CPU= 47.776
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_28.zip
Uploading the intermediate file: upload_file_28.zip
  12:38:57 STEP  697 H= 697:00 +CPU= 26.159
  12:39:24 STEP  698 H= 698:00 +CPU= 27.007
  12:40:06 STEP  699 H= 699:00 +CPU= 41.559
  12:40:33 STEP  700 H= 700:00 +CPU= 26.673
  12:40:59 STEP  701 H= 701:00 +CPU= 25.657
  12:41:41 STEP  702 H= 702:00 +CPU= 42.570
  12:42:07 STEP  703 H= 703:00 +CPU= 25.563
  12:42:34 STEP  704 H= 704:00 +CPU= 26.628
  12:43:16 STEP  705 H= 705:00 +CPU= 42.349
  12:43:43 STEP  706 H= 706:00 +CPU= 26.933
  12:44:09 STEP  707 H= 707:00 +CPU= 25.511
  12:44:55 STEP  708 H= 708:00 +CPU= 46.227
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000708
  12:45:22 STEP  709 H= 709:00 +CPU= 26.761
  12:45:55 STEP  710 H= 710:00 +CPU= 33.089
  14:44:55 STEP  711 H= 711:00 +CPU= 49.121
  14:45:27 STEP  712 H= 712:00 +CPU= 32.041
  14:46:00 STEP  713 H= 713:00 +CPU= 32.598
  14:46:52 STEP  714 H= 714:00 +CPU= 51.833
  14:47:23 STEP  715 H= 715:00 +CPU= 29.974
  14:47:52 STEP  716 H= 716:00 +CPU= 29.036
  14:48:35 STEP  717 H= 717:00 +CPU= 43.402
  14:49:03 STEP  718 H= 718:00 +CPU= 27.452
  14:49:31 STEP  719 H= 719:00 +CPU= 27.768
  14:50:25 STEP  720 H= 720:00 +CPU= 52.488
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_29.zip
Uploading the intermediate file: upload_file_29.zip
  14:50:52 STEP  721 H= 721:00 +CPU= 26.388
Uploading trickle at timestep: 2592000
  14:51:19 STEP  722 H= 722:00 +CPU= 27.428
  14:52:03 STEP  723 H= 723:00 +CPU= 43.040
  14:52:30 STEP  724 H= 724:00 +CPU= 27.329
  14:52:56 STEP  725 H= 725:00 +CPU= 25.898
  14:53:40 STEP  726 H= 726:00 +CPU= 44.068
  14:54:07 STEP  727 H= 727:00 +CPU= 26.754
  14:54:34 STEP  728 H= 728:00 +CPU= 27.018
  14:55:17 STEP  729 H= 729:00 +CPU= 42.611
  14:55:45 STEP  730 H= 730:00 +CPU= 27.185
  14:56:11 STEP  731 H= 731:00 +CPU= 26.367
  14:56:58 STEP  732 H= 732:00 +CPU= 46.684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000732
  14:57:28 STEP  733 H= 733:00 +CPU= 30.020
  14:58:04 STEP  734 H= 734:00 +CPU= 35.740
  14:58:53 STEP  735 H= 735:00 +CPU= 47.892
  14:59:20 STEP  736 H= 736:00 +CPU= 27.005
  14:59:47 STEP  737 H= 737:00 +CPU= 26.955
  15:00:30 STEP  738 H= 738:00 +CPU= 42.683
  15:00:56 STEP  739 H= 739:00 +CPU= 26.384
  15:01:23 STEP  740 H= 740:00 +CPU= 26.821
  15:02:06 STEP  741 H= 741:00 +CPU= 42.265
  15:02:33 STEP  742 H= 742:00 +CPU= 26.787
  15:03:00 STEP  743 H= 743:00 +CPU= 27.008
  15:03:48 STEP  744 H= 744:00 +CPU= 48.189
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_30.zip
Uploading the intermediate file: upload_file_30.zip
  15:04:15 STEP  745 H= 745:00 +CPU= 26.592
  15:04:42 STEP  746 H= 746:00 +CPU= 26.771
  15:05:25 STEP  747 H= 747:00 +CPU= 42.311
  15:05:52 STEP  748 H= 748:00 +CPU= 27.082
  15:06:17 STEP  749 H= 749:00 +CPU= 25.527
  15:07:01 STEP  750 H= 750:00 +CPU= 43.382
  15:07:28 STEP  751 H= 751:00 +CPU= 26.470
  15:07:55 STEP  752 H= 752:00 +CPU= 26.863
  15:08:37 STEP  753 H= 753:00 +CPU= 41.826
  15:09:03 STEP  754 H= 754:00 +CPU= 26.343
  15:09:29 STEP  755 H= 755:00 +CPU= 25.846
  15:10:16 STEP  756 H= 756:00 +CPU= 47.174
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000756
  15:10:42 STEP  757 H= 757:00 +CPU= 25.693
  15:11:09 STEP  758 H= 758:00 +CPU= 27.009
  15:11:52 STEP  759 H= 759:00 +CPU= 42.159
  15:12:18 STEP  760 H= 760:00 +CPU= 26.532
  15:12:45 STEP  761 H= 761:00 +CPU= 26.877
  15:13:35 STEP  762 H= 762:00 +CPU= 49.280
  15:14:03 STEP  763 H= 763:00 +CPU= 28.074
  15:14:33 STEP  764 H= 764:00 +CPU= 29.053
  15:15:24 STEP  765 H= 765:00 +CPU= 50.385
  15:15:57 STEP  766 H= 766:00 +CPU= 32.885
  15:16:33 STEP  767 H= 767:00 +CPU= 35.782
  15:17:39 STEP  768 H= 768:00 +CPU= 65.177
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_31.zip
Uploading the intermediate file: upload_file_31.zip
  15:18:14 STEP  769 H= 769:00 +CPU= 33.968
  15:18:49 STEP  770 H= 770:00 +CPU= 34.429
  15:19:46 STEP  771 H= 771:00 +CPU= 55.990
  15:20:13 STEP  772 H= 772:00 +CPU= 27.065
  15:20:48 STEP  773 H= 773:00 +CPU= 34.616
  15:21:40 STEP  774 H= 774:00 +CPU= 52.008
  15:22:15 STEP  775 H= 775:00 +CPU= 34.313
  15:22:52 STEP  776 H= 776:00 +CPU= 36.111
  15:23:49 STEP  777 H= 777:00 +CPU= 57.161
  15:24:26 STEP  778 H= 778:00 +CPU= 36.146
  15:25:00 STEP  779 H= 779:00 +CPU= 33.883
  15:26:02 STEP  780 H= 780:00 +CPU= 60.758
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000780
  15:26:36 STEP  781 H= 781:00 +CPU= 34.097
  15:27:12 STEP  782 H= 782:00 +CPU= 35.329
  15:28:08 STEP  783 H= 783:00 +CPU= 55.599
  15:28:44 STEP  784 H= 784:00 +CPU= 35.500
  15:29:19 STEP  785 H= 785:00 +CPU= 34.549
  15:30:16 STEP  786 H= 786:00 +CPU= 56.774
  15:30:51 STEP  787 H= 787:00 +CPU= 34.777
  15:31:27 STEP  788 H= 788:00 +CPU= 35.260
  15:32:23 STEP  789 H= 789:00 +CPU= 55.396
  15:32:58 STEP  790 H= 790:00 +CPU= 35.143
  15:33:32 STEP  791 H= 791:00 +CPU= 33.583
  15:34:37 STEP  792 H= 792:00 +CPU= 64.582
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000780
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_32.zip
Uploading the intermediate file: upload_file_32.zip
  15:35:13 STEP  793 H= 793:00 +CPU= 34.982
  15:35:49 STEP  794 H= 794:00 +CPU= 35.435
  15:36:44 STEP  795 H= 795:00 +CPU= 55.125
  15:37:20 STEP  796 H= 796:00 +CPU= 35.041
  15:37:54 STEP  797 H= 797:00 +CPU= 33.920
  15:38:51 STEP  798 H= 798:00 +CPU= 56.303
  15:39:26 STEP  799 H= 799:00 +CPU= 34.488
  15:40:02 STEP  800 H= 800:00 +CPU= 35.763
  15:40:59 STEP  801 H= 801:00 +CPU= 56.132
  15:41:34 STEP  802 H= 802:00 +CPU= 34.792
  15:42:09 STEP  803 H= 803:00 +CPU= 34.647
  15:43:10 STEP  804 H= 804:00 +CPU= 60.903
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000804
  15:43:45 STEP  805 H= 805:00 +CPU= 34.485
  15:44:21 STEP  806 H= 806:00 +CPU= 35.464
  15:45:19 STEP  807 H= 807:00 +CPU= 56.789
  15:45:54 STEP  808 H= 808:00 +CPU= 34.993
  15:46:29 STEP  809 H= 809:00 +CPU= 34.418
  15:47:27 STEP  810 H= 810:00 +CPU= 57.320
  15:48:02 STEP  811 H= 811:00 +CPU= 34.892
  15:48:38 STEP  812 H= 812:00 +CPU= 35.024
  15:49:32 STEP  813 H= 813:00 +CPU= 53.942
  15:50:08 STEP  814 H= 814:00 +CPU= 35.880
  15:50:44 STEP  815 H= 815:00 +CPU= 35.274
  15:51:45 STEP  816 H= 816:00 +CPU= 60.729
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000804
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_33.zip
Uploading the intermediate file: upload_file_33.zip
  15:52:12 STEP  817 H= 817:00 +CPU= 26.178
  15:52:39 STEP  818 H= 818:00 +CPU= 26.603
  15:53:20 STEP  819 H= 819:00 +CPU= 41.736
  15:53:47 STEP  820 H= 820:00 +CPU= 26.545
  15:54:13 STEP  821 H= 821:00 +CPU= 25.647
  15:54:56 STEP  822 H= 822:00 +CPU= 42.746
  15:55:21 STEP  823 H= 823:00 +CPU= 25.429
  15:55:47 STEP  824 H= 824:00 +CPU= 26.166
  15:56:29 STEP  825 H= 825:00 +CPU= 41.671
  15:56:56 STEP  826 H= 826:00 +CPU= 26.629
  15:57:22 STEP  827 H= 827:00 +CPU= 26.275
  15:58:09 STEP  828 H= 828:00 +CPU= 45.946
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000828
  15:58:34 STEP  829 H= 829:00 +CPU= 25.660
  15:59:01 STEP  830 H= 830:00 +CPU= 26.264
  15:59:43 STEP  831 H= 831:00 +CPU= 42.476
  16:00:10 STEP  832 H= 832:00 +CPU= 26.219
  16:00:35 STEP  833 H= 833:00 +CPU= 25.581
  16:01:24 STEP  834 H= 834:00 +CPU= 47.822
  16:01:58 STEP  835 H= 835:00 +CPU= 33.852
  16:02:33 STEP  836 H= 836:00 +CPU= 35.219
  16:03:20 STEP  837 H= 837:00 +CPU= 46.050
  16:03:46 STEP  838 H= 838:00 +CPU= 26.342
  16:04:13 STEP  839 H= 839:00 +CPU= 26.551
  16:05:02 STEP  840 H= 840:00 +CPU= 49.046
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000828
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_34.zip
Uploading the intermediate file: upload_file_34.zip
  16:05:28 STEP  841 H= 841:00 +CPU= 25.639
  16:05:54 STEP  842 H= 842:00 +CPU= 26.002
  16:06:36 STEP  843 H= 843:00 +CPU= 41.866
  16:07:03 STEP  844 H= 844:00 +CPU= 26.590
  16:07:29 STEP  845 H= 845:00 +CPU= 25.824
  16:08:11 STEP  846 H= 846:00 +CPU= 42.262
  16:08:37 STEP  847 H= 847:00 +CPU= 25.429
  16:09:03 STEP  848 H= 848:00 +CPU= 26.466
  16:09:45 STEP  849 H= 849:00 +CPU= 41.891
  16:10:12 STEP  850 H= 850:00 +CPU= 26.805
  16:10:37 STEP  851 H= 851:00 +CPU= 25.122
  16:11:24 STEP  852 H= 852:00 +CPU= 45.977
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000852
  16:11:49 STEP  853 H= 853:00 +CPU= 25.348
  16:12:15 STEP  854 H= 854:00 +CPU= 25.987
  16:12:57 STEP  855 H= 855:00 +CPU= 41.596
  16:13:23 STEP  856 H= 856:00 +CPU= 26.368
  16:13:48 STEP  857 H= 857:00 +CPU= 25.029
  16:14:31 STEP  858 H= 858:00 +CPU= 42.904
  16:14:57 STEP  859 H= 859:00 +CPU= 25.655
  16:15:24 STEP  860 H= 860:00 +CPU= 26.279
  16:16:05 STEP  861 H= 861:00 +CPU= 41.127
  16:16:31 STEP  862 H= 862:00 +CPU= 26.260
  16:16:58 STEP  863 H= 863:00 +CPU= 26.377
  16:17:46 STEP  864 H= 864:00 +CPU= 47.793
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_35.zip
Uploading the intermediate file: upload_file_35.zip
  16:18:12 STEP  865 H= 865:00 +CPU= 25.965
  16:18:38 STEP  866 H= 866:00 +CPU= 26.291
  16:19:19 STEP  867 H= 867:00 +CPU= 40.945
  16:19:46 STEP  868 H= 868:00 +CPU= 26.370
  16:20:12 STEP  869 H= 869:00 +CPU= 25.702
  16:20:54 STEP  870 H= 870:00 +CPU= 42.377
  16:21:20 STEP  871 H= 871:00 +CPU= 25.882
  16:21:46 STEP  872 H= 872:00 +CPU= 26.046
  16:22:28 STEP  873 H= 873:00 +CPU= 41.995
  16:22:55 STEP  874 H= 874:00 +CPU= 26.261
  16:23:21 STEP  875 H= 875:00 +CPU= 25.709
  16:24:07 STEP  876 H= 876:00 +CPU= 45.783
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000876
  16:24:32 STEP  877 H= 877:00 +CPU= 25.655
  16:24:59 STEP  878 H= 878:00 +CPU= 26.746
  16:25:41 STEP  879 H= 879:00 +CPU= 41.429
  16:26:07 STEP  880 H= 880:00 +CPU= 26.141
  16:26:32 STEP  881 H= 881:00 +CPU= 25.393
  16:27:15 STEP  882 H= 882:00 +CPU= 42.058
  16:27:40 STEP  883 H= 883:00 +CPU= 25.347
  16:28:06 STEP  884 H= 884:00 +CPU= 26.155
  16:28:47 STEP  885 H= 885:00 +CPU= 41.042
  16:29:14 STEP  886 H= 886:00 +CPU= 26.394
  16:29:40 STEP  887 H= 887:00 +CPU= 26.229
  16:30:29 STEP  888 H= 888:00 +CPU= 48.446
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000876
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_36.zip
Uploading the intermediate file: upload_file_36.zip
  16:30:55 STEP  889 H= 889:00 +CPU= 25.677
  16:31:22 STEP  890 H= 890:00 +CPU= 26.299
  16:32:03 STEP  891 H= 891:00 +CPU= 41.392
  16:32:29 STEP  892 H= 892:00 +CPU= 25.977
  16:32:55 STEP  893 H= 893:00 +CPU= 25.771
  16:33:37 STEP  894 H= 894:00 +CPU= 41.863
  16:34:02 STEP  895 H= 895:00 +CPU= 25.285
  16:34:29 STEP  896 H= 896:00 +CPU= 26.411
  16:35:10 STEP  897 H= 897:00 +CPU= 41.118
  16:35:37 STEP  898 H= 898:00 +CPU= 26.324
  16:36:02 STEP  899 H= 899:00 +CPU= 25.400
  16:36:48 STEP  900 H= 900:00 +CPU= 45.694
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000900
  16:37:14 STEP  901 H= 901:00 +CPU= 25.978
  16:37:41 STEP  902 H= 902:00 +CPU= 26.603
  16:38:22 STEP  903 H= 903:00 +CPU= 41.223
  16:38:49 STEP  904 H= 904:00 +CPU= 26.634
  16:39:15 STEP  905 H= 905:00 +CPU= 25.628
  16:39:57 STEP  906 H= 906:00 +CPU= 42.509
  16:40:23 STEP  907 H= 907:00 +CPU= 25.930
  16:40:49 STEP  908 H= 908:00 +CPU= 25.977
  16:41:31 STEP  909 H= 909:00 +CPU= 41.544
  16:41:57 STEP  910 H= 910:00 +CPU= 25.934
  16:42:23 STEP  911 H= 911:00 +CPU= 26.038
  16:43:11 STEP  912 H= 912:00 +CPU= 47.706
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000900
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_37.zip
Uploading the intermediate file: upload_file_37.zip
  16:43:37 STEP  913 H= 913:00 +CPU= 25.634
  16:44:04 STEP  914 H= 914:00 +CPU= 26.504
  16:44:45 STEP  915 H= 915:00 +CPU= 40.917
  16:45:11 STEP  916 H= 916:00 +CPU= 26.447
  16:45:37 STEP  917 H= 917:00 +CPU= 25.725
  16:46:20 STEP  918 H= 918:00 +CPU= 42.517
  16:46:45 STEP  919 H= 919:00 +CPU= 25.684
  16:47:12 STEP  920 H= 920:00 +CPU= 26.381
  16:47:54 STEP  921 H= 921:00 +CPU= 41.458
  16:48:20 STEP  922 H= 922:00 +CPU= 26.265
  16:48:46 STEP  923 H= 923:00 +CPU= 25.666
  16:49:32 STEP  924 H= 924:00 +CPU= 46.141
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000924
  16:49:58 STEP  925 H= 925:00 +CPU= 26.102
  16:50:25 STEP  926 H= 926:00 +CPU= 26.393
  16:51:06 STEP  927 H= 927:00 +CPU= 41.561
  16:51:34 STEP  928 H= 928:00 +CPU= 27.040
  16:51:59 STEP  929 H= 929:00 +CPU= 25.784
  16:52:42 STEP  930 H= 930:00 +CPU= 42.227
  16:53:08 STEP  931 H= 931:00 +CPU= 25.582
  16:53:34 STEP  932 H= 932:00 +CPU= 26.216
  16:54:15 STEP  933 H= 933:00 +CPU= 41.393
  16:54:42 STEP  934 H= 934:00 +CPU= 26.098
  16:55:08 STEP  935 H= 935:00 +CPU= 26.009
  16:55:56 STEP  936 H= 936:00 +CPU= 47.990
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000924
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_38.zip
Uploading the intermediate file: upload_file_38.zip
  16:56:22 STEP  937 H= 937:00 +CPU= 25.442
  16:56:48 STEP  938 H= 938:00 +CPU= 26.376
  16:57:30 STEP  939 H= 939:00 +CPU= 41.336
  16:57:56 STEP  940 H= 940:00 +CPU= 26.157
  16:58:21 STEP  941 H= 941:00 +CPU= 25.210
  16:59:04 STEP  942 H= 942:00 +CPU= 42.478
  16:59:29 STEP  943 H= 943:00 +CPU= 25.586
  16:59:56 STEP  944 H= 944:00 +CPU= 26.441
  17:00:37 STEP  945 H= 945:00 +CPU= 40.975
  17:01:04 STEP  946 H= 946:00 +CPU= 26.506
  17:01:29 STEP  947 H= 947:00 +CPU= 25.742
  17:02:16 STEP  948 H= 948:00 +CPU= 46.048
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000948
  17:02:42 STEP  949 H= 949:00 +CPU= 25.862
  17:03:08 STEP  950 H= 950:00 +CPU= 26.273
  17:03:50 STEP  951 H= 951:00 +CPU= 42.047
  17:04:16 STEP  952 H= 952:00 +CPU= 26.116
  17:04:42 STEP  953 H= 953:00 +CPU= 25.691
  17:05:25 STEP  954 H= 954:00 +CPU= 42.158
  17:05:50 STEP  955 H= 955:00 +CPU= 25.610
  17:06:17 STEP  956 H= 956:00 +CPU= 26.438
  17:06:58 STEP  957 H= 957:00 +CPU= 41.168
  17:07:25 STEP  958 H= 958:00 +CPU= 26.475
  17:07:50 STEP  959 H= 959:00 +CPU= 25.722
  17:08:39 STEP  960 H= 960:00 +CPU= 47.837
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_39.zip
Uploading the intermediate file: upload_file_39.zip
Uploading trickle at timestep: 3456000
  17:09:05 STEP  961 H= 961:00 +CPU= 25.768
  17:09:31 STEP  962 H= 962:00 +CPU= 26.069
  17:10:12 STEP  963 H= 963:00 +CPU= 40.918
  17:10:38 STEP  964 H= 964:00 +CPU= 26.331
  17:11:04 STEP  965 H= 965:00 +CPU= 25.519
  17:11:47 STEP  966 H= 966:00 +CPU= 42.640
  17:12:12 STEP  967 H= 967:00 +CPU= 25.600
  17:12:39 STEP  968 H= 968:00 +CPU= 26.712
  17:13:20 STEP  969 H= 969:00 +CPU= 41.016
  17:13:47 STEP  970 H= 970:00 +CPU= 26.885
  17:14:13 STEP  971 H= 971:00 +CPU= 25.789
  17:14:59 STEP  972 H= 972:00 +CPU= 45.841
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000972
  17:15:25 STEP  973 H= 973:00 +CPU= 25.943
  17:15:52 STEP  974 H= 974:00 +CPU= 26.705
  17:16:34 STEP  975 H= 975:00 +CPU= 42.206
  17:17:00 STEP  976 H= 976:00 +CPU= 25.933
  17:17:26 STEP  977 H= 977:00 +CPU= 25.856
  17:18:09 STEP  978 H= 978:00 +CPU= 43.018
  17:18:40 STEP  979 H= 979:00 +CPU= 29.801
  17:19:12 STEP  980 H= 980:00 +CPU= 32.112
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.502] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.508] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.508] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.509] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.509] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.509] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.509] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.510] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.511] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.511] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.514] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.514] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.514] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.514] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.514] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.514] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS CLDPP T/S    (1002) 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171950:1671985190:24177.515] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                   CLOUDVAR 
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171951:1671985191:24178.517] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:2316:2316] [20221225:171951:1671985191:24178.517] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000FD0587  cloudvar_                 246  cloudvar.F90
oifs_43r3_model.e  0000000000FB3DC7  vdfmain_                  890  vdfmain.F90
oifs_43r3_model.e  0000000000F92699  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  0000000000F88512  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000F0B683  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1981050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0173
(argv4) batchid: 950
(argv5) wuid: 12166817
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1daad804fab2edf987a6ca4ced7e9504 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0173_1981050100_123_950_12166817.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0173_1981050100_123_950_12166817.zip
ic_ancil_file: ic_ancil_12166817
ifsdata_file: ifsdata_12166817
climate_data_file: clim_data_12166817
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_1f3c2aef13312eb4374e108567765d18 to: /var/lib/boinc-client/slots/0/ic_ancil_12166817.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12166817.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3f25508c9783a854a25b614f861a4df2 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12166817.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12166817.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12166817.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12166817.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12166817.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12166817.xml
-- Model is restarting --
Adjusting last_iter, 981, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12166817.xml
last_cpu_time: 36694
upload_file_number: 40
last_iter: 959
last_upload: 3456000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986
The child process has been launched with process id: 1909
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:rm-HP-Pavilion-dv7-Notebook-PC:1:1:1909:1909] [20221226:084559:1672040759:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:46:54 STEP  960 H= 960:00 +CPU= 47.176
  08:47:23 STEP  961 H= 961:00 +CPU= 28.361
  08:47:54 STEP  962 H= 962:00 +CPU= 30.520
  08:48:46 STEP  963 H= 963:00 +CPU= 51.771
  08:49:19 STEP  964 H= 964:00 +CPU= 32.183
  08:49:50 STEP  965 H= 965:00 +CPU= 30.669
  08:50:40 STEP  966 H= 966:00 +CPU= 50.263
  08:51:11 STEP  967 H= 967:00 +CPU= 30.394
  08:51:43 STEP  968 H= 968:00 +CPU= 31.088
  08:52:30 STEP  969 H= 969:00 +CPU= 47.273
  08:53:03 STEP  970 H= 970:00 +CPU= 32.733
  08:53:34 STEP  971 H= 971:00 +CPU= 30.920
  08:54:28 STEP  972 H= 972:00 +CPU= 53.333
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000972
  08:54:58 STEP  973 H= 973:00 +CPU= 29.448
  08:55:34 STEP  974 H= 974:00 +CPU= 35.324
  08:56:34 STEP  975 H= 975:00 +CPU= 59.265
  08:57:09 STEP  976 H= 976:00 +CPU= 34.383
  08:57:44 STEP  977 H= 977:00 +CPU= 34.395
  08:58:40 STEP  978 H= 978:00 +CPU= 54.913
  08:59:14 STEP  979 H= 979:00 +CPU= 33.211
  08:59:48 STEP  980 H= 980:00 +CPU= 33.629
  09:00:45 STEP  981 H= 981:00 +CPU= 55.888
  09:01:16 STEP  982 H= 982:00 +CPU= 30.624
  09:01:47 STEP  983 H= 983:00 +CPU= 31.201
  09:02:52 STEP  984 H= 984:00 +CPU= 62.984
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166817/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0173_1981050100_123_950_12166817_0_r1228176986_40.zip
Uploading the intermediate file: upload_file_40.zip
  09:03:26 STEP  985 H= 985:00 +CPU= 33.539
  09:04:00 STEP  986 H= 986:00 +CPU= 33.265
  09:04:54 STEP  987 H= 987:00 +CPU= 53.188
  09:05:30 STEP  988 H= 988:00 +CPU= 35.173
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:55:14 1533691 22251028 oifs_43r3_ps_0173_1981050100_123_950_12166817_0 3,456,000 36,035 0.0104
09 May 2023 15:55:01 1533691 22251028 oifs_43r3_ps_0173_1981050100_123_950_12166817_0 2,592,000 27,798 0.0107


©2024 cpdn.org