climateprediction.net home page
Task 22302467

Task 22302467

Name oifs_43r3_ps_0274_2017050100_123_986_12202918_1
Workunit 12202918
Created 23 Jan 2023, 17:51:32 UTC
Sent 23 Jan 2023, 17:52:34 UTC
Report deadline 24 Mar 2023, 17:52:34 UTC
Received 28 Jan 2023, 14:03:28 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1220995
Run time 6 hours 43 min 14 sec
CPU time 6 hours 38 min 25 sec
Validate state Invalid
Credit 5,322.24
Device peak FLOPS 3.23 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,797.30 MB
Peak swap size 4,978.08 MB
Peak disk usage 2,821.72 MB
Stderr
<core_client_version>7.21.0</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
fs_43r3_ps_12202918/ICMUAhq0f+000552
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000564
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000564
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000564
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_23.zip
Uploading the intermediate file: upload_file_23.zip
  14:43:44 STEP  577 H= 577:00 +CPU= 17.053
Uploading trickle at timestep: 2073600
  14:44:01 STEP  578 H= 578:00 +CPU= 17.645
  14:44:29 STEP  579 H= 579:00 +CPU= 27.426
  14:44:46 STEP  580 H= 580:00 +CPU= 17.321
  14:45:03 STEP  581 H= 581:00 +CPU= 16.898
  14:45:32 STEP  582 H= 582:00 +CPU= 28.279
  14:45:49 STEP  583 H= 583:00 +CPU= 17.074
  14:46:07 STEP  584 H= 584:00 +CPU= 17.681
  14:46:34 STEP  585 H= 585:00 +CPU= 27.306
  14:46:52 STEP  586 H= 586:00 +CPU= 17.439
  14:47:08 STEP  587 H= 587:00 +CPU= 16.773
  14:47:38 STEP  588 H= 588:00 +CPU= 30.006
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000588
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000588
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000588
  14:47:55 STEP  589 H= 589:00 +CPU= 16.652
  14:48:13 STEP  590 H= 590:00 +CPU= 17.450
  14:48:40 STEP  591 H= 591:00 +CPU= 27.539
  14:48:58 STEP  592 H= 592:00 +CPU= 17.294
  14:49:14 STEP  593 H= 593:00 +CPU= 16.783
  14:49:43 STEP  594 H= 594:00 +CPU= 28.004
  14:49:59 STEP  595 H= 595:00 +CPU= 16.556
  14:50:17 STEP  596 H= 596:00 +CPU= 17.441
  14:50:44 STEP  597 H= 597:00 +CPU= 27.037
  14:51:01 STEP  598 H= 598:00 +CPU= 17.137
  14:51:18 STEP  599 H= 599:00 +CPU= 17.131
  14:51:49 STEP  600 H= 600:00 +CPU= 30.999
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000600
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000600
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000600
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000576
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000576
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000576
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000588
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000588
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000588
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_24.zip
Uploading the intermediate file: upload_file_24.zip
  14:52:06 STEP  601 H= 601:00 +CPU= 16.770
  14:52:24 STEP  602 H= 602:00 +CPU= 17.511
  14:52:51 STEP  603 H= 603:00 +CPU= 27.018
  14:53:09 STEP  604 H= 604:00 +CPU= 17.597
  14:53:25 STEP  605 H= 605:00 +CPU= 16.637
  14:53:53 STEP  606 H= 606:00 +CPU= 27.683
  14:54:10 STEP  607 H= 607:00 +CPU= 17.145
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:03:25 STEP  608 H= 608:00 +CPU= 18.427
  17:03:54 STEP  609 H= 609:00 +CPU= 29.196
  17:04:13 STEP  610 H= 610:00 +CPU= 19.137
  17:04:33 STEP  611 H= 611:00 +CPU= 19.435
  17:05:04 STEP  612 H= 612:00 +CPU= 31.578
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000612
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000612
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000612
  17:05:22 STEP  613 H= 613:00 +CPU= 17.426
  17:05:40 STEP  614 H= 614:00 +CPU= 18.169
  17:06:08 STEP  615 H= 615:00 +CPU= 28.121
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:07:10 STEP  616 H= 616:00 +CPU= 18.442
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:07:51 STEP  617 H= 617:00 +CPU= 19.520
  17:08:22 STEP  618 H= 618:00 +CPU= 30.968
  17:08:39 STEP  619 H= 619:00 +CPU= 17.033
  17:08:58 STEP  620 H= 620:00 +CPU= 18.714
  17:09:27 STEP  621 H= 621:00 +CPU= 29.654
  17:09:47 STEP  622 H= 622:00 +CPU= 19.298
  17:10:04 STEP  623 H= 623:00 +CPU= 17.056
  17:10:37 STEP  624 H= 624:00 +CPU= 32.748
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000624
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000624
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000624
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000600
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000600
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000600
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000612
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000612
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000612
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_25.zip
Uploading the intermediate file: upload_file_25.zip
  17:10:55 STEP  625 H= 625:00 +CPU= 18.298
  17:11:13 STEP  626 H= 626:00 +CPU= 17.920
  17:11:43 STEP  627 H= 627:00 +CPU= 29.634
  17:12:00 STEP  628 H= 628:00 +CPU= 17.137
  17:12:18 STEP  629 H= 629:00 +CPU= 17.850
  17:12:48 STEP  630 H= 630:00 +CPU= 29.974
  17:13:06 STEP  631 H= 631:00 +CPU= 17.504
  17:13:24 STEP  632 H= 632:00 +CPU= 18.404
  17:13:53 STEP  633 H= 633:00 +CPU= 29.196
  17:14:11 STEP  634 H= 634:00 +CPU= 17.700
  17:14:30 STEP  635 H= 635:00 +CPU= 18.513
  17:15:01 STEP  636 H= 636:00 +CPU= 31.093
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000636
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000636
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000636
  17:15:19 STEP  637 H= 637:00 +CPU= 17.606
  17:15:38 STEP  638 H= 638:00 +CPU= 19.637
  17:16:08 STEP  639 H= 639:00 +CPU= 30.035
  17:16:27 STEP  640 H= 640:00 +CPU= 18.482
  17:16:45 STEP  641 H= 641:00 +CPU= 18.080
  17:17:13 STEP  642 H= 642:00 +CPU= 28.221
  17:17:32 STEP  643 H= 643:00 +CPU= 18.928
  17:17:50 STEP  644 H= 644:00 +CPU= 17.815
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:18:41 STEP  645 H= 645:00 +CPU= 29.874
  17:18:59 STEP  646 H= 646:00 +CPU= 17.563
  17:19:17 STEP  647 H= 647:00 +CPU= 17.495
  17:19:48 STEP  648 H= 648:00 +CPU= 31.509
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000648
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000648
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000648
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000624
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000624
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000624
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000636
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000636
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000636
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_26.zip
Uploading the intermediate file: upload_file_26.zip
  17:20:06 STEP  649 H= 649:00 +CPU= 17.547
  17:20:25 STEP  650 H= 650:00 +CPU= 18.898
  17:20:53 STEP  651 H= 651:00 +CPU= 27.715
  17:21:12 STEP  652 H= 652:00 +CPU= 19.581
  17:21:32 STEP  653 H= 653:00 +CPU= 19.593
  17:22:03 STEP  654 H= 654:00 +CPU= 30.971
  17:22:20 STEP  655 H= 655:00 +CPU= 17.220
  17:22:39 STEP  656 H= 656:00 +CPU= 18.267
  17:23:08 STEP  657 H= 657:00 +CPU= 29.071
  17:23:25 STEP  658 H= 658:00 +CPU= 17.453
  17:23:43 STEP  659 H= 659:00 +CPU= 17.353
  17:24:16 STEP  660 H= 660:00 +CPU= 32.736
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000660
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000660
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000660
  17:24:34 STEP  661 H= 661:00 +CPU= 17.974
  17:24:53 STEP  662 H= 662:00 +CPU= 19.201
  17:25:24 STEP  663 H= 663:00 +CPU= 31.036
  17:25:42 STEP  664 H= 664:00 +CPU= 17.400
  17:25:59 STEP  665 H= 665:00 +CPU= 16.939
  17:26:28 STEP  666 H= 666:00 +CPU= 28.955
  17:26:46 STEP  667 H= 667:00 +CPU= 18.341
  17:27:05 STEP  668 H= 668:00 +CPU= 18.967
  17:27:34 STEP  669 H= 669:00 +CPU= 29.040
  17:27:52 STEP  670 H= 670:00 +CPU= 17.706
  17:28:10 STEP  671 H= 671:00 +CPU= 18.088
  17:28:45 STEP  672 H= 672:00 +CPU= 34.850
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000672
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000672
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000672
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000648
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000648
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000648
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000660
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000660
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000660
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_27.zip
Uploading the intermediate file: upload_file_27.zip
  17:29:03 STEP  673 H= 673:00 +CPU= 17.386
  17:29:21 STEP  674 H= 674:00 +CPU= 18.209
  17:29:51 STEP  675 H= 675:00 +CPU= 30.420
  17:30:10 STEP  676 H= 676:00 +CPU= 18.319
  17:30:28 STEP  677 H= 677:00 +CPU= 18.140
  17:30:58 STEP  678 H= 678:00 +CPU= 29.349
  17:31:16 STEP  679 H= 679:00 +CPU= 18.533
  17:31:35 STEP  680 H= 680:00 +CPU= 18.345
  17:32:04 STEP  681 H= 681:00 +CPU= 29.597
  17:32:24 STEP  682 H= 682:00 +CPU= 19.593
  17:32:42 STEP  683 H= 683:00 +CPU= 17.819
  17:33:14 STEP  684 H= 684:00 +CPU= 31.952
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000684
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000684
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000684
  17:33:31 STEP  685 H= 685:00 +CPU= 17.307
  17:33:51 STEP  686 H= 686:00 +CPU= 19.217
  17:34:18 STEP  687 H= 687:00 +CPU= 27.274
  17:34:37 STEP  688 H= 688:00 +CPU= 18.811
  17:34:54 STEP  689 H= 689:00 +CPU= 16.894
  17:35:22 STEP  690 H= 690:00 +CPU= 28.280
  17:35:40 STEP  691 H= 691:00 +CPU= 17.609
  17:35:58 STEP  692 H= 692:00 +CPU= 18.200
  17:36:27 STEP  693 H= 693:00 +CPU= 29.051
  17:36:47 STEP  694 H= 694:00 +CPU= 19.340
  17:37:06 STEP  695 H= 695:00 +CPU= 18.657
  17:37:39 STEP  696 H= 696:00 +CPU= 33.189
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000696
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000696
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000696
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000672
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000672
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000672
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000684
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000684
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000684
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_28.zip
Uploading the intermediate file: upload_file_28.zip
  17:37:57 STEP  697 H= 697:00 +CPU= 18.318
  17:38:17 STEP  698 H= 698:00 +CPU= 19.628
  17:38:48 STEP  699 H= 699:00 +CPU= 30.480
  17:39:07 STEP  700 H= 700:00 +CPU= 19.206
  17:39:26 STEP  701 H= 701:00 +CPU= 18.578
  17:39:57 STEP  702 H= 702:00 +CPU= 31.153
  17:40:15 STEP  703 H= 703:00 +CPU= 17.992
  17:40:33 STEP  704 H= 704:00 +CPU= 18.239
  17:41:03 STEP  705 H= 705:00 +CPU= 29.649
  17:41:23 STEP  706 H= 706:00 +CPU= 19.805
  17:41:42 STEP  707 H= 707:00 +CPU= 19.219
  17:42:13 STEP  708 H= 708:00 +CPU= 31.198
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000708
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000708
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000708
  17:42:32 STEP  709 H= 709:00 +CPU= 18.902
  17:42:50 STEP  710 H= 710:00 +CPU= 17.738
  17:43:20 STEP  711 H= 711:00 +CPU= 29.422
  17:43:38 STEP  712 H= 712:00 +CPU= 17.941
  17:43:55 STEP  713 H= 713:00 +CPU= 16.812
  17:44:24 STEP  714 H= 714:00 +CPU= 28.825
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:45:03 STEP  715 H= 715:00 +CPU= 17.413
  17:45:20 STEP  716 H= 716:00 +CPU= 17.363
  17:45:50 STEP  717 H= 717:00 +CPU= 29.280
  17:46:08 STEP  718 H= 718:00 +CPU= 17.781
  17:46:27 STEP  719 H= 719:00 +CPU= 19.362
  17:47:01 STEP  720 H= 720:00 +CPU= 33.767
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000720
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000720
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000720
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000696
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000696
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000696
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000708
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000708
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000708
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_29.zip
Uploading the intermediate file: upload_file_29.zip
  17:47:19 STEP  721 H= 721:00 +CPU= 17.973
  17:47:38 STEP  722 H= 722:00 +CPU= 18.627
  17:48:08 STEP  723 H= 723:00 +CPU= 30.114
  17:48:27 STEP  724 H= 724:00 +CPU= 19.009
  17:48:45 STEP  725 H= 725:00 +CPU= 18.142
  17:49:15 STEP  726 H= 726:00 +CPU= 29.855
  17:49:33 STEP  727 H= 727:00 +CPU= 17.589
  17:49:51 STEP  728 H= 728:00 +CPU= 17.928
  17:50:21 STEP  729 H= 729:00 +CPU= 29.561
  17:50:40 STEP  730 H= 730:00 +CPU= 19.258
  17:51:00 STEP  731 H= 731:00 +CPU= 19.418
  17:51:33 STEP  732 H= 732:00 +CPU= 32.870
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000732
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000732
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000732
  17:51:51 STEP  733 H= 733:00 +CPU= 18.476
  17:52:09 STEP  734 H= 734:00 +CPU= 18.343
  17:52:37 STEP  735 H= 735:00 +CPU= 27.644
  17:52:57 STEP  736 H= 736:00 +CPU= 19.753
  17:53:15 STEP  737 H= 737:00 +CPU= 18.076
  17:53:47 STEP  738 H= 738:00 +CPU= 31.286
  17:54:05 STEP  739 H= 739:00 +CPU= 17.888
  17:54:24 STEP  740 H= 740:00 +CPU= 18.921
  17:54:52 STEP  741 H= 741:00 +CPU= 28.138
  17:55:10 STEP  742 H= 742:00 +CPU= 18.194
  17:55:29 STEP  743 H= 743:00 +CPU= 18.348
  17:56:03 STEP  744 H= 744:00 +CPU= 34.237
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000744
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000744
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000744
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000720
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000720
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000720
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000732
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000732
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000732
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_30.zip
Uploading the intermediate file: upload_file_30.zip
  17:56:21 STEP  745 H= 745:00 +CPU= 17.560
  17:56:38 STEP  746 H= 746:00 +CPU= 17.761
  17:57:09 STEP  747 H= 747:00 +CPU= 29.982
  17:57:26 STEP  748 H= 748:00 +CPU= 17.811
  17:57:44 STEP  749 H= 749:00 +CPU= 17.642
  17:58:16 STEP  750 H= 750:00 +CPU= 31.740
  17:58:34 STEP  751 H= 751:00 +CPU= 18.143
  17:58:52 STEP  752 H= 752:00 +CPU= 17.498
  17:59:21 STEP  753 H= 753:00 +CPU= 28.726
  17:59:40 STEP  754 H= 754:00 +CPU= 19.052
  17:59:58 STEP  755 H= 755:00 +CPU= 17.917
  18:00:30 STEP  756 H= 756:00 +CPU= 32.180
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000756
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000756
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000756
  18:00:48 STEP  757 H= 757:00 +CPU= 18.254
  18:01:08 STEP  758 H= 758:00 +CPU= 19.341
  18:01:39 STEP  759 H= 759:00 +CPU= 31.080
  18:01:58 STEP  760 H= 760:00 +CPU= 18.468
  18:02:17 STEP  761 H= 761:00 +CPU= 19.065
  18:02:46 STEP  762 H= 762:00 +CPU= 29.066
  18:03:03 STEP  763 H= 763:00 +CPU= 17.286
  18:03:21 STEP  764 H= 764:00 +CPU= 17.254
  18:03:50 STEP  765 H= 765:00 +CPU= 29.363
  18:04:09 STEP  766 H= 766:00 +CPU= 18.517
  18:04:29 STEP  767 H= 767:00 +CPU= 19.958
  18:05:03 STEP  768 H= 768:00 +CPU= 33.703
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000768
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000768
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000768
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000744
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000744
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000744
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000756
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000756
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000756
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_31.zip
Uploading the intermediate file: upload_file_31.zip
  18:05:21 STEP  769 H= 769:00 +CPU= 18.457
  18:05:40 STEP  770 H= 770:00 +CPU= 18.688
  18:06:09 STEP  771 H= 771:00 +CPU= 28.573
  18:06:27 STEP  772 H= 772:00 +CPU= 18.829
  18:06:45 STEP  773 H= 773:00 +CPU= 17.751
  18:07:15 STEP  774 H= 774:00 +CPU= 30.058
  18:07:34 STEP  775 H= 775:00 +CPU= 18.570
  18:07:52 STEP  776 H= 776:00 +CPU= 17.371
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:08:42 STEP  777 H= 777:00 +CPU= 28.851
  18:09:00 STEP  778 H= 778:00 +CPU= 18.218
  18:09:18 STEP  779 H= 779:00 +CPU= 17.725
  18:09:50 STEP  780 H= 780:00 +CPU= 31.796
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000780
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000780
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000780
  18:10:06 STEP  781 H= 781:00 +CPU= 16.632
  18:10:27 STEP  782 H= 782:00 +CPU= 20.503
  18:10:57 STEP  783 H= 783:00 +CPU= 30.420
  18:11:15 STEP  784 H= 784:00 +CPU= 17.670
  18:11:32 STEP  785 H= 785:00 +CPU= 16.975
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:12:22 STEP  786 H= 786:00 +CPU= 28.858
  18:12:39 STEP  787 H= 787:00 +CPU= 16.570
  18:12:56 STEP  788 H= 788:00 +CPU= 17.227
  18:13:23 STEP  789 H= 789:00 +CPU= 27.232
  18:13:41 STEP  790 H= 790:00 +CPU= 17.410
  18:13:59 STEP  791 H= 791:00 +CPU= 17.533
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:24:51 STEP  792 H= 792:00 +CPU= 31.937
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000792
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000792
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000792
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000768
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000768
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000768
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000780
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000780
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000780
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_32.zip
Uploading the intermediate file: upload_file_32.zip
  18:25:09 STEP  793 H= 793:00 +CPU= 17.671
  18:25:27 STEP  794 H= 794:00 +CPU= 18.371
  18:25:56 STEP  795 H= 795:00 +CPU= 28.094
  18:26:14 STEP  796 H= 796:00 +CPU= 18.277
  18:26:31 STEP  797 H= 797:00 +CPU= 17.409
  18:27:01 STEP  798 H= 798:00 +CPU= 29.043
  18:27:18 STEP  799 H= 799:00 +CPU= 17.240
  18:27:37 STEP  800 H= 800:00 +CPU= 18.751
  18:28:05 STEP  801 H= 801:00 +CPU= 28.114
  18:28:23 STEP  802 H= 802:00 +CPU= 18.219
  18:28:40 STEP  803 H= 803:00 +CPU= 17.060
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:29:33 STEP  804 H= 804:00 +CPU= 31.305
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000804
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000804
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000804
  18:29:50 STEP  805 H= 805:00 +CPU= 17.400
  18:30:08 STEP  806 H= 806:00 +CPU= 17.890
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:33:40 STEP  807 H= 807:00 +CPU= 27.473
  19:33:57 STEP  808 H= 808:00 +CPU= 17.268
  19:34:14 STEP  809 H= 809:00 +CPU= 16.870
  19:34:42 STEP  810 H= 810:00 +CPU= 28.288
  19:35:00 STEP  811 H= 811:00 +CPU= 17.152
  19:35:17 STEP  812 H= 812:00 +CPU= 17.551
  19:35:45 STEP  813 H= 813:00 +CPU= 27.494
  19:36:02 STEP  814 H= 814:00 +CPU= 17.317
  19:36:24 STEP  815 H= 815:00 +CPU= 21.961
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:40:17 STEP  816 H= 816:00 +CPU= 34.994
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000816
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000816
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000816
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000792
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000792
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000792
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000804
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000804
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000804
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_33.zip
Uploading the intermediate file: upload_file_33.zip
  19:40:34 STEP  817 H= 817:00 +CPU= 17.287
Uploading trickle at timestep: 2937600
  19:40:52 STEP  818 H= 818:00 +CPU= 17.803
Quit request received from BOINC client, ending the child process
19:41:44 (16486): BOINC client no longer exists - exiting
19:41:44 (16486): timer handler: client dead, exiting
19:41:49 (16486): BOINC client no longer exists - exiting
19:41:49 (16486): timer handler: client dead, exiting
19:41:54 (16486): BOINC client no longer exists - exiting
19:41:54 (16486): timer handler: client dead, exiting
19:41:59 (16486): BOINC client no longer exists - exiting
19:41:59 (16486): timer handler: client dead, exiting
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2017050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0274
(argv4) batchid: 986
(argv5) wuid: 12202918
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /mnt/sdc/home/seti/BOINC/slots/6
Project directory is: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918
..mkdir for temp folder for results failed
Copying: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /mnt/sdc/home/seti/BOINC/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /mnt/sdc/home/seti/BOINC/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_544426576586cd936d95ba70c2c49a9d to: /mnt/sdc/home/seti/BOINC/slots/6/oifs_43r3_ps_0274_2017050100_123_986_12202918.zip
Unzipping the namelist zip file: /mnt/sdc/home/seti/BOINC/slots/6/oifs_43r3_ps_0274_2017050100_123_986_12202918.zip
ic_ancil_file: ic_ancil_12202918
ifsdata_file: ifsdata_12202918
climate_data_file: clim_data_12202918
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6368220425fcca61853a7790c6c3a415 to: /mnt/sdc/home/seti/BOINC/slots/6/ic_ancil_12202918.zip
Unzipping the IC ancils zip file: /mnt/sdc/home/seti/BOINC/slots/6/ic_ancil_12202918.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_fbff3c30716d8ee44b1b36878d1f4291 to: /mnt/sdc/home/seti/BOINC/slots/6/ifsdata/ifsdata_12202918.zip
Unzipping the ifsdata_zip file: /mnt/sdc/home/seti/BOINC/slots/6/ifsdata/ifsdata_12202918.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/sdc/home/seti/BOINC/slots/6/159l_2/clim_data_12202918.zip
Unzipping the climate data zip file: /mnt/sdc/home/seti/BOINC/slots/6/159l_2/clim_data_12202918.zip
Checking for progress XML file: /mnt/sdc/home/seti/BOINC/slots/6/progress_file_12202918.xml
Opened progress file ok : /mnt/sdc/home/seti/BOINC/slots/6/progress_file_12202918.xml
-- Model is restarting --
Adjusting last_iter, 819, to previous model restart step.
Creating progress file: /mnt/sdc/home/seti/BOINC/slots/6/progress_file_12202918.xml
last_cpu_time: 18069
upload_file_number: 34
last_iter: 815
last_upload: 2937600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624
The child process has been launched with process id: 4762
Executing the command: /mnt/sdc/home/seti/BOINC/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:4762:4762] [20230128:112143:1674901303:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  11:23:08 STEP  816 H= 816:00 +CPU= 30.901
  11:23:26 STEP  817 H= 817:00 +CPU= 17.348
  11:23:44 STEP  818 H= 818:00 +CPU= 18.118
  11:24:12 STEP  819 H= 819:00 +CPU= 28.250
  11:24:31 STEP  820 H= 820:00 +CPU= 18.159
  11:24:48 STEP  821 H= 821:00 +CPU= 17.325
  11:25:17 STEP  822 H= 822:00 +CPU= 28.546
  11:25:34 STEP  823 H= 823:00 +CPU= 17.311
  11:25:53 STEP  824 H= 824:00 +CPU= 18.084
  11:26:20 STEP  825 H= 825:00 +CPU= 27.549
  11:26:38 STEP  826 H= 826:00 +CPU= 17.809
  11:26:56 STEP  827 H= 827:00 +CPU= 17.591
  11:27:27 STEP  828 H= 828:00 +CPU= 31.112
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000828
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000828
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000828
  11:27:45 STEP  829 H= 829:00 +CPU= 17.415
  11:28:03 STEP  830 H= 830:00 +CPU= 18.188
  11:28:34 STEP  831 H= 831:00 +CPU= 30.857
  11:28:54 STEP  832 H= 832:00 +CPU= 19.620
  11:29:13 STEP  833 H= 833:00 +CPU= 19.302
  11:29:45 STEP  834 H= 834:00 +CPU= 31.528
  11:30:04 STEP  835 H= 835:00 +CPU= 18.630
  11:30:21 STEP  836 H= 836:00 +CPU= 17.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:51 STEP  837 H= 837:00 +CPU= 27.886
  12:01:08 STEP  838 H= 838:00 +CPU= 17.526
  12:01:26 STEP  839 H= 839:00 +CPU= 17.649
  12:01:57 STEP  840 H= 840:00 +CPU= 31.471
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000840
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000840
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000840
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000816
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000816
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000816
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000828
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000828
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000828
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_34.zip
  12:02:14 STEP  841 H= 841:00 +CPU= 16.971
Uploading the intermediate file: upload_file_34.zip
  12:02:32 STEP  842 H= 842:00 +CPU= 17.798
  12:03:00 STEP  843 H= 843:00 +CPU= 27.710
  12:03:18 STEP  844 H= 844:00 +CPU= 17.806
  12:03:35 STEP  845 H= 845:00 +CPU= 17.151
  12:04:04 STEP  846 H= 846:00 +CPU= 28.313
  12:04:21 STEP  847 H= 847:00 +CPU= 17.227
  12:04:39 STEP  848 H= 848:00 +CPU= 17.815
  12:05:06 STEP  849 H= 849:00 +CPU= 27.654
  12:05:24 STEP  850 H= 850:00 +CPU= 17.690
  12:05:42 STEP  851 H= 851:00 +CPU= 17.283
  12:06:13 STEP  852 H= 852:00 +CPU= 30.906
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000852
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000852
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000852
  12:06:30 STEP  853 H= 853:00 +CPU= 17.112
  12:06:48 STEP  854 H= 854:00 +CPU= 18.118
  12:07:16 STEP  855 H= 855:00 +CPU= 28.011
  12:07:34 STEP  856 H= 856:00 +CPU= 17.524
  12:07:51 STEP  857 H= 857:00 +CPU= 17.216
  12:08:19 STEP  858 H= 858:00 +CPU= 28.514
  12:08:37 STEP  859 H= 859:00 +CPU= 17.147
  12:08:54 STEP  860 H= 860:00 +CPU= 17.653
  12:09:22 STEP  861 H= 861:00 +CPU= 27.658
  12:09:40 STEP  862 H= 862:00 +CPU= 17.717
  12:09:58 STEP  863 H= 863:00 +CPU= 17.619
  12:10:30 STEP  864 H= 864:00 +CPU= 32.102
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000864
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000864
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000864
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000840
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000840
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000840
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000852
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000852
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000852
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_35.zip
Uploading the intermediate file: upload_file_35.zip
  12:10:47 STEP  865 H= 865:00 +CPU= 17.117
  12:11:05 STEP  866 H= 866:00 +CPU= 17.607
  12:11:33 STEP  867 H= 867:00 +CPU= 27.706
  12:11:50 STEP  868 H= 868:00 +CPU= 17.699
  12:12:08 STEP  869 H= 869:00 +CPU= 17.213
  12:12:36 STEP  870 H= 870:00 +CPU= 28.291
  12:12:53 STEP  871 H= 871:00 +CPU= 16.950
  12:13:11 STEP  872 H= 872:00 +CPU= 17.718
  12:13:38 STEP  873 H= 873:00 +CPU= 27.557
  12:13:56 STEP  874 H= 874:00 +CPU= 17.765
  12:14:13 STEP  875 H= 875:00 +CPU= 17.191
  12:14:44 STEP  876 H= 876:00 +CPU= 30.616
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000876
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000876
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000876
  12:15:01 STEP  877 H= 877:00 +CPU= 17.150
  12:15:19 STEP  878 H= 878:00 +CPU= 17.554
  12:15:47 STEP  879 H= 879:00 +CPU= 27.905
  12:16:05 STEP  880 H= 880:00 +CPU= 17.697
  12:16:22 STEP  881 H= 881:00 +CPU= 17.118
  12:16:50 STEP  882 H= 882:00 +CPU= 28.185
  12:17:07 STEP  883 H= 883:00 +CPU= 17.113
  12:17:25 STEP  884 H= 884:00 +CPU= 17.653
  12:17:53 STEP  885 H= 885:00 +CPU= 27.732
  12:18:10 STEP  886 H= 886:00 +CPU= 17.449
  12:18:28 STEP  887 H= 887:00 +CPU= 17.633
  12:19:00 STEP  888 H= 888:00 +CPU= 31.712
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000888
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000888
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000888
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000864
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000864
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000864
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000876
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000876
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000876
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_36.zip
Uploading the intermediate file: upload_file_36.zip
  12:19:17 STEP  889 H= 889:00 +CPU= 17.326
  12:19:35 STEP  890 H= 890:00 +CPU= 17.746
  12:20:03 STEP  891 H= 891:00 +CPU= 27.613
  12:20:21 STEP  892 H= 892:00 +CPU= 17.997
  12:20:38 STEP  893 H= 893:00 +CPU= 17.297
  12:21:07 STEP  894 H= 894:00 +CPU= 28.462
  12:21:24 STEP  895 H= 895:00 +CPU= 17.020
  12:21:42 STEP  896 H= 896:00 +CPU= 17.906
  12:22:09 STEP  897 H= 897:00 +CPU= 27.570
  12:22:27 STEP  898 H= 898:00 +CPU= 17.714
  12:22:44 STEP  899 H= 899:00 +CPU= 17.156
  12:23:15 STEP  900 H= 900:00 +CPU= 30.925
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000900
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000900
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000900
  12:23:32 STEP  901 H= 901:00 +CPU= 17.077
  12:23:50 STEP  902 H= 902:00 +CPU= 17.727
  12:24:18 STEP  903 H= 903:00 +CPU= 27.751
  12:24:36 STEP  904 H= 904:00 +CPU= 17.950
  12:24:53 STEP  905 H= 905:00 +CPU= 17.129
  12:25:24 STEP  906 H= 906:00 +CPU= 30.148
  12:25:41 STEP  907 H= 907:00 +CPU= 17.401
  12:26:00 STEP  908 H= 908:00 +CPU= 18.480
  12:26:28 STEP  909 H= 909:00 +CPU= 28.219
  12:26:46 STEP  910 H= 910:00 +CPU= 18.376
  12:27:05 STEP  911 H= 911:00 +CPU= 18.337
  12:27:38 STEP  912 H= 912:00 +CPU= 32.623
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000912
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000912
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000912
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000888
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000888
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000888
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000900
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000900
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000900
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_37.zip
Uploading the intermediate file: upload_file_37.zip
  12:27:56 STEP  913 H= 913:00 +CPU= 18.170
  12:28:14 STEP  914 H= 914:00 +CPU= 18.209
  12:28:42 STEP  915 H= 915:00 +CPU= 27.951
  12:29:00 STEP  916 H= 916:00 +CPU= 17.796
  12:29:18 STEP  917 H= 917:00 +CPU= 17.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:30:08 STEP  918 H= 918:00 +CPU= 28.883
  12:30:26 STEP  919 H= 919:00 +CPU= 17.830
  12:30:44 STEP  920 H= 920:00 +CPU= 18.095
  12:31:12 STEP  921 H= 921:00 +CPU= 28.288
  12:31:31 STEP  922 H= 922:00 +CPU= 18.402
  12:31:49 STEP  923 H= 923:00 +CPU= 18.319
  12:32:21 STEP  924 H= 924:00 +CPU= 31.572
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000924
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000924
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000924
  12:32:39 STEP  925 H= 925:00 +CPU= 17.990
  12:32:57 STEP  926 H= 926:00 +CPU= 17.968
  12:33:26 STEP  927 H= 927:00 +CPU= 29.009
  12:33:44 STEP  928 H= 928:00 +CPU= 18.051
  12:34:02 STEP  929 H= 929:00 +CPU= 17.785
  12:34:31 STEP  930 H= 930:00 +CPU= 28.804
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:35:09 STEP  931 H= 931:00 +CPU= 17.257
  12:35:28 STEP  932 H= 932:00 +CPU= 18.173
  12:35:56 STEP  933 H= 933:00 +CPU= 28.784
  12:36:15 STEP  934 H= 934:00 +CPU= 18.297
  12:36:33 STEP  935 H= 935:00 +CPU= 18.405
  12:37:06 STEP  936 H= 936:00 +CPU= 32.819
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000936
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000936
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000936
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000912
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000912
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000912
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000924
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000924
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000924
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_38.zip
Uploading the intermediate file: upload_file_38.zip
  12:37:24 STEP  937 H= 937:00 +CPU= 17.876
  12:37:42 STEP  938 H= 938:00 +CPU= 18.066
  12:38:11 STEP  939 H= 939:00 +CPU= 28.692
  12:38:30 STEP  940 H= 940:00 +CPU= 18.426
  12:38:48 STEP  941 H= 941:00 +CPU= 18.169
  12:39:18 STEP  942 H= 942:00 +CPU= 29.337
  12:39:36 STEP  943 H= 943:00 +CPU= 17.885
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:40:16 STEP  944 H= 944:00 +CPU= 17.980
  12:40:45 STEP  945 H= 945:00 +CPU= 28.870
  12:41:03 STEP  946 H= 946:00 +CPU= 18.489
  12:41:20 STEP  947 H= 947:00 +CPU= 17.151
  12:41:52 STEP  948 H= 948:00 +CPU= 31.600
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000948
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000948
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000948
  12:42:10 STEP  949 H= 949:00 +CPU= 17.779
  12:42:28 STEP  950 H= 950:00 +CPU= 18.415
  12:42:57 STEP  951 H= 951:00 +CPU= 28.269
  12:43:15 STEP  952 H= 952:00 +CPU= 18.232
  12:43:33 STEP  953 H= 953:00 +CPU= 17.472
  12:44:02 STEP  954 H= 954:00 +CPU= 28.678
  12:44:19 STEP  955 H= 955:00 +CPU= 17.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:44:58 STEP  956 H= 956:00 +CPU= 17.919
  12:45:26 STEP  957 H= 957:00 +CPU= 27.905
  12:45:44 STEP  958 H= 958:00 +CPU= 17.585
  12:46:01 STEP  959 H= 959:00 +CPU= 17.378
  12:46:33 STEP  960 H= 960:00 +CPU= 31.526
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000960
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000960
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000960
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000936
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000936
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000936
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000948
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000948
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000948
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_39.zip
Uploading the intermediate file: upload_file_39.zip
  12:46:50 STEP  961 H= 961:00 +CPU= 17.110
  12:47:08 STEP  962 H= 962:00 +CPU= 17.614
  12:47:36 STEP  963 H= 963:00 +CPU= 27.577
  12:47:53 STEP  964 H= 964:00 +CPU= 17.677
  12:48:11 STEP  965 H= 965:00 +CPU= 17.195
  12:48:39 STEP  966 H= 966:00 +CPU= 28.237
  12:48:56 STEP  967 H= 967:00 +CPU= 17.041
  12:49:14 STEP  968 H= 968:00 +CPU= 17.816
  12:49:42 STEP  969 H= 969:00 +CPU= 27.587
  12:49:59 STEP  970 H= 970:00 +CPU= 17.690
  12:50:17 STEP  971 H= 971:00 +CPU= 17.196
  12:50:47 STEP  972 H= 972:00 +CPU= 30.653
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000972
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000972
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000972
  12:51:05 STEP  973 H= 973:00 +CPU= 17.582
  12:51:23 STEP  974 H= 974:00 +CPU= 17.856
  12:51:50 STEP  975 H= 975:00 +CPU= 27.494
  12:52:08 STEP  976 H= 976:00 +CPU= 17.628
  12:52:25 STEP  977 H= 977:00 +CPU= 17.054
  12:52:54 STEP  978 H= 978:00 +CPU= 28.567
  12:53:11 STEP  979 H= 979:00 +CPU= 16.997
  12:53:29 STEP  980 H= 980:00 +CPU= 17.629
  12:53:56 STEP  981 H= 981:00 +CPU= 27.380
  12:54:13 STEP  982 H= 982:00 +CPU= 17.384
  12:54:31 STEP  983 H= 983:00 +CPU= 17.464
  12:55:03 STEP  984 H= 984:00 +CPU= 31.752
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000984
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000984
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000984
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000960
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000960
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000960
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000972
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000972
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000972
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_40.zip
Uploading the intermediate file: upload_file_40.zip
  12:55:20 STEP  985 H= 985:00 +CPU= 17.180
  12:55:38 STEP  986 H= 986:00 +CPU= 17.539
  12:56:05 STEP  987 H= 987:00 +CPU= 27.512
  12:56:23 STEP  988 H= 988:00 +CPU= 17.657
  12:56:40 STEP  989 H= 989:00 +CPU= 17.202
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:57:31 STEP  990 H= 990:00 +CPU= 29.819
  12:57:49 STEP  991 H= 991:00 +CPU= 17.383
  12:58:08 STEP  992 H= 992:00 +CPU= 19.194
  12:58:37 STEP  993 H= 993:00 +CPU= 29.024
  12:58:56 STEP  994 H= 994:00 +CPU= 18.726
  12:59:15 STEP  995 H= 995:00 +CPU= 18.745
  12:59:49 STEP  996 H= 996:00 +CPU= 33.904
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+000996
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+000996
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+000996
  13:00:07 STEP  997 H= 997:00 +CPU= 18.531
  13:00:25 STEP  998 H= 998:00 +CPU= 18.209
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:30:57 STEP  999 H= 999:00 +CPU= 29.397
  13:31:15 STEP 1000 H=1000:00 +CPU= 17.978
  13:31:34 STEP 1001 H=1001:00 +CPU= 18.591
  13:32:04 STEP 1002 H=1002:00 +CPU= 29.694
  13:32:21 STEP 1003 H=1003:00 +CPU= 17.119
  13:32:39 STEP 1004 H=1004:00 +CPU= 18.431
  13:33:09 STEP 1005 H=1005:00 +CPU= 29.596
  13:33:27 STEP 1006 H=1006:00 +CPU= 18.219
  13:33:46 STEP 1007 H=1007:00 +CPU= 18.284
  13:34:20 STEP 1008 H=1008:00 +CPU= 33.853
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+001008
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+001008
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+001008
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000984
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000984
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000984
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+000996
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+000996
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+000996
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_41.zip
Uploading the intermediate file: upload_file_41.zip
  13:34:38 STEP 1009 H=1009:00 +CPU= 18.186
  13:34:59 STEP 1010 H=1010:00 +CPU= 20.904
  13:35:27 STEP 1011 H=1011:00 +CPU= 27.949
  13:35:45 STEP 1012 H=1012:00 +CPU= 17.625
  13:36:02 STEP 1013 H=1013:00 +CPU= 17.692
  13:36:32 STEP 1014 H=1014:00 +CPU= 29.057
  13:36:51 STEP 1015 H=1015:00 +CPU= 19.578
  13:37:09 STEP 1016 H=1016:00 +CPU= 17.920
  13:37:38 STEP 1017 H=1017:00 +CPU= 28.399
  13:37:57 STEP 1018 H=1018:00 +CPU= 19.148
  13:38:14 STEP 1019 H=1019:00 +CPU= 16.762
  13:38:46 STEP 1020 H=1020:00 +CPU= 32.132
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+001020
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+001020
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+001020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:39:28 STEP 1021 H=1021:00 +CPU= 19.869
  13:39:48 STEP 1022 H=1022:00 +CPU= 19.872
  13:40:19 STEP 1023 H=1023:00 +CPU= 31.199
  13:40:40 STEP 1024 H=1024:00 +CPU= 20.520
  13:40:59 STEP 1025 H=1025:00 +CPU= 18.531
  13:41:29 STEP 1026 H=1026:00 +CPU= 30.422
  13:41:46 STEP 1027 H=1027:00 +CPU= 17.079
  13:42:04 STEP 1028 H=1028:00 +CPU= 17.254
  13:42:32 STEP 1029 H=1029:00 +CPU= 27.970
  13:42:50 STEP 1030 H=1030:00 +CPU= 18.835
  13:43:08 STEP 1031 H=1031:00 +CPU= 17.251
  13:43:42 STEP 1032 H=1032:00 +CPU= 34.068
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+001032
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+001032
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+001032
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+001008
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+001008
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+001008
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+001020
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+001020
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+001020
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_42.zip
Uploading the intermediate file: upload_file_42.zip
  13:44:01 STEP 1033 H=1033:00 +CPU= 18.923
  13:44:20 STEP 1034 H=1034:00 +CPU= 19.234
  13:44:52 STEP 1035 H=1035:00 +CPU= 31.196
  13:45:11 STEP 1036 H=1036:00 +CPU= 18.885
  13:45:29 STEP 1037 H=1037:00 +CPU= 18.433
  13:46:01 STEP 1038 H=1038:00 +CPU= 31.580
  13:46:20 STEP 1039 H=1039:00 +CPU= 18.859
  13:46:40 STEP 1040 H=1040:00 +CPU= 19.961
  13:47:10 STEP 1041 H=1041:00 +CPU= 29.757
  13:47:28 STEP 1042 H=1042:00 +CPU= 18.455
  13:47:47 STEP 1043 H=1043:00 +CPU= 18.567
  13:48:20 STEP 1044 H=1044:00 +CPU= 32.989
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+001044
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+001044
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+001044
  13:48:38 STEP 1045 H=1045:00 +CPU= 17.929
  13:48:57 STEP 1046 H=1046:00 +CPU= 19.282
  13:49:26 STEP 1047 H=1047:00 +CPU= 29.072
  13:49:45 STEP 1048 H=1048:00 +CPU= 18.390
  13:50:02 STEP 1049 H=1049:00 +CPU= 17.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:50:52 STEP 1050 H=1050:00 +CPU= 28.655
  13:51:09 STEP 1051 H=1051:00 +CPU= 17.023
  13:51:27 STEP 1052 H=1052:00 +CPU= 17.865
  13:51:57 STEP 1053 H=1053:00 +CPU= 29.688
  13:52:16 STEP 1054 H=1054:00 +CPU= 19.402
  13:52:36 STEP 1055 H=1055:00 +CPU= 19.377
  13:53:11 STEP 1056 H=1056:00 +CPU= 35.088
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+001056
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+001056
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+001056
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+001032
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+001032
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+001032
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+001044
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+001044
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+001044
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_43.zip
Uploading the intermediate file: upload_file_43.zip
  13:53:29 STEP 1057 H=1057:00 +CPU= 17.646
Uploading trickle at timestep: 3801600
  13:53:47 STEP 1058 H=1058:00 +CPU= 18.442
  13:54:17 STEP 1059 H=1059:00 +CPU= 29.095
  13:54:34 STEP 1060 H=1060:00 +CPU= 17.884
  13:54:53 STEP 1061 H=1061:00 +CPU= 18.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:55:42 STEP 1062 H=1062:00 +CPU= 28.663
  13:56:02 STEP 1063 H=1063:00 +CPU= 19.167
  13:56:21 STEP 1064 H=1064:00 +CPU= 19.678
  13:56:52 STEP 1065 H=1065:00 +CPU= 30.160
  13:57:11 STEP 1066 H=1066:00 +CPU= 18.801
  13:57:28 STEP 1067 H=1067:00 +CPU= 17.405
  13:58:01 STEP 1068 H=1068:00 +CPU= 33.248
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+001068
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+001068
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+001068
  13:58:20 STEP 1069 H=1069:00 +CPU= 18.531
  13:58:39 STEP 1070 H=1070:00 +CPU= 18.907
  13:59:08 STEP 1071 H=1071:00 +CPU= 29.010
  13:59:26 STEP 1072 H=1072:00 +CPU= 17.501
  13:59:43 STEP 1073 H=1073:00 +CPU= 16.933
  14:00:12 STEP 1074 H=1074:00 +CPU= 28.858
  14:00:31 STEP 1075 H=1075:00 +CPU= 19.411
  14:00:51 STEP 1076 H=1076:00 +CPU= 20.236
  14:01:22 STEP 1077 H=1077:00 +CPU= 30.801
  14:01:40 STEP 1078 H=1078:00 +CPU= 17.605
  14:01:58 STEP 1079 H=1079:00 +CPU= 17.909
  14:02:31 STEP 1080 H=1080:00 +CPU= 32.745
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMGGhq0f+001080
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMSHhq0f+001080
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/6/ICMUAhq0f+001080
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+001056
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+001056
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+001056
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMGGhq0f+001068
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMSHhq0f+001068
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12202918/ICMUAhq0f+001068
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0274_2017050100_123_986_12202918_1_r41734624_44.zip
Uploading the intermediate file: upload_file_44.zip
  14:02:49 STEP 1081 H=1081:00 +CPU= 18.530
  14:03:09 STEP 1082 H=1082:00 +CPU= 19.738
double free or corruption (out)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:44:16 1220995 22302467 oifs_43r3_ps_0274_2017050100_123_986_12202918_1 3,801,600 23,335 0.0061
09 May 2023 16:44:10 1220995 22302467 oifs_43r3_ps_0274_2017050100_123_986_12202918_1 2,937,600 18,028 0.0061
09 May 2023 16:39:25 1220995 22302467 oifs_43r3_ps_0274_2017050100_123_986_12202918_1 2,073,600 12,748 0.0061
09 May 2023 16:39:25 1220995 22302467 oifs_43r3_ps_0274_2017050100_123_986_12202918_1 2,073,600 12,748 0.0061
09 May 2023 16:39:20 1220995 22302467 oifs_43r3_ps_0274_2017050100_123_986_12202918_1 1,209,600 7,701 0.0064


©2024 climateprediction.net