climateprediction.net home page
Task 22259322

Task 22259322

Name oifs_43r3_ps_0009_1989050100_123_958_12174653_0
Workunit 12174653
Created 21 Dec 2022, 15:25:11 UTC
Sent 16 Jan 2023, 17:43:37 UTC
Report deadline 17 Mar 2023, 17:43:37 UTC
Received 17 Jan 2023, 18:35:21 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1532237
Run time 18 hours 32 min 32 sec
CPU time 15 hours 26 min 28 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 7.29 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,409.18 MB
Peak swap size 4,978.08 MB
Peak disk usage 5,267.79 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
216:1673966536:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1742875:1742875] [20230117:074216:1673966536:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1742875:1742875] [20230117:074216:1673966536:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1742875:1742875] [20230117:074216:1673966536:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1742875:1742875] [20230117:074216:1673966536:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  07:42:54 STEP 2544 H=2544:00 +CPU= 34.308
  07:43:13 STEP 2545 H=2545:00 +CPU= 18.695
  07:43:32 STEP 2546 H=2546:00 +CPU= 19.442
  07:44:01 STEP 2547 H=2547:00 +CPU= 28.279
  07:44:22 STEP 2548 H=2548:00 +CPU= 20.440
  07:44:41 STEP 2549 H=2549:00 +CPU= 19.400
  07:45:10 STEP 2550 H=2550:00 +CPU= 28.079
  07:45:28 STEP 2551 H=2551:00 +CPU= 18.013
  07:45:45 STEP 2552 H=2552:00 +CPU= 17.146
  07:46:13 STEP 2553 H=2553:00 +CPU= 27.864
  07:46:33 STEP 2554 H=2554:00 +CPU= 20.040
  07:46:52 STEP 2555 H=2555:00 +CPU= 18.607
  07:47:22 STEP 2556 H=2556:00 +CPU= 29.454
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002556
  07:47:40 STEP 2557 H=2557:00 +CPU= 17.866
  07:48:00 STEP 2558 H=2558:00 +CPU= 19.222
  07:48:27 STEP 2559 H=2559:00 +CPU= 26.511
  07:48:45 STEP 2560 H=2560:00 +CPU= 18.383
  07:49:04 STEP 2561 H=2561:00 +CPU= 18.545
  07:49:32 STEP 2562 H=2562:00 +CPU= 28.013
  07:49:49 STEP 2563 H=2563:00 +CPU= 16.070
  07:50:08 STEP 2564 H=2564:00 +CPU= 18.785
  07:50:37 STEP 2565 H=2565:00 +CPU= 28.663
  07:50:58 STEP 2566 H=2566:00 +CPU= 20.669
  07:51:15 STEP 2567 H=2567:00 +CPU= 17.415
  07:51:49 STEP 2568 H=2568:00 +CPU= 32.660
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_106.zip
Uploading the intermediate file: upload_file_106.zip
  07:52:07 STEP 2569 H=2569:00 +CPU= 18.618
  07:52:27 STEP 2570 H=2570:00 +CPU= 19.509
  07:52:54 STEP 2571 H=2571:00 +CPU= 26.664
  07:53:13 STEP 2572 H=2572:00 +CPU= 18.148
  07:53:31 STEP 2573 H=2573:00 +CPU= 18.332
  07:54:02 STEP 2574 H=2574:00 +CPU= 30.135
  07:54:20 STEP 2575 H=2575:00 +CPU= 18.379
  07:54:39 STEP 2576 H=2576:00 +CPU= 18.766
  07:55:08 STEP 2577 H=2577:00 +CPU= 28.125
  07:55:25 STEP 2578 H=2578:00 +CPU= 16.790
  07:55:44 STEP 2579 H=2579:00 +CPU= 18.630
  07:56:16 STEP 2580 H=2580:00 +CPU= 32.509
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002580
  07:56:36 STEP 2581 H=2581:00 +CPU= 19.030
  07:56:53 STEP 2582 H=2582:00 +CPU= 17.523
  07:57:21 STEP 2583 H=2583:00 +CPU= 27.354
  07:57:39 STEP 2584 H=2584:00 +CPU= 17.537
  07:57:56 STEP 2585 H=2585:00 +CPU= 16.843
  07:58:25 STEP 2586 H=2586:00 +CPU= 29.016
  07:58:46 STEP 2587 H=2587:00 +CPU= 19.587
  07:59:04 STEP 2588 H=2588:00 +CPU= 17.692
  07:59:29 STEP 2589 H=2589:00 +CPU= 25.667
  07:59:48 STEP 2590 H=2590:00 +CPU= 18.804
  08:00:09 STEP 2591 H=2591:00 +CPU= 20.642
  08:00:43 STEP 2592 H=2592:00 +CPU= 33.495
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_107.zip
Uploading the intermediate file: upload_file_107.zip
  08:01:00 STEP 2593 H=2593:00 +CPU= 15.999
  08:01:18 STEP 2594 H=2594:00 +CPU= 18.283
  08:01:47 STEP 2595 H=2595:00 +CPU= 28.301
  08:02:05 STEP 2596 H=2596:00 +CPU= 18.035
  08:02:23 STEP 2597 H=2597:00 +CPU= 17.890
  08:02:53 STEP 2598 H=2598:00 +CPU= 29.551
  08:03:13 STEP 2599 H=2599:00 +CPU= 19.557
  08:03:31 STEP 2600 H=2600:00 +CPU= 17.965
  08:03:59 STEP 2601 H=2601:00 +CPU= 27.194
  08:04:18 STEP 2602 H=2602:00 +CPU= 19.377
  08:04:39 STEP 2603 H=2603:00 +CPU= 20.087
  08:05:11 STEP 2604 H=2604:00 +CPU= 31.927
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002604
  08:05:29 STEP 2605 H=2605:00 +CPU= 17.683
  08:05:46 STEP 2606 H=2606:00 +CPU= 17.317
  08:06:14 STEP 2607 H=2607:00 +CPU= 27.696
  08:06:34 STEP 2608 H=2608:00 +CPU= 19.596
  08:06:54 STEP 2609 H=2609:00 +CPU= 19.374
  08:07:24 STEP 2610 H=2610:00 +CPU= 29.731
  08:07:44 STEP 2611 H=2611:00 +CPU= 18.881
  08:08:03 STEP 2612 H=2612:00 +CPU= 19.019
  08:08:29 STEP 2613 H=2613:00 +CPU= 25.572
  08:08:48 STEP 2614 H=2614:00 +CPU= 19.459
  08:09:10 STEP 2615 H=2615:00 +CPU= 21.052
  08:09:43 STEP 2616 H=2616:00 +CPU= 33.532
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_108.zip
Uploading the intermediate file: upload_file_108.zip
  08:10:01 STEP 2617 H=2617:00 +CPU= 17.360
  08:10:20 STEP 2618 H=2618:00 +CPU= 19.078
  08:10:47 STEP 2619 H=2619:00 +CPU= 26.061
  08:11:06 STEP 2620 H=2620:00 +CPU= 18.668
  08:11:25 STEP 2621 H=2621:00 +CPU= 19.601
  08:11:54 STEP 2622 H=2622:00 +CPU= 27.809
  08:12:11 STEP 2623 H=2623:00 +CPU= 17.476
  08:12:30 STEP 2624 H=2624:00 +CPU= 18.906
  08:12:58 STEP 2625 H=2625:00 +CPU= 26.850
  08:13:16 STEP 2626 H=2626:00 +CPU= 18.192
  08:13:35 STEP 2627 H=2627:00 +CPU= 18.685
  08:14:05 STEP 2628 H=2628:00 +CPU= 29.775
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002628
  08:14:23 STEP 2629 H=2629:00 +CPU= 17.833
  08:14:44 STEP 2630 H=2630:00 +CPU= 20.030
  08:15:13 STEP 2631 H=2631:00 +CPU= 28.938
  08:15:32 STEP 2632 H=2632:00 +CPU= 18.590
  08:15:49 STEP 2633 H=2633:00 +CPU= 17.205
  08:16:17 STEP 2634 H=2634:00 +CPU= 27.854
  08:16:36 STEP 2635 H=2635:00 +CPU= 18.570
  08:16:56 STEP 2636 H=2636:00 +CPU= 19.548
  08:17:24 STEP 2637 H=2637:00 +CPU= 27.048
  08:17:42 STEP 2638 H=2638:00 +CPU= 18.581
  08:18:02 STEP 2639 H=2639:00 +CPU= 19.103
  08:18:34 STEP 2640 H=2640:00 +CPU= 32.149
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_109.zip
Uploading the intermediate file: upload_file_109.zip
  08:18:51 STEP 2641 H=2641:00 +CPU= 16.870
  08:19:11 STEP 2642 H=2642:00 +CPU= 19.542
  08:19:41 STEP 2643 H=2643:00 +CPU= 28.784
  08:19:58 STEP 2644 H=2644:00 +CPU= 17.015
  08:20:15 STEP 2645 H=2645:00 +CPU= 17.271
  08:20:46 STEP 2646 H=2646:00 +CPU= 30.138
  08:21:06 STEP 2647 H=2647:00 +CPU= 19.645
  08:21:26 STEP 2648 H=2648:00 +CPU= 19.958
  08:21:56 STEP 2649 H=2649:00 +CPU= 30.006
  08:22:15 STEP 2650 H=2650:00 +CPU= 18.640
  08:22:34 STEP 2651 H=2651:00 +CPU= 18.473
  08:23:03 STEP 2652 H=2652:00 +CPU= 28.619
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002652
  08:23:20 STEP 2653 H=2653:00 +CPU= 16.926
  08:23:41 STEP 2654 H=2654:00 +CPU= 20.677
  08:24:09 STEP 2655 H=2655:00 +CPU= 27.955
  08:24:26 STEP 2656 H=2656:00 +CPU= 16.229
  08:24:44 STEP 2657 H=2657:00 +CPU= 17.727
  08:25:15 STEP 2658 H=2658:00 +CPU= 30.815
  08:25:35 STEP 2659 H=2659:00 +CPU= 19.189
  08:25:52 STEP 2660 H=2660:00 +CPU= 17.386
  08:26:20 STEP 2661 H=2661:00 +CPU= 27.790
  08:26:39 STEP 2662 H=2662:00 +CPU= 18.632
  08:26:57 STEP 2663 H=2663:00 +CPU= 18.002
  08:27:32 STEP 2664 H=2664:00 +CPU= 34.307
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_110.zip
Uploading the intermediate file: upload_file_110.zip
  08:27:52 STEP 2665 H=2665:00 +CPU= 18.919
  08:28:13 STEP 2666 H=2666:00 +CPU= 20.968
  08:28:43 STEP 2667 H=2667:00 +CPU= 29.194
  08:29:02 STEP 2668 H=2668:00 +CPU= 19.333
  08:29:20 STEP 2669 H=2669:00 +CPU= 18.077
  08:29:50 STEP 2670 H=2670:00 +CPU= 28.361
  08:30:09 STEP 2671 H=2671:00 +CPU= 18.939
  08:30:30 STEP 2672 H=2672:00 +CPU= 20.501
  08:31:01 STEP 2673 H=2673:00 +CPU= 30.897
  08:31:20 STEP 2674 H=2674:00 +CPU= 19.219
  08:31:40 STEP 2675 H=2675:00 +CPU= 20.031
  08:32:15 STEP 2676 H=2676:00 +CPU= 33.595
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002676
  08:32:35 STEP 2677 H=2677:00 +CPU= 19.670
  08:32:56 STEP 2678 H=2678:00 +CPU= 20.959
  08:33:26 STEP 2679 H=2679:00 +CPU= 29.928
  08:33:46 STEP 2680 H=2680:00 +CPU= 19.511
  08:34:06 STEP 2681 H=2681:00 +CPU= 19.647
  08:34:33 STEP 2682 H=2682:00 +CPU= 26.841
  08:34:52 STEP 2683 H=2683:00 +CPU= 18.256
  08:35:12 STEP 2684 H=2684:00 +CPU= 19.621
  08:35:40 STEP 2685 H=2685:00 +CPU= 28.120
  08:35:59 STEP 2686 H=2686:00 +CPU= 18.659
  08:36:18 STEP 2687 H=2687:00 +CPU= 18.915
  08:36:53 STEP 2688 H=2688:00 +CPU= 34.414
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_111.zip
Uploading the intermediate file: upload_file_111.zip
  08:37:13 STEP 2689 H=2689:00 +CPU= 19.083
  08:37:33 STEP 2690 H=2690:00 +CPU= 20.273
  08:38:05 STEP 2691 H=2691:00 +CPU= 30.726
  08:38:24 STEP 2692 H=2692:00 +CPU= 18.888
  08:38:43 STEP 2693 H=2693:00 +CPU= 19.145
  08:39:11 STEP 2694 H=2694:00 +CPU= 27.773
  08:39:30 STEP 2695 H=2695:00 +CPU= 18.219
  08:39:48 STEP 2696 H=2696:00 +CPU= 17.603
  08:40:16 STEP 2697 H=2697:00 +CPU= 28.121
  08:40:35 STEP 2698 H=2698:00 +CPU= 18.958
  08:40:56 STEP 2699 H=2699:00 +CPU= 20.331
  08:41:29 STEP 2700 H=2700:00 +CPU= 32.390
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002700
  08:41:48 STEP 2701 H=2701:00 +CPU= 19.438
  08:42:09 STEP 2702 H=2702:00 +CPU= 20.156
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0009
(argv4) batchid: 958
(argv5) wuid: 12174653
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1e58a936600140bd2336b9c685981f23 to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0009_1989050100_123_958_12174653.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0009_1989050100_123_958_12174653.zip
ic_ancil_file: ic_ancil_12174653
ifsdata_file: ifsdata_12174653
climate_data_file: clim_data_12174653
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0137b20002a9f2430630ed19560dd74c to: /var/lib/boinc-client/slots/13/ic_ancil_12174653.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12174653.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6c26c7f9bfb080e2778d5d28e340f7be to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174653.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174653.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174653.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174653.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12174653.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12174653.xml
-- Model is restarting --
Adjusting last_iter, 2703, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12174653.xml
last_cpu_time: 49318
upload_file_number: 112
last_iter: 2687
last_upload: 9676800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099
The child process has been launched with process id: 1744796
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1744796:1744796] [20230117:084338:1673970218:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:44:14 STEP 2688 H=2688:00 +CPU= 33.061
  08:44:34 STEP 2689 H=2689:00 +CPU= 19.415
  08:44:52 STEP 2690 H=2690:00 +CPU= 18.225
  08:45:22 STEP 2691 H=2691:00 +CPU= 29.357
  08:45:42 STEP 2692 H=2692:00 +CPU= 20.082
  08:46:02 STEP 2693 H=2693:00 +CPU= 19.514
  08:46:32 STEP 2694 H=2694:00 +CPU= 29.807
  08:46:51 STEP 2695 H=2695:00 +CPU= 18.822
  08:47:08 STEP 2696 H=2696:00 +CPU= 16.832
  08:47:36 STEP 2697 H=2697:00 +CPU= 27.305
  08:47:55 STEP 2698 H=2698:00 +CPU= 19.131
  08:48:14 STEP 2699 H=2699:00 +CPU= 18.460
  08:48:44 STEP 2700 H=2700:00 +CPU= 29.743
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002700
  08:49:04 STEP 2701 H=2701:00 +CPU= 19.150
  08:49:23 STEP 2702 H=2702:00 +CPU= 18.859
  08:49:50 STEP 2703 H=2703:00 +CPU= 26.841
  08:50:09 STEP 2704 H=2704:00 +CPU= 18.911
  08:50:28 STEP 2705 H=2705:00 +CPU= 18.678
  08:50:58 STEP 2706 H=2706:00 +CPU= 29.259
  08:51:16 STEP 2707 H=2707:00 +CPU= 17.635
  08:51:37 STEP 2708 H=2708:00 +CPU= 20.190
  08:52:06 STEP 2709 H=2709:00 +CPU= 29.415
  08:52:26 STEP 2710 H=2710:00 +CPU= 18.799
  08:52:44 STEP 2711 H=2711:00 +CPU= 17.833
  08:53:20 STEP 2712 H=2712:00 +CPU= 35.650
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_112.zip
Uploading the intermediate file: upload_file_112.zip
  08:53:40 STEP 2713 H=2713:00 +CPU= 19.896
  08:54:00 STEP 2714 H=2714:00 +CPU= 19.493
  08:54:30 STEP 2715 H=2715:00 +CPU= 29.202
  08:54:50 STEP 2716 H=2716:00 +CPU= 19.812
  08:55:09 STEP 2717 H=2717:00 +CPU= 18.652
  08:55:38 STEP 2718 H=2718:00 +CPU= 28.612
  08:55:57 STEP 2719 H=2719:00 +CPU= 18.415
  08:56:16 STEP 2720 H=2720:00 +CPU= 18.597
  08:56:45 STEP 2721 H=2721:00 +CPU= 29.415
  08:57:04 STEP 2722 H=2722:00 +CPU= 18.106
  08:57:19 STEP 2723 H=2723:00 +CPU= 15.685
  08:57:50 STEP 2724 H=2724:00 +CPU= 29.965
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002724
  08:58:09 STEP 2725 H=2725:00 +CPU= 18.734
  08:58:27 STEP 2726 H=2726:00 +CPU= 17.852
  08:58:56 STEP 2727 H=2727:00 +CPU= 28.665
  08:59:11 STEP 2728 H=2728:00 +CPU= 15.195
  08:59:29 STEP 2729 H=2729:00 +CPU= 16.914
  09:00:00 STEP 2730 H=2730:00 +CPU= 30.873
  09:00:20 STEP 2731 H=2731:00 +CPU= 19.329
  09:00:38 STEP 2732 H=2732:00 +CPU= 17.513
  09:01:05 STEP 2733 H=2733:00 +CPU= 26.981
  09:01:25 STEP 2734 H=2734:00 +CPU= 19.699
  09:01:46 STEP 2735 H=2735:00 +CPU= 20.770
  09:02:22 STEP 2736 H=2736:00 +CPU= 35.324
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_113.zip
Uploading the intermediate file: upload_file_113.zip
  09:02:42 STEP 2737 H=2737:00 +CPU= 19.401
  09:03:02 STEP 2738 H=2738:00 +CPU= 19.635
  09:03:31 STEP 2739 H=2739:00 +CPU= 28.227
  09:03:49 STEP 2740 H=2740:00 +CPU= 18.658
  09:04:08 STEP 2741 H=2741:00 +CPU= 18.267
  09:04:39 STEP 2742 H=2742:00 +CPU= 29.994
  09:04:58 STEP 2743 H=2743:00 +CPU= 18.721
  09:05:17 STEP 2744 H=2744:00 +CPU= 18.819
  09:05:48 STEP 2745 H=2745:00 +CPU= 30.798
  09:06:08 STEP 2746 H=2746:00 +CPU= 20.044
  09:06:27 STEP 2747 H=2747:00 +CPU= 18.759
  09:07:01 STEP 2748 H=2748:00 +CPU= 33.202
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002748
  09:07:20 STEP 2749 H=2749:00 +CPU= 18.471
  09:07:38 STEP 2750 H=2750:00 +CPU= 17.988
  09:08:08 STEP 2751 H=2751:00 +CPU= 29.796
  09:08:30 STEP 2752 H=2752:00 +CPU= 20.882
  09:08:47 STEP 2753 H=2753:00 +CPU= 17.064
  09:09:17 STEP 2754 H=2754:00 +CPU= 29.805
  09:09:37 STEP 2755 H=2755:00 +CPU= 19.249
  09:09:57 STEP 2756 H=2756:00 +CPU= 19.655
  09:10:25 STEP 2757 H=2757:00 +CPU= 27.810
  09:10:45 STEP 2758 H=2758:00 +CPU= 20.015
  09:11:04 STEP 2759 H=2759:00 +CPU= 18.597
  09:11:38 STEP 2760 H=2760:00 +CPU= 33.600
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_114.zip
Uploading the intermediate file: upload_file_114.zip
  09:11:58 STEP 2761 H=2761:00 +CPU= 19.737
  09:12:17 STEP 2762 H=2762:00 +CPU= 18.415
  09:12:48 STEP 2763 H=2763:00 +CPU= 29.948
  09:13:05 STEP 2764 H=2764:00 +CPU= 17.585
  09:13:25 STEP 2765 H=2765:00 +CPU= 19.451
  09:13:56 STEP 2766 H=2766:00 +CPU= 30.185
  09:14:15 STEP 2767 H=2767:00 +CPU= 19.383
  09:14:33 STEP 2768 H=2768:00 +CPU= 17.851
  09:15:04 STEP 2769 H=2769:00 +CPU= 30.305
  09:15:24 STEP 2770 H=2770:00 +CPU= 19.129
  09:15:42 STEP 2771 H=2771:00 +CPU= 18.180
  09:16:15 STEP 2772 H=2772:00 +CPU= 32.531
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002772
  09:16:34 STEP 2773 H=2773:00 +CPU= 19.009
  09:16:53 STEP 2774 H=2774:00 +CPU= 18.532
  09:17:22 STEP 2775 H=2775:00 +CPU= 28.973
  09:17:42 STEP 2776 H=2776:00 +CPU= 19.154
  09:18:01 STEP 2777 H=2777:00 +CPU= 18.993
  09:18:31 STEP 2778 H=2778:00 +CPU= 29.440
  09:18:51 STEP 2779 H=2779:00 +CPU= 19.188
  09:19:10 STEP 2780 H=2780:00 +CPU= 19.075
  09:19:41 STEP 2781 H=2781:00 +CPU= 30.408
  09:20:00 STEP 2782 H=2782:00 +CPU= 19.174
  09:20:20 STEP 2783 H=2783:00 +CPU= 19.162
  09:20:51 STEP 2784 H=2784:00 +CPU= 30.920
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_115.zip
Uploading the intermediate file: upload_file_115.zip
  09:21:11 STEP 2785 H=2785:00 +CPU= 19.129
  09:21:29 STEP 2786 H=2786:00 +CPU= 18.393
  09:21:59 STEP 2787 H=2787:00 +CPU= 29.358
  09:22:17 STEP 2788 H=2788:00 +CPU= 17.483
  09:22:36 STEP 2789 H=2789:00 +CPU= 18.820
  09:23:07 STEP 2790 H=2790:00 +CPU= 30.543
  09:23:26 STEP 2791 H=2791:00 +CPU= 19.026
  09:23:45 STEP 2792 H=2792:00 +CPU= 18.482
  09:24:15 STEP 2793 H=2793:00 +CPU= 29.145
  09:24:34 STEP 2794 H=2794:00 +CPU= 18.895
  09:24:52 STEP 2795 H=2795:00 +CPU= 17.785
  09:25:23 STEP 2796 H=2796:00 +CPU= 30.724
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002796
  09:25:43 STEP 2797 H=2797:00 +CPU= 19.472
  09:26:02 STEP 2798 H=2798:00 +CPU= 19.355
  09:26:30 STEP 2799 H=2799:00 +CPU= 27.521
  09:26:50 STEP 2800 H=2800:00 +CPU= 18.861
  09:27:07 STEP 2801 H=2801:00 +CPU= 17.560
  09:27:36 STEP 2802 H=2802:00 +CPU= 28.378
  09:27:56 STEP 2803 H=2803:00 +CPU= 19.578
  09:28:13 STEP 2804 H=2804:00 +CPU= 16.671
  09:28:43 STEP 2805 H=2805:00 +CPU= 29.568
  09:29:04 STEP 2806 H=2806:00 +CPU= 20.281
  09:29:23 STEP 2807 H=2807:00 +CPU= 19.055
  09:30:00 STEP 2808 H=2808:00 +CPU= 35.850
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_116.zip
Uploading the intermediate file: upload_file_116.zip
  09:30:18 STEP 2809 H=2809:00 +CPU= 18.231
  09:30:37 STEP 2810 H=2810:00 +CPU= 18.397
  09:31:05 STEP 2811 H=2811:00 +CPU= 27.886
  09:31:26 STEP 2812 H=2812:00 +CPU= 20.414
  09:31:45 STEP 2813 H=2813:00 +CPU= 18.678
  09:32:15 STEP 2814 H=2814:00 +CPU= 29.454
  09:32:32 STEP 2815 H=2815:00 +CPU= 17.348
  09:32:51 STEP 2816 H=2816:00 +CPU= 17.817
  09:33:19 STEP 2817 H=2817:00 +CPU= 28.118
  09:33:38 STEP 2818 H=2818:00 +CPU= 18.665
  09:33:56 STEP 2819 H=2819:00 +CPU= 18.089
  09:34:26 STEP 2820 H=2820:00 +CPU= 30.120
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002820
  09:34:44 STEP 2821 H=2821:00 +CPU= 17.584
  09:35:03 STEP 2822 H=2822:00 +CPU= 18.313
  09:35:32 STEP 2823 H=2823:00 +CPU= 28.985
  09:35:49 STEP 2824 H=2824:00 +CPU= 17.244
  09:36:08 STEP 2825 H=2825:00 +CPU= 18.109
  09:36:36 STEP 2826 H=2826:00 +CPU= 27.888
  09:36:54 STEP 2827 H=2827:00 +CPU= 17.700
  09:37:13 STEP 2828 H=2828:00 +CPU= 18.578
  09:37:42 STEP 2829 H=2829:00 +CPU= 29.082
  09:38:00 STEP 2830 H=2830:00 +CPU= 18.158
  09:38:18 STEP 2831 H=2831:00 +CPU= 17.889
  09:38:52 STEP 2832 H=2832:00 +CPU= 32.939
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_117.zip
Uploading the intermediate file: upload_file_117.zip
  09:39:10 STEP 2833 H=2833:00 +CPU= 17.551
  09:39:29 STEP 2834 H=2834:00 +CPU= 18.666
  09:39:59 STEP 2835 H=2835:00 +CPU= 29.547
  09:40:18 STEP 2836 H=2836:00 +CPU= 18.353
  09:40:36 STEP 2837 H=2837:00 +CPU= 18.180
  09:41:06 STEP 2838 H=2838:00 +CPU= 29.212
  09:41:23 STEP 2839 H=2839:00 +CPU= 17.249
  09:41:42 STEP 2840 H=2840:00 +CPU= 18.329
  09:42:10 STEP 2841 H=2841:00 +CPU= 27.461
  09:42:30 STEP 2842 H=2842:00 +CPU= 19.312
  09:42:49 STEP 2843 H=2843:00 +CPU= 19.110
  09:43:21 STEP 2844 H=2844:00 +CPU= 31.661
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002844
  09:43:38 STEP 2845 H=2845:00 +CPU= 16.940
  09:43:58 STEP 2846 H=2846:00 +CPU= 19.335
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0009
(argv4) batchid: 958
(argv5) wuid: 12174653
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1e58a936600140bd2336b9c685981f23 to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0009_1989050100_123_958_12174653.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0009_1989050100_123_958_12174653.zip
ic_ancil_file: ic_ancil_12174653
ifsdata_file: ifsdata_12174653
climate_data_file: clim_data_12174653
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0137b20002a9f2430630ed19560dd74c to: /var/lib/boinc-client/slots/13/ic_ancil_12174653.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12174653.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6c26c7f9bfb080e2778d5d28e340f7be to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174653.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174653.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174653.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174653.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12174653.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12174653.xml
-- Model is restarting --
Adjusting last_iter, 2847, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12174653.xml
last_cpu_time: 52884
upload_file_number: 118
last_iter: 2831
last_upload: 10195200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099
The child process has been launched with process id: 1746969
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:1746969:1746969] [20230117:094504:1673973904:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:45:40 STEP 2832 H=2832:00 +CPU= 32.728
  09:45:59 STEP 2833 H=2833:00 +CPU= 18.812
  09:46:19 STEP 2834 H=2834:00 +CPU= 19.256
  09:46:46 STEP 2835 H=2835:00 +CPU= 26.933
  09:47:05 STEP 2836 H=2836:00 +CPU= 19.058
  09:47:23 STEP 2837 H=2837:00 +CPU= 17.921
  09:47:53 STEP 2838 H=2838:00 +CPU= 29.117
  09:48:10 STEP 2839 H=2839:00 +CPU= 17.103
  09:48:28 STEP 2840 H=2840:00 +CPU= 17.382
  09:48:58 STEP 2841 H=2841:00 +CPU= 29.431
  09:49:16 STEP 2842 H=2842:00 +CPU= 18.451
  09:49:35 STEP 2843 H=2843:00 +CPU= 18.177
  09:50:06 STEP 2844 H=2844:00 +CPU= 30.633
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002844
  09:50:25 STEP 2845 H=2845:00 +CPU= 18.952
  09:50:43 STEP 2846 H=2846:00 +CPU= 17.893
  09:51:09 STEP 2847 H=2847:00 +CPU= 25.612
  09:51:29 STEP 2848 H=2848:00 +CPU= 19.459
  09:51:50 STEP 2849 H=2849:00 +CPU= 20.527
  09:52:20 STEP 2850 H=2850:00 +CPU= 29.151
  09:52:39 STEP 2851 H=2851:00 +CPU= 18.579
  09:52:58 STEP 2852 H=2852:00 +CPU= 18.969
  09:53:26 STEP 2853 H=2853:00 +CPU= 27.843
  09:53:46 STEP 2854 H=2854:00 +CPU= 19.187
  09:54:06 STEP 2855 H=2855:00 +CPU= 19.836
  09:54:40 STEP 2856 H=2856:00 +CPU= 33.544
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_118.zip
Uploading the intermediate file: upload_file_118.zip
  09:55:00 STEP 2857 H=2857:00 +CPU= 20.377
  09:55:21 STEP 2858 H=2858:00 +CPU= 19.601
  09:55:49 STEP 2859 H=2859:00 +CPU= 27.979
  09:56:06 STEP 2860 H=2860:00 +CPU= 16.593
  09:56:25 STEP 2861 H=2861:00 +CPU= 18.575
  09:56:55 STEP 2862 H=2862:00 +CPU= 29.708
  09:57:14 STEP 2863 H=2863:00 +CPU= 19.352
  09:57:34 STEP 2864 H=2864:00 +CPU= 19.394
  09:58:04 STEP 2865 H=2865:00 +CPU= 29.829
  09:58:24 STEP 2866 H=2866:00 +CPU= 19.049
  09:58:43 STEP 2867 H=2867:00 +CPU= 18.550
  09:59:13 STEP 2868 H=2868:00 +CPU= 29.861
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002868
  09:59:33 STEP 2869 H=2869:00 +CPU= 19.485
  09:59:54 STEP 2870 H=2870:00 +CPU= 20.702
  10:00:23 STEP 2871 H=2871:00 +CPU= 29.026
  10:00:42 STEP 2872 H=2872:00 +CPU= 18.425
  10:01:01 STEP 2873 H=2873:00 +CPU= 18.847
  10:01:30 STEP 2874 H=2874:00 +CPU= 28.801
  10:01:50 STEP 2875 H=2875:00 +CPU= 18.642
  10:02:08 STEP 2876 H=2876:00 +CPU= 18.390
  10:02:36 STEP 2877 H=2877:00 +CPU= 26.995
  10:02:55 STEP 2878 H=2878:00 +CPU= 19.304
  10:03:17 STEP 2879 H=2879:00 +CPU= 21.180
  10:03:52 STEP 2880 H=2880:00 +CPU= 34.402
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_119.zip
Uploading the intermediate file: upload_file_119.zip
  10:04:11 STEP 2881 H=2881:00 +CPU= 19.479
  10:04:32 STEP 2882 H=2882:00 +CPU= 19.893
  10:05:00 STEP 2883 H=2883:00 +CPU= 28.092
  10:05:18 STEP 2884 H=2884:00 +CPU= 17.914
  10:05:37 STEP 2885 H=2885:00 +CPU= 17.895
  10:06:06 STEP 2886 H=2886:00 +CPU= 28.922
  10:06:25 STEP 2887 H=2887:00 +CPU= 18.657
  10:06:43 STEP 2888 H=2888:00 +CPU= 17.689
  10:07:14 STEP 2889 H=2889:00 +CPU= 30.563
  10:07:32 STEP 2890 H=2890:00 +CPU= 18.178
  10:07:52 STEP 2891 H=2891:00 +CPU= 19.441
  10:08:24 STEP 2892 H=2892:00 +CPU= 31.841
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002892
  10:08:43 STEP 2893 H=2893:00 +CPU= 18.519
  10:09:03 STEP 2894 H=2894:00 +CPU= 19.647
  10:09:31 STEP 2895 H=2895:00 +CPU= 27.532
  10:09:51 STEP 2896 H=2896:00 +CPU= 19.838
  10:10:11 STEP 2897 H=2897:00 +CPU= 19.885
  10:10:42 STEP 2898 H=2898:00 +CPU= 30.614
  10:11:03 STEP 2899 H=2899:00 +CPU= 20.166
  10:11:23 STEP 2900 H=2900:00 +CPU= 20.186
  10:11:52 STEP 2901 H=2901:00 +CPU= 27.979
  10:12:10 STEP 2902 H=2902:00 +CPU= 18.178
  10:12:29 STEP 2903 H=2903:00 +CPU= 18.836
  10:13:04 STEP 2904 H=2904:00 +CPU= 33.850
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_120.zip
Uploading the intermediate file: upload_file_120.zip
  10:13:23 STEP 2905 H=2905:00 +CPU= 19.359
  10:13:42 STEP 2906 H=2906:00 +CPU= 18.083
  10:14:12 STEP 2907 H=2907:00 +CPU= 29.709
  10:14:31 STEP 2908 H=2908:00 +CPU= 18.560
  10:14:49 STEP 2909 H=2909:00 +CPU= 17.510
  10:15:17 STEP 2910 H=2910:00 +CPU= 28.181
  10:15:36 STEP 2911 H=2911:00 +CPU= 18.794
  10:15:55 STEP 2912 H=2912:00 +CPU= 18.864
  10:16:25 STEP 2913 H=2913:00 +CPU= 29.431
  10:16:44 STEP 2914 H=2914:00 +CPU= 18.976
  10:17:04 STEP 2915 H=2915:00 +CPU= 19.192
  10:17:35 STEP 2916 H=2916:00 +CPU= 30.925
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002916
  10:17:55 STEP 2917 H=2917:00 +CPU= 19.457
  10:18:15 STEP 2918 H=2918:00 +CPU= 19.301
  10:18:43 STEP 2919 H=2919:00 +CPU= 27.775
  10:19:03 STEP 2920 H=2920:00 +CPU= 19.478
  10:19:22 STEP 2921 H=2921:00 +CPU= 18.731
  10:19:51 STEP 2922 H=2922:00 +CPU= 28.116
  10:20:09 STEP 2923 H=2923:00 +CPU= 18.259
  10:20:29 STEP 2924 H=2924:00 +CPU= 19.238
  10:20:59 STEP 2925 H=2925:00 +CPU= 29.255
  10:21:18 STEP 2926 H=2926:00 +CPU= 18.973
  10:21:38 STEP 2927 H=2927:00 +CPU= 20.252
  10:22:14 STEP 2928 H=2928:00 +CPU= 34.763
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_121.zip
Uploading the intermediate file: upload_file_121.zip
  10:22:32 STEP 2929 H=2929:00 +CPU= 18.700
  10:22:52 STEP 2930 H=2930:00 +CPU= 19.106
  10:23:21 STEP 2931 H=2931:00 +CPU= 28.798
  10:23:40 STEP 2932 H=2932:00 +CPU= 18.049
  10:23:58 STEP 2933 H=2933:00 +CPU= 17.897
  10:24:25 STEP 2934 H=2934:00 +CPU= 26.893
  10:24:42 STEP 2935 H=2935:00 +CPU= 16.783
  10:24:58 STEP 2936 H=2936:00 +CPU= 16.443
  10:25:24 STEP 2937 H=2937:00 +CPU= 25.248
  10:25:41 STEP 2938 H=2938:00 +CPU= 16.908
  10:25:57 STEP 2939 H=2939:00 +CPU= 16.232
  10:26:26 STEP 2940 H=2940:00 +CPU= 28.451
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002940
  10:26:43 STEP 2941 H=2941:00 +CPU= 16.599
  10:27:01 STEP 2942 H=2942:00 +CPU= 18.084
  10:27:30 STEP 2943 H=2943:00 +CPU= 29.065
  10:27:49 STEP 2944 H=2944:00 +CPU= 17.918
  10:28:06 STEP 2945 H=2945:00 +CPU= 16.927
  10:28:35 STEP 2946 H=2946:00 +CPU= 28.403
  10:28:52 STEP 2947 H=2947:00 +CPU= 16.541
  10:29:11 STEP 2948 H=2948:00 +CPU= 18.841
  10:29:40 STEP 2949 H=2949:00 +CPU= 28.919
  10:29:58 STEP 2950 H=2950:00 +CPU= 17.873
  10:30:16 STEP 2951 H=2951:00 +CPU= 17.485
  10:30:49 STEP 2952 H=2952:00 +CPU= 32.476
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174653/ICMGGhq0f+002952
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0009_1989050100_123_958_12174653_0_r217548099_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
10:34:12 (1746949): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:24:36 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 10,623,600 55,588 0.0052
09 May 2023 16:24:11 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 7,776,000 36,944 0.0048
09 May 2023 16:24:06 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 6,912,000 32,805 0.0047
09 May 2023 16:24:01 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 6,048,000 28,747 0.0048
09 May 2023 16:23:55 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 5,184,000 24,696 0.0048
09 May 2023 16:23:46 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 4,320,000 20,846 0.0048
09 May 2023 16:23:41 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 3,456,000 16,828 0.0049
09 May 2023 16:23:36 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 2,592,000 12,902 0.0050
09 May 2023 16:23:32 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 1,728,000 8,792 0.0051
09 May 2023 16:23:27 1532237 22259322 oifs_43r3_ps_0009_1989050100_123_958_12174653_0 864,000 4,600 0.0053


©2024 cpdn.org