climateprediction.net home page
Task 22265293

Task 22265293

Name oifs_43r3_ps_0690_1994050100_123_963_12180334_0
Workunit 12180334
Created 22 Dec 2022, 6:23:57 UTC
Sent 31 Dec 2022, 18:55:58 UTC
Report deadline 30 Jan 2023, 18:55:58 UTC
Received 2 Jan 2023, 11:08:18 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1494992
Run time 16 hours 47 min 42 sec
CPU time 16 hours 20 min 55 sec
Validate state Invalid
Credit 10,886.40
Device peak FLOPS 5.32 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.34 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,379.81 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ithub/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:2079:2079] [20230102:053719:1672634239:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:2079:2079] [20230102:053719:1672634239:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:2079:2079] [20230102:053719:1672634239:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:2079:2079] [20230102:053719:1672634239:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:2079:2079] [20230102:053719:1672634239:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:38:05 STEP 1680 H=1680:00 +CPU= 38.511
  05:38:28 STEP 1681 H=1681:00 +CPU= 21.813
  05:38:53 STEP 1682 H=1682:00 +CPU= 22.811
  05:39:27 STEP 1683 H=1683:00 +CPU= 34.370
  05:39:52 STEP 1684 H=1684:00 +CPU= 22.785
  05:40:15 STEP 1685 H=1685:00 +CPU= 22.003
  05:40:52 STEP 1686 H=1686:00 +CPU= 34.883
  05:41:16 STEP 1687 H=1687:00 +CPU= 22.089
  05:41:40 STEP 1688 H=1688:00 +CPU= 23.010
  05:42:15 STEP 1689 H=1689:00 +CPU= 34.341
  05:42:38 STEP 1690 H=1690:00 +CPU= 22.958
  05:43:02 STEP 1691 H=1691:00 +CPU= 22.193
  05:43:40 STEP 1692 H=1692:00 +CPU= 37.469
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001692
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001692
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001692
  05:44:03 STEP 1693 H=1693:00 +CPU= 21.899
  05:44:27 STEP 1694 H=1694:00 +CPU= 23.065
  05:45:02 STEP 1695 H=1695:00 +CPU= 34.228
  05:45:26 STEP 1696 H=1696:00 +CPU= 22.862
  05:45:49 STEP 1697 H=1697:00 +CPU= 22.021
  05:46:24 STEP 1698 H=1698:00 +CPU= 34.628
  05:46:46 STEP 1699 H=1699:00 +CPU= 21.583
  05:47:10 STEP 1700 H=1700:00 +CPU= 22.573
  05:47:46 STEP 1701 H=1701:00 +CPU= 34.262
  05:48:09 STEP 1702 H=1702:00 +CPU= 22.960
  05:48:33 STEP 1703 H=1703:00 +CPU= 23.091
  05:49:15 STEP 1704 H=1704:00 +CPU= 40.328
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001704
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001704
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001692
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_70.zip
Uploading the intermediate file: upload_file_70.zip
  05:49:38 STEP 1705 H=1705:00 +CPU= 22.021
  05:50:02 STEP 1706 H=1706:00 +CPU= 22.853
  05:50:38 STEP 1707 H=1707:00 +CPU= 34.569
  05:51:02 STEP 1708 H=1708:00 +CPU= 22.786
  05:51:25 STEP 1709 H=1709:00 +CPU= 22.261
  05:52:02 STEP 1710 H=1710:00 +CPU= 35.061
  05:52:25 STEP 1711 H=1711:00 +CPU= 22.223
  05:52:48 STEP 1712 H=1712:00 +CPU= 22.745
  05:53:24 STEP 1713 H=1713:00 +CPU= 34.156
  05:53:48 STEP 1714 H=1714:00 +CPU= 22.710
  05:54:11 STEP 1715 H=1715:00 +CPU= 22.136
  05:54:51 STEP 1716 H=1716:00 +CPU= 38.176
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001716
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001716
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001716
  05:55:14 STEP 1717 H=1717:00 +CPU= 22.122
  05:55:38 STEP 1718 H=1718:00 +CPU= 22.912
  05:56:14 STEP 1719 H=1719:00 +CPU= 34.122
  05:56:37 STEP 1720 H=1720:00 +CPU= 22.779
  05:57:00 STEP 1721 H=1721:00 +CPU= 21.919
  05:57:37 STEP 1722 H=1722:00 +CPU= 35.024
  05:58:00 STEP 1723 H=1723:00 +CPU= 22.222
  05:58:24 STEP 1724 H=1724:00 +CPU= 22.862
  05:58:59 STEP 1725 H=1725:00 +CPU= 34.172
  05:59:22 STEP 1726 H=1726:00 +CPU= 22.815
  05:59:46 STEP 1727 H=1727:00 +CPU= 22.810
  06:00:29 STEP 1728 H=1728:00 +CPU= 40.609
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001728
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001728
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001716
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_71.zip
Uploading the intermediate file: upload_file_71.zip
  06:00:52 STEP 1729 H=1729:00 +CPU= 22.249
  06:01:16 STEP 1730 H=1730:00 +CPU= 22.770
  06:01:51 STEP 1731 H=1731:00 +CPU= 34.072
  06:02:15 STEP 1732 H=1732:00 +CPU= 22.876
  06:02:37 STEP 1733 H=1733:00 +CPU= 22.033
  06:03:15 STEP 1734 H=1734:00 +CPU= 34.992
  06:03:38 STEP 1735 H=1735:00 +CPU= 22.024
  06:04:02 STEP 1736 H=1736:00 +CPU= 22.798
  06:04:37 STEP 1737 H=1737:00 +CPU= 34.466
  06:05:01 STEP 1738 H=1738:00 +CPU= 22.970
  06:05:24 STEP 1739 H=1739:00 +CPU= 22.374
  06:06:05 STEP 1740 H=1740:00 +CPU= 38.239
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001740
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001740
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001740
  06:06:29 STEP 1741 H=1741:00 +CPU= 22.377
  06:06:53 STEP 1742 H=1742:00 +CPU= 23.282
  06:07:28 STEP 1743 H=1743:00 +CPU= 34.259
  06:07:51 STEP 1744 H=1744:00 +CPU= 23.019
  06:08:14 STEP 1745 H=1745:00 +CPU= 21.777
  06:08:51 STEP 1746 H=1746:00 +CPU= 35.406
  06:09:12 STEP 1747 H=1747:00 +CPU= 21.484
  06:09:35 STEP 1748 H=1748:00 +CPU= 22.335
  06:10:10 STEP 1749 H=1749:00 +CPU= 34.550
  06:10:33 STEP 1750 H=1750:00 +CPU= 22.649
  06:10:56 STEP 1751 H=1751:00 +CPU= 22.556
  06:11:37 STEP 1752 H=1752:00 +CPU= 40.883
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001752
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001752
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001740
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_72.zip
Uploading the intermediate file: upload_file_72.zip
  06:12:00 STEP 1753 H=1753:00 +CPU= 22.243
  06:12:24 STEP 1754 H=1754:00 +CPU= 22.552
  06:13:01 STEP 1755 H=1755:00 +CPU= 33.987
  06:13:25 STEP 1756 H=1756:00 +CPU= 22.464
  06:13:48 STEP 1757 H=1757:00 +CPU= 21.731
  06:14:25 STEP 1758 H=1758:00 +CPU= 34.530
  06:14:49 STEP 1759 H=1759:00 +CPU= 21.908
  06:15:13 STEP 1760 H=1760:00 +CPU= 22.542
  06:15:50 STEP 1761 H=1761:00 +CPU= 33.836
  06:16:14 STEP 1762 H=1762:00 +CPU= 22.569
  06:16:38 STEP 1763 H=1763:00 +CPU= 21.708
  06:17:18 STEP 1764 H=1764:00 +CPU= 37.952
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001764
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001764
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001764
  06:17:39 STEP 1765 H=1765:00 +CPU= 21.574
  06:18:02 STEP 1766 H=1766:00 +CPU= 22.332
  06:18:36 STEP 1767 H=1767:00 +CPU= 33.200
  06:18:58 STEP 1768 H=1768:00 +CPU= 22.486
  06:19:20 STEP 1769 H=1769:00 +CPU= 21.908
  06:19:57 STEP 1770 H=1770:00 +CPU= 35.053
  06:20:19 STEP 1771 H=1771:00 +CPU= 22.353
  06:20:43 STEP 1772 H=1772:00 +CPU= 22.738
  06:21:18 STEP 1773 H=1773:00 +CPU= 33.802
  06:21:41 STEP 1774 H=1774:00 +CPU= 22.685
  06:22:04 STEP 1775 H=1775:00 +CPU= 22.754
  06:22:46 STEP 1776 H=1776:00 +CPU= 40.461
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001776
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001776
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001764
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_73.zip
Uploading the intermediate file: upload_file_73.zip
  06:23:10 STEP 1777 H=1777:00 +CPU= 22.011
  06:23:33 STEP 1778 H=1778:00 +CPU= 22.913
  06:24:08 STEP 1779 H=1779:00 +CPU= 33.579
  06:24:31 STEP 1780 H=1780:00 +CPU= 22.296
  06:24:54 STEP 1781 H=1781:00 +CPU= 22.257
  06:25:30 STEP 1782 H=1782:00 +CPU= 35.091
  06:25:52 STEP 1783 H=1783:00 +CPU= 21.633
  06:26:15 STEP 1784 H=1784:00 +CPU= 22.509
  06:26:50 STEP 1785 H=1785:00 +CPU= 34.045
  06:27:13 STEP 1786 H=1786:00 +CPU= 22.912
  06:27:36 STEP 1787 H=1787:00 +CPU= 22.371
  06:28:16 STEP 1788 H=1788:00 +CPU= 38.780
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001788
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001788
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001788
  06:28:38 STEP 1789 H=1789:00 +CPU= 22.185
  06:29:01 STEP 1790 H=1790:00 +CPU= 22.646
  06:29:38 STEP 1791 H=1791:00 +CPU= 34.951
  06:30:01 STEP 1792 H=1792:00 +CPU= 22.976
  06:30:28 STEP 1793 H=1793:00 +CPU= 25.583
  06:31:17 STEP 1794 H=1794:00 +CPU= 45.494
  06:31:48 STEP 1795 H=1795:00 +CPU= 28.862
  06:32:16 STEP 1796 H=1796:00 +CPU= 26.288
  06:32:51 STEP 1797 H=1797:00 +CPU= 34.689
  06:33:14 STEP 1798 H=1798:00 +CPU= 22.702
  06:33:38 STEP 1799 H=1799:00 +CPU= 23.167
  06:34:19 STEP 1800 H=1800:00 +CPU= 39.856
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001800
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001800
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001788
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_74.zip
Uploading the intermediate file: upload_file_74.zip
  06:34:41 STEP 1801 H=1801:00 +CPU= 21.844
  06:35:04 STEP 1802 H=1802:00 +CPU= 22.565
  06:35:39 STEP 1803 H=1803:00 +CPU= 34.450
  06:36:02 STEP 1804 H=1804:00 +CPU= 22.921
  06:36:25 STEP 1805 H=1805:00 +CPU= 22.334
  06:36:59 STEP 1806 H=1806:00 +CPU= 34.019
  06:37:22 STEP 1807 H=1807:00 +CPU= 21.912
  06:37:44 STEP 1808 H=1808:00 +CPU= 22.169
  06:38:17 STEP 1809 H=1809:00 +CPU= 33.059
  06:38:40 STEP 1810 H=1810:00 +CPU= 22.430
  06:39:03 STEP 1811 H=1811:00 +CPU= 22.184
  06:39:42 STEP 1812 H=1812:00 +CPU= 37.562
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001812
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001812
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001812
  06:40:04 STEP 1813 H=1813:00 +CPU= 21.630
  06:40:27 STEP 1814 H=1814:00 +CPU= 22.513
  06:41:01 STEP 1815 H=1815:00 +CPU= 34.505
  06:41:25 STEP 1816 H=1816:00 +CPU= 22.912
  06:41:47 STEP 1817 H=1817:00 +CPU= 22.485
  06:42:23 STEP 1818 H=1818:00 +CPU= 34.984
  06:42:45 STEP 1819 H=1819:00 +CPU= 21.924
  06:43:08 STEP 1820 H=1820:00 +CPU= 22.605
  06:43:42 STEP 1821 H=1821:00 +CPU= 34.164
  06:44:05 STEP 1822 H=1822:00 +CPU= 22.969
  06:44:29 STEP 1823 H=1823:00 +CPU= 23.105
  06:45:10 STEP 1824 H=1824:00 +CPU= 41.074
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001824
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001824
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001812
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_75.zip
Uploading the intermediate file: upload_file_75.zip
  06:45:33 STEP 1825 H=1825:00 +CPU= 22.483
  06:45:56 STEP 1826 H=1826:00 +CPU= 23.065
  06:46:30 STEP 1827 H=1827:00 +CPU= 33.963
  06:46:53 STEP 1828 H=1828:00 +CPU= 22.349
  06:47:16 STEP 1829 H=1829:00 +CPU= 22.327
  06:47:52 STEP 1830 H=1830:00 +CPU= 35.113
  06:48:14 STEP 1831 H=1831:00 +CPU= 22.199
  06:48:38 STEP 1832 H=1832:00 +CPU= 23.269
  06:49:13 STEP 1833 H=1833:00 +CPU= 34.209
  06:49:36 STEP 1834 H=1834:00 +CPU= 23.174
  06:49:59 STEP 1835 H=1835:00 +CPU= 22.747
  06:50:48 STEP 1836 H=1836:00 +CPU= 45.637
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001836
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001836
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001836
  06:51:20 STEP 1837 H=1837:00 +CPU= 29.586
  06:51:53 STEP 1838 H=1838:00 +CPU= 30.229
  06:52:30 STEP 1839 H=1839:00 +CPU= 36.376
  06:52:54 STEP 1840 H=1840:00 +CPU= 22.644
  06:53:17 STEP 1841 H=1841:00 +CPU= 22.227
  06:53:52 STEP 1842 H=1842:00 +CPU= 34.694
  06:54:14 STEP 1843 H=1843:00 +CPU= 22.072
  06:54:38 STEP 1844 H=1844:00 +CPU= 23.019
  06:55:12 STEP 1845 H=1845:00 +CPU= 33.877
  06:55:35 STEP 1846 H=1846:00 +CPU= 22.679
  06:55:59 STEP 1847 H=1847:00 +CPU= 22.959
  06:56:41 STEP 1848 H=1848:00 +CPU= 40.825
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_76.zip
Uploading the intermediate file: upload_file_76.zip
  06:57:03 STEP 1849 H=1849:00 +CPU= 22.048
  06:57:26 STEP 1850 H=1850:00 +CPU= 22.417
  06:58:02 STEP 1851 H=1851:00 +CPU= 34.546
  06:58:25 STEP 1852 H=1852:00 +CPU= 22.687
  06:58:48 STEP 1853 H=1853:00 +CPU= 21.981
  06:59:24 STEP 1854 H=1854:00 +CPU= 35.004
  06:59:46 STEP 1855 H=1855:00 +CPU= 22.102
  07:00:09 STEP 1856 H=1856:00 +CPU= 22.559
  07:00:42 STEP 1857 H=1857:00 +CPU= 33.178
  07:01:05 STEP 1858 H=1858:00 +CPU= 22.307
  07:01:26 STEP 1859 H=1859:00 +CPU= 21.457
  07:02:04 STEP 1860 H=1860:00 +CPU= 37.120
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001860
  07:02:27 STEP 1861 H=1861:00 +CPU= 22.376
  07:02:50 STEP 1862 H=1862:00 +CPU= 22.828
  07:03:25 STEP 1863 H=1863:00 +CPU= 34.194
  07:03:48 STEP 1864 H=1864:00 +CPU= 22.602
  07:04:10 STEP 1865 H=1865:00 +CPU= 21.590
  07:04:45 STEP 1866 H=1866:00 +CPU= 34.746
  07:05:08 STEP 1867 H=1867:00 +CPU= 22.100
  07:05:30 STEP 1868 H=1868:00 +CPU= 22.221
  07:06:04 STEP 1869 H=1869:00 +CPU= 33.231
  07:06:27 STEP 1870 H=1870:00 +CPU= 22.632
  07:06:50 STEP 1871 H=1871:00 +CPU= 23.116
  07:07:31 STEP 1872 H=1872:00 +CPU= 40.653
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_77.zip
Uploading the intermediate file: upload_file_77.zip
  07:07:53 STEP 1873 H=1873:00 +CPU= 21.617
  07:08:16 STEP 1874 H=1874:00 +CPU= 22.543
  07:08:49 STEP 1875 H=1875:00 +CPU= 33.118
  07:09:12 STEP 1876 H=1876:00 +CPU= 22.187
  07:09:34 STEP 1877 H=1877:00 +CPU= 22.058
  07:10:11 STEP 1878 H=1878:00 +CPU= 35.082
  07:10:32 STEP 1879 H=1879:00 +CPU= 21.598
  07:10:56 STEP 1880 H=1880:00 +CPU= 22.685
  07:11:31 STEP 1881 H=1881:00 +CPU= 34.787
  07:11:54 STEP 1882 H=1882:00 +CPU= 22.619
  07:12:17 STEP 1883 H=1883:00 +CPU= 22.504
  07:12:57 STEP 1884 H=1884:00 +CPU= 38.393
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001884
  07:13:20 STEP 1885 H=1885:00 +CPU= 22.144
  07:13:43 STEP 1886 H=1886:00 +CPU= 23.231
  07:14:19 STEP 1887 H=1887:00 +CPU= 34.248
  07:14:43 STEP 1888 H=1888:00 +CPU= 23.013
  07:15:06 STEP 1889 H=1889:00 +CPU= 22.259
  07:15:42 STEP 1890 H=1890:00 +CPU= 34.978
  07:16:04 STEP 1891 H=1891:00 +CPU= 22.069
  07:16:28 STEP 1892 H=1892:00 +CPU= 22.726
  07:17:04 STEP 1893 H=1893:00 +CPU= 34.509
  07:17:27 STEP 1894 H=1894:00 +CPU= 22.796
  07:17:51 STEP 1895 H=1895:00 +CPU= 22.782
  07:18:33 STEP 1896 H=1896:00 +CPU= 40.898
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_78.zip
Uploading the intermediate file: upload_file_78.zip
  07:18:55 STEP 1897 H=1897:00 +CPU= 22.224
  07:19:19 STEP 1898 H=1898:00 +CPU= 22.795
  07:19:55 STEP 1899 H=1899:00 +CPU= 34.408
  07:20:18 STEP 1900 H=1900:00 +CPU= 22.747
  07:20:41 STEP 1901 H=1901:00 +CPU= 22.235
  07:21:17 STEP 1902 H=1902:00 +CPU= 34.847
  07:21:39 STEP 1903 H=1903:00 +CPU= 21.706
  07:22:02 STEP 1904 H=1904:00 +CPU= 22.629
  07:22:36 STEP 1905 H=1905:00 +CPU= 33.515
  07:22:59 STEP 1906 H=1906:00 +CPU= 22.532
  07:23:22 STEP 1907 H=1907:00 +CPU= 22.238
  07:24:01 STEP 1908 H=1908:00 +CPU= 38.332
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001908
  07:24:24 STEP 1909 H=1909:00 +CPU= 21.849
  07:24:47 STEP 1910 H=1910:00 +CPU= 23.333
  07:25:22 STEP 1911 H=1911:00 +CPU= 34.262
  07:25:45 STEP 1912 H=1912:00 +CPU= 22.437
  07:26:08 STEP 1913 H=1913:00 +CPU= 22.343
  07:26:43 STEP 1914 H=1914:00 +CPU= 34.794
  07:27:05 STEP 1915 H=1915:00 +CPU= 21.831
  07:27:28 STEP 1916 H=1916:00 +CPU= 22.501
  07:28:03 STEP 1917 H=1917:00 +CPU= 33.885
  07:28:27 STEP 1918 H=1918:00 +CPU= 22.548
  07:28:50 STEP 1919 H=1919:00 +CPU= 22.508
  07:29:32 STEP 1920 H=1920:00 +CPU= 41.104
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_79.zip
Uploading the intermediate file: upload_file_79.zip
  07:29:55 STEP 1921 H=1921:00 +CPU= 21.733
Uploading trickle at timestep: 6912000
  07:30:18 STEP 1922 H=1922:00 +CPU= 22.443
  07:30:53 STEP 1923 H=1923:00 +CPU= 34.067
  07:31:17 STEP 1924 H=1924:00 +CPU= 22.592
  07:31:40 STEP 1925 H=1925:00 +CPU= 21.990
  07:32:16 STEP 1926 H=1926:00 +CPU= 35.124
  07:32:39 STEP 1927 H=1927:00 +CPU= 22.381
  07:33:02 STEP 1928 H=1928:00 +CPU= 22.803
  07:33:38 STEP 1929 H=1929:00 +CPU= 34.384
  07:34:01 STEP 1930 H=1930:00 +CPU= 22.951
  07:34:23 STEP 1931 H=1931:00 +CPU= 22.214
  07:35:02 STEP 1932 H=1932:00 +CPU= 38.048
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001932
  07:35:25 STEP 1933 H=1933:00 +CPU= 22.495
  07:35:49 STEP 1934 H=1934:00 +CPU= 23.181
  07:36:24 STEP 1935 H=1935:00 +CPU= 34.195
  07:36:48 STEP 1936 H=1936:00 +CPU= 22.643
  07:37:10 STEP 1937 H=1937:00 +CPU= 21.730
  07:37:45 STEP 1938 H=1938:00 +CPU= 34.831
  07:38:07 STEP 1939 H=1939:00 +CPU= 21.685
  07:38:30 STEP 1940 H=1940:00 +CPU= 22.332
  07:39:04 STEP 1941 H=1941:00 +CPU= 33.736
  07:39:27 STEP 1942 H=1942:00 +CPU= 22.378
  07:39:49 STEP 1943 H=1943:00 +CPU= 22.418
  07:40:31 STEP 1944 H=1944:00 +CPU= 40.623
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_80.zip
Uploading the intermediate file: upload_file_80.zip
  07:40:53 STEP 1945 H=1945:00 +CPU= 22.078
  07:41:16 STEP 1946 H=1946:00 +CPU= 23.055
  07:41:51 STEP 1947 H=1947:00 +CPU= 34.356
  07:42:15 STEP 1948 H=1948:00 +CPU= 23.267
  07:42:38 STEP 1949 H=1949:00 +CPU= 22.448
  07:43:14 STEP 1950 H=1950:00 +CPU= 35.731
  07:43:37 STEP 1951 H=1951:00 +CPU= 22.347
  07:44:01 STEP 1952 H=1952:00 +CPU= 23.117
  07:44:36 STEP 1953 H=1953:00 +CPU= 34.779
  07:45:00 STEP 1954 H=1954:00 +CPU= 23.233
  07:45:27 STEP 1955 H=1955:00 +CPU= 25.525
  07:46:21 STEP 1956 H=1956:00 +CPU= 49.829
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001956
  07:46:52 STEP 1957 H=1957:00 +CPU= 29.309
  07:47:19 STEP 1958 H=1958:00 +CPU= 25.301
  07:47:53 STEP 1959 H=1959:00 +CPU= 33.412
  07:48:15 STEP 1960 H=1960:00 +CPU= 22.670
  07:48:38 STEP 1961 H=1961:00 +CPU= 22.323
  07:49:13 STEP 1962 H=1962:00 +CPU= 34.231
  07:49:34 STEP 1963 H=1963:00 +CPU= 21.494
  07:49:57 STEP 1964 H=1964:00 +CPU= 22.717
  07:50:31 STEP 1965 H=1965:00 +CPU= 33.181
  07:50:53 STEP 1966 H=1966:00 +CPU= 22.103
  07:51:17 STEP 1967 H=1967:00 +CPU= 22.951
  07:51:59 STEP 1968 H=1968:00 +CPU= 41.171
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_81.zip
Uploading the intermediate file: upload_file_81.zip
  07:52:21 STEP 1969 H=1969:00 +CPU= 22.061
  07:52:44 STEP 1970 H=1970:00 +CPU= 23.180
  07:53:19 STEP 1971 H=1971:00 +CPU= 34.472
  07:53:42 STEP 1972 H=1972:00 +CPU= 22.587
  07:54:05 STEP 1973 H=1973:00 +CPU= 22.412
  07:54:40 STEP 1974 H=1974:00 +CPU= 35.396
  07:55:03 STEP 1975 H=1975:00 +CPU= 22.125
  07:55:26 STEP 1976 H=1976:00 +CPU= 22.873
  07:56:01 STEP 1977 H=1977:00 +CPU= 34.514
  07:56:24 STEP 1978 H=1978:00 +CPU= 23.305
  07:56:47 STEP 1979 H=1979:00 +CPU= 22.460
  07:57:25 STEP 1980 H=1980:00 +CPU= 38.194
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001980
  07:57:47 STEP 1981 H=1981:00 +CPU= 21.673
  07:58:10 STEP 1982 H=1982:00 +CPU= 22.153
  07:58:43 STEP 1983 H=1983:00 +CPU= 33.273
  07:59:06 STEP 1984 H=1984:00 +CPU= 22.291
  07:59:27 STEP 1985 H=1985:00 +CPU= 21.359
  08:00:03 STEP 1986 H=1986:00 +CPU= 35.064
  08:00:26 STEP 1987 H=1987:00 +CPU= 22.445
  08:00:49 STEP 1988 H=1988:00 +CPU= 23.285
  08:01:24 STEP 1989 H=1989:00 +CPU= 34.057
  08:01:47 STEP 1990 H=1990:00 +CPU= 22.507
  08:02:10 STEP 1991 H=1991:00 +CPU= 22.899
  08:02:51 STEP 1992 H=1992:00 +CPU= 40.607
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_82.zip
Uploading the intermediate file: upload_file_82.zip
  08:03:13 STEP 1993 H=1993:00 +CPU= 21.616
  08:03:35 STEP 1994 H=1994:00 +CPU= 22.206
  08:04:09 STEP 1995 H=1995:00 +CPU= 33.358
  08:04:31 STEP 1996 H=1996:00 +CPU= 22.196
  08:04:53 STEP 1997 H=1997:00 +CPU= 21.560
  08:05:27 STEP 1998 H=1998:00 +CPU= 34.252
  08:05:49 STEP 1999 H=1999:00 +CPU= 21.632
  08:06:11 STEP 2000 H=2000:00 +CPU= 22.175
  08:06:45 STEP 2001 H=2001:00 +CPU= 33.550
  08:07:08 STEP 2002 H=2002:00 +CPU= 23.207
  08:07:31 STEP 2003 H=2003:00 +CPU= 22.234
  08:08:10 STEP 2004 H=2004:00 +CPU= 38.585
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002004
  08:08:32 STEP 2005 H=2005:00 +CPU= 22.335
  08:08:55 STEP 2006 H=2006:00 +CPU= 23.011
  08:09:29 STEP 2007 H=2007:00 +CPU= 33.557
  08:09:52 STEP 2008 H=2008:00 +CPU= 22.781
  08:10:14 STEP 2009 H=2009:00 +CPU= 22.042
  08:10:49 STEP 2010 H=2010:00 +CPU= 34.719
  08:11:11 STEP 2011 H=2011:00 +CPU= 21.638
  08:11:33 STEP 2012 H=2012:00 +CPU= 22.277
  08:12:08 STEP 2013 H=2013:00 +CPU= 34.052
  08:12:31 STEP 2014 H=2014:00 +CPU= 23.055
  08:12:54 STEP 2015 H=2015:00 +CPU= 22.951
  08:13:34 STEP 2016 H=2016:00 +CPU= 39.939
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_83.zip
Uploading the intermediate file: upload_file_83.zip
  08:13:56 STEP 2017 H=2017:00 +CPU= 21.999
  08:14:20 STEP 2018 H=2018:00 +CPU= 23.007
  08:14:55 STEP 2019 H=2019:00 +CPU= 34.800
  08:15:18 STEP 2020 H=2020:00 +CPU= 23.239
  08:15:40 STEP 2021 H=2021:00 +CPU= 22.366
  08:16:16 STEP 2022 H=2022:00 +CPU= 35.397
  08:16:38 STEP 2023 H=2023:00 +CPU= 22.196
  08:17:01 STEP 2024 H=2024:00 +CPU= 22.938
  08:17:36 STEP 2025 H=2025:00 +CPU= 34.452
  08:17:59 STEP 2026 H=2026:00 +CPU= 22.541
  08:18:21 STEP 2027 H=2027:00 +CPU= 21.512
  08:18:58 STEP 2028 H=2028:00 +CPU= 36.925
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002028
  08:19:20 STEP 2029 H=2029:00 +CPU= 21.933
  08:19:43 STEP 2030 H=2030:00 +CPU= 22.583
  08:20:17 STEP 2031 H=2031:00 +CPU= 34.466
  08:20:40 STEP 2032 H=2032:00 +CPU= 23.007
  08:21:02 STEP 2033 H=2033:00 +CPU= 21.777
  08:21:37 STEP 2034 H=2034:00 +CPU= 34.035
  08:21:59 STEP 2035 H=2035:00 +CPU= 21.870
  08:22:22 STEP 2036 H=2036:00 +CPU= 22.982
  08:22:57 STEP 2037 H=2037:00 +CPU= 34.492
  08:23:20 STEP 2038 H=2038:00 +CPU= 23.130
  08:23:43 STEP 2039 H=2039:00 +CPU= 22.952
  08:24:23 STEP 2040 H=2040:00 +CPU= 39.917
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_84.zip
Uploading the intermediate file: upload_file_84.zip
  08:24:46 STEP 2041 H=2041:00 +CPU= 22.336
  08:25:09 STEP 2042 H=2042:00 +CPU= 22.837
  08:25:43 STEP 2043 H=2043:00 +CPU= 33.681
  08:26:06 STEP 2044 H=2044:00 +CPU= 22.976
  08:26:28 STEP 2045 H=2045:00 +CPU= 21.763
  08:27:03 STEP 2046 H=2046:00 +CPU= 34.670
  08:27:24 STEP 2047 H=2047:00 +CPU= 21.436
  08:27:47 STEP 2048 H=2048:00 +CPU= 22.293
  08:28:20 STEP 2049 H=2049:00 +CPU= 33.381
  08:28:43 STEP 2050 H=2050:00 +CPU= 22.269
  08:29:04 STEP 2051 H=2051:00 +CPU= 21.490
  08:29:42 STEP 2052 H=2052:00 +CPU= 37.368
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002052
  08:30:04 STEP 2053 H=2053:00 +CPU= 22.071
  08:30:27 STEP 2054 H=2054:00 +CPU= 22.839
  08:31:02 STEP 2055 H=2055:00 +CPU= 34.166
  08:31:25 STEP 2056 H=2056:00 +CPU= 23.051
  08:31:47 STEP 2057 H=2057:00 +CPU= 22.352
  08:32:23 STEP 2058 H=2058:00 +CPU= 35.330
  08:32:45 STEP 2059 H=2059:00 +CPU= 21.546
  08:33:07 STEP 2060 H=2060:00 +CPU= 22.474
  08:33:41 STEP 2061 H=2061:00 +CPU= 33.265
  08:34:04 STEP 2062 H=2062:00 +CPU= 22.552
  08:34:26 STEP 2063 H=2063:00 +CPU= 22.279
  08:35:06 STEP 2064 H=2064:00 +CPU= 39.649
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_85.zip
Uploading the intermediate file: upload_file_85.zip
  08:35:28 STEP 2065 H=2065:00 +CPU= 22.179
  08:35:52 STEP 2066 H=2066:00 +CPU= 23.113
  08:36:26 STEP 2067 H=2067:00 +CPU= 33.850
  08:36:49 STEP 2068 H=2068:00 +CPU= 23.121
  08:37:11 STEP 2069 H=2069:00 +CPU= 21.992
  08:37:46 STEP 2070 H=2070:00 +CPU= 34.754
  08:38:09 STEP 2071 H=2071:00 +CPU= 22.596
  08:38:32 STEP 2072 H=2072:00 +CPU= 22.809
  08:39:06 STEP 2073 H=2073:00 +CPU= 33.469
  08:39:29 STEP 2074 H=2074:00 +CPU= 23.020
  08:39:51 STEP 2075 H=2075:00 +CPU= 22.494
  08:40:30 STEP 2076 H=2076:00 +CPU= 38.399
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002076
  08:40:52 STEP 2077 H=2077:00 +CPU= 21.469
  08:41:14 STEP 2078 H=2078:00 +CPU= 22.127
  08:41:47 STEP 2079 H=2079:00 +CPU= 33.074
  08:42:10 STEP 2080 H=2080:00 +CPU= 22.122
  08:42:32 STEP 2081 H=2081:00 +CPU= 22.382
  08:43:08 STEP 2082 H=2082:00 +CPU= 35.501
  08:43:30 STEP 2083 H=2083:00 +CPU= 22.273
  08:43:52 STEP 2084 H=2084:00 +CPU= 21.987
  08:44:27 STEP 2085 H=2085:00 +CPU= 33.908
  08:44:50 STEP 2086 H=2086:00 +CPU= 22.658
  08:45:12 STEP 2087 H=2087:00 +CPU= 22.536
  08:45:53 STEP 2088 H=2088:00 +CPU= 39.617
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_86.zip
Uploading the intermediate file: upload_file_86.zip
  08:46:16 STEP 2089 H=2089:00 +CPU= 21.990
  08:46:40 STEP 2090 H=2090:00 +CPU= 22.664
  08:47:15 STEP 2091 H=2091:00 +CPU= 34.244
  08:47:38 STEP 2092 H=2092:00 +CPU= 22.389
  08:48:00 STEP 2093 H=2093:00 +CPU= 22.016
  08:48:36 STEP 2094 H=2094:00 +CPU= 35.409
  08:48:59 STEP 2095 H=2095:00 +CPU= 22.433
  08:49:23 STEP 2096 H=2096:00 +CPU= 23.059
  08:49:58 STEP 2097 H=2097:00 +CPU= 33.987
  08:50:22 STEP 2098 H=2098:00 +CPU= 23.156
  08:50:45 STEP 2099 H=2099:00 +CPU= 22.286
  08:51:24 STEP 2100 H=2100:00 +CPU= 38.578
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002100
  08:51:46 STEP 2101 H=2101:00 +CPU= 22.358
  08:52:10 STEP 2102 H=2102:00 +CPU= 22.796
  08:52:45 STEP 2103 H=2103:00 +CPU= 34.401
  08:53:07 STEP 2104 H=2104:00 +CPU= 22.370
  08:53:29 STEP 2105 H=2105:00 +CPU= 21.500
  08:54:04 STEP 2106 H=2106:00 +CPU= 34.690
  08:54:27 STEP 2107 H=2107:00 +CPU= 22.181
  08:54:50 STEP 2108 H=2108:00 +CPU= 22.579
  08:55:24 STEP 2109 H=2109:00 +CPU= 33.880
  08:55:47 STEP 2110 H=2110:00 +CPU= 22.873
  08:56:10 STEP 2111 H=2111:00 +CPU= 22.440
  08:56:51 STEP 2112 H=2112:00 +CPU= 40.749
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_87.zip
Uploading the intermediate file: upload_file_87.zip
  08:57:14 STEP 2113 H=2113:00 +CPU= 22.104
  08:57:37 STEP 2114 H=2114:00 +CPU= 23.162
  08:58:12 STEP 2115 H=2115:00 +CPU= 34.224
  08:58:35 STEP 2116 H=2116:00 +CPU= 22.745
  08:58:58 STEP 2117 H=2117:00 +CPU= 22.337
  08:59:34 STEP 2118 H=2118:00 +CPU= 35.047
  08:59:57 STEP 2119 H=2119:00 +CPU= 22.642
  09:00:21 STEP 2120 H=2120:00 +CPU= 23.027
  09:00:54 STEP 2121 H=2121:00 +CPU= 33.103
  09:01:17 STEP 2122 H=2122:00 +CPU= 22.492
  09:01:39 STEP 2123 H=2123:00 +CPU= 21.865
  09:02:17 STEP 2124 H=2124:00 +CPU= 37.469
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002124
  09:02:39 STEP 2125 H=2125:00 +CPU= 21.762
  09:03:01 STEP 2126 H=2126:00 +CPU= 22.011
  09:03:35 STEP 2127 H=2127:00 +CPU= 33.793
  09:03:58 STEP 2128 H=2128:00 +CPU= 22.633
  09:04:20 STEP 2129 H=2129:00 +CPU= 22.421
  09:04:56 STEP 2130 H=2130:00 +CPU= 35.078
  09:05:19 STEP 2131 H=2131:00 +CPU= 21.970
  09:05:42 STEP 2132 H=2132:00 +CPU= 22.589
  09:06:16 STEP 2133 H=2133:00 +CPU= 34.297
  09:06:39 STEP 2134 H=2134:00 +CPU= 22.485
  09:07:03 STEP 2135 H=2135:00 +CPU= 23.039
  09:07:44 STEP 2136 H=2136:00 +CPU= 40.263
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_88.zip
Uploading the intermediate file: upload_file_88.zip
  09:08:06 STEP 2137 H=2137:00 +CPU= 21.687
  09:08:30 STEP 2138 H=2138:00 +CPU= 23.111
  09:09:05 STEP 2139 H=2139:00 +CPU= 34.417
  09:09:27 STEP 2140 H=2140:00 +CPU= 22.597
  09:09:49 STEP 2141 H=2141:00 +CPU= 22.063
  09:10:24 STEP 2142 H=2142:00 +CPU= 34.120
  09:10:46 STEP 2143 H=2143:00 +CPU= 21.735
  09:11:09 STEP 2144 H=2144:00 +CPU= 22.115
  09:11:43 STEP 2145 H=2145:00 +CPU= 33.768
  09:12:05 STEP 2146 H=2146:00 +CPU= 22.171
  09:12:28 STEP 2147 H=2147:00 +CPU= 22.376
  09:13:06 STEP 2148 H=2148:00 +CPU= 37.694
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002148
  09:13:29 STEP 2149 H=2149:00 +CPU= 22.475
  09:13:54 STEP 2150 H=2150:00 +CPU= 22.750
  09:14:30 STEP 2151 H=2151:00 +CPU= 34.044
  09:14:54 STEP 2152 H=2152:00 +CPU= 22.541
  09:15:18 STEP 2153 H=2153:00 +CPU= 21.962
  09:15:56 STEP 2154 H=2154:00 +CPU= 34.800
  09:16:19 STEP 2155 H=2155:00 +CPU= 21.881
  09:16:43 STEP 2156 H=2156:00 +CPU= 22.687
  09:17:20 STEP 2157 H=2157:00 +CPU= 33.843
  09:17:44 STEP 2158 H=2158:00 +CPU= 22.448
  09:18:08 STEP 2159 H=2159:00 +CPU= 22.732
  09:18:49 STEP 2160 H=2160:00 +CPU= 40.468
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_89.zip
Uploading the intermediate file: upload_file_89.zip
  09:19:11 STEP 2161 H=2161:00 +CPU= 21.786
Uploading trickle at timestep: 7776000
  09:19:34 STEP 2162 H=2162:00 +CPU= 22.555
  09:20:10 STEP 2163 H=2163:00 +CPU= 34.161
  09:20:35 STEP 2164 H=2164:00 +CPU= 23.274
  09:20:57 STEP 2165 H=2165:00 +CPU= 22.572
  09:21:33 STEP 2166 H=2166:00 +CPU= 35.218
  09:21:56 STEP 2167 H=2167:00 +CPU= 21.940
  09:22:19 STEP 2168 H=2168:00 +CPU= 22.964
  09:22:53 STEP 2169 H=2169:00 +CPU= 33.428
  09:23:15 STEP 2170 H=2170:00 +CPU= 22.570
  09:23:37 STEP 2171 H=2171:00 +CPU= 21.635
  09:24:17 STEP 2172 H=2172:00 +CPU= 37.913
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002172
  09:24:40 STEP 2173 H=2173:00 +CPU= 22.223
  09:25:03 STEP 2174 H=2174:00 +CPU= 22.609
  09:25:39 STEP 2175 H=2175:00 +CPU= 34.669
  09:26:03 STEP 2176 H=2176:00 +CPU= 22.740
  09:26:25 STEP 2177 H=2177:00 +CPU= 21.824
  09:27:00 STEP 2178 H=2178:00 +CPU= 34.212
  09:27:23 STEP 2179 H=2179:00 +CPU= 22.321
  09:27:47 STEP 2180 H=2180:00 +CPU= 23.073
  09:28:21 STEP 2181 H=2181:00 +CPU= 33.777
  09:28:44 STEP 2182 H=2182:00 +CPU= 22.568
  09:29:07 STEP 2183 H=2183:00 +CPU= 23.035
  09:29:49 STEP 2184 H=2184:00 +CPU= 40.728
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_90.zip
Uploading the intermediate file: upload_file_90.zip
  09:30:11 STEP 2185 H=2185:00 +CPU= 21.678
  09:30:34 STEP 2186 H=2186:00 +CPU= 22.486
  09:31:09 STEP 2187 H=2187:00 +CPU= 34.208
  09:31:33 STEP 2188 H=2188:00 +CPU= 22.549
  09:31:55 STEP 2189 H=2189:00 +CPU= 22.076
  09:32:31 STEP 2190 H=2190:00 +CPU= 35.178
  09:32:54 STEP 2191 H=2191:00 +CPU= 22.218
  09:33:18 STEP 2192 H=2192:00 +CPU= 23.147
  09:33:54 STEP 2193 H=2193:00 +CPU= 34.714
  09:34:17 STEP 2194 H=2194:00 +CPU= 22.902
  09:34:39 STEP 2195 H=2195:00 +CPU= 22.058
  09:35:18 STEP 2196 H=2196:00 +CPU= 38.302
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002196
  09:35:41 STEP 2197 H=2197:00 +CPU= 22.324
  09:36:05 STEP 2198 H=2198:00 +CPU= 22.603
  09:36:41 STEP 2199 H=2199:00 +CPU= 34.318
  09:37:04 STEP 2200 H=2200:00 +CPU= 22.889
  09:37:29 STEP 2201 H=2201:00 +CPU= 22.511
  09:38:04 STEP 2202 H=2202:00 +CPU= 35.191
  09:38:26 STEP 2203 H=2203:00 +CPU= 21.795
  09:38:49 STEP 2204 H=2204:00 +CPU= 22.363
  09:39:26 STEP 2205 H=2205:00 +CPU= 34.394
  09:39:49 STEP 2206 H=2206:00 +CPU= 22.791
  09:40:13 STEP 2207 H=2207:00 +CPU= 23.090
  09:40:55 STEP 2208 H=2208:00 +CPU= 40.859
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_91.zip
Uploading the intermediate file: upload_file_91.zip
  09:41:17 STEP 2209 H=2209:00 +CPU= 21.129
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1994050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0690
(argv4) batchid: 963
(argv5) wuid: 12180334
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/13
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f8b60072dac1a09c8b3b37653952f162 to: /var/lib/boinc/slots/13/oifs_43r3_ps_0690_1994050100_123_963_12180334.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/13/oifs_43r3_ps_0690_1994050100_123_963_12180334.zip
ic_ancil_file: ic_ancil_12180334
ifsdata_file: ifsdata_12180334
climate_data_file: clim_data_12180334
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_08b4b7f939545c62b0c2947f92e21774 to: /var/lib/boinc/slots/13/ic_ancil_12180334.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/13/ic_ancil_12180334.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_443e3b63ff431b5654b0f38e717b884b to: /var/lib/boinc/slots/13/ifsdata/ifsdata_12180334.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/13/ifsdata/ifsdata_12180334.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/13/159l_2/clim_data_12180334.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/13/159l_2/clim_data_12180334.zip
Checking for progress XML file: /var/lib/boinc/slots/13/progress_file_12180334.xml
Opened progress file ok : /var/lib/boinc/slots/13/progress_file_12180334.xml
-- Model is restarting --
Adjusting last_iter, 2209, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/13/progress_file_12180334.xml
last_cpu_time: 58138
upload_file_number: 92
last_iter: 2207
last_upload: 7948800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339
The child process has been launched with process id: 21655
Executing the command: /var/lib/boinc/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:proxy-server:1:1:21655:21655] [20230102:105540:1672653340:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:56:19 STEP 2208 H=2208:00 +CPU= 36.170
  10:56:40 STEP 2209 H=2209:00 +CPU= 20.227
  10:57:01 STEP 2210 H=2210:00 +CPU= 20.833
  10:57:33 STEP 2211 H=2211:00 +CPU= 31.615
  10:57:54 STEP 2212 H=2212:00 +CPU= 21.433
  10:58:15 STEP 2213 H=2213:00 +CPU= 20.673
  10:58:48 STEP 2214 H=2214:00 +CPU= 32.312
  10:59:08 STEP 2215 H=2215:00 +CPU= 20.188
  10:59:29 STEP 2216 H=2216:00 +CPU= 20.966
  11:00:02 STEP 2217 H=2217:00 +CPU= 32.179
  11:00:23 STEP 2218 H=2218:00 +CPU= 21.174
  11:00:43 STEP 2219 H=2219:00 +CPU= 20.224
  11:01:19 STEP 2220 H=2220:00 +CPU= 35.365
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002220
  11:01:40 STEP 2221 H=2221:00 +CPU= 20.676
  11:02:01 STEP 2222 H=2222:00 +CPU= 21.389
  11:02:34 STEP 2223 H=2223:00 +CPU= 32.052
  11:02:55 STEP 2224 H=2224:00 +CPU= 21.426
  11:03:16 STEP 2225 H=2225:00 +CPU= 20.444
  11:03:49 STEP 2226 H=2226:00 +CPU= 32.745
  11:04:10 STEP 2227 H=2227:00 +CPU= 20.690
  11:04:31 STEP 2228 H=2228:00 +CPU= 20.809
  11:05:03 STEP 2229 H=2229:00 +CPU= 32.674
  11:05:25 STEP 2230 H=2230:00 +CPU= 21.003
  11:05:47 STEP 2231 H=2231:00 +CPU= 21.468
  11:06:26 STEP 2232 H=2232:00 +CPU= 38.003
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180334/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0690_1994050100_123_963_12180334_0_r1739401339_92.zip
Uploading the intermediate file: upload_file_92.zip
  11:06:46 STEP 2233 H=2233:00 +CPU= 20.342
  11:07:08 STEP 2234 H=2234:00 +CPU= 21.126
  11:07:41 STEP 2235 H=2235:00 +CPU= 32.124
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:05:50 1494992 22265293 oifs_43r3_ps_0690_1994050100_123_963_12180334_0 7,776,000 56,850 0.0073
09 May 2023 16:05:50 1494992 22265293 oifs_43r3_ps_0690_1994050100_123_963_12180334_0 7,776,000 56,850 0.0073
09 May 2023 16:05:48 1494992 22265293 oifs_43r3_ps_0690_1994050100_123_963_12180334_0 6,912,000 50,397 0.0073
09 May 2023 16:05:22 1494992 22265293 oifs_43r3_ps_0690_1994050100_123_963_12180334_0 5,443,200 39,454 0.0072
09 May 2023 16:04:57 1494992 22265293 oifs_43r3_ps_0690_1994050100_123_963_12180334_0 2,937,600 21,447 0.0073
09 May 2023 16:04:52 1494992 22265293 oifs_43r3_ps_0690_1994050100_123_963_12180334_0 2,073,600 15,149 0.0073
09 May 2023 16:04:25 1494992 22265293 oifs_43r3_ps_0690_1994050100_123_963_12180334_0 864,000 6,326 0.0073


©2024 cpdn.org