climateprediction.net home page
Task 22303116

Task 22303116

Name oifs_43r3_ps_0661_2020050100_123_989_12206305_1
Workunit 12206305
Created 24 Jan 2023, 20:44:47 UTC
Sent 24 Jan 2023, 21:41:05 UTC
Report deadline 25 Mar 2023, 21:41:05 UTC
Received 2 Feb 2023, 21:14:40 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1527792
Run time 15 hours 37 min 21 sec
CPU time 14 hours 15 min 49 sec
Validate state Invalid
Credit 11,128.32
Device peak FLOPS 6.73 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.25 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,674.32 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
:32:42 STEP 1813 H=1813:00 +CPU= 20.049
  00:33:05 STEP 1814 H=1814:00 +CPU= 20.187
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:34:02 STEP 1815 H=1815:00 +CPU= 31.209
  00:34:27 STEP 1816 H=1816:00 +CPU= 21.408
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:35:01 STEP 1817 H=1817:00 +CPU= 20.644
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:35:50 STEP 1818 H=1818:00 +CPU= 32.934
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:36:26 STEP 1819 H=1819:00 +CPU= 21.209
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:37:11 STEP 1820 H=1820:00 +CPU= 20.425
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:38:38 STEP 1821 H=1821:00 +CPU= 31.413
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:39:24 STEP 1822 H=1822:00 +CPU= 20.887
  00:39:48 STEP 1823 H=1823:00 +CPU= 21.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:40:51 STEP 1824 H=1824:00 +CPU= 36.932
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_75.zip
Uploading the intermediate file: upload_file_75.zip
  00:41:11 STEP 1825 H=1825:00 +CPU= 18.763
  00:41:31 STEP 1826 H=1826:00 +CPU= 18.930
  00:42:01 STEP 1827 H=1827:00 +CPU= 28.437
  00:42:21 STEP 1828 H=1828:00 +CPU= 19.232
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:42:54 STEP 1829 H=1829:00 +CPU= 19.695
  00:43:28 STEP 1830 H=1830:00 +CPU= 30.015
  00:43:48 STEP 1831 H=1831:00 +CPU= 18.579
  00:44:09 STEP 1832 H=1832:00 +CPU= 19.295
  00:44:40 STEP 1833 H=1833:00 +CPU= 29.193
  00:45:01 STEP 1834 H=1834:00 +CPU= 19.254
  00:45:21 STEP 1835 H=1835:00 +CPU= 18.456
  00:45:55 STEP 1836 H=1836:00 +CPU= 32.109
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001836
  00:46:15 STEP 1837 H=1837:00 +CPU= 18.445
  00:46:36 STEP 1838 H=1838:00 +CPU= 19.170
  00:47:07 STEP 1839 H=1839:00 +CPU= 28.762
  00:47:27 STEP 1840 H=1840:00 +CPU= 19.349
  00:47:48 STEP 1841 H=1841:00 +CPU= 18.941
  00:48:20 STEP 1842 H=1842:00 +CPU= 30.000
  00:48:40 STEP 1843 H=1843:00 +CPU= 19.228
  00:49:01 STEP 1844 H=1844:00 +CPU= 19.607
  00:49:33 STEP 1845 H=1845:00 +CPU= 29.566
  00:49:54 STEP 1846 H=1846:00 +CPU= 19.270
  00:50:14 STEP 1847 H=1847:00 +CPU= 19.271
  00:50:54 STEP 1848 H=1848:00 +CPU= 36.626
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_76.zip
Uploading the intermediate file: upload_file_76.zip
  00:51:22 STEP 1849 H=1849:00 +CPU= 22.969
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:53:45 STEP 1850 H=1850:00 +CPU= 23.576
  00:54:23 STEP 1851 H=1851:00 +CPU= 33.796
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:01 STEP 1852 H=1852:00 +CPU= 25.070
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:47 STEP 1853 H=1853:00 +CPU= 21.851
  00:56:21 STEP 1854 H=1854:00 +CPU= 32.538
  00:56:42 STEP 1855 H=1855:00 +CPU= 20.318
  00:57:04 STEP 1856 H=1856:00 +CPU= 21.495
  00:57:37 STEP 1857 H=1857:00 +CPU= 31.232
  00:57:57 STEP 1858 H=1858:00 +CPU= 19.253
  00:58:18 STEP 1859 H=1859:00 +CPU= 18.920
  00:58:53 STEP 1860 H=1860:00 +CPU= 34.549
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001860
  00:59:14 STEP 1861 H=1861:00 +CPU= 20.215
  00:59:36 STEP 1862 H=1862:00 +CPU= 21.004
  01:00:09 STEP 1863 H=1863:00 +CPU= 31.360
  01:00:30 STEP 1864 H=1864:00 +CPU= 20.435
  01:00:51 STEP 1865 H=1865:00 +CPU= 20.311
  01:01:25 STEP 1866 H=1866:00 +CPU= 32.583
  01:01:46 STEP 1867 H=1867:00 +CPU= 20.206
  01:02:08 STEP 1868 H=1868:00 +CPU= 21.059
  01:02:40 STEP 1869 H=1869:00 +CPU= 31.106
  01:03:02 STEP 1870 H=1870:00 +CPU= 20.571
  01:03:24 STEP 1871 H=1871:00 +CPU= 21.245
  01:03:57 STEP 1872 H=1872:00 +CPU= 32.724
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_77.zip
Uploading the intermediate file: upload_file_77.zip
  01:04:16 STEP 1873 H=1873:00 +CPU= 18.285
  01:04:35 STEP 1874 H=1874:00 +CPU= 18.634
  01:05:05 STEP 1875 H=1875:00 +CPU= 29.234
  01:05:24 STEP 1876 H=1876:00 +CPU= 18.691
  01:05:43 STEP 1877 H=1877:00 +CPU= 18.006
  01:06:13 STEP 1878 H=1878:00 +CPU= 28.908
  01:06:32 STEP 1879 H=1879:00 +CPU= 18.371
  01:06:52 STEP 1880 H=1880:00 +CPU= 19.286
  01:07:20 STEP 1881 H=1881:00 +CPU= 27.957
  01:07:40 STEP 1882 H=1882:00 +CPU= 18.950
  01:07:59 STEP 1883 H=1883:00 +CPU= 18.545
  01:08:31 STEP 1884 H=1884:00 +CPU= 31.207
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001884
  01:08:50 STEP 1885 H=1885:00 +CPU= 18.287
  01:09:10 STEP 1886 H=1886:00 +CPU= 18.950
  01:09:38 STEP 1887 H=1887:00 +CPU= 28.393
  01:09:59 STEP 1888 H=1888:00 +CPU= 19.327
  01:10:18 STEP 1889 H=1889:00 +CPU= 18.512
  01:10:47 STEP 1890 H=1890:00 +CPU= 28.714
  01:11:06 STEP 1891 H=1891:00 +CPU= 18.149
  01:11:25 STEP 1892 H=1892:00 +CPU= 18.542
  01:11:53 STEP 1893 H=1893:00 +CPU= 28.279
  01:12:13 STEP 1894 H=1894:00 +CPU= 18.811
  01:12:32 STEP 1895 H=1895:00 +CPU= 18.900
  01:13:07 STEP 1896 H=1896:00 +CPU= 33.628
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_78.zip
Uploading the intermediate file: upload_file_78.zip
  01:13:26 STEP 1897 H=1897:00 +CPU= 18.458
  01:13:45 STEP 1898 H=1898:00 +CPU= 18.942
  01:14:15 STEP 1899 H=1899:00 +CPU= 29.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:14:49 STEP 1900 H=1900:00 +CPU= 18.980
  01:15:07 STEP 1901 H=1901:00 +CPU= 18.335
  01:15:37 STEP 1902 H=1902:00 +CPU= 29.374
  01:15:56 STEP 1903 H=1903:00 +CPU= 18.262
  01:16:16 STEP 1904 H=1904:00 +CPU= 19.047
  01:16:46 STEP 1905 H=1905:00 +CPU= 28.788
  01:17:05 STEP 1906 H=1906:00 +CPU= 19.016
  01:17:25 STEP 1907 H=1907:00 +CPU= 18.505
  01:17:57 STEP 1908 H=1908:00 +CPU= 31.550
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001908
  01:18:16 STEP 1909 H=1909:00 +CPU= 18.188
  01:18:35 STEP 1910 H=1910:00 +CPU= 18.698
  01:19:04 STEP 1911 H=1911:00 +CPU= 28.461
  01:19:24 STEP 1912 H=1912:00 +CPU= 19.152
  01:19:42 STEP 1913 H=1913:00 +CPU= 17.976
  01:20:12 STEP 1914 H=1914:00 +CPU= 28.584
  01:20:31 STEP 1915 H=1915:00 +CPU= 18.118
  01:20:50 STEP 1916 H=1916:00 +CPU= 18.872
  01:21:20 STEP 1917 H=1917:00 +CPU= 28.510
  01:21:39 STEP 1918 H=1918:00 +CPU= 18.962
  01:21:59 STEP 1919 H=1919:00 +CPU= 19.003
  01:22:33 STEP 1920 H=1920:00 +CPU= 32.681
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_79.zip
Uploading the intermediate file: upload_file_79.zip
  01:22:52 STEP 1921 H=1921:00 +CPU= 18.198
  01:23:11 STEP 1922 H=1922:00 +CPU= 19.120
  01:23:40 STEP 1923 H=1923:00 +CPU= 28.146
  01:24:00 STEP 1924 H=1924:00 +CPU= 18.793
  01:24:18 STEP 1925 H=1925:00 +CPU= 17.938
  01:24:48 STEP 1926 H=1926:00 +CPU= 28.980
  01:25:06 STEP 1927 H=1927:00 +CPU= 18.016
  01:25:26 STEP 1928 H=1928:00 +CPU= 19.110
  01:25:54 STEP 1929 H=1929:00 +CPU= 28.068
  01:26:13 STEP 1930 H=1930:00 +CPU= 18.338
  01:26:31 STEP 1931 H=1931:00 +CPU= 17.393
  01:27:03 STEP 1932 H=1932:00 +CPU= 31.382
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001932
  01:27:23 STEP 1933 H=1933:00 +CPU= 18.680
  01:27:42 STEP 1934 H=1934:00 +CPU= 18.872
  01:28:11 STEP 1935 H=1935:00 +CPU= 27.866
  01:28:31 STEP 1936 H=1936:00 +CPU= 18.933
  01:28:50 STEP 1937 H=1937:00 +CPU= 18.844
  01:29:20 STEP 1938 H=1938:00 +CPU= 29.093
  01:29:39 STEP 1939 H=1939:00 +CPU= 18.381
  01:29:59 STEP 1940 H=1940:00 +CPU= 19.022
  01:30:27 STEP 1941 H=1941:00 +CPU= 27.806
  01:30:47 STEP 1942 H=1942:00 +CPU= 19.181
  01:31:07 STEP 1943 H=1943:00 +CPU= 19.209
  01:31:41 STEP 1944 H=1944:00 +CPU= 33.438
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_80.zip
Uploading the intermediate file: upload_file_80.zip
  01:32:00 STEP 1945 H=1945:00 +CPU= 18.359
  01:32:19 STEP 1946 H=1946:00 +CPU= 18.699
  01:32:49 STEP 1947 H=1947:00 +CPU= 28.887
  01:33:08 STEP 1948 H=1948:00 +CPU= 18.492
  01:33:27 STEP 1949 H=1949:00 +CPU= 18.361
  01:33:57 STEP 1950 H=1950:00 +CPU= 29.464
  01:34:16 STEP 1951 H=1951:00 +CPU= 17.966
  01:34:35 STEP 1952 H=1952:00 +CPU= 18.544
  01:35:04 STEP 1953 H=1953:00 +CPU= 28.455
  01:35:23 STEP 1954 H=1954:00 +CPU= 19.006
  01:35:43 STEP 1955 H=1955:00 +CPU= 18.974
  01:36:16 STEP 1956 H=1956:00 +CPU= 31.797
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001956
  01:36:35 STEP 1957 H=1957:00 +CPU= 18.331
  01:36:54 STEP 1958 H=1958:00 +CPU= 18.833
  01:37:24 STEP 1959 H=1959:00 +CPU= 28.293
  01:37:42 STEP 1960 H=1960:00 +CPU= 18.313
  01:38:01 STEP 1961 H=1961:00 +CPU= 18.307
  01:38:32 STEP 1962 H=1962:00 +CPU= 29.575
  01:38:50 STEP 1963 H=1963:00 +CPU= 17.987
  01:39:09 STEP 1964 H=1964:00 +CPU= 18.764
  01:39:38 STEP 1965 H=1965:00 +CPU= 28.146
  01:39:57 STEP 1966 H=1966:00 +CPU= 18.682
  01:40:17 STEP 1967 H=1967:00 +CPU= 18.691
  01:40:50 STEP 1968 H=1968:00 +CPU= 33.107
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_81.zip
Uploading the intermediate file: upload_file_81.zip
  01:41:10 STEP 1969 H=1969:00 +CPU= 18.696
Uploading trickle at timestep: 7084800
  01:41:29 STEP 1970 H=1970:00 +CPU= 19.189
  01:41:58 STEP 1971 H=1971:00 +CPU= 28.702
  01:42:17 STEP 1972 H=1972:00 +CPU= 18.914
  01:42:36 STEP 1973 H=1973:00 +CPU= 18.524
  01:43:05 STEP 1974 H=1974:00 +CPU= 28.926
  01:43:24 STEP 1975 H=1975:00 +CPU= 18.542
  01:43:43 STEP 1976 H=1976:00 +CPU= 19.517
  01:44:12 STEP 1977 H=1977:00 +CPU= 28.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:44:40 STEP 1978 H=1978:00 +CPU= 17.741
  01:45:15 STEP 1979 H=1979:00 +CPU= 21.508
  01:46:20 STEP 1980 H=1980:00 +CPU= 37.176
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001980
  01:46:55 STEP 1981 H=1981:00 +CPU= 22.044
  01:47:21 STEP 1982 H=1982:00 +CPU= 21.449
  01:47:59 STEP 1983 H=1983:00 +CPU= 32.441
  01:48:24 STEP 1984 H=1984:00 +CPU= 21.322
  01:48:49 STEP 1985 H=1985:00 +CPU= 20.957
  01:49:28 STEP 1986 H=1986:00 +CPU= 32.851
  01:49:52 STEP 1987 H=1987:00 +CPU= 20.621
  01:50:17 STEP 1988 H=1988:00 +CPU= 21.398
  01:50:54 STEP 1989 H=1989:00 +CPU= 31.690
  01:51:20 STEP 1990 H=1990:00 +CPU= 21.624
  01:51:44 STEP 1991 H=1991:00 +CPU= 21.089
  01:52:28 STEP 1992 H=1992:00 +CPU= 37.510
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_82.zip
Uploading the intermediate file: upload_file_82.zip
  01:52:53 STEP 1993 H=1993:00 +CPU= 20.701
  01:53:19 STEP 1994 H=1994:00 +CPU= 21.513
  01:53:57 STEP 1995 H=1995:00 +CPU= 31.743
  01:54:23 STEP 1996 H=1996:00 +CPU= 21.592
  01:54:47 STEP 1997 H=1997:00 +CPU= 20.893
  01:55:26 STEP 1998 H=1998:00 +CPU= 32.891
  01:55:50 STEP 1999 H=1999:00 +CPU= 20.603
  01:56:15 STEP 2000 H=2000:00 +CPU= 21.469
  01:56:53 STEP 2001 H=2001:00 +CPU= 31.946
  01:57:18 STEP 2002 H=2002:00 +CPU= 21.685
  01:57:42 STEP 2003 H=2003:00 +CPU= 20.580
  01:58:24 STEP 2004 H=2004:00 +CPU= 35.502
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002004
  01:58:49 STEP 2005 H=2005:00 +CPU= 20.241
  01:59:15 STEP 2006 H=2006:00 +CPU= 21.626
  01:59:53 STEP 2007 H=2007:00 +CPU= 31.870
  02:00:19 STEP 2008 H=2008:00 +CPU= 21.534
  02:00:44 STEP 2009 H=2009:00 +CPU= 20.777
  02:01:23 STEP 2010 H=2010:00 +CPU= 32.742
  02:01:48 STEP 2011 H=2011:00 +CPU= 21.095
  02:02:13 STEP 2012 H=2012:00 +CPU= 21.221
  02:02:50 STEP 2013 H=2013:00 +CPU= 31.787
  02:03:15 STEP 2014 H=2014:00 +CPU= 21.311
  02:03:40 STEP 2015 H=2015:00 +CPU= 21.148
  02:04:24 STEP 2016 H=2016:00 +CPU= 37.417
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_83.zip
Uploading the intermediate file: upload_file_83.zip
  02:04:49 STEP 2017 H=2017:00 +CPU= 20.715
  02:05:14 STEP 2018 H=2018:00 +CPU= 21.283
  02:05:53 STEP 2019 H=2019:00 +CPU= 32.006
  02:06:18 STEP 2020 H=2020:00 +CPU= 21.228
  02:06:43 STEP 2021 H=2021:00 +CPU= 20.913
  02:07:21 STEP 2022 H=2022:00 +CPU= 32.293
  02:07:45 STEP 2023 H=2023:00 +CPU= 20.408
  02:08:11 STEP 2024 H=2024:00 +CPU= 21.680
  02:08:49 STEP 2025 H=2025:00 +CPU= 32.495
  02:09:15 STEP 2026 H=2026:00 +CPU= 22.201
  02:09:40 STEP 2027 H=2027:00 +CPU= 21.104
  02:10:24 STEP 2028 H=2028:00 +CPU= 37.499
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002028
  02:10:49 STEP 2029 H=2029:00 +CPU= 21.025
  02:11:15 STEP 2030 H=2030:00 +CPU= 22.416
  02:11:54 STEP 2031 H=2031:00 +CPU= 33.584
  02:12:21 STEP 2032 H=2032:00 +CPU= 22.410
  02:12:46 STEP 2033 H=2033:00 +CPU= 21.531
  02:13:27 STEP 2034 H=2034:00 +CPU= 35.120
  02:13:53 STEP 2035 H=2035:00 +CPU= 21.799
  02:14:19 STEP 2036 H=2036:00 +CPU= 22.280
  02:14:58 STEP 2037 H=2037:00 +CPU= 33.403
  02:15:24 STEP 2038 H=2038:00 +CPU= 22.468
  02:15:50 STEP 2039 H=2039:00 +CPU= 22.272
  02:16:36 STEP 2040 H=2040:00 +CPU= 39.107
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_84.zip
Uploading the intermediate file: upload_file_84.zip
  02:17:01 STEP 2041 H=2041:00 +CPU= 21.433
  02:17:28 STEP 2042 H=2042:00 +CPU= 22.511
  02:18:06 STEP 2043 H=2043:00 +CPU= 33.043
  02:18:33 STEP 2044 H=2044:00 +CPU= 22.669
  02:18:58 STEP 2045 H=2045:00 +CPU= 21.624
  02:19:39 STEP 2046 H=2046:00 +CPU= 34.793
  02:20:06 STEP 2047 H=2047:00 +CPU= 22.439
  02:20:33 STEP 2048 H=2048:00 +CPU= 22.743
  02:21:13 STEP 2049 H=2049:00 +CPU= 34.305
  02:21:39 STEP 2050 H=2050:00 +CPU= 22.334
  02:22:05 STEP 2051 H=2051:00 +CPU= 22.206
  02:22:49 STEP 2052 H=2052:00 +CPU= 37.483
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002052
  02:23:15 STEP 2053 H=2053:00 +CPU= 22.159
  02:23:41 STEP 2054 H=2054:00 +CPU= 21.966
  02:24:21 STEP 2055 H=2055:00 +CPU= 33.868
  02:24:47 STEP 2056 H=2056:00 +CPU= 22.461
  02:25:13 STEP 2057 H=2057:00 +CPU= 21.876
  02:25:54 STEP 2058 H=2058:00 +CPU= 34.871
  02:26:19 STEP 2059 H=2059:00 +CPU= 21.902
  02:26:46 STEP 2060 H=2060:00 +CPU= 22.457
  02:27:25 STEP 2061 H=2061:00 +CPU= 33.758
  02:27:52 STEP 2062 H=2062:00 +CPU= 22.624
  02:28:18 STEP 2063 H=2063:00 +CPU= 22.279
  02:29:04 STEP 2064 H=2064:00 +CPU= 39.006
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_85.zip
Uploading the intermediate file: upload_file_85.zip
  02:29:30 STEP 2065 H=2065:00 +CPU= 21.663
  02:29:57 STEP 2066 H=2066:00 +CPU= 22.775
  02:30:36 STEP 2067 H=2067:00 +CPU= 33.726
  02:31:02 STEP 2068 H=2068:00 +CPU= 22.323
  02:31:28 STEP 2069 H=2069:00 +CPU= 22.420
  02:32:09 STEP 2070 H=2070:00 +CPU= 34.851
  02:32:34 STEP 2071 H=2071:00 +CPU= 21.730
  02:33:01 STEP 2072 H=2072:00 +CPU= 22.623
  02:33:41 STEP 2073 H=2073:00 +CPU= 34.006
  02:34:09 STEP 2074 H=2074:00 +CPU= 23.011
  02:34:35 STEP 2075 H=2075:00 +CPU= 22.229
  02:35:19 STEP 2076 H=2076:00 +CPU= 37.653
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002076
  02:35:45 STEP 2077 H=2077:00 +CPU= 22.136
  02:36:13 STEP 2078 H=2078:00 +CPU= 22.609
  02:36:52 STEP 2079 H=2079:00 +CPU= 33.573
  02:37:18 STEP 2080 H=2080:00 +CPU= 22.559
  02:37:44 STEP 2081 H=2081:00 +CPU= 21.743
  02:38:25 STEP 2082 H=2082:00 +CPU= 35.044
  02:38:51 STEP 2083 H=2083:00 +CPU= 21.896
  02:39:18 STEP 2084 H=2084:00 +CPU= 22.840
  02:39:58 STEP 2085 H=2085:00 +CPU= 34.488
  02:40:25 STEP 2086 H=2086:00 +CPU= 22.661
  02:40:53 STEP 2087 H=2087:00 +CPU= 23.098
  02:41:40 STEP 2088 H=2088:00 +CPU= 39.814
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_86.zip
Uploading the intermediate file: upload_file_86.zip
  02:42:06 STEP 2089 H=2089:00 +CPU= 22.499
  02:42:31 STEP 2090 H=2090:00 +CPU= 21.842
  02:43:11 STEP 2091 H=2091:00 +CPU= 33.695
  02:43:37 STEP 2092 H=2092:00 +CPU= 22.565
  02:44:03 STEP 2093 H=2093:00 +CPU= 22.034
  02:44:43 STEP 2094 H=2094:00 +CPU= 34.418
  02:45:10 STEP 2095 H=2095:00 +CPU= 22.457
  02:45:36 STEP 2096 H=2096:00 +CPU= 22.079
  02:46:15 STEP 2097 H=2097:00 +CPU= 33.521
  02:46:42 STEP 2098 H=2098:00 +CPU= 22.815
  02:47:08 STEP 2099 H=2099:00 +CPU= 22.152
  02:47:52 STEP 2100 H=2100:00 +CPU= 37.615
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002100
  02:48:17 STEP 2101 H=2101:00 +CPU= 21.810
  02:48:44 STEP 2102 H=2102:00 +CPU= 22.736
  02:49:23 STEP 2103 H=2103:00 +CPU= 33.808
  02:49:50 STEP 2104 H=2104:00 +CPU= 22.686
  02:50:15 STEP 2105 H=2105:00 +CPU= 22.023
  02:50:56 STEP 2106 H=2106:00 +CPU= 34.321
  02:51:22 STEP 2107 H=2107:00 +CPU= 22.143
  02:51:48 STEP 2108 H=2108:00 +CPU= 22.570
  02:52:28 STEP 2109 H=2109:00 +CPU= 34.145
  02:52:54 STEP 2110 H=2110:00 +CPU= 22.662
  02:53:21 STEP 2111 H=2111:00 +CPU= 22.893
  02:54:08 STEP 2112 H=2112:00 +CPU= 39.387
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_87.zip
Uploading the intermediate file: upload_file_87.zip
  02:54:33 STEP 2113 H=2113:00 +CPU= 22.106
  02:55:01 STEP 2114 H=2114:00 +CPU= 22.998
  02:55:40 STEP 2115 H=2115:00 +CPU= 33.631
  02:56:07 STEP 2116 H=2116:00 +CPU= 22.844
  02:56:33 STEP 2117 H=2117:00 +CPU= 21.773
  02:57:15 STEP 2118 H=2118:00 +CPU= 34.796
  02:57:40 STEP 2119 H=2119:00 +CPU= 21.888
  02:58:07 STEP 2120 H=2120:00 +CPU= 22.881
  02:58:48 STEP 2121 H=2121:00 +CPU= 33.851
  02:59:14 STEP 2122 H=2122:00 +CPU= 22.508
  02:59:41 STEP 2123 H=2123:00 +CPU= 22.372
  03:00:26 STEP 2124 H=2124:00 +CPU= 37.955
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002124
  03:00:52 STEP 2125 H=2125:00 +CPU= 22.519
  03:01:19 STEP 2126 H=2126:00 +CPU= 22.653
  03:01:59 STEP 2127 H=2127:00 +CPU= 34.104
  03:02:25 STEP 2128 H=2128:00 +CPU= 22.524
  03:02:51 STEP 2129 H=2129:00 +CPU= 22.065
  03:03:32 STEP 2130 H=2130:00 +CPU= 34.692
  03:03:58 STEP 2131 H=2131:00 +CPU= 21.824
  03:04:24 STEP 2132 H=2132:00 +CPU= 22.294
  03:05:04 STEP 2133 H=2133:00 +CPU= 33.883
  03:05:31 STEP 2134 H=2134:00 +CPU= 22.901
  03:05:58 STEP 2135 H=2135:00 +CPU= 22.590
  03:06:44 STEP 2136 H=2136:00 +CPU= 39.358
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_88.zip
Uploading the intermediate file: upload_file_88.zip
  03:07:09 STEP 2137 H=2137:00 +CPU= 21.897
  03:07:36 STEP 2138 H=2138:00 +CPU= 22.480
  03:08:16 STEP 2139 H=2139:00 +CPU= 33.822
  03:08:44 STEP 2140 H=2140:00 +CPU= 23.008
  03:09:13 STEP 2141 H=2141:00 +CPU= 23.055
  03:09:57 STEP 2142 H=2142:00 +CPU= 35.489
  03:10:24 STEP 2143 H=2143:00 +CPU= 22.291
  03:10:53 STEP 2144 H=2144:00 +CPU= 23.212
  03:11:37 STEP 2145 H=2145:00 +CPU= 35.176
  03:12:04 STEP 2146 H=2146:00 +CPU= 23.097
  03:12:32 STEP 2147 H=2147:00 +CPU= 22.182
  03:13:19 STEP 2148 H=2148:00 +CPU= 38.762
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002148
  03:13:47 STEP 2149 H=2149:00 +CPU= 22.940
  03:14:19 STEP 2150 H=2150:00 +CPU= 24.694
  03:15:00 STEP 2151 H=2151:00 +CPU= 34.497
  03:15:27 STEP 2152 H=2152:00 +CPU= 23.080
  03:15:54 STEP 2153 H=2153:00 +CPU= 22.255
  03:16:35 STEP 2154 H=2154:00 +CPU= 34.983
  03:17:01 STEP 2155 H=2155:00 +CPU= 22.253
  03:17:28 STEP 2156 H=2156:00 +CPU= 22.512
  03:18:08 STEP 2157 H=2157:00 +CPU= 34.074
  03:18:34 STEP 2158 H=2158:00 +CPU= 22.771
  03:19:03 STEP 2159 H=2159:00 +CPU= 23.459
  03:19:49 STEP 2160 H=2160:00 +CPU= 39.473
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_89.zip
Uploading the intermediate file: upload_file_89.zip
  03:20:15 STEP 2161 H=2161:00 +CPU= 22.279
  03:20:42 STEP 2162 H=2162:00 +CPU= 22.908
  03:21:23 STEP 2163 H=2163:00 +CPU= 34.290
  03:21:49 STEP 2164 H=2164:00 +CPU= 22.548
  03:22:15 STEP 2165 H=2165:00 +CPU= 22.608
  03:22:56 STEP 2166 H=2166:00 +CPU= 34.759
  03:23:23 STEP 2167 H=2167:00 +CPU= 22.194
  03:23:50 STEP 2168 H=2168:00 +CPU= 23.320
  03:24:30 STEP 2169 H=2169:00 +CPU= 34.291
  03:24:58 STEP 2170 H=2170:00 +CPU= 23.266
  03:25:23 STEP 2171 H=2171:00 +CPU= 22.087
  03:26:08 STEP 2172 H=2172:00 +CPU= 37.821
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002172
  03:26:34 STEP 2173 H=2173:00 +CPU= 22.358
  03:27:01 STEP 2174 H=2174:00 +CPU= 23.155
  03:27:42 STEP 2175 H=2175:00 +CPU= 34.355
  03:28:09 STEP 2176 H=2176:00 +CPU= 23.012
  03:28:34 STEP 2177 H=2177:00 +CPU= 22.063
  03:29:14 STEP 2178 H=2178:00 +CPU= 34.346
  03:29:41 STEP 2179 H=2179:00 +CPU= 22.377
  03:30:07 STEP 2180 H=2180:00 +CPU= 22.608
  03:30:47 STEP 2181 H=2181:00 +CPU= 34.121
  03:31:14 STEP 2182 H=2182:00 +CPU= 23.115
  03:31:41 STEP 2183 H=2183:00 +CPU= 23.055
  03:32:27 STEP 2184 H=2184:00 +CPU= 39.385
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_90.zip
Uploading the intermediate file: upload_file_90.zip
  03:32:53 STEP 2185 H=2185:00 +CPU= 22.228
  03:33:20 STEP 2186 H=2186:00 +CPU= 23.210
  03:34:01 STEP 2187 H=2187:00 +CPU= 34.018
  03:34:28 STEP 2188 H=2188:00 +CPU= 23.119
  03:34:55 STEP 2189 H=2189:00 +CPU= 22.094
  03:35:36 STEP 2190 H=2190:00 +CPU= 35.313
  03:36:02 STEP 2191 H=2191:00 +CPU= 22.047
  03:36:29 STEP 2192 H=2192:00 +CPU= 23.019
  03:37:09 STEP 2193 H=2193:00 +CPU= 33.786
  03:37:36 STEP 2194 H=2194:00 +CPU= 22.763
  03:38:02 STEP 2195 H=2195:00 +CPU= 22.314
  03:38:47 STEP 2196 H=2196:00 +CPU= 37.832
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002196
  03:39:15 STEP 2197 H=2197:00 +CPU= 22.487
  03:39:42 STEP 2198 H=2198:00 +CPU= 22.705
  03:40:22 STEP 2199 H=2199:00 +CPU= 34.375
  03:40:50 STEP 2200 H=2200:00 +CPU= 22.967
  03:41:16 STEP 2201 H=2201:00 +CPU= 22.385
  03:41:57 STEP 2202 H=2202:00 +CPU= 34.661
  03:42:23 STEP 2203 H=2203:00 +CPU= 21.736
  03:42:51 STEP 2204 H=2204:00 +CPU= 23.097
  03:43:32 STEP 2205 H=2205:00 +CPU= 33.889
  03:43:59 STEP 2206 H=2206:00 +CPU= 22.855
  03:44:25 STEP 2207 H=2207:00 +CPU= 22.905
  03:45:13 STEP 2208 H=2208:00 +CPU= 40.074
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_91.zip
Uploading the intermediate file: upload_file_91.zip
  03:45:40 STEP 2209 H=2209:00 +CPU= 22.642
Uploading trickle at timestep: 7948800
  03:46:07 STEP 2210 H=2210:00 +CPU= 23.168
  03:46:46 STEP 2211 H=2211:00 +CPU= 33.603
  03:47:13 STEP 2212 H=2212:00 +CPU= 22.996
  03:47:39 STEP 2213 H=2213:00 +CPU= 21.960
  03:48:20 STEP 2214 H=2214:00 +CPU= 34.979
  03:48:46 STEP 2215 H=2215:00 +CPU= 22.488
  03:49:14 STEP 2216 H=2216:00 +CPU= 23.180
  03:49:53 STEP 2217 H=2217:00 +CPU= 33.802
  03:50:20 STEP 2218 H=2218:00 +CPU= 22.228
  03:50:47 STEP 2219 H=2219:00 +CPU= 22.094
  03:51:31 STEP 2220 H=2220:00 +CPU= 37.414
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002220
  03:51:57 STEP 2221 H=2221:00 +CPU= 22.894
  03:52:24 STEP 2222 H=2222:00 +CPU= 22.815
  03:53:04 STEP 2223 H=2223:00 +CPU= 34.103
  03:53:30 STEP 2224 H=2224:00 +CPU= 22.498
  03:53:56 STEP 2225 H=2225:00 +CPU= 21.733
  03:54:37 STEP 2226 H=2226:00 +CPU= 34.790
  03:55:03 STEP 2227 H=2227:00 +CPU= 21.919
  03:55:31 STEP 2228 H=2228:00 +CPU= 22.984
  03:56:11 STEP 2229 H=2229:00 +CPU= 33.942
  03:56:37 STEP 2230 H=2230:00 +CPU= 22.597
  03:57:04 STEP 2231 H=2231:00 +CPU= 22.495
  03:57:50 STEP 2232 H=2232:00 +CPU= 39.249
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_92.zip
Uploading the intermediate file: upload_file_92.zip
  03:58:16 STEP 2233 H=2233:00 +CPU= 22.500
  03:58:43 STEP 2234 H=2234:00 +CPU= 22.919
  03:59:23 STEP 2235 H=2235:00 +CPU= 33.641
  03:59:49 STEP 2236 H=2236:00 +CPU= 22.746
  04:00:16 STEP 2237 H=2237:00 +CPU= 22.189
  04:00:58 STEP 2238 H=2238:00 +CPU= 35.697
  04:01:24 STEP 2239 H=2239:00 +CPU= 22.100
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.179] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.192] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.193] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.193] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.193] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.193] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.193] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.195] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040136:1675306896:21195.196] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               POSTPHY_LAYER 
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040137:1675306897:21196.196] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2907:2907] [20230202:040137:1675306897:21196.196] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000EAB6D4  postphy_layer_            215  postphy_layer.F90
oifs_43r3_model.e  0000000000E5022E  ec_phys_                  945  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2020050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0661
(argv4) batchid: 989
(argv5) wuid: 12206305
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c6e07ff7217e52c9d4728898e3c490a3 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0661_2020050100_123_989_12206305.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0661_2020050100_123_989_12206305.zip
ic_ancil_file: ic_ancil_12206305
ifsdata_file: ifsdata_12206305
climate_data_file: clim_data_12206305
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a92720b805546eb4aa78c7294ffacf9f to: /var/lib/boinc-client/slots/2/ic_ancil_12206305.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12206305.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_713f25b176b0993d734cef21e70ae3a4 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12206305.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12206305.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12206305.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12206305.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12206305.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12206305.xml
-- Model is restarting --
Adjusting last_iter, 2240, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12206305.xml
last_cpu_time: 50558
upload_file_number: 93
last_iter: 2231
last_upload: 8035200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180
The child process has been launched with process id: 2897
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2897:2897] [20230202:215803:1675371483:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:58:44 STEP 2232 H=2232:00 +CPU= 35.266
  21:59:05 STEP 2233 H=2233:00 +CPU= 19.728
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:00:04 STEP 2234 H=2234:00 +CPU= 21.925
  22:00:34 STEP 2235 H=2235:00 +CPU= 29.632
  22:00:54 STEP 2236 H=2236:00 +CPU= 19.623
  22:01:16 STEP 2237 H=2237:00 +CPU= 20.562
  22:01:51 STEP 2238 H=2238:00 +CPU= 31.792
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:02:36 STEP 2239 H=2239:00 +CPU= 21.757
  22:02:59 STEP 2240 H=2240:00 +CPU= 21.829
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:03:44 STEP 2241 H=2241:00 +CPU= 32.768
  22:04:05 STEP 2242 H=2242:00 +CPU= 20.346
  22:04:23 STEP 2243 H=2243:00 +CPU= 17.879
  22:04:58 STEP 2244 H=2244:00 +CPU= 33.545
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002244
  22:05:18 STEP 2245 H=2245:00 +CPU= 20.000
  22:05:39 STEP 2246 H=2246:00 +CPU= 19.772
  22:06:11 STEP 2247 H=2247:00 +CPU= 31.467
  22:06:35 STEP 2248 H=2248:00 +CPU= 23.118
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:07:08 STEP 2249 H=2249:00 +CPU= 21.760
  22:07:42 STEP 2250 H=2250:00 +CPU= 33.506
  22:08:02 STEP 2251 H=2251:00 +CPU= 19.516
  22:08:23 STEP 2252 H=2252:00 +CPU= 20.580
  22:08:53 STEP 2253 H=2253:00 +CPU= 29.655
  22:09:13 STEP 2254 H=2254:00 +CPU= 19.790
  22:09:33 STEP 2255 H=2255:00 +CPU= 19.416
  22:10:10 STEP 2256 H=2256:00 +CPU= 36.766
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206305/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_2020050100_123_989_12206305_1_r2137801180_93.zip
Uploading the intermediate file: upload_file_93.zip
  22:10:29 STEP 2257 H=2257:00 +CPU= 18.677
  22:10:49 STEP 2258 H=2258:00 +CPU= 20.034
  22:11:19 STEP 2259 H=2259:00 +CPU= 29.945
  22:11:40 STEP 2260 H=2260:00 +CPU= 20.421
  22:11:58 STEP 2261 H=2261:00 +CPU= 18.239
  22:12:29 STEP 2262 H=2262:00 +CPU= 30.192
  22:12:48 STEP 2263 H=2263:00 +CPU= 18.692
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:46:46 1527792 22303116 oifs_43r3_ps_0661_2020050100_123_989_12206305_1 7,948,800 49,730 0.0063
09 May 2023 16:46:45 1527792 22303116 oifs_43r3_ps_0661_2020050100_123_989_12206305_1 7,084,800 43,420 0.0061
09 May 2023 16:46:42 1527792 22303116 oifs_43r3_ps_0661_2020050100_123_989_12206305_1 6,220,800 37,858 0.0061
09 May 2023 16:46:15 1527792 22303116 oifs_43r3_ps_0661_2020050100_123_989_12206305_1 4,665,600 27,554 0.0059
09 May 2023 16:46:22 1527792 22303116 oifs_43r3_ps_0661_2020050100_123_989_12206305_1 4,665,600 27,554 0.0059
09 May 2023 16:43:40 1527792 22303116 oifs_43r3_ps_0661_2020050100_123_989_12206305_1 2,851,200 14,883 0.0052
09 May 2023 16:41:17 1527792 22303116 oifs_43r3_ps_0661_2020050100_123_989_12206305_1 1,209,600 6,611 0.0055


©2024 cpdn.org