climateprediction.net home page
Task 22257042

Task 22257042

Name oifs_43r3_ps_0907_1986050100_123_955_12172551_0
Workunit 12172551
Created 21 Dec 2022, 9:51:26 UTC
Sent 12 Jan 2023, 16:04:54 UTC
Report deadline 11 Feb 2023, 16:04:54 UTC
Received 14 Jan 2023, 10:53:56 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1532184
Run time 12 hours 51 min 25 sec
CPU time 12 hours 49 min 18 sec
Validate state Invalid
Credit 11,128.32
Device peak FLOPS 6.86 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,417.64 MB
Peak swap size 4,978.08 MB
Peak disk usage 3,242.57 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
inc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_77.zip
Uploading the intermediate file: upload_file_77.zip
  00:33:34 STEP 1873 H=1873:00 +CPU= 15.211
  00:33:50 STEP 1874 H=1874:00 +CPU= 15.685
  00:34:14 STEP 1875 H=1875:00 +CPU= 24.459
  00:34:30 STEP 1876 H=1876:00 +CPU= 15.738
  00:34:45 STEP 1877 H=1877:00 +CPU= 15.179
  00:35:10 STEP 1878 H=1878:00 +CPU= 24.491
  00:35:25 STEP 1879 H=1879:00 +CPU= 15.333
  00:35:41 STEP 1880 H=1880:00 +CPU= 15.750
  00:36:05 STEP 1881 H=1881:00 +CPU= 24.189
  00:36:21 STEP 1882 H=1882:00 +CPU= 15.569
  00:36:36 STEP 1883 H=1883:00 +CPU= 15.152
  00:37:02 STEP 1884 H=1884:00 +CPU= 26.580
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001884
  00:37:18 STEP 1885 H=1885:00 +CPU= 15.164
  00:37:33 STEP 1886 H=1886:00 +CPU= 15.617
  00:37:57 STEP 1887 H=1887:00 +CPU= 24.118
  00:38:13 STEP 1888 H=1888:00 +CPU= 15.664
  00:38:28 STEP 1889 H=1889:00 +CPU= 15.079
  00:38:53 STEP 1890 H=1890:00 +CPU= 24.528
  00:39:08 STEP 1891 H=1891:00 +CPU= 15.342
  00:39:24 STEP 1892 H=1892:00 +CPU= 15.516
  00:39:48 STEP 1893 H=1893:00 +CPU= 24.162
  00:40:03 STEP 1894 H=1894:00 +CPU= 15.642
  00:40:19 STEP 1895 H=1895:00 +CPU= 15.518
  00:40:47 STEP 1896 H=1896:00 +CPU= 28.539
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_78.zip
Uploading the intermediate file: upload_file_78.zip
  00:41:03 STEP 1897 H=1897:00 +CPU= 15.136
  00:41:18 STEP 1898 H=1898:00 +CPU= 15.423
  00:41:42 STEP 1899 H=1899:00 +CPU= 24.057
  00:41:58 STEP 1900 H=1900:00 +CPU= 15.547
  00:42:13 STEP 1901 H=1901:00 +CPU= 15.326
  00:42:37 STEP 1902 H=1902:00 +CPU= 24.540
  00:42:53 STEP 1903 H=1903:00 +CPU= 15.147
  00:43:08 STEP 1904 H=1904:00 +CPU= 15.655
  00:43:32 STEP 1905 H=1905:00 +CPU= 24.168
  00:43:48 STEP 1906 H=1906:00 +CPU= 15.664
  00:44:03 STEP 1907 H=1907:00 +CPU= 15.085
  00:44:30 STEP 1908 H=1908:00 +CPU= 26.288
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001908
  00:44:45 STEP 1909 H=1909:00 +CPU= 15.112
  00:45:00 STEP 1910 H=1910:00 +CPU= 15.587
  00:45:24 STEP 1911 H=1911:00 +CPU= 24.040
  00:45:40 STEP 1912 H=1912:00 +CPU= 15.690
  00:45:55 STEP 1913 H=1913:00 +CPU= 15.393
  00:46:20 STEP 1914 H=1914:00 +CPU= 24.787
  00:46:36 STEP 1915 H=1915:00 +CPU= 15.343
  00:46:51 STEP 1916 H=1916:00 +CPU= 15.661
  00:47:15 STEP 1917 H=1917:00 +CPU= 24.182
  00:47:31 STEP 1918 H=1918:00 +CPU= 15.785
  00:47:47 STEP 1919 H=1919:00 +CPU= 15.666
  00:48:15 STEP 1920 H=1920:00 +CPU= 28.357
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_79.zip
Uploading the intermediate file: upload_file_79.zip
  00:48:30 STEP 1921 H=1921:00 +CPU= 15.315
  00:48:46 STEP 1922 H=1922:00 +CPU= 15.594
  00:49:10 STEP 1923 H=1923:00 +CPU= 24.078
  00:49:26 STEP 1924 H=1924:00 +CPU= 15.701
  00:49:41 STEP 1925 H=1925:00 +CPU= 15.189
  00:50:05 STEP 1926 H=1926:00 +CPU= 24.419
  00:50:21 STEP 1927 H=1927:00 +CPU= 15.338
  00:50:37 STEP 1928 H=1928:00 +CPU= 15.808
  00:51:01 STEP 1929 H=1929:00 +CPU= 24.096
  00:51:16 STEP 1930 H=1930:00 +CPU= 15.682
  00:51:32 STEP 1931 H=1931:00 +CPU= 15.218
  00:51:58 STEP 1932 H=1932:00 +CPU= 26.491
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001932
  00:52:13 STEP 1933 H=1933:00 +CPU= 15.262
  00:52:29 STEP 1934 H=1934:00 +CPU= 15.584
  00:52:53 STEP 1935 H=1935:00 +CPU= 24.032
  00:53:09 STEP 1936 H=1936:00 +CPU= 15.721
  00:53:24 STEP 1937 H=1937:00 +CPU= 15.132
  00:53:48 STEP 1938 H=1938:00 +CPU= 24.466
  00:54:03 STEP 1939 H=1939:00 +CPU= 15.161
  00:54:19 STEP 1940 H=1940:00 +CPU= 15.788
  00:54:43 STEP 1941 H=1941:00 +CPU= 24.167
  00:54:59 STEP 1942 H=1942:00 +CPU= 15.715
  00:55:15 STEP 1943 H=1943:00 +CPU= 15.563
  00:55:43 STEP 1944 H=1944:00 +CPU= 28.392
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_80.zip
Uploading the intermediate file: upload_file_80.zip
  00:55:58 STEP 1945 H=1945:00 +CPU= 15.302
  00:56:14 STEP 1946 H=1946:00 +CPU= 15.697
  00:56:38 STEP 1947 H=1947:00 +CPU= 24.182
  00:56:54 STEP 1948 H=1948:00 +CPU= 15.760
  00:57:09 STEP 1949 H=1949:00 +CPU= 15.221
  00:57:34 STEP 1950 H=1950:00 +CPU= 24.741
  00:57:50 STEP 1951 H=1951:00 +CPU= 15.391
  00:58:05 STEP 1952 H=1952:00 +CPU= 15.602
  00:58:29 STEP 1953 H=1953:00 +CPU= 24.177
  00:58:45 STEP 1954 H=1954:00 +CPU= 15.600
  00:59:00 STEP 1955 H=1955:00 +CPU= 15.064
  00:59:26 STEP 1956 H=1956:00 +CPU= 26.447
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001956
  00:59:42 STEP 1957 H=1957:00 +CPU= 15.242
  00:59:57 STEP 1958 H=1958:00 +CPU= 15.644
  01:00:22 STEP 1959 H=1959:00 +CPU= 24.192
  01:00:37 STEP 1960 H=1960:00 +CPU= 15.708
  01:00:53 STEP 1961 H=1961:00 +CPU= 15.323
  01:01:17 STEP 1962 H=1962:00 +CPU= 24.730
  01:01:32 STEP 1963 H=1963:00 +CPU= 15.185
  01:01:48 STEP 1964 H=1964:00 +CPU= 15.690
  01:02:12 STEP 1965 H=1965:00 +CPU= 24.131
  01:02:28 STEP 1966 H=1966:00 +CPU= 15.671
  01:02:44 STEP 1967 H=1967:00 +CPU= 15.768
  01:03:12 STEP 1968 H=1968:00 +CPU= 28.244
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_81.zip
Uploading the intermediate file: upload_file_81.zip
  01:03:28 STEP 1969 H=1969:00 +CPU= 15.578
Uploading trickle at timestep: 7084800
  01:03:43 STEP 1970 H=1970:00 +CPU= 15.749
  01:04:08 STEP 1971 H=1971:00 +CPU= 24.233
  01:04:23 STEP 1972 H=1972:00 +CPU= 15.706
  01:04:39 STEP 1973 H=1973:00 +CPU= 15.269
  01:05:03 STEP 1974 H=1974:00 +CPU= 24.667
  01:05:18 STEP 1975 H=1975:00 +CPU= 15.290
  01:05:34 STEP 1976 H=1976:00 +CPU= 15.656
  01:05:58 STEP 1977 H=1977:00 +CPU= 24.238
  01:06:14 STEP 1978 H=1978:00 +CPU= 15.695
  01:06:29 STEP 1979 H=1979:00 +CPU= 15.286
  01:06:56 STEP 1980 H=1980:00 +CPU= 26.565
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001980
  01:07:11 STEP 1981 H=1981:00 +CPU= 15.294
  01:07:27 STEP 1982 H=1982:00 +CPU= 15.647
  01:07:51 STEP 1983 H=1983:00 +CPU= 23.997
  01:08:07 STEP 1984 H=1984:00 +CPU= 15.648
  01:08:22 STEP 1985 H=1985:00 +CPU= 15.272
  01:08:47 STEP 1986 H=1986:00 +CPU= 24.740
  01:09:02 STEP 1987 H=1987:00 +CPU= 15.401
  01:09:18 STEP 1988 H=1988:00 +CPU= 15.681
  01:09:42 STEP 1989 H=1989:00 +CPU= 24.118
  01:09:57 STEP 1990 H=1990:00 +CPU= 15.621
  01:10:13 STEP 1991 H=1991:00 +CPU= 15.786
  01:10:41 STEP 1992 H=1992:00 +CPU= 28.040
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_82.zip
Uploading the intermediate file: upload_file_82.zip
  01:10:56 STEP 1993 H=1993:00 +CPU= 15.249
  01:11:12 STEP 1994 H=1994:00 +CPU= 15.669
  01:11:36 STEP 1995 H=1995:00 +CPU= 24.318
  01:11:52 STEP 1996 H=1996:00 +CPU= 15.650
  01:12:07 STEP 1997 H=1997:00 +CPU= 15.150
  01:12:32 STEP 1998 H=1998:00 +CPU= 24.749
  01:12:47 STEP 1999 H=1999:00 +CPU= 15.226
  01:13:03 STEP 2000 H=2000:00 +CPU= 15.593
  01:13:27 STEP 2001 H=2001:00 +CPU= 24.086
  01:13:43 STEP 2002 H=2002:00 +CPU= 15.577
  01:13:58 STEP 2003 H=2003:00 +CPU= 15.246
  01:14:24 STEP 2004 H=2004:00 +CPU= 26.658
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002004
  01:14:40 STEP 2005 H=2005:00 +CPU= 15.420
  01:14:56 STEP 2006 H=2006:00 +CPU= 15.687
  01:15:20 STEP 2007 H=2007:00 +CPU= 24.070
  01:15:35 STEP 2008 H=2008:00 +CPU= 15.628
  01:15:50 STEP 2009 H=2009:00 +CPU= 15.263
  01:16:15 STEP 2010 H=2010:00 +CPU= 24.511
  01:16:30 STEP 2011 H=2011:00 +CPU= 15.244
  01:16:46 STEP 2012 H=2012:00 +CPU= 15.636
  01:17:10 STEP 2013 H=2013:00 +CPU= 24.103
  01:17:26 STEP 2014 H=2014:00 +CPU= 15.588
  01:17:41 STEP 2015 H=2015:00 +CPU= 15.679
  01:18:10 STEP 2016 H=2016:00 +CPU= 28.281
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_83.zip
Uploading the intermediate file: upload_file_83.zip
  01:18:25 STEP 2017 H=2017:00 +CPU= 15.220
  01:18:40 STEP 2018 H=2018:00 +CPU= 15.691
  01:19:05 STEP 2019 H=2019:00 +CPU= 24.091
  01:19:20 STEP 2020 H=2020:00 +CPU= 15.669
  01:19:35 STEP 2021 H=2021:00 +CPU= 15.193
  01:20:00 STEP 2022 H=2022:00 +CPU= 24.765
  01:20:15 STEP 2023 H=2023:00 +CPU= 15.199
  01:20:31 STEP 2024 H=2024:00 +CPU= 15.650
  01:20:55 STEP 2025 H=2025:00 +CPU= 24.135
  01:21:11 STEP 2026 H=2026:00 +CPU= 15.671
  01:21:26 STEP 2027 H=2027:00 +CPU= 15.211
  01:21:53 STEP 2028 H=2028:00 +CPU= 26.630
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002028
  01:22:08 STEP 2029 H=2029:00 +CPU= 15.372
  01:22:24 STEP 2030 H=2030:00 +CPU= 15.699
  01:22:48 STEP 2031 H=2031:00 +CPU= 24.180
  01:23:04 STEP 2032 H=2032:00 +CPU= 15.818
  01:23:19 STEP 2033 H=2033:00 +CPU= 15.286
  01:23:44 STEP 2034 H=2034:00 +CPU= 24.653
  01:23:59 STEP 2035 H=2035:00 +CPU= 15.356
  01:24:15 STEP 2036 H=2036:00 +CPU= 15.695
  01:24:39 STEP 2037 H=2037:00 +CPU= 24.198
  01:24:55 STEP 2038 H=2038:00 +CPU= 15.741
  01:25:10 STEP 2039 H=2039:00 +CPU= 15.659
  01:25:39 STEP 2040 H=2040:00 +CPU= 28.340
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_84.zip
Uploading the intermediate file: upload_file_84.zip
  01:25:54 STEP 2041 H=2041:00 +CPU= 15.115
  01:26:10 STEP 2042 H=2042:00 +CPU= 15.641
  01:26:34 STEP 2043 H=2043:00 +CPU= 24.086
  01:26:49 STEP 2044 H=2044:00 +CPU= 15.645
  01:27:05 STEP 2045 H=2045:00 +CPU= 15.281
  01:27:29 STEP 2046 H=2046:00 +CPU= 24.523
  01:27:44 STEP 2047 H=2047:00 +CPU= 15.222
  01:28:00 STEP 2048 H=2048:00 +CPU= 15.528
  01:28:24 STEP 2049 H=2049:00 +CPU= 24.165
  01:28:40 STEP 2050 H=2050:00 +CPU= 15.739
  01:28:55 STEP 2051 H=2051:00 +CPU= 15.179
  01:29:21 STEP 2052 H=2052:00 +CPU= 26.397
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002052
  01:29:37 STEP 2053 H=2053:00 +CPU= 15.208
  01:29:52 STEP 2054 H=2054:00 +CPU= 15.549
  01:30:16 STEP 2055 H=2055:00 +CPU= 24.123
  01:30:32 STEP 2056 H=2056:00 +CPU= 15.706
  01:30:47 STEP 2057 H=2057:00 +CPU= 15.258
  01:31:12 STEP 2058 H=2058:00 +CPU= 24.573
  01:31:27 STEP 2059 H=2059:00 +CPU= 15.292
  01:31:43 STEP 2060 H=2060:00 +CPU= 15.603
  01:32:07 STEP 2061 H=2061:00 +CPU= 23.902
  01:32:22 STEP 2062 H=2062:00 +CPU= 15.715
  01:32:38 STEP 2063 H=2063:00 +CPU= 15.634
  01:33:06 STEP 2064 H=2064:00 +CPU= 28.279
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_85.zip
Uploading the intermediate file: upload_file_85.zip
  01:33:21 STEP 2065 H=2065:00 +CPU= 15.199
  01:33:37 STEP 2066 H=2066:00 +CPU= 15.685
  01:34:02 STEP 2067 H=2067:00 +CPU= 25.016
  01:34:18 STEP 2068 H=2068:00 +CPU= 15.696
  01:34:33 STEP 2069 H=2069:00 +CPU= 15.033
  01:34:57 STEP 2070 H=2070:00 +CPU= 24.454
  01:35:13 STEP 2071 H=2071:00 +CPU= 15.208
  01:35:28 STEP 2072 H=2072:00 +CPU= 15.612
  01:35:52 STEP 2073 H=2073:00 +CPU= 24.145
  01:36:08 STEP 2074 H=2074:00 +CPU= 15.530
  01:36:23 STEP 2075 H=2075:00 +CPU= 15.075
  01:36:49 STEP 2076 H=2076:00 +CPU= 26.516
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002076
  01:37:05 STEP 2077 H=2077:00 +CPU= 15.130
  01:37:20 STEP 2078 H=2078:00 +CPU= 15.689
  01:37:44 STEP 2079 H=2079:00 +CPU= 24.001
  01:38:00 STEP 2080 H=2080:00 +CPU= 15.592
  01:38:15 STEP 2081 H=2081:00 +CPU= 15.072
  01:38:39 STEP 2082 H=2082:00 +CPU= 24.280
  01:38:54 STEP 2083 H=2083:00 +CPU= 15.121
  01:39:10 STEP 2084 H=2084:00 +CPU= 15.612
  01:39:34 STEP 2085 H=2085:00 +CPU= 24.047
  01:39:50 STEP 2086 H=2086:00 +CPU= 15.517
  01:40:05 STEP 2087 H=2087:00 +CPU= 15.780
  01:40:34 STEP 2088 H=2088:00 +CPU= 28.389
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_86.zip
Uploading the intermediate file: upload_file_86.zip
  01:40:49 STEP 2089 H=2089:00 +CPU= 15.178
  01:41:05 STEP 2090 H=2090:00 +CPU= 15.996
  01:41:29 STEP 2091 H=2091:00 +CPU= 24.242
  01:41:45 STEP 2092 H=2092:00 +CPU= 15.569
  01:42:00 STEP 2093 H=2093:00 +CPU= 15.013
  01:42:24 STEP 2094 H=2094:00 +CPU= 24.598
  01:42:40 STEP 2095 H=2095:00 +CPU= 15.304
  01:42:55 STEP 2096 H=2096:00 +CPU= 15.656
  01:43:19 STEP 2097 H=2097:00 +CPU= 24.037
  01:43:35 STEP 2098 H=2098:00 +CPU= 15.684
  01:43:50 STEP 2099 H=2099:00 +CPU= 15.274
  01:44:17 STEP 2100 H=2100:00 +CPU= 26.663
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002100
  01:44:32 STEP 2101 H=2101:00 +CPU= 15.282
  01:44:48 STEP 2102 H=2102:00 +CPU= 15.582
  01:45:12 STEP 2103 H=2103:00 +CPU= 24.177
  01:45:28 STEP 2104 H=2104:00 +CPU= 15.737
  01:45:43 STEP 2105 H=2105:00 +CPU= 15.273
  01:46:08 STEP 2106 H=2106:00 +CPU= 24.506
  01:46:23 STEP 2107 H=2107:00 +CPU= 15.219
  01:46:39 STEP 2108 H=2108:00 +CPU= 15.725
  01:47:03 STEP 2109 H=2109:00 +CPU= 24.142
  01:47:18 STEP 2110 H=2110:00 +CPU= 15.679
  01:47:34 STEP 2111 H=2111:00 +CPU= 15.512
  01:48:02 STEP 2112 H=2112:00 +CPU= 28.251
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_87.zip
Uploading the intermediate file: upload_file_87.zip
  01:48:17 STEP 2113 H=2113:00 +CPU= 15.229
  01:48:33 STEP 2114 H=2114:00 +CPU= 15.782
  01:48:57 STEP 2115 H=2115:00 +CPU= 24.161
  01:49:13 STEP 2116 H=2116:00 +CPU= 15.744
  01:49:28 STEP 2117 H=2117:00 +CPU= 15.203
  01:49:53 STEP 2118 H=2118:00 +CPU= 24.665
  01:50:08 STEP 2119 H=2119:00 +CPU= 15.186
  01:50:24 STEP 2120 H=2120:00 +CPU= 15.556
  01:50:48 STEP 2121 H=2121:00 +CPU= 23.950
  01:51:03 STEP 2122 H=2122:00 +CPU= 15.578
  01:51:18 STEP 2123 H=2123:00 +CPU= 15.244
  01:51:45 STEP 2124 H=2124:00 +CPU= 26.450
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002124
  01:52:00 STEP 2125 H=2125:00 +CPU= 15.258
  01:52:16 STEP 2126 H=2126:00 +CPU= 15.589
  01:52:40 STEP 2127 H=2127:00 +CPU= 24.230
  01:52:56 STEP 2128 H=2128:00 +CPU= 15.713
  01:53:11 STEP 2129 H=2129:00 +CPU= 15.157
  01:53:35 STEP 2130 H=2130:00 +CPU= 24.572
  01:53:51 STEP 2131 H=2131:00 +CPU= 15.259
  01:54:06 STEP 2132 H=2132:00 +CPU= 15.512
  01:54:30 STEP 2133 H=2133:00 +CPU= 24.101
  01:54:46 STEP 2134 H=2134:00 +CPU= 15.604
  01:55:02 STEP 2135 H=2135:00 +CPU= 15.672
  01:55:30 STEP 2136 H=2136:00 +CPU= 28.439
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_88.zip
Uploading the intermediate file: upload_file_88.zip
  01:55:45 STEP 2137 H=2137:00 +CPU= 15.185
  01:56:01 STEP 2138 H=2138:00 +CPU= 15.615
  01:56:25 STEP 2139 H=2139:00 +CPU= 24.132
  01:56:41 STEP 2140 H=2140:00 +CPU= 15.706
  01:56:56 STEP 2141 H=2141:00 +CPU= 15.248
  01:57:21 STEP 2142 H=2142:00 +CPU= 24.667
  01:57:36 STEP 2143 H=2143:00 +CPU= 15.164
  01:57:51 STEP 2144 H=2144:00 +CPU= 15.569
  01:58:16 STEP 2145 H=2145:00 +CPU= 24.119
  01:58:31 STEP 2146 H=2146:00 +CPU= 15.642
  01:58:46 STEP 2147 H=2147:00 +CPU= 15.246
  01:59:13 STEP 2148 H=2148:00 +CPU= 26.567
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002148
  01:59:28 STEP 2149 H=2149:00 +CPU= 15.327
  01:59:44 STEP 2150 H=2150:00 +CPU= 15.678
  02:00:08 STEP 2151 H=2151:00 +CPU= 24.197
  02:00:24 STEP 2152 H=2152:00 +CPU= 15.516
  02:00:39 STEP 2153 H=2153:00 +CPU= 15.266
  02:01:04 STEP 2154 H=2154:00 +CPU= 24.598
  02:01:19 STEP 2155 H=2155:00 +CPU= 15.255
  02:01:34 STEP 2156 H=2156:00 +CPU= 15.638
  02:01:59 STEP 2157 H=2157:00 +CPU= 24.066
  02:02:14 STEP 2158 H=2158:00 +CPU= 15.565
  02:02:30 STEP 2159 H=2159:00 +CPU= 15.576
  02:02:59 STEP 2160 H=2160:00 +CPU= 28.966
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_89.zip
Uploading the intermediate file: upload_file_89.zip
  02:03:14 STEP 2161 H=2161:00 +CPU= 15.199
  02:03:29 STEP 2162 H=2162:00 +CPU= 15.550
  02:03:54 STEP 2163 H=2163:00 +CPU= 24.218
  02:04:09 STEP 2164 H=2164:00 +CPU= 15.589
  02:04:24 STEP 2165 H=2165:00 +CPU= 15.092
  02:04:49 STEP 2166 H=2166:00 +CPU= 24.562
  02:05:04 STEP 2167 H=2167:00 +CPU= 15.247
  02:05:20 STEP 2168 H=2168:00 +CPU= 15.699
  02:05:44 STEP 2169 H=2169:00 +CPU= 24.059
  02:06:00 STEP 2170 H=2170:00 +CPU= 15.682
  02:06:15 STEP 2171 H=2171:00 +CPU= 15.248
  02:06:41 STEP 2172 H=2172:00 +CPU= 26.459
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002172
  02:06:57 STEP 2173 H=2173:00 +CPU= 15.233
  02:07:12 STEP 2174 H=2174:00 +CPU= 15.741
  02:07:36 STEP 2175 H=2175:00 +CPU= 23.979
  02:07:52 STEP 2176 H=2176:00 +CPU= 15.591
  02:08:07 STEP 2177 H=2177:00 +CPU= 15.268
  02:08:32 STEP 2178 H=2178:00 +CPU= 24.541
  02:08:47 STEP 2179 H=2179:00 +CPU= 15.161
  02:09:02 STEP 2180 H=2180:00 +CPU= 15.597
  02:09:27 STEP 2181 H=2181:00 +CPU= 24.105
  02:09:42 STEP 2182 H=2182:00 +CPU= 15.684
  02:09:58 STEP 2183 H=2183:00 +CPU= 15.945
  02:10:27 STEP 2184 H=2184:00 +CPU= 28.444
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_90.zip
Uploading the intermediate file: upload_file_90.zip
  02:10:42 STEP 2185 H=2185:00 +CPU= 15.120
  02:10:57 STEP 2186 H=2186:00 +CPU= 15.618
  02:11:22 STEP 2187 H=2187:00 +CPU= 24.156
  02:11:37 STEP 2188 H=2188:00 +CPU= 15.675
  02:11:52 STEP 2189 H=2189:00 +CPU= 15.182
  02:12:17 STEP 2190 H=2190:00 +CPU= 24.528
  02:12:32 STEP 2191 H=2191:00 +CPU= 15.213
  02:12:48 STEP 2192 H=2192:00 +CPU= 15.665
  02:13:12 STEP 2193 H=2193:00 +CPU= 24.162
  02:13:28 STEP 2194 H=2194:00 +CPU= 15.673
  02:13:43 STEP 2195 H=2195:00 +CPU= 15.190
  02:14:09 STEP 2196 H=2196:00 +CPU= 26.473
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002196
  02:14:25 STEP 2197 H=2197:00 +CPU= 15.352
  02:14:40 STEP 2198 H=2198:00 +CPU= 15.721
  02:15:05 STEP 2199 H=2199:00 +CPU= 24.169
  02:15:20 STEP 2200 H=2200:00 +CPU= 15.720
  02:15:35 STEP 2201 H=2201:00 +CPU= 15.187
  02:16:00 STEP 2202 H=2202:00 +CPU= 24.583
  02:16:15 STEP 2203 H=2203:00 +CPU= 15.205
  02:16:31 STEP 2204 H=2204:00 +CPU= 15.663
  02:16:55 STEP 2205 H=2205:00 +CPU= 24.097
  02:17:11 STEP 2206 H=2206:00 +CPU= 15.699
  02:17:26 STEP 2207 H=2207:00 +CPU= 15.700
  02:17:55 STEP 2208 H=2208:00 +CPU= 28.188
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_91.zip
Uploading the intermediate file: upload_file_91.zip
  02:18:10 STEP 2209 H=2209:00 +CPU= 15.145
Uploading trickle at timestep: 7948800
  02:18:25 STEP 2210 H=2210:00 +CPU= 15.586
  02:18:49 STEP 2211 H=2211:00 +CPU= 24.067
  02:19:05 STEP 2212 H=2212:00 +CPU= 15.988
  02:19:21 STEP 2213 H=2213:00 +CPU= 15.118
  02:19:45 STEP 2214 H=2214:00 +CPU= 24.599
  02:20:00 STEP 2215 H=2215:00 +CPU= 15.296
  02:20:16 STEP 2216 H=2216:00 +CPU= 15.539
  02:20:40 STEP 2217 H=2217:00 +CPU= 24.251
  02:20:56 STEP 2218 H=2218:00 +CPU= 15.644
  02:21:11 STEP 2219 H=2219:00 +CPU= 15.184
  02:21:38 STEP 2220 H=2220:00 +CPU= 26.615
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002220
  02:21:53 STEP 2221 H=2221:00 +CPU= 15.245
  02:22:09 STEP 2222 H=2222:00 +CPU= 15.650
  02:22:33 STEP 2223 H=2223:00 +CPU= 24.093
  02:22:49 STEP 2224 H=2224:00 +CPU= 15.797
  02:23:04 STEP 2225 H=2225:00 +CPU= 15.153
  02:23:28 STEP 2226 H=2226:00 +CPU= 24.428
  02:23:43 STEP 2227 H=2227:00 +CPU= 15.199
  02:23:59 STEP 2228 H=2228:00 +CPU= 15.540
  02:24:23 STEP 2229 H=2229:00 +CPU= 24.111
  02:24:39 STEP 2230 H=2230:00 +CPU= 15.618
  02:24:54 STEP 2231 H=2231:00 +CPU= 15.632
  02:25:23 STEP 2232 H=2232:00 +CPU= 28.318
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_92.zip
Uploading the intermediate file: upload_file_92.zip
  02:25:38 STEP 2233 H=2233:00 +CPU= 15.251
  02:25:53 STEP 2234 H=2234:00 +CPU= 15.664
  02:26:18 STEP 2235 H=2235:00 +CPU= 24.100
  02:26:33 STEP 2236 H=2236:00 +CPU= 15.821
  02:26:49 STEP 2237 H=2237:00 +CPU= 15.119
  02:27:13 STEP 2238 H=2238:00 +CPU= 24.617
  02:27:28 STEP 2239 H=2239:00 +CPU= 15.275
  02:27:44 STEP 2240 H=2240:00 +CPU= 15.589
  02:28:08 STEP 2241 H=2241:00 +CPU= 24.115
  02:28:24 STEP 2242 H=2242:00 +CPU= 15.575
  02:28:39 STEP 2243 H=2243:00 +CPU= 15.264
  02:29:06 STEP 2244 H=2244:00 +CPU= 26.539
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002244
  02:29:21 STEP 2245 H=2245:00 +CPU= 15.258
  02:29:36 STEP 2246 H=2246:00 +CPU= 15.515
  02:30:01 STEP 2247 H=2247:00 +CPU= 24.227
  02:30:16 STEP 2248 H=2248:00 +CPU= 15.736
  02:30:32 STEP 2249 H=2249:00 +CPU= 15.235
  02:30:56 STEP 2250 H=2250:00 +CPU= 24.546
  02:31:11 STEP 2251 H=2251:00 +CPU= 15.216
  02:31:27 STEP 2252 H=2252:00 +CPU= 15.640
  02:31:51 STEP 2253 H=2253:00 +CPU= 24.117
  02:32:07 STEP 2254 H=2254:00 +CPU= 15.610
  02:32:22 STEP 2255 H=2255:00 +CPU= 15.603
  02:32:50 STEP 2256 H=2256:00 +CPU= 28.069
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_93.zip
Uploading the intermediate file: upload_file_93.zip
  02:33:05 STEP 2257 H=2257:00 +CPU= 14.690
  02:33:20 STEP 2258 H=2258:00 +CPU= 15.214
  02:33:44 STEP 2259 H=2259:00 +CPU= 23.667
  02:33:59 STEP 2260 H=2260:00 +CPU= 15.507
  02:34:14 STEP 2261 H=2261:00 +CPU= 14.742
  02:34:38 STEP 2262 H=2262:00 +CPU= 24.113
  02:34:53 STEP 2263 H=2263:00 +CPU= 14.729
  02:35:08 STEP 2264 H=2264:00 +CPU= 15.150
  02:35:32 STEP 2265 H=2265:00 +CPU= 23.554
  02:35:47 STEP 2266 H=2266:00 +CPU= 15.236
  02:36:02 STEP 2267 H=2267:00 +CPU= 14.745
  02:36:28 STEP 2268 H=2268:00 +CPU= 25.865
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002268
  02:36:42 STEP 2269 H=2269:00 +CPU= 14.845
  02:36:58 STEP 2270 H=2270:00 +CPU= 15.237
  02:37:21 STEP 2271 H=2271:00 +CPU= 23.575
  02:37:36 STEP 2272 H=2272:00 +CPU= 15.169
  02:37:51 STEP 2273 H=2273:00 +CPU= 14.643
  02:38:15 STEP 2274 H=2274:00 +CPU= 23.791
  02:38:30 STEP 2275 H=2275:00 +CPU= 14.766
  02:38:45 STEP 2276 H=2276:00 +CPU= 15.258
  02:39:08 STEP 2277 H=2277:00 +CPU= 23.282
  02:39:23 STEP 2278 H=2278:00 +CPU= 15.155
  02:39:38 STEP 2279 H=2279:00 +CPU= 15.068
  02:40:06 STEP 2280 H=2280:00 +CPU= 27.378
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_94.zip
Uploading the intermediate file: upload_file_94.zip
  02:40:21 STEP 2281 H=2281:00 +CPU= 14.737
  02:40:36 STEP 2282 H=2282:00 +CPU= 15.190
  02:40:59 STEP 2283 H=2283:00 +CPU= 23.511
  02:41:14 STEP 2284 H=2284:00 +CPU= 15.224
  02:41:29 STEP 2285 H=2285:00 +CPU= 14.672
  02:41:53 STEP 2286 H=2286:00 +CPU= 24.040
  02:42:08 STEP 2287 H=2287:00 +CPU= 14.774
  02:42:23 STEP 2288 H=2288:00 +CPU= 15.267
  02:42:47 STEP 2289 H=2289:00 +CPU= 23.595
  02:43:02 STEP 2290 H=2290:00 +CPU= 15.259
  02:43:17 STEP 2291 H=2291:00 +CPU= 14.786
  02:43:43 STEP 2292 H=2292:00 +CPU= 25.638
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002292
  02:43:57 STEP 2293 H=2293:00 +CPU= 14.741
  02:44:12 STEP 2294 H=2294:00 +CPU= 15.152
  02:44:36 STEP 2295 H=2295:00 +CPU= 23.299
  02:44:51 STEP 2296 H=2296:00 +CPU= 15.130
  02:45:06 STEP 2297 H=2297:00 +CPU= 14.878
  02:45:30 STEP 2298 H=2298:00 +CPU= 23.933
  02:45:44 STEP 2299 H=2299:00 +CPU= 14.800
  02:46:00 STEP 2300 H=2300:00 +CPU= 15.108
  02:46:23 STEP 2301 H=2301:00 +CPU= 23.492
  02:46:38 STEP 2302 H=2302:00 +CPU= 15.223
  02:46:53 STEP 2303 H=2303:00 +CPU= 15.159
  02:47:21 STEP 2304 H=2304:00 +CPU= 27.417
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_95.zip
Uploading the intermediate file: upload_file_95.zip
  02:47:36 STEP 2305 H=2305:00 +CPU= 14.728
  02:47:51 STEP 2306 H=2306:00 +CPU= 15.232
  02:48:14 STEP 2307 H=2307:00 +CPU= 23.364
  02:48:29 STEP 2308 H=2308:00 +CPU= 15.146
  02:48:44 STEP 2309 H=2309:00 +CPU= 14.784
  02:49:08 STEP 2310 H=2310:00 +CPU= 23.956
  02:49:23 STEP 2311 H=2311:00 +CPU= 14.679
  02:49:38 STEP 2312 H=2312:00 +CPU= 15.139
  02:50:01 STEP 2313 H=2313:00 +CPU= 23.237
  02:50:16 STEP 2314 H=2314:00 +CPU= 15.054
  02:50:31 STEP 2315 H=2315:00 +CPU= 14.714
  02:50:57 STEP 2316 H=2316:00 +CPU= 25.803
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002316
  02:51:11 STEP 2317 H=2317:00 +CPU= 14.716
  02:51:27 STEP 2318 H=2318:00 +CPU= 15.218
  02:51:50 STEP 2319 H=2319:00 +CPU= 23.457
  02:52:05 STEP 2320 H=2320:00 +CPU= 15.128
  02:52:20 STEP 2321 H=2321:00 +CPU= 14.606
  02:52:44 STEP 2322 H=2322:00 +CPU= 23.924
  02:52:58 STEP 2323 H=2323:00 +CPU= 14.618
  02:53:13 STEP 2324 H=2324:00 +CPU= 14.990
  02:53:37 STEP 2325 H=2325:00 +CPU= 23.300
  02:53:52 STEP 2326 H=2326:00 +CPU= 15.287
  02:54:07 STEP 2327 H=2327:00 +CPU= 15.101
  02:54:35 STEP 2328 H=2328:00 +CPU= 27.459
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_96.zip
Uploading the intermediate file: upload_file_96.zip
  02:54:49 STEP 2329 H=2329:00 +CPU= 14.748
  02:55:05 STEP 2330 H=2330:00 +CPU= 15.196
  02:55:28 STEP 2331 H=2331:00 +CPU= 23.635
  02:55:43 STEP 2332 H=2332:00 +CPU= 15.170
  02:55:58 STEP 2333 H=2333:00 +CPU= 14.574
  02:56:22 STEP 2334 H=2334:00 +CPU= 23.894
  02:56:37 STEP 2335 H=2335:00 +CPU= 14.667
  02:56:52 STEP 2336 H=2336:00 +CPU= 15.114
  02:57:15 STEP 2337 H=2337:00 +CPU= 23.234
  02:57:30 STEP 2338 H=2338:00 +CPU= 15.165
  02:57:45 STEP 2339 H=2339:00 +CPU= 14.689
  02:58:10 STEP 2340 H=2340:00 +CPU= 25.484
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002340
  02:58:25 STEP 2341 H=2341:00 +CPU= 14.797
  02:58:40 STEP 2342 H=2342:00 +CPU= 15.308
  02:59:04 STEP 2343 H=2343:00 +CPU= 23.343
  02:59:19 STEP 2344 H=2344:00 +CPU= 15.141
  02:59:34 STEP 2345 H=2345:00 +CPU= 14.867
  02:59:58 STEP 2346 H=2346:00 +CPU= 23.860
  03:00:12 STEP 2347 H=2347:00 +CPU= 14.657
  03:00:27 STEP 2348 H=2348:00 +CPU= 15.242
  03:00:51 STEP 2349 H=2349:00 +CPU= 23.393
  03:01:06 STEP 2350 H=2350:00 +CPU= 15.114
  03:01:21 STEP 2351 H=2351:00 +CPU= 15.230
  03:01:49 STEP 2352 H=2352:00 +CPU= 27.338
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027_97.zip
Uploading the intermediate file: upload_file_97.zip
  03:02:03 STEP 2353 H=2353:00 +CPU= 14.630
  03:02:18 STEP 2354 H=2354:00 +CPU= 15.178
  03:02:42 STEP 2355 H=2355:00 +CPU= 23.407
  03:02:57 STEP 2356 H=2356:00 +CPU= 15.222
  03:03:12 STEP 2357 H=2357:00 +CPU= 14.708
  03:03:36 STEP 2358 H=2358:00 +CPU= 23.991
  03:03:50 STEP 2359 H=2359:00 +CPU= 14.677
  03:04:06 STEP 2360 H=2360:00 +CPU= 15.207
  03:04:29 STEP 2361 H=2361:00 +CPU= 23.477
  03:04:44 STEP 2362 H=2362:00 +CPU= 15.326
  03:04:59 STEP 2363 H=2363:00 +CPU= 14.749
  03:05:25 STEP 2364 H=2364:00 +CPU= 25.793
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002364
  03:05:40 STEP 2365 H=2365:00 +CPU= 14.704
  03:05:55 STEP 2366 H=2366:00 +CPU= 15.174
  03:06:18 STEP 2367 H=2367:00 +CPU= 23.444
  03:06:34 STEP 2368 H=2368:00 +CPU= 15.217
  03:06:48 STEP 2369 H=2369:00 +CPU= 14.817
  03:07:12 STEP 2370 H=2370:00 +CPU= 23.762
  03:07:27 STEP 2371 H=2371:00 +CPU= 14.761
  03:07:42 STEP 2372 H=2372:00 +CPU= 15.070
  03:08:05 STEP 2373 H=2373:00 +CPU= 23.103
  03:08:20 STEP 2374 H=2374:00 +CPU= 15.231
  03:08:36 STEP 2375 H=2375:00 +CPU= 15.200
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0907
(argv4) batchid: 955
(argv5) wuid: 12172551
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172551
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_e158b290169254828b8e791f668621f6 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0907_1986050100_123_955_12172551.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0907_1986050100_123_955_12172551.zip
ic_ancil_file: ic_ancil_12172551
ifsdata_file: ifsdata_12172551
climate_data_file: clim_data_12172551
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_2a4b4edb114951ab22aa5dee7fd0e8c1 to: /var/lib/boinc-client/slots/3/ic_ancil_12172551.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12172551.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f39a5f5c3fd1db3b70df69f6ae90df7c to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12172551.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12172551.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12172551.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12172551.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12172551.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12172551.xml
-- Model is restarting --
Adjusting last_iter, 2376, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12172551.xml
last_cpu_time: 46158
upload_file_number: 98
last_iter: 2375
last_upload: 8467200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0907_1986050100_123_955_12172551_0_r1684595027
The child process has been launched with process id: 1429
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114824:1673693304:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
ECCODES ERROR   :  IO ERROR: Inappropriate ioctl for device (Inappropriate ioctl for device)
 GRIB_READ_BYTES_INT        50000                       4  FAILED         -11
 GRIB_API ERROR MSG: Input output problem
MPL_ABORT: CALLED FROM PROCESSOR      1 THRD     1
 MPL_ABORT: THRD           1   GRIB_READ_BYTES_INT FAILED
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 1135 MB (maxheap), 361 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       RERESF 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        IOSTREAM_MIX:IO_GET 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         IOSTREAM_MIX:READ_RECORD 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          >MPL-IOSTREAM READ_RECORD ( 650) 
[EC_DRHOOK:W31Ub20:1:1:1429:1429] [20230114:114825:1673693305:1.612] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GRIB_API:IGRIB_READ_BYTES_INT 
 SDL_TRACEBACK: Calling INTEL_TRBK, THRD =            1
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/3/oifs_43r3_model.exe' : sigcontextptr=0x7ffd5598bae0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 16) with plain hex-dump
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [0]: [0x839273] : 0x839273
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [1]: [0x839879] : 0x839879
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [2]: [0x8c2957] : 0x8c2957
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [3]: [0x8c3229] : 0x8c3229
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [4]: [0xa044bc] : 0xa044bc
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [5]: [0x9e49b1] : 0x9e49b1
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [6]: [0x9d8ba9] : 0x9d8ba9
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [7]: [0x54b4a0] : 0x54b4a0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [8]: [0x546a18] : 0x546a18
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [9]: [0x546412] : 0x546412
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [10]: [0x545e28] : 0x545e28
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [11]: [0x40708d] : 0x40708d
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [12]: [0x40220f] : 0x40220f
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [13]: [0x4021a2] : 0x4021a2
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [14]: [0x1dca390] : 0x1dca390
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [15]: [0x40206e] : 0x40206e
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 SDL_TRACEBACK: Done INTEL_TRBK, THRD =            1
abort: 
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:19:07 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 7,948,800 43,106 0.0054
09 May 2023 16:19:07 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 7,948,800 43,106 0.0054
09 May 2023 16:19:07 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 7,084,800 38,618 0.0055
09 May 2023 16:19:07 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 7,084,800 38,618 0.0055
09 May 2023 16:19:07 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 6,220,800 34,141 0.0055
09 May 2023 16:18:27 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 5,356,800 29,656 0.0055
09 May 2023 16:18:27 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 5,356,800 29,656 0.0055
09 May 2023 16:18:24 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 4,492,800 25,160 0.0056
09 May 2023 16:18:24 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 4,492,800 25,160 0.0056
09 May 2023 16:18:21 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 3,628,800 20,666 0.0057
09 May 2023 16:18:21 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 3,628,800 20,666 0.0057
09 May 2023 16:18:15 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 2,764,800 15,873 0.0057
09 May 2023 16:18:15 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 1,900,800 11,009 0.0058
09 May 2023 16:18:07 1532184 22257042 oifs_43r3_ps_0907_1986050100_123_955_12172551_0 864,000 4,952 0.0057


©2024 climateprediction.net