climateprediction.net home page
Task 22253874

Task 22253874

Name oifs_43r3_ps_0934_1983050100_123_952_12169578_0
Workunit 12169578
Created 21 Dec 2022, 2:04:08 UTC
Sent 15 Jan 2023, 14:07:49 UTC
Report deadline 14 Feb 2023, 14:07:49 UTC
Received 16 Jan 2023, 11:18:39 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1515226
Run time 18 hours 54 min 57 sec
CPU time 18 hours 41 min 33 sec
Validate state Invalid
Credit 12,700.80
Device peak FLOPS 4.58 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,410.93 MB
Peak swap size 4,978.06 MB
Peak disk usage 2,842.49 MB
Stderr
<core_client_version>7.9.3</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
89 H=1989:00 +CPU= 33.763
  21:59:18 STEP 1990 H=1990:00 +CPU= 22.144
  21:59:40 STEP 1991 H=1991:00 +CPU= 22.098
  22:00:24 STEP 1992 H=1992:00 +CPU= 39.703
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_82.zip
Uploading the intermediate file: upload_file_82.zip
  22:00:45 STEP 1993 H=1993:00 +CPU= 21.313
  22:01:07 STEP 1994 H=1994:00 +CPU= 21.967
  22:01:41 STEP 1995 H=1995:00 +CPU= 33.589
  22:02:03 STEP 1996 H=1996:00 +CPU= 21.998
  22:02:24 STEP 1997 H=1997:00 +CPU= 21.261
  22:02:59 STEP 1998 H=1998:00 +CPU= 34.382
  22:03:20 STEP 1999 H=1999:00 +CPU= 21.303
  22:03:42 STEP 2000 H=2000:00 +CPU= 22.031
  22:04:16 STEP 2001 H=2001:00 +CPU= 33.642
  22:04:38 STEP 2002 H=2002:00 +CPU= 22.033
  22:04:59 STEP 2003 H=2003:00 +CPU= 21.300
  22:05:36 STEP 2004 H=2004:00 +CPU= 37.370
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002004
  22:05:58 STEP 2005 H=2005:00 +CPU= 21.311
  22:06:20 STEP 2006 H=2006:00 +CPU= 22.043
  22:06:53 STEP 2007 H=2007:00 +CPU= 33.587
  22:07:15 STEP 2008 H=2008:00 +CPU= 22.015
  22:07:37 STEP 2009 H=2009:00 +CPU= 21.306
  22:08:11 STEP 2010 H=2010:00 +CPU= 34.366
  22:08:32 STEP 2011 H=2011:00 +CPU= 21.291
  22:08:54 STEP 2012 H=2012:00 +CPU= 21.992
  22:09:28 STEP 2013 H=2013:00 +CPU= 33.556
  22:09:50 STEP 2014 H=2014:00 +CPU= 21.974
  22:10:12 STEP 2015 H=2015:00 +CPU= 21.920
  22:10:55 STEP 2016 H=2016:00 +CPU= 39.370
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_83.zip
Uploading the intermediate file: upload_file_83.zip
  22:11:16 STEP 2017 H=2017:00 +CPU= 21.311
  22:11:39 STEP 2018 H=2018:00 +CPU= 22.075
  22:12:12 STEP 2019 H=2019:00 +CPU= 33.665
  22:12:34 STEP 2020 H=2020:00 +CPU= 22.096
  22:12:56 STEP 2021 H=2021:00 +CPU= 21.391
  22:13:30 STEP 2022 H=2022:00 +CPU= 34.553
  22:13:52 STEP 2023 H=2023:00 +CPU= 21.422
  22:14:14 STEP 2024 H=2024:00 +CPU= 22.134
  22:14:48 STEP 2025 H=2025:00 +CPU= 33.681
  22:15:10 STEP 2026 H=2026:00 +CPU= 22.133
  22:15:31 STEP 2027 H=2027:00 +CPU= 21.430
  22:16:09 STEP 2028 H=2028:00 +CPU= 37.464
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002028
  22:16:30 STEP 2029 H=2029:00 +CPU= 21.430
  22:16:52 STEP 2030 H=2030:00 +CPU= 22.131
  22:17:26 STEP 2031 H=2031:00 +CPU= 33.682
  22:17:48 STEP 2032 H=2032:00 +CPU= 22.109
  22:18:10 STEP 2033 H=2033:00 +CPU= 21.308
  22:18:44 STEP 2034 H=2034:00 +CPU= 34.373
  22:19:05 STEP 2035 H=2035:00 +CPU= 21.277
  22:19:27 STEP 2036 H=2036:00 +CPU= 22.022
  22:20:01 STEP 2037 H=2037:00 +CPU= 33.521
  22:20:23 STEP 2038 H=2038:00 +CPU= 21.967
  22:20:45 STEP 2039 H=2039:00 +CPU= 21.937
  22:21:28 STEP 2040 H=2040:00 +CPU= 39.467
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_84.zip
Uploading the intermediate file: upload_file_84.zip
  22:21:49 STEP 2041 H=2041:00 +CPU= 21.315
Uploading trickle at timestep: 7344000
  22:22:11 STEP 2042 H=2042:00 +CPU= 22.016
  22:22:45 STEP 2043 H=2043:00 +CPU= 33.490
  22:23:07 STEP 2044 H=2044:00 +CPU= 21.936
  22:23:28 STEP 2045 H=2045:00 +CPU= 21.218
  22:24:02 STEP 2046 H=2046:00 +CPU= 34.312
  22:24:24 STEP 2047 H=2047:00 +CPU= 21.266
  22:24:46 STEP 2048 H=2048:00 +CPU= 21.985
  22:25:19 STEP 2049 H=2049:00 +CPU= 33.528
  22:25:41 STEP 2050 H=2050:00 +CPU= 21.972
  22:26:02 STEP 2051 H=2051:00 +CPU= 21.255
  22:26:40 STEP 2052 H=2052:00 +CPU= 37.297
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002052
  22:27:01 STEP 2053 H=2053:00 +CPU= 21.251
  22:27:23 STEP 2054 H=2054:00 +CPU= 21.985
  22:27:56 STEP 2055 H=2055:00 +CPU= 33.508
  22:28:18 STEP 2056 H=2056:00 +CPU= 21.946
  22:28:40 STEP 2057 H=2057:00 +CPU= 21.218
  22:29:14 STEP 2058 H=2058:00 +CPU= 34.285
  22:29:35 STEP 2059 H=2059:00 +CPU= 21.211
  22:29:57 STEP 2060 H=2060:00 +CPU= 21.939
  22:30:31 STEP 2061 H=2061:00 +CPU= 33.488
  22:30:53 STEP 2062 H=2062:00 +CPU= 21.907
  22:31:15 STEP 2063 H=2063:00 +CPU= 21.874
  22:31:58 STEP 2064 H=2064:00 +CPU= 39.532
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_85.zip
Uploading the intermediate file: upload_file_85.zip
  22:32:20 STEP 2065 H=2065:00 +CPU= 21.218
  22:32:42 STEP 2066 H=2066:00 +CPU= 21.919
  22:33:15 STEP 2067 H=2067:00 +CPU= 33.456
  22:33:37 STEP 2068 H=2068:00 +CPU= 21.936
  22:33:58 STEP 2069 H=2069:00 +CPU= 21.222
  22:34:33 STEP 2070 H=2070:00 +CPU= 34.304
  22:34:54 STEP 2071 H=2071:00 +CPU= 21.276
  22:35:16 STEP 2072 H=2072:00 +CPU= 21.961
  22:35:49 STEP 2073 H=2073:00 +CPU= 33.490
  22:36:11 STEP 2074 H=2074:00 +CPU= 21.985
  22:36:33 STEP 2075 H=2075:00 +CPU= 21.265
  22:37:10 STEP 2076 H=2076:00 +CPU= 37.244
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002076
  22:37:31 STEP 2077 H=2077:00 +CPU= 21.254
  22:37:53 STEP 2078 H=2078:00 +CPU= 21.967
  22:38:27 STEP 2079 H=2079:00 +CPU= 33.475
  22:38:49 STEP 2080 H=2080:00 +CPU= 21.964
  22:39:10 STEP 2081 H=2081:00 +CPU= 21.244
  22:39:44 STEP 2082 H=2082:00 +CPU= 34.296
  22:40:06 STEP 2083 H=2083:00 +CPU= 21.237
  22:40:27 STEP 2084 H=2084:00 +CPU= 21.940
  22:41:01 STEP 2085 H=2085:00 +CPU= 33.483
  22:41:23 STEP 2086 H=2086:00 +CPU= 21.940
  22:41:45 STEP 2087 H=2087:00 +CPU= 21.892
  22:42:28 STEP 2088 H=2088:00 +CPU= 39.418
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_86.zip
Uploading the intermediate file: upload_file_86.zip
  22:42:50 STEP 2089 H=2089:00 +CPU= 21.211
  22:43:12 STEP 2090 H=2090:00 +CPU= 21.953
  22:43:45 STEP 2091 H=2091:00 +CPU= 33.478
  22:44:07 STEP 2092 H=2092:00 +CPU= 21.964
  22:44:29 STEP 2093 H=2093:00 +CPU= 21.218
  22:45:03 STEP 2094 H=2094:00 +CPU= 34.301
  22:45:24 STEP 2095 H=2095:00 +CPU= 21.270
  22:45:46 STEP 2096 H=2096:00 +CPU= 21.989
  22:46:20 STEP 2097 H=2097:00 +CPU= 33.502
  22:46:42 STEP 2098 H=2098:00 +CPU= 22.001
  22:47:03 STEP 2099 H=2099:00 +CPU= 21.254
  22:47:41 STEP 2100 H=2100:00 +CPU= 37.255
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002100
  22:48:02 STEP 2101 H=2101:00 +CPU= 21.255
  22:48:24 STEP 2102 H=2102:00 +CPU= 21.979
  22:48:57 STEP 2103 H=2103:00 +CPU= 33.502
  22:49:19 STEP 2104 H=2104:00 +CPU= 21.966
  22:49:41 STEP 2105 H=2105:00 +CPU= 21.257
  22:50:15 STEP 2106 H=2106:00 +CPU= 34.294
  22:50:36 STEP 2107 H=2107:00 +CPU= 21.235
  22:50:58 STEP 2108 H=2108:00 +CPU= 21.941
  22:51:32 STEP 2109 H=2109:00 +CPU= 33.486
  22:51:54 STEP 2110 H=2110:00 +CPU= 21.932
  22:52:16 STEP 2111 H=2111:00 +CPU= 21.865
  22:53:00 STEP 2112 H=2112:00 +CPU= 39.511
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_87.zip
Uploading the intermediate file: upload_file_87.zip
  22:53:22 STEP 2113 H=2113:00 +CPU= 21.257
  22:53:44 STEP 2114 H=2114:00 +CPU= 22.119
  22:54:17 STEP 2115 H=2115:00 +CPU= 33.720
  22:54:40 STEP 2116 H=2116:00 +CPU= 22.118
  22:55:01 STEP 2117 H=2117:00 +CPU= 21.419
  22:55:36 STEP 2118 H=2118:00 +CPU= 34.567
  22:55:57 STEP 2119 H=2119:00 +CPU= 21.439
  22:56:19 STEP 2120 H=2120:00 +CPU= 22.157
  22:56:53 STEP 2121 H=2121:00 +CPU= 33.706
  22:57:15 STEP 2122 H=2122:00 +CPU= 22.147
  22:57:36 STEP 2123 H=2123:00 +CPU= 21.439
  22:58:14 STEP 2124 H=2124:00 +CPU= 37.519
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002124
  22:58:35 STEP 2125 H=2125:00 +CPU= 21.434
  22:58:58 STEP 2126 H=2126:00 +CPU= 22.163
  22:59:31 STEP 2127 H=2127:00 +CPU= 33.697
  22:59:53 STEP 2128 H=2128:00 +CPU= 22.129
  23:00:15 STEP 2129 H=2129:00 +CPU= 21.411
  23:00:49 STEP 2130 H=2130:00 +CPU= 34.512
  23:01:11 STEP 2131 H=2131:00 +CPU= 21.389
  23:01:33 STEP 2132 H=2132:00 +CPU= 22.112
  23:02:07 STEP 2133 H=2133:00 +CPU= 33.659
  23:02:29 STEP 2134 H=2134:00 +CPU= 22.087
  23:02:51 STEP 2135 H=2135:00 +CPU= 22.058
  23:03:34 STEP 2136 H=2136:00 +CPU= 39.622
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_88.zip
Uploading the intermediate file: upload_file_88.zip
  23:03:56 STEP 2137 H=2137:00 +CPU= 21.343
  23:04:18 STEP 2138 H=2138:00 +CPU= 22.043
  23:04:52 STEP 2139 H=2139:00 +CPU= 33.619
  23:05:14 STEP 2140 H=2140:00 +CPU= 22.053
  23:05:35 STEP 2141 H=2141:00 +CPU= 21.343
  23:06:10 STEP 2142 H=2142:00 +CPU= 34.464
  23:06:31 STEP 2143 H=2143:00 +CPU= 21.365
  23:06:53 STEP 2144 H=2144:00 +CPU= 22.093
  23:07:27 STEP 2145 H=2145:00 +CPU= 33.677
  23:07:49 STEP 2146 H=2146:00 +CPU= 22.094
  23:08:11 STEP 2147 H=2147:00 +CPU= 21.354
  23:08:48 STEP 2148 H=2148:00 +CPU= 37.465
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002148
  23:09:10 STEP 2149 H=2149:00 +CPU= 21.391
  23:09:32 STEP 2150 H=2150:00 +CPU= 22.083
  23:10:05 STEP 2151 H=2151:00 +CPU= 33.667
  23:10:28 STEP 2152 H=2152:00 +CPU= 22.106
  23:10:49 STEP 2153 H=2153:00 +CPU= 21.380
  23:11:23 STEP 2154 H=2154:00 +CPU= 34.462
  23:11:45 STEP 2155 H=2155:00 +CPU= 21.353
  23:12:07 STEP 2156 H=2156:00 +CPU= 22.084
  23:12:41 STEP 2157 H=2157:00 +CPU= 33.611
  23:13:03 STEP 2158 H=2158:00 +CPU= 22.051
  23:13:25 STEP 2159 H=2159:00 +CPU= 22.010
  23:14:07 STEP 2160 H=2160:00 +CPU= 39.568
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_89.zip
Uploading the intermediate file: upload_file_89.zip
  23:14:28 STEP 2161 H=2161:00 +CPU= 21.323
  23:14:51 STEP 2162 H=2162:00 +CPU= 22.048
  23:15:24 STEP 2163 H=2163:00 +CPU= 33.609
  23:15:46 STEP 2164 H=2164:00 +CPU= 22.071
  23:16:08 STEP 2165 H=2165:00 +CPU= 21.348
  23:16:42 STEP 2166 H=2166:00 +CPU= 34.367
  23:17:04 STEP 2167 H=2167:00 +CPU= 21.281
  23:17:26 STEP 2168 H=2168:00 +CPU= 21.998
  23:17:59 STEP 2169 H=2169:00 +CPU= 33.522
  23:18:21 STEP 2170 H=2170:00 +CPU= 22.017
  23:18:42 STEP 2171 H=2171:00 +CPU= 21.283
  23:19:20 STEP 2172 H=2172:00 +CPU= 37.292
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002172
  23:19:41 STEP 2173 H=2173:00 +CPU= 21.286
  23:20:03 STEP 2174 H=2174:00 +CPU= 22.007
  23:20:37 STEP 2175 H=2175:00 +CPU= 33.526
  23:20:59 STEP 2176 H=2176:00 +CPU= 21.981
  23:21:20 STEP 2177 H=2177:00 +CPU= 21.260
  23:21:54 STEP 2178 H=2178:00 +CPU= 34.329
  23:22:16 STEP 2179 H=2179:00 +CPU= 21.262
  23:22:38 STEP 2180 H=2180:00 +CPU= 21.981
  23:23:11 STEP 2181 H=2181:00 +CPU= 33.521
  23:23:33 STEP 2182 H=2182:00 +CPU= 21.989
  23:23:55 STEP 2183 H=2183:00 +CPU= 22.007
  23:24:39 STEP 2184 H=2184:00 +CPU= 39.621
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_90.zip
Uploading the intermediate file: upload_file_90.zip
  23:25:01 STEP 2185 H=2185:00 +CPU= 21.363
  23:25:23 STEP 2186 H=2186:00 +CPU= 22.076
  23:25:56 STEP 2187 H=2187:00 +CPU= 33.692
  23:26:19 STEP 2188 H=2188:00 +CPU= 22.084
  23:26:40 STEP 2189 H=2189:00 +CPU= 21.371
  23:27:15 STEP 2190 H=2190:00 +CPU= 34.507
  23:27:36 STEP 2191 H=2191:00 +CPU= 21.420
  23:27:58 STEP 2192 H=2192:00 +CPU= 22.135
  23:28:32 STEP 2193 H=2193:00 +CPU= 33.716
  23:28:55 STEP 2194 H=2194:00 +CPU= 22.497
  23:29:17 STEP 2195 H=2195:00 +CPU= 21.762
  23:29:55 STEP 2196 H=2196:00 +CPU= 38.193
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002196
  23:30:17 STEP 2197 H=2197:00 +CPU= 21.678
  23:30:39 STEP 2198 H=2198:00 +CPU= 22.300
  23:31:13 STEP 2199 H=2199:00 +CPU= 33.975
  23:31:36 STEP 2200 H=2200:00 +CPU= 22.385
  23:31:57 STEP 2201 H=2201:00 +CPU= 21.599
  23:32:32 STEP 2202 H=2202:00 +CPU= 34.774
  23:32:54 STEP 2203 H=2203:00 +CPU= 21.535
  23:33:16 STEP 2204 H=2204:00 +CPU= 22.223
  23:33:50 STEP 2205 H=2205:00 +CPU= 33.905
  23:34:12 STEP 2206 H=2206:00 +CPU= 22.194
  23:34:34 STEP 2207 H=2207:00 +CPU= 22.108
  23:35:21 STEP 2208 H=2208:00 +CPU= 39.677
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_91.zip
Uploading the intermediate file: upload_file_91.zip
  23:35:42 STEP 2209 H=2209:00 +CPU= 21.456
  23:36:04 STEP 2210 H=2210:00 +CPU= 22.175
  23:36:38 STEP 2211 H=2211:00 +CPU= 33.793
  23:37:00 STEP 2212 H=2212:00 +CPU= 22.159
  23:37:22 STEP 2213 H=2213:00 +CPU= 21.468
  23:37:56 STEP 2214 H=2214:00 +CPU= 34.646
  23:38:18 STEP 2215 H=2215:00 +CPU= 21.484
  23:38:40 STEP 2216 H=2216:00 +CPU= 22.224
  23:39:14 STEP 2217 H=2217:00 +CPU= 33.815
  23:39:36 STEP 2218 H=2218:00 +CPU= 22.169
  23:39:58 STEP 2219 H=2219:00 +CPU= 21.460
  23:40:35 STEP 2220 H=2220:00 +CPU= 37.536
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002220
  23:40:57 STEP 2221 H=2221:00 +CPU= 21.448
  23:41:19 STEP 2222 H=2222:00 +CPU= 22.202
  23:41:53 STEP 2223 H=2223:00 +CPU= 33.834
  23:42:15 STEP 2224 H=2224:00 +CPU= 22.162
  23:42:36 STEP 2225 H=2225:00 +CPU= 21.455
  23:43:11 STEP 2226 H=2226:00 +CPU= 34.606
  23:43:32 STEP 2227 H=2227:00 +CPU= 21.424
  23:43:54 STEP 2228 H=2228:00 +CPU= 22.161
  23:44:28 STEP 2229 H=2229:00 +CPU= 33.810
  23:44:50 STEP 2230 H=2230:00 +CPU= 22.145
  23:45:13 STEP 2231 H=2231:00 +CPU= 22.095
  23:45:59 STEP 2232 H=2232:00 +CPU= 39.737
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_92.zip
Uploading the intermediate file: upload_file_92.zip
  23:46:20 STEP 2233 H=2233:00 +CPU= 21.418
  23:46:42 STEP 2234 H=2234:00 +CPU= 22.121
  23:47:16 STEP 2235 H=2235:00 +CPU= 33.795
  23:47:39 STEP 2236 H=2236:00 +CPU= 22.113
  23:48:00 STEP 2237 H=2237:00 +CPU= 21.407
  23:48:35 STEP 2238 H=2238:00 +CPU= 34.627
  23:48:56 STEP 2239 H=2239:00 +CPU= 21.360
  23:49:18 STEP 2240 H=2240:00 +CPU= 22.010
  23:49:52 STEP 2241 H=2241:00 +CPU= 33.575
  23:50:14 STEP 2242 H=2242:00 +CPU= 22.012
  23:50:35 STEP 2243 H=2243:00 +CPU= 21.292
  23:51:12 STEP 2244 H=2244:00 +CPU= 37.315
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002244
  23:51:34 STEP 2245 H=2245:00 +CPU= 21.283
  23:51:56 STEP 2246 H=2246:00 +CPU= 21.979
  23:52:29 STEP 2247 H=2247:00 +CPU= 33.504
  23:52:51 STEP 2248 H=2248:00 +CPU= 21.973
  23:53:12 STEP 2249 H=2249:00 +CPU= 21.280
  23:53:47 STEP 2250 H=2250:00 +CPU= 34.377
  23:54:08 STEP 2251 H=2251:00 +CPU= 21.281
  23:54:30 STEP 2252 H=2252:00 +CPU= 21.958
  23:55:04 STEP 2253 H=2253:00 +CPU= 33.545
  23:55:26 STEP 2254 H=2254:00 +CPU= 21.932
  23:55:48 STEP 2255 H=2255:00 +CPU= 21.905
  23:56:35 STEP 2256 H=2256:00 +CPU= 39.421
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_93.zip
Uploading the intermediate file: upload_file_93.zip
  23:56:57 STEP 2257 H=2257:00 +CPU= 21.241
  23:57:19 STEP 2258 H=2258:00 +CPU= 21.904
  23:57:52 STEP 2259 H=2259:00 +CPU= 33.424
  23:58:14 STEP 2260 H=2260:00 +CPU= 21.900
  23:58:35 STEP 2261 H=2261:00 +CPU= 21.195
  23:59:10 STEP 2262 H=2262:00 +CPU= 34.256
  23:59:31 STEP 2263 H=2263:00 +CPU= 21.209
  23:59:53 STEP 2264 H=2264:00 +CPU= 21.915
  00:00:26 STEP 2265 H=2265:00 +CPU= 33.475
  00:00:48 STEP 2266 H=2266:00 +CPU= 21.930
  00:01:09 STEP 2267 H=2267:00 +CPU= 21.208
  00:01:46 STEP 2268 H=2268:00 +CPU= 37.181
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002268
  00:02:08 STEP 2269 H=2269:00 +CPU= 21.206
  00:02:30 STEP 2270 H=2270:00 +CPU= 21.923
  00:03:03 STEP 2271 H=2271:00 +CPU= 33.443
  00:03:25 STEP 2272 H=2272:00 +CPU= 21.912
  00:03:46 STEP 2273 H=2273:00 +CPU= 21.190
  00:04:20 STEP 2274 H=2274:00 +CPU= 34.227
  00:04:42 STEP 2275 H=2275:00 +CPU= 21.172
  00:05:04 STEP 2276 H=2276:00 +CPU= 21.882
  00:05:37 STEP 2277 H=2277:00 +CPU= 33.407
  00:05:59 STEP 2278 H=2278:00 +CPU= 21.871
  00:06:21 STEP 2279 H=2279:00 +CPU= 21.826
  00:07:05 STEP 2280 H=2280:00 +CPU= 39.258
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_94.zip
Uploading the intermediate file: upload_file_94.zip
  00:07:26 STEP 2281 H=2281:00 +CPU= 21.139
Uploading trickle at timestep: 8208000
  00:07:48 STEP 2282 H=2282:00 +CPU= 21.925
  00:08:22 STEP 2283 H=2283:00 +CPU= 33.515
  00:08:44 STEP 2284 H=2284:00 +CPU= 21.948
  00:09:05 STEP 2285 H=2285:00 +CPU= 21.251
  00:09:40 STEP 2286 H=2286:00 +CPU= 34.358
  00:10:01 STEP 2287 H=2287:00 +CPU= 21.313
  00:10:23 STEP 2288 H=2288:00 +CPU= 22.061
  00:10:57 STEP 2289 H=2289:00 +CPU= 33.638
  00:11:20 STEP 2290 H=2290:00 +CPU= 22.015
  00:11:41 STEP 2291 H=2291:00 +CPU= 21.271
  00:12:19 STEP 2292 H=2292:00 +CPU= 37.285
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002292
  00:12:40 STEP 2293 H=2293:00 +CPU= 21.271
  00:13:02 STEP 2294 H=2294:00 +CPU= 21.978
  00:13:36 STEP 2295 H=2295:00 +CPU= 33.517
  00:13:58 STEP 2296 H=2296:00 +CPU= 21.975
  00:14:19 STEP 2297 H=2297:00 +CPU= 21.290
  00:14:54 STEP 2298 H=2298:00 +CPU= 34.418
  00:15:15 STEP 2299 H=2299:00 +CPU= 21.298
  00:15:37 STEP 2300 H=2300:00 +CPU= 22.026
  00:16:11 STEP 2301 H=2301:00 +CPU= 33.612
  00:16:33 STEP 2302 H=2302:00 +CPU= 22.023
  00:16:55 STEP 2303 H=2303:00 +CPU= 21.947
  00:17:41 STEP 2304 H=2304:00 +CPU= 39.657
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_95.zip
Uploading the intermediate file: upload_file_95.zip
  00:18:02 STEP 2305 H=2305:00 +CPU= 21.398
  00:18:24 STEP 2306 H=2306:00 +CPU= 22.112
  00:18:58 STEP 2307 H=2307:00 +CPU= 33.780
  00:19:20 STEP 2308 H=2308:00 +CPU= 22.134
  00:19:42 STEP 2309 H=2309:00 +CPU= 21.432
  00:20:16 STEP 2310 H=2310:00 +CPU= 34.598
  00:20:38 STEP 2311 H=2311:00 +CPU= 21.459
  00:21:00 STEP 2312 H=2312:00 +CPU= 22.179
  00:21:34 STEP 2313 H=2313:00 +CPU= 33.810
  00:21:56 STEP 2314 H=2314:00 +CPU= 22.213
  00:22:18 STEP 2315 H=2315:00 +CPU= 21.492
  00:22:55 STEP 2316 H=2316:00 +CPU= 37.580
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002316
  00:23:17 STEP 2317 H=2317:00 +CPU= 21.478
  00:23:39 STEP 2318 H=2318:00 +CPU= 22.174
  00:24:13 STEP 2319 H=2319:00 +CPU= 33.768
  00:24:35 STEP 2320 H=2320:00 +CPU= 22.162
  00:24:56 STEP 2321 H=2321:00 +CPU= 21.453
  00:25:31 STEP 2322 H=2322:00 +CPU= 34.585
  00:25:52 STEP 2323 H=2323:00 +CPU= 21.413
  00:26:14 STEP 2324 H=2324:00 +CPU= 22.130
  00:26:48 STEP 2325 H=2325:00 +CPU= 33.777
  00:27:10 STEP 2326 H=2326:00 +CPU= 22.130
  00:27:32 STEP 2327 H=2327:00 +CPU= 22.064
  00:28:18 STEP 2328 H=2328:00 +CPU= 39.725
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_96.zip
Uploading the intermediate file: upload_file_96.zip
  00:28:39 STEP 2329 H=2329:00 +CPU= 21.379
  00:29:02 STEP 2330 H=2330:00 +CPU= 22.047
  00:29:35 STEP 2331 H=2331:00 +CPU= 33.514
  00:29:57 STEP 2332 H=2332:00 +CPU= 21.929
  00:30:18 STEP 2333 H=2333:00 +CPU= 21.230
  00:30:55 STEP 2334 H=2334:00 +CPU= 34.467
  00:31:17 STEP 2335 H=2335:00 +CPU= 21.381
  00:31:39 STEP 2336 H=2336:00 +CPU= 22.105
  00:32:13 STEP 2337 H=2337:00 +CPU= 33.648
  00:32:35 STEP 2338 H=2338:00 +CPU= 22.075
  00:32:56 STEP 2339 H=2339:00 +CPU= 21.356
  00:33:34 STEP 2340 H=2340:00 +CPU= 37.450
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002340
  00:33:56 STEP 2341 H=2341:00 +CPU= 21.367
  00:34:18 STEP 2342 H=2342:00 +CPU= 22.096
  00:34:51 STEP 2343 H=2343:00 +CPU= 33.640
  00:35:13 STEP 2344 H=2344:00 +CPU= 22.071
  00:35:35 STEP 2345 H=2345:00 +CPU= 21.381
  00:36:09 STEP 2346 H=2346:00 +CPU= 34.577
  00:36:31 STEP 2347 H=2347:00 +CPU= 21.412
  00:36:53 STEP 2348 H=2348:00 +CPU= 22.172
  00:37:27 STEP 2349 H=2349:00 +CPU= 33.797
  00:37:49 STEP 2350 H=2350:00 +CPU= 22.152
  00:38:11 STEP 2351 H=2351:00 +CPU= 22.080
  00:39:01 STEP 2352 H=2352:00 +CPU= 39.520
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_97.zip
Uploading the intermediate file: upload_file_97.zip
  00:39:23 STEP 2353 H=2353:00 +CPU= 21.323
  00:39:45 STEP 2354 H=2354:00 +CPU= 22.045
  00:40:19 STEP 2355 H=2355:00 +CPU= 33.632
  00:40:41 STEP 2356 H=2356:00 +CPU= 22.040
  00:41:02 STEP 2357 H=2357:00 +CPU= 21.361
  00:41:37 STEP 2358 H=2358:00 +CPU= 34.538
  00:41:59 STEP 2359 H=2359:00 +CPU= 21.372
  00:42:21 STEP 2360 H=2360:00 +CPU= 22.063
  00:42:55 STEP 2361 H=2361:00 +CPU= 33.770
  00:43:21 STEP 2362 H=2362:00 +CPU= 22.372
  00:43:43 STEP 2363 H=2363:00 +CPU= 21.524
  00:44:21 STEP 2364 H=2364:00 +CPU= 37.729
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002364
  00:44:43 STEP 2365 H=2365:00 +CPU= 21.501
  00:45:05 STEP 2366 H=2366:00 +CPU= 22.241
  00:45:41 STEP 2367 H=2367:00 +CPU= 33.970
  00:46:03 STEP 2368 H=2368:00 +CPU= 22.150
  00:46:25 STEP 2369 H=2369:00 +CPU= 21.410
  00:46:59 STEP 2370 H=2370:00 +CPU= 34.568
  00:47:21 STEP 2371 H=2371:00 +CPU= 21.395
  00:47:43 STEP 2372 H=2372:00 +CPU= 22.125
  00:48:17 STEP 2373 H=2373:00 +CPU= 33.721
  00:48:39 STEP 2374 H=2374:00 +CPU= 22.065
  00:49:01 STEP 2375 H=2375:00 +CPU= 22.022
  00:49:50 STEP 2376 H=2376:00 +CPU= 39.596
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_98.zip
Uploading the intermediate file: upload_file_98.zip
  00:50:11 STEP 2377 H=2377:00 +CPU= 21.344
  00:50:33 STEP 2378 H=2378:00 +CPU= 22.182
  00:51:07 STEP 2379 H=2379:00 +CPU= 33.847
  00:51:29 STEP 2380 H=2380:00 +CPU= 22.161
  00:51:51 STEP 2381 H=2381:00 +CPU= 21.486
  00:52:26 STEP 2382 H=2382:00 +CPU= 34.690
  00:52:47 STEP 2383 H=2383:00 +CPU= 21.499
  00:53:09 STEP 2384 H=2384:00 +CPU= 22.208
  00:53:43 STEP 2385 H=2385:00 +CPU= 33.862
  00:54:05 STEP 2386 H=2386:00 +CPU= 22.211
  00:54:27 STEP 2387 H=2387:00 +CPU= 21.503
  00:55:05 STEP 2388 H=2388:00 +CPU= 37.632
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002388
  00:55:26 STEP 2389 H=2389:00 +CPU= 21.519
  00:55:48 STEP 2390 H=2390:00 +CPU= 22.214
  00:56:22 STEP 2391 H=2391:00 +CPU= 33.814
  00:56:44 STEP 2392 H=2392:00 +CPU= 22.180
  00:57:06 STEP 2393 H=2393:00 +CPU= 21.459
  00:57:40 STEP 2394 H=2394:00 +CPU= 34.596
  00:58:02 STEP 2395 H=2395:00 +CPU= 21.474
  00:58:24 STEP 2396 H=2396:00 +CPU= 22.170
  00:58:58 STEP 2397 H=2397:00 +CPU= 33.775
  00:59:20 STEP 2398 H=2398:00 +CPU= 22.131
  00:59:43 STEP 2399 H=2399:00 +CPU= 22.088
  01:00:29 STEP 2400 H=2400:00 +CPU= 39.673
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_99.zip
Uploading the intermediate file: upload_file_99.zip
  01:00:51 STEP 2401 H=2401:00 +CPU= 21.375
  01:01:13 STEP 2402 H=2402:00 +CPU= 22.103
  01:01:47 STEP 2403 H=2403:00 +CPU= 33.641
  01:02:09 STEP 2404 H=2404:00 +CPU= 22.095
  01:02:46 STEP 2405 H=2405:00 +CPU= 21.967
  01:03:25 STEP 2406 H=2406:00 +CPU= 34.727
  01:03:47 STEP 2407 H=2407:00 +CPU= 21.414
  01:04:09 STEP 2408 H=2408:00 +CPU= 22.097
  01:04:43 STEP 2409 H=2409:00 +CPU= 33.663
  01:05:06 STEP 2410 H=2410:00 +CPU= 22.309
  01:05:28 STEP 2411 H=2411:00 +CPU= 22.005
  01:06:07 STEP 2412 H=2412:00 +CPU= 38.274
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002412
  01:06:29 STEP 2413 H=2413:00 +CPU= 22.047
  01:06:52 STEP 2414 H=2414:00 +CPU= 22.739
  01:07:27 STEP 2415 H=2415:00 +CPU= 34.644
  01:07:50 STEP 2416 H=2416:00 +CPU= 22.817
  01:08:12 STEP 2417 H=2417:00 +CPU= 22.038
  01:08:48 STEP 2418 H=2418:00 +CPU= 35.484
  01:09:10 STEP 2419 H=2419:00 +CPU= 22.079
  01:09:33 STEP 2420 H=2420:00 +CPU= 22.845
  01:10:08 STEP 2421 H=2421:00 +CPU= 34.726
  01:10:31 STEP 2422 H=2422:00 +CPU= 22.801
  01:10:54 STEP 2423 H=2423:00 +CPU= 22.762
  01:11:39 STEP 2424 H=2424:00 +CPU= 40.745
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_100.zip
Uploading the intermediate file: upload_file_100.zip
  01:12:00 STEP 2425 H=2425:00 +CPU= 21.620
  01:12:23 STEP 2426 H=2426:00 +CPU= 22.126
  01:12:56 STEP 2427 H=2427:00 +CPU= 33.783
  01:13:19 STEP 2428 H=2428:00 +CPU= 22.142
  01:13:40 STEP 2429 H=2429:00 +CPU= 21.432
  01:14:15 STEP 2430 H=2430:00 +CPU= 34.583
  01:14:36 STEP 2431 H=2431:00 +CPU= 21.439
  01:14:58 STEP 2432 H=2432:00 +CPU= 22.163
  01:15:32 STEP 2433 H=2433:00 +CPU= 33.772
  01:15:54 STEP 2434 H=2434:00 +CPU= 22.154
  01:16:16 STEP 2435 H=2435:00 +CPU= 21.464
  01:16:54 STEP 2436 H=2436:00 +CPU= 37.552
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002436
  01:17:15 STEP 2437 H=2437:00 +CPU= 21.426
  01:17:37 STEP 2438 H=2438:00 +CPU= 22.128
  01:18:11 STEP 2439 H=2439:00 +CPU= 33.684
  01:18:33 STEP 2440 H=2440:00 +CPU= 22.097
  01:18:55 STEP 2441 H=2441:00 +CPU= 21.374
  01:19:29 STEP 2442 H=2442:00 +CPU= 34.499
  01:19:51 STEP 2443 H=2443:00 +CPU= 21.364
  01:20:13 STEP 2444 H=2444:00 +CPU= 22.068
  01:20:47 STEP 2445 H=2445:00 +CPU= 33.683
  01:21:09 STEP 2446 H=2446:00 +CPU= 22.059
  01:21:31 STEP 2447 H=2447:00 +CPU= 21.997
  01:22:13 STEP 2448 H=2448:00 +CPU= 39.723
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_101.zip
Uploading the intermediate file: upload_file_101.zip
  01:22:35 STEP 2449 H=2449:00 +CPU= 21.367
  01:22:57 STEP 2450 H=2450:00 +CPU= 22.070
  01:23:31 STEP 2451 H=2451:00 +CPU= 33.739
  01:23:53 STEP 2452 H=2452:00 +CPU= 22.091
  01:24:15 STEP 2453 H=2453:00 +CPU= 21.356
  01:24:49 STEP 2454 H=2454:00 +CPU= 34.560
  01:25:11 STEP 2455 H=2455:00 +CPU= 21.395
  01:25:33 STEP 2456 H=2456:00 +CPU= 22.120
  01:26:07 STEP 2457 H=2457:00 +CPU= 34.018
  01:26:29 STEP 2458 H=2458:00 +CPU= 22.282
  01:26:51 STEP 2459 H=2459:00 +CPU= 21.525
  01:27:29 STEP 2460 H=2460:00 +CPU= 37.823
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002460
  01:27:51 STEP 2461 H=2461:00 +CPU= 21.602
  01:28:13 STEP 2462 H=2462:00 +CPU= 22.340
  01:28:47 STEP 2463 H=2463:00 +CPU= 33.849
  01:29:09 STEP 2464 H=2464:00 +CPU= 22.225
  01:29:31 STEP 2465 H=2465:00 +CPU= 21.411
  01:30:05 STEP 2466 H=2466:00 +CPU= 34.587
  01:30:27 STEP 2467 H=2467:00 +CPU= 21.406
  01:30:49 STEP 2468 H=2468:00 +CPU= 22.110
  01:31:23 STEP 2469 H=2469:00 +CPU= 33.716
  01:31:45 STEP 2470 H=2470:00 +CPU= 22.105
  01:32:07 STEP 2471 H=2471:00 +CPU= 22.049
  01:32:49 STEP 2472 H=2472:00 +CPU= 39.670
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_102.zip
Uploading the intermediate file: upload_file_102.zip
  01:33:10 STEP 2473 H=2473:00 +CPU= 21.375
  01:33:33 STEP 2474 H=2474:00 +CPU= 22.085
  01:34:06 STEP 2475 H=2475:00 +CPU= 33.696
  01:34:29 STEP 2476 H=2476:00 +CPU= 22.114
  01:34:50 STEP 2477 H=2477:00 +CPU= 21.415
  01:35:25 STEP 2478 H=2478:00 +CPU= 34.577
  01:35:46 STEP 2479 H=2479:00 +CPU= 21.431
  01:36:08 STEP 2480 H=2480:00 +CPU= 22.142
  01:36:42 STEP 2481 H=2481:00 +CPU= 33.757
  01:37:05 STEP 2482 H=2482:00 +CPU= 22.216
  01:37:26 STEP 2483 H=2483:00 +CPU= 21.421
  01:38:04 STEP 2484 H=2484:00 +CPU= 37.524
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002484
  01:38:25 STEP 2485 H=2485:00 +CPU= 21.429
  01:38:47 STEP 2486 H=2486:00 +CPU= 22.165
  01:39:21 STEP 2487 H=2487:00 +CPU= 33.749
  01:39:43 STEP 2488 H=2488:00 +CPU= 22.137
  01:40:05 STEP 2489 H=2489:00 +CPU= 21.417
  01:40:40 STEP 2490 H=2490:00 +CPU= 34.552
  01:41:01 STEP 2491 H=2491:00 +CPU= 21.408
  01:41:23 STEP 2492 H=2492:00 +CPU= 22.172
  01:41:58 STEP 2493 H=2493:00 +CPU= 34.778
  01:42:21 STEP 2494 H=2494:00 +CPU= 22.809
  01:42:44 STEP 2495 H=2495:00 +CPU= 22.822
  01:43:27 STEP 2496 H=2496:00 +CPU= 41.009
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_103.zip
Uploading the intermediate file: upload_file_103.zip
  01:43:49 STEP 2497 H=2497:00 +CPU= 21.661
  01:44:11 STEP 2498 H=2498:00 +CPU= 22.088
  01:44:45 STEP 2499 H=2499:00 +CPU= 33.722
  01:45:07 STEP 2500 H=2500:00 +CPU= 22.086
  01:45:29 STEP 2501 H=2501:00 +CPU= 21.371
  01:46:03 STEP 2502 H=2502:00 +CPU= 34.546
  01:46:25 STEP 2503 H=2503:00 +CPU= 21.397
  01:46:47 STEP 2504 H=2504:00 +CPU= 22.132
  01:47:20 STEP 2505 H=2505:00 +CPU= 33.722
  01:47:43 STEP 2506 H=2506:00 +CPU= 22.133
  01:48:04 STEP 2507 H=2507:00 +CPU= 21.404
  01:48:42 STEP 2508 H=2508:00 +CPU= 37.496
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002508
  01:49:03 STEP 2509 H=2509:00 +CPU= 21.432
  01:49:25 STEP 2510 H=2510:00 +CPU= 22.127
  01:49:59 STEP 2511 H=2511:00 +CPU= 33.747
  01:50:21 STEP 2512 H=2512:00 +CPU= 22.092
  01:50:43 STEP 2513 H=2513:00 +CPU= 21.410
  01:51:17 STEP 2514 H=2514:00 +CPU= 34.568
  01:51:39 STEP 2515 H=2515:00 +CPU= 21.377
  01:52:01 STEP 2516 H=2516:00 +CPU= 22.098
  01:52:34 STEP 2517 H=2517:00 +CPU= 33.677
  01:52:57 STEP 2518 H=2518:00 +CPU= 22.067
  01:53:19 STEP 2519 H=2519:00 +CPU= 22.054
  01:54:01 STEP 2520 H=2520:00 +CPU= 39.671
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/9/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285_104.zip
Uploading the intermediate file: upload_file_104.zip
Uploading trickle at timestep: 9072000
  01:54:22 STEP 2521 H=2521:00 +CPU= 21.334
  01:54:44 STEP 2522 H=2522:00 +CPU= 22.120
  01:55:18 STEP 2523 H=2523:00 +CPU= 33.639
  01:55:40 STEP 2524 H=2524:00 +CPU= 22.082
  01:56:02 STEP 2525 H=2525:00 +CPU= 21.355
  01:56:36 STEP 2526 H=2526:00 +CPU= 34.448
  01:56:57 STEP 2527 H=2527:00 +CPU= 21.360
  01:57:20 STEP 2528 H=2528:00 +CPU= 22.112
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1983050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0934
(argv4) batchid: 952
(argv5) wuid: 12169578
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/9
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12169578
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4bf44d00608a00d56a44ae6de1ddc0d3 to: /var/lib/boinc-client/slots/9/oifs_43r3_ps_0934_1983050100_123_952_12169578.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/9/oifs_43r3_ps_0934_1983050100_123_952_12169578.zip
ic_ancil_file: ic_ancil_12169578
ifsdata_file: ifsdata_12169578
climate_data_file: clim_data_12169578
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_636b8e4bcf2948bc431d4fe5ee1074dc to: /var/lib/boinc-client/slots/9/ic_ancil_12169578.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/9/ic_ancil_12169578.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5ddb20ea42dbdc41042c431ab25be209 to: /var/lib/boinc-client/slots/9/ifsdata/ifsdata_12169578.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/9/ifsdata/ifsdata_12169578.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/9/159l_2/clim_data_12169578.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/9/159l_2/clim_data_12169578.zip
Checking for progress XML file: /var/lib/boinc-client/slots/9/progress_file_12169578.xml
Opened progress file ok : /var/lib/boinc-client/slots/9/progress_file_12169578.xml
-- Model is restarting --
Adjusting last_iter, 2529, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/9/progress_file_12169578.xml
last_cpu_time: 67061
upload_file_number: 105
last_iter: 2519
last_upload: 9072000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0934_1983050100_123_952_12169578_0_r545311285
The child process has been launched with process id: 21175
Executing the command: /var/lib/boinc-client/slots/9/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Ken-Boinc-4:1:1:21175:21175] [20230116:020726:1673863646:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  02:08:36 STEP 2520 H=2520:00 +CPU= 40.851
  02:09:30 STEP 2521 H=2521:00 +CPU= 22.694
  02:10:02 STEP 2522 H=2522:00 +CPU= 21.010
  02:10:37 STEP 2523 H=2523:00 +CPU= 33.597
  02:11:08 STEP 2524 H=2524:00 +CPU= 22.943
  02:11:48 STEP 2525 H=2525:00 +CPU= 23.086
  02:12:34 STEP 2526 H=2526:00 +CPU= 36.945
  02:13:00 STEP 2527 H=2527:00 +CPU= 23.340
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:22:10 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 9,072,000 66,829 0.0074
09 May 2023 16:22:02 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 8,208,000 60,516 0.0074
09 May 2023 16:21:56 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 7,344,000 54,243 0.0074
09 May 2023 16:21:49 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 6,480,000 47,950 0.0074
09 May 2023 16:21:42 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 5,616,000 41,620 0.0074
09 May 2023 16:21:38 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 4,752,000 35,278 0.0074
09 May 2023 16:21:27 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 3,888,000 28,966 0.0075
09 May 2023 16:21:15 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 2,592,000 19,075 0.0074
09 May 2023 16:21:08 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 1,728,000 12,726 0.0074
09 May 2023 16:21:02 1515226 22253874 oifs_43r3_ps_0934_1983050100_123_952_12169578_0 864,000 6,354 0.0074


©2024 cpdn.org