climateprediction.net home page
Task 22269019

Task 22269019

Name oifs_43r3_ps_0180_1998050100_123_967_12183824_0
Workunit 12183824
Created 22 Dec 2022, 15:31:57 UTC
Sent 22 Dec 2022, 16:23:22 UTC
Report deadline 21 Jan 2023, 16:23:22 UTC
Received 26 Dec 2022, 0:43:20 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1509857
Run time 22 hours 38 min 24 sec
CPU time 21 hours 33 min 28 sec
Validate state Invalid
Credit 11,249.28
Device peak FLOPS 5.51 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,410.10 MB
Peak swap size 4,978.16 MB
Peak disk usage 7,531.63 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
95
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_85.zip
Uploading the intermediate file: upload_file_85.zip
  20:43:39 STEP 2065 H=2065:00 +CPU= 24.474
  20:44:06 STEP 2066 H=2066:00 +CPU= 24.951
  20:44:49 STEP 2067 H=2067:00 +CPU= 38.228
  20:45:19 STEP 2068 H=2068:00 +CPU= 25.497
  20:45:45 STEP 2069 H=2069:00 +CPU= 24.226
  20:46:30 STEP 2070 H=2070:00 +CPU= 39.186
  20:46:57 STEP 2071 H=2071:00 +CPU= 24.506
  20:47:26 STEP 2072 H=2072:00 +CPU= 25.082
  20:48:11 STEP 2073 H=2073:00 +CPU= 38.140
  20:48:39 STEP 2074 H=2074:00 +CPU= 25.157
  20:49:07 STEP 2075 H=2075:00 +CPU= 24.248
  20:49:55 STEP 2076 H=2076:00 +CPU= 42.241
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002076
  20:50:23 STEP 2077 H=2077:00 +CPU= 24.445
  20:50:52 STEP 2078 H=2078:00 +CPU= 25.041
  20:51:36 STEP 2079 H=2079:00 +CPU= 37.838
  20:52:05 STEP 2080 H=2080:00 +CPU= 25.067
  20:52:33 STEP 2081 H=2081:00 +CPU= 24.203
  20:53:16 STEP 2082 H=2082:00 +CPU= 38.628
  20:53:44 STEP 2083 H=2083:00 +CPU= 24.402
  20:54:13 STEP 2084 H=2084:00 +CPU= 25.017
  20:54:56 STEP 2085 H=2085:00 +CPU= 37.947
  20:55:25 STEP 2086 H=2086:00 +CPU= 25.146
  20:55:54 STEP 2087 H=2087:00 +CPU= 25.073
  20:56:47 STEP 2088 H=2088:00 +CPU= 45.151
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_86.zip
Uploading the intermediate file: upload_file_86.zip
  20:57:15 STEP 2089 H=2089:00 +CPU= 24.294
  20:57:43 STEP 2090 H=2090:00 +CPU= 24.897
  20:58:25 STEP 2091 H=2091:00 +CPU= 37.973
  20:58:54 STEP 2092 H=2092:00 +CPU= 25.189
  20:59:22 STEP 2093 H=2093:00 +CPU= 24.283
  21:00:07 STEP 2094 H=2094:00 +CPU= 38.814
  21:00:35 STEP 2095 H=2095:00 +CPU= 24.490
  21:01:05 STEP 2096 H=2096:00 +CPU= 25.206
  21:01:48 STEP 2097 H=2097:00 +CPU= 38.169
  21:02:18 STEP 2098 H=2098:00 +CPU= 25.302
  21:02:44 STEP 2099 H=2099:00 +CPU= 24.418
  21:03:32 STEP 2100 H=2100:00 +CPU= 42.499
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002100
  21:04:01 STEP 2101 H=2101:00 +CPU= 24.498
  21:04:30 STEP 2102 H=2102:00 +CPU= 25.040
  21:05:13 STEP 2103 H=2103:00 +CPU= 37.974
  21:05:43 STEP 2104 H=2104:00 +CPU= 25.263
  21:06:10 STEP 2105 H=2105:00 +CPU= 24.179
  21:06:55 STEP 2106 H=2106:00 +CPU= 39.118
  21:07:23 STEP 2107 H=2107:00 +CPU= 24.363
  21:07:51 STEP 2108 H=2108:00 +CPU= 25.051
  21:08:35 STEP 2109 H=2109:00 +CPU= 38.103
  21:09:05 STEP 2110 H=2110:00 +CPU= 25.288
  21:09:34 STEP 2111 H=2111:00 +CPU= 25.174
  21:10:28 STEP 2112 H=2112:00 +CPU= 45.088
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_87.zip
Uploading the intermediate file: upload_file_87.zip
  21:10:57 STEP 2113 H=2113:00 +CPU= 24.383
  21:11:25 STEP 2114 H=2114:00 +CPU= 25.011
  21:12:08 STEP 2115 H=2115:00 +CPU= 37.896
  21:12:36 STEP 2116 H=2116:00 +CPU= 25.239
  21:13:04 STEP 2117 H=2117:00 +CPU= 24.164
  21:13:48 STEP 2118 H=2118:00 +CPU= 38.576
  21:14:17 STEP 2119 H=2119:00 +CPU= 24.512
  21:14:47 STEP 2120 H=2120:00 +CPU= 25.095
  21:15:32 STEP 2121 H=2121:00 +CPU= 38.113
  21:16:01 STEP 2122 H=2122:00 +CPU= 25.238
  21:16:29 STEP 2123 H=2123:00 +CPU= 24.608
  21:17:19 STEP 2124 H=2124:00 +CPU= 42.320
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002124
  21:17:47 STEP 2125 H=2125:00 +CPU= 24.499
  21:18:15 STEP 2126 H=2126:00 +CPU= 25.029
  21:19:00 STEP 2127 H=2127:00 +CPU= 37.938
  21:19:30 STEP 2128 H=2128:00 +CPU= 25.207
  21:19:57 STEP 2129 H=2129:00 +CPU= 24.171
  21:20:40 STEP 2130 H=2130:00 +CPU= 38.767
  21:21:08 STEP 2131 H=2131:00 +CPU= 24.326
  21:21:36 STEP 2132 H=2132:00 +CPU= 24.897
  21:22:19 STEP 2133 H=2133:00 +CPU= 37.927
  21:22:48 STEP 2134 H=2134:00 +CPU= 25.206
  21:23:17 STEP 2135 H=2135:00 +CPU= 25.032
  21:24:10 STEP 2136 H=2136:00 +CPU= 45.590
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_88.zip
Uploading the intermediate file: upload_file_88.zip
  21:24:38 STEP 2137 H=2137:00 +CPU= 24.377
  21:25:06 STEP 2138 H=2138:00 +CPU= 24.970
  21:25:49 STEP 2139 H=2139:00 +CPU= 37.906
  21:26:17 STEP 2140 H=2140:00 +CPU= 25.182
  21:26:45 STEP 2141 H=2141:00 +CPU= 24.405
  21:27:29 STEP 2142 H=2142:00 +CPU= 38.888
  21:27:57 STEP 2143 H=2143:00 +CPU= 24.425
  21:28:27 STEP 2144 H=2144:00 +CPU= 25.136
  21:29:11 STEP 2145 H=2145:00 +CPU= 38.204
  21:29:39 STEP 2146 H=2146:00 +CPU= 25.182
  21:30:07 STEP 2147 H=2147:00 +CPU= 24.249
  21:30:56 STEP 2148 H=2148:00 +CPU= 42.412
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002148
  21:31:25 STEP 2149 H=2149:00 +CPU= 24.423
  21:31:54 STEP 2150 H=2150:00 +CPU= 25.171
  21:32:38 STEP 2151 H=2151:00 +CPU= 37.880
  21:33:06 STEP 2152 H=2152:00 +CPU= 25.147
  21:33:33 STEP 2153 H=2153:00 +CPU= 24.201
  21:34:18 STEP 2154 H=2154:00 +CPU= 38.862
  21:34:47 STEP 2155 H=2155:00 +CPU= 24.532
  21:35:15 STEP 2156 H=2156:00 +CPU= 24.947
  21:35:58 STEP 2157 H=2157:00 +CPU= 38.091
  21:36:26 STEP 2158 H=2158:00 +CPU= 25.228
  21:36:55 STEP 2159 H=2159:00 +CPU= 25.062
  21:37:46 STEP 2160 H=2160:00 +CPU= 45.374
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_89.zip
Uploading the intermediate file: upload_file_89.zip
  21:38:13 STEP 2161 H=2161:00 +CPU= 24.283
  21:38:41 STEP 2162 H=2162:00 +CPU= 25.017
  21:39:25 STEP 2163 H=2163:00 +CPU= 38.180
  21:39:54 STEP 2164 H=2164:00 +CPU= 25.339
  21:40:21 STEP 2165 H=2165:00 +CPU= 24.396
  21:41:06 STEP 2166 H=2166:00 +CPU= 39.037
  21:41:35 STEP 2167 H=2167:00 +CPU= 24.631
  21:42:04 STEP 2168 H=2168:00 +CPU= 25.121
  21:42:47 STEP 2169 H=2169:00 +CPU= 37.964
  21:43:16 STEP 2170 H=2170:00 +CPU= 25.356
  21:43:44 STEP 2171 H=2171:00 +CPU= 24.362
  21:44:33 STEP 2172 H=2172:00 +CPU= 42.272
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002172
  21:45:02 STEP 2173 H=2173:00 +CPU= 24.513
  21:45:32 STEP 2174 H=2174:00 +CPU= 25.081
  21:46:18 STEP 2175 H=2175:00 +CPU= 38.298
  21:46:47 STEP 2176 H=2176:00 +CPU= 25.315
  21:47:16 STEP 2177 H=2177:00 +CPU= 24.378
  21:48:01 STEP 2178 H=2178:00 +CPU= 38.834
  21:48:29 STEP 2179 H=2179:00 +CPU= 24.444
  21:48:57 STEP 2180 H=2180:00 +CPU= 25.030
  21:49:42 STEP 2181 H=2181:00 +CPU= 38.234
  21:50:11 STEP 2182 H=2182:00 +CPU= 25.398
  21:50:40 STEP 2183 H=2183:00 +CPU= 25.140
  21:51:32 STEP 2184 H=2184:00 +CPU= 45.289
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_90.zip
Uploading the intermediate file: upload_file_90.zip
  21:52:01 STEP 2185 H=2185:00 +CPU= 24.555
  21:52:30 STEP 2186 H=2186:00 +CPU= 25.045
  21:53:13 STEP 2187 H=2187:00 +CPU= 38.117
  21:53:43 STEP 2188 H=2188:00 +CPU= 25.325
  21:54:11 STEP 2189 H=2189:00 +CPU= 24.425
  21:54:56 STEP 2190 H=2190:00 +CPU= 39.046
  21:55:25 STEP 2191 H=2191:00 +CPU= 24.559
  21:55:54 STEP 2192 H=2192:00 +CPU= 25.344
  21:56:39 STEP 2193 H=2193:00 +CPU= 38.311
  21:57:08 STEP 2194 H=2194:00 +CPU= 25.355
  21:57:35 STEP 2195 H=2195:00 +CPU= 24.424
  21:58:24 STEP 2196 H=2196:00 +CPU= 42.643
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002196
  21:58:52 STEP 2197 H=2197:00 +CPU= 24.668
  21:59:21 STEP 2198 H=2198:00 +CPU= 25.137
  22:00:05 STEP 2199 H=2199:00 +CPU= 38.321
  22:00:35 STEP 2200 H=2200:00 +CPU= 25.267
  22:01:04 STEP 2201 H=2201:00 +CPU= 24.426
  22:01:49 STEP 2202 H=2202:00 +CPU= 38.907
  22:02:19 STEP 2203 H=2203:00 +CPU= 24.591
  22:02:48 STEP 2204 H=2204:00 +CPU= 25.209
  22:03:32 STEP 2205 H=2205:00 +CPU= 38.110
  22:04:02 STEP 2206 H=2206:00 +CPU= 25.419
  22:04:32 STEP 2207 H=2207:00 +CPU= 25.342
  22:05:26 STEP 2208 H=2208:00 +CPU= 45.947
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_91.zip
Uploading the intermediate file: upload_file_91.zip
  22:05:53 STEP 2209 H=2209:00 +CPU= 24.409
  22:06:23 STEP 2210 H=2210:00 +CPU= 25.211
  22:07:07 STEP 2211 H=2211:00 +CPU= 38.173
  22:07:36 STEP 2212 H=2212:00 +CPU= 25.403
  22:08:03 STEP 2213 H=2213:00 +CPU= 24.478
  22:08:49 STEP 2214 H=2214:00 +CPU= 39.115
  22:09:18 STEP 2215 H=2215:00 +CPU= 24.785
  22:09:49 STEP 2216 H=2216:00 +CPU= 25.307
  22:10:34 STEP 2217 H=2217:00 +CPU= 38.335
  22:11:04 STEP 2218 H=2218:00 +CPU= 25.664
  22:11:33 STEP 2219 H=2219:00 +CPU= 24.562
  22:12:24 STEP 2220 H=2220:00 +CPU= 42.889
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002220
  22:12:51 STEP 2221 H=2221:00 +CPU= 24.523
  22:13:19 STEP 2222 H=2222:00 +CPU= 25.282
  22:14:04 STEP 2223 H=2223:00 +CPU= 38.163
  22:14:31 STEP 2224 H=2224:00 +CPU= 25.302
  22:14:59 STEP 2225 H=2225:00 +CPU= 24.417
  22:15:44 STEP 2226 H=2226:00 +CPU= 39.102
  22:16:11 STEP 2227 H=2227:00 +CPU= 24.630
  22:16:40 STEP 2228 H=2228:00 +CPU= 25.096
  22:17:25 STEP 2229 H=2229:00 +CPU= 38.151
  22:17:53 STEP 2230 H=2230:00 +CPU= 25.285
  22:18:22 STEP 2231 H=2231:00 +CPU= 25.205
  22:19:14 STEP 2232 H=2232:00 +CPU= 45.396
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_92.zip
Uploading the intermediate file: upload_file_92.zip
Uploading trickle at timestep: 8035200
  22:19:42 STEP 2233 H=2233:00 +CPU= 24.492
  22:20:10 STEP 2234 H=2234:00 +CPU= 25.233
  22:20:54 STEP 2235 H=2235:00 +CPU= 38.156
  22:21:24 STEP 2236 H=2236:00 +CPU= 25.297
  22:21:52 STEP 2237 H=2237:00 +CPU= 24.540
  22:22:39 STEP 2238 H=2238:00 +CPU= 39.315
  22:23:07 STEP 2239 H=2239:00 +CPU= 24.724
  22:23:37 STEP 2240 H=2240:00 +CPU= 25.334
  22:24:21 STEP 2241 H=2241:00 +CPU= 38.256
  22:24:52 STEP 2242 H=2242:00 +CPU= 25.677
  22:25:20 STEP 2243 H=2243:00 +CPU= 24.446
  22:26:09 STEP 2244 H=2244:00 +CPU= 42.698
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002244
  22:26:38 STEP 2245 H=2245:00 +CPU= 24.745
  22:27:07 STEP 2246 H=2246:00 +CPU= 25.404
  22:27:51 STEP 2247 H=2247:00 +CPU= 38.228
  22:28:20 STEP 2248 H=2248:00 +CPU= 25.286
  22:28:48 STEP 2249 H=2249:00 +CPU= 24.412
  22:29:31 STEP 2250 H=2250:00 +CPU= 39.077
  22:29:59 STEP 2251 H=2251:00 +CPU= 24.575
  22:30:28 STEP 2252 H=2252:00 +CPU= 25.347
  22:31:10 STEP 2253 H=2253:00 +CPU= 38.072
  22:31:39 STEP 2254 H=2254:00 +CPU= 25.410
  22:32:09 STEP 2255 H=2255:00 +CPU= 25.213
  22:33:03 STEP 2256 H=2256:00 +CPU= 45.472
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_93.zip
Uploading the intermediate file: upload_file_93.zip
  22:33:31 STEP 2257 H=2257:00 +CPU= 24.513
  22:34:00 STEP 2258 H=2258:00 +CPU= 25.323
  22:34:45 STEP 2259 H=2259:00 +CPU= 38.282
  22:35:16 STEP 2260 H=2260:00 +CPU= 25.438
  22:35:44 STEP 2261 H=2261:00 +CPU= 24.456
  22:36:30 STEP 2262 H=2262:00 +CPU= 39.204
  22:36:58 STEP 2263 H=2263:00 +CPU= 24.551
  22:37:27 STEP 2264 H=2264:00 +CPU= 25.525
  22:38:11 STEP 2265 H=2265:00 +CPU= 38.262
  22:38:40 STEP 2266 H=2266:00 +CPU= 25.269
  22:39:08 STEP 2267 H=2267:00 +CPU= 24.516
  22:39:57 STEP 2268 H=2268:00 +CPU= 42.504
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002268
  22:40:25 STEP 2269 H=2269:00 +CPU= 24.676
  22:40:53 STEP 2270 H=2270:00 +CPU= 25.110
  22:41:37 STEP 2271 H=2271:00 +CPU= 38.203
  22:42:06 STEP 2272 H=2272:00 +CPU= 25.404
  22:42:34 STEP 2273 H=2273:00 +CPU= 24.459
  22:43:20 STEP 2274 H=2274:00 +CPU= 39.225
  22:43:49 STEP 2275 H=2275:00 +CPU= 24.795
  22:44:17 STEP 2276 H=2276:00 +CPU= 25.358
  22:45:02 STEP 2277 H=2277:00 +CPU= 38.299
  22:45:32 STEP 2278 H=2278:00 +CPU= 25.556
  22:46:02 STEP 2279 H=2279:00 +CPU= 25.396
  22:46:55 STEP 2280 H=2280:00 +CPU= 45.392
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_94.zip
Uploading the intermediate file: upload_file_94.zip
  22:47:24 STEP 2281 H=2281:00 +CPU= 24.736
  22:47:53 STEP 2282 H=2282:00 +CPU= 24.807
  22:48:33 STEP 2283 H=2283:00 +CPU= 37.170
  22:49:01 STEP 2284 H=2284:00 +CPU= 24.905
  22:49:27 STEP 2285 H=2285:00 +CPU= 23.966
  22:50:09 STEP 2286 H=2286:00 +CPU= 38.088
  22:50:35 STEP 2287 H=2287:00 +CPU= 24.282
  22:51:04 STEP 2288 H=2288:00 +CPU= 25.285
  22:51:44 STEP 2289 H=2289:00 +CPU= 37.595
  22:52:08 STEP 2290 H=2290:00 +CPU= 23.311
  22:52:31 STEP 2291 H=2291:00 +CPU= 23.146
  22:53:13 STEP 2292 H=2292:00 +CPU= 40.679
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002292
  22:53:35 STEP 2293 H=2293:00 +CPU= 22.587
  22:53:59 STEP 2294 H=2294:00 +CPU= 23.737
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1998050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0180
(argv4) batchid: 967
(argv5) wuid: 12183824
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c4207c6d9721b1189e4da373f0124007 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0180_1998050100_123_967_12183824.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0180_1998050100_123_967_12183824.zip
ic_ancil_file: ic_ancil_12183824
ifsdata_file: ifsdata_12183824
climate_data_file: clim_data_12183824
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_cf7ab91c23ba43d73f16dc08356f78a6 to: /var/lib/boinc-client/slots/8/ic_ancil_12183824.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12183824.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c2416c7c3839a8e76007e592492566f4 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12183824.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12183824.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12183824.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12183824.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12183824.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12183824.xml
-- Model is restarting --
Adjusting last_iter, 2295, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12183824.xml
last_cpu_time: 71371
upload_file_number: 95
last_iter: 2279
last_upload: 8208000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445
The child process has been launched with process id: 125224
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:125224:125224] [20221225:225521:1672005321:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  22:56:02 STEP 2280 H=2280:00 +CPU= 37.949
  22:56:23 STEP 2281 H=2281:00 +CPU= 20.879
  22:56:45 STEP 2282 H=2282:00 +CPU= 21.648
  22:57:20 STEP 2283 H=2283:00 +CPU= 35.328
  22:57:44 STEP 2284 H=2284:00 +CPU= 23.270
  22:58:07 STEP 2285 H=2285:00 +CPU= 23.032
  22:58:45 STEP 2286 H=2286:00 +CPU= 37.705
  22:59:09 STEP 2287 H=2287:00 +CPU= 23.477
  22:59:33 STEP 2288 H=2288:00 +CPU= 24.190
  23:00:09 STEP 2289 H=2289:00 +CPU= 36.500
  23:00:34 STEP 2290 H=2290:00 +CPU= 24.692
  23:00:58 STEP 2291 H=2291:00 +CPU= 23.695
  23:01:39 STEP 2292 H=2292:00 +CPU= 40.946
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002292
  23:02:03 STEP 2293 H=2293:00 +CPU= 23.545
  23:02:27 STEP 2294 H=2294:00 +CPU= 24.168
  23:03:04 STEP 2295 H=2295:00 +CPU= 36.656
  23:03:28 STEP 2296 H=2296:00 +CPU= 24.283
  23:03:52 STEP 2297 H=2297:00 +CPU= 23.348
  23:04:30 STEP 2298 H=2298:00 +CPU= 37.433
  23:04:54 STEP 2299 H=2299:00 +CPU= 23.377
  23:05:18 STEP 2300 H=2300:00 +CPU= 24.298
  23:05:55 STEP 2301 H=2301:00 +CPU= 36.760
  23:06:20 STEP 2302 H=2302:00 +CPU= 24.210
  23:06:44 STEP 2303 H=2303:00 +CPU= 24.193
  23:07:29 STEP 2304 H=2304:00 +CPU= 43.880
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_95.zip
Uploading the intermediate file: upload_file_95.zip
  23:07:53 STEP 2305 H=2305:00 +CPU= 23.579
  23:08:18 STEP 2306 H=2306:00 +CPU= 24.563
  23:08:54 STEP 2307 H=2307:00 +CPU= 36.261
  23:09:17 STEP 2308 H=2308:00 +CPU= 22.242
  23:09:39 STEP 2309 H=2309:00 +CPU= 22.276
  23:10:15 STEP 2310 H=2310:00 +CPU= 35.631
  23:10:39 STEP 2311 H=2311:00 +CPU= 23.454
  23:11:04 STEP 2312 H=2312:00 +CPU= 24.438
  23:11:41 STEP 2313 H=2313:00 +CPU= 36.897
  23:12:05 STEP 2314 H=2314:00 +CPU= 24.460
  23:12:29 STEP 2315 H=2315:00 +CPU= 23.760
  23:13:11 STEP 2316 H=2316:00 +CPU= 41.107
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002316
  23:13:35 STEP 2317 H=2317:00 +CPU= 23.665
  23:13:59 STEP 2318 H=2318:00 +CPU= 24.307
  23:14:36 STEP 2319 H=2319:00 +CPU= 36.674
  23:15:01 STEP 2320 H=2320:00 +CPU= 24.608
  23:15:25 STEP 2321 H=2321:00 +CPU= 23.841
  23:16:03 STEP 2322 H=2322:00 +CPU= 37.527
  23:16:27 STEP 2323 H=2323:00 +CPU= 23.576
  23:16:52 STEP 2324 H=2324:00 +CPU= 24.356
  23:17:29 STEP 2325 H=2325:00 +CPU= 36.659
  23:17:53 STEP 2326 H=2326:00 +CPU= 24.497
  23:18:18 STEP 2327 H=2327:00 +CPU= 24.230
  23:19:02 STEP 2328 H=2328:00 +CPU= 43.979
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_96.zip
Uploading the intermediate file: upload_file_96.zip
  23:19:26 STEP 2329 H=2329:00 +CPU= 23.513
  23:19:50 STEP 2330 H=2330:00 +CPU= 24.163
  23:20:27 STEP 2331 H=2331:00 +CPU= 36.700
  23:20:52 STEP 2332 H=2332:00 +CPU= 24.482
  23:21:16 STEP 2333 H=2333:00 +CPU= 23.665
  23:21:53 STEP 2334 H=2334:00 +CPU= 36.765
  23:22:15 STEP 2335 H=2335:00 +CPU= 21.281
  23:22:38 STEP 2336 H=2336:00 +CPU= 22.721
  23:23:13 STEP 2337 H=2337:00 +CPU= 35.270
  23:23:37 STEP 2338 H=2338:00 +CPU= 23.534
  23:24:00 STEP 2339 H=2339:00 +CPU= 23.147
  23:24:39 STEP 2340 H=2340:00 +CPU= 38.827
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002340
  23:25:02 STEP 2341 H=2341:00 +CPU= 23.018
  23:25:26 STEP 2342 H=2342:00 +CPU= 23.860
  23:26:02 STEP 2343 H=2343:00 +CPU= 34.935
  23:26:26 STEP 2344 H=2344:00 +CPU= 24.239
  23:26:50 STEP 2345 H=2345:00 +CPU= 23.537
  23:27:27 STEP 2346 H=2346:00 +CPU= 37.533
  23:27:51 STEP 2347 H=2347:00 +CPU= 23.815
  23:28:16 STEP 2348 H=2348:00 +CPU= 24.186
  23:28:53 STEP 2349 H=2349:00 +CPU= 36.940
  23:29:18 STEP 2350 H=2350:00 +CPU= 24.502
  23:29:42 STEP 2351 H=2351:00 +CPU= 24.452
  23:30:27 STEP 2352 H=2352:00 +CPU= 43.642
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_97.zip
Uploading the intermediate file: upload_file_97.zip
  23:30:51 STEP 2353 H=2353:00 +CPU= 23.826
  23:31:14 STEP 2354 H=2354:00 +CPU= 22.664
  23:31:48 STEP 2355 H=2355:00 +CPU= 33.934
  23:32:11 STEP 2356 H=2356:00 +CPU= 22.801
  23:32:34 STEP 2357 H=2357:00 +CPU= 23.036
  23:33:12 STEP 2358 H=2358:00 +CPU= 37.672
  23:33:36 STEP 2359 H=2359:00 +CPU= 23.988
  23:34:01 STEP 2360 H=2360:00 +CPU= 24.334
  23:34:38 STEP 2361 H=2361:00 +CPU= 36.675
  23:35:03 STEP 2362 H=2362:00 +CPU= 24.992
  23:35:27 STEP 2363 H=2363:00 +CPU= 23.845
  23:36:08 STEP 2364 H=2364:00 +CPU= 41.090
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002364
  23:36:32 STEP 2365 H=2365:00 +CPU= 23.725
  23:36:57 STEP 2366 H=2366:00 +CPU= 24.336
  23:37:34 STEP 2367 H=2367:00 +CPU= 37.003
  23:37:58 STEP 2368 H=2368:00 +CPU= 24.306
  23:38:22 STEP 2369 H=2369:00 +CPU= 23.491
  23:38:59 STEP 2370 H=2370:00 +CPU= 36.435
  23:39:21 STEP 2371 H=2371:00 +CPU= 21.762
  23:39:44 STEP 2372 H=2372:00 +CPU= 22.871
  23:40:20 STEP 2373 H=2373:00 +CPU= 35.532
  23:40:45 STEP 2374 H=2374:00 +CPU= 24.397
  23:41:09 STEP 2375 H=2375:00 +CPU= 24.457
  23:41:54 STEP 2376 H=2376:00 +CPU= 44.001
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_98.zip
Uploading the intermediate file: upload_file_98.zip
  23:42:18 STEP 2377 H=2377:00 +CPU= 23.610
  23:42:43 STEP 2378 H=2378:00 +CPU= 24.568
  23:43:21 STEP 2379 H=2379:00 +CPU= 37.028
  23:43:45 STEP 2380 H=2380:00 +CPU= 24.333
  23:44:09 STEP 2381 H=2381:00 +CPU= 23.581
  23:44:47 STEP 2382 H=2382:00 +CPU= 37.535
  23:45:11 STEP 2383 H=2383:00 +CPU= 23.873
  23:45:36 STEP 2384 H=2384:00 +CPU= 24.886
  23:46:13 STEP 2385 H=2385:00 +CPU= 36.334
  23:46:35 STEP 2386 H=2386:00 +CPU= 22.005
  23:46:57 STEP 2387 H=2387:00 +CPU= 22.307
  23:47:36 STEP 2388 H=2388:00 +CPU= 38.894
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002388
  23:48:00 STEP 2389 H=2389:00 +CPU= 23.303
  23:48:25 STEP 2390 H=2390:00 +CPU= 25.030
  23:49:02 STEP 2391 H=2391:00 +CPU= 36.810
  23:49:28 STEP 2392 H=2392:00 +CPU= 24.971
  23:49:51 STEP 2393 H=2393:00 +CPU= 23.688
  23:50:30 STEP 2394 H=2394:00 +CPU= 37.989
  23:50:54 STEP 2395 H=2395:00 +CPU= 23.768
  23:51:18 STEP 2396 H=2396:00 +CPU= 24.526
  23:51:57 STEP 2397 H=2397:00 +CPU= 37.686
  23:52:21 STEP 2398 H=2398:00 +CPU= 24.367
  23:52:45 STEP 2399 H=2399:00 +CPU= 22.833
  23:53:27 STEP 2400 H=2400:00 +CPU= 41.745
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_99.zip
Uploading the intermediate file: upload_file_99.zip
  23:53:48 STEP 2401 H=2401:00 +CPU= 21.222
  23:54:10 STEP 2402 H=2402:00 +CPU= 21.960
  23:54:46 STEP 2403 H=2403:00 +CPU= 35.486
  23:55:11 STEP 2404 H=2404:00 +CPU= 24.764
  23:55:35 STEP 2405 H=2405:00 +CPU= 23.967
  23:56:13 STEP 2406 H=2406:00 +CPU= 37.654
  23:56:36 STEP 2407 H=2407:00 +CPU= 23.481
  23:57:01 STEP 2408 H=2408:00 +CPU= 24.472
  23:57:38 STEP 2409 H=2409:00 +CPU= 36.619
  23:58:02 STEP 2410 H=2410:00 +CPU= 24.643
  23:58:26 STEP 2411 H=2411:00 +CPU= 23.896
  23:59:07 STEP 2412 H=2412:00 +CPU= 40.691
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002412
  23:59:30 STEP 2413 H=2413:00 +CPU= 23.000
  23:59:52 STEP 2414 H=2414:00 +CPU= 21.895
  00:00:27 STEP 2415 H=2415:00 +CPU= 34.415
  00:00:50 STEP 2416 H=2416:00 +CPU= 23.175
  00:01:14 STEP 2417 H=2417:00 +CPU= 23.669
  00:01:52 STEP 2418 H=2418:00 +CPU= 37.780
  00:02:16 STEP 2419 H=2419:00 +CPU= 23.709
  00:02:40 STEP 2420 H=2420:00 +CPU= 24.212
  00:03:17 STEP 2421 H=2421:00 +CPU= 36.933
  00:03:42 STEP 2422 H=2422:00 +CPU= 24.373
  00:04:07 STEP 2423 H=2423:00 +CPU= 24.769
  00:04:52 STEP 2424 H=2424:00 +CPU= 44.221
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_100.zip
Uploading the intermediate file: upload_file_100.zip
  00:05:16 STEP 2425 H=2425:00 +CPU= 23.575
  00:05:41 STEP 2426 H=2426:00 +CPU= 24.567
  00:06:18 STEP 2427 H=2427:00 +CPU= 37.294
  00:06:43 STEP 2428 H=2428:00 +CPU= 24.452
  00:07:07 STEP 2429 H=2429:00 +CPU= 23.349
  00:07:41 STEP 2430 H=2430:00 +CPU= 34.371
  00:08:04 STEP 2431 H=2431:00 +CPU= 22.156
  00:08:27 STEP 2432 H=2432:00 +CPU= 23.088
  00:09:04 STEP 2433 H=2433:00 +CPU= 36.835
  00:09:29 STEP 2434 H=2434:00 +CPU= 24.691
  00:09:54 STEP 2435 H=2435:00 +CPU= 23.661
  00:10:37 STEP 2436 H=2436:00 +CPU= 41.559
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002436
  00:11:01 STEP 2437 H=2437:00 +CPU= 24.069
  00:11:26 STEP 2438 H=2438:00 +CPU= 24.647
  00:12:03 STEP 2439 H=2439:00 +CPU= 36.820
  00:12:28 STEP 2440 H=2440:00 +CPU= 24.780
  00:12:52 STEP 2441 H=2441:00 +CPU= 23.616
  00:13:31 STEP 2442 H=2442:00 +CPU= 38.264
  00:13:54 STEP 2443 H=2443:00 +CPU= 23.547
  00:14:19 STEP 2444 H=2444:00 +CPU= 23.860
  00:14:52 STEP 2445 H=2445:00 +CPU= 33.280
  00:15:16 STEP 2446 H=2446:00 +CPU= 23.079
  00:15:40 STEP 2447 H=2447:00 +CPU= 23.473
  00:16:25 STEP 2448 H=2448:00 +CPU= 44.533
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_101.zip
Uploading the intermediate file: upload_file_101.zip
  00:16:49 STEP 2449 H=2449:00 +CPU= 23.740
  00:17:14 STEP 2450 H=2450:00 +CPU= 24.545
  00:17:51 STEP 2451 H=2451:00 +CPU= 36.938
  00:18:16 STEP 2452 H=2452:00 +CPU= 24.447
  00:18:39 STEP 2453 H=2453:00 +CPU= 23.535
  00:19:18 STEP 2454 H=2454:00 +CPU= 38.369
  00:19:42 STEP 2455 H=2455:00 +CPU= 23.694
  00:20:06 STEP 2456 H=2456:00 +CPU= 24.316
  00:20:40 STEP 2457 H=2457:00 +CPU= 33.161
  00:21:03 STEP 2458 H=2458:00 +CPU= 22.967
  00:21:26 STEP 2459 H=2459:00 +CPU= 22.740
  00:22:07 STEP 2460 H=2460:00 +CPU= 40.666
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002460
  00:22:30 STEP 2461 H=2461:00 +CPU= 22.963
  00:22:53 STEP 2462 H=2462:00 +CPU= 23.521
  00:23:30 STEP 2463 H=2463:00 +CPU= 36.199
  00:23:54 STEP 2464 H=2464:00 +CPU= 24.449
  00:24:18 STEP 2465 H=2465:00 +CPU= 23.727
  00:24:56 STEP 2466 H=2466:00 +CPU= 37.923
  00:25:20 STEP 2467 H=2467:00 +CPU= 23.634
  00:25:45 STEP 2468 H=2468:00 +CPU= 24.488
  00:26:19 STEP 2469 H=2469:00 +CPU= 33.994
  00:26:43 STEP 2470 H=2470:00 +CPU= 23.280
  00:27:07 STEP 2471 H=2471:00 +CPU= 23.896
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1998050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0180
(argv4) batchid: 967
(argv5) wuid: 12183824
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c4207c6d9721b1189e4da373f0124007 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0180_1998050100_123_967_12183824.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0180_1998050100_123_967_12183824.zip
ic_ancil_file: ic_ancil_12183824
ifsdata_file: ifsdata_12183824
climate_data_file: clim_data_12183824
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_cf7ab91c23ba43d73f16dc08356f78a6 to: /var/lib/boinc-client/slots/8/ic_ancil_12183824.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12183824.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c2416c7c3839a8e76007e592492566f4 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12183824.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12183824.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12183824.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12183824.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12183824.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12183824.xml
-- Model is restarting --
Adjusting last_iter, 2471, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12183824.xml
last_cpu_time: 76824
upload_file_number: 102
last_iter: 2447
last_upload: 8812800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445
The child process has been launched with process id: 126128
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:bart-MintII:1:1:126128:126128] [20221226:002818:1672010898:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  00:29:04 STEP 2472 H=2472:00 +CPU= 42.432
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_102.zip
Uploading the intermediate file: upload_file_102.zip
  00:29:26 STEP 2473 H=2473:00 +CPU= 22.107
  00:29:50 STEP 2474 H=2474:00 +CPU= 24.031
  00:30:25 STEP 2475 H=2475:00 +CPU= 34.928
  00:30:48 STEP 2476 H=2476:00 +CPU= 22.371
  00:31:07 STEP 2477 H=2477:00 +CPU= 19.515
  00:31:42 STEP 2478 H=2478:00 +CPU= 34.995
  00:32:05 STEP 2479 H=2479:00 +CPU= 22.220
  00:32:29 STEP 2480 H=2480:00 +CPU= 24.451
  00:33:06 STEP 2481 H=2481:00 +CPU= 36.918
  00:33:31 STEP 2482 H=2482:00 +CPU= 24.774
  00:33:55 STEP 2483 H=2483:00 +CPU= 23.532
  00:34:36 STEP 2484 H=2484:00 +CPU= 40.858
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002484
  00:34:59 STEP 2485 H=2485:00 +CPU= 23.403
  00:35:24 STEP 2486 H=2486:00 +CPU= 24.377
  00:36:01 STEP 2487 H=2487:00 +CPU= 36.684
  00:36:25 STEP 2488 H=2488:00 +CPU= 24.074
  00:36:48 STEP 2489 H=2489:00 +CPU= 23.299
  00:37:26 STEP 2490 H=2490:00 +CPU= 37.378
  00:37:49 STEP 2491 H=2491:00 +CPU= 23.445
  00:38:13 STEP 2492 H=2492:00 +CPU= 23.509
  00:38:47 STEP 2493 H=2493:00 +CPU= 33.453
  00:39:10 STEP 2494 H=2494:00 +CPU= 22.730
  00:39:33 STEP 2495 H=2495:00 +CPU= 23.183
  00:40:17 STEP 2496 H=2496:00 +CPU= 43.121
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183824/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0180_1998050100_123_967_12183824_0_r2093716445_103.zip
Uploading the intermediate file: upload_file_103.zip
  00:40:41 STEP 2497 H=2497:00 +CPU= 23.492
  00:41:05 STEP 2498 H=2498:00 +CPU= 24.089
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:55:42 1509857 22269019 oifs_43r3_ps_0180_1998050100_123_967_12183824_0 8,035,200 69,529 0.0087
09 May 2023 15:55:31 1509857 22269019 oifs_43r3_ps_0180_1998050100_123_967_12183824_0 7,171,200 62,372 0.0087
09 May 2023 15:55:20 1509857 22269019 oifs_43r3_ps_0180_1998050100_123_967_12183824_0 6,307,200 55,339 0.0088
09 May 2023 15:55:09 1509857 22269019 oifs_43r3_ps_0180_1998050100_123_967_12183824_0 5,443,200 48,176 0.0089
09 May 2023 15:54:58 1509857 22269019 oifs_43r3_ps_0180_1998050100_123_967_12183824_0 4,579,200 40,998 0.0090
09 May 2023 15:54:46 1509857 22269019 oifs_43r3_ps_0180_1998050100_123_967_12183824_0 3,715,200 33,667 0.0091
09 May 2023 15:53:59 1509857 22269019 oifs_43r3_ps_0180_1998050100_123_967_12183824_0 864,000 7,651 0.0089


©2024 climateprediction.net