climateprediction.net home page
Task 22290089

Task 22290089

Name oifs_43r3_ps_0647_2010050100_123_979_12196291_1
Workunit 12196291
Created 24 Dec 2022, 19:26:00 UTC
Sent 19 Jan 2023, 16:27:29 UTC
Report deadline 20 Mar 2023, 16:27:29 UTC
Received 31 Jan 2023, 8:10:36 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1478188
Run time 1 days 3 hours 10 min 42 sec
CPU time 13 hours 28 min 56 sec
Validate state Invalid
Credit 12,700.80
Device peak FLOPS 4.20 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,413.54 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,572.17 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
002172
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002172
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:41:01 STEP 2173 H=2173:00 +CPU= 24.900
  10:41:27 STEP 2174 H=2174:00 +CPU= 25.413
  10:42:10 STEP 2175 H=2175:00 +CPU= 42.171
  10:42:38 STEP 2176 H=2176:00 +CPU= 27.523
  10:43:02 STEP 2177 H=2177:00 +CPU= 23.944
  10:43:46 STEP 2178 H=2178:00 +CPU= 42.698
  10:44:08 STEP 2179 H=2179:00 +CPU= 21.579
  10:44:31 STEP 2180 H=2180:00 +CPU= 22.640
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:49:47 STEP 2181 H=2181:00 +CPU= 43.407
  10:50:13 STEP 2182 H=2182:00 +CPU= 25.488
  10:50:39 STEP 2183 H=2183:00 +CPU= 25.560
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:55:33 STEP 2184 H=2184:00 +CPU= 50.447
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002184
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002184
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002184
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002160
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002160
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002160
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002172
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002172
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002172
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_37.zip
Uploading the intermediate file: upload_file_37.zip
  10:55:59 STEP 2185 H=2185:00 +CPU= 25.344
  10:56:22 STEP 2186 H=2186:00 +CPU= 22.239
  10:57:04 STEP 2187 H=2187:00 +CPU= 41.481
  10:57:30 STEP 2188 H=2188:00 +CPU= 25.865
  10:57:54 STEP 2189 H=2189:00 +CPU= 23.376
  10:58:31 STEP 2190 H=2190:00 +CPU= 36.337
  10:58:57 STEP 2191 H=2191:00 +CPU= 25.655
  10:59:24 STEP 2192 H=2192:00 +CPU= 26.467
  11:00:03 STEP 2193 H=2193:00 +CPU= 37.578
  11:00:29 STEP 2194 H=2194:00 +CPU= 25.876
  11:00:51 STEP 2195 H=2195:00 +CPU= 21.884
  11:01:36 STEP 2196 H=2196:00 +CPU= 43.847
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002196
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002196
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002196
  11:01:59 STEP 2197 H=2197:00 +CPU= 22.005
  11:02:24 STEP 2198 H=2198:00 +CPU= 25.121
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:18:07 STEP 2199 H=2199:00 +CPU= 37.993
  11:18:31 STEP 2200 H=2200:00 +CPU= 23.394
  11:18:58 STEP 2201 H=2201:00 +CPU= 25.800
  11:19:40 STEP 2202 H=2202:00 +CPU= 39.526
  11:20:07 STEP 2203 H=2203:00 +CPU= 26.444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:23:11 STEP 2204 H=2204:00 +CPU= 29.147
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:59:31 STEP 2205 H=2205:00 +CPU= 45.936
  11:59:58 STEP 2206 H=2206:00 +CPU= 27.115
  12:00:25 STEP 2207 H=2207:00 +CPU= 26.059
  12:01:16 STEP 2208 H=2208:00 +CPU= 49.420
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002208
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002208
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002208
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002184
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002184
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002184
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002196
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002196
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002196
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_38.zip
Uploading the intermediate file: upload_file_38.zip
  12:01:43 STEP 2209 H=2209:00 +CPU= 26.526
  12:02:09 STEP 2210 H=2210:00 +CPU= 25.237
  12:02:51 STEP 2211 H=2211:00 +CPU= 41.211
  12:03:18 STEP 2212 H=2212:00 +CPU= 26.178
  12:03:46 STEP 2213 H=2213:00 +CPU= 27.586
  12:04:29 STEP 2214 H=2214:00 +CPU= 41.812
  12:04:54 STEP 2215 H=2215:00 +CPU= 24.621
  12:05:19 STEP 2216 H=2216:00 +CPU= 24.131
  12:06:00 STEP 2217 H=2217:00 +CPU= 39.654
  12:06:30 STEP 2218 H=2218:00 +CPU= 29.100
  12:06:56 STEP 2219 H=2219:00 +CPU= 25.896
  12:07:45 STEP 2220 H=2220:00 +CPU= 47.614
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002220
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002220
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002220
  12:08:14 STEP 2221 H=2221:00 +CPU= 28.373
  12:08:41 STEP 2222 H=2222:00 +CPU= 26.450
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:11:27 STEP 2223 H=2223:00 +CPU= 43.229
  12:11:55 STEP 2224 H=2224:00 +CPU= 27.469
  12:12:22 STEP 2225 H=2225:00 +CPU= 27.169
  12:13:09 STEP 2226 H=2226:00 +CPU= 45.960
  12:13:34 STEP 2227 H=2227:00 +CPU= 23.856
  12:14:05 STEP 2228 H=2228:00 +CPU= 29.232
  12:14:47 STEP 2229 H=2229:00 +CPU= 41.623
  12:15:14 STEP 2230 H=2230:00 +CPU= 25.896
  12:15:43 STEP 2231 H=2231:00 +CPU= 28.288
  12:16:36 STEP 2232 H=2232:00 +CPU= 51.943
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002232
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002232
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002232
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002208
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002208
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002208
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002220
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002220
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002220
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_39.zip
Uploading the intermediate file: upload_file_39.zip
  12:17:03 STEP 2233 H=2233:00 +CPU= 26.544
  12:17:31 STEP 2234 H=2234:00 +CPU= 26.708
  12:18:17 STEP 2235 H=2235:00 +CPU= 45.122
  12:18:45 STEP 2236 H=2236:00 +CPU= 27.510
  12:19:12 STEP 2237 H=2237:00 +CPU= 26.151
  12:19:53 STEP 2238 H=2238:00 +CPU= 40.609
  12:20:17 STEP 2239 H=2239:00 +CPU= 23.754
  12:20:44 STEP 2240 H=2240:00 +CPU= 26.014
  12:21:28 STEP 2241 H=2241:00 +CPU= 43.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:23:55 STEP 2242 H=2242:00 +CPU= 26.316
  12:24:20 STEP 2243 H=2243:00 +CPU= 24.567
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:31:52 STEP 2244 H=2244:00 +CPU= 43.678
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002244
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002244
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002244
  12:32:23 STEP 2245 H=2245:00 +CPU= 28.846
  12:32:57 STEP 2246 H=2246:00 +CPU= 31.380
  12:33:49 STEP 2247 H=2247:00 +CPU= 48.815
  12:34:23 STEP 2248 H=2248:00 +CPU= 30.870
  12:34:56 STEP 2249 H=2249:00 +CPU= 30.946
  12:35:47 STEP 2250 H=2250:00 +CPU= 46.994
  12:36:19 STEP 2251 H=2251:00 +CPU= 29.301
  12:36:48 STEP 2252 H=2252:00 +CPU= 27.371
  12:37:41 STEP 2253 H=2253:00 +CPU= 48.389
  12:38:14 STEP 2254 H=2254:00 +CPU= 31.112
  12:38:45 STEP 2255 H=2255:00 +CPU= 28.883
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:52:39 STEP 2256 H=2256:00 +CPU= 55.420
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002256
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002256
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002256
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002232
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002232
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002232
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002244
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002244
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002244
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_40.zip
Uploading the intermediate file: upload_file_40.zip
  12:53:08 STEP 2257 H=2257:00 +CPU= 28.332
  12:53:37 STEP 2258 H=2258:00 +CPU= 28.766
  12:54:19 STEP 2259 H=2259:00 +CPU= 40.051
  12:54:48 STEP 2260 H=2260:00 +CPU= 28.421
  12:55:16 STEP 2261 H=2261:00 +CPU= 26.877
  12:55:59 STEP 2262 H=2262:00 +CPU= 42.667
  12:56:27 STEP 2263 H=2263:00 +CPU= 26.773
  12:56:56 STEP 2264 H=2264:00 +CPU= 28.218
  12:57:40 STEP 2265 H=2265:00 +CPU= 43.209
  12:58:06 STEP 2266 H=2266:00 +CPU= 25.621
  12:58:34 STEP 2267 H=2267:00 +CPU= 27.588
  12:59:20 STEP 2268 H=2268:00 +CPU= 44.296
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002268
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002268
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002268
  12:59:47 STEP 2269 H=2269:00 +CPU= 26.895
  13:00:16 STEP 2270 H=2270:00 +CPU= 27.639
  13:00:58 STEP 2271 H=2271:00 +CPU= 41.534
  13:01:26 STEP 2272 H=2272:00 +CPU= 26.981
  13:01:55 STEP 2273 H=2273:00 +CPU= 28.655
  13:02:45 STEP 2274 H=2274:00 +CPU= 49.917
  13:03:16 STEP 2275 H=2275:00 +CPU= 29.636
  13:03:47 STEP 2276 H=2276:00 +CPU= 30.554
  13:04:35 STEP 2277 H=2277:00 +CPU= 47.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:40 STEP 2278 H=2278:00 +CPU= 28.548
  14:05:08 STEP 2279 H=2279:00 +CPU= 27.312
  14:06:02 STEP 2280 H=2280:00 +CPU= 53.551
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002280
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002280
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002280
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002256
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002256
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002256
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002268
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002268
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002268
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_41.zip
Uploading the intermediate file: upload_file_41.zip
  14:06:32 STEP 2281 H=2281:00 +CPU= 29.638
Uploading trickle at timestep: 8208000
  14:07:02 STEP 2282 H=2282:00 +CPU= 29.378
  14:07:50 STEP 2283 H=2283:00 +CPU= 47.614
  14:08:21 STEP 2284 H=2284:00 +CPU= 30.794
  14:08:52 STEP 2285 H=2285:00 +CPU= 30.413
  14:09:41 STEP 2286 H=2286:00 +CPU= 47.902
  14:10:11 STEP 2287 H=2287:00 +CPU= 29.903
  14:10:42 STEP 2288 H=2288:00 +CPU= 30.749
  14:11:31 STEP 2289 H=2289:00 +CPU= 48.450
  14:12:02 STEP 2290 H=2290:00 +CPU= 29.859
  14:12:32 STEP 2291 H=2291:00 +CPU= 29.710
  14:13:25 STEP 2292 H=2292:00 +CPU= 52.172
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002292
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002292
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002292
  14:13:55 STEP 2293 H=2293:00 +CPU= 29.697
  14:14:25 STEP 2294 H=2294:00 +CPU= 29.370
  14:15:13 STEP 2295 H=2295:00 +CPU= 47.723
  14:15:44 STEP 2296 H=2296:00 +CPU= 30.729
  14:16:13 STEP 2297 H=2297:00 +CPU= 28.576
  14:17:03 STEP 2298 H=2298:00 +CPU= 48.912
  14:17:33 STEP 2299 H=2299:00 +CPU= 29.737
  14:18:03 STEP 2300 H=2300:00 +CPU= 29.414
  14:18:50 STEP 2301 H=2301:00 +CPU= 47.079
  14:19:21 STEP 2302 H=2302:00 +CPU= 29.966
  14:19:52 STEP 2303 H=2303:00 +CPU= 30.554
  14:20:48 STEP 2304 H=2304:00 +CPU= 55.773
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002304
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002304
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002304
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002280
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002280
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002280
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002292
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002292
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002292
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_42.zip
Uploading the intermediate file: upload_file_42.zip
  14:21:19 STEP 2305 H=2305:00 +CPU= 30.155
  14:21:50 STEP 2306 H=2306:00 +CPU= 30.945
  14:22:39 STEP 2307 H=2307:00 +CPU= 47.845
  14:23:09 STEP 2308 H=2308:00 +CPU= 29.827
  14:23:37 STEP 2309 H=2309:00 +CPU= 28.369
  14:24:27 STEP 2310 H=2310:00 +CPU= 48.131
  14:24:56 STEP 2311 H=2311:00 +CPU= 29.018
  14:25:27 STEP 2312 H=2312:00 +CPU= 30.221
  14:26:14 STEP 2313 H=2313:00 +CPU= 46.668
  14:26:45 STEP 2314 H=2314:00 +CPU= 30.841
  14:27:15 STEP 2315 H=2315:00 +CPU= 28.881
  14:28:08 STEP 2316 H=2316:00 +CPU= 52.571
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002316
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002316
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002316
  14:28:36 STEP 2317 H=2317:00 +CPU= 27.600
  14:29:06 STEP 2318 H=2318:00 +CPU= 29.728
  14:29:53 STEP 2319 H=2319:00 +CPU= 45.955
  14:30:24 STEP 2320 H=2320:00 +CPU= 30.533
  14:30:53 STEP 2321 H=2321:00 +CPU= 29.247
  14:31:41 STEP 2322 H=2322:00 +CPU= 47.030
  14:32:11 STEP 2323 H=2323:00 +CPU= 29.342
  14:32:41 STEP 2324 H=2324:00 +CPU= 29.426
  14:33:29 STEP 2325 H=2325:00 +CPU= 47.659
  14:34:00 STEP 2326 H=2326:00 +CPU= 30.363
  14:34:30 STEP 2327 H=2327:00 +CPU= 29.991
  14:35:26 STEP 2328 H=2328:00 +CPU= 55.424
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002328
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002328
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002328
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002304
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002304
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002304
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002316
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002316
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002316
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_43.zip
Uploading the intermediate file: upload_file_43.zip
  14:35:56 STEP 2329 H=2329:00 +CPU= 29.064
  14:36:27 STEP 2330 H=2330:00 +CPU= 30.932
  14:37:14 STEP 2331 H=2331:00 +CPU= 45.892
  14:37:44 STEP 2332 H=2332:00 +CPU= 29.527
  14:38:13 STEP 2333 H=2333:00 +CPU= 29.140
  14:39:01 STEP 2334 H=2334:00 +CPU= 47.688
  14:39:31 STEP 2335 H=2335:00 +CPU= 28.861
  14:40:02 STEP 2336 H=2336:00 +CPU= 30.636
  14:40:50 STEP 2337 H=2337:00 +CPU= 47.153
  14:41:19 STEP 2338 H=2338:00 +CPU= 29.390
  14:41:48 STEP 2339 H=2339:00 +CPU= 28.183
  14:42:41 STEP 2340 H=2340:00 +CPU= 52.150
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002340
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002340
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002340
  14:43:11 STEP 2341 H=2341:00 +CPU= 29.256
  14:43:41 STEP 2342 H=2342:00 +CPU= 30.224
  14:44:29 STEP 2343 H=2343:00 +CPU= 47.522
  14:44:59 STEP 2344 H=2344:00 +CPU= 29.240
  14:45:29 STEP 2345 H=2345:00 +CPU= 29.667
  14:46:17 STEP 2346 H=2346:00 +CPU= 47.153
  14:46:46 STEP 2347 H=2347:00 +CPU= 28.867
  14:47:16 STEP 2348 H=2348:00 +CPU= 29.436
  14:48:03 STEP 2349 H=2349:00 +CPU= 46.396
  14:48:34 STEP 2350 H=2350:00 +CPU= 30.631
  14:49:04 STEP 2351 H=2351:00 +CPU= 29.089
  14:50:00 STEP 2352 H=2352:00 +CPU= 55.366
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002352
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002352
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002328
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002328
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002328
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002340
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002340
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002340
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_44.zip
Uploading the intermediate file: upload_file_44.zip
  14:50:30 STEP 2353 H=2353:00 +CPU= 29.645
  14:51:00 STEP 2354 H=2354:00 +CPU= 30.264
  14:51:48 STEP 2355 H=2355:00 +CPU= 47.496
  14:52:19 STEP 2356 H=2356:00 +CPU= 30.012
  14:52:48 STEP 2357 H=2357:00 +CPU= 28.657
  14:53:35 STEP 2358 H=2358:00 +CPU= 46.902
  14:54:04 STEP 2359 H=2359:00 +CPU= 28.121
  14:54:34 STEP 2360 H=2360:00 +CPU= 29.477
  14:55:21 STEP 2361 H=2361:00 +CPU= 46.525
  14:55:51 STEP 2362 H=2362:00 +CPU= 29.888
  14:56:20 STEP 2363 H=2363:00 +CPU= 28.441
  14:57:12 STEP 2364 H=2364:00 +CPU= 51.001
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002364
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002364
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002364
  14:57:41 STEP 2365 H=2365:00 +CPU= 28.911
  14:58:12 STEP 2366 H=2366:00 +CPU= 30.136
  14:58:58 STEP 2367 H=2367:00 +CPU= 45.493
  14:59:27 STEP 2368 H=2368:00 +CPU= 28.858
  14:59:55 STEP 2369 H=2369:00 +CPU= 28.196
  15:00:42 STEP 2370 H=2370:00 +CPU= 45.975
  15:01:10 STEP 2371 H=2371:00 +CPU= 27.592
  15:01:40 STEP 2372 H=2372:00 +CPU= 29.737
  15:02:24 STEP 2373 H=2373:00 +CPU= 43.366
  15:02:53 STEP 2374 H=2374:00 +CPU= 27.997
  15:03:21 STEP 2375 H=2375:00 +CPU= 27.825
  15:04:17 STEP 2376 H=2376:00 +CPU= 54.293
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002376
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002376
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002364
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002364
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002364
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_45.zip
Uploading the intermediate file: upload_file_45.zip
  15:04:44 STEP 2377 H=2377:00 +CPU= 27.296
  15:05:14 STEP 2378 H=2378:00 +CPU= 28.907
  15:05:59 STEP 2379 H=2379:00 +CPU= 44.575
  15:06:28 STEP 2380 H=2380:00 +CPU= 28.660
  15:06:57 STEP 2381 H=2381:00 +CPU= 28.567
  15:07:45 STEP 2382 H=2382:00 +CPU= 46.975
  15:08:14 STEP 2383 H=2383:00 +CPU= 29.008
  15:08:44 STEP 2384 H=2384:00 +CPU= 30.145
  15:09:32 STEP 2385 H=2385:00 +CPU= 46.670
  15:10:02 STEP 2386 H=2386:00 +CPU= 30.143
  15:10:33 STEP 2387 H=2387:00 +CPU= 30.007
  15:11:26 STEP 2388 H=2388:00 +CPU= 52.177
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002388
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002388
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002388
  15:11:56 STEP 2389 H=2389:00 +CPU= 30.335
  15:12:28 STEP 2390 H=2390:00 +CPU= 31.089
  15:13:17 STEP 2391 H=2391:00 +CPU= 48.140
  15:13:48 STEP 2392 H=2392:00 +CPU= 30.830
  15:14:18 STEP 2393 H=2393:00 +CPU= 29.375
  15:15:07 STEP 2394 H=2394:00 +CPU= 48.612
  15:15:36 STEP 2395 H=2395:00 +CPU= 28.322
  15:16:07 STEP 2396 H=2396:00 +CPU= 30.463
  15:16:57 STEP 2397 H=2397:00 +CPU= 49.148
  15:17:28 STEP 2398 H=2398:00 +CPU= 30.717
  15:17:59 STEP 2399 H=2399:00 +CPU= 30.665
  15:18:51 STEP 2400 H=2400:00 +CPU= 51.736
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002400
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002400
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002388
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002388
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002388
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_46.zip
Uploading the intermediate file: upload_file_46.zip
  15:19:19 STEP 2401 H=2401:00 +CPU= 27.523
  15:19:47 STEP 2402 H=2402:00 +CPU= 26.988
  15:20:30 STEP 2403 H=2403:00 +CPU= 42.982
  15:20:58 STEP 2404 H=2404:00 +CPU= 27.268
  15:21:22 STEP 2405 H=2405:00 +CPU= 24.517
  15:22:05 STEP 2406 H=2406:00 +CPU= 42.065
  15:22:30 STEP 2407 H=2407:00 +CPU= 24.475
  15:22:58 STEP 2408 H=2408:00 +CPU= 27.522
  15:23:39 STEP 2409 H=2409:00 +CPU= 41.251
  15:24:05 STEP 2410 H=2410:00 +CPU= 25.630
  15:24:31 STEP 2411 H=2411:00 +CPU= 25.083
  15:25:16 STEP 2412 H=2412:00 +CPU= 44.457
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002412
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002412
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002412
  15:25:42 STEP 2413 H=2413:00 +CPU= 26.121
  15:26:08 STEP 2414 H=2414:00 +CPU= 25.953
  15:26:49 STEP 2415 H=2415:00 +CPU= 39.942
  15:27:13 STEP 2416 H=2416:00 +CPU= 24.394
  15:27:37 STEP 2417 H=2417:00 +CPU= 23.190
  15:28:15 STEP 2418 H=2418:00 +CPU= 37.905
  15:28:37 STEP 2419 H=2419:00 +CPU= 21.435
  15:28:58 STEP 2420 H=2420:00 +CPU= 21.230
  15:29:33 STEP 2421 H=2421:00 +CPU= 34.302
  15:29:55 STEP 2422 H=2422:00 +CPU= 22.180
  15:30:18 STEP 2423 H=2423:00 +CPU= 22.092
  15:30:59 STEP 2424 H=2424:00 +CPU= 40.468
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002424
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002424
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002412
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002412
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002412
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_47.zip
Uploading the intermediate file: upload_file_47.zip
  15:31:20 STEP 2425 H=2425:00 +CPU= 21.626
  15:31:43 STEP 2426 H=2426:00 +CPU= 21.881
  15:32:17 STEP 2427 H=2427:00 +CPU= 34.566
  15:32:40 STEP 2428 H=2428:00 +CPU= 22.010
  15:33:01 STEP 2429 H=2429:00 +CPU= 21.144
  15:33:37 STEP 2430 H=2430:00 +CPU= 35.219
  15:33:59 STEP 2431 H=2431:00 +CPU= 21.855
  15:34:22 STEP 2432 H=2432:00 +CPU= 22.633
  15:34:58 STEP 2433 H=2433:00 +CPU= 35.454
  15:35:20 STEP 2434 H=2434:00 +CPU= 21.877
  15:35:42 STEP 2435 H=2435:00 +CPU= 20.832
  15:36:22 STEP 2436 H=2436:00 +CPU= 39.382
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002436
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002436
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002436
  15:36:44 STEP 2437 H=2437:00 +CPU= 21.956
  15:37:07 STEP 2438 H=2438:00 +CPU= 22.160
  15:37:41 STEP 2439 H=2439:00 +CPU= 33.686
  15:38:03 STEP 2440 H=2440:00 +CPU= 21.804
  15:38:24 STEP 2441 H=2441:00 +CPU= 21.003
  15:39:00 STEP 2442 H=2442:00 +CPU= 35.379
  15:39:21 STEP 2443 H=2443:00 +CPU= 21.269
  15:39:43 STEP 2444 H=2444:00 +CPU= 21.784
  15:40:18 STEP 2445 H=2445:00 +CPU= 34.375
  15:40:40 STEP 2446 H=2446:00 +CPU= 21.939
  15:41:02 STEP 2447 H=2447:00 +CPU= 21.701
  15:41:58 STEP 2448 H=2448:00 +CPU= 54.921
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002448
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002448
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002436
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002436
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002436
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_48.zip
Uploading the intermediate file: upload_file_48.zip
  15:42:26 STEP 2449 H=2449:00 +CPU= 27.990
  15:42:57 STEP 2450 H=2450:00 +CPU= 30.173
  15:43:44 STEP 2451 H=2451:00 +CPU= 45.943
  15:44:14 STEP 2452 H=2452:00 +CPU= 29.298
  15:44:42 STEP 2453 H=2453:00 +CPU= 28.289
  15:45:30 STEP 2454 H=2454:00 +CPU= 47.586
  15:45:59 STEP 2455 H=2455:00 +CPU= 28.139
  15:46:29 STEP 2456 H=2456:00 +CPU= 29.606
  15:47:16 STEP 2457 H=2457:00 +CPU= 47.043
  15:47:47 STEP 2458 H=2458:00 +CPU= 30.557
  15:48:16 STEP 2459 H=2459:00 +CPU= 28.338
  15:49:09 STEP 2460 H=2460:00 +CPU= 51.930
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002460
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002460
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002460
  15:49:38 STEP 2461 H=2461:00 +CPU= 29.377
  15:50:10 STEP 2462 H=2462:00 +CPU= 31.109
  15:50:58 STEP 2463 H=2463:00 +CPU= 47.138
  15:51:28 STEP 2464 H=2464:00 +CPU= 30.295
  15:51:59 STEP 2465 H=2465:00 +CPU= 30.152
  15:52:48 STEP 2466 H=2466:00 +CPU= 48.718
  15:53:18 STEP 2467 H=2467:00 +CPU= 29.058
  15:53:49 STEP 2468 H=2468:00 +CPU= 31.309
  15:54:38 STEP 2469 H=2469:00 +CPU= 48.073
  15:55:09 STEP 2470 H=2470:00 +CPU= 30.382
  15:55:40 STEP 2471 H=2471:00 +CPU= 31.058
  15:56:36 STEP 2472 H=2472:00 +CPU= 55.274
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002472
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002472
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002460
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002460
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002460
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_49.zip
Uploading the intermediate file: upload_file_49.zip
  15:57:06 STEP 2473 H=2473:00 +CPU= 29.013
  15:57:36 STEP 2474 H=2474:00 +CPU= 29.523
  15:58:24 STEP 2475 H=2475:00 +CPU= 47.255
  15:58:54 STEP 2476 H=2476:00 +CPU= 30.156
  15:59:24 STEP 2477 H=2477:00 +CPU= 29.645
  16:00:13 STEP 2478 H=2478:00 +CPU= 47.892
  16:00:41 STEP 2479 H=2479:00 +CPU= 28.368
  16:01:12 STEP 2480 H=2480:00 +CPU= 30.104
  16:02:00 STEP 2481 H=2481:00 +CPU= 47.227
  16:02:30 STEP 2482 H=2482:00 +CPU= 29.843
  16:02:59 STEP 2483 H=2483:00 +CPU= 28.567
  16:03:51 STEP 2484 H=2484:00 +CPU= 51.686
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002484
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002484
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002484
  16:04:21 STEP 2485 H=2485:00 +CPU= 28.997
  16:04:50 STEP 2486 H=2486:00 +CPU= 29.090
  16:05:38 STEP 2487 H=2487:00 +CPU= 47.074
  16:06:08 STEP 2488 H=2488:00 +CPU= 29.643
  16:06:38 STEP 2489 H=2489:00 +CPU= 29.328
  16:07:26 STEP 2490 H=2490:00 +CPU= 47.877
  16:07:56 STEP 2491 H=2491:00 +CPU= 29.608
  16:08:26 STEP 2492 H=2492:00 +CPU= 29.405
  16:09:15 STEP 2493 H=2493:00 +CPU= 48.117
  16:09:45 STEP 2494 H=2494:00 +CPU= 30.071
  16:10:16 STEP 2495 H=2495:00 +CPU= 30.173
  16:11:10 STEP 2496 H=2496:00 +CPU= 53.846
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002496
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002496
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002484
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002484
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002484
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_50.zip
Uploading the intermediate file: upload_file_50.zip
  16:11:39 STEP 2497 H=2497:00 +CPU= 28.213
  16:12:10 STEP 2498 H=2498:00 +CPU= 30.735
  16:12:59 STEP 2499 H=2499:00 +CPU= 48.468
  16:13:30 STEP 2500 H=2500:00 +CPU= 29.908
  16:14:00 STEP 2501 H=2501:00 +CPU= 29.807
  16:14:47 STEP 2502 H=2502:00 +CPU= 46.290
  16:15:17 STEP 2503 H=2503:00 +CPU= 29.456
  16:15:47 STEP 2504 H=2504:00 +CPU= 30.266
  16:16:34 STEP 2505 H=2505:00 +CPU= 46.047
  16:16:58 STEP 2506 H=2506:00 +CPU= 23.217
  16:17:20 STEP 2507 H=2507:00 +CPU= 22.450
  16:18:00 STEP 2508 H=2508:00 +CPU= 39.145
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002508
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002508
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002508
  16:18:23 STEP 2509 H=2509:00 +CPU= 22.305
  16:18:46 STEP 2510 H=2510:00 +CPU= 22.923
  16:19:23 STEP 2511 H=2511:00 +CPU= 36.061
  16:19:46 STEP 2512 H=2512:00 +CPU= 22.376
  16:20:08 STEP 2513 H=2513:00 +CPU= 22.183
  16:20:44 STEP 2514 H=2514:00 +CPU= 35.196
  16:21:06 STEP 2515 H=2515:00 +CPU= 21.620
  16:21:28 STEP 2516 H=2516:00 +CPU= 22.207
  16:22:04 STEP 2517 H=2517:00 +CPU= 35.644
  16:22:26 STEP 2518 H=2518:00 +CPU= 21.584
  16:22:49 STEP 2519 H=2519:00 +CPU= 22.837
  16:23:32 STEP 2520 H=2520:00 +CPU= 42.262
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002520
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002520
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002508
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002508
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002508
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_51.zip
Uploading the intermediate file: upload_file_51.zip
  16:23:54 STEP 2521 H=2521:00 +CPU= 21.493
Uploading trickle at timestep: 9072000
  16:24:18 STEP 2522 H=2522:00 +CPU= 22.997
  16:24:54 STEP 2523 H=2523:00 +CPU= 36.032
  16:25:18 STEP 2524 H=2524:00 +CPU= 23.268
  16:25:40 STEP 2525 H=2525:00 +CPU= 22.328
  16:26:18 STEP 2526 H=2526:00 +CPU= 37.160
  16:26:41 STEP 2527 H=2527:00 +CPU= 22.607
  16:27:06 STEP 2528 H=2528:00 +CPU= 23.729
  16:27:44 STEP 2529 H=2529:00 +CPU= 36.927
  16:28:09 STEP 2530 H=2530:00 +CPU= 24.234
  16:28:34 STEP 2531 H=2531:00 +CPU= 23.336
  16:29:14 STEP 2532 H=2532:00 +CPU= 39.908
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002532
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002532
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002532
  16:29:37 STEP 2533 H=2533:00 +CPU= 22.530
  16:30:02 STEP 2534 H=2534:00 +CPU= 23.639
  16:30:39 STEP 2535 H=2535:00 +CPU= 36.325
  16:31:02 STEP 2536 H=2536:00 +CPU= 22.519
  16:31:24 STEP 2537 H=2537:00 +CPU= 22.108
  16:32:01 STEP 2538 H=2538:00 +CPU= 36.087
  16:32:23 STEP 2539 H=2539:00 +CPU= 22.031
  16:32:46 STEP 2540 H=2540:00 +CPU= 22.451
  16:33:22 STEP 2541 H=2541:00 +CPU= 35.361
  16:33:45 STEP 2542 H=2542:00 +CPU= 22.895
  16:34:09 STEP 2543 H=2543:00 +CPU= 23.000
  16:34:52 STEP 2544 H=2544:00 +CPU= 41.999
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002544
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002544
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002532
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002532
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002532
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_52.zip
Uploading the intermediate file: upload_file_52.zip
  16:35:13 STEP 2545 H=2545:00 +CPU= 21.503
  16:35:36 STEP 2546 H=2546:00 +CPU= 22.704
  16:36:12 STEP 2547 H=2547:00 +CPU= 35.589
  16:36:34 STEP 2548 H=2548:00 +CPU= 21.534
  16:36:56 STEP 2549 H=2549:00 +CPU= 21.339
  16:37:32 STEP 2550 H=2550:00 +CPU= 35.324
  16:37:53 STEP 2551 H=2551:00 +CPU= 21.507
  16:38:17 STEP 2552 H=2552:00 +CPU= 23.090
  16:38:52 STEP 2553 H=2553:00 +CPU= 34.371
  16:39:13 STEP 2554 H=2554:00 +CPU= 21.374
  16:39:36 STEP 2555 H=2555:00 +CPU= 21.875
  16:40:16 STEP 2556 H=2556:00 +CPU= 39.618
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002556
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002556
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002556
  16:40:39 STEP 2557 H=2557:00 +CPU= 22.476
  16:41:01 STEP 2558 H=2558:00 +CPU= 21.561
  16:41:39 STEP 2559 H=2559:00 +CPU= 37.525
  16:42:07 STEP 2560 H=2560:00 +CPU= 27.223
  16:42:36 STEP 2561 H=2561:00 +CPU= 28.359
  16:43:21 STEP 2562 H=2562:00 +CPU= 43.676
  16:43:48 STEP 2563 H=2563:00 +CPU= 26.972
  16:44:16 STEP 2564 H=2564:00 +CPU= 26.705
  16:44:56 STEP 2565 H=2565:00 +CPU= 39.669
  16:45:24 STEP 2566 H=2566:00 +CPU= 27.418
  16:45:52 STEP 2567 H=2567:00 +CPU= 27.459
  16:46:44 STEP 2568 H=2568:00 +CPU= 51.164
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002568
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002568
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002556
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002556
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002556
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_53.zip
Uploading the intermediate file: upload_file_53.zip
  16:47:14 STEP 2569 H=2569:00 +CPU= 28.756
  16:47:40 STEP 2570 H=2570:00 +CPU= 25.277
  16:48:23 STEP 2571 H=2571:00 +CPU= 42.356
  16:48:51 STEP 2572 H=2572:00 +CPU= 26.927
  16:49:17 STEP 2573 H=2573:00 +CPU= 25.897
  16:50:00 STEP 2574 H=2574:00 +CPU= 41.934
  16:50:27 STEP 2575 H=2575:00 +CPU= 25.961
  16:50:54 STEP 2576 H=2576:00 +CPU= 26.672
  16:51:41 STEP 2577 H=2577:00 +CPU= 45.925
  16:52:08 STEP 2578 H=2578:00 +CPU= 26.702
  16:52:36 STEP 2579 H=2579:00 +CPU= 26.878
  16:53:25 STEP 2580 H=2580:00 +CPU= 47.786
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002580
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002580
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002580
  16:53:48 STEP 2581 H=2581:00 +CPU= 22.978
  16:54:18 STEP 2582 H=2582:00 +CPU= 29.105
  16:55:00 STEP 2583 H=2583:00 +CPU= 40.881
  16:55:27 STEP 2584 H=2584:00 +CPU= 27.270
  16:55:56 STEP 2585 H=2585:00 +CPU= 27.682
  16:56:43 STEP 2586 H=2586:00 +CPU= 46.832
  16:57:12 STEP 2587 H=2587:00 +CPU= 27.439
  16:57:39 STEP 2588 H=2588:00 +CPU= 27.175
  16:58:18 STEP 2589 H=2589:00 +CPU= 37.626
  16:58:47 STEP 2590 H=2590:00 +CPU= 28.007
  16:59:14 STEP 2591 H=2591:00 +CPU= 26.494
  17:00:05 STEP 2592 H=2592:00 +CPU= 49.919
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002592
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002592
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002592
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMGGhq0f+002580
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMSHhq0f+002580
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291/ICMUAhq0f+002580
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250_54.zip
Uploading the intermediate file: upload_file_54.zip
  17:00:33 STEP 2593 H=2593:00 +CPU= 27.136
  17:01:00 STEP 2594 H=2594:00 +CPU= 27.316
  17:01:44 STEP 2595 H=2595:00 +CPU= 43.041
  17:02:10 STEP 2596 H=2596:00 +CPU= 25.402
  17:02:38 STEP 2597 H=2597:00 +CPU= 27.148
  17:03:24 STEP 2598 H=2598:00 +CPU= 44.834
  17:03:51 STEP 2599 H=2599:00 +CPU= 26.165
  17:04:19 STEP 2600 H=2600:00 +CPU= 26.483
  17:05:00 STEP 2601 H=2601:00 +CPU= 40.250
  17:05:28 STEP 2602 H=2602:00 +CPU= 27.427
  17:05:55 STEP 2603 H=2603:00 +CPU= 26.829
  17:06:44 STEP 2604 H=2604:00 +CPU= 47.421
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002604
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002604
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002604
  17:07:11 STEP 2605 H=2605:00 +CPU= 27.212
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171221:1674839541:37033.624] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4416MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4416 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.712] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                CONVECTION_LAYER 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 CUCALLN 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171222:1674839542:37033.713] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                  CUMASTRN 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171223:1674839543:37034.743] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:14984:14984] [20230127:171223:1674839543:37034.743] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000010D3A76  cumastrn_                 989  cumastrn.F90
oifs_43r3_model.e  00000000010C7792  cucalln_                  352  cucalln.F90
oifs_43r3_model.e  00000000010C4C15  convection_layer_         121  convection_layer.F90
oifs_43r3_model.e  0000000000F11F5B  callpar_                 1064  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2010050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0647
(argv4) batchid: 979
(argv5) wuid: 12196291
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /data/boinc/boinc-client/slots/28
Project directory is: /data/boinc/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196291
..mkdir for temp folder for results failed
Copying: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /data/boinc/boinc-client/slots/28/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /data/boinc/boinc-client/slots/28/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_3da391b9764f995b06aff9abeecfb441 to: /data/boinc/boinc-client/slots/28/oifs_43r3_ps_0647_2010050100_123_979_12196291.zip
Unzipping the namelist zip file: /data/boinc/boinc-client/slots/28/oifs_43r3_ps_0647_2010050100_123_979_12196291.zip
ic_ancil_file: ic_ancil_12196291
ifsdata_file: ifsdata_12196291
climate_data_file: clim_data_12196291
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_c3122ac1b9035eac7559807d89a337ef to: /data/boinc/boinc-client/slots/28/ic_ancil_12196291.zip
Unzipping the IC ancils zip file: /data/boinc/boinc-client/slots/28/ic_ancil_12196291.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c6750a1d3278d64826b0c3fa3d10faf4 to: /data/boinc/boinc-client/slots/28/ifsdata/ifsdata_12196291.zip
Unzipping the ifsdata_zip file: /data/boinc/boinc-client/slots/28/ifsdata/ifsdata_12196291.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /data/boinc/boinc-client/slots/28/159l_2/clim_data_12196291.zip
Unzipping the climate data zip file: /data/boinc/boinc-client/slots/28/159l_2/clim_data_12196291.zip
Checking for progress XML file: /data/boinc/boinc-client/slots/28/progress_file_12196291.xml
Opened progress file ok : /data/boinc/boinc-client/slots/28/progress_file_12196291.xml
-- Model is restarting --
Adjusting last_iter, 2606, to previous model restart step.
Creating progress file: /data/boinc/boinc-client/slots/28/progress_file_12196291.xml
last_cpu_time: 47786
upload_file_number: 55
last_iter: 2591
last_upload: 9331200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0647_2010050100_123_979_12196291_1_r1565294250
The child process has been launched with process id: 84622
Executing the command: /data/boinc/boinc-client/slots/28/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:84622:84622] [20230130:212018:1675113618:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:21:16 STEP 2592 H=2592:00 +CPU= 50.581
  21:21:44 STEP 2593 H=2593:00 +CPU= 27.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:59:29 STEP 2594 H=2594:00 +CPU= 25.790
  07:00:03 STEP 2595 H=2595:00 +CPU= 34.529
  07:00:26 STEP 2596 H=2596:00 +CPU= 22.136
  07:00:47 STEP 2597 H=2597:00 +CPU= 21.410
  07:01:25 STEP 2598 H=2598:00 +CPU= 37.012
  07:01:46 STEP 2599 H=2599:00 +CPU= 21.046
  07:02:07 STEP 2600 H=2600:00 +CPU= 20.767
  07:02:41 STEP 2601 H=2601:00 +CPU= 33.577
  07:03:06 STEP 2602 H=2602:00 +CPU= 25.292
  07:03:33 STEP 2603 H=2603:00 +CPU= 26.530
  07:04:21 STEP 2604 H=2604:00 +CPU= 47.746
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMGGhq0f+002604
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMSHhq0f+002604
Moving to projects directory: /data/boinc/boinc-client/slots/28/ICMUAhq0f+002604
  07:04:48 STEP 2605 H=2605:00 +CPU= 26.491
  07:05:15 STEP 2606 H=2606:00 +CPU= 26.122
  07:05:57 STEP 2607 H=2607:00 +CPU= 42.344
  07:06:25 STEP 2608 H=2608:00 +CPU= 27.321
  07:06:51 STEP 2609 H=2609:00 +CPU= 26.019
  07:07:34 STEP 2610 H=2610:00 +CPU= 42.294
  07:07:58 STEP 2611 H=2611:00 +CPU= 23.585
  07:08:27 STEP 2612 H=2612:00 +CPU= 28.569
  07:09:10 STEP 2613 H=2613:00 +CPU= 42.657
  07:09:41 STEP 2614 H=2614:00 +CPU= 30.480
  07:10:10 STEP 2615 H=2615:00 +CPU= 29.183
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:43:27 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 9,072,000 45,222 0.0050
09 May 2023 16:43:21 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 8,208,000 37,086 0.0045
09 May 2023 16:41:01 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 6,393,600 17,762 0.0028
09 May 2023 16:40:51 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 5,529,600 8,922 0.0016
09 May 2023 16:39:38 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 4,057,200 36,587 0.0090
09 May 2023 16:36:40 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 3,193,200 28,703 0.0090
09 May 2023 16:31:41 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 2,156,400 18,213 0.0084
09 May 2023 16:31:28 1478188 22290089 oifs_43r3_ps_0647_2010050100_123_979_12196291_1 1,292,400 9,858 0.0076


©2024 cpdn.org