climateprediction.net (CPDN) home page
Task 22444492

Task 22444492

Name oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0
Workunit 12290894
Created 12 Jun 2024, 23:45:30 UTC
Sent 13 Jun 2024, 12:02:14 UTC
Report deadline 12 Aug 2024, 12:02:14 UTC
Received 18 Jun 2024, 22:01:57 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1522755
Run time 12 hours 52 min 26 sec
CPU time 12 hours 46 min 52 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 7.59 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 4.91 GB
Peak swap size 5.84 GB
Peak disk usage 1.32 GB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
PU= 23.573
  17:41:57 STEP 1275 H= 318:45 +CPU= 23.001
  17:42:20 STEP 1276 H= 319:00 +CPU= 23.098
  17:42:44 STEP 1277 H= 319:15 +CPU= 23.549
  17:43:08 STEP 1278 H= 319:30 +CPU= 24.294
  17:43:32 STEP 1279 H= 319:45 +CPU= 23.545
  17:43:55 STEP 1280 H= 320:00 +CPU= 23.099
  17:44:17 STEP 1281 H= 320:15 +CPU= 22.108
  17:44:40 STEP 1282 H= 320:30 +CPU= 22.849
  17:45:04 STEP 1283 H= 320:45 +CPU= 24.150
  17:45:34 STEP 1284 H= 321:00 +CPU= 30.250
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001284
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001284
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001284
  17:45:59 STEP 1285 H= 321:15 +CPU= 24.857
  17:46:23 STEP 1286 H= 321:30 +CPU= 23.675
  17:46:45 STEP 1287 H= 321:45 +CPU= 22.415
  17:47:07 STEP 1288 H= 322:00 +CPU= 21.508
  17:47:30 STEP 1289 H= 322:15 +CPU= 23.033
  17:47:54 STEP 1290 H= 322:30 +CPU= 23.941
  17:48:18 STEP 1291 H= 322:45 +CPU= 23.488
  17:48:42 STEP 1292 H= 323:00 +CPU= 24.145
  17:49:04 STEP 1293 H= 323:15 +CPU= 22.350
  17:49:26 STEP 1294 H= 323:30 +CPU= 21.992
  17:49:49 STEP 1295 H= 323:45 +CPU= 22.658
  17:50:21 STEP 1296 H= 324:00 +CPU= 31.878
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001296
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001296
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001296
  17:50:42 STEP 1297 H= 324:15 +CPU= 21.343
  17:51:06 STEP 1298 H= 324:30 +CPU= 23.443
  17:51:29 STEP 1299 H= 324:45 +CPU= 22.813
  17:51:51 STEP 1300 H= 325:00 +CPU= 22.334
  17:52:15 STEP 1301 H= 325:15 +CPU= 23.789
  17:52:38 STEP 1302 H= 325:30 +CPU= 23.356
  17:53:02 STEP 1303 H= 325:45 +CPU= 23.384
  17:53:25 STEP 1304 H= 326:00 +CPU= 22.852
  17:53:48 STEP 1305 H= 326:15 +CPU= 23.284
  17:54:13 STEP 1306 H= 326:30 +CPU= 24.480
  17:54:37 STEP 1307 H= 326:45 +CPU= 24.604
  17:55:09 STEP 1308 H= 327:00 +CPU= 31.685
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001308
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001308
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001308
  17:55:34 STEP 1309 H= 327:15 +CPU= 24.524
  17:55:55 STEP 1310 H= 327:30 +CPU= 20.738
  17:56:19 STEP 1311 H= 327:45 +CPU= 23.885
  17:56:42 STEP 1312 H= 328:00 +CPU= 23.481
  17:57:05 STEP 1313 H= 328:15 +CPU= 22.762
  17:57:29 STEP 1314 H= 328:30 +CPU= 24.073
  17:57:53 STEP 1315 H= 328:45 +CPU= 23.649
  17:58:17 STEP 1316 H= 329:00 +CPU= 23.848
  17:58:40 STEP 1317 H= 329:15 +CPU= 23.404
  17:59:03 STEP 1318 H= 329:30 +CPU= 22.543
  17:59:25 STEP 1319 H= 329:45 +CPU= 21.682
  17:59:54 STEP 1320 H= 330:00 +CPU= 29.020
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001320
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001320
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001320
  18:00:17 STEP 1321 H= 330:15 +CPU= 23.217
  18:00:41 STEP 1322 H= 330:30 +CPU= 24.253
  18:01:04 STEP 1323 H= 330:45 +CPU= 23.015
  18:01:28 STEP 1324 H= 331:00 +CPU= 24.173
  18:01:51 STEP 1325 H= 331:15 +CPU= 22.614
  18:02:13 STEP 1326 H= 331:30 +CPU= 22.022
  18:02:36 STEP 1327 H= 331:45 +CPU= 22.449
  18:02:59 STEP 1328 H= 332:00 +CPU= 23.441
  18:03:23 STEP 1329 H= 332:15 +CPU= 23.584
  18:03:45 STEP 1330 H= 332:30 +CPU= 21.955
  18:04:07 STEP 1331 H= 332:45 +CPU= 21.725
  18:04:37 STEP 1332 H= 333:00 +CPU= 30.251
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001332
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001332
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001332
  18:05:01 STEP 1333 H= 333:15 +CPU= 24.339
  18:05:23 STEP 1334 H= 333:30 +CPU= 21.177
  18:05:47 STEP 1335 H= 333:45 +CPU= 23.976
  18:06:10 STEP 1336 H= 334:00 +CPU= 23.449
  18:06:32 STEP 1337 H= 334:15 +CPU= 22.178
  18:06:54 STEP 1338 H= 334:30 +CPU= 22.069
  18:07:18 STEP 1339 H= 334:45 +CPU= 23.299
  18:07:41 STEP 1340 H= 335:00 +CPU= 22.723
  18:08:03 STEP 1341 H= 335:15 +CPU= 22.359
  18:08:25 STEP 1342 H= 335:30 +CPU= 22.251
  18:08:50 STEP 1343 H= 335:45 +CPU= 24.285
  18:09:20 STEP 1344 H= 336:00 +CPU= 30.585
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001344
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001344
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001332
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001332
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001332
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0_r60511044_13.zip
Uploading the intermediate file: upload_file_13.zip
  18:09:43 STEP 1345 H= 336:15 +CPU= 23.085
  18:10:06 STEP 1346 H= 336:30 +CPU= 22.418
  18:10:31 STEP 1347 H= 336:45 +CPU= 24.901
  18:10:54 STEP 1348 H= 337:00 +CPU= 22.827
  18:11:18 STEP 1349 H= 337:15 +CPU= 23.915
  18:11:41 STEP 1350 H= 337:30 +CPU= 23.331
  18:12:07 STEP 1351 H= 337:45 +CPU= 25.186
  18:12:29 STEP 1352 H= 338:00 +CPU= 22.728
  18:12:52 STEP 1353 H= 338:15 +CPU= 22.602
  18:13:16 STEP 1354 H= 338:30 +CPU= 23.511
  18:13:37 STEP 1355 H= 338:45 +CPU= 21.461
  18:14:06 STEP 1356 H= 339:00 +CPU= 29.137
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001356
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001356
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001356
  18:14:30 STEP 1357 H= 339:15 +CPU= 23.402
  18:14:55 STEP 1358 H= 339:30 +CPU= 24.952
  18:15:18 STEP 1359 H= 339:45 +CPU= 22.932
  18:15:43 STEP 1360 H= 340:00 +CPU= 24.888
  18:16:05 STEP 1361 H= 340:15 +CPU= 22.587
  18:16:28 STEP 1362 H= 340:30 +CPU= 22.080
  18:16:52 STEP 1363 H= 340:45 +CPU= 24.366
  18:17:17 STEP 1364 H= 341:00 +CPU= 24.838
  18:17:41 STEP 1365 H= 341:15 +CPU= 24.011
  18:18:05 STEP 1366 H= 341:30 +CPU= 23.555
  18:18:26 STEP 1367 H= 341:45 +CPU= 21.438
  18:18:55 STEP 1368 H= 342:00 +CPU= 28.693
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001368
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001368
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001368
  18:19:16 STEP 1369 H= 342:15 +CPU= 21.004
  18:19:38 STEP 1370 H= 342:30 +CPU= 22.353
  18:20:02 STEP 1371 H= 342:45 +CPU= 23.177
  18:20:26 STEP 1372 H= 343:00 +CPU= 23.870
  18:20:50 STEP 1373 H= 343:15 +CPU= 24.263
  18:21:12 STEP 1374 H= 343:30 +CPU= 22.537
  18:21:35 STEP 1375 H= 343:45 +CPU= 22.459
  18:21:58 STEP 1376 H= 344:00 +CPU= 22.628
  18:22:22 STEP 1377 H= 344:15 +CPU= 24.405
  18:22:42 STEP 1378 H= 344:30 +CPU= 20.348
  18:23:04 STEP 1379 H= 344:45 +CPU= 21.390
  18:23:35 STEP 1380 H= 345:00 +CPU= 30.684
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001380
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001380
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001380
  18:23:58 STEP 1381 H= 345:15 +CPU= 23.381
  18:24:23 STEP 1382 H= 345:30 +CPU= 24.826
  18:24:48 STEP 1383 H= 345:45 +CPU= 24.852
  18:25:12 STEP 1384 H= 346:00 +CPU= 24.244
  18:25:37 STEP 1385 H= 346:15 +CPU= 24.520
  18:26:01 STEP 1386 H= 346:30 +CPU= 23.784
  18:26:25 STEP 1387 H= 346:45 +CPU= 24.551
  18:26:50 STEP 1388 H= 347:00 +CPU= 24.413
  18:27:12 STEP 1389 H= 347:15 +CPU= 22.270
  18:27:36 STEP 1390 H= 347:30 +CPU= 23.541
  18:27:59 STEP 1391 H= 347:45 +CPU= 23.048
  18:28:34 STEP 1392 H= 348:00 +CPU= 35.345
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001392
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001392
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001392
  18:28:57 STEP 1393 H= 348:15 +CPU= 22.905
  18:29:20 STEP 1394 H= 348:30 +CPU= 22.360
  18:29:42 STEP 1395 H= 348:45 +CPU= 22.717
  18:30:07 STEP 1396 H= 349:00 +CPU= 24.484
  18:30:32 STEP 1397 H= 349:15 +CPU= 24.838
  18:30:54 STEP 1398 H= 349:30 +CPU= 22.181
  18:31:18 STEP 1399 H= 349:45 +CPU= 23.694
  18:31:41 STEP 1400 H= 350:00 +CPU= 23.484
  18:32:06 STEP 1401 H= 350:15 +CPU= 24.279
  18:32:30 STEP 1402 H= 350:30 +CPU= 24.402
  18:32:53 STEP 1403 H= 350:45 +CPU= 22.914
  18:33:21 STEP 1404 H= 351:00 +CPU= 27.873
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001404
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001404
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001404
  18:33:46 STEP 1405 H= 351:15 +CPU= 25.135
  18:34:10 STEP 1406 H= 351:30 +CPU= 23.997
  18:34:31 STEP 1407 H= 351:45 +CPU= 20.474
  18:34:55 STEP 1408 H= 352:00 +CPU= 24.377
  18:35:18 STEP 1409 H= 352:15 +CPU= 22.590
  18:35:40 STEP 1410 H= 352:30 +CPU= 21.817
  18:36:05 STEP 1411 H= 352:45 +CPU= 24.606
  18:36:28 STEP 1412 H= 353:00 +CPU= 23.870
  18:36:51 STEP 1413 H= 353:15 +CPU= 22.902
  18:37:15 STEP 1414 H= 353:30 +CPU= 23.347
  18:37:39 STEP 1415 H= 353:45 +CPU= 23.759
  18:38:08 STEP 1416 H= 354:00 +CPU= 28.981
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001416
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001416
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001416
  18:38:31 STEP 1417 H= 354:15 +CPU= 23.276
  18:38:56 STEP 1418 H= 354:30 +CPU= 25.001
  18:39:21 STEP 1419 H= 354:45 +CPU= 24.558
  18:39:46 STEP 1420 H= 355:00 +CPU= 25.623
  18:40:09 STEP 1421 H= 355:15 +CPU= 22.876
  18:40:33 STEP 1422 H= 355:30 +CPU= 23.390
  18:40:57 STEP 1423 H= 355:45 +CPU= 23.565
  18:41:21 STEP 1424 H= 356:00 +CPU= 24.659
  18:41:46 STEP 1425 H= 356:15 +CPU= 24.660
  18:42:10 STEP 1426 H= 356:30 +CPU= 23.827
  18:42:35 STEP 1427 H= 356:45 +CPU= 24.577
  18:43:04 STEP 1428 H= 357:00 +CPU= 29.789
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001428
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001428
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001428
  18:43:28 STEP 1429 H= 357:15 +CPU= 23.436
  18:43:51 STEP 1430 H= 357:30 +CPU= 23.243
  18:44:14 STEP 1431 H= 357:45 +CPU= 22.499
  18:44:36 STEP 1432 H= 358:00 +CPU= 22.052
  18:45:00 STEP 1433 H= 358:15 +CPU= 23.812
  18:45:23 STEP 1434 H= 358:30 +CPU= 23.398
  18:45:46 STEP 1435 H= 358:45 +CPU= 22.791
  18:46:11 STEP 1436 H= 359:00 +CPU= 24.976
  18:46:36 STEP 1437 H= 359:15 +CPU= 24.640
  18:47:01 STEP 1438 H= 359:30 +CPU= 24.920
  18:47:26 STEP 1439 H= 359:45 +CPU= 24.617
  18:47:59 STEP 1440 H= 360:00 +CPU= 33.221
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001440
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001440
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001428
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001428
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001428
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0_r60511044_14.zip
Uploading the intermediate file: upload_file_14.zip
  18:48:24 STEP 1441 H= 360:15 +CPU= 24.985
  18:48:49 STEP 1442 H= 360:30 +CPU= 25.037
  18:49:14 STEP 1443 H= 360:45 +CPU= 25.165
  18:49:40 STEP 1444 H= 361:00 +CPU= 25.421
  18:50:03 STEP 1445 H= 361:15 +CPU= 23.421
  18:50:29 STEP 1446 H= 361:30 +CPU= 25.186
  18:50:53 STEP 1447 H= 361:45 +CPU= 24.324
  18:51:18 STEP 1448 H= 362:00 +CPU= 24.937
  18:51:42 STEP 1449 H= 362:15 +CPU= 23.499
  18:52:06 STEP 1450 H= 362:30 +CPU= 24.271
  18:52:29 STEP 1451 H= 362:45 +CPU= 22.760
  18:52:59 STEP 1452 H= 363:00 +CPU= 30.337
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001452
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001452
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001452
  18:53:21 STEP 1453 H= 363:15 +CPU= 22.053
  18:53:44 STEP 1454 H= 363:30 +CPU= 22.803
  18:54:06 STEP 1455 H= 363:45 +CPU= 21.870
  18:54:29 STEP 1456 H= 364:00 +CPU= 23.267
  18:54:51 STEP 1457 H= 364:15 +CPU= 22.098
  18:55:14 STEP 1458 H= 364:30 +CPU= 22.624
  18:55:36 STEP 1459 H= 364:45 +CPU= 22.154
  18:56:01 STEP 1460 H= 365:00 +CPU= 24.394
  18:56:24 STEP 1461 H= 365:15 +CPU= 23.136
  18:56:48 STEP 1462 H= 365:30 +CPU= 23.676
  18:57:12 STEP 1463 H= 365:45 +CPU= 23.914
  18:57:41 STEP 1464 H= 366:00 +CPU= 29.332
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001464
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001464
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001464
  18:58:05 STEP 1465 H= 366:15 +CPU= 23.745
  18:58:29 STEP 1466 H= 366:30 +CPU= 24.403
  18:58:52 STEP 1467 H= 366:45 +CPU= 22.906
  18:59:18 STEP 1468 H= 367:00 +CPU= 26.016
  18:59:41 STEP 1469 H= 367:15 +CPU= 22.003
  19:00:04 STEP 1470 H= 367:30 +CPU= 23.289
  19:00:28 STEP 1471 H= 367:45 +CPU= 23.715
  19:00:51 STEP 1472 H= 368:00 +CPU= 23.423
  19:01:13 STEP 1473 H= 368:15 +CPU= 22.032
  19:01:37 STEP 1474 H= 368:30 +CPU= 24.004
  19:02:01 STEP 1475 H= 368:45 +CPU= 23.355
  19:02:33 STEP 1476 H= 369:00 +CPU= 31.813
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001476
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001476
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001476
  19:02:57 STEP 1477 H= 369:15 +CPU= 24.772
  19:03:20 STEP 1478 H= 369:30 +CPU= 22.932
  19:03:43 STEP 1479 H= 369:45 +CPU= 22.308
  19:04:07 STEP 1480 H= 370:00 +CPU= 23.893
  19:04:29 STEP 1481 H= 370:15 +CPU= 21.677
  19:04:53 STEP 1482 H= 370:30 +CPU= 23.943
  19:05:17 STEP 1483 H= 370:45 +CPU= 24.316
  19:05:41 STEP 1484 H= 371:00 +CPU= 23.781
  19:06:03 STEP 1485 H= 371:15 +CPU= 22.192
  19:06:26 STEP 1486 H= 371:30 +CPU= 22.620
  19:06:49 STEP 1487 H= 371:45 +CPU= 23.525
  19:07:21 STEP 1488 H= 372:00 +CPU= 31.379
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001488
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001488
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001488
  19:07:42 STEP 1489 H= 372:15 +CPU= 21.575
  19:08:04 STEP 1490 H= 372:30 +CPU= 21.041
  19:08:25 STEP 1491 H= 372:45 +CPU= 21.008
  19:08:48 STEP 1492 H= 373:00 +CPU= 23.021
  19:09:10 STEP 1493 H= 373:15 +CPU= 22.198
  19:09:32 STEP 1494 H= 373:30 +CPU= 22.218
  19:09:55 STEP 1495 H= 373:45 +CPU= 22.857
  19:10:19 STEP 1496 H= 374:00 +CPU= 23.583
  19:10:42 STEP 1497 H= 374:15 +CPU= 23.275
  19:11:05 STEP 1498 H= 374:30 +CPU= 22.898
  19:11:30 STEP 1499 H= 374:45 +CPU= 24.705
  19:12:00 STEP 1500 H= 375:00 +CPU= 29.599
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001500
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001500
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001500
  19:12:23 STEP 1501 H= 375:15 +CPU= 23.522
  19:12:45 STEP 1502 H= 375:30 +CPU= 22.145
  19:13:07 STEP 1503 H= 375:45 +CPU= 21.710
  19:13:30 STEP 1504 H= 376:00 +CPU= 23.253
  19:13:54 STEP 1505 H= 376:15 +CPU= 23.672
  19:14:17 STEP 1506 H= 376:30 +CPU= 22.475
  19:14:40 STEP 1507 H= 376:45 +CPU= 22.885
  19:15:03 STEP 1508 H= 377:00 +CPU= 23.291
  19:15:27 STEP 1509 H= 377:15 +CPU= 24.260
  19:15:50 STEP 1510 H= 377:30 +CPU= 22.637
  19:16:13 STEP 1511 H= 377:45 +CPU= 22.994
  19:16:43 STEP 1512 H= 378:00 +CPU= 30.361
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001512
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001512
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001512
  19:17:07 STEP 1513 H= 378:15 +CPU= 23.311
  19:17:31 STEP 1514 H= 378:30 +CPU= 24.175
  19:17:55 STEP 1515 H= 378:45 +CPU= 24.275
  19:18:19 STEP 1516 H= 379:00 +CPU= 23.970
  19:18:41 STEP 1517 H= 379:15 +CPU= 21.556
  19:19:05 STEP 1518 H= 379:30 +CPU= 23.716
  19:19:30 STEP 1519 H= 379:45 +CPU= 25.023
  19:19:55 STEP 1520 H= 380:00 +CPU= 24.949
  19:20:18 STEP 1521 H= 380:15 +CPU= 23.199
  19:20:43 STEP 1522 H= 380:30 +CPU= 24.913
  19:21:07 STEP 1523 H= 380:45 +CPU= 24.075
  19:21:35 STEP 1524 H= 381:00 +CPU= 28.048
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001524
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001524
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001524
  19:21:59 STEP 1525 H= 381:15 +CPU= 23.589
  19:22:21 STEP 1526 H= 381:30 +CPU= 21.748
  19:22:45 STEP 1527 H= 381:45 +CPU= 24.452
  19:23:10 STEP 1528 H= 382:00 +CPU= 24.163
  19:23:34 STEP 1529 H= 382:15 +CPU= 24.699
  19:23:57 STEP 1530 H= 382:30 +CPU= 22.925
  19:24:20 STEP 1531 H= 382:45 +CPU= 22.987
  19:24:46 STEP 1532 H= 383:00 +CPU= 25.716
  19:25:08 STEP 1533 H= 383:15 +CPU= 22.007
  19:25:33 STEP 1534 H= 383:30 +CPU= 24.276
  19:25:55 STEP 1535 H= 383:45 +CPU= 22.539
  19:26:29 STEP 1536 H= 384:00 +CPU= 33.333
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001536
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001536
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001524
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001524
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001524
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0_r60511044_15.zip
Uploading the intermediate file: upload_file_15.zip
  19:26:53 STEP 1537 H= 384:15 +CPU= 24.463
  19:27:16 STEP 1538 H= 384:30 +CPU= 23.114
  19:27:40 STEP 1539 H= 384:45 +CPU= 23.657
  19:28:05 STEP 1540 H= 385:00 +CPU= 24.368
  19:28:29 STEP 1541 H= 385:15 +CPU= 24.584
  19:28:54 STEP 1542 H= 385:30 +CPU= 24.643
  19:29:17 STEP 1543 H= 385:45 +CPU= 23.156
  19:29:39 STEP 1544 H= 386:00 +CPU= 21.467
  19:30:00 STEP 1545 H= 386:15 +CPU= 21.533
  19:30:25 STEP 1546 H= 386:30 +CPU= 24.629
  19:30:49 STEP 1547 H= 386:45 +CPU= 23.629
  19:31:17 STEP 1548 H= 387:00 +CPU= 27.943
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001548
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001548
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001548
  19:31:39 STEP 1549 H= 387:15 +CPU= 22.163
  19:32:01 STEP 1550 H= 387:30 +CPU= 22.042
  19:32:23 STEP 1551 H= 387:45 +CPU= 22.241
  19:32:45 STEP 1552 H= 388:00 +CPU= 21.733
  19:33:08 STEP 1553 H= 388:15 +CPU= 23.002
  19:33:32 STEP 1554 H= 388:30 +CPU= 23.611
  19:33:53 STEP 1555 H= 388:45 +CPU= 21.077
  19:34:14 STEP 1556 H= 389:00 +CPU= 21.317
  19:34:37 STEP 1557 H= 389:15 +CPU= 22.765
  19:34:59 STEP 1558 H= 389:30 +CPU= 21.706
  19:35:22 STEP 1559 H= 389:45 +CPU= 22.539
  19:35:52 STEP 1560 H= 390:00 +CPU= 30.161
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001560
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001560
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001560
  19:36:17 STEP 1561 H= 390:15 +CPU= 24.760
  19:36:40 STEP 1562 H= 390:30 +CPU= 23.422
  19:37:04 STEP 1563 H= 390:45 +CPU= 23.594
  19:37:28 STEP 1564 H= 391:00 +CPU= 24.603
  19:37:52 STEP 1565 H= 391:15 +CPU= 23.357
  19:38:15 STEP 1566 H= 391:30 +CPU= 23.273
  19:38:39 STEP 1567 H= 391:45 +CPU= 23.592
  19:39:05 STEP 1568 H= 392:00 +CPU= 25.691
  19:39:30 STEP 1569 H= 392:15 +CPU= 25.849
  19:39:54 STEP 1570 H= 392:30 +CPU= 23.543
  19:40:16 STEP 1571 H= 392:45 +CPU= 22.264
  19:40:45 STEP 1572 H= 393:00 +CPU= 28.574
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001572
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001572
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001572
  19:41:07 STEP 1573 H= 393:15 +CPU= 21.846
  19:41:28 STEP 1574 H= 393:30 +CPU= 20.904
  19:41:48 STEP 1575 H= 393:45 +CPU= 19.877
  19:42:11 STEP 1576 H= 394:00 +CPU= 22.659
  19:42:34 STEP 1577 H= 394:15 +CPU= 22.981
  19:42:56 STEP 1578 H= 394:30 +CPU= 22.428
  19:43:19 STEP 1579 H= 394:45 +CPU= 23.133
  19:43:43 STEP 1580 H= 395:00 +CPU= 23.637
  19:44:05 STEP 1581 H= 395:15 +CPU= 22.336
  19:44:29 STEP 1582 H= 395:30 +CPU= 23.028
  19:44:51 STEP 1583 H= 395:45 +CPU= 22.508
  19:45:23 STEP 1584 H= 396:00 +CPU= 32.098
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001584
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001584
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001584
  19:45:46 STEP 1585 H= 396:15 +CPU= 22.672
  19:46:10 STEP 1586 H= 396:30 +CPU= 24.147
  19:46:34 STEP 1587 H= 396:45 +CPU= 23.438
  19:46:59 STEP 1588 H= 397:00 +CPU= 25.559
  19:47:22 STEP 1589 H= 397:15 +CPU= 22.322
  19:47:46 STEP 1590 H= 397:30 +CPU= 23.997
  19:48:09 STEP 1591 H= 397:45 +CPU= 23.145
  19:48:34 STEP 1592 H= 398:00 +CPU= 24.816
  19:48:58 STEP 1593 H= 398:15 +CPU= 24.375
  19:49:23 STEP 1594 H= 398:30 +CPU= 24.416
  19:49:46 STEP 1595 H= 398:45 +CPU= 22.838
  19:50:16 STEP 1596 H= 399:00 +CPU= 30.041
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001596
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001596
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001596
  19:50:40 STEP 1597 H= 399:15 +CPU= 23.916
  19:51:03 STEP 1598 H= 399:30 +CPU= 23.049
  19:51:26 STEP 1599 H= 399:45 +CPU= 23.282
  19:51:52 STEP 1600 H= 400:00 +CPU= 25.425
  19:52:15 STEP 1601 H= 400:15 +CPU= 22.926
  19:52:37 STEP 1602 H= 400:30 +CPU= 22.008
  19:53:00 STEP 1603 H= 400:45 +CPU= 22.641
  19:53:22 STEP 1604 H= 401:00 +CPU= 22.051
  19:53:44 STEP 1605 H= 401:15 +CPU= 22.260
  19:54:05 STEP 1606 H= 401:30 +CPU= 20.937
  19:54:28 STEP 1607 H= 401:45 +CPU= 22.636
  19:54:57 STEP 1608 H= 402:00 +CPU= 29.363
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001608
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001608
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001608
  19:55:20 STEP 1609 H= 402:15 +CPU= 22.737
  19:55:42 STEP 1610 H= 402:30 +CPU= 22.131
  19:56:04 STEP 1611 H= 402:45 +CPU= 21.847
  19:56:27 STEP 1612 H= 403:00 +CPU= 23.169
  19:56:50 STEP 1613 H= 403:15 +CPU= 22.566
  19:57:12 STEP 1614 H= 403:30 +CPU= 22.572
  19:57:37 STEP 1615 H= 403:45 +CPU= 24.358
  19:58:00 STEP 1616 H= 404:00 +CPU= 23.552
  19:58:24 STEP 1617 H= 404:15 +CPU= 23.227
  19:58:46 STEP 1618 H= 404:30 +CPU= 21.982
  19:59:09 STEP 1619 H= 404:45 +CPU= 23.381
  19:59:38 STEP 1620 H= 405:00 +CPU= 28.792
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001620
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001620
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001620
  20:00:02 STEP 1621 H= 405:15 +CPU= 23.407
  20:00:25 STEP 1622 H= 405:30 +CPU= 23.522
  20:00:48 STEP 1623 H= 405:45 +CPU= 22.919
  20:01:12 STEP 1624 H= 406:00 +CPU= 24.084
  20:01:35 STEP 1625 H= 406:15 +CPU= 22.927
  20:01:58 STEP 1626 H= 406:30 +CPU= 23.110
  20:02:23 STEP 1627 H= 406:45 +CPU= 24.122
  20:02:47 STEP 1628 H= 407:00 +CPU= 24.335
  20:03:11 STEP 1629 H= 407:15 +CPU= 24.038
  20:03:35 STEP 1630 H= 407:30 +CPU= 24.031
  20:03:59 STEP 1631 H= 407:45 +CPU= 24.165
  20:04:32 STEP 1632 H= 408:00 +CPU= 32.528
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001632
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001632
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001620
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001620
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001620
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0_r60511044_16.zip
Uploading the intermediate file: upload_file_16.zip
  20:04:56 STEP 1633 H= 408:15 +CPU= 24.389
  20:05:20 STEP 1634 H= 408:30 +CPU= 23.752
  20:05:44 STEP 1635 H= 408:45 +CPU= 23.253
  20:06:09 STEP 1636 H= 409:00 +CPU= 25.218
  20:06:32 STEP 1637 H= 409:15 +CPU= 23.538
  20:06:56 STEP 1638 H= 409:30 +CPU= 23.761
  20:07:20 STEP 1639 H= 409:45 +CPU= 23.674
  20:07:43 STEP 1640 H= 410:00 +CPU= 22.985
  20:08:07 STEP 1641 H= 410:15 +CPU= 23.800
  20:08:31 STEP 1642 H= 410:30 +CPU= 23.581
  20:08:54 STEP 1643 H= 410:45 +CPU= 23.067
  20:09:24 STEP 1644 H= 411:00 +CPU= 29.999
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001644
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001644
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001644
  20:09:48 STEP 1645 H= 411:15 +CPU= 23.747
  20:10:10 STEP 1646 H= 411:30 +CPU= 22.751
  20:10:33 STEP 1647 H= 411:45 +CPU= 22.060
  20:10:56 STEP 1648 H= 412:00 +CPU= 23.402
  20:11:19 STEP 1649 H= 412:15 +CPU= 23.064
  20:11:43 STEP 1650 H= 412:30 +CPU= 23.807
  20:12:06 STEP 1651 H= 412:45 +CPU= 23.036
  20:12:30 STEP 1652 H= 413:00 +CPU= 23.520
  20:12:53 STEP 1653 H= 413:15 +CPU= 23.601
  20:13:15 STEP 1654 H= 413:30 +CPU= 21.350
  20:13:35 STEP 1655 H= 413:45 +CPU= 20.674
  20:14:02 STEP 1656 H= 414:00 +CPU= 26.926
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001656
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001656
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001656
  20:14:24 STEP 1657 H= 414:15 +CPU= 21.236
  20:14:44 STEP 1658 H= 414:30 +CPU= 20.500
  20:15:06 STEP 1659 H= 414:45 +CPU= 22.065
  20:15:28 STEP 1660 H= 415:00 +CPU= 21.893
  20:15:51 STEP 1661 H= 415:15 +CPU= 22.805
  20:16:13 STEP 1662 H= 415:30 +CPU= 21.785
  20:16:36 STEP 1663 H= 415:45 +CPU= 22.413
  20:16:58 STEP 1664 H= 416:00 +CPU= 22.324
  20:17:21 STEP 1665 H= 416:15 +CPU= 23.277
  20:17:45 STEP 1666 H= 416:30 +CPU= 23.961
  20:18:10 STEP 1667 H= 416:45 +CPU= 24.422
  20:18:38 STEP 1668 H= 417:00 +CPU= 28.269
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001668
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001668
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001668
  20:19:01 STEP 1669 H= 417:15 +CPU= 22.925
  20:19:21 STEP 1670 H= 417:30 +CPU= 20.135
  20:19:44 STEP 1671 H= 417:45 +CPU= 22.434
  20:20:08 STEP 1672 H= 418:00 +CPU= 23.919
  20:20:33 STEP 1673 H= 418:15 +CPU= 24.732
  20:20:57 STEP 1674 H= 418:30 +CPU= 24.139
  20:21:20 STEP 1675 H= 418:45 +CPU= 23.491
  20:21:45 STEP 1676 H= 419:00 +CPU= 24.682
  20:22:10 STEP 1677 H= 419:15 +CPU= 24.453
  20:22:33 STEP 1678 H= 419:30 +CPU= 23.643
  20:22:57 STEP 1679 H= 419:45 +CPU= 23.626
  20:23:28 STEP 1680 H= 420:00 +CPU= 30.826
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001680
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001680
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001680
  20:23:49 STEP 1681 H= 420:15 +CPU= 21.329
  20:24:13 STEP 1682 H= 420:30 +CPU= 23.978
  20:24:37 STEP 1683 H= 420:45 +CPU= 23.198
  20:25:00 STEP 1684 H= 421:00 +CPU= 23.247
  20:25:23 STEP 1685 H= 421:15 +CPU= 22.883
  20:25:47 STEP 1686 H= 421:30 +CPU= 23.997
  20:26:10 STEP 1687 H= 421:45 +CPU= 23.431
  20:26:34 STEP 1688 H= 422:00 +CPU= 23.819
  20:26:59 STEP 1689 H= 422:15 +CPU= 24.281
  20:27:22 STEP 1690 H= 422:30 +CPU= 23.104
  20:27:45 STEP 1691 H= 422:45 +CPU= 23.074
  20:28:15 STEP 1692 H= 423:00 +CPU= 30.502
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001692
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001692
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001692
  20:28:40 STEP 1693 H= 423:15 +CPU= 24.100
  20:29:02 STEP 1694 H= 423:30 +CPU= 22.335
  20:29:27 STEP 1695 H= 423:45 +CPU= 24.787
  20:29:51 STEP 1696 H= 424:00 +CPU= 23.787
  20:30:14 STEP 1697 H= 424:15 +CPU= 23.624
  20:30:38 STEP 1698 H= 424:30 +CPU= 23.293
  20:31:01 STEP 1699 H= 424:45 +CPU= 22.893
  20:31:25 STEP 1700 H= 425:00 +CPU= 24.323
  20:31:49 STEP 1701 H= 425:15 +CPU= 23.906
  20:32:12 STEP 1702 H= 425:30 +CPU= 22.908
  20:32:36 STEP 1703 H= 425:45 +CPU= 23.570
  20:33:05 STEP 1704 H= 426:00 +CPU= 29.296
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001704
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001704
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001704
  20:33:30 STEP 1705 H= 426:15 +CPU= 24.652
  20:33:54 STEP 1706 H= 426:30 +CPU= 23.962
  20:34:18 STEP 1707 H= 426:45 +CPU= 23.809
  20:34:43 STEP 1708 H= 427:00 +CPU= 25.200
  20:35:06 STEP 1709 H= 427:15 +CPU= 23.315
  20:35:31 STEP 1710 H= 427:30 +CPU= 24.515
  20:35:54 STEP 1711 H= 427:45 +CPU= 23.583
  20:36:18 STEP 1712 H= 428:00 +CPU= 23.919
  20:36:41 STEP 1713 H= 428:15 +CPU= 22.946
  20:37:04 STEP 1714 H= 428:30 +CPU= 22.301
  20:37:27 STEP 1715 H= 428:45 +CPU= 23.513
  20:37:58 STEP 1716 H= 429:00 +CPU= 30.199
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001716
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001716
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001716
  20:38:22 STEP 1717 H= 429:15 +CPU= 23.843
  20:38:46 STEP 1718 H= 429:30 +CPU= 23.967
  20:39:10 STEP 1719 H= 429:45 +CPU= 24.471
  20:39:34 STEP 1720 H= 430:00 +CPU= 23.570
  20:39:58 STEP 1721 H= 430:15 +CPU= 23.883
  20:40:22 STEP 1722 H= 430:30 +CPU= 24.444
  20:40:46 STEP 1723 H= 430:45 +CPU= 23.790
  20:41:11 STEP 1724 H= 431:00 +CPU= 24.953
  20:41:32 STEP 1725 H= 431:15 +CPU= 21.026
  20:41:56 STEP 1726 H= 431:30 +CPU= 23.444
  20:42:21 STEP 1727 H= 431:45 +CPU= 24.874
  20:42:54 STEP 1728 H= 432:00 +CPU= 33.618
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001728
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001728
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001716
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001716
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001716
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0_r60511044_17.zip
Uploading the intermediate file: upload_file_17.zip
  20:43:19 STEP 1729 H= 432:15 +CPU= 24.973
  20:43:44 STEP 1730 H= 432:30 +CPU= 24.956
  20:44:10 STEP 1731 H= 432:45 +CPU= 25.129
  20:44:33 STEP 1732 H= 433:00 +CPU= 23.761
  20:44:55 STEP 1733 H= 433:15 +CPU= 21.776
  20:45:17 STEP 1734 H= 433:30 +CPU= 21.283
  20:45:39 STEP 1735 H= 433:45 +CPU= 22.678
  20:46:04 STEP 1736 H= 434:00 +CPU= 24.147
  20:46:27 STEP 1737 H= 434:15 +CPU= 23.067
  20:46:49 STEP 1738 H= 434:30 +CPU= 22.551
  20:47:13 STEP 1739 H= 434:45 +CPU= 23.223
  20:47:43 STEP 1740 H= 435:00 +CPU= 30.341
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001740
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001740
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001740
  20:48:05 STEP 1741 H= 435:15 +CPU= 22.185
  20:48:28 STEP 1742 H= 435:30 +CPU= 23.129
  20:48:52 STEP 1743 H= 435:45 +CPU= 24.058
  20:49:16 STEP 1744 H= 436:00 +CPU= 23.092
  20:49:39 STEP 1745 H= 436:15 +CPU= 23.065
  20:50:03 STEP 1746 H= 436:30 +CPU= 24.360
  20:50:24 STEP 1747 H= 436:45 +CPU= 21.331
  20:50:47 STEP 1748 H= 437:00 +CPU= 22.921
  20:51:09 STEP 1749 H= 437:15 +CPU= 21.219
  20:51:31 STEP 1750 H= 437:30 +CPU= 22.268
  20:51:54 STEP 1751 H= 437:45 +CPU= 22.602
  20:52:23 STEP 1752 H= 438:00 +CPU= 29.694
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001752
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001752
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001752
  20:52:47 STEP 1753 H= 438:15 +CPU= 23.129
  20:53:11 STEP 1754 H= 438:30 +CPU= 24.545
  20:53:34 STEP 1755 H= 438:45 +CPU= 22.663
  20:53:56 STEP 1756 H= 439:00 +CPU= 21.973
  20:54:19 STEP 1757 H= 439:15 +CPU= 23.216
  20:54:42 STEP 1758 H= 439:30 +CPU= 22.236
  20:55:03 STEP 1759 H= 439:45 +CPU= 21.436
  20:55:26 STEP 1760 H= 440:00 +CPU= 22.518
  20:55:48 STEP 1761 H= 440:15 +CPU= 21.815
  20:56:11 STEP 1762 H= 440:30 +CPU= 23.270
  20:56:35 STEP 1763 H= 440:45 +CPU= 24.190
  20:57:04 STEP 1764 H= 441:00 +CPU= 28.699
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001764
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001764
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001764
  20:57:27 STEP 1765 H= 441:15 +CPU= 22.649
  20:57:51 STEP 1766 H= 441:30 +CPU= 24.218
  20:58:13 STEP 1767 H= 441:45 +CPU= 21.605
  20:58:37 STEP 1768 H= 442:00 +CPU= 24.536
  20:59:00 STEP 1769 H= 442:15 +CPU= 22.524
  20:59:23 STEP 1770 H= 442:30 +CPU= 23.089
  20:59:47 STEP 1771 H= 442:45 +CPU= 23.971
  21:00:10 STEP 1772 H= 443:00 +CPU= 22.941
  21:00:33 STEP 1773 H= 443:15 +CPU= 23.347
  21:00:57 STEP 1774 H= 443:30 +CPU= 23.909
  21:01:20 STEP 1775 H= 443:45 +CPU= 22.278
  21:01:53 STEP 1776 H= 444:00 +CPU= 33.156
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001776
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001776
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001776
  21:02:16 STEP 1777 H= 444:15 +CPU= 22.750
  21:02:37 STEP 1778 H= 444:30 +CPU= 21.067
  21:02:59 STEP 1779 H= 444:45 +CPU= 22.244
  21:03:21 STEP 1780 H= 445:00 +CPU= 21.437
  21:03:43 STEP 1781 H= 445:15 +CPU= 21.824
  21:04:06 STEP 1782 H= 445:30 +CPU= 23.159
  21:04:28 STEP 1783 H= 445:45 +CPU= 21.895
  21:04:51 STEP 1784 H= 446:00 +CPU= 22.923
  21:05:12 STEP 1785 H= 446:15 +CPU= 21.574
  21:05:35 STEP 1786 H= 446:30 +CPU= 22.434
  21:06:00 STEP 1787 H= 446:45 +CPU= 25.055
  21:06:29 STEP 1788 H= 447:00 +CPU= 28.998
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001788
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001788
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001788
  21:06:52 STEP 1789 H= 447:15 +CPU= 23.222
  21:07:13 STEP 1790 H= 447:30 +CPU= 21.094
  21:07:36 STEP 1791 H= 447:45 +CPU= 22.741
  21:08:00 STEP 1792 H= 448:00 +CPU= 23.366
  21:08:21 STEP 1793 H= 448:15 +CPU= 20.858
  21:08:44 STEP 1794 H= 448:30 +CPU= 23.039
  21:09:06 STEP 1795 H= 448:45 +CPU= 22.265
  21:09:29 STEP 1796 H= 449:00 +CPU= 23.286
  21:09:54 STEP 1797 H= 449:15 +CPU= 24.942
  21:10:18 STEP 1798 H= 449:30 +CPU= 23.331
  21:10:41 STEP 1799 H= 449:45 +CPU= 22.952
  21:11:10 STEP 1800 H= 450:00 +CPU= 29.446
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001800
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001800
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001800
  21:11:34 STEP 1801 H= 450:15 +CPU= 23.519
  21:11:56 STEP 1802 H= 450:30 +CPU= 21.623
  21:12:19 STEP 1803 H= 450:45 +CPU= 23.245
  21:12:44 STEP 1804 H= 451:00 +CPU= 25.113
  21:13:10 STEP 1805 H= 451:15 +CPU= 25.865
  21:13:34 STEP 1806 H= 451:30 +CPU= 23.916
  21:13:58 STEP 1807 H= 451:45 +CPU= 23.509
  21:14:21 STEP 1808 H= 452:00 +CPU= 23.338
  21:14:43 STEP 1809 H= 452:15 +CPU= 21.972
  21:15:06 STEP 1810 H= 452:30 +CPU= 22.834
  21:15:27 STEP 1811 H= 452:45 +CPU= 20.911
  21:15:54 STEP 1812 H= 453:00 +CPU= 27.479
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001812
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001812
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001812
  21:16:17 STEP 1813 H= 453:15 +CPU= 22.712
  21:16:38 STEP 1814 H= 453:30 +CPU= 21.370
  21:17:03 STEP 1815 H= 453:45 +CPU= 24.825
  21:17:29 STEP 1816 H= 454:00 +CPU= 25.415
  21:17:52 STEP 1817 H= 454:15 +CPU= 23.409
  21:18:16 STEP 1818 H= 454:30 +CPU= 23.350
  21:18:39 STEP 1819 H= 454:45 +CPU= 23.275
  21:19:04 STEP 1820 H= 455:00 +CPU= 25.020
  21:19:29 STEP 1821 H= 455:15 +CPU= 24.664
  21:19:53 STEP 1822 H= 455:30 +CPU= 24.341
  21:20:17 STEP 1823 H= 455:45 +CPU= 23.978
  21:20:50 STEP 1824 H= 456:00 +CPU= 32.641
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001824
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001824
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001812
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001812
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001812
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0_r60511044_18.zip
Uploading the intermediate file: upload_file_18.zip
  21:21:15 STEP 1825 H= 456:15 +CPU= 25.388
  21:21:41 STEP 1826 H= 456:30 +CPU= 25.219
  21:22:06 STEP 1827 H= 456:45 +CPU= 24.887
  21:22:31 STEP 1828 H= 457:00 +CPU= 25.272
  21:22:54 STEP 1829 H= 457:15 +CPU= 22.781
  21:23:17 STEP 1830 H= 457:30 +CPU= 22.888
  21:23:40 STEP 1831 H= 457:45 +CPU= 22.751
  21:24:05 STEP 1832 H= 458:00 +CPU= 24.821
  21:24:29 STEP 1833 H= 458:15 +CPU= 24.140
  21:24:52 STEP 1834 H= 458:30 +CPU= 23.380
  21:25:17 STEP 1835 H= 458:45 +CPU= 24.401
  21:25:47 STEP 1836 H= 459:00 +CPU= 30.121
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001836
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001836
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001836
  21:26:10 STEP 1837 H= 459:15 +CPU= 22.626
  21:26:33 STEP 1838 H= 459:30 +CPU= 23.458
  21:26:56 STEP 1839 H= 459:45 +CPU= 23.008
  21:27:21 STEP 1840 H= 460:00 +CPU= 24.600
  21:27:43 STEP 1841 H= 460:15 +CPU= 22.520
  21:28:07 STEP 1842 H= 460:30 +CPU= 23.690
  21:28:31 STEP 1843 H= 460:45 +CPU= 24.068
  21:28:57 STEP 1844 H= 461:00 +CPU= 25.492
  21:29:21 STEP 1845 H= 461:15 +CPU= 24.155
  21:29:45 STEP 1846 H= 461:30 +CPU= 24.266
  21:30:08 STEP 1847 H= 461:45 +CPU= 22.689
  21:30:37 STEP 1848 H= 462:00 +CPU= 28.565
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001848
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001848
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001848
  21:31:01 STEP 1849 H= 462:15 +CPU= 23.692
  21:31:25 STEP 1850 H= 462:30 +CPU= 24.430
  21:31:49 STEP 1851 H= 462:45 +CPU= 23.827
  21:32:13 STEP 1852 H= 463:00 +CPU= 23.982
  21:32:37 STEP 1853 H= 463:15 +CPU= 23.412
  21:33:01 STEP 1854 H= 463:30 +CPU= 24.385
  21:33:22 STEP 1855 H= 463:45 +CPU= 21.143
  21:33:45 STEP 1856 H= 464:00 +CPU= 22.658
  21:34:08 STEP 1857 H= 464:15 +CPU= 23.274
  21:34:31 STEP 1858 H= 464:30 +CPU= 22.614
  21:34:55 STEP 1859 H= 464:45 +CPU= 23.639
  21:35:24 STEP 1860 H= 465:00 +CPU= 29.033
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001860
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001860
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001860
  21:35:48 STEP 1861 H= 465:15 +CPU= 24.451
  21:36:11 STEP 1862 H= 465:30 +CPU= 22.519
  21:36:35 STEP 1863 H= 465:45 +CPU= 23.723
  21:36:58 STEP 1864 H= 466:00 +CPU= 23.440
  21:37:21 STEP 1865 H= 466:15 +CPU= 22.646
  21:37:43 STEP 1866 H= 466:30 +CPU= 22.380
  21:38:07 STEP 1867 H= 466:45 +CPU= 24.078
  21:38:31 STEP 1868 H= 467:00 +CPU= 23.300
  21:38:53 STEP 1869 H= 467:15 +CPU= 21.859
  21:39:15 STEP 1870 H= 467:30 +CPU= 22.426
  21:39:39 STEP 1871 H= 467:45 +CPU= 23.993
  21:40:13 STEP 1872 H= 468:00 +CPU= 33.905
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001872
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001872
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001872
  21:40:36 STEP 1873 H= 468:15 +CPU= 22.662
  21:41:01 STEP 1874 H= 468:30 +CPU= 24.504
  21:41:24 STEP 1875 H= 468:45 +CPU= 23.245
  21:41:47 STEP 1876 H= 469:00 +CPU= 23.423
  21:42:11 STEP 1877 H= 469:15 +CPU= 23.933
  21:42:34 STEP 1878 H= 469:30 +CPU= 22.306
  21:42:56 STEP 1879 H= 469:45 +CPU= 22.594
  21:43:20 STEP 1880 H= 470:00 +CPU= 23.703
  21:43:43 STEP 1881 H= 470:15 +CPU= 22.834
  21:44:07 STEP 1882 H= 470:30 +CPU= 23.756
  21:44:31 STEP 1883 H= 470:45 +CPU= 23.630
  21:45:01 STEP 1884 H= 471:00 +CPU= 30.612
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001884
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001884
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001884
  21:45:26 STEP 1885 H= 471:15 +CPU= 25.004
  21:45:51 STEP 1886 H= 471:30 +CPU= 24.686
  21:46:15 STEP 1887 H= 471:45 +CPU= 24.358
  21:46:40 STEP 1888 H= 472:00 +CPU= 24.040
  21:47:03 STEP 1889 H= 472:15 +CPU= 23.697
  21:47:28 STEP 1890 H= 472:30 +CPU= 24.320
  21:47:52 STEP 1891 H= 472:45 +CPU= 24.116
  21:48:16 STEP 1892 H= 473:00 +CPU= 24.038
  21:48:39 STEP 1893 H= 473:15 +CPU= 23.223
  21:49:03 STEP 1894 H= 473:30 +CPU= 23.703
  21:49:24 STEP 1895 H= 473:45 +CPU= 21.402
  21:49:53 STEP 1896 H= 474:00 +CPU= 28.802
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001896
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001896
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001896
  21:50:16 STEP 1897 H= 474:15 +CPU= 22.553
  21:50:41 STEP 1898 H= 474:30 +CPU= 24.479
  21:51:05 STEP 1899 H= 474:45 +CPU= 24.405
  21:51:31 STEP 1900 H= 475:00 +CPU= 25.705
  21:51:55 STEP 1901 H= 475:15 +CPU= 23.888
  21:52:20 STEP 1902 H= 475:30 +CPU= 25.122
  21:52:43 STEP 1903 H= 475:45 +CPU= 23.457
  21:53:08 STEP 1904 H= 476:00 +CPU= 24.511
  21:53:31 STEP 1905 H= 476:15 +CPU= 22.973
  21:53:55 STEP 1906 H= 476:30 +CPU= 24.391
  21:54:20 STEP 1907 H= 476:45 +CPU= 24.986
  21:54:49 STEP 1908 H= 477:00 +CPU= 28.883
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001908
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001908
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001908
  21:55:13 STEP 1909 H= 477:15 +CPU= 23.342
  21:55:36 STEP 1910 H= 477:30 +CPU= 23.350
  21:56:00 STEP 1911 H= 477:45 +CPU= 23.406
  21:56:26 STEP 1912 H= 478:00 +CPU= 26.036
  21:56:51 STEP 1913 H= 478:15 +CPU= 24.664
  21:57:15 STEP 1914 H= 478:30 +CPU= 24.527
  21:57:38 STEP 1915 H= 478:45 +CPU= 22.957
  21:58:03 STEP 1916 H= 479:00 +CPU= 24.602
  21:58:26 STEP 1917 H= 479:15 +CPU= 22.961
  21:58:49 STEP 1918 H= 479:30 +CPU= 23.076
  21:59:12 STEP 1919 H= 479:45 +CPU= 22.943
  21:59:42 STEP 1920 H= 480:00 +CPU= 29.958
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.6045708509E-21      I       0.7908715273E-06      I
ICloud fractionI      0.0000000000E+00      I       0.9153225890E-02      I
I  Relat. Hum. I      0.4854968551E+00      I       0.4955321709E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1541994031E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    315662756 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS631325.5 Mbytes/s, TOTAL TIME=    0.0091( 5.5%)
 21:59:42 STEP 1920 H= 480:00 +CPU= 29.958
 END CNT3
 NSTEP =  1920 CNT0   000000000  22.472  22.472  22.526   0.998               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        57546/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1008    0        0            0    57546 ************    72.6     0.0    72.6     0.0   158.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.99      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 2082    4026.8    4026.8     18.17      0.00
   9 SPCM     - SPECTRAL COMP.                    1920     709.7     709.7      2.95      0.00
  10 SCAN2M - PHYSICS                             1921   11078.3   11078.3     46.13      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                161     650.1     650.1      0.23      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 83      50.8      50.8      0.01      0.00
  14 SUINIF                                          1    3867.4    3867.4      0.01      0.00
  17 GRIDFPOS IN CNT4                              161      26.2      26.2      0.01      0.00
  18 SUSPECG                                         1    2220.7    2220.7      0.00      0.00
  19 SUSPEC                                          1    2224.9    2224.9      0.00      0.00
  24 SUGRIDU                                         1    1340.8    1340.8      0.00      0.00
  25 SPECRT                                          1     209.4     209.4      0.00      0.00
  26 SUGRIDF                                         1      92.3      92.3      0.00      0.00
  27 RESTART FILES - WRITING                        40    2616.5    2616.5      0.23      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               161       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                               161    5172.8    5172.8      1.81      0.00
  31 POSDDH IN STEPO                                 9       0.7       0.7      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1921    5133.5    5133.5     21.38      0.00
  39 SU0YOMB                                         1     259.0     259.0      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1921      19.6      19.6      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1920     335.8     335.8      1.40      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1920      45.5      45.5      0.19      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 83       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     4165     323.2     323.2      2.92      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      4006     209.7     209.7      1.82      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       4006      75.0      75.0      0.65      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      4165      99.2      99.2      0.90      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      37.8      37.8      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           4165      45.3      45.3      0.41      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           4006      23.6      23.6      0.20      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           4165      66.3      66.3      0.60      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           4006      31.1      31.1      0.27      0.00
 400 GSTATS                                     223224       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                207696       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    46134.5 CPU TIME   46012.3 VECTOR TIME    46012.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       37.5
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMGGh7zg+001920
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMSHh7zg+001920
Moving to projects directory: /home/pdw/Mo/1/slots/3/ICMUAh7zg+001920
Adding to the zip: /home/pdw/Mo/1/slots/3/NODE.001_01
Adding to the zip: /home/pdw/Mo/1/slots/3/ifs.stat
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001860
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001860
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMUAh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMSHh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12290894/ICMGGh7zg+001860
Zipping up the final file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0_r60511044_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
22:03:10 (217106): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Jun 2024 21:02:04 1522755 22444492 oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0 1,727,100 46,012 0.0266
18 Jun 2024 14:37:19 1522755 22444492 oifs_43r3_bl_a1zb_2016092300_20_1018_12290894_0 864,000 23,066 0.0267


©2024 cpdn.org