climateprediction.net home page
Task 22316388

Task 22316388

Name oifs_43r3_0179_2019110100_123_993_12214995_0
Workunit 12214995
Created 24 Feb 2023, 17:00:29 UTC
Sent 24 Feb 2023, 17:03:55 UTC
Report deadline 25 Apr 2023, 17:03:55 UTC
Received 25 Feb 2023, 8:14:03 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1498009
Run time 13 hours 39 min 53 sec
CPU time 13 hours 35 min 45 sec
Validate state Invalid
Credit 14,873.04
Device peak FLOPS 7.33 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,416.22 MB
Peak swap size 4,974.23 MB
Peak disk usage 1,279.91 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
ip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002340
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002340
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_97.zip
Uploading the intermediate file: upload_file_97.zip
  04:47:03 STEP 2353 H=2353:00 +CPU= 13.951
  04:47:17 STEP 2354 H=2354:00 +CPU= 14.023
  04:47:38 STEP 2355 H=2355:00 +CPU= 21.125
  04:47:52 STEP 2356 H=2356:00 +CPU= 14.371
  04:48:06 STEP 2357 H=2357:00 +CPU= 13.491
  04:48:27 STEP 2358 H=2358:00 +CPU= 21.309
  04:48:41 STEP 2359 H=2359:00 +CPU= 13.963
  04:48:55 STEP 2360 H=2360:00 +CPU= 13.986
  04:49:16 STEP 2361 H=2361:00 +CPU= 20.965
  04:49:31 STEP 2362 H=2362:00 +CPU= 14.455
  04:49:44 STEP 2363 H=2363:00 +CPU= 13.667
  04:50:07 STEP 2364 H=2364:00 +CPU= 22.669
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002364
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002364
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002364
  04:50:21 STEP 2365 H=2365:00 +CPU= 13.923
  04:50:35 STEP 2366 H=2366:00 +CPU= 14.024
  04:50:56 STEP 2367 H=2367:00 +CPU= 20.727
  04:51:10 STEP 2368 H=2368:00 +CPU= 14.300
  04:51:24 STEP 2369 H=2369:00 +CPU= 13.595
  04:51:45 STEP 2370 H=2370:00 +CPU= 21.079
  04:51:59 STEP 2371 H=2371:00 +CPU= 14.012
  04:52:13 STEP 2372 H=2372:00 +CPU= 14.080
  04:52:33 STEP 2373 H=2373:00 +CPU= 20.632
  04:52:48 STEP 2374 H=2374:00 +CPU= 14.478
  04:53:02 STEP 2375 H=2375:00 +CPU= 14.342
  04:53:27 STEP 2376 H=2376:00 +CPU= 24.449
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002376
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002376
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002352
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002352
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002352
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002364
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002364
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002364
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_98.zip
Uploading the intermediate file: upload_file_98.zip
  04:53:41 STEP 2377 H=2377:00 +CPU= 14.066
  04:53:55 STEP 2378 H=2378:00 +CPU= 14.111
  04:54:16 STEP 2379 H=2379:00 +CPU= 21.281
  04:54:31 STEP 2380 H=2380:00 +CPU= 14.575
  04:54:45 STEP 2381 H=2381:00 +CPU= 13.651
  04:55:06 STEP 2382 H=2382:00 +CPU= 21.309
  04:55:20 STEP 2383 H=2383:00 +CPU= 14.018
  04:55:34 STEP 2384 H=2384:00 +CPU= 13.990
  04:55:55 STEP 2385 H=2385:00 +CPU= 21.020
  04:56:09 STEP 2386 H=2386:00 +CPU= 14.430
  04:56:23 STEP 2387 H=2387:00 +CPU= 13.709
  04:56:46 STEP 2388 H=2388:00 +CPU= 22.713
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002388
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002388
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002388
  04:57:00 STEP 2389 H=2389:00 +CPU= 14.033
  04:57:14 STEP 2390 H=2390:00 +CPU= 14.109
  04:57:35 STEP 2391 H=2391:00 +CPU= 20.884
  04:57:49 STEP 2392 H=2392:00 +CPU= 14.364
  04:58:03 STEP 2393 H=2393:00 +CPU= 13.560
  04:58:24 STEP 2394 H=2394:00 +CPU= 21.264
  04:58:38 STEP 2395 H=2395:00 +CPU= 14.051
  04:58:52 STEP 2396 H=2396:00 +CPU= 13.963
  04:59:13 STEP 2397 H=2397:00 +CPU= 20.672
  04:59:27 STEP 2398 H=2398:00 +CPU= 14.321
  04:59:41 STEP 2399 H=2399:00 +CPU= 14.168
  05:00:06 STEP 2400 H=2400:00 +CPU= 24.497
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002400
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002400
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002388
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002388
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002388
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_99.zip
Uploading the intermediate file: upload_file_99.zip
  05:00:20 STEP 2401 H=2401:00 +CPU= 13.964
Uploading trickle at timestep: 8640000
  05:00:34 STEP 2402 H=2402:00 +CPU= 13.906
  05:00:55 STEP 2403 H=2403:00 +CPU= 21.195
  05:01:09 STEP 2404 H=2404:00 +CPU= 14.311
  05:01:23 STEP 2405 H=2405:00 +CPU= 13.506
  05:01:45 STEP 2406 H=2406:00 +CPU= 21.749
  05:01:59 STEP 2407 H=2407:00 +CPU= 14.024
  05:02:13 STEP 2408 H=2408:00 +CPU= 13.901
  05:02:34 STEP 2409 H=2409:00 +CPU= 21.032
  05:02:48 STEP 2410 H=2410:00 +CPU= 14.295
  05:03:02 STEP 2411 H=2411:00 +CPU= 13.572
  05:03:24 STEP 2412 H=2412:00 +CPU= 22.712
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002412
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002412
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002412
  05:03:38 STEP 2413 H=2413:00 +CPU= 14.074
  05:03:52 STEP 2414 H=2414:00 +CPU= 14.032
  05:04:13 STEP 2415 H=2415:00 +CPU= 20.996
  05:04:28 STEP 2416 H=2416:00 +CPU= 14.333
  05:04:41 STEP 2417 H=2417:00 +CPU= 13.550
  05:05:03 STEP 2418 H=2418:00 +CPU= 21.252
  05:05:17 STEP 2419 H=2419:00 +CPU= 14.077
  05:05:31 STEP 2420 H=2420:00 +CPU= 13.942
  05:05:51 STEP 2421 H=2421:00 +CPU= 20.705
  05:06:06 STEP 2422 H=2422:00 +CPU= 14.398
  05:06:20 STEP 2423 H=2423:00 +CPU= 14.224
  05:06:44 STEP 2424 H=2424:00 +CPU= 24.442
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002424
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002424
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002412
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002412
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002412
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_100.zip
Uploading the intermediate file: upload_file_100.zip
  05:06:59 STEP 2425 H=2425:00 +CPU= 14.090
  05:07:12 STEP 2426 H=2426:00 +CPU= 13.908
  05:07:34 STEP 2427 H=2427:00 +CPU= 21.129
  05:07:48 STEP 2428 H=2428:00 +CPU= 14.423
  05:08:02 STEP 2429 H=2429:00 +CPU= 13.562
  05:08:23 STEP 2430 H=2430:00 +CPU= 21.313
  05:08:37 STEP 2431 H=2431:00 +CPU= 14.125
  05:08:51 STEP 2432 H=2432:00 +CPU= 13.935
  05:09:12 STEP 2433 H=2433:00 +CPU= 21.028
  05:09:27 STEP 2434 H=2434:00 +CPU= 14.440
  05:09:40 STEP 2435 H=2435:00 +CPU= 13.580
  05:10:03 STEP 2436 H=2436:00 +CPU= 22.837
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002436
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002436
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002436
  05:10:17 STEP 2437 H=2437:00 +CPU= 14.040
  05:10:31 STEP 2438 H=2438:00 +CPU= 13.987
  05:10:52 STEP 2439 H=2439:00 +CPU= 21.051
  05:11:06 STEP 2440 H=2440:00 +CPU= 14.341
  05:11:20 STEP 2441 H=2441:00 +CPU= 13.442
  05:11:41 STEP 2442 H=2442:00 +CPU= 21.355
  05:11:55 STEP 2443 H=2443:00 +CPU= 14.024
  05:12:09 STEP 2444 H=2444:00 +CPU= 13.913
  05:12:30 STEP 2445 H=2445:00 +CPU= 20.775
  05:12:44 STEP 2446 H=2446:00 +CPU= 14.431
  05:12:59 STEP 2447 H=2447:00 +CPU= 14.169
  05:13:23 STEP 2448 H=2448:00 +CPU= 24.405
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002448
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002448
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002436
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002436
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002436
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_101.zip
Uploading the intermediate file: upload_file_101.zip
  05:13:37 STEP 2449 H=2449:00 +CPU= 13.944
  05:13:51 STEP 2450 H=2450:00 +CPU= 13.844
  05:14:12 STEP 2451 H=2451:00 +CPU= 21.122
  05:14:26 STEP 2452 H=2452:00 +CPU= 14.384
  05:14:40 STEP 2453 H=2453:00 +CPU= 13.442
  05:15:01 STEP 2454 H=2454:00 +CPU= 21.462
  05:15:15 STEP 2455 H=2455:00 +CPU= 14.022
  05:15:29 STEP 2456 H=2456:00 +CPU= 13.926
  05:15:50 STEP 2457 H=2457:00 +CPU= 21.034
  05:16:05 STEP 2458 H=2458:00 +CPU= 14.424
  05:16:18 STEP 2459 H=2459:00 +CPU= 13.648
  05:16:41 STEP 2460 H=2460:00 +CPU= 22.732
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002460
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002460
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002460
  05:16:55 STEP 2461 H=2461:00 +CPU= 13.934
  05:17:09 STEP 2462 H=2462:00 +CPU= 13.853
  05:17:30 STEP 2463 H=2463:00 +CPU= 20.953
  05:17:44 STEP 2464 H=2464:00 +CPU= 14.248
  05:17:58 STEP 2465 H=2465:00 +CPU= 13.495
  05:18:19 STEP 2466 H=2466:00 +CPU= 21.235
  05:18:33 STEP 2467 H=2467:00 +CPU= 14.002
  05:18:47 STEP 2468 H=2468:00 +CPU= 13.947
  05:19:08 STEP 2469 H=2469:00 +CPU= 20.707
  05:19:22 STEP 2470 H=2470:00 +CPU= 14.353
  05:19:36 STEP 2471 H=2471:00 +CPU= 14.345
  05:20:01 STEP 2472 H=2472:00 +CPU= 24.519
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002472
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002472
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002460
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002460
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002460
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_102.zip
Uploading the intermediate file: upload_file_102.zip
  05:20:15 STEP 2473 H=2473:00 +CPU= 13.972
  05:20:29 STEP 2474 H=2474:00 +CPU= 14.017
  05:20:50 STEP 2475 H=2475:00 +CPU= 21.328
  05:21:05 STEP 2476 H=2476:00 +CPU= 14.526
  05:21:18 STEP 2477 H=2477:00 +CPU= 13.596
  05:21:40 STEP 2478 H=2478:00 +CPU= 21.451
  05:21:54 STEP 2479 H=2479:00 +CPU= 14.077
  05:22:08 STEP 2480 H=2480:00 +CPU= 14.005
  05:22:29 STEP 2481 H=2481:00 +CPU= 21.061
  05:22:44 STEP 2482 H=2482:00 +CPU= 14.464
  05:22:57 STEP 2483 H=2483:00 +CPU= 13.736
  05:23:20 STEP 2484 H=2484:00 +CPU= 22.783
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002484
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002484
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002484
  05:23:34 STEP 2485 H=2485:00 +CPU= 14.038
  05:23:48 STEP 2486 H=2486:00 +CPU= 14.097
  05:24:09 STEP 2487 H=2487:00 +CPU= 21.075
  05:24:24 STEP 2488 H=2488:00 +CPU= 14.442
  05:24:37 STEP 2489 H=2489:00 +CPU= 13.580
  05:24:59 STEP 2490 H=2490:00 +CPU= 21.373
  05:25:13 STEP 2491 H=2491:00 +CPU= 14.039
  05:25:27 STEP 2492 H=2492:00 +CPU= 14.011
  05:25:48 STEP 2493 H=2493:00 +CPU= 20.851
  05:26:02 STEP 2494 H=2494:00 +CPU= 14.367
  05:26:16 STEP 2495 H=2495:00 +CPU= 14.198
  05:26:41 STEP 2496 H=2496:00 +CPU= 24.275
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002496
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002496
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002484
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002484
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002484
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_103.zip
Uploading the intermediate file: upload_file_103.zip
  05:26:55 STEP 2497 H=2497:00 +CPU= 13.907
  05:27:08 STEP 2498 H=2498:00 +CPU= 13.830
  05:27:29 STEP 2499 H=2499:00 +CPU= 20.997
  05:27:44 STEP 2500 H=2500:00 +CPU= 14.297
  05:27:57 STEP 2501 H=2501:00 +CPU= 13.378
  05:28:19 STEP 2502 H=2502:00 +CPU= 21.391
  05:28:33 STEP 2503 H=2503:00 +CPU= 14.016
  05:28:47 STEP 2504 H=2504:00 +CPU= 13.923
  05:29:08 STEP 2505 H=2505:00 +CPU= 21.157
  05:29:22 STEP 2506 H=2506:00 +CPU= 14.433
  05:29:36 STEP 2507 H=2507:00 +CPU= 13.704
  05:29:59 STEP 2508 H=2508:00 +CPU= 22.764
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002508
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002508
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002508
  05:30:13 STEP 2509 H=2509:00 +CPU= 13.978
  05:30:27 STEP 2510 H=2510:00 +CPU= 13.985
  05:30:48 STEP 2511 H=2511:00 +CPU= 20.900
  05:31:02 STEP 2512 H=2512:00 +CPU= 14.235
  05:31:15 STEP 2513 H=2513:00 +CPU= 13.548
  05:31:36 STEP 2514 H=2514:00 +CPU= 21.107
  05:31:50 STEP 2515 H=2515:00 +CPU= 13.969
  05:32:04 STEP 2516 H=2516:00 +CPU= 14.024
  05:32:25 STEP 2517 H=2517:00 +CPU= 20.708
  05:32:40 STEP 2518 H=2518:00 +CPU= 14.382
  05:32:54 STEP 2519 H=2519:00 +CPU= 14.424
  05:33:18 STEP 2520 H=2520:00 +CPU= 24.303
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002520
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002520
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002508
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002508
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002508
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_104.zip
Uploading the intermediate file: upload_file_104.zip
  05:33:32 STEP 2521 H=2521:00 +CPU= 14.024
  05:33:46 STEP 2522 H=2522:00 +CPU= 13.986
  05:34:08 STEP 2523 H=2523:00 +CPU= 21.243
  05:34:22 STEP 2524 H=2524:00 +CPU= 14.447
  05:34:36 STEP 2525 H=2525:00 +CPU= 13.740
  05:34:57 STEP 2526 H=2526:00 +CPU= 21.210
  05:35:11 STEP 2527 H=2527:00 +CPU= 13.995
  05:35:25 STEP 2528 H=2528:00 +CPU= 14.075
  05:35:46 STEP 2529 H=2529:00 +CPU= 21.046
  05:36:01 STEP 2530 H=2530:00 +CPU= 14.553
  05:36:15 STEP 2531 H=2531:00 +CPU= 13.788
  05:36:37 STEP 2532 H=2532:00 +CPU= 22.702
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002532
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002532
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002532
  05:36:51 STEP 2533 H=2533:00 +CPU= 13.869
  05:37:05 STEP 2534 H=2534:00 +CPU= 14.023
  05:37:26 STEP 2535 H=2535:00 +CPU= 20.722
  05:37:40 STEP 2536 H=2536:00 +CPU= 14.287
  05:37:54 STEP 2537 H=2537:00 +CPU= 13.664
  05:38:15 STEP 2538 H=2538:00 +CPU= 21.094
  05:38:29 STEP 2539 H=2539:00 +CPU= 14.058
  05:38:43 STEP 2540 H=2540:00 +CPU= 14.069
  05:39:04 STEP 2541 H=2541:00 +CPU= 20.563
  05:39:18 STEP 2542 H=2542:00 +CPU= 14.449
  05:39:33 STEP 2543 H=2543:00 +CPU= 14.322
  05:39:57 STEP 2544 H=2544:00 +CPU= 24.346
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002544
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002544
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002532
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002532
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002532
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_105.zip
Uploading the intermediate file: upload_file_105.zip
  05:40:11 STEP 2545 H=2545:00 +CPU= 14.202
  05:40:25 STEP 2546 H=2546:00 +CPU= 14.158
  05:40:46 STEP 2547 H=2547:00 +CPU= 21.119
  05:41:01 STEP 2548 H=2548:00 +CPU= 14.409
  05:41:14 STEP 2549 H=2549:00 +CPU= 13.605
  05:41:36 STEP 2550 H=2550:00 +CPU= 21.220
  05:41:50 STEP 2551 H=2551:00 +CPU= 14.029
  05:42:04 STEP 2552 H=2552:00 +CPU= 14.037
  05:42:25 STEP 2553 H=2553:00 +CPU= 21.059
  05:42:39 STEP 2554 H=2554:00 +CPU= 14.492
  05:42:53 STEP 2555 H=2555:00 +CPU= 13.778
  05:43:16 STEP 2556 H=2556:00 +CPU= 22.682
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002556
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002556
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002556
  05:43:30 STEP 2557 H=2557:00 +CPU= 14.021
  05:43:44 STEP 2558 H=2558:00 +CPU= 14.105
  05:44:05 STEP 2559 H=2559:00 +CPU= 20.785
  05:44:19 STEP 2560 H=2560:00 +CPU= 14.357
  05:44:33 STEP 2561 H=2561:00 +CPU= 13.660
  05:44:54 STEP 2562 H=2562:00 +CPU= 21.155
  05:45:08 STEP 2563 H=2563:00 +CPU= 14.042
  05:45:22 STEP 2564 H=2564:00 +CPU= 14.049
  05:45:43 STEP 2565 H=2565:00 +CPU= 20.657
  05:45:57 STEP 2566 H=2566:00 +CPU= 14.374
  05:46:12 STEP 2567 H=2567:00 +CPU= 14.350
  05:46:36 STEP 2568 H=2568:00 +CPU= 24.346
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002568
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002568
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002556
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002556
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002556
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_106.zip
Uploading the intermediate file: upload_file_106.zip
  05:46:50 STEP 2569 H=2569:00 +CPU= 14.138
  05:47:04 STEP 2570 H=2570:00 +CPU= 14.085
  05:47:25 STEP 2571 H=2571:00 +CPU= 21.131
  05:47:40 STEP 2572 H=2572:00 +CPU= 14.375
  05:47:53 STEP 2573 H=2573:00 +CPU= 13.686
  05:48:15 STEP 2574 H=2574:00 +CPU= 21.229
  05:48:28 STEP 2575 H=2575:00 +CPU= 12.887
  05:48:40 STEP 2576 H=2576:00 +CPU= 12.543
  05:48:59 STEP 2577 H=2577:00 +CPU= 18.865
  05:49:12 STEP 2578 H=2578:00 +CPU= 12.818
  05:49:24 STEP 2579 H=2579:00 +CPU= 12.298
  05:49:45 STEP 2580 H=2580:00 +CPU= 20.485
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002580
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002580
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002580
  05:49:57 STEP 2581 H=2581:00 +CPU= 12.502
  05:50:10 STEP 2582 H=2582:00 +CPU= 12.637
  05:50:29 STEP 2583 H=2583:00 +CPU= 18.851
  05:50:41 STEP 2584 H=2584:00 +CPU= 12.780
  05:50:54 STEP 2585 H=2585:00 +CPU= 12.213
  05:51:13 STEP 2586 H=2586:00 +CPU= 19.170
  05:51:25 STEP 2587 H=2587:00 +CPU= 12.451
  05:51:38 STEP 2588 H=2588:00 +CPU= 12.585
  05:51:58 STEP 2589 H=2589:00 +CPU= 19.885
  05:52:12 STEP 2590 H=2590:00 +CPU= 14.279
  05:52:26 STEP 2591 H=2591:00 +CPU= 14.338
  05:52:51 STEP 2592 H=2592:00 +CPU= 24.139
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002592
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002592
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002580
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002580
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002580
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_107.zip
Uploading the intermediate file: upload_file_107.zip
  05:53:05 STEP 2593 H=2593:00 +CPU= 13.996
  05:53:19 STEP 2594 H=2594:00 +CPU= 14.139
  05:53:40 STEP 2595 H=2595:00 +CPU= 20.869
  05:53:54 STEP 2596 H=2596:00 +CPU= 14.214
  05:54:08 STEP 2597 H=2597:00 +CPU= 13.696
  05:54:29 STEP 2598 H=2598:00 +CPU= 21.156
  05:54:43 STEP 2599 H=2599:00 +CPU= 13.907
  05:54:57 STEP 2600 H=2600:00 +CPU= 14.167
  05:55:18 STEP 2601 H=2601:00 +CPU= 20.788
  05:55:32 STEP 2602 H=2602:00 +CPU= 14.188
  05:55:46 STEP 2603 H=2603:00 +CPU= 13.889
  05:56:08 STEP 2604 H=2604:00 +CPU= 22.560
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002604
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002604
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002604
  05:56:22 STEP 2605 H=2605:00 +CPU= 13.960
  05:56:37 STEP 2606 H=2606:00 +CPU= 14.353
  05:56:58 STEP 2607 H=2607:00 +CPU= 20.873
  05:57:12 STEP 2608 H=2608:00 +CPU= 14.201
  05:57:26 STEP 2609 H=2609:00 +CPU= 13.865
  05:57:47 STEP 2610 H=2610:00 +CPU= 21.112
  05:58:01 STEP 2611 H=2611:00 +CPU= 13.808
  05:58:15 STEP 2612 H=2612:00 +CPU= 14.193
  05:58:36 STEP 2613 H=2613:00 +CPU= 20.750
  05:58:50 STEP 2614 H=2614:00 +CPU= 13.974
  05:59:04 STEP 2615 H=2615:00 +CPU= 14.670
  05:59:27 STEP 2616 H=2616:00 +CPU= 23.077
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002616
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002616
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002604
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002604
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002604
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_108.zip
Uploading the intermediate file: upload_file_108.zip
  05:59:40 STEP 2617 H=2617:00 +CPU= 12.362
  05:59:52 STEP 2618 H=2618:00 +CPU= 12.593
  06:00:11 STEP 2619 H=2619:00 +CPU= 18.786
  06:00:24 STEP 2620 H=2620:00 +CPU= 12.639
  06:00:36 STEP 2621 H=2621:00 +CPU= 12.337
  06:00:55 STEP 2622 H=2622:00 +CPU= 19.183
  06:01:08 STEP 2623 H=2623:00 +CPU= 12.382
  06:01:20 STEP 2624 H=2624:00 +CPU= 12.655
  06:01:39 STEP 2625 H=2625:00 +CPU= 18.735
  06:01:52 STEP 2626 H=2626:00 +CPU= 12.477
  06:02:04 STEP 2627 H=2627:00 +CPU= 12.497
  06:02:24 STEP 2628 H=2628:00 +CPU= 20.321
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002628
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002628
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002628
  06:02:37 STEP 2629 H=2629:00 +CPU= 12.250
  06:02:50 STEP 2630 H=2630:00 +CPU= 13.077
  06:03:10 STEP 2631 H=2631:00 +CPU= 20.661
  06:03:25 STEP 2632 H=2632:00 +CPU= 14.141
  06:03:39 STEP 2633 H=2633:00 +CPU= 13.943
  06:03:59 STEP 2634 H=2634:00 +CPU= 20.904
  06:04:13 STEP 2635 H=2635:00 +CPU= 13.776
  06:04:28 STEP 2636 H=2636:00 +CPU= 14.430
  06:04:48 STEP 2637 H=2637:00 +CPU= 20.580
  06:05:02 STEP 2638 H=2638:00 +CPU= 14.056
  06:05:17 STEP 2639 H=2639:00 +CPU= 14.564
  06:05:41 STEP 2640 H=2640:00 +CPU= 24.048
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002640
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002640
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002628
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002628
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002628
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_109.zip
Uploading the intermediate file: upload_file_109.zip
  06:05:55 STEP 2641 H=2641:00 +CPU= 13.732
Uploading trickle at timestep: 9504000
  06:06:09 STEP 2642 H=2642:00 +CPU= 14.288
  06:06:30 STEP 2643 H=2643:00 +CPU= 20.634
  06:06:44 STEP 2644 H=2644:00 +CPU= 14.029
  06:06:58 STEP 2645 H=2645:00 +CPU= 14.114
  06:07:19 STEP 2646 H=2646:00 +CPU= 21.016
  06:07:33 STEP 2647 H=2647:00 +CPU= 13.722
  06:07:47 STEP 2648 H=2648:00 +CPU= 14.510
  06:08:08 STEP 2649 H=2649:00 +CPU= 20.659
  06:08:22 STEP 2650 H=2650:00 +CPU= 14.003
  06:08:36 STEP 2651 H=2651:00 +CPU= 14.001
  06:08:58 STEP 2652 H=2652:00 +CPU= 22.648
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002652
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002652
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002652
  06:09:12 STEP 2653 H=2653:00 +CPU= 13.677
  06:09:27 STEP 2654 H=2654:00 +CPU= 14.527
  06:09:47 STEP 2655 H=2655:00 +CPU= 20.743
  06:10:01 STEP 2656 H=2656:00 +CPU= 13.932
  06:10:15 STEP 2657 H=2657:00 +CPU= 13.972
  06:10:37 STEP 2658 H=2658:00 +CPU= 21.141
  06:10:50 STEP 2659 H=2659:00 +CPU= 13.533
  06:11:05 STEP 2660 H=2660:00 +CPU= 14.396
  06:11:25 STEP 2661 H=2661:00 +CPU= 20.568
  06:11:39 STEP 2662 H=2662:00 +CPU= 14.159
  06:11:54 STEP 2663 H=2663:00 +CPU= 14.335
  06:12:18 STEP 2664 H=2664:00 +CPU= 24.356
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002664
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002664
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002652
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002652
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002652
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_110.zip
Uploading the intermediate file: upload_file_110.zip
  06:12:32 STEP 2665 H=2665:00 +CPU= 13.826
  06:12:46 STEP 2666 H=2666:00 +CPU= 14.377
  06:13:07 STEP 2667 H=2667:00 +CPU= 20.838
  06:13:21 STEP 2668 H=2668:00 +CPU= 13.987
  06:13:35 STEP 2669 H=2669:00 +CPU= 14.024
  06:13:56 STEP 2670 H=2670:00 +CPU= 21.029
  06:14:10 STEP 2671 H=2671:00 +CPU= 13.638
  06:14:24 STEP 2672 H=2672:00 +CPU= 14.473
  06:14:45 STEP 2673 H=2673:00 +CPU= 20.684
  06:14:59 STEP 2674 H=2674:00 +CPU= 14.145
  06:15:13 STEP 2675 H=2675:00 +CPU= 13.900
  06:15:36 STEP 2676 H=2676:00 +CPU= 22.801
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002676
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002676
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002676
  06:15:50 STEP 2677 H=2677:00 +CPU= 13.867
  06:16:04 STEP 2678 H=2678:00 +CPU= 14.571
  06:16:25 STEP 2679 H=2679:00 +CPU= 21.055
  06:16:40 STEP 2680 H=2680:00 +CPU= 14.181
  06:16:54 STEP 2681 H=2681:00 +CPU= 14.179
  06:17:15 STEP 2682 H=2682:00 +CPU= 21.348
  06:17:29 STEP 2683 H=2683:00 +CPU= 13.711
  06:17:43 STEP 2684 H=2684:00 +CPU= 14.460
  06:18:04 STEP 2685 H=2685:00 +CPU= 20.682
  06:18:18 STEP 2686 H=2686:00 +CPU= 14.222
  06:18:33 STEP 2687 H=2687:00 +CPU= 14.367
  06:18:57 STEP 2688 H=2688:00 +CPU= 24.416
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002688
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002688
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002676
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002676
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002676
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_111.zip
Uploading the intermediate file: upload_file_111.zip
  06:19:11 STEP 2689 H=2689:00 +CPU= 13.799
  06:19:25 STEP 2690 H=2690:00 +CPU= 14.312
  06:19:46 STEP 2691 H=2691:00 +CPU= 20.895
  06:20:00 STEP 2692 H=2692:00 +CPU= 14.011
  06:20:14 STEP 2693 H=2693:00 +CPU= 14.064
  06:20:36 STEP 2694 H=2694:00 +CPU= 21.232
  06:20:49 STEP 2695 H=2695:00 +CPU= 13.704
  06:21:04 STEP 2696 H=2696:00 +CPU= 14.495
  06:21:25 STEP 2697 H=2697:00 +CPU= 20.825
  06:21:39 STEP 2698 H=2698:00 +CPU= 14.182
  06:21:53 STEP 2699 H=2699:00 +CPU= 14.071
  06:22:16 STEP 2700 H=2700:00 +CPU= 22.712
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002700
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002700
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002700
  06:22:29 STEP 2701 H=2701:00 +CPU= 13.681
  06:22:44 STEP 2702 H=2702:00 +CPU= 14.385
  06:23:04 STEP 2703 H=2703:00 +CPU= 20.802
  06:23:18 STEP 2704 H=2704:00 +CPU= 13.904
  06:23:32 STEP 2705 H=2705:00 +CPU= 13.998
  06:23:54 STEP 2706 H=2706:00 +CPU= 21.196
  06:24:07 STEP 2707 H=2707:00 +CPU= 13.637
  06:24:22 STEP 2708 H=2708:00 +CPU= 14.251
  06:24:42 STEP 2709 H=2709:00 +CPU= 20.570
  06:24:56 STEP 2710 H=2710:00 +CPU= 14.162
  06:25:11 STEP 2711 H=2711:00 +CPU= 14.341
  06:25:35 STEP 2712 H=2712:00 +CPU= 24.441
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002712
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002712
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002700
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002700
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002700
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_112.zip
Uploading the intermediate file: upload_file_112.zip
  06:25:49 STEP 2713 H=2713:00 +CPU= 13.897
  06:26:04 STEP 2714 H=2714:00 +CPU= 14.484
  06:26:25 STEP 2715 H=2715:00 +CPU= 20.974
  06:26:39 STEP 2716 H=2716:00 +CPU= 14.117
  06:26:53 STEP 2717 H=2717:00 +CPU= 14.167
  06:27:14 STEP 2718 H=2718:00 +CPU= 21.226
  06:27:28 STEP 2719 H=2719:00 +CPU= 13.780
  06:27:42 STEP 2720 H=2720:00 +CPU= 14.538
  06:28:03 STEP 2721 H=2721:00 +CPU= 20.820
  06:28:18 STEP 2722 H=2722:00 +CPU= 14.256
  06:28:32 STEP 2723 H=2723:00 +CPU= 14.178
  06:28:54 STEP 2724 H=2724:00 +CPU= 22.560
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002724
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002724
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002724
  06:29:08 STEP 2725 H=2725:00 +CPU= 13.743
  06:29:22 STEP 2726 H=2726:00 +CPU= 14.363
  06:29:43 STEP 2727 H=2727:00 +CPU= 20.788
  06:29:57 STEP 2728 H=2728:00 +CPU= 13.996
  06:30:11 STEP 2729 H=2729:00 +CPU= 14.029
  06:30:33 STEP 2730 H=2730:00 +CPU= 21.194
  06:30:46 STEP 2731 H=2731:00 +CPU= 13.607
  06:31:00 STEP 2732 H=2732:00 +CPU= 14.294
  06:31:21 STEP 2733 H=2733:00 +CPU= 20.547
  06:31:35 STEP 2734 H=2734:00 +CPU= 14.121
  06:31:50 STEP 2735 H=2735:00 +CPU= 14.382
  06:32:14 STEP 2736 H=2736:00 +CPU= 24.566
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002736
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002736
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002724
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002724
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002724
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_113.zip
Uploading the intermediate file: upload_file_113.zip
  06:32:28 STEP 2737 H=2737:00 +CPU= 13.870
  06:32:42 STEP 2738 H=2738:00 +CPU= 14.424
  06:33:04 STEP 2739 H=2739:00 +CPU= 21.070
  06:33:18 STEP 2740 H=2740:00 +CPU= 14.043
  06:33:32 STEP 2741 H=2741:00 +CPU= 14.124
  06:33:53 STEP 2742 H=2742:00 +CPU= 21.234
  06:34:07 STEP 2743 H=2743:00 +CPU= 13.715
  06:34:21 STEP 2744 H=2744:00 +CPU= 14.513
  06:34:42 STEP 2745 H=2745:00 +CPU= 20.847
  06:34:56 STEP 2746 H=2746:00 +CPU= 14.142
  06:35:10 STEP 2747 H=2747:00 +CPU= 14.131
  06:35:33 STEP 2748 H=2748:00 +CPU= 22.642
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002748
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002748
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002748
  06:35:47 STEP 2749 H=2749:00 +CPU= 13.859
  06:36:01 STEP 2750 H=2750:00 +CPU= 14.471
  06:36:22 STEP 2751 H=2751:00 +CPU= 20.917
  06:36:36 STEP 2752 H=2752:00 +CPU= 14.032
  06:36:50 STEP 2753 H=2753:00 +CPU= 14.048
  06:37:12 STEP 2754 H=2754:00 +CPU= 21.342
  06:37:25 STEP 2755 H=2755:00 +CPU= 13.648
  06:37:40 STEP 2756 H=2756:00 +CPU= 14.411
  06:38:00 STEP 2757 H=2757:00 +CPU= 20.627
  06:38:15 STEP 2758 H=2758:00 +CPU= 14.143
  06:38:29 STEP 2759 H=2759:00 +CPU= 14.402
  06:38:54 STEP 2760 H=2760:00 +CPU= 24.537
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002760
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002760
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002748
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002748
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002748
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_114.zip
Uploading the intermediate file: upload_file_114.zip
  06:39:08 STEP 2761 H=2761:00 +CPU= 14.013
  06:39:22 STEP 2762 H=2762:00 +CPU= 14.631
  06:39:43 STEP 2763 H=2763:00 +CPU= 21.109
  06:39:58 STEP 2764 H=2764:00 +CPU= 14.236
  06:40:12 STEP 2765 H=2765:00 +CPU= 14.256
  06:40:33 STEP 2766 H=2766:00 +CPU= 21.454
  06:40:47 STEP 2767 H=2767:00 +CPU= 13.988
  06:41:02 STEP 2768 H=2768:00 +CPU= 14.607
  06:41:23 STEP 2769 H=2769:00 +CPU= 21.041
  06:41:37 STEP 2770 H=2770:00 +CPU= 14.322
  06:41:52 STEP 2771 H=2771:00 +CPU= 14.257
  06:42:14 STEP 2772 H=2772:00 +CPU= 22.645
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002772
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002772
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002772
  06:42:28 STEP 2773 H=2773:00 +CPU= 13.842
  06:42:42 STEP 2774 H=2774:00 +CPU= 14.248
  06:43:03 STEP 2775 H=2775:00 +CPU= 20.779
  06:43:17 STEP 2776 H=2776:00 +CPU= 14.177
  06:43:31 STEP 2777 H=2777:00 +CPU= 13.958
  06:43:53 STEP 2778 H=2778:00 +CPU= 21.174
  06:44:06 STEP 2779 H=2779:00 +CPU= 13.823
  06:44:21 STEP 2780 H=2780:00 +CPU= 14.378
  06:44:41 STEP 2781 H=2781:00 +CPU= 20.627
  06:44:56 STEP 2782 H=2782:00 +CPU= 14.167
  06:45:10 STEP 2783 H=2783:00 +CPU= 14.563
  06:45:35 STEP 2784 H=2784:00 +CPU= 24.418
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002784
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002784
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002772
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002772
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002772
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_115.zip
Uploading the intermediate file: upload_file_115.zip
  06:45:49 STEP 2785 H=2785:00 +CPU= 13.876
  06:46:03 STEP 2786 H=2786:00 +CPU= 14.221
  06:46:24 STEP 2787 H=2787:00 +CPU= 20.729
  06:46:38 STEP 2788 H=2788:00 +CPU= 14.109
  06:46:52 STEP 2789 H=2789:00 +CPU= 13.965
  06:47:13 STEP 2790 H=2790:00 +CPU= 21.136
  06:47:27 STEP 2791 H=2791:00 +CPU= 13.798
  06:47:41 STEP 2792 H=2792:00 +CPU= 14.335
  06:48:02 STEP 2793 H=2793:00 +CPU= 20.793
  06:48:16 STEP 2794 H=2794:00 +CPU= 14.038
  06:48:30 STEP 2795 H=2795:00 +CPU= 14.019
  06:48:52 STEP 2796 H=2796:00 +CPU= 22.679
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002796
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002796
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002796
  06:49:06 STEP 2797 H=2797:00 +CPU= 13.833
  06:49:21 STEP 2798 H=2798:00 +CPU= 14.308
  06:49:42 STEP 2799 H=2799:00 +CPU= 20.922
  06:49:56 STEP 2800 H=2800:00 +CPU= 14.195
  06:50:10 STEP 2801 H=2801:00 +CPU= 14.002
  06:50:31 STEP 2802 H=2802:00 +CPU= 21.430
  06:50:45 STEP 2803 H=2803:00 +CPU= 13.825
  06:50:59 STEP 2804 H=2804:00 +CPU= 14.332
  06:51:20 STEP 2805 H=2805:00 +CPU= 20.760
  06:51:34 STEP 2806 H=2806:00 +CPU= 14.170
  06:51:49 STEP 2807 H=2807:00 +CPU= 14.682
  06:52:14 STEP 2808 H=2808:00 +CPU= 24.566
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002808
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002808
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002796
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002796
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002796
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_116.zip
Uploading the intermediate file: upload_file_116.zip
  06:52:28 STEP 2809 H=2809:00 +CPU= 13.975
  06:52:42 STEP 2810 H=2810:00 +CPU= 14.183
  06:53:03 STEP 2811 H=2811:00 +CPU= 20.919
  06:53:17 STEP 2812 H=2812:00 +CPU= 14.121
  06:53:31 STEP 2813 H=2813:00 +CPU= 13.758
  06:53:52 STEP 2814 H=2814:00 +CPU= 21.155
  06:54:06 STEP 2815 H=2815:00 +CPU= 13.779
  06:54:20 STEP 2816 H=2816:00 +CPU= 14.240
  06:54:41 STEP 2817 H=2817:00 +CPU= 20.800
  06:54:55 STEP 2818 H=2818:00 +CPU= 14.133
  06:55:09 STEP 2819 H=2819:00 +CPU= 13.961
  06:55:32 STEP 2820 H=2820:00 +CPU= 22.739
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002820
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002820
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002820
  06:55:46 STEP 2821 H=2821:00 +CPU= 13.903
  06:56:00 STEP 2822 H=2822:00 +CPU= 14.198
  06:56:21 STEP 2823 H=2823:00 +CPU= 20.956
  06:56:35 STEP 2824 H=2824:00 +CPU= 14.226
  06:56:49 STEP 2825 H=2825:00 +CPU= 13.893
  06:57:10 STEP 2826 H=2826:00 +CPU= 21.407
  06:57:24 STEP 2827 H=2827:00 +CPU= 13.825
  06:57:38 STEP 2828 H=2828:00 +CPU= 14.187
  06:57:59 STEP 2829 H=2829:00 +CPU= 20.779
  06:58:13 STEP 2830 H=2830:00 +CPU= 14.120
  06:58:28 STEP 2831 H=2831:00 +CPU= 14.546
  06:58:52 STEP 2832 H=2832:00 +CPU= 24.570
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002832
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002832
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002820
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002820
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002820
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_117.zip
Uploading the intermediate file: upload_file_117.zip
  06:59:06 STEP 2833 H=2833:00 +CPU= 13.907
  06:59:20 STEP 2834 H=2834:00 +CPU= 14.103
  06:59:41 STEP 2835 H=2835:00 +CPU= 20.974
  06:59:56 STEP 2836 H=2836:00 +CPU= 14.193
  07:00:09 STEP 2837 H=2837:00 +CPU= 13.825
  07:00:31 STEP 2838 H=2838:00 +CPU= 21.247
  07:00:45 STEP 2839 H=2839:00 +CPU= 13.873
  07:00:59 STEP 2840 H=2840:00 +CPU= 14.260
  07:01:20 STEP 2841 H=2841:00 +CPU= 20.878
  07:01:34 STEP 2842 H=2842:00 +CPU= 14.252
  07:01:48 STEP 2843 H=2843:00 +CPU= 14.272
  07:02:11 STEP 2844 H=2844:00 +CPU= 22.741
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002844
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002844
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002844
  07:02:25 STEP 2845 H=2845:00 +CPU= 13.754
  07:02:39 STEP 2846 H=2846:00 +CPU= 14.113
  07:03:00 STEP 2847 H=2847:00 +CPU= 20.917
  07:03:14 STEP 2848 H=2848:00 +CPU= 14.215
  07:03:28 STEP 2849 H=2849:00 +CPU= 13.858
  07:03:50 STEP 2850 H=2850:00 +CPU= 21.467
  07:04:03 STEP 2851 H=2851:00 +CPU= 13.830
  07:04:18 STEP 2852 H=2852:00 +CPU= 14.194
  07:04:38 STEP 2853 H=2853:00 +CPU= 20.700
  07:04:53 STEP 2854 H=2854:00 +CPU= 14.163
  07:05:07 STEP 2855 H=2855:00 +CPU= 14.465
  07:05:31 STEP 2856 H=2856:00 +CPU= 24.407
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002856
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002856
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002844
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002844
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002844
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_118.zip
Uploading the intermediate file: upload_file_118.zip
  07:05:45 STEP 2857 H=2857:00 +CPU= 13.952
  07:06:00 STEP 2858 H=2858:00 +CPU= 14.225
  07:06:21 STEP 2859 H=2859:00 +CPU= 20.964
  07:06:35 STEP 2860 H=2860:00 +CPU= 14.189
  07:06:49 STEP 2861 H=2861:00 +CPU= 13.860
  07:07:10 STEP 2862 H=2862:00 +CPU= 21.321
  07:07:24 STEP 2863 H=2863:00 +CPU= 13.838
  07:07:38 STEP 2864 H=2864:00 +CPU= 14.217
  07:07:59 STEP 2865 H=2865:00 +CPU= 20.934
  07:08:13 STEP 2866 H=2866:00 +CPU= 14.111
  07:08:27 STEP 2867 H=2867:00 +CPU= 14.025
  07:08:50 STEP 2868 H=2868:00 +CPU= 22.668
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002868
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002868
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002868
  07:09:04 STEP 2869 H=2869:00 +CPU= 13.942
  07:09:18 STEP 2870 H=2870:00 +CPU= 14.321
  07:09:39 STEP 2871 H=2871:00 +CPU= 21.008
  07:09:53 STEP 2872 H=2872:00 +CPU= 14.312
  07:10:08 STEP 2873 H=2873:00 +CPU= 13.997
  07:10:29 STEP 2874 H=2874:00 +CPU= 21.363
  07:10:43 STEP 2875 H=2875:00 +CPU= 13.919
  07:10:57 STEP 2876 H=2876:00 +CPU= 14.326
  07:11:18 STEP 2877 H=2877:00 +CPU= 20.858
  07:11:32 STEP 2878 H=2878:00 +CPU= 14.040
  07:11:47 STEP 2879 H=2879:00 +CPU= 14.509
  07:12:11 STEP 2880 H=2880:00 +CPU= 24.622
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002880
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002880
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002868
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002868
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002868
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_119.zip
Uploading the intermediate file: upload_file_119.zip
  07:12:25 STEP 2881 H=2881:00 +CPU= 13.979
Uploading trickle at timestep: 10368000
  07:12:40 STEP 2882 H=2882:00 +CPU= 14.270
  07:13:01 STEP 2883 H=2883:00 +CPU= 21.129
  07:13:15 STEP 2884 H=2884:00 +CPU= 14.186
  07:13:29 STEP 2885 H=2885:00 +CPU= 13.868
  07:13:50 STEP 2886 H=2886:00 +CPU= 21.328
  07:14:04 STEP 2887 H=2887:00 +CPU= 13.942
  07:14:18 STEP 2888 H=2888:00 +CPU= 14.406
  07:14:40 STEP 2889 H=2889:00 +CPU= 21.020
  07:14:54 STEP 2890 H=2890:00 +CPU= 14.256
  07:15:08 STEP 2891 H=2891:00 +CPU= 13.887
  07:15:30 STEP 2892 H=2892:00 +CPU= 22.750
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002892
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002892
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002892
  07:15:44 STEP 2893 H=2893:00 +CPU= 13.854
  07:15:59 STEP 2894 H=2894:00 +CPU= 14.304
  07:16:20 STEP 2895 H=2895:00 +CPU= 20.993
  07:16:34 STEP 2896 H=2896:00 +CPU= 14.173
  07:16:48 STEP 2897 H=2897:00 +CPU= 13.955
  07:17:09 STEP 2898 H=2898:00 +CPU= 21.293
  07:17:23 STEP 2899 H=2899:00 +CPU= 13.934
  07:17:37 STEP 2900 H=2900:00 +CPU= 14.354
  07:17:58 STEP 2901 H=2901:00 +CPU= 20.864
  07:18:12 STEP 2902 H=2902:00 +CPU= 14.094
  07:18:27 STEP 2903 H=2903:00 +CPU= 14.562
  07:18:51 STEP 2904 H=2904:00 +CPU= 24.437
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002904
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002904
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002892
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002892
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002892
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_120.zip
Uploading the intermediate file: upload_file_120.zip
  07:19:06 STEP 2905 H=2905:00 +CPU= 14.133
  07:19:20 STEP 2906 H=2906:00 +CPU= 14.431
  07:19:41 STEP 2907 H=2907:00 +CPU= 21.044
  07:19:55 STEP 2908 H=2908:00 +CPU= 14.321
  07:20:10 STEP 2909 H=2909:00 +CPU= 14.038
  07:20:31 STEP 2910 H=2910:00 +CPU= 21.331
  07:20:45 STEP 2911 H=2911:00 +CPU= 14.042
  07:20:59 STEP 2912 H=2912:00 +CPU= 14.341
  07:21:20 STEP 2913 H=2913:00 +CPU= 20.984
  07:21:34 STEP 2914 H=2914:00 +CPU= 14.091
  07:21:48 STEP 2915 H=2915:00 +CPU= 13.728
  07:22:11 STEP 2916 H=2916:00 +CPU= 22.769
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002916
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002916
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002916
  07:22:25 STEP 2917 H=2917:00 +CPU= 14.007
  07:22:39 STEP 2918 H=2918:00 +CPU= 14.396
  07:23:00 STEP 2919 H=2919:00 +CPU= 20.964
  07:23:15 STEP 2920 H=2920:00 +CPU= 14.345
  07:23:29 STEP 2921 H=2921:00 +CPU= 13.980
  07:23:50 STEP 2922 H=2922:00 +CPU= 21.362
  07:24:04 STEP 2923 H=2923:00 +CPU= 13.939
  07:24:18 STEP 2924 H=2924:00 +CPU= 14.214
  07:24:39 STEP 2925 H=2925:00 +CPU= 21.056
  07:24:53 STEP 2926 H=2926:00 +CPU= 14.071
  07:25:08 STEP 2927 H=2927:00 +CPU= 14.604
  07:25:33 STEP 2928 H=2928:00 +CPU= 24.583
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002928
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002928
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002928
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002916
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002916
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002916
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_121.zip
Uploading the intermediate file: upload_file_121.zip
  07:25:47 STEP 2929 H=2929:00 +CPU= 14.032
  07:26:01 STEP 2930 H=2930:00 +CPU= 14.228
  07:26:22 STEP 2931 H=2931:00 +CPU= 20.940
  07:26:36 STEP 2932 H=2932:00 +CPU= 14.321
  07:26:50 STEP 2933 H=2933:00 +CPU= 13.941
  07:27:11 STEP 2934 H=2934:00 +CPU= 21.273
  07:27:25 STEP 2935 H=2935:00 +CPU= 13.985
  07:27:40 STEP 2936 H=2936:00 +CPU= 14.258
  07:28:01 STEP 2937 H=2937:00 +CPU= 20.893
  07:28:15 STEP 2938 H=2938:00 +CPU= 14.132
  07:28:28 STEP 2939 H=2939:00 +CPU= 13.718
  07:28:51 STEP 2940 H=2940:00 +CPU= 22.702
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002940
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002940
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002940
  07:29:05 STEP 2941 H=2941:00 +CPU= 14.094
  07:29:20 STEP 2942 H=2942:00 +CPU= 14.395
  07:29:41 STEP 2943 H=2943:00 +CPU= 21.009
  07:29:55 STEP 2944 H=2944:00 +CPU= 14.493
  07:30:09 STEP 2945 H=2945:00 +CPU= 14.062
  07:30:31 STEP 2946 H=2946:00 +CPU= 21.376
  07:30:45 STEP 2947 H=2947:00 +CPU= 14.015
  07:30:59 STEP 2948 H=2948:00 +CPU= 14.340
  07:31:20 STEP 2949 H=2949:00 +CPU= 21.038
  07:31:34 STEP 2950 H=2950:00 +CPU= 14.220
  07:31:49 STEP 2951 H=2951:00 +CPU= 14.629
  07:32:13 STEP 2952 H=2952:00 +CPU= 23.704
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.19925486346338E-02 -1.00000000000000E+00  1.00000000000000E+00
  MAXGPFV : MAX. VALUE =    1.10460382867833     
  MAXGPFV : MAX. VALUE =   0.741897331108798     
 NSTEP =  2952 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhnw4+002952 MODE=a
 IO-STREAM CLOSED - ICMGGhnw4+002952
 MPI-TASK:   1 -    312189610 BYTES IN      8 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0078( 3.8%)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302231500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302233000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302234500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200303000000  !!!!!!!!!!!! 
 07:32:13 STEP 2952 H=2952:00 +CPU= 23.704
 END CNT3
 NSTEP =  2952 CNT0   000000000  20.516  20.516  20.536   0.999               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ          654/    28953814 WRITTEN        46048/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1116    0      654        28275    46048 ************   120.8     0.0   120.8     0.9   200.1
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********    100.00      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 3200    1711.0    1711.0     11.18      0.00
   9 SPCM     - SPECTRAL COMP.                    2952     280.4     280.4      1.69      0.00
  10 SCAN2M - PHYSICS                             2953    6839.1    6839.1     41.23      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                247     777.0     777.0      0.39      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                126      11.3      11.3      0.00      0.00
  13 SCAN2M - RADIATION CALC.                      985    7001.8    7001.8     14.08      0.00
  14 SUINIF                                          1    1414.3    1414.3      0.00      0.00
  17 GRIDFPOS IN CNT4                              247      27.0      27.0      0.01      0.00
  18 SUSPECG                                         1     256.9     256.9      0.00      0.00
  19 SUSPEC                                          1     263.3     263.3      0.00      0.00
  24 SUGRIDU                                         1     901.0     901.0      0.00      0.00
  25 SPECRT                                          1     118.0     118.0      0.00      0.00
  26 SUGRIDF                                         1     132.0     132.0      0.00      0.00
  27 RESTART FILES - WRITING                       123    1560.3    1560.3      0.39      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               247       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                               247    1321.1    1321.1      0.67      0.00
  31 POSDDH IN STEPO                                13       0.5       0.5      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   2953    2273.6    2273.6     13.71      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL          2952    1685.7    1685.7     10.16      0.00
  39 SU0YOMB                                         1     154.3     154.3      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   2953      16.0      16.0      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             2952     251.3     251.3      1.51      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             2952      40.7      40.7      0.25      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                126       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM    10094      90.1      90.1      1.86      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      6152      90.5      90.5      1.14      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       6152      32.1      32.1      0.40      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM     10094      32.6      32.6      0.67      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2      12.8      12.8      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION          10094      13.0      13.0      0.27      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           6152      12.1      12.1      0.15      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION          10094      14.8      14.8      0.30      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           6152      10.7      10.7      0.13      0.00
 400 GSTATS                                     589499       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                564603       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    48986.7 CPU TIME   48945.1 VECTOR TIME    48945.1


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      216.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /hdd/boinc-client/slots/0/ICMGGhnw4+002952
Moving to projects directory: /hdd/boinc-client/slots/0/ICMSHhnw4+002952
Moving to projects directory: /hdd/boinc-client/slots/0/ICMUAhnw4+002952
Adding to the zip: /hdd/boinc-client/slots/0/NODE.001_01
Adding to the zip: /hdd/boinc-client/slots/0/ifs.stat
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002928
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002952
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002940
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMUAhnw4+002928
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002928
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002940
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMGGhnw4+002952
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002952
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214995/ICMSHhnw4+002940
Zipping up the final file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_0179_2019110100_123_993_12214995_0_r1793681168_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
07:35:35 (41942): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:53:24 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 10,623,600 48,945 0.0046
09 May 2023 16:53:20 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 10,368,000 47,744 0.0046
09 May 2023 16:53:17 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 9,504,000 43,761 0.0046
09 May 2023 16:53:14 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 8,640,000 39,827 0.0046
09 May 2023 16:53:10 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 7,776,000 35,873 0.0046
09 May 2023 16:53:07 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 6,912,000 31,889 0.0046
09 May 2023 16:53:04 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 6,048,000 27,945 0.0046
09 May 2023 16:53:01 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 5,184,000 23,961 0.0046
09 May 2023 16:52:58 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 4,320,000 19,988 0.0046
09 May 2023 16:52:55 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 3,456,000 16,004 0.0046
09 May 2023 16:52:52 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 2,592,000 12,010 0.0046
09 May 2023 16:52:50 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 1,728,000 8,015 0.0046
09 May 2023 16:52:45 1498009 22316388 oifs_43r3_0179_2019110100_123_993_12214995_0 864,000 4,022 0.0047


©2024 climateprediction.net