climateprediction.net home page
Task 22315930

Task 22315930

Name oifs_43r3_00v3_2019110100_123_993_12214557_0
Workunit 12214557
Created 24 Feb 2023, 15:25:54 UTC
Sent 24 Feb 2023, 22:32:42 UTC
Report deadline 25 Apr 2023, 22:32:42 UTC
Received 25 Feb 2023, 23:50:07 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1497236
Run time 13 hours 14 min 29 sec
CPU time 13 hours 10 min 24 sec
Validate state Valid
Credit 14,515.20
Device peak FLOPS 7.27 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,416.04 MB
Peak swap size 4,974.22 MB
Peak disk usage 1,850.33 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
ip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002340
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002340
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_97.zip
Uploading the intermediate file: upload_file_97.zip
  21:03:08 STEP 2353 H=2353:00 +CPU= 12.862
  21:03:21 STEP 2354 H=2354:00 +CPU= 13.059
  21:03:41 STEP 2355 H=2355:00 +CPU= 19.290
  21:03:54 STEP 2356 H=2356:00 +CPU= 13.126
  21:04:07 STEP 2357 H=2357:00 +CPU= 12.854
  21:04:26 STEP 2358 H=2358:00 +CPU= 19.718
  21:04:39 STEP 2359 H=2359:00 +CPU= 12.781
  21:04:52 STEP 2360 H=2360:00 +CPU= 13.200
  21:05:11 STEP 2361 H=2361:00 +CPU= 19.240
  21:05:25 STEP 2362 H=2362:00 +CPU= 13.082
  21:05:37 STEP 2363 H=2363:00 +CPU= 12.779
  21:05:58 STEP 2364 H=2364:00 +CPU= 20.823
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002364
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002364
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002364
  21:06:11 STEP 2365 H=2365:00 +CPU= 12.783
  21:06:24 STEP 2366 H=2366:00 +CPU= 13.132
  21:06:43 STEP 2367 H=2367:00 +CPU= 19.140
  21:06:56 STEP 2368 H=2368:00 +CPU= 13.090
  21:07:09 STEP 2369 H=2369:00 +CPU= 12.913
  21:07:29 STEP 2370 H=2370:00 +CPU= 19.515
  21:07:41 STEP 2371 H=2371:00 +CPU= 12.671
  21:07:55 STEP 2372 H=2372:00 +CPU= 13.206
  21:08:14 STEP 2373 H=2373:00 +CPU= 19.227
  21:08:27 STEP 2374 H=2374:00 +CPU= 13.052
  21:08:40 STEP 2375 H=2375:00 +CPU= 13.266
  21:09:03 STEP 2376 H=2376:00 +CPU= 22.427
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002376
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002376
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002352
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002352
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002352
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002364
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002364
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002364
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_98.zip
Uploading the intermediate file: upload_file_98.zip
  21:09:16 STEP 2377 H=2377:00 +CPU= 12.854
  21:09:29 STEP 2378 H=2378:00 +CPU= 13.067
  21:09:48 STEP 2379 H=2379:00 +CPU= 19.235
  21:10:01 STEP 2380 H=2380:00 +CPU= 13.167
  21:10:14 STEP 2381 H=2381:00 +CPU= 12.905
  21:10:34 STEP 2382 H=2382:00 +CPU= 19.636
  21:10:47 STEP 2383 H=2383:00 +CPU= 12.824
  21:11:00 STEP 2384 H=2384:00 +CPU= 13.172
  21:11:19 STEP 2385 H=2385:00 +CPU= 19.192
  21:11:32 STEP 2386 H=2386:00 +CPU= 13.087
  21:11:45 STEP 2387 H=2387:00 +CPU= 12.810
  21:12:06 STEP 2388 H=2388:00 +CPU= 20.899
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002388
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002388
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002388
  21:12:19 STEP 2389 H=2389:00 +CPU= 12.822
  21:12:32 STEP 2390 H=2390:00 +CPU= 13.220
  21:12:51 STEP 2391 H=2391:00 +CPU= 19.166
  21:13:04 STEP 2392 H=2392:00 +CPU= 13.133
  21:13:17 STEP 2393 H=2393:00 +CPU= 12.867
  21:13:37 STEP 2394 H=2394:00 +CPU= 19.557
  21:13:49 STEP 2395 H=2395:00 +CPU= 12.618
  21:14:02 STEP 2396 H=2396:00 +CPU= 13.134
  21:14:22 STEP 2397 H=2397:00 +CPU= 19.331
  21:14:35 STEP 2398 H=2398:00 +CPU= 13.138
  21:14:48 STEP 2399 H=2399:00 +CPU= 13.231
  21:15:10 STEP 2400 H=2400:00 +CPU= 22.389
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002400
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002400
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002376
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002388
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002388
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002388
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_99.zip
Uploading the intermediate file: upload_file_99.zip
  21:15:23 STEP 2401 H=2401:00 +CPU= 12.859
Uploading trickle at timestep: 8640000
  21:15:36 STEP 2402 H=2402:00 +CPU= 13.098
  21:15:56 STEP 2403 H=2403:00 +CPU= 19.273
  21:16:09 STEP 2404 H=2404:00 +CPU= 13.180
  21:16:22 STEP 2405 H=2405:00 +CPU= 12.841
  21:16:42 STEP 2406 H=2406:00 +CPU= 19.735
  21:16:54 STEP 2407 H=2407:00 +CPU= 12.708
  21:17:07 STEP 2408 H=2408:00 +CPU= 13.178
  21:17:27 STEP 2409 H=2409:00 +CPU= 19.242
  21:17:40 STEP 2410 H=2410:00 +CPU= 13.075
  21:17:53 STEP 2411 H=2411:00 +CPU= 12.768
  21:18:13 STEP 2412 H=2412:00 +CPU= 20.878
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002412
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002412
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002412
  21:18:26 STEP 2413 H=2413:00 +CPU= 12.806
  21:18:39 STEP 2414 H=2414:00 +CPU= 13.247
  21:18:59 STEP 2415 H=2415:00 +CPU= 19.184
  21:19:12 STEP 2416 H=2416:00 +CPU= 13.201
  21:19:25 STEP 2417 H=2417:00 +CPU= 12.901
  21:19:44 STEP 2418 H=2418:00 +CPU= 19.527
  21:19:57 STEP 2419 H=2419:00 +CPU= 12.716
  21:20:10 STEP 2420 H=2420:00 +CPU= 13.173
  21:20:30 STEP 2421 H=2421:00 +CPU= 19.285
  21:20:43 STEP 2422 H=2422:00 +CPU= 13.125
  21:20:56 STEP 2423 H=2423:00 +CPU= 13.304
  21:21:18 STEP 2424 H=2424:00 +CPU= 22.496
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002424
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002424
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002400
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002412
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002412
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002412
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_100.zip
Uploading the intermediate file: upload_file_100.zip
  21:21:31 STEP 2425 H=2425:00 +CPU= 12.906
  21:21:45 STEP 2426 H=2426:00 +CPU= 13.159
  21:22:04 STEP 2427 H=2427:00 +CPU= 19.376
  21:22:17 STEP 2428 H=2428:00 +CPU= 13.195
  21:22:30 STEP 2429 H=2429:00 +CPU= 12.906
  21:22:50 STEP 2430 H=2430:00 +CPU= 19.646
  21:23:03 STEP 2431 H=2431:00 +CPU= 12.816
  21:23:16 STEP 2432 H=2432:00 +CPU= 13.158
  21:23:35 STEP 2433 H=2433:00 +CPU= 19.289
  21:23:48 STEP 2434 H=2434:00 +CPU= 13.118
  21:24:01 STEP 2435 H=2435:00 +CPU= 12.815
  21:24:22 STEP 2436 H=2436:00 +CPU= 20.903
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002436
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002436
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002436
  21:24:35 STEP 2437 H=2437:00 +CPU= 12.857
  21:24:48 STEP 2438 H=2438:00 +CPU= 13.370
  21:25:07 STEP 2439 H=2439:00 +CPU= 19.188
  21:25:21 STEP 2440 H=2440:00 +CPU= 13.177
  21:25:34 STEP 2441 H=2441:00 +CPU= 12.920
  21:25:53 STEP 2442 H=2442:00 +CPU= 19.642
  21:26:06 STEP 2443 H=2443:00 +CPU= 12.838
  21:26:19 STEP 2444 H=2444:00 +CPU= 13.385
  21:26:39 STEP 2445 H=2445:00 +CPU= 19.361
  21:26:52 STEP 2446 H=2446:00 +CPU= 13.165
  21:27:05 STEP 2447 H=2447:00 +CPU= 13.328
  21:27:28 STEP 2448 H=2448:00 +CPU= 22.501
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002448
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002448
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002424
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002436
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002436
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002436
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_101.zip
Uploading the intermediate file: upload_file_101.zip
  21:27:41 STEP 2449 H=2449:00 +CPU= 12.871
  21:27:54 STEP 2450 H=2450:00 +CPU= 13.072
  21:28:13 STEP 2451 H=2451:00 +CPU= 19.301
  21:28:26 STEP 2452 H=2452:00 +CPU= 13.217
  21:28:39 STEP 2453 H=2453:00 +CPU= 12.870
  21:28:59 STEP 2454 H=2454:00 +CPU= 19.645
  21:29:12 STEP 2455 H=2455:00 +CPU= 12.976
  21:29:25 STEP 2456 H=2456:00 +CPU= 13.187
  21:29:44 STEP 2457 H=2457:00 +CPU= 19.267
  21:29:58 STEP 2458 H=2458:00 +CPU= 13.213
  21:30:10 STEP 2459 H=2459:00 +CPU= 12.870
  21:30:31 STEP 2460 H=2460:00 +CPU= 20.871
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002460
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002460
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002460
  21:30:44 STEP 2461 H=2461:00 +CPU= 12.819
  21:30:57 STEP 2462 H=2462:00 +CPU= 13.167
  21:31:16 STEP 2463 H=2463:00 +CPU= 19.188
  21:31:30 STEP 2464 H=2464:00 +CPU= 13.145
  21:31:42 STEP 2465 H=2465:00 +CPU= 12.836
  21:32:02 STEP 2466 H=2466:00 +CPU= 19.562
  21:32:15 STEP 2467 H=2467:00 +CPU= 12.748
  21:32:28 STEP 2468 H=2468:00 +CPU= 13.225
  21:32:47 STEP 2469 H=2469:00 +CPU= 19.143
  21:33:01 STEP 2470 H=2470:00 +CPU= 13.252
  21:33:14 STEP 2471 H=2471:00 +CPU= 13.185
  21:33:36 STEP 2472 H=2472:00 +CPU= 22.461
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002472
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002472
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002448
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002460
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002460
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002460
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_102.zip
Uploading the intermediate file: upload_file_102.zip
  21:33:49 STEP 2473 H=2473:00 +CPU= 12.882
  21:34:02 STEP 2474 H=2474:00 +CPU= 13.122
  21:34:22 STEP 2475 H=2475:00 +CPU= 19.353
  21:34:35 STEP 2476 H=2476:00 +CPU= 13.232
  21:34:48 STEP 2477 H=2477:00 +CPU= 12.827
  21:35:07 STEP 2478 H=2478:00 +CPU= 19.677
  21:35:20 STEP 2479 H=2479:00 +CPU= 12.918
  21:35:34 STEP 2480 H=2480:00 +CPU= 13.242
  21:35:53 STEP 2481 H=2481:00 +CPU= 19.268
  21:36:06 STEP 2482 H=2482:00 +CPU= 13.156
  21:36:19 STEP 2483 H=2483:00 +CPU= 12.841
  21:36:40 STEP 2484 H=2484:00 +CPU= 20.889
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002484
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002484
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002484
  21:36:53 STEP 2485 H=2485:00 +CPU= 12.838
  21:37:06 STEP 2486 H=2486:00 +CPU= 13.160
  21:37:25 STEP 2487 H=2487:00 +CPU= 19.183
  21:37:38 STEP 2488 H=2488:00 +CPU= 13.189
  21:37:51 STEP 2489 H=2489:00 +CPU= 12.822
  21:38:11 STEP 2490 H=2490:00 +CPU= 19.543
  21:38:23 STEP 2491 H=2491:00 +CPU= 12.767
  21:38:37 STEP 2492 H=2492:00 +CPU= 13.183
  21:38:56 STEP 2493 H=2493:00 +CPU= 19.164
  21:39:09 STEP 2494 H=2494:00 +CPU= 13.183
  21:39:22 STEP 2495 H=2495:00 +CPU= 13.138
  21:39:45 STEP 2496 H=2496:00 +CPU= 22.543
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002496
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002496
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002472
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002484
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002484
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002484
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_103.zip
Uploading the intermediate file: upload_file_103.zip
  21:39:57 STEP 2497 H=2497:00 +CPU= 12.843
  21:40:11 STEP 2498 H=2498:00 +CPU= 13.079
  21:40:30 STEP 2499 H=2499:00 +CPU= 19.320
  21:40:43 STEP 2500 H=2500:00 +CPU= 13.300
  21:40:56 STEP 2501 H=2501:00 +CPU= 12.875
  21:41:16 STEP 2502 H=2502:00 +CPU= 19.785
  21:41:29 STEP 2503 H=2503:00 +CPU= 12.836
  21:41:42 STEP 2504 H=2504:00 +CPU= 13.167
  21:42:01 STEP 2505 H=2505:00 +CPU= 19.366
  21:42:14 STEP 2506 H=2506:00 +CPU= 13.177
  21:42:27 STEP 2507 H=2507:00 +CPU= 12.806
  21:42:48 STEP 2508 H=2508:00 +CPU= 20.943
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002508
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002508
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002508
  21:43:01 STEP 2509 H=2509:00 +CPU= 12.901
  21:43:14 STEP 2510 H=2510:00 +CPU= 13.169
  21:43:34 STEP 2511 H=2511:00 +CPU= 19.253
  21:43:47 STEP 2512 H=2512:00 +CPU= 13.238
  21:44:00 STEP 2513 H=2513:00 +CPU= 12.825
  21:44:19 STEP 2514 H=2514:00 +CPU= 19.604
  21:44:32 STEP 2515 H=2515:00 +CPU= 12.869
  21:44:45 STEP 2516 H=2516:00 +CPU= 13.123
  21:45:05 STEP 2517 H=2517:00 +CPU= 19.334
  21:45:18 STEP 2518 H=2518:00 +CPU= 13.216
  21:45:31 STEP 2519 H=2519:00 +CPU= 13.164
  21:45:54 STEP 2520 H=2520:00 +CPU= 22.573
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002520
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002520
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002496
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002508
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002508
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002508
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_104.zip
Uploading the intermediate file: upload_file_104.zip
  21:46:07 STEP 2521 H=2521:00 +CPU= 12.966
  21:46:20 STEP 2522 H=2522:00 +CPU= 13.029
  21:46:39 STEP 2523 H=2523:00 +CPU= 19.445
  21:46:52 STEP 2524 H=2524:00 +CPU= 13.248
  21:47:05 STEP 2525 H=2525:00 +CPU= 12.933
  21:47:25 STEP 2526 H=2526:00 +CPU= 19.680
  21:47:38 STEP 2527 H=2527:00 +CPU= 12.902
  21:47:51 STEP 2528 H=2528:00 +CPU= 13.137
  21:48:10 STEP 2529 H=2529:00 +CPU= 19.431
  21:48:24 STEP 2530 H=2530:00 +CPU= 13.204
  21:48:36 STEP 2531 H=2531:00 +CPU= 12.745
  21:48:58 STEP 2532 H=2532:00 +CPU= 21.177
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002532
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002532
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002532
  21:49:11 STEP 2533 H=2533:00 +CPU= 13.001
  21:49:24 STEP 2534 H=2534:00 +CPU= 13.095
  21:49:43 STEP 2535 H=2535:00 +CPU= 19.187
  21:49:56 STEP 2536 H=2536:00 +CPU= 13.181
  21:50:09 STEP 2537 H=2537:00 +CPU= 12.844
  21:50:29 STEP 2538 H=2538:00 +CPU= 19.632
  21:50:42 STEP 2539 H=2539:00 +CPU= 12.880
  21:50:55 STEP 2540 H=2540:00 +CPU= 13.102
  21:51:14 STEP 2541 H=2541:00 +CPU= 19.364
  21:51:27 STEP 2542 H=2542:00 +CPU= 13.256
  21:51:41 STEP 2543 H=2543:00 +CPU= 13.168
  21:52:03 STEP 2544 H=2544:00 +CPU= 22.699
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002544
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002544
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002520
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002532
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002532
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002532
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_105.zip
Uploading the intermediate file: upload_file_105.zip
  21:52:16 STEP 2545 H=2545:00 +CPU= 12.914
  21:52:29 STEP 2546 H=2546:00 +CPU= 13.027
  21:52:49 STEP 2547 H=2547:00 +CPU= 19.529
  21:53:02 STEP 2548 H=2548:00 +CPU= 13.272
  21:53:15 STEP 2549 H=2549:00 +CPU= 12.865
  21:53:35 STEP 2550 H=2550:00 +CPU= 19.766
  21:53:48 STEP 2551 H=2551:00 +CPU= 12.874
  21:54:01 STEP 2552 H=2552:00 +CPU= 13.283
  21:54:20 STEP 2553 H=2553:00 +CPU= 19.337
  21:54:33 STEP 2554 H=2554:00 +CPU= 13.237
  21:54:46 STEP 2555 H=2555:00 +CPU= 12.812
  21:55:07 STEP 2556 H=2556:00 +CPU= 20.987
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002556
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002556
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002556
  21:55:20 STEP 2557 H=2557:00 +CPU= 12.849
  21:55:33 STEP 2558 H=2558:00 +CPU= 13.090
  21:55:52 STEP 2559 H=2559:00 +CPU= 19.215
  21:56:06 STEP 2560 H=2560:00 +CPU= 13.173
  21:56:18 STEP 2561 H=2561:00 +CPU= 12.803
  21:56:38 STEP 2562 H=2562:00 +CPU= 19.616
  21:56:51 STEP 2563 H=2563:00 +CPU= 12.942
  21:57:04 STEP 2564 H=2564:00 +CPU= 13.107
  21:57:23 STEP 2565 H=2565:00 +CPU= 19.279
  21:57:37 STEP 2566 H=2566:00 +CPU= 13.201
  21:57:50 STEP 2567 H=2567:00 +CPU= 13.175
  21:58:13 STEP 2568 H=2568:00 +CPU= 22.762
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002568
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002568
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002544
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002556
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002556
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002556
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_106.zip
Uploading the intermediate file: upload_file_106.zip
  21:58:26 STEP 2569 H=2569:00 +CPU= 12.938
  21:58:39 STEP 2570 H=2570:00 +CPU= 13.063
  21:58:58 STEP 2571 H=2571:00 +CPU= 19.430
  21:59:11 STEP 2572 H=2572:00 +CPU= 13.259
  21:59:24 STEP 2573 H=2573:00 +CPU= 12.759
  21:59:44 STEP 2574 H=2574:00 +CPU= 19.636
  21:59:57 STEP 2575 H=2575:00 +CPU= 12.967
  22:00:10 STEP 2576 H=2576:00 +CPU= 13.271
  22:00:29 STEP 2577 H=2577:00 +CPU= 19.315
  22:00:43 STEP 2578 H=2578:00 +CPU= 13.241
  22:00:55 STEP 2579 H=2579:00 +CPU= 12.849
  22:01:16 STEP 2580 H=2580:00 +CPU= 20.954
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002580
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002580
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002580
  22:01:29 STEP 2581 H=2581:00 +CPU= 12.897
  22:01:42 STEP 2582 H=2582:00 +CPU= 13.115
  22:02:02 STEP 2583 H=2583:00 +CPU= 19.124
  22:02:15 STEP 2584 H=2584:00 +CPU= 13.196
  22:02:28 STEP 2585 H=2585:00 +CPU= 12.805
  22:02:47 STEP 2586 H=2586:00 +CPU= 19.593
  22:03:00 STEP 2587 H=2587:00 +CPU= 12.786
  22:03:13 STEP 2588 H=2588:00 +CPU= 13.103
  22:03:32 STEP 2589 H=2589:00 +CPU= 19.179
  22:03:46 STEP 2590 H=2590:00 +CPU= 13.261
  22:03:59 STEP 2591 H=2591:00 +CPU= 13.176
  22:04:21 STEP 2592 H=2592:00 +CPU= 22.543
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002592
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002592
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002568
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002580
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002580
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002580
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_107.zip
Uploading the intermediate file: upload_file_107.zip
  22:04:34 STEP 2593 H=2593:00 +CPU= 12.842
  22:04:47 STEP 2594 H=2594:00 +CPU= 13.022
  22:05:07 STEP 2595 H=2595:00 +CPU= 19.437
  22:05:20 STEP 2596 H=2596:00 +CPU= 13.254
  22:05:33 STEP 2597 H=2597:00 +CPU= 12.848
  22:05:52 STEP 2598 H=2598:00 +CPU= 19.655
  22:06:05 STEP 2599 H=2599:00 +CPU= 12.922
  22:06:19 STEP 2600 H=2600:00 +CPU= 13.237
  22:06:38 STEP 2601 H=2601:00 +CPU= 19.366
  22:06:51 STEP 2602 H=2602:00 +CPU= 13.183
  22:07:04 STEP 2603 H=2603:00 +CPU= 12.828
  22:07:25 STEP 2604 H=2604:00 +CPU= 20.933
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002604
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002604
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002604
  22:07:38 STEP 2605 H=2605:00 +CPU= 12.823
  22:07:51 STEP 2606 H=2606:00 +CPU= 13.100
  22:08:10 STEP 2607 H=2607:00 +CPU= 19.153
  22:08:23 STEP 2608 H=2608:00 +CPU= 13.234
  22:08:36 STEP 2609 H=2609:00 +CPU= 12.798
  22:08:56 STEP 2610 H=2610:00 +CPU= 19.594
  22:09:09 STEP 2611 H=2611:00 +CPU= 12.801
  22:09:22 STEP 2612 H=2612:00 +CPU= 13.112
  22:09:41 STEP 2613 H=2613:00 +CPU= 19.273
  22:09:54 STEP 2614 H=2614:00 +CPU= 13.148
  22:10:07 STEP 2615 H=2615:00 +CPU= 13.114
  22:10:30 STEP 2616 H=2616:00 +CPU= 22.651
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002616
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002616
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002592
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002604
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002604
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002604
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_108.zip
Uploading the intermediate file: upload_file_108.zip
  22:10:43 STEP 2617 H=2617:00 +CPU= 12.864
  22:10:56 STEP 2618 H=2618:00 +CPU= 12.996
  22:11:15 STEP 2619 H=2619:00 +CPU= 19.474
  22:11:29 STEP 2620 H=2620:00 +CPU= 13.277
  22:11:41 STEP 2621 H=2621:00 +CPU= 12.725
  22:12:01 STEP 2622 H=2622:00 +CPU= 19.737
  22:12:14 STEP 2623 H=2623:00 +CPU= 12.936
  22:12:27 STEP 2624 H=2624:00 +CPU= 13.172
  22:12:47 STEP 2625 H=2625:00 +CPU= 19.334
  22:13:00 STEP 2626 H=2626:00 +CPU= 13.241
  22:13:13 STEP 2627 H=2627:00 +CPU= 12.806
  22:13:34 STEP 2628 H=2628:00 +CPU= 20.952
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002628
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002628
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002628
  22:13:46 STEP 2629 H=2629:00 +CPU= 12.877
  22:14:00 STEP 2630 H=2630:00 +CPU= 13.094
  22:14:19 STEP 2631 H=2631:00 +CPU= 19.181
  22:14:32 STEP 2632 H=2632:00 +CPU= 13.174
  22:14:45 STEP 2633 H=2633:00 +CPU= 12.865
  22:15:05 STEP 2634 H=2634:00 +CPU= 19.785
  22:15:17 STEP 2635 H=2635:00 +CPU= 12.784
  22:15:31 STEP 2636 H=2636:00 +CPU= 13.121
  22:15:50 STEP 2637 H=2637:00 +CPU= 19.314
  22:16:03 STEP 2638 H=2638:00 +CPU= 13.193
  22:16:16 STEP 2639 H=2639:00 +CPU= 13.174
  22:16:39 STEP 2640 H=2640:00 +CPU= 22.602
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002640
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002640
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002616
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002628
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002628
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002628
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_109.zip
Uploading the intermediate file: upload_file_109.zip
  22:16:52 STEP 2641 H=2641:00 +CPU= 12.837
  22:17:05 STEP 2642 H=2642:00 +CPU= 12.998
Uploading trickle at timestep: 9504000
  22:17:24 STEP 2643 H=2643:00 +CPU= 19.409
  22:17:37 STEP 2644 H=2644:00 +CPU= 13.219
  22:17:50 STEP 2645 H=2645:00 +CPU= 12.813
  22:18:10 STEP 2646 H=2646:00 +CPU= 19.614
  22:18:23 STEP 2647 H=2647:00 +CPU= 12.944
  22:18:36 STEP 2648 H=2648:00 +CPU= 13.194
  22:18:55 STEP 2649 H=2649:00 +CPU= 19.305
  22:19:09 STEP 2650 H=2650:00 +CPU= 13.192
  22:19:21 STEP 2651 H=2651:00 +CPU= 12.819
  22:19:42 STEP 2652 H=2652:00 +CPU= 20.966
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002652
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002652
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002652
  22:19:55 STEP 2653 H=2653:00 +CPU= 12.887
  22:20:08 STEP 2654 H=2654:00 +CPU= 13.161
  22:20:28 STEP 2655 H=2655:00 +CPU= 19.240
  22:20:41 STEP 2656 H=2656:00 +CPU= 13.211
  22:20:54 STEP 2657 H=2657:00 +CPU= 12.919
  22:21:13 STEP 2658 H=2658:00 +CPU= 19.681
  22:21:26 STEP 2659 H=2659:00 +CPU= 12.852
  22:21:40 STEP 2660 H=2660:00 +CPU= 13.188
  22:21:59 STEP 2661 H=2661:00 +CPU= 19.208
  22:22:12 STEP 2662 H=2662:00 +CPU= 13.220
  22:22:25 STEP 2663 H=2663:00 +CPU= 13.200
  22:22:48 STEP 2664 H=2664:00 +CPU= 22.563
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002664
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002664
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002640
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002652
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002652
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002652
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_110.zip
Uploading the intermediate file: upload_file_110.zip
  22:23:01 STEP 2665 H=2665:00 +CPU= 12.902
  22:23:14 STEP 2666 H=2666:00 +CPU= 13.058
  22:23:33 STEP 2667 H=2667:00 +CPU= 19.385
  22:23:46 STEP 2668 H=2668:00 +CPU= 13.190
  22:23:59 STEP 2669 H=2669:00 +CPU= 12.889
  22:24:19 STEP 2670 H=2670:00 +CPU= 19.706
  22:24:32 STEP 2671 H=2671:00 +CPU= 12.805
  22:24:45 STEP 2672 H=2672:00 +CPU= 13.247
  22:25:04 STEP 2673 H=2673:00 +CPU= 19.200
  22:25:17 STEP 2674 H=2674:00 +CPU= 13.154
  22:25:30 STEP 2675 H=2675:00 +CPU= 12.921
  22:25:51 STEP 2676 H=2676:00 +CPU= 20.919
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002676
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002676
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002676
  22:26:04 STEP 2677 H=2677:00 +CPU= 12.837
  22:26:17 STEP 2678 H=2678:00 +CPU= 13.149
  22:26:36 STEP 2679 H=2679:00 +CPU= 19.177
  22:26:50 STEP 2680 H=2680:00 +CPU= 13.151
  22:27:03 STEP 2681 H=2681:00 +CPU= 12.885
  22:27:22 STEP 2682 H=2682:00 +CPU= 19.457
  22:27:35 STEP 2683 H=2683:00 +CPU= 12.830
  22:27:48 STEP 2684 H=2684:00 +CPU= 13.260
  22:28:07 STEP 2685 H=2685:00 +CPU= 19.318
  22:28:21 STEP 2686 H=2686:00 +CPU= 13.232
  22:28:34 STEP 2687 H=2687:00 +CPU= 13.378
  22:28:57 STEP 2688 H=2688:00 +CPU= 22.564
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002688
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002688
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002664
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002676
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002676
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002676
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_111.zip
Uploading the intermediate file: upload_file_111.zip
  22:29:10 STEP 2689 H=2689:00 +CPU= 12.912
  22:29:23 STEP 2690 H=2690:00 +CPU= 13.152
  22:29:42 STEP 2691 H=2691:00 +CPU= 19.342
  22:29:55 STEP 2692 H=2692:00 +CPU= 13.294
  22:30:08 STEP 2693 H=2693:00 +CPU= 12.929
  22:30:28 STEP 2694 H=2694:00 +CPU= 19.646
  22:30:41 STEP 2695 H=2695:00 +CPU= 12.978
  22:30:54 STEP 2696 H=2696:00 +CPU= 13.356
  22:31:14 STEP 2697 H=2697:00 +CPU= 19.347
  22:31:27 STEP 2698 H=2698:00 +CPU= 13.242
  22:31:40 STEP 2699 H=2699:00 +CPU= 12.957
  22:32:01 STEP 2700 H=2700:00 +CPU= 20.798
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002700
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002700
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002700
  22:32:14 STEP 2701 H=2701:00 +CPU= 12.800
  22:32:27 STEP 2702 H=2702:00 +CPU= 13.259
  22:32:46 STEP 2703 H=2703:00 +CPU= 19.083
  22:32:59 STEP 2704 H=2704:00 +CPU= 13.379
  22:33:12 STEP 2705 H=2705:00 +CPU= 12.941
  22:33:32 STEP 2706 H=2706:00 +CPU= 19.661
  22:33:45 STEP 2707 H=2707:00 +CPU= 12.957
  22:33:58 STEP 2708 H=2708:00 +CPU= 13.301
  22:34:18 STEP 2709 H=2709:00 +CPU= 19.392
  22:34:31 STEP 2710 H=2710:00 +CPU= 13.286
  22:34:44 STEP 2711 H=2711:00 +CPU= 13.411
  22:35:07 STEP 2712 H=2712:00 +CPU= 22.470
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002712
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002712
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002688
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002700
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002700
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002700
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_112.zip
Uploading the intermediate file: upload_file_112.zip
  22:35:20 STEP 2713 H=2713:00 +CPU= 12.868
  22:35:33 STEP 2714 H=2714:00 +CPU= 13.218
  22:35:52 STEP 2715 H=2715:00 +CPU= 19.368
  22:36:06 STEP 2716 H=2716:00 +CPU= 13.233
  22:36:19 STEP 2717 H=2717:00 +CPU= 12.947
  22:36:38 STEP 2718 H=2718:00 +CPU= 19.772
  22:36:51 STEP 2719 H=2719:00 +CPU= 12.881
  22:37:04 STEP 2720 H=2720:00 +CPU= 13.259
  22:37:24 STEP 2721 H=2721:00 +CPU= 19.293
  22:37:37 STEP 2722 H=2722:00 +CPU= 13.163
  22:37:50 STEP 2723 H=2723:00 +CPU= 12.805
  22:38:11 STEP 2724 H=2724:00 +CPU= 20.960
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002724
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002724
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002724
  22:38:24 STEP 2725 H=2725:00 +CPU= 12.900
  22:38:37 STEP 2726 H=2726:00 +CPU= 13.252
  22:38:56 STEP 2727 H=2727:00 +CPU= 19.109
  22:39:09 STEP 2728 H=2728:00 +CPU= 13.123
  22:39:22 STEP 2729 H=2729:00 +CPU= 12.784
  22:39:42 STEP 2730 H=2730:00 +CPU= 19.677
  22:39:54 STEP 2731 H=2731:00 +CPU= 12.742
  22:40:08 STEP 2732 H=2732:00 +CPU= 13.128
  22:40:27 STEP 2733 H=2733:00 +CPU= 19.259
  22:40:40 STEP 2734 H=2734:00 +CPU= 13.060
  22:40:53 STEP 2735 H=2735:00 +CPU= 13.110
  22:41:15 STEP 2736 H=2736:00 +CPU= 22.363
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002736
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002736
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002712
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002724
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002724
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002724
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_113.zip
Uploading the intermediate file: upload_file_113.zip
  22:41:28 STEP 2737 H=2737:00 +CPU= 12.831
  22:41:41 STEP 2738 H=2738:00 +CPU= 13.158
  22:42:01 STEP 2739 H=2739:00 +CPU= 19.219
  22:42:14 STEP 2740 H=2740:00 +CPU= 13.253
  22:42:27 STEP 2741 H=2741:00 +CPU= 12.818
  22:42:47 STEP 2742 H=2742:00 +CPU= 19.873
  22:43:00 STEP 2743 H=2743:00 +CPU= 12.910
  22:43:13 STEP 2744 H=2744:00 +CPU= 13.254
  22:43:32 STEP 2745 H=2745:00 +CPU= 19.502
  22:43:46 STEP 2746 H=2746:00 +CPU= 13.239
  22:43:58 STEP 2747 H=2747:00 +CPU= 12.845
  22:44:20 STEP 2748 H=2748:00 +CPU= 21.139
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002748
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002748
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002748
  22:44:32 STEP 2749 H=2749:00 +CPU= 12.790
  22:44:45 STEP 2750 H=2750:00 +CPU= 13.162
  22:45:05 STEP 2751 H=2751:00 +CPU= 19.293
  22:45:18 STEP 2752 H=2752:00 +CPU= 13.284
  22:45:31 STEP 2753 H=2753:00 +CPU= 12.849
  22:45:51 STEP 2754 H=2754:00 +CPU= 19.732
  22:46:04 STEP 2755 H=2755:00 +CPU= 12.838
  22:46:17 STEP 2756 H=2756:00 +CPU= 13.125
  22:46:36 STEP 2757 H=2757:00 +CPU= 19.294
  22:46:49 STEP 2758 H=2758:00 +CPU= 13.242
  22:47:02 STEP 2759 H=2759:00 +CPU= 13.200
  22:47:25 STEP 2760 H=2760:00 +CPU= 22.427
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002760
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002760
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002736
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002748
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002748
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002748
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_114.zip
Uploading the intermediate file: upload_file_114.zip
  22:47:38 STEP 2761 H=2761:00 +CPU= 12.787
  22:47:51 STEP 2762 H=2762:00 +CPU= 13.142
  22:48:10 STEP 2763 H=2763:00 +CPU= 19.371
  22:48:23 STEP 2764 H=2764:00 +CPU= 13.221
  22:48:36 STEP 2765 H=2765:00 +CPU= 12.808
  22:48:56 STEP 2766 H=2766:00 +CPU= 19.865
  22:49:09 STEP 2767 H=2767:00 +CPU= 12.798
  22:49:22 STEP 2768 H=2768:00 +CPU= 13.173
  22:49:42 STEP 2769 H=2769:00 +CPU= 19.444
  22:49:55 STEP 2770 H=2770:00 +CPU= 13.135
  22:50:08 STEP 2771 H=2771:00 +CPU= 12.783
  22:50:29 STEP 2772 H=2772:00 +CPU= 20.978
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002772
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002772
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002772
  22:50:41 STEP 2773 H=2773:00 +CPU= 12.854
  22:50:55 STEP 2774 H=2774:00 +CPU= 13.135
  22:51:14 STEP 2775 H=2775:00 +CPU= 19.201
  22:51:27 STEP 2776 H=2776:00 +CPU= 13.181
  22:51:40 STEP 2777 H=2777:00 +CPU= 12.843
  22:52:00 STEP 2778 H=2778:00 +CPU= 19.828
  22:52:13 STEP 2779 H=2779:00 +CPU= 12.813
  22:52:26 STEP 2780 H=2780:00 +CPU= 13.210
  22:52:45 STEP 2781 H=2781:00 +CPU= 19.263
  22:52:58 STEP 2782 H=2782:00 +CPU= 13.267
  22:53:12 STEP 2783 H=2783:00 +CPU= 13.235
  22:53:34 STEP 2784 H=2784:00 +CPU= 22.440
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002784
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002784
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002760
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002772
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002772
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002772
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_115.zip
Uploading the intermediate file: upload_file_115.zip
  22:53:47 STEP 2785 H=2785:00 +CPU= 12.786
  22:54:00 STEP 2786 H=2786:00 +CPU= 13.091
  22:54:19 STEP 2787 H=2787:00 +CPU= 19.301
  22:54:32 STEP 2788 H=2788:00 +CPU= 13.170
  22:54:45 STEP 2789 H=2789:00 +CPU= 12.764
  22:55:05 STEP 2790 H=2790:00 +CPU= 19.832
  22:55:18 STEP 2791 H=2791:00 +CPU= 12.780
  22:55:31 STEP 2792 H=2792:00 +CPU= 13.090
  22:55:50 STEP 2793 H=2793:00 +CPU= 19.421
  22:56:03 STEP 2794 H=2794:00 +CPU= 13.100
  22:56:16 STEP 2795 H=2795:00 +CPU= 12.769
  22:56:37 STEP 2796 H=2796:00 +CPU= 20.973
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002796
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002796
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002796
  22:56:50 STEP 2797 H=2797:00 +CPU= 12.827
  22:57:03 STEP 2798 H=2798:00 +CPU= 13.171
  22:57:22 STEP 2799 H=2799:00 +CPU= 19.141
  22:57:36 STEP 2800 H=2800:00 +CPU= 13.167
  22:57:48 STEP 2801 H=2801:00 +CPU= 12.823
  22:58:08 STEP 2802 H=2802:00 +CPU= 19.708
  22:58:21 STEP 2803 H=2803:00 +CPU= 12.731
  22:58:34 STEP 2804 H=2804:00 +CPU= 13.114
  22:58:53 STEP 2805 H=2805:00 +CPU= 19.285
  22:59:07 STEP 2806 H=2806:00 +CPU= 13.257
  22:59:20 STEP 2807 H=2807:00 +CPU= 13.206
  22:59:42 STEP 2808 H=2808:00 +CPU= 22.475
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002808
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002808
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002784
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002796
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002796
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002796
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_116.zip
Uploading the intermediate file: upload_file_116.zip
  22:59:55 STEP 2809 H=2809:00 +CPU= 12.785
  23:00:08 STEP 2810 H=2810:00 +CPU= 13.141
  23:00:28 STEP 2811 H=2811:00 +CPU= 19.330
  23:00:41 STEP 2812 H=2812:00 +CPU= 13.266
  23:00:54 STEP 2813 H=2813:00 +CPU= 12.833
  23:01:14 STEP 2814 H=2814:00 +CPU= 19.948
  23:01:27 STEP 2815 H=2815:00 +CPU= 12.954
  23:01:40 STEP 2816 H=2816:00 +CPU= 13.213
  23:01:59 STEP 2817 H=2817:00 +CPU= 19.506
  23:02:12 STEP 2818 H=2818:00 +CPU= 13.174
  23:02:25 STEP 2819 H=2819:00 +CPU= 12.839
  23:02:46 STEP 2820 H=2820:00 +CPU= 21.099
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002820
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002820
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002820
  23:02:59 STEP 2821 H=2821:00 +CPU= 12.787
  23:03:12 STEP 2822 H=2822:00 +CPU= 13.103
  23:03:32 STEP 2823 H=2823:00 +CPU= 19.210
  23:03:45 STEP 2824 H=2824:00 +CPU= 13.231
  23:03:58 STEP 2825 H=2825:00 +CPU= 12.842
  23:04:17 STEP 2826 H=2826:00 +CPU= 19.778
  23:04:30 STEP 2827 H=2827:00 +CPU= 12.745
  23:04:43 STEP 2828 H=2828:00 +CPU= 13.166
  23:05:03 STEP 2829 H=2829:00 +CPU= 19.412
  23:05:16 STEP 2830 H=2830:00 +CPU= 13.159
  23:05:29 STEP 2831 H=2831:00 +CPU= 13.309
  23:05:52 STEP 2832 H=2832:00 +CPU= 22.499
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002832
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002832
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002808
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002820
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002820
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002820
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_117.zip
Uploading the intermediate file: upload_file_117.zip
  23:06:05 STEP 2833 H=2833:00 +CPU= 12.790
  23:06:18 STEP 2834 H=2834:00 +CPU= 13.127
  23:06:37 STEP 2835 H=2835:00 +CPU= 19.341
  23:06:50 STEP 2836 H=2836:00 +CPU= 13.293
  23:07:03 STEP 2837 H=2837:00 +CPU= 12.821
  23:07:23 STEP 2838 H=2838:00 +CPU= 19.944
  23:07:36 STEP 2839 H=2839:00 +CPU= 12.916
  23:07:49 STEP 2840 H=2840:00 +CPU= 13.224
  23:08:09 STEP 2841 H=2841:00 +CPU= 19.613
  23:08:22 STEP 2842 H=2842:00 +CPU= 13.181
  23:08:35 STEP 2843 H=2843:00 +CPU= 12.851
  23:08:56 STEP 2844 H=2844:00 +CPU= 21.008
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002844
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002844
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002844
  23:09:09 STEP 2845 H=2845:00 +CPU= 12.726
  23:09:22 STEP 2846 H=2846:00 +CPU= 13.089
  23:09:41 STEP 2847 H=2847:00 +CPU= 19.209
  23:09:54 STEP 2848 H=2848:00 +CPU= 13.224
  23:10:07 STEP 2849 H=2849:00 +CPU= 12.862
  23:10:27 STEP 2850 H=2850:00 +CPU= 19.700
  23:10:40 STEP 2851 H=2851:00 +CPU= 12.766
  23:10:53 STEP 2852 H=2852:00 +CPU= 13.199
  23:11:12 STEP 2853 H=2853:00 +CPU= 19.358
  23:11:26 STEP 2854 H=2854:00 +CPU= 13.283
  23:11:39 STEP 2855 H=2855:00 +CPU= 13.214
  23:12:01 STEP 2856 H=2856:00 +CPU= 22.407
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002856
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002856
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002832
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002844
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002844
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002844
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_118.zip
Uploading the intermediate file: upload_file_118.zip
  23:12:14 STEP 2857 H=2857:00 +CPU= 12.807
  23:12:27 STEP 2858 H=2858:00 +CPU= 13.144
  23:12:47 STEP 2859 H=2859:00 +CPU= 19.371
  23:13:00 STEP 2860 H=2860:00 +CPU= 13.252
  23:13:13 STEP 2861 H=2861:00 +CPU= 12.788
  23:13:32 STEP 2862 H=2862:00 +CPU= 19.831
  23:13:45 STEP 2863 H=2863:00 +CPU= 12.881
  23:13:59 STEP 2864 H=2864:00 +CPU= 13.204
  23:14:18 STEP 2865 H=2865:00 +CPU= 19.504
  23:14:31 STEP 2866 H=2866:00 +CPU= 13.243
  23:14:44 STEP 2867 H=2867:00 +CPU= 12.839
  23:15:05 STEP 2868 H=2868:00 +CPU= 20.910
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002868
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002868
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002868
  23:15:18 STEP 2869 H=2869:00 +CPU= 12.792
  23:15:31 STEP 2870 H=2870:00 +CPU= 13.114
  23:15:50 STEP 2871 H=2871:00 +CPU= 19.219
  23:16:03 STEP 2872 H=2872:00 +CPU= 13.223
  23:16:16 STEP 2873 H=2873:00 +CPU= 12.879
  23:16:36 STEP 2874 H=2874:00 +CPU= 19.678
  23:16:49 STEP 2875 H=2875:00 +CPU= 12.812
  23:17:02 STEP 2876 H=2876:00 +CPU= 13.260
  23:17:21 STEP 2877 H=2877:00 +CPU= 19.317
  23:17:35 STEP 2878 H=2878:00 +CPU= 13.205
  23:17:48 STEP 2879 H=2879:00 +CPU= 13.270
  23:18:10 STEP 2880 H=2880:00 +CPU= 22.459
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002880
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002880
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002856
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002868
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002868
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002868
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_119.zip
Uploading the intermediate file: upload_file_119.zip
  23:18:23 STEP 2881 H=2881:00 +CPU= 12.783
  23:18:36 STEP 2882 H=2882:00 +CPU= 13.085
Uploading trickle at timestep: 10368000
  23:18:56 STEP 2883 H=2883:00 +CPU= 19.370
  23:19:09 STEP 2884 H=2884:00 +CPU= 13.251
  23:19:22 STEP 2885 H=2885:00 +CPU= 12.805
  23:19:42 STEP 2886 H=2886:00 +CPU= 19.808
  23:19:54 STEP 2887 H=2887:00 +CPU= 12.779
  23:20:08 STEP 2888 H=2888:00 +CPU= 13.214
  23:20:27 STEP 2889 H=2889:00 +CPU= 19.393
  23:20:40 STEP 2890 H=2890:00 +CPU= 13.168
  23:20:53 STEP 2891 H=2891:00 +CPU= 12.863
  23:21:14 STEP 2892 H=2892:00 +CPU= 20.991
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002892
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002892
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002892
  23:21:27 STEP 2893 H=2893:00 +CPU= 12.846
  23:21:40 STEP 2894 H=2894:00 +CPU= 13.094
  23:21:59 STEP 2895 H=2895:00 +CPU= 19.241
  23:22:12 STEP 2896 H=2896:00 +CPU= 13.214
  23:22:25 STEP 2897 H=2897:00 +CPU= 12.866
  23:22:45 STEP 2898 H=2898:00 +CPU= 19.657
  23:22:58 STEP 2899 H=2899:00 +CPU= 12.768
  23:23:11 STEP 2900 H=2900:00 +CPU= 13.260
  23:23:30 STEP 2901 H=2901:00 +CPU= 19.246
  23:23:44 STEP 2902 H=2902:00 +CPU= 13.254
  23:23:57 STEP 2903 H=2903:00 +CPU= 13.182
  23:24:19 STEP 2904 H=2904:00 +CPU= 22.430
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002904
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002904
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002880
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002892
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002892
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002892
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_120.zip
Uploading the intermediate file: upload_file_120.zip
  23:24:32 STEP 2905 H=2905:00 +CPU= 12.897
  23:24:45 STEP 2906 H=2906:00 +CPU= 13.108
  23:25:05 STEP 2907 H=2907:00 +CPU= 19.373
  23:25:18 STEP 2908 H=2908:00 +CPU= 13.201
  23:25:31 STEP 2909 H=2909:00 +CPU= 12.773
  23:25:50 STEP 2910 H=2910:00 +CPU= 19.364
  23:26:02 STEP 2911 H=2911:00 +CPU= 11.696
  23:26:14 STEP 2912 H=2912:00 +CPU= 12.029
  23:26:32 STEP 2913 H=2913:00 +CPU= 17.942
  23:26:44 STEP 2914 H=2914:00 +CPU= 12.030
  23:26:55 STEP 2915 H=2915:00 +CPU= 11.699
  23:27:15 STEP 2916 H=2916:00 +CPU= 19.580
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002916
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002916
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002916
  23:27:27 STEP 2917 H=2917:00 +CPU= 11.786
  23:27:39 STEP 2918 H=2918:00 +CPU= 12.027
  23:27:57 STEP 2919 H=2919:00 +CPU= 17.897
  23:28:09 STEP 2920 H=2920:00 +CPU= 12.079
  23:28:21 STEP 2921 H=2921:00 +CPU= 11.658
  23:28:39 STEP 2922 H=2922:00 +CPU= 18.407
  23:28:50 STEP 2923 H=2923:00 +CPU= 11.472
  23:29:02 STEP 2924 H=2924:00 +CPU= 11.139
  23:29:19 STEP 2925 H=2925:00 +CPU= 17.005
  23:29:30 STEP 2926 H=2926:00 +CPU= 11.296
  23:29:41 STEP 2927 H=2927:00 +CPU= 11.334
  23:30:01 STEP 2928 H=2928:00 +CPU= 20.164
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002928
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002928
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002928
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002904
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002916
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002916
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002916
Zipping up the intermediate file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_121.zip
Uploading the intermediate file: upload_file_121.zip
  23:30:12 STEP 2929 H=2929:00 +CPU= 10.986
  23:30:24 STEP 2930 H=2930:00 +CPU= 11.333
  23:30:41 STEP 2931 H=2931:00 +CPU= 17.102
  23:30:52 STEP 2932 H=2932:00 +CPU= 11.309
  23:31:03 STEP 2933 H=2933:00 +CPU= 10.957
  23:31:21 STEP 2934 H=2934:00 +CPU= 17.529
  23:31:32 STEP 2935 H=2935:00 +CPU= 10.969
  23:31:43 STEP 2936 H=2936:00 +CPU= 11.282
  23:32:00 STEP 2937 H=2937:00 +CPU= 17.084
  23:32:11 STEP 2938 H=2938:00 +CPU= 11.250
  23:32:22 STEP 2939 H=2939:00 +CPU= 11.042
  23:32:41 STEP 2940 H=2940:00 +CPU= 18.845
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002940
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002940
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002940
  23:32:52 STEP 2941 H=2941:00 +CPU= 11.132
  23:33:04 STEP 2942 H=2942:00 +CPU= 11.437
  23:33:21 STEP 2943 H=2943:00 +CPU= 17.222
  23:33:32 STEP 2944 H=2944:00 +CPU= 11.398
  23:33:43 STEP 2945 H=2945:00 +CPU= 11.092
  23:34:01 STEP 2946 H=2946:00 +CPU= 17.609
  23:34:12 STEP 2947 H=2947:00 +CPU= 11.070
  23:34:24 STEP 2948 H=2948:00 +CPU= 11.390
  23:34:41 STEP 2949 H=2949:00 +CPU= 17.204
  23:34:52 STEP 2950 H=2950:00 +CPU= 11.384
  23:35:04 STEP 2951 H=2951:00 +CPU= 11.419
  23:35:23 STEP 2952 H=2952:00 +CPU= 19.556
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.19925486346338E-02 -1.00000000000000E+00  1.00000000000000E+00
  MAXGPFV : MAX. VALUE =    1.15500598430440     
  MAXGPFV : MAX. VALUE =   0.737460887821664     
 NSTEP =  2952 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhnw4+002952 MODE=a
 IO-STREAM CLOSED - ICMGGhnw4+002952
 MPI-TASK:   1 -    321020586 BYTES IN      8 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0055( 3.6%)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302231500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302233000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302234500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200303000000  !!!!!!!!!!!! 
 23:35:23 STEP 2952 H=2952:00 +CPU= 19.556
 END CNT3
 NSTEP =  2952 CNT0   000000000  16.898  16.898  16.905   1.000               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ          654/    28953814 WRITTEN        46048/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1116    0      654        28275    46048 ************   132.0     0.0   132.0     0.8   210.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********    100.00      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 3200    1561.2    1561.2     10.53      0.00
   9 SPCM     - SPECTRAL COMP.                    2952     255.1     255.1      1.59      0.00
  10 SCAN2M - PHYSICS                             2953    6649.4    6649.4     41.37      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                247     819.7     819.7      0.43      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                126      11.1      11.1      0.00      0.00
  13 SCAN2M - RADIATION CALC.                      985    6802.7    6802.7     14.12      0.00
  14 SUINIF                                          1    1337.4    1337.4      0.00      0.00
  17 GRIDFPOS IN CNT4                              247      26.4      26.4      0.01      0.00
  18 SUSPECG                                         1     256.0     256.0      0.00      0.00
  19 SUSPEC                                          1     261.7     261.7      0.00      0.00
  24 SUGRIDU                                         1     835.7     835.7      0.00      0.00
  25 SPECRT                                          1     106.0     106.0      0.00      0.00
  26 SUGRIDF                                         1     134.0     134.0      0.00      0.00
  27 RESTART FILES - WRITING                       123    1646.0    1646.0      0.43      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               247       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                               247    1278.6    1278.6      0.67      0.00
  31 POSDDH IN STEPO                                13       0.5       0.5      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   2953    2297.4    2297.4     14.29      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL          2952    1625.6    1625.6     10.11      0.00
  39 SU0YOMB                                         1     178.5     178.5      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   2953      15.1      15.1      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             2952     242.0     242.0      1.50      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             2952      37.6      37.6      0.23      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                126       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM    10094      88.4      88.4      1.88      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      6152      84.5      84.5      1.09      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       6152      34.4      34.4      0.45      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM     10094      30.8      30.8      0.66      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2      12.8      12.8      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION          10094      14.8      14.8      0.31      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           6152      13.0      13.0      0.17      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION          10094      14.6      14.6      0.31      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           6152       9.7       9.7      0.13      0.00
 400 GSTATS                                     589499       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                564603       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    47462.3 CPU TIME   47424.6 VECTOR TIME    47424.6


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      223.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /hdd/boinc-client/slots/2/ICMGGhnw4+002952
Moving to projects directory: /hdd/boinc-client/slots/2/ICMSHhnw4+002952
Moving to projects directory: /hdd/boinc-client/slots/2/ICMUAhnw4+002952
Adding to the zip: /hdd/boinc-client/slots/2/NODE.001_01
Adding to the zip: /hdd/boinc-client/slots/2/ifs.stat
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002952
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002952
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002940
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002940
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002928
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002952
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMGGhnw4+002940
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMSHhnw4+002928
Adding to the zip: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_12214557/ICMUAhnw4+002928
Zipping up the final file: /hdd/boinc-client/projects/climateprediction.net/oifs_43r3_00v3_2019110100_123_993_12214557_0_r1879186590_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
23:38:46 (66880): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:54:06 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 10,623,600 47,424 0.0045
09 May 2023 16:54:06 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 10,368,000 46,401 0.0045
09 May 2023 16:54:04 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 9,504,000 42,710 0.0045
09 May 2023 16:54:02 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 8,640,000 39,019 0.0045
09 May 2023 16:53:59 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 7,776,000 35,338 0.0045
09 May 2023 16:53:57 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 6,912,000 31,648 0.0046
09 May 2023 16:53:54 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 6,048,000 27,957 0.0046
09 May 2023 16:53:52 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 5,184,000 24,266 0.0047
09 May 2023 16:53:49 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 4,320,000 20,355 0.0047
09 May 2023 16:53:46 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 3,456,000 16,292 0.0047
09 May 2023 16:53:43 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 2,592,000 12,230 0.0047
09 May 2023 16:53:37 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 1,728,000 8,141 0.0047
09 May 2023 16:53:34 1497236 22315930 oifs_43r3_00v3_2019110100_123_993_12214557_0 864,000 4,070 0.0047


©2024 climateprediction.net