climateprediction.net home page
Task 22299828

Task 22299828

Name oifs_43r3_ps_0200_2009050100_123_978_12194844_2
Workunit 12194844
Created 19 Jan 2023, 11:40:28 UTC
Sent 19 Jan 2023, 12:01:20 UTC
Report deadline 20 Mar 2023, 12:01:20 UTC
Received 20 Jan 2023, 2:38:44 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1501535
Run time 13 hours 30 min 51 sec
CPU time 13 hours 19 min 27 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 7.22 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.84 MB
Peak swap size 4,978.08 MB
Peak disk usage 2,821.62 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
194844/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_99.zip
Uploading the intermediate file: upload_file_99.zip
  23:00:31 STEP 2401 H=2401:00 +CPU= 13.627
  23:00:45 STEP 2402 H=2402:00 +CPU= 13.620
  23:01:05 STEP 2403 H=2403:00 +CPU= 20.641
  23:01:19 STEP 2404 H=2404:00 +CPU= 13.902
  23:01:33 STEP 2405 H=2405:00 +CPU= 13.639
  23:01:54 STEP 2406 H=2406:00 +CPU= 20.854
  23:02:07 STEP 2407 H=2407:00 +CPU= 13.549
  23:02:21 STEP 2408 H=2408:00 +CPU= 13.940
  23:02:42 STEP 2409 H=2409:00 +CPU= 20.037
  23:02:56 STEP 2410 H=2410:00 +CPU= 13.905
  23:03:09 STEP 2411 H=2411:00 +CPU= 13.714
  23:03:32 STEP 2412 H=2412:00 +CPU= 22.805
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002412
  23:03:45 STEP 2413 H=2413:00 +CPU= 13.190
  23:03:59 STEP 2414 H=2414:00 +CPU= 13.976
  23:04:20 STEP 2415 H=2415:00 +CPU= 20.608
  23:04:34 STEP 2416 H=2416:00 +CPU= 13.956
  23:04:47 STEP 2417 H=2417:00 +CPU= 13.181
  23:05:09 STEP 2418 H=2418:00 +CPU= 21.085
  23:05:22 STEP 2419 H=2419:00 +CPU= 13.559
  23:05:36 STEP 2420 H=2420:00 +CPU= 13.948
  23:05:57 STEP 2421 H=2421:00 +CPU= 20.199
  23:06:11 STEP 2422 H=2422:00 +CPU= 13.965
  23:06:25 STEP 2423 H=2423:00 +CPU= 14.019
  23:06:52 STEP 2424 H=2424:00 +CPU= 24.824
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_100.zip
Uploading the intermediate file: upload_file_100.zip
  23:07:05 STEP 2425 H=2425:00 +CPU= 13.595
  23:07:19 STEP 2426 H=2426:00 +CPU= 13.942
  23:07:39 STEP 2427 H=2427:00 +CPU= 20.201
  23:07:53 STEP 2428 H=2428:00 +CPU= 13.818
  23:08:07 STEP 2429 H=2429:00 +CPU= 13.631
  23:08:28 STEP 2430 H=2430:00 +CPU= 20.945
  23:08:41 STEP 2431 H=2431:00 +CPU= 13.123
  23:08:55 STEP 2432 H=2432:00 +CPU= 14.079
  23:09:16 STEP 2433 H=2433:00 +CPU= 20.624
  23:09:30 STEP 2434 H=2434:00 +CPU= 13.900
  23:09:43 STEP 2435 H=2435:00 +CPU= 13.368
  23:10:06 STEP 2436 H=2436:00 +CPU= 22.870
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002436
  23:10:20 STEP 2437 H=2437:00 +CPU= 13.697
  23:10:34 STEP 2438 H=2438:00 +CPU= 14.079
  23:10:55 STEP 2439 H=2439:00 +CPU= 20.232
  23:11:09 STEP 2440 H=2440:00 +CPU= 13.945
  23:11:22 STEP 2441 H=2441:00 +CPU= 13.660
  23:11:43 STEP 2442 H=2442:00 +CPU= 20.808
  23:11:57 STEP 2443 H=2443:00 +CPU= 13.669
  23:12:11 STEP 2444 H=2444:00 +CPU= 13.981
  23:12:32 STEP 2445 H=2445:00 +CPU= 20.613
  23:12:45 STEP 2446 H=2446:00 +CPU= 13.477
  23:13:00 STEP 2447 H=2447:00 +CPU= 14.033
  23:13:26 STEP 2448 H=2448:00 +CPU= 25.075
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_101.zip
Uploading the intermediate file: upload_file_101.zip
  23:13:39 STEP 2449 H=2449:00 +CPU= 13.120
  23:13:53 STEP 2450 H=2450:00 +CPU= 13.890
  23:14:14 STEP 2451 H=2451:00 +CPU= 20.690
  23:14:28 STEP 2452 H=2452:00 +CPU= 13.971
  23:14:41 STEP 2453 H=2453:00 +CPU= 13.096
  23:15:02 STEP 2454 H=2454:00 +CPU= 20.975
  23:15:16 STEP 2455 H=2455:00 +CPU= 13.615
  23:15:30 STEP 2456 H=2456:00 +CPU= 13.982
  23:15:50 STEP 2457 H=2457:00 +CPU= 20.366
  23:16:04 STEP 2458 H=2458:00 +CPU= 13.999
  23:16:18 STEP 2459 H=2459:00 +CPU= 13.688
  23:16:40 STEP 2460 H=2460:00 +CPU= 22.349
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002460
  23:16:54 STEP 2461 H=2461:00 +CPU= 13.711
  23:17:08 STEP 2462 H=2462:00 +CPU= 13.982
  23:17:29 STEP 2463 H=2463:00 +CPU= 20.544
  23:17:43 STEP 2464 H=2464:00 +CPU= 13.745
  23:17:56 STEP 2465 H=2465:00 +CPU= 13.673
  23:18:18 STEP 2466 H=2466:00 +CPU= 20.963
  23:18:31 STEP 2467 H=2467:00 +CPU= 13.607
  23:18:45 STEP 2468 H=2468:00 +CPU= 13.568
  23:19:06 STEP 2469 H=2469:00 +CPU= 20.600
  23:19:20 STEP 2470 H=2470:00 +CPU= 13.976
  23:19:34 STEP 2471 H=2471:00 +CPU= 14.006
  23:20:01 STEP 2472 H=2472:00 +CPU= 24.754
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_102.zip
Uploading the intermediate file: upload_file_102.zip
  23:20:15 STEP 2473 H=2473:00 +CPU= 13.509
  23:20:29 STEP 2474 H=2474:00 +CPU= 13.982
  23:20:49 STEP 2475 H=2475:00 +CPU= 20.277
  23:21:03 STEP 2476 H=2476:00 +CPU= 13.835
  23:21:17 STEP 2477 H=2477:00 +CPU= 13.626
  23:21:37 STEP 2478 H=2478:00 +CPU= 20.558
  23:21:51 STEP 2479 H=2479:00 +CPU= 13.352
  23:22:05 STEP 2480 H=2480:00 +CPU= 13.991
  23:22:25 STEP 2481 H=2481:00 +CPU= 20.569
  23:22:39 STEP 2482 H=2482:00 +CPU= 13.381
  23:22:53 STEP 2483 H=2483:00 +CPU= 13.615
  23:23:15 STEP 2484 H=2484:00 +CPU= 22.834
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002484
  23:23:29 STEP 2485 H=2485:00 +CPU= 13.718
  23:23:43 STEP 2486 H=2486:00 +CPU= 13.561
  23:24:04 STEP 2487 H=2487:00 +CPU= 20.564
  23:24:18 STEP 2488 H=2488:00 +CPU= 13.931
  23:24:31 STEP 2489 H=2489:00 +CPU= 13.602
  23:24:52 STEP 2490 H=2490:00 +CPU= 20.697
  23:25:06 STEP 2491 H=2491:00 +CPU= 13.616
  23:25:20 STEP 2492 H=2492:00 +CPU= 13.948
  23:25:40 STEP 2493 H=2493:00 +CPU= 20.123
  23:25:54 STEP 2494 H=2494:00 +CPU= 13.943
  23:26:08 STEP 2495 H=2495:00 +CPU= 14.056
  23:26:35 STEP 2496 H=2496:00 +CPU= 24.713
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_103.zip
Uploading the intermediate file: upload_file_103.zip
  23:26:49 STEP 2497 H=2497:00 +CPU= 13.394
  23:27:03 STEP 2498 H=2498:00 +CPU= 13.935
Uploading trickle at timestep: 8985600
  23:27:23 STEP 2499 H=2499:00 +CPU= 20.659
  23:27:37 STEP 2500 H=2500:00 +CPU= 13.524
  23:27:50 STEP 2501 H=2501:00 +CPU= 13.403
  23:28:12 STEP 2502 H=2502:00 +CPU= 20.997
  23:28:25 STEP 2503 H=2503:00 +CPU= 13.619
  23:28:39 STEP 2504 H=2504:00 +CPU= 13.420
  23:28:59 STEP 2505 H=2505:00 +CPU= 20.625
  23:29:13 STEP 2506 H=2506:00 +CPU= 13.971
  23:29:27 STEP 2507 H=2507:00 +CPU= 13.659
  23:29:50 STEP 2508 H=2508:00 +CPU= 22.524
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002508
  23:30:03 STEP 2509 H=2509:00 +CPU= 13.617
  23:30:18 STEP 2510 H=2510:00 +CPU= 14.020
  23:30:38 STEP 2511 H=2511:00 +CPU= 20.110
  23:30:52 STEP 2512 H=2512:00 +CPU= 14.012
  23:31:06 STEP 2513 H=2513:00 +CPU= 13.663
  23:31:27 STEP 2514 H=2514:00 +CPU= 20.930
  23:31:40 STEP 2515 H=2515:00 +CPU= 13.295
  23:31:54 STEP 2516 H=2516:00 +CPU= 14.028
  23:32:15 STEP 2517 H=2517:00 +CPU= 20.588
  23:32:29 STEP 2518 H=2518:00 +CPU= 13.960
  23:32:43 STEP 2519 H=2519:00 +CPU= 13.651
  23:33:09 STEP 2520 H=2520:00 +CPU= 25.066
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_104.zip
Uploading the intermediate file: upload_file_104.zip
  23:33:23 STEP 2521 H=2521:00 +CPU= 13.637
  23:33:36 STEP 2522 H=2522:00 +CPU= 13.384
  23:33:57 STEP 2523 H=2523:00 +CPU= 20.490
  23:34:11 STEP 2524 H=2524:00 +CPU= 13.967
  23:34:25 STEP 2525 H=2525:00 +CPU= 13.622
  23:34:45 STEP 2526 H=2526:00 +CPU= 20.510
  23:34:59 STEP 2527 H=2527:00 +CPU= 13.673
  23:35:13 STEP 2528 H=2528:00 +CPU= 13.914
  23:35:34 STEP 2529 H=2529:00 +CPU= 20.486
  23:35:47 STEP 2530 H=2530:00 +CPU= 13.619
  23:36:01 STEP 2531 H=2531:00 +CPU= 13.735
  23:36:24 STEP 2532 H=2532:00 +CPU= 22.783
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002532
  23:36:37 STEP 2533 H=2533:00 +CPU= 13.047
  23:36:51 STEP 2534 H=2534:00 +CPU= 14.018
  23:37:12 STEP 2535 H=2535:00 +CPU= 20.577
  23:37:26 STEP 2536 H=2536:00 +CPU= 13.957
  23:37:39 STEP 2537 H=2537:00 +CPU= 13.217
  23:38:00 STEP 2538 H=2538:00 +CPU= 20.974
  23:38:14 STEP 2539 H=2539:00 +CPU= 13.638
  23:38:28 STEP 2540 H=2540:00 +CPU= 13.978
  23:38:49 STEP 2541 H=2541:00 +CPU= 20.363
  23:39:03 STEP 2542 H=2542:00 +CPU= 13.961
  23:39:17 STEP 2543 H=2543:00 +CPU= 13.954
  23:39:44 STEP 2544 H=2544:00 +CPU= 24.598
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_105.zip
Uploading the intermediate file: upload_file_105.zip
  23:39:57 STEP 2545 H=2545:00 +CPU= 13.548
  23:40:11 STEP 2546 H=2546:00 +CPU= 13.974
  23:40:32 STEP 2547 H=2547:00 +CPU= 20.613
  23:40:46 STEP 2548 H=2548:00 +CPU= 13.489
  23:40:59 STEP 2549 H=2549:00 +CPU= 13.724
  23:41:21 STEP 2550 H=2550:00 +CPU= 20.967
  23:41:34 STEP 2551 H=2551:00 +CPU= 13.345
  23:41:48 STEP 2552 H=2552:00 +CPU= 13.783
  23:42:09 STEP 2553 H=2553:00 +CPU= 20.673
  23:42:23 STEP 2554 H=2554:00 +CPU= 14.014
  23:42:36 STEP 2555 H=2555:00 +CPU= 13.156
  23:42:59 STEP 2556 H=2556:00 +CPU= 22.687
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002556
  23:43:12 STEP 2557 H=2557:00 +CPU= 13.592
  23:43:26 STEP 2558 H=2558:00 +CPU= 13.969
  23:43:47 STEP 2559 H=2559:00 +CPU= 20.384
  23:44:01 STEP 2560 H=2560:00 +CPU= 14.006
  23:44:15 STEP 2561 H=2561:00 +CPU= 13.651
  23:44:35 STEP 2562 H=2562:00 +CPU= 20.482
  23:44:49 STEP 2563 H=2563:00 +CPU= 13.558
  23:45:03 STEP 2564 H=2564:00 +CPU= 13.938
  23:45:24 STEP 2565 H=2565:00 +CPU= 20.603
  23:45:37 STEP 2566 H=2566:00 +CPU= 13.409
  23:45:51 STEP 2567 H=2567:00 +CPU= 13.992
  23:46:18 STEP 2568 H=2568:00 +CPU= 25.039
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_106.zip
Uploading the intermediate file: upload_file_106.zip
  23:46:32 STEP 2569 H=2569:00 +CPU= 13.524
  23:46:45 STEP 2570 H=2570:00 +CPU= 13.388
  23:47:06 STEP 2571 H=2571:00 +CPU= 20.605
  23:47:20 STEP 2572 H=2572:00 +CPU= 13.933
  23:47:33 STEP 2573 H=2573:00 +CPU= 13.337
  23:47:54 STEP 2574 H=2574:00 +CPU= 20.608
  23:48:08 STEP 2575 H=2575:00 +CPU= 13.721
  23:48:22 STEP 2576 H=2576:00 +CPU= 13.988
  23:48:42 STEP 2577 H=2577:00 +CPU= 20.109
  23:48:56 STEP 2578 H=2578:00 +CPU= 13.951
  23:49:10 STEP 2579 H=2579:00 +CPU= 13.634
  23:49:32 STEP 2580 H=2580:00 +CPU= 22.583
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002580
  23:49:46 STEP 2581 H=2581:00 +CPU= 13.304
  23:50:00 STEP 2582 H=2582:00 +CPU= 14.031
  23:50:20 STEP 2583 H=2583:00 +CPU= 20.589
  23:50:34 STEP 2584 H=2584:00 +CPU= 13.564
  23:50:48 STEP 2585 H=2585:00 +CPU= 13.558
  23:51:09 STEP 2586 H=2586:00 +CPU= 20.960
  23:51:23 STEP 2587 H=2587:00 +CPU= 13.630
  23:51:36 STEP 2588 H=2588:00 +CPU= 13.507
  23:51:57 STEP 2589 H=2589:00 +CPU= 20.615
  23:52:11 STEP 2590 H=2590:00 +CPU= 13.924
  23:52:25 STEP 2591 H=2591:00 +CPU= 14.013
  23:52:51 STEP 2592 H=2592:00 +CPU= 24.617
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_107.zip
Uploading the intermediate file: upload_file_107.zip
  23:53:05 STEP 2593 H=2593:00 +CPU= 13.688
  23:53:19 STEP 2594 H=2594:00 +CPU= 13.946
  23:53:39 STEP 2595 H=2595:00 +CPU= 20.250
  23:53:54 STEP 2596 H=2596:00 +CPU= 14.028
  23:54:07 STEP 2597 H=2597:00 +CPU= 13.653
  23:54:28 STEP 2598 H=2598:00 +CPU= 20.995
  23:54:42 STEP 2599 H=2599:00 +CPU= 13.178
  23:54:56 STEP 2600 H=2600:00 +CPU= 14.047
  23:55:16 STEP 2601 H=2601:00 +CPU= 20.638
  23:55:31 STEP 2602 H=2602:00 +CPU= 13.971
  23:55:42 STEP 2603 H=2603:00 +CPU= 11.822
  23:56:03 STEP 2604 H=2604:00 +CPU= 20.229
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002604
  23:56:15 STEP 2605 H=2605:00 +CPU= 11.839
  23:56:27 STEP 2606 H=2606:00 +CPU= 12.189
  23:56:45 STEP 2607 H=2607:00 +CPU= 18.250
  23:56:57 STEP 2608 H=2608:00 +CPU= 12.163
  23:57:09 STEP 2609 H=2609:00 +CPU= 11.845
  23:57:28 STEP 2610 H=2610:00 +CPU= 18.710
  23:57:39 STEP 2611 H=2611:00 +CPU= 11.232
  23:57:52 STEP 2612 H=2612:00 +CPU= 12.287
  23:58:12 STEP 2613 H=2613:00 +CPU= 19.977
  23:58:26 STEP 2614 H=2614:00 +CPU= 13.885
  23:58:40 STEP 2615 H=2615:00 +CPU= 13.929
  23:59:06 STEP 2616 H=2616:00 +CPU= 24.654
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_108.zip
Uploading the intermediate file: upload_file_108.zip
  23:59:20 STEP 2617 H=2617:00 +CPU= 13.591
  23:59:34 STEP 2618 H=2618:00 +CPU= 13.915
  23:59:55 STEP 2619 H=2619:00 +CPU= 20.660
  00:00:08 STEP 2620 H=2620:00 +CPU= 13.487
  00:00:22 STEP 2621 H=2621:00 +CPU= 13.599
  00:00:43 STEP 2622 H=2622:00 +CPU= 20.937
  00:00:57 STEP 2623 H=2623:00 +CPU= 13.517
  00:01:10 STEP 2624 H=2624:00 +CPU= 13.527
  00:01:31 STEP 2625 H=2625:00 +CPU= 20.576
  00:01:45 STEP 2626 H=2626:00 +CPU= 13.898
  00:01:58 STEP 2627 H=2627:00 +CPU= 13.365
  00:02:21 STEP 2628 H=2628:00 +CPU= 22.404
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002628
  00:02:34 STEP 2629 H=2629:00 +CPU= 13.543
  00:02:48 STEP 2630 H=2630:00 +CPU= 13.978
  00:03:09 STEP 2631 H=2631:00 +CPU= 20.121
  00:03:23 STEP 2632 H=2632:00 +CPU= 13.990
  00:03:36 STEP 2633 H=2633:00 +CPU= 13.577
  00:03:57 STEP 2634 H=2634:00 +CPU= 20.819
  00:04:11 STEP 2635 H=2635:00 +CPU= 13.186
  00:04:24 STEP 2636 H=2636:00 +CPU= 13.869
  00:04:45 STEP 2637 H=2637:00 +CPU= 20.530
  00:04:59 STEP 2638 H=2638:00 +CPU= 13.562
  00:05:13 STEP 2639 H=2639:00 +CPU= 13.749
  00:05:40 STEP 2640 H=2640:00 +CPU= 24.847
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_109.zip
Uploading the intermediate file: upload_file_109.zip
  00:05:54 STEP 2641 H=2641:00 +CPU= 13.513
  00:06:07 STEP 2642 H=2642:00 +CPU= 13.473
  00:06:28 STEP 2643 H=2643:00 +CPU= 20.656
  00:06:42 STEP 2644 H=2644:00 +CPU= 13.958
  00:06:56 STEP 2645 H=2645:00 +CPU= 13.550
  00:07:16 STEP 2646 H=2646:00 +CPU= 20.403
  00:07:30 STEP 2647 H=2647:00 +CPU= 13.517
  00:07:44 STEP 2648 H=2648:00 +CPU= 13.872
  00:08:04 STEP 2649 H=2649:00 +CPU= 20.170
  00:08:18 STEP 2650 H=2650:00 +CPU= 13.885
  00:08:32 STEP 2651 H=2651:00 +CPU= 13.560
  00:08:54 STEP 2652 H=2652:00 +CPU= 22.669
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002652
  00:09:08 STEP 2653 H=2653:00 +CPU= 13.113
  00:09:22 STEP 2654 H=2654:00 +CPU= 13.829
  00:09:42 STEP 2655 H=2655:00 +CPU= 20.496
  00:09:56 STEP 2656 H=2656:00 +CPU= 13.889
  00:10:09 STEP 2657 H=2657:00 +CPU= 13.093
  00:10:30 STEP 2658 H=2658:00 +CPU= 20.905
  00:10:44 STEP 2659 H=2659:00 +CPU= 13.535
  00:10:58 STEP 2660 H=2660:00 +CPU= 13.659
  00:11:18 STEP 2661 H=2661:00 +CPU= 20.118
  00:11:32 STEP 2662 H=2662:00 +CPU= 13.848
  00:11:46 STEP 2663 H=2663:00 +CPU= 13.952
  00:12:12 STEP 2664 H=2664:00 +CPU= 24.768
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_110.zip
Uploading the intermediate file: upload_file_110.zip
  00:12:26 STEP 2665 H=2665:00 +CPU= 13.494
  00:12:40 STEP 2666 H=2666:00 +CPU= 13.876
  00:13:00 STEP 2667 H=2667:00 +CPU= 20.180
  00:13:14 STEP 2668 H=2668:00 +CPU= 13.774
  00:13:27 STEP 2669 H=2669:00 +CPU= 13.541
  00:13:48 STEP 2670 H=2670:00 +CPU= 20.921
  00:14:01 STEP 2671 H=2671:00 +CPU= 13.057
  00:14:15 STEP 2672 H=2672:00 +CPU= 13.896
  00:14:36 STEP 2673 H=2673:00 +CPU= 20.568
  00:14:50 STEP 2674 H=2674:00 +CPU= 13.913
  00:15:03 STEP 2675 H=2675:00 +CPU= 13.168
  00:15:26 STEP 2676 H=2676:00 +CPU= 22.676
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002676
  00:15:40 STEP 2677 H=2677:00 +CPU= 13.500
  00:15:54 STEP 2678 H=2678:00 +CPU= 13.918
  00:16:14 STEP 2679 H=2679:00 +CPU= 20.316
  00:16:28 STEP 2680 H=2680:00 +CPU= 13.904
  00:16:42 STEP 2681 H=2681:00 +CPU= 13.526
  00:17:02 STEP 2682 H=2682:00 +CPU= 20.478
  00:17:16 STEP 2683 H=2683:00 +CPU= 13.505
  00:17:30 STEP 2684 H=2684:00 +CPU= 13.962
  00:17:50 STEP 2685 H=2685:00 +CPU= 20.536
  00:18:04 STEP 2686 H=2686:00 +CPU= 13.620
  00:18:18 STEP 2687 H=2687:00 +CPU= 13.929
  00:18:45 STEP 2688 H=2688:00 +CPU= 25.154
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_111.zip
Uploading the intermediate file: upload_file_111.zip
  00:18:58 STEP 2689 H=2689:00 +CPU= 13.227
  00:19:12 STEP 2690 H=2690:00 +CPU= 13.569
  00:19:32 STEP 2691 H=2691:00 +CPU= 20.477
  00:19:46 STEP 2692 H=2692:00 +CPU= 13.881
  00:19:59 STEP 2693 H=2693:00 +CPU= 13.119
  00:20:20 STEP 2694 H=2694:00 +CPU= 20.753
  00:20:34 STEP 2695 H=2695:00 +CPU= 13.513
  00:20:48 STEP 2696 H=2696:00 +CPU= 13.919
  00:21:08 STEP 2697 H=2697:00 +CPU= 20.045
  00:21:22 STEP 2698 H=2698:00 +CPU= 13.874
  00:21:35 STEP 2699 H=2699:00 +CPU= 13.561
  00:21:58 STEP 2700 H=2700:00 +CPU= 22.516
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002700
  00:22:12 STEP 2701 H=2701:00 +CPU= 13.389
  00:22:26 STEP 2702 H=2702:00 +CPU= 13.848
  00:22:46 STEP 2703 H=2703:00 +CPU= 20.517
  00:23:00 STEP 2704 H=2704:00 +CPU= 13.463
  00:23:13 STEP 2705 H=2705:00 +CPU= 13.453
  00:23:34 STEP 2706 H=2706:00 +CPU= 20.907
  00:23:48 STEP 2707 H=2707:00 +CPU= 13.546
  00:24:01 STEP 2708 H=2708:00 +CPU= 13.407
  00:24:22 STEP 2709 H=2709:00 +CPU= 20.400
  00:24:36 STEP 2710 H=2710:00 +CPU= 13.852
  00:24:50 STEP 2711 H=2711:00 +CPU= 13.974
  00:25:16 STEP 2712 H=2712:00 +CPU= 24.684
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_112.zip
Uploading the intermediate file: upload_file_112.zip
  00:25:30 STEP 2713 H=2713:00 +CPU= 13.605
  00:25:44 STEP 2714 H=2714:00 +CPU= 13.885
  00:26:04 STEP 2715 H=2715:00 +CPU= 20.271
  00:26:18 STEP 2716 H=2716:00 +CPU= 13.925
  00:26:32 STEP 2717 H=2717:00 +CPU= 13.581
  00:26:53 STEP 2718 H=2718:00 +CPU= 20.929
  00:27:06 STEP 2719 H=2719:00 +CPU= 13.182
  00:27:20 STEP 2720 H=2720:00 +CPU= 13.973
  00:27:40 STEP 2721 H=2721:00 +CPU= 20.555
  00:27:54 STEP 2722 H=2722:00 +CPU= 13.906
  00:28:08 STEP 2723 H=2723:00 +CPU= 13.204
  00:28:31 STEP 2724 H=2724:00 +CPU= 22.672
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002724
  00:28:44 STEP 2725 H=2725:00 +CPU= 13.569
  00:28:58 STEP 2726 H=2726:00 +CPU= 13.634
  00:29:18 STEP 2727 H=2727:00 +CPU= 20.128
  00:29:32 STEP 2728 H=2728:00 +CPU= 13.890
  00:29:46 STEP 2729 H=2729:00 +CPU= 13.555
  00:30:06 STEP 2730 H=2730:00 +CPU= 20.383
  00:30:20 STEP 2731 H=2731:00 +CPU= 13.530
  00:30:34 STEP 2732 H=2732:00 +CPU= 13.888
  00:30:54 STEP 2733 H=2733:00 +CPU= 20.470
  00:31:08 STEP 2734 H=2734:00 +CPU= 13.407
  00:31:22 STEP 2735 H=2735:00 +CPU= 13.934
  00:31:48 STEP 2736 H=2736:00 +CPU= 24.990
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_113.zip
Uploading the intermediate file: upload_file_113.zip
  00:32:01 STEP 2737 H=2737:00 +CPU= 12.972
  00:32:15 STEP 2738 H=2738:00 +CPU= 13.925
Uploading trickle at timestep: 9849600
  00:32:36 STEP 2739 H=2739:00 +CPU= 20.558
  00:32:50 STEP 2740 H=2740:00 +CPU= 13.907
  00:33:03 STEP 2741 H=2741:00 +CPU= 13.256
  00:33:24 STEP 2742 H=2742:00 +CPU= 20.879
  00:33:38 STEP 2743 H=2743:00 +CPU= 13.574
  00:33:52 STEP 2744 H=2744:00 +CPU= 13.962
  00:34:12 STEP 2745 H=2745:00 +CPU= 20.241
  00:34:26 STEP 2746 H=2746:00 +CPU= 13.899
  00:34:40 STEP 2747 H=2747:00 +CPU= 13.556
  00:35:02 STEP 2748 H=2748:00 +CPU= 22.248
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002748
  00:35:16 STEP 2749 H=2749:00 +CPU= 13.640
  00:35:30 STEP 2750 H=2750:00 +CPU= 13.936
  00:35:50 STEP 2751 H=2751:00 +CPU= 20.451
  00:36:04 STEP 2752 H=2752:00 +CPU= 13.575
  00:36:18 STEP 2753 H=2753:00 +CPU= 13.573
  00:36:39 STEP 2754 H=2754:00 +CPU= 20.860
  00:36:52 STEP 2755 H=2755:00 +CPU= 13.504
  00:37:06 STEP 2756 H=2756:00 +CPU= 13.568
  00:37:26 STEP 2757 H=2757:00 +CPU= 20.440
  00:37:41 STEP 2758 H=2758:00 +CPU= 14.180
  00:37:55 STEP 2759 H=2759:00 +CPU= 13.977
  00:38:22 STEP 2760 H=2760:00 +CPU= 24.586
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_114.zip
Uploading the intermediate file: upload_file_114.zip
  00:38:35 STEP 2761 H=2761:00 +CPU= 13.479
  00:38:49 STEP 2762 H=2762:00 +CPU= 13.876
  00:39:09 STEP 2763 H=2763:00 +CPU= 20.052
  00:39:23 STEP 2764 H=2764:00 +CPU= 13.873
  00:39:37 STEP 2765 H=2765:00 +CPU= 13.565
  00:39:58 STEP 2766 H=2766:00 +CPU= 20.793
  00:40:11 STEP 2767 H=2767:00 +CPU= 12.994
  00:40:25 STEP 2768 H=2768:00 +CPU= 13.894
  00:40:45 STEP 2769 H=2769:00 +CPU= 20.457
  00:40:59 STEP 2770 H=2770:00 +CPU= 13.595
  00:41:12 STEP 2771 H=2771:00 +CPU= 13.304
  00:41:35 STEP 2772 H=2772:00 +CPU= 22.725
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002772
  00:41:49 STEP 2773 H=2773:00 +CPU= 13.599
  00:42:02 STEP 2774 H=2774:00 +CPU= 13.409
  00:42:23 STEP 2775 H=2775:00 +CPU= 20.526
  00:42:37 STEP 2776 H=2776:00 +CPU= 13.928
  00:42:51 STEP 2777 H=2777:00 +CPU= 13.581
  00:43:11 STEP 2778 H=2778:00 +CPU= 20.502
  00:43:25 STEP 2779 H=2779:00 +CPU= 13.538
  00:43:39 STEP 2780 H=2780:00 +CPU= 13.968
  00:43:59 STEP 2781 H=2781:00 +CPU= 20.149
  00:44:13 STEP 2782 H=2782:00 +CPU= 13.629
  00:44:27 STEP 2783 H=2783:00 +CPU= 13.917
  00:44:54 STEP 2784 H=2784:00 +CPU= 24.876
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_115.zip
Uploading the intermediate file: upload_file_115.zip
  00:45:08 STEP 2785 H=2785:00 +CPU= 13.132
  00:45:21 STEP 2786 H=2786:00 +CPU= 13.756
  00:45:42 STEP 2787 H=2787:00 +CPU= 20.451
  00:45:56 STEP 2788 H=2788:00 +CPU= 13.918
  00:46:09 STEP 2789 H=2789:00 +CPU= 13.061
  00:46:30 STEP 2790 H=2790:00 +CPU= 20.757
  00:46:44 STEP 2791 H=2791:00 +CPU= 13.517
  00:46:57 STEP 2792 H=2792:00 +CPU= 13.828
  00:47:18 STEP 2793 H=2793:00 +CPU= 20.080
  00:47:32 STEP 2794 H=2794:00 +CPU= 13.830
  00:47:45 STEP 2795 H=2795:00 +CPU= 13.523
  00:48:08 STEP 2796 H=2796:00 +CPU= 22.303
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002796
  00:48:21 STEP 2797 H=2797:00 +CPU= 13.502
  00:48:35 STEP 2798 H=2798:00 +CPU= 13.955
  00:48:56 STEP 2799 H=2799:00 +CPU= 20.479
  00:49:09 STEP 2800 H=2800:00 +CPU= 13.325
  00:49:23 STEP 2801 H=2801:00 +CPU= 13.550
  00:49:44 STEP 2802 H=2802:00 +CPU= 20.862
  00:49:57 STEP 2803 H=2803:00 +CPU= 13.433
  00:50:11 STEP 2804 H=2804:00 +CPU= 13.367
  00:50:31 STEP 2805 H=2805:00 +CPU= 20.417
  00:50:45 STEP 2806 H=2806:00 +CPU= 13.765
  00:50:59 STEP 2807 H=2807:00 +CPU= 13.657
  00:51:25 STEP 2808 H=2808:00 +CPU= 24.584
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_116.zip
Uploading the intermediate file: upload_file_116.zip
  00:51:39 STEP 2809 H=2809:00 +CPU= 13.610
  00:51:53 STEP 2810 H=2810:00 +CPU= 13.832
  00:52:13 STEP 2811 H=2811:00 +CPU= 20.389
  00:52:27 STEP 2812 H=2812:00 +CPU= 13.978
  00:52:41 STEP 2813 H=2813:00 +CPU= 13.560
  00:53:02 STEP 2814 H=2814:00 +CPU= 20.443
  00:53:15 STEP 2815 H=2815:00 +CPU= 13.579
  00:53:29 STEP 2816 H=2816:00 +CPU= 13.897
  00:53:50 STEP 2817 H=2817:00 +CPU= 20.567
  00:54:03 STEP 2818 H=2818:00 +CPU= 13.624
  00:54:17 STEP 2819 H=2819:00 +CPU= 13.538
  00:54:40 STEP 2820 H=2820:00 +CPU= 22.603
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002820
  00:54:54 STEP 2821 H=2821:00 +CPU= 13.631
  00:55:07 STEP 2822 H=2822:00 +CPU= 13.543
  00:55:28 STEP 2823 H=2823:00 +CPU= 20.457
  00:55:42 STEP 2824 H=2824:00 +CPU= 13.884
  00:55:55 STEP 2825 H=2825:00 +CPU= 13.572
  00:56:16 STEP 2826 H=2826:00 +CPU= 20.477
  00:56:30 STEP 2827 H=2827:00 +CPU= 13.585
  00:56:43 STEP 2828 H=2828:00 +CPU= 13.824
  00:57:04 STEP 2829 H=2829:00 +CPU= 20.118
  00:57:17 STEP 2830 H=2830:00 +CPU= 13.782
  00:57:31 STEP 2831 H=2831:00 +CPU= 13.934
  00:57:57 STEP 2832 H=2832:00 +CPU= 24.704
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_117.zip
Uploading the intermediate file: upload_file_117.zip
  00:58:11 STEP 2833 H=2833:00 +CPU= 13.119
  00:58:25 STEP 2834 H=2834:00 +CPU= 13.838
  00:58:45 STEP 2835 H=2835:00 +CPU= 20.566
  00:58:59 STEP 2836 H=2836:00 +CPU= 13.483
  00:59:12 STEP 2837 H=2837:00 +CPU= 13.355
  00:59:33 STEP 2838 H=2838:00 +CPU= 20.852
  00:59:47 STEP 2839 H=2839:00 +CPU= 13.551
  01:00:00 STEP 2840 H=2840:00 +CPU= 13.489
  01:00:21 STEP 2841 H=2841:00 +CPU= 20.425
  01:00:35 STEP 2842 H=2842:00 +CPU= 13.928
  01:00:49 STEP 2843 H=2843:00 +CPU= 13.628
  01:01:11 STEP 2844 H=2844:00 +CPU= 22.484
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002844
  01:01:25 STEP 2845 H=2845:00 +CPU= 13.536
  01:01:39 STEP 2846 H=2846:00 +CPU= 13.788
  01:01:59 STEP 2847 H=2847:00 +CPU= 20.110
  01:02:13 STEP 2848 H=2848:00 +CPU= 13.716
  01:02:26 STEP 2849 H=2849:00 +CPU= 13.592
  01:02:47 STEP 2850 H=2850:00 +CPU= 20.898
  01:03:00 STEP 2851 H=2851:00 +CPU= 13.047
  01:03:14 STEP 2852 H=2852:00 +CPU= 13.757
  01:03:35 STEP 2853 H=2853:00 +CPU= 20.536
  01:03:49 STEP 2854 H=2854:00 +CPU= 13.882
  01:04:02 STEP 2855 H=2855:00 +CPU= 13.464
  01:04:29 STEP 2856 H=2856:00 +CPU= 24.833
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_118.zip
Uploading the intermediate file: upload_file_118.zip
  01:04:42 STEP 2857 H=2857:00 +CPU= 13.571
  01:04:56 STEP 2858 H=2858:00 +CPU= 13.913
  01:05:17 STEP 2859 H=2859:00 +CPU= 20.226
  01:05:31 STEP 2860 H=2860:00 +CPU= 13.935
  01:05:44 STEP 2861 H=2861:00 +CPU= 13.537
  01:06:05 STEP 2862 H=2862:00 +CPU= 20.654
  01:06:19 STEP 2863 H=2863:00 +CPU= 13.475
  01:06:32 STEP 2864 H=2864:00 +CPU= 13.893
  01:06:53 STEP 2865 H=2865:00 +CPU= 20.589
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2009050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0200
(argv4) batchid: 978
(argv5) wuid: 12194844
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/2
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c423866234ef2cb2cf83165e98902b00 to: /var/lib/boinc/slots/2/oifs_43r3_ps_0200_2009050100_123_978_12194844.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/2/oifs_43r3_ps_0200_2009050100_123_978_12194844.zip
ic_ancil_file: ic_ancil_12194844
ifsdata_file: ifsdata_12194844
climate_data_file: clim_data_12194844
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0042d62531240ea6c6d1975d7e91ccb2 to: /var/lib/boinc/slots/2/ic_ancil_12194844.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/2/ic_ancil_12194844.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_201782110a7d4b947def37dd9bb51483 to: /var/lib/boinc/slots/2/ifsdata/ifsdata_12194844.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/2/ifsdata/ifsdata_12194844.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/2/159l_2/clim_data_12194844.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/2/159l_2/clim_data_12194844.zip
Checking for progress XML file: /var/lib/boinc/slots/2/progress_file_12194844.xml
Opened progress file ok : /var/lib/boinc/slots/2/progress_file_12194844.xml
-- Model is restarting --
Adjusting last_iter, 2865, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/2/progress_file_12194844.xml
last_cpu_time: 46392
upload_file_number: 119
last_iter: 2855
last_upload: 10281600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339
The child process has been launched with process id: 77572
Executing the command: /var/lib/boinc/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:scruffy.lockley.uk:1:1:77572:77572] [20230120:010802:1674176882:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  01:08:32 STEP 2856 H=2856:00 +CPU= 23.244
  01:08:45 STEP 2857 H=2857:00 +CPU= 13.451
  01:08:59 STEP 2858 H=2858:00 +CPU= 13.351
  01:09:19 STEP 2859 H=2859:00 +CPU= 20.414
  01:09:33 STEP 2860 H=2860:00 +CPU= 13.810
  01:09:46 STEP 2861 H=2861:00 +CPU= 13.385
  01:10:07 STEP 2862 H=2862:00 +CPU= 20.509
  01:10:21 STEP 2863 H=2863:00 +CPU= 13.488
  01:10:35 STEP 2864 H=2864:00 +CPU= 13.924
  01:10:55 STEP 2865 H=2865:00 +CPU= 19.982
  01:11:09 STEP 2866 H=2866:00 +CPU= 14.002
  01:11:22 STEP 2867 H=2867:00 +CPU= 13.577
  01:11:45 STEP 2868 H=2868:00 +CPU= 22.531
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002868
  01:11:58 STEP 2869 H=2869:00 +CPU= 13.095
  01:12:12 STEP 2870 H=2870:00 +CPU= 13.889
  01:12:33 STEP 2871 H=2871:00 +CPU= 20.444
  01:12:46 STEP 2872 H=2872:00 +CPU= 13.497
  01:13:00 STEP 2873 H=2873:00 +CPU= 13.292
  01:13:21 STEP 2874 H=2874:00 +CPU= 20.904
  01:13:34 STEP 2875 H=2875:00 +CPU= 13.551
  01:13:48 STEP 2876 H=2876:00 +CPU= 13.361
  01:14:08 STEP 2877 H=2877:00 +CPU= 20.250
  01:14:22 STEP 2878 H=2878:00 +CPU= 13.863
  01:14:36 STEP 2879 H=2879:00 +CPU= 13.906
  01:15:02 STEP 2880 H=2880:00 +CPU= 24.699
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_119.zip
Uploading the intermediate file: upload_file_119.zip
  01:15:16 STEP 2881 H=2881:00 +CPU= 13.439
  01:15:30 STEP 2882 H=2882:00 +CPU= 13.838
  01:15:50 STEP 2883 H=2883:00 +CPU= 20.053
  01:16:04 STEP 2884 H=2884:00 +CPU= 13.856
  01:16:17 STEP 2885 H=2885:00 +CPU= 13.496
  01:16:38 STEP 2886 H=2886:00 +CPU= 20.806
  01:16:52 STEP 2887 H=2887:00 +CPU= 13.253
  01:17:06 STEP 2888 H=2888:00 +CPU= 13.929
  01:17:26 STEP 2889 H=2889:00 +CPU= 20.522
  01:17:40 STEP 2890 H=2890:00 +CPU= 13.884
  01:17:54 STEP 2891 H=2891:00 +CPU= 13.213
  01:18:16 STEP 2892 H=2892:00 +CPU= 22.778
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002892
  01:18:30 STEP 2893 H=2893:00 +CPU= 13.558
  01:18:44 STEP 2894 H=2894:00 +CPU= 13.907
  01:19:04 STEP 2895 H=2895:00 +CPU= 20.119
  01:19:18 STEP 2896 H=2896:00 +CPU= 13.910
  01:19:32 STEP 2897 H=2897:00 +CPU= 13.560
  01:19:52 STEP 2898 H=2898:00 +CPU= 20.524
  01:20:06 STEP 2899 H=2899:00 +CPU= 13.526
  01:20:20 STEP 2900 H=2900:00 +CPU= 13.869
  01:20:40 STEP 2901 H=2901:00 +CPU= 20.419
  01:20:54 STEP 2902 H=2902:00 +CPU= 13.461
  01:21:08 STEP 2903 H=2903:00 +CPU= 13.914
  01:21:35 STEP 2904 H=2904:00 +CPU= 24.880
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_120.zip
Uploading the intermediate file: upload_file_120.zip
  01:21:48 STEP 2905 H=2905:00 +CPU= 13.148
  01:22:02 STEP 2906 H=2906:00 +CPU= 13.710
  01:22:22 STEP 2907 H=2907:00 +CPU= 20.442
  01:22:36 STEP 2908 H=2908:00 +CPU= 13.893
  01:22:49 STEP 2909 H=2909:00 +CPU= 12.959
  01:23:10 STEP 2910 H=2910:00 +CPU= 20.767
  01:23:24 STEP 2911 H=2911:00 +CPU= 13.535
  01:23:38 STEP 2912 H=2912:00 +CPU= 13.907
  01:23:58 STEP 2913 H=2913:00 +CPU= 20.025
  01:24:12 STEP 2914 H=2914:00 +CPU= 13.842
  01:24:25 STEP 2915 H=2915:00 +CPU= 13.476
  01:24:47 STEP 2916 H=2916:00 +CPU= 22.140
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002916
  01:25:01 STEP 2917 H=2917:00 +CPU= 13.457
  01:25:15 STEP 2918 H=2918:00 +CPU= 13.805
  01:25:35 STEP 2919 H=2919:00 +CPU= 20.342
  01:25:49 STEP 2920 H=2920:00 +CPU= 13.339
  01:26:02 STEP 2921 H=2921:00 +CPU= 13.461
  01:26:23 STEP 2922 H=2922:00 +CPU= 20.789
  01:26:37 STEP 2923 H=2923:00 +CPU= 13.577
  01:26:50 STEP 2924 H=2924:00 +CPU= 13.389
  01:27:11 STEP 2925 H=2925:00 +CPU= 20.439
  01:27:24 STEP 2926 H=2926:00 +CPU= 13.759
  01:27:38 STEP 2927 H=2927:00 +CPU= 13.917
  01:28:05 STEP 2928 H=2928:00 +CPU= 24.550
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_121.zip
Uploading the intermediate file: upload_file_121.zip
  01:28:18 STEP 2929 H=2929:00 +CPU= 13.456
  01:28:32 STEP 2930 H=2930:00 +CPU= 13.875
  01:28:52 STEP 2931 H=2931:00 +CPU= 20.122
  01:29:06 STEP 2932 H=2932:00 +CPU= 13.845
  01:29:20 STEP 2933 H=2933:00 +CPU= 13.461
  01:29:41 STEP 2934 H=2934:00 +CPU= 20.874
  01:29:54 STEP 2935 H=2935:00 +CPU= 13.143
  01:30:08 STEP 2936 H=2936:00 +CPU= 13.849
  01:30:28 STEP 2937 H=2937:00 +CPU= 20.383
  01:30:42 STEP 2938 H=2938:00 +CPU= 13.964
  01:30:56 STEP 2939 H=2939:00 +CPU= 13.164
  01:31:18 STEP 2940 H=2940:00 +CPU= 22.616
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002940
  01:31:32 STEP 2941 H=2941:00 +CPU= 13.521
  01:31:45 STEP 2942 H=2942:00 +CPU= 13.524
  01:32:06 STEP 2943 H=2943:00 +CPU= 20.187
  01:32:20 STEP 2944 H=2944:00 +CPU= 13.842
  01:32:33 STEP 2945 H=2945:00 +CPU= 13.517
  01:32:54 STEP 2946 H=2946:00 +CPU= 20.394
  01:33:07 STEP 2947 H=2947:00 +CPU= 13.524
  01:33:21 STEP 2948 H=2948:00 +CPU= 13.787
  01:33:42 STEP 2949 H=2949:00 +CPU= 20.546
  01:33:55 STEP 2950 H=2950:00 +CPU= 13.436
  01:34:09 STEP 2951 H=2951:00 +CPU= 13.912
  01:34:35 STEP 2952 H=2952:00 +CPU= 23.758
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12194844/ICMGGhq0f+002928
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0200_2009050100_123_978_12194844_2_r1008566339_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
01:37:58 (77567): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:29:45 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 10,623,600 47,967 0.0045
09 May 2023 16:29:39 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 9,849,600 44,316 0.0045
09 May 2023 16:29:33 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 8,985,600 40,441 0.0045
09 May 2023 16:29:24 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 7,776,000 34,953 0.0045
09 May 2023 16:29:18 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 6,912,000 31,066 0.0045
09 May 2023 16:29:12 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 6,048,000 27,192 0.0045
09 May 2023 16:29:06 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 5,184,000 23,308 0.0045
09 May 2023 16:28:59 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 4,320,000 19,432 0.0045
09 May 2023 16:28:53 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 3,456,000 15,546 0.0045
09 May 2023 16:28:47 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 2,592,000 11,660 0.0045
09 May 2023 16:28:41 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 1,728,000 7,778 0.0045
09 May 2023 16:28:35 1501535 22299828 oifs_43r3_ps_0200_2009050100_123_978_12194844_2 864,000 3,892 0.0045


©2024 climateprediction.net