climateprediction.net home page
Task 22278575

Task 22278575

Name oifs_43r3_ps_0169_2007050100_123_976_12192813_0
Workunit 12192813
Created 23 Dec 2022, 15:05:57 UTC
Sent 20 Jan 2023, 21:56:47 UTC
Report deadline 21 Mar 2023, 21:56:47 UTC
Received 26 Jan 2023, 1:26:40 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1538241
Run time 21 hours 42 min 11 sec
CPU time 21 hours 35 min 51 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 6.24 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,414.43 MB
Peak swap size 4,978.14 MB
Peak disk usage 4,569.07 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
:40 STEP 2433 H=2433:00 +CPU= 31.882
  13:10:56 STEP 2434 H=2434:00 +CPU= 16.768
  13:11:19 STEP 2435 H=2435:00 +CPU= 22.240
  13:11:45 STEP 2436 H=2436:00 +CPU= 26.668
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002436
  13:12:00 STEP 2437 H=2437:00 +CPU= 14.564
  13:12:15 STEP 2438 H=2438:00 +CPU= 15.445
  13:12:42 STEP 2439 H=2439:00 +CPU= 26.247
  13:12:57 STEP 2440 H=2440:00 +CPU= 15.087
  13:13:16 STEP 2441 H=2441:00 +CPU= 19.216
  13:13:46 STEP 2442 H=2442:00 +CPU= 30.046
  13:14:08 STEP 2443 H=2443:00 +CPU= 22.314
  13:14:31 STEP 2444 H=2444:00 +CPU= 22.657
  13:15:03 STEP 2445 H=2445:00 +CPU= 32.041
  13:15:23 STEP 2446 H=2446:00 +CPU= 19.829
  13:15:41 STEP 2447 H=2447:00 +CPU= 18.493
  13:16:11 STEP 2448 H=2448:00 +CPU= 29.334
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_101.zip
Uploading the intermediate file: upload_file_101.zip
  13:16:28 STEP 2449 H=2449:00 +CPU= 16.980
  13:16:42 STEP 2450 H=2450:00 +CPU= 14.754
  13:17:12 STEP 2451 H=2451:00 +CPU= 30.084
  13:17:35 STEP 2452 H=2452:00 +CPU= 22.471
  13:17:56 STEP 2453 H=2453:00 +CPU= 21.198
  13:18:29 STEP 2454 H=2454:00 +CPU= 32.532
  13:18:43 STEP 2455 H=2455:00 +CPU= 14.678
  13:19:00 STEP 2456 H=2456:00 +CPU= 16.826
  13:19:35 STEP 2457 H=2457:00 +CPU= 34.446
  13:19:56 STEP 2458 H=2458:00 +CPU= 21.497
  13:20:17 STEP 2459 H=2459:00 +CPU= 21.339
  13:20:44 STEP 2460 H=2460:00 +CPU= 26.216
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002460
  13:21:03 STEP 2461 H=2461:00 +CPU= 19.494
  13:21:26 STEP 2462 H=2462:00 +CPU= 22.882
  13:21:55 STEP 2463 H=2463:00 +CPU= 29.311
  13:22:11 STEP 2464 H=2464:00 +CPU= 15.454
  13:22:27 STEP 2465 H=2465:00 +CPU= 16.207
  13:22:50 STEP 2466 H=2466:00 +CPU= 23.387
  13:23:08 STEP 2467 H=2467:00 +CPU= 17.828
  13:23:31 STEP 2468 H=2468:00 +CPU= 22.825
  13:23:58 STEP 2469 H=2469:00 +CPU= 26.956
  13:24:13 STEP 2470 H=2470:00 +CPU= 14.791
  13:24:28 STEP 2471 H=2471:00 +CPU= 14.767
  13:24:54 STEP 2472 H=2472:00 +CPU= 26.756
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_102.zip
Uploading the intermediate file: upload_file_102.zip
  13:25:11 STEP 2473 H=2473:00 +CPU= 16.245
  13:25:33 STEP 2474 H=2474:00 +CPU= 22.309
  13:26:03 STEP 2475 H=2475:00 +CPU= 29.798
  13:26:20 STEP 2476 H=2476:00 +CPU= 16.931
  13:26:35 STEP 2477 H=2477:00 +CPU= 15.208
  13:27:05 STEP 2478 H=2478:00 +CPU= 30.146
  13:27:24 STEP 2479 H=2479:00 +CPU= 19.269
  13:27:47 STEP 2480 H=2480:00 +CPU= 23.167
  13:28:16 STEP 2481 H=2481:00 +CPU= 28.737
  13:28:35 STEP 2482 H=2482:00 +CPU= 18.997
  13:28:52 STEP 2483 H=2483:00 +CPU= 17.039
  13:29:27 STEP 2484 H=2484:00 +CPU= 34.324
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002484
  13:29:46 STEP 2485 H=2485:00 +CPU= 19.425
  13:30:07 STEP 2486 H=2486:00 +CPU= 20.673
  13:30:36 STEP 2487 H=2487:00 +CPU= 29.068
  13:30:59 STEP 2488 H=2488:00 +CPU= 22.867
  13:31:21 STEP 2489 H=2489:00 +CPU= 22.669
  13:31:55 STEP 2490 H=2490:00 +CPU= 33.880
  13:32:15 STEP 2491 H=2491:00 +CPU= 19.783
  13:32:34 STEP 2492 H=2492:00 +CPU= 18.894
  13:33:06 STEP 2493 H=2493:00 +CPU= 32.581
  13:33:26 STEP 2494 H=2494:00 +CPU= 19.129
  13:33:48 STEP 2495 H=2495:00 +CPU= 22.618
  13:34:25 STEP 2496 H=2496:00 +CPU= 37.209
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_103.zip
Uploading the intermediate file: upload_file_103.zip
  13:34:45 STEP 2497 H=2497:00 +CPU= 19.325
  13:35:01 STEP 2498 H=2498:00 +CPU= 16.084
  13:35:33 STEP 2499 H=2499:00 +CPU= 32.591
  13:35:50 STEP 2500 H=2500:00 +CPU= 16.576
  13:36:07 STEP 2501 H=2501:00 +CPU= 17.158
  13:36:35 STEP 2502 H=2502:00 +CPU= 28.113
  13:36:50 STEP 2503 H=2503:00 +CPU= 14.772
  13:37:09 STEP 2504 H=2504:00 +CPU= 19.438
  13:37:38 STEP 2505 H=2505:00 +CPU= 28.646
  13:37:54 STEP 2506 H=2506:00 +CPU= 16.077
  13:38:11 STEP 2507 H=2507:00 +CPU= 17.092
  13:38:37 STEP 2508 H=2508:00 +CPU= 25.784
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002508
  13:38:56 STEP 2509 H=2509:00 +CPU= 18.955
  13:39:18 STEP 2510 H=2510:00 +CPU= 22.332
  13:39:44 STEP 2511 H=2511:00 +CPU= 25.541
  13:39:59 STEP 2512 H=2512:00 +CPU= 14.959
  13:40:17 STEP 2513 H=2513:00 +CPU= 18.512
  13:40:49 STEP 2514 H=2514:00 +CPU= 31.847
  13:41:05 STEP 2515 H=2515:00 +CPU= 16.180
  13:41:23 STEP 2516 H=2516:00 +CPU= 17.940
  13:41:49 STEP 2517 H=2517:00 +CPU= 25.720
  13:42:11 STEP 2518 H=2518:00 +CPU= 21.475
  13:42:29 STEP 2519 H=2519:00 +CPU= 18.795
  13:43:09 STEP 2520 H=2520:00 +CPU= 39.711
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_104.zip
Uploading the intermediate file: upload_file_104.zip
  13:43:29 STEP 2521 H=2521:00 +CPU= 19.984
  13:43:51 STEP 2522 H=2522:00 +CPU= 21.705
  13:44:18 STEP 2523 H=2523:00 +CPU= 27.407
  13:44:37 STEP 2524 H=2524:00 +CPU= 18.645
  13:44:57 STEP 2525 H=2525:00 +CPU= 19.875
  13:45:30 STEP 2526 H=2526:00 +CPU= 33.130
  13:45:45 STEP 2527 H=2527:00 +CPU= 14.805
  13:46:02 STEP 2528 H=2528:00 +CPU= 17.420
  13:46:31 STEP 2529 H=2529:00 +CPU= 28.809
  13:46:50 STEP 2530 H=2530:00 +CPU= 19.480
  13:47:10 STEP 2531 H=2531:00 +CPU= 19.646
  13:47:39 STEP 2532 H=2532:00 +CPU= 29.266
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002532
  13:47:59 STEP 2533 H=2533:00 +CPU= 20.247
  13:48:18 STEP 2534 H=2534:00 +CPU= 18.854
  13:48:47 STEP 2535 H=2535:00 +CPU= 28.673
  13:49:04 STEP 2536 H=2536:00 +CPU= 17.052
  13:49:22 STEP 2537 H=2537:00 +CPU= 17.965
  13:49:47 STEP 2538 H=2538:00 +CPU= 25.438
  13:50:08 STEP 2539 H=2539:00 +CPU= 20.321
  13:50:26 STEP 2540 H=2540:00 +CPU= 17.725
  13:50:49 STEP 2541 H=2541:00 +CPU= 23.309
  13:51:07 STEP 2542 H=2542:00 +CPU= 18.565
  13:51:24 STEP 2543 H=2543:00 +CPU= 17.015
  13:52:00 STEP 2544 H=2544:00 +CPU= 35.712
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_105.zip
Uploading the intermediate file: upload_file_105.zip
  13:52:18 STEP 2545 H=2545:00 +CPU= 17.848
  13:52:40 STEP 2546 H=2546:00 +CPU= 22.491
  13:53:09 STEP 2547 H=2547:00 +CPU= 28.944
  13:53:33 STEP 2548 H=2548:00 +CPU= 23.633
  13:53:52 STEP 2549 H=2549:00 +CPU= 19.164
  13:54:25 STEP 2550 H=2550:00 +CPU= 32.920
  13:54:48 STEP 2551 H=2551:00 +CPU= 23.164
  13:55:13 STEP 2552 H=2552:00 +CPU= 24.659
  13:56:05 STEP 2553 H=2553:00 +CPU= 52.273
  13:56:34 STEP 2554 H=2554:00 +CPU= 28.846
  13:57:00 STEP 2555 H=2555:00 +CPU= 25.687
  13:57:36 STEP 2556 H=2556:00 +CPU= 36.009
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002556
  13:57:56 STEP 2557 H=2557:00 +CPU= 20.280
  13:58:25 STEP 2558 H=2558:00 +CPU= 28.856
  13:59:05 STEP 2559 H=2559:00 +CPU= 39.850
  13:59:28 STEP 2560 H=2560:00 +CPU= 23.194
  13:59:45 STEP 2561 H=2561:00 +CPU= 17.080
  14:00:21 STEP 2562 H=2562:00 +CPU= 35.702
  14:00:37 STEP 2563 H=2563:00 +CPU= 16.211
  14:00:59 STEP 2564 H=2564:00 +CPU= 21.754
  14:01:31 STEP 2565 H=2565:00 +CPU= 31.850
  14:01:55 STEP 2566 H=2566:00 +CPU= 24.245
  14:02:15 STEP 2567 H=2567:00 +CPU= 20.186
  14:02:51 STEP 2568 H=2568:00 +CPU= 36.373
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_106.zip
Uploading the intermediate file: upload_file_106.zip
  14:03:10 STEP 2569 H=2569:00 +CPU= 18.921
  14:03:36 STEP 2570 H=2570:00 +CPU= 25.302
  14:04:19 STEP 2571 H=2571:00 +CPU= 43.490
  14:04:41 STEP 2572 H=2572:00 +CPU= 21.397
  14:05:00 STEP 2573 H=2573:00 +CPU= 19.330
  14:05:30 STEP 2574 H=2574:00 +CPU= 29.664
  14:05:49 STEP 2575 H=2575:00 +CPU= 19.350
  14:06:11 STEP 2576 H=2576:00 +CPU= 21.838
  14:06:43 STEP 2577 H=2577:00 +CPU= 32.485
  14:07:04 STEP 2578 H=2578:00 +CPU= 20.732
  14:07:25 STEP 2579 H=2579:00 +CPU= 21.508
  14:08:10 STEP 2580 H=2580:00 +CPU= 44.631
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002580
  14:08:35 STEP 2581 H=2581:00 +CPU= 24.792
  14:08:58 STEP 2582 H=2582:00 +CPU= 22.965
  14:09:34 STEP 2583 H=2583:00 +CPU= 35.702
  14:10:01 STEP 2584 H=2584:00 +CPU= 27.264
  14:10:26 STEP 2585 H=2585:00 +CPU= 25.349
  14:11:07 STEP 2586 H=2586:00 +CPU= 40.426
  14:11:30 STEP 2587 H=2587:00 +CPU= 22.888
  14:11:54 STEP 2588 H=2588:00 +CPU= 24.804
  14:12:27 STEP 2589 H=2589:00 +CPU= 32.481
  14:12:46 STEP 2590 H=2590:00 +CPU= 19.484
  14:13:07 STEP 2591 H=2591:00 +CPU= 20.597
  14:13:44 STEP 2592 H=2592:00 +CPU= 37.551
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_107.zip
Uploading the intermediate file: upload_file_107.zip
  14:14:10 STEP 2593 H=2593:00 +CPU= 25.037
Uploading trickle at timestep: 9331200
  14:14:30 STEP 2594 H=2594:00 +CPU= 20.263
  14:14:59 STEP 2595 H=2595:00 +CPU= 28.906
  14:15:20 STEP 2596 H=2596:00 +CPU= 21.771
  14:15:44 STEP 2597 H=2597:00 +CPU= 23.486
  14:16:18 STEP 2598 H=2598:00 +CPU= 34.053
  14:16:35 STEP 2599 H=2599:00 +CPU= 16.652
  14:16:55 STEP 2600 H=2600:00 +CPU= 20.052
  14:17:26 STEP 2601 H=2601:00 +CPU= 31.032
  14:17:46 STEP 2602 H=2602:00 +CPU= 20.639
  14:18:05 STEP 2603 H=2603:00 +CPU= 18.237
  14:18:34 STEP 2604 H=2604:00 +CPU= 29.439
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002604
  14:18:54 STEP 2605 H=2605:00 +CPU= 19.815
  14:19:15 STEP 2606 H=2606:00 +CPU= 20.898
  14:19:42 STEP 2607 H=2607:00 +CPU= 26.751
  14:19:59 STEP 2608 H=2608:00 +CPU= 17.594
  14:20:17 STEP 2609 H=2609:00 +CPU= 17.731
  14:20:44 STEP 2610 H=2610:00 +CPU= 26.688
  14:20:59 STEP 2611 H=2611:00 +CPU= 15.751
  14:21:17 STEP 2612 H=2612:00 +CPU= 17.772
  14:21:41 STEP 2613 H=2613:00 +CPU= 23.836
  14:21:57 STEP 2614 H=2614:00 +CPU= 16.201
  14:22:19 STEP 2615 H=2615:00 +CPU= 22.060
  14:22:53 STEP 2616 H=2616:00 +CPU= 33.283
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_108.zip
Uploading the intermediate file: upload_file_108.zip
  14:23:09 STEP 2617 H=2617:00 +CPU= 16.863
  14:23:30 STEP 2618 H=2618:00 +CPU= 20.370
  14:23:58 STEP 2619 H=2619:00 +CPU= 28.180
  14:24:15 STEP 2620 H=2620:00 +CPU= 17.350
  14:24:34 STEP 2621 H=2621:00 +CPU= 18.817
  14:24:59 STEP 2622 H=2622:00 +CPU= 24.587
  14:25:19 STEP 2623 H=2623:00 +CPU= 20.768
  14:25:36 STEP 2624 H=2624:00 +CPU= 16.666
  14:26:10 STEP 2625 H=2625:00 +CPU= 33.977
  14:26:28 STEP 2626 H=2626:00 +CPU= 18.094
  14:26:46 STEP 2627 H=2627:00 +CPU= 17.342
  14:27:16 STEP 2628 H=2628:00 +CPU= 30.277
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002628
  14:27:32 STEP 2629 H=2629:00 +CPU= 15.808
  14:27:49 STEP 2630 H=2630:00 +CPU= 17.424
  14:28:17 STEP 2631 H=2631:00 +CPU= 28.265
  14:28:39 STEP 2632 H=2632:00 +CPU= 21.284
  14:28:56 STEP 2633 H=2633:00 +CPU= 17.012
  14:29:20 STEP 2634 H=2634:00 +CPU= 24.078
  14:29:36 STEP 2635 H=2635:00 +CPU= 16.604
  14:29:54 STEP 2636 H=2636:00 +CPU= 17.915
  14:30:25 STEP 2637 H=2637:00 +CPU= 30.941
  14:30:42 STEP 2638 H=2638:00 +CPU= 16.671
  14:30:59 STEP 2639 H=2639:00 +CPU= 17.180
  14:31:32 STEP 2640 H=2640:00 +CPU= 32.618
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_109.zip
Uploading the intermediate file: upload_file_109.zip
  14:31:50 STEP 2641 H=2641:00 +CPU= 18.054
  14:32:10 STEP 2642 H=2642:00 +CPU= 20.207
  14:32:39 STEP 2643 H=2643:00 +CPU= 29.152
  14:32:58 STEP 2644 H=2644:00 +CPU= 18.488
  14:33:20 STEP 2645 H=2645:00 +CPU= 22.648
  14:33:51 STEP 2646 H=2646:00 +CPU= 30.468
  14:34:11 STEP 2647 H=2647:00 +CPU= 20.223
  14:34:33 STEP 2648 H=2648:00 +CPU= 21.721
  14:35:01 STEP 2649 H=2649:00 +CPU= 27.911
  14:35:20 STEP 2650 H=2650:00 +CPU= 19.831
  14:35:41 STEP 2651 H=2651:00 +CPU= 20.870
  14:36:08 STEP 2652 H=2652:00 +CPU= 26.923
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002652
  14:36:27 STEP 2653 H=2653:00 +CPU= 18.317
  14:36:46 STEP 2654 H=2654:00 +CPU= 19.250
  14:37:14 STEP 2655 H=2655:00 +CPU= 28.154
  14:37:31 STEP 2656 H=2656:00 +CPU= 16.949
  14:37:48 STEP 2657 H=2657:00 +CPU= 17.181
  14:38:17 STEP 2658 H=2658:00 +CPU= 28.552
  14:38:34 STEP 2659 H=2659:00 +CPU= 16.908
  14:38:53 STEP 2660 H=2660:00 +CPU= 19.316
  14:39:24 STEP 2661 H=2661:00 +CPU= 31.240
  14:39:43 STEP 2662 H=2662:00 +CPU= 18.472
  14:40:01 STEP 2663 H=2663:00 +CPU= 18.255
  14:40:37 STEP 2664 H=2664:00 +CPU= 35.735
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_110.zip
Uploading the intermediate file: upload_file_110.zip
  14:40:54 STEP 2665 H=2665:00 +CPU= 17.692
  14:41:13 STEP 2666 H=2666:00 +CPU= 18.727
  14:41:43 STEP 2667 H=2667:00 +CPU= 29.762
  14:42:01 STEP 2668 H=2668:00 +CPU= 17.948
  14:42:19 STEP 2669 H=2669:00 +CPU= 18.128
  14:42:48 STEP 2670 H=2670:00 +CPU= 28.673
  14:43:03 STEP 2671 H=2671:00 +CPU= 15.790
  14:43:19 STEP 2672 H=2672:00 +CPU= 15.320
  14:43:50 STEP 2673 H=2673:00 +CPU= 30.977
  14:44:13 STEP 2674 H=2674:00 +CPU= 22.939
  14:44:33 STEP 2675 H=2675:00 +CPU= 20.065
  14:45:04 STEP 2676 H=2676:00 +CPU= 30.731
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002676
  14:45:21 STEP 2677 H=2677:00 +CPU= 17.723
  14:45:39 STEP 2678 H=2678:00 +CPU= 17.357
  14:46:08 STEP 2679 H=2679:00 +CPU= 29.359
  14:46:27 STEP 2680 H=2680:00 +CPU= 18.807
  14:46:42 STEP 2681 H=2681:00 +CPU= 15.040
  14:47:10 STEP 2682 H=2682:00 +CPU= 28.262
  14:47:30 STEP 2683 H=2683:00 +CPU= 19.803
  14:47:48 STEP 2684 H=2684:00 +CPU= 17.614
  14:48:13 STEP 2685 H=2685:00 +CPU= 25.424
  14:48:33 STEP 2686 H=2686:00 +CPU= 20.115
  14:48:49 STEP 2687 H=2687:00 +CPU= 16.139
  14:49:23 STEP 2688 H=2688:00 +CPU= 34.189
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_111.zip
Uploading the intermediate file: upload_file_111.zip
  14:49:44 STEP 2689 H=2689:00 +CPU= 20.035
  14:50:02 STEP 2690 H=2690:00 +CPU= 17.999
  14:50:27 STEP 2691 H=2691:00 +CPU= 25.776
  14:50:46 STEP 2692 H=2692:00 +CPU= 18.564
  14:51:07 STEP 2693 H=2693:00 +CPU= 21.043
  14:51:33 STEP 2694 H=2694:00 +CPU= 26.167
  14:51:48 STEP 2695 H=2695:00 +CPU= 15.218
  14:52:06 STEP 2696 H=2696:00 +CPU= 18.075
  14:52:31 STEP 2697 H=2697:00 +CPU= 24.884
  14:52:47 STEP 2698 H=2698:00 +CPU= 15.724
  14:53:05 STEP 2699 H=2699:00 +CPU= 17.569
  14:53:38 STEP 2700 H=2700:00 +CPU= 33.575
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002700
  14:53:55 STEP 2701 H=2701:00 +CPU= 17.000
  14:54:13 STEP 2702 H=2702:00 +CPU= 17.474
  14:54:38 STEP 2703 H=2703:00 +CPU= 25.417
  14:55:00 STEP 2704 H=2704:00 +CPU= 21.490
  14:55:17 STEP 2705 H=2705:00 +CPU= 17.568
  14:55:42 STEP 2706 H=2706:00 +CPU= 24.868
  14:56:02 STEP 2707 H=2707:00 +CPU= 19.758
  14:56:23 STEP 2708 H=2708:00 +CPU= 21.346
  14:56:49 STEP 2709 H=2709:00 +CPU= 26.266
  14:57:06 STEP 2710 H=2710:00 +CPU= 16.161
  14:57:24 STEP 2711 H=2711:00 +CPU= 18.896
  14:57:59 STEP 2712 H=2712:00 +CPU= 34.042
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_112.zip
Uploading the intermediate file: upload_file_112.zip
  14:58:16 STEP 2713 H=2713:00 +CPU= 17.807
  14:58:34 STEP 2714 H=2714:00 +CPU= 18.084
  14:59:04 STEP 2715 H=2715:00 +CPU= 29.389
  14:59:20 STEP 2716 H=2716:00 +CPU= 15.989
  14:59:36 STEP 2717 H=2717:00 +CPU= 16.048
  15:00:07 STEP 2718 H=2718:00 +CPU= 30.977
  15:00:25 STEP 2719 H=2719:00 +CPU= 18.371
  15:00:46 STEP 2720 H=2720:00 +CPU= 20.857
  15:01:12 STEP 2721 H=2721:00 +CPU= 25.554
  15:01:28 STEP 2722 H=2722:00 +CPU= 16.769
  15:01:45 STEP 2723 H=2723:00 +CPU= 17.007
  15:02:20 STEP 2724 H=2724:00 +CPU= 34.433
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002724
  15:02:37 STEP 2725 H=2725:00 +CPU= 16.750
  15:02:52 STEP 2726 H=2726:00 +CPU= 15.007
  15:03:24 STEP 2727 H=2727:00 +CPU= 32.191
  15:03:43 STEP 2728 H=2728:00 +CPU= 19.538
  15:04:02 STEP 2729 H=2729:00 +CPU= 18.886
  15:04:33 STEP 2730 H=2730:00 +CPU= 31.267
  15:04:52 STEP 2731 H=2731:00 +CPU= 18.217
  15:05:07 STEP 2732 H=2732:00 +CPU= 15.711
  15:05:36 STEP 2733 H=2733:00 +CPU= 28.077
  15:05:50 STEP 2734 H=2734:00 +CPU= 14.723
  15:06:07 STEP 2735 H=2735:00 +CPU= 16.859
  15:06:39 STEP 2736 H=2736:00 +CPU= 31.922
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_113.zip
Uploading the intermediate file: upload_file_113.zip
  15:06:59 STEP 2737 H=2737:00 +CPU= 20.126
  15:07:22 STEP 2738 H=2738:00 +CPU= 22.620
  15:07:49 STEP 2739 H=2739:00 +CPU= 27.137
  15:08:06 STEP 2740 H=2740:00 +CPU= 16.691
  15:08:24 STEP 2741 H=2741:00 +CPU= 18.311
  15:08:58 STEP 2742 H=2742:00 +CPU= 34.273
  15:09:22 STEP 2743 H=2743:00 +CPU= 23.658
  15:09:41 STEP 2744 H=2744:00 +CPU= 19.418
  15:10:10 STEP 2745 H=2745:00 +CPU= 28.597
  15:10:32 STEP 2746 H=2746:00 +CPU= 22.006
  15:10:56 STEP 2747 H=2747:00 +CPU= 24.193
  15:11:33 STEP 2748 H=2748:00 +CPU= 37.090
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002748
  15:11:52 STEP 2749 H=2749:00 +CPU= 18.911
  15:12:09 STEP 2750 H=2750:00 +CPU= 17.102
  15:12:39 STEP 2751 H=2751:00 +CPU= 29.735
  15:12:57 STEP 2752 H=2752:00 +CPU= 18.440
  15:13:15 STEP 2753 H=2753:00 +CPU= 17.776
  15:13:50 STEP 2754 H=2754:00 +CPU= 34.863
  15:14:11 STEP 2755 H=2755:00 +CPU= 21.239
  15:14:32 STEP 2756 H=2756:00 +CPU= 21.006
  15:14:58 STEP 2757 H=2757:00 +CPU= 26.044
  15:15:18 STEP 2758 H=2758:00 +CPU= 19.879
  15:15:40 STEP 2759 H=2759:00 +CPU= 22.043
  15:16:13 STEP 2760 H=2760:00 +CPU= 32.525
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_114.zip
Uploading the intermediate file: upload_file_114.zip
  15:16:32 STEP 2761 H=2761:00 +CPU= 18.675
  15:16:54 STEP 2762 H=2762:00 +CPU= 22.537
  15:17:22 STEP 2763 H=2763:00 +CPU= 28.175
  15:17:42 STEP 2764 H=2764:00 +CPU= 20.192
  15:18:02 STEP 2765 H=2765:00 +CPU= 19.612
  15:18:34 STEP 2766 H=2766:00 +CPU= 31.983
  15:18:54 STEP 2767 H=2767:00 +CPU= 20.331
  15:19:17 STEP 2768 H=2768:00 +CPU= 22.169
  15:19:50 STEP 2769 H=2769:00 +CPU= 33.260
  15:20:06 STEP 2770 H=2770:00 +CPU= 15.836
  15:20:22 STEP 2771 H=2771:00 +CPU= 16.453
  15:20:53 STEP 2772 H=2772:00 +CPU= 30.622
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002772
  15:21:11 STEP 2773 H=2773:00 +CPU= 18.003
  15:21:30 STEP 2774 H=2774:00 +CPU= 19.241
  15:21:56 STEP 2775 H=2775:00 +CPU= 25.790
  15:22:14 STEP 2776 H=2776:00 +CPU= 17.956
  15:22:32 STEP 2777 H=2777:00 +CPU= 17.818
  15:23:01 STEP 2778 H=2778:00 +CPU= 29.543
  15:23:18 STEP 2779 H=2779:00 +CPU= 17.122
  15:23:37 STEP 2780 H=2780:00 +CPU= 18.392
  15:24:08 STEP 2781 H=2781:00 +CPU= 31.586
  15:24:28 STEP 2782 H=2782:00 +CPU= 19.852
  15:24:50 STEP 2783 H=2783:00 +CPU= 22.296
  15:25:29 STEP 2784 H=2784:00 +CPU= 38.309
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_115.zip
Uploading the intermediate file: upload_file_115.zip
  15:25:48 STEP 2785 H=2785:00 +CPU= 18.842
  15:26:07 STEP 2786 H=2786:00 +CPU= 19.852
  15:26:39 STEP 2787 H=2787:00 +CPU= 31.219
  15:26:58 STEP 2788 H=2788:00 +CPU= 19.536
  15:27:20 STEP 2789 H=2789:00 +CPU= 22.043
  15:27:50 STEP 2790 H=2790:00 +CPU= 29.868
  15:28:10 STEP 2791 H=2791:00 +CPU= 19.827
  15:28:32 STEP 2792 H=2792:00 +CPU= 22.076
  15:28:59 STEP 2793 H=2793:00 +CPU= 26.815
  15:29:16 STEP 2794 H=2794:00 +CPU= 17.058
  15:29:36 STEP 2795 H=2795:00 +CPU= 20.567
  15:30:07 STEP 2796 H=2796:00 +CPU= 30.799
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002796
  15:30:27 STEP 2797 H=2797:00 +CPU= 19.352
  15:30:47 STEP 2798 H=2798:00 +CPU= 20.821
  15:31:21 STEP 2799 H=2799:00 +CPU= 33.293
  15:31:44 STEP 2800 H=2800:00 +CPU= 23.398
  15:32:09 STEP 2801 H=2801:00 +CPU= 24.485
  15:32:39 STEP 2802 H=2802:00 +CPU= 30.796
  15:32:58 STEP 2803 H=2803:00 +CPU= 18.957
  15:33:21 STEP 2804 H=2804:00 +CPU= 23.056
  15:34:00 STEP 2805 H=2805:00 +CPU= 38.580
  15:34:25 STEP 2806 H=2806:00 +CPU= 24.806
  15:34:55 STEP 2807 H=2807:00 +CPU= 30.335
  15:35:34 STEP 2808 H=2808:00 +CPU= 38.351
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_116.zip
Uploading the intermediate file: upload_file_116.zip
  15:35:53 STEP 2809 H=2809:00 +CPU= 19.118
  15:36:16 STEP 2810 H=2810:00 +CPU= 23.295
  15:36:56 STEP 2811 H=2811:00 +CPU= 39.675
  15:37:27 STEP 2812 H=2812:00 +CPU= 31.800
  15:37:50 STEP 2813 H=2813:00 +CPU= 22.759
  15:38:24 STEP 2814 H=2814:00 +CPU= 33.516
  15:38:45 STEP 2815 H=2815:00 +CPU= 20.779
  15:39:06 STEP 2816 H=2816:00 +CPU= 21.917
  15:39:40 STEP 2817 H=2817:00 +CPU= 33.925
  15:40:05 STEP 2818 H=2818:00 +CPU= 24.151
  15:40:27 STEP 2819 H=2819:00 +CPU= 22.782
  15:41:09 STEP 2820 H=2820:00 +CPU= 41.635
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002820
  15:41:34 STEP 2821 H=2821:00 +CPU= 24.781
  15:42:00 STEP 2822 H=2822:00 +CPU= 26.190
  15:42:38 STEP 2823 H=2823:00 +CPU= 37.638
  15:43:06 STEP 2824 H=2824:00 +CPU= 27.907
  15:43:22 STEP 2825 H=2825:00 +CPU= 16.908
  15:43:52 STEP 2826 H=2826:00 +CPU= 29.914
  15:44:10 STEP 2827 H=2827:00 +CPU= 17.943
  15:44:32 STEP 2828 H=2828:00 +CPU= 21.715
  15:45:03 STEP 2829 H=2829:00 +CPU= 30.863
  15:45:22 STEP 2830 H=2830:00 +CPU= 19.114
  15:45:42 STEP 2831 H=2831:00 +CPU= 19.927
  15:46:17 STEP 2832 H=2832:00 +CPU= 35.223
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_117.zip
Uploading the intermediate file: upload_file_117.zip
Uploading trickle at timestep: 10195200
  15:46:41 STEP 2833 H=2833:00 +CPU= 23.981
  15:47:02 STEP 2834 H=2834:00 +CPU= 20.741
  15:47:33 STEP 2835 H=2835:00 +CPU= 31.227
  15:47:52 STEP 2836 H=2836:00 +CPU= 18.779
  15:48:14 STEP 2837 H=2837:00 +CPU= 22.131
  15:48:50 STEP 2838 H=2838:00 +CPU= 36.001
  15:49:12 STEP 2839 H=2839:00 +CPU= 21.990
  15:49:32 STEP 2840 H=2840:00 +CPU= 19.765
  15:50:05 STEP 2841 H=2841:00 +CPU= 33.060
  15:50:28 STEP 2842 H=2842:00 +CPU= 23.229
  15:50:49 STEP 2843 H=2843:00 +CPU= 20.834
  15:51:31 STEP 2844 H=2844:00 +CPU= 41.877
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002844
  15:51:51 STEP 2845 H=2845:00 +CPU= 20.209
  15:52:18 STEP 2846 H=2846:00 +CPU= 26.752
  15:52:59 STEP 2847 H=2847:00 +CPU= 41.351
  15:53:26 STEP 2848 H=2848:00 +CPU= 26.758
  15:53:53 STEP 2849 H=2849:00 +CPU= 26.737
  15:54:28 STEP 2850 H=2850:00 +CPU= 35.336
  15:54:49 STEP 2851 H=2851:00 +CPU= 21.293
  15:55:21 STEP 2852 H=2852:00 +CPU= 31.887
  15:56:00 STEP 2853 H=2853:00 +CPU= 39.083
  15:56:23 STEP 2854 H=2854:00 +CPU= 22.813
  15:56:42 STEP 2855 H=2855:00 +CPU= 18.819
  15:57:24 STEP 2856 H=2856:00 +CPU= 42.237
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_118.zip
Uploading the intermediate file: upload_file_118.zip
  15:57:48 STEP 2857 H=2857:00 +CPU= 23.638
  15:58:12 STEP 2858 H=2858:00 +CPU= 23.799
  15:58:51 STEP 2859 H=2859:00 +CPU= 39.016
  15:59:15 STEP 2860 H=2860:00 +CPU= 24.551
  15:59:35 STEP 2861 H=2861:00 +CPU= 19.624
  16:00:05 STEP 2862 H=2862:00 +CPU= 30.179
  16:00:28 STEP 2863 H=2863:00 +CPU= 22.955
  16:00:49 STEP 2864 H=2864:00 +CPU= 20.971
  16:01:24 STEP 2865 H=2865:00 +CPU= 34.533
  16:01:47 STEP 2866 H=2866:00 +CPU= 23.612
  16:02:09 STEP 2867 H=2867:00 +CPU= 21.859
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0169
(argv4) batchid: 976
(argv5) wuid: 12192813
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client_40001/slots/2
Project directory is: /var/lib/boinc-client_40001/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client_40001/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client_40001/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ed31ec1bb850f15413ecae1211e4a061 to: /var/lib/boinc-client_40001/slots/2/oifs_43r3_ps_0169_2007050100_123_976_12192813.zip
Unzipping the namelist zip file: /var/lib/boinc-client_40001/slots/2/oifs_43r3_ps_0169_2007050100_123_976_12192813.zip
ic_ancil_file: ic_ancil_12192813
ifsdata_file: ifsdata_12192813
climate_data_file: clim_data_12192813
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_90264709eeb6199ccb273bbfe85d72e6 to: /var/lib/boinc-client_40001/slots/2/ic_ancil_12192813.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client_40001/slots/2/ic_ancil_12192813.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_b23322b0e9c9ff320bab499971a155cc to: /var/lib/boinc-client_40001/slots/2/ifsdata/ifsdata_12192813.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client_40001/slots/2/ifsdata/ifsdata_12192813.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client_40001/slots/2/159l_2/clim_data_12192813.zip
Unzipping the climate data zip file: /var/lib/boinc-client_40001/slots/2/159l_2/clim_data_12192813.zip
Checking for progress XML file: /var/lib/boinc-client_40001/slots/2/progress_file_12192813.xml
Opened progress file ok : /var/lib/boinc-client_40001/slots/2/progress_file_12192813.xml
-- Model is restarting --
Adjusting last_iter, 2868, to previous model restart step.
Creating progress file: /var/lib/boinc-client_40001/slots/2/progress_file_12192813.xml
last_cpu_time: 75137
upload_file_number: 119
last_iter: 2855
last_upload: 10281600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705
The child process has been launched with process id: 1412
Executing the command: /var/lib/boinc-client_40001/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1412:1412] [20230124:162702:1674577622:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:28:15 STEP 2856 H=2856:00 +CPU= 36.974
  16:28:38 STEP 2857 H=2857:00 +CPU= 23.129
  16:29:01 STEP 2858 H=2858:00 +CPU= 23.296
  16:29:31 STEP 2859 H=2859:00 +CPU= 29.716
  16:29:49 STEP 2860 H=2860:00 +CPU= 17.677
  16:30:06 STEP 2861 H=2861:00 +CPU= 17.621
  16:30:40 STEP 2862 H=2862:00 +CPU= 33.655
  16:31:00 STEP 2863 H=2863:00 +CPU= 19.958
  16:31:21 STEP 2864 H=2864:00 +CPU= 20.603
  16:31:54 STEP 2865 H=2865:00 +CPU= 33.780
  16:32:17 STEP 2866 H=2866:00 +CPU= 22.870
  16:32:43 STEP 2867 H=2867:00 +CPU= 26.143
  16:33:17 STEP 2868 H=2868:00 +CPU= 33.857
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002868
  16:33:38 STEP 2869 H=2869:00 +CPU= 20.201
  16:34:00 STEP 2870 H=2870:00 +CPU= 22.413
  16:34:37 STEP 2871 H=2871:00 +CPU= 36.969
  16:34:58 STEP 2872 H=2872:00 +CPU= 20.956
  16:35:16 STEP 2873 H=2873:00 +CPU= 18.421
  16:35:54 STEP 2874 H=2874:00 +CPU= 37.782
  16:36:16 STEP 2875 H=2875:00 +CPU= 22.334
  16:36:37 STEP 2876 H=2876:00 +CPU= 20.186
  16:37:10 STEP 2877 H=2877:00 +CPU= 33.671
  16:37:30 STEP 2878 H=2878:00 +CPU= 19.824
  16:37:50 STEP 2879 H=2879:00 +CPU= 19.921
  16:38:35 STEP 2880 H=2880:00 +CPU= 44.806
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_119.zip
Uploading the intermediate file: upload_file_119.zip
  16:39:00 STEP 2881 H=2881:00 +CPU= 25.126
  16:39:26 STEP 2882 H=2882:00 +CPU= 26.192
  16:40:03 STEP 2883 H=2883:00 +CPU= 36.443
  16:40:22 STEP 2884 H=2884:00 +CPU= 19.575
  16:40:45 STEP 2885 H=2885:00 +CPU= 22.947
  16:41:17 STEP 2886 H=2886:00 +CPU= 31.440
  16:41:34 STEP 2887 H=2887:00 +CPU= 17.496
  16:41:53 STEP 2888 H=2888:00 +CPU= 18.399
  16:42:21 STEP 2889 H=2889:00 +CPU= 28.310
  16:42:38 STEP 2890 H=2890:00 +CPU= 17.196
  16:43:01 STEP 2891 H=2891:00 +CPU= 23.168
  16:43:46 STEP 2892 H=2892:00 +CPU= 45.057
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002892
  16:44:04 STEP 2893 H=2893:00 +CPU= 17.891
  16:44:23 STEP 2894 H=2894:00 +CPU= 18.760
  16:44:54 STEP 2895 H=2895:00 +CPU= 30.679
  16:45:13 STEP 2896 H=2896:00 +CPU= 19.697
  16:45:37 STEP 2897 H=2897:00 +CPU= 23.892
  16:46:12 STEP 2898 H=2898:00 +CPU= 34.232
  16:46:29 STEP 2899 H=2899:00 +CPU= 17.386
  16:46:47 STEP 2900 H=2900:00 +CPU= 17.908
  16:47:23 STEP 2901 H=2901:00 +CPU= 36.438
  16:47:46 STEP 2902 H=2902:00 +CPU= 22.211
  16:48:04 STEP 2903 H=2903:00 +CPU= 18.860
  16:48:40 STEP 2904 H=2904:00 +CPU= 35.929
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_120.zip
Uploading the intermediate file: upload_file_120.zip
  16:49:01 STEP 2905 H=2905:00 +CPU= 20.790
  16:49:21 STEP 2906 H=2906:00 +CPU= 19.967
  16:49:54 STEP 2907 H=2907:00 +CPU= 32.656
  16:50:11 STEP 2908 H=2908:00 +CPU= 17.161
  16:50:30 STEP 2909 H=2909:00 +CPU= 18.829
  16:51:01 STEP 2910 H=2910:00 +CPU= 31.400
  16:51:22 STEP 2911 H=2911:00 +CPU= 20.584
  16:51:44 STEP 2912 H=2912:00 +CPU= 22.050
  16:52:16 STEP 2913 H=2913:00 +CPU= 32.455
  16:52:43 STEP 2914 H=2914:00 +CPU= 27.155
  16:53:05 STEP 2915 H=2915:00 +CPU= 21.231
  16:53:40 STEP 2916 H=2916:00 +CPU= 34.945
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002916
  16:54:07 STEP 2917 H=2917:00 +CPU= 26.980
  16:54:30 STEP 2918 H=2918:00 +CPU= 23.020
  16:55:02 STEP 2919 H=2919:00 +CPU= 32.436
  16:55:25 STEP 2920 H=2920:00 +CPU= 22.870
  16:55:44 STEP 2921 H=2921:00 +CPU= 19.492
  16:56:18 STEP 2922 H=2922:00 +CPU= 33.224
  16:56:41 STEP 2923 H=2923:00 +CPU= 23.234
  16:57:04 STEP 2924 H=2924:00 +CPU= 23.307
  16:57:46 STEP 2925 H=2925:00 +CPU= 42.057
  16:58:13 STEP 2926 H=2926:00 +CPU= 26.923
  16:58:33 STEP 2927 H=2927:00 +CPU= 20.231
  16:59:17 STEP 2928 H=2928:00 +CPU= 43.420
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_121.zip
Uploading the intermediate file: upload_file_121.zip
  16:59:43 STEP 2929 H=2929:00 +CPU= 25.836
  17:00:07 STEP 2930 H=2930:00 +CPU= 24.716
  17:00:42 STEP 2931 H=2931:00 +CPU= 34.606
  17:01:02 STEP 2932 H=2932:00 +CPU= 20.415
  17:01:26 STEP 2933 H=2933:00 +CPU= 23.643
  17:02:04 STEP 2934 H=2934:00 +CPU= 37.903
  17:02:27 STEP 2935 H=2935:00 +CPU= 22.961
  17:02:51 STEP 2936 H=2936:00 +CPU= 24.025
  17:03:26 STEP 2937 H=2937:00 +CPU= 34.638
  17:03:49 STEP 2938 H=2938:00 +CPU= 23.611
  17:04:13 STEP 2939 H=2939:00 +CPU= 23.681
  17:05:00 STEP 2940 H=2940:00 +CPU= 47.029
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002940
  17:05:28 STEP 2941 H=2941:00 +CPU= 27.872
  17:05:56 STEP 2942 H=2942:00 +CPU= 28.732
  17:06:37 STEP 2943 H=2943:00 +CPU= 40.525
  17:06:59 STEP 2944 H=2944:00 +CPU= 21.824
  17:07:25 STEP 2945 H=2945:00 +CPU= 25.872
  17:07:56 STEP 2946 H=2946:00 +CPU= 31.694
  17:08:24 STEP 2947 H=2947:00 +CPU= 27.759
  17:08:51 STEP 2948 H=2948:00 +CPU= 27.222
  17:09:31 STEP 2949 H=2949:00 +CPU= 39.633
  17:09:59 STEP 2950 H=2950:00 +CPU= 28.408
  17:10:24 STEP 2951 H=2951:00 +CPU= 24.865
  17:11:09 STEP 2952 H=2952:00 +CPU= 44.775
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client_40001/slots/2/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12192813/ICMSHhq0f+002928
Zipping up the final file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0169_2007050100_123_976_12192813_0_r922115705_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
17:14:32 (1161): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:39:37 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 10,623,600 77,751 0.0073
09 May 2023 16:39:34 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 10,195,200 74,164 0.0073
09 May 2023 16:39:34 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 10,195,200 74,164 0.0073
09 May 2023 16:39:29 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 9,331,200 68,618 0.0074
09 May 2023 16:39:20 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 8,467,200 62,961 0.0074
09 May 2023 16:39:20 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 8,467,200 62,961 0.0074
09 May 2023 16:39:09 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 7,430,400 54,849 0.0074
09 May 2023 16:39:02 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 6,566,400 49,502 0.0075
09 May 2023 16:38:58 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 5,702,400 44,207 0.0078
09 May 2023 16:38:58 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 5,702,400 44,207 0.0078
09 May 2023 16:38:50 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 4,838,400 38,850 0.0080
09 May 2023 16:38:50 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 4,838,400 38,850 0.0080
09 May 2023 16:38:41 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 3,456,000 28,454 0.0082
09 May 2023 16:38:34 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 2,592,000 21,412 0.0083
09 May 2023 16:38:26 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 1,728,000 14,199 0.0082
09 May 2023 16:38:17 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 864,000 7,269 0.0084
09 May 2023 16:38:17 1538241 22278575 oifs_43r3_ps_0169_2007050100_123_976_12192813_0 864,000 7,269 0.0084


©2024 cpdn.org