climateprediction.net home page
Task 22255662

Task 22255662

Name oifs_43r3_ps_0661_1985050100_123_954_12171305_0
Workunit 12171305
Created 21 Dec 2022, 6:35:37 UTC
Sent 21 Dec 2022, 6:43:50 UTC
Report deadline 20 Jan 2023, 6:43:50 UTC
Received 25 Dec 2022, 9:34:12 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1478188
Run time 21 hours 46 min 25 sec
CPU time 21 hours 24 min 11 sec
Validate state Invalid
Credit 8,588.16
Device peak FLOPS 4.18 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.39 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,171.73 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
a/boinc/boinc-client/slots/21/ICMUAhq0f+001536
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001512
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001512
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001512
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001524
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001524
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001524
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_63.zip
Uploading the intermediate file: upload_file_63.zip
  17:56:29 STEP 1537 H=1537:00 +CPU= 29.950
  17:57:00 STEP 1538 H=1538:00 +CPU= 30.395
  17:57:47 STEP 1539 H=1539:00 +CPU= 46.870
  17:58:18 STEP 1540 H=1540:00 +CPU= 30.764
  17:58:48 STEP 1541 H=1541:00 +CPU= 30.281
  17:59:36 STEP 1542 H=1542:00 +CPU= 47.047
  18:00:06 STEP 1543 H=1543:00 +CPU= 30.580
  18:00:37 STEP 1544 H=1544:00 +CPU= 29.949
  18:01:24 STEP 1545 H=1545:00 +CPU= 46.994
  18:01:54 STEP 1546 H=1546:00 +CPU= 30.012
  18:02:24 STEP 1547 H=1547:00 +CPU= 30.176
  18:03:18 STEP 1548 H=1548:00 +CPU= 52.821
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001548
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001548
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001548
  18:03:49 STEP 1549 H=1549:00 +CPU= 31.160
  18:04:20 STEP 1550 H=1550:00 +CPU= 30.575
  18:05:08 STEP 1551 H=1551:00 +CPU= 47.975
  18:05:40 STEP 1552 H=1552:00 +CPU= 31.594
  18:06:10 STEP 1553 H=1553:00 +CPU= 29.946
  18:06:59 STEP 1554 H=1554:00 +CPU= 48.624
  18:07:29 STEP 1555 H=1555:00 +CPU= 30.271
  18:08:01 STEP 1556 H=1556:00 +CPU= 31.632
  18:08:50 STEP 1557 H=1557:00 +CPU= 47.980
  18:09:22 STEP 1558 H=1558:00 +CPU= 32.068
  18:09:53 STEP 1559 H=1559:00 +CPU= 30.774
  18:10:49 STEP 1560 H=1560:00 +CPU= 55.959
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001560
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001560
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001560
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001536
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001536
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001536
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001548
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001548
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001548
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_64.zip
Uploading the intermediate file: upload_file_64.zip
  18:11:19 STEP 1561 H=1561:00 +CPU= 29.836
  18:11:51 STEP 1562 H=1562:00 +CPU= 31.852
  18:12:39 STEP 1563 H=1563:00 +CPU= 47.130
  18:13:11 STEP 1564 H=1564:00 +CPU= 31.782
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:40:41 STEP 1565 H=1565:00 +CPU= 30.308
  18:41:29 STEP 1566 H=1566:00 +CPU= 48.275
  18:42:00 STEP 1567 H=1567:00 +CPU= 30.937
  18:42:31 STEP 1568 H=1568:00 +CPU= 30.509
  18:43:17 STEP 1569 H=1569:00 +CPU= 46.324
  18:43:47 STEP 1570 H=1570:00 +CPU= 29.445
  18:44:15 STEP 1571 H=1571:00 +CPU= 28.349
  18:45:09 STEP 1572 H=1572:00 +CPU= 53.533
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001572
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001572
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001572
  18:45:39 STEP 1573 H=1573:00 +CPU= 30.031
  18:46:13 STEP 1574 H=1574:00 +CPU= 33.583
  18:47:00 STEP 1575 H=1575:00 +CPU= 46.997
  18:47:31 STEP 1576 H=1576:00 +CPU= 30.743
  18:48:01 STEP 1577 H=1577:00 +CPU= 30.227
  18:48:50 STEP 1578 H=1578:00 +CPU= 48.708
  18:49:20 STEP 1579 H=1579:00 +CPU= 29.695
  18:49:50 STEP 1580 H=1580:00 +CPU= 29.651
  18:50:37 STEP 1581 H=1581:00 +CPU= 47.390
  18:51:10 STEP 1582 H=1582:00 +CPU= 32.555
  18:51:40 STEP 1583 H=1583:00 +CPU= 29.705
  18:52:36 STEP 1584 H=1584:00 +CPU= 55.975
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001584
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001584
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001584
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001560
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001560
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001560
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001572
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001572
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001572
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_65.zip
Uploading the intermediate file: upload_file_65.zip
  18:53:06 STEP 1585 H=1585:00 +CPU= 29.769
  18:53:36 STEP 1586 H=1586:00 +CPU= 30.332
  18:54:25 STEP 1587 H=1587:00 +CPU= 48.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:57:30 STEP 1588 H=1588:00 +CPU= 31.659
  18:57:59 STEP 1589 H=1589:00 +CPU= 29.514
  18:58:48 STEP 1590 H=1590:00 +CPU= 48.206
  18:59:17 STEP 1591 H=1591:00 +CPU= 29.132
  18:59:48 STEP 1592 H=1592:00 +CPU= 30.592
  19:00:33 STEP 1593 H=1593:00 +CPU= 45.102
  19:01:04 STEP 1594 H=1594:00 +CPU= 31.500
  19:01:33 STEP 1595 H=1595:00 +CPU= 28.982
  19:02:28 STEP 1596 H=1596:00 +CPU= 54.020
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001596
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001596
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001596
  19:02:56 STEP 1597 H=1597:00 +CPU= 28.165
  19:03:28 STEP 1598 H=1598:00 +CPU= 31.577
  19:04:16 STEP 1599 H=1599:00 +CPU= 47.756
  19:04:47 STEP 1600 H=1600:00 +CPU= 31.267
  19:05:16 STEP 1601 H=1601:00 +CPU= 29.359
  19:06:04 STEP 1602 H=1602:00 +CPU= 47.859
  19:06:35 STEP 1603 H=1603:00 +CPU= 30.825
  19:07:07 STEP 1604 H=1604:00 +CPU= 31.473
  19:07:55 STEP 1605 H=1605:00 +CPU= 47.764
  19:08:25 STEP 1606 H=1606:00 +CPU= 30.151
  19:08:57 STEP 1607 H=1607:00 +CPU= 31.348
  19:09:53 STEP 1608 H=1608:00 +CPU= 55.711
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001608
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001608
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001608
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001584
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001584
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001584
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001596
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001596
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001596
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_66.zip
Uploading the intermediate file: upload_file_66.zip
  19:10:22 STEP 1609 H=1609:00 +CPU= 29.431
  19:10:54 STEP 1610 H=1610:00 +CPU= 31.510
  19:11:41 STEP 1611 H=1611:00 +CPU= 47.231
  19:12:13 STEP 1612 H=1612:00 +CPU= 31.494
  19:12:44 STEP 1613 H=1613:00 +CPU= 31.290
  19:13:31 STEP 1614 H=1614:00 +CPU= 46.684
  19:14:00 STEP 1615 H=1615:00 +CPU= 28.985
  19:14:33 STEP 1616 H=1616:00 +CPU= 32.213
  19:15:22 STEP 1617 H=1617:00 +CPU= 48.866
  19:15:52 STEP 1618 H=1618:00 +CPU= 29.973
  19:16:22 STEP 1619 H=1619:00 +CPU= 29.906
  19:17:14 STEP 1620 H=1620:00 +CPU= 51.985
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001620
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001620
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001620
  19:17:44 STEP 1621 H=1621:00 +CPU= 30.443
  19:18:16 STEP 1622 H=1622:00 +CPU= 31.287
  19:19:02 STEP 1623 H=1623:00 +CPU= 46.564
  19:19:34 STEP 1624 H=1624:00 +CPU= 31.428
  19:20:04 STEP 1625 H=1625:00 +CPU= 30.286
  19:20:54 STEP 1626 H=1626:00 +CPU= 49.229
  19:21:23 STEP 1627 H=1627:00 +CPU= 29.584
  19:21:55 STEP 1628 H=1628:00 +CPU= 31.074
  19:22:43 STEP 1629 H=1629:00 +CPU= 48.539
  19:23:13 STEP 1630 H=1630:00 +CPU= 29.169
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:32:29 STEP 1631 H=1631:00 +CPU= 31.083
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:43:40 STEP 1632 H=1632:00 +CPU= 58.116
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001632
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001632
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001632
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001608
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001608
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001608
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001620
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001620
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001620
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_67.zip
Uploading the intermediate file: upload_file_67.zip
  20:44:10 STEP 1633 H=1633:00 +CPU= 29.741
  20:44:43 STEP 1634 H=1634:00 +CPU= 32.189
  20:45:31 STEP 1635 H=1635:00 +CPU= 47.556
  20:46:02 STEP 1636 H=1636:00 +CPU= 31.315
  20:46:33 STEP 1637 H=1637:00 +CPU= 30.164
  20:47:22 STEP 1638 H=1638:00 +CPU= 48.806
  20:47:51 STEP 1639 H=1639:00 +CPU= 29.533
  20:48:23 STEP 1640 H=1640:00 +CPU= 31.026
  20:49:10 STEP 1641 H=1641:00 +CPU= 46.852
  20:49:42 STEP 1642 H=1642:00 +CPU= 32.132
  20:50:12 STEP 1643 H=1643:00 +CPU= 29.965
  20:51:04 STEP 1644 H=1644:00 +CPU= 51.976
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001644
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001644
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001644
  20:51:35 STEP 1645 H=1645:00 +CPU= 30.273
  20:52:06 STEP 1646 H=1646:00 +CPU= 31.479
  20:52:55 STEP 1647 H=1647:00 +CPU= 47.783
  20:53:26 STEP 1648 H=1648:00 +CPU= 31.539
  20:53:57 STEP 1649 H=1649:00 +CPU= 30.185
  20:54:45 STEP 1650 H=1650:00 +CPU= 48.133
  20:55:17 STEP 1651 H=1651:00 +CPU= 32.065
  20:55:48 STEP 1652 H=1652:00 +CPU= 30.772
  20:56:37 STEP 1653 H=1653:00 +CPU= 48.173
  20:57:10 STEP 1654 H=1654:00 +CPU= 32.276
  20:57:41 STEP 1655 H=1655:00 +CPU= 30.703
  20:58:39 STEP 1656 H=1656:00 +CPU= 58.052
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001656
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001656
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001656
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001632
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001632
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001632
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001644
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001644
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001644
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_68.zip
Uploading the intermediate file: upload_file_68.zip
  20:59:10 STEP 1657 H=1657:00 +CPU= 30.829
  20:59:42 STEP 1658 H=1658:00 +CPU= 31.001
  21:00:29 STEP 1659 H=1659:00 +CPU= 47.335
  21:01:02 STEP 1660 H=1660:00 +CPU= 32.950
  21:01:34 STEP 1661 H=1661:00 +CPU= 31.461
  21:02:23 STEP 1662 H=1662:00 +CPU= 48.340
  21:02:53 STEP 1663 H=1663:00 +CPU= 30.334
  21:03:24 STEP 1664 H=1664:00 +CPU= 30.590
  21:04:11 STEP 1665 H=1665:00 +CPU= 46.610
  21:04:42 STEP 1666 H=1666:00 +CPU= 30.930
  21:05:13 STEP 1667 H=1667:00 +CPU= 30.557
  21:06:06 STEP 1668 H=1668:00 +CPU= 52.887
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001668
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001668
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001668
  21:06:36 STEP 1669 H=1669:00 +CPU= 30.164
  21:07:08 STEP 1670 H=1670:00 +CPU= 31.390
  21:07:57 STEP 1671 H=1671:00 +CPU= 49.097
  21:08:28 STEP 1672 H=1672:00 +CPU= 31.184
  21:08:59 STEP 1673 H=1673:00 +CPU= 30.541
  21:09:50 STEP 1674 H=1674:00 +CPU= 50.219
  21:10:22 STEP 1675 H=1675:00 +CPU= 32.067
  21:10:54 STEP 1676 H=1676:00 +CPU= 31.622
  21:11:41 STEP 1677 H=1677:00 +CPU= 47.179
  21:12:13 STEP 1678 H=1678:00 +CPU= 31.688
  21:12:45 STEP 1679 H=1679:00 +CPU= 31.071
  21:13:43 STEP 1680 H=1680:00 +CPU= 57.707
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001680
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001680
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001680
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001656
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001656
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001656
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001668
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001668
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001668
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_69.zip
Uploading the intermediate file: upload_file_69.zip
  21:14:12 STEP 1681 H=1681:00 +CPU= 29.169
  21:14:44 STEP 1682 H=1682:00 +CPU= 31.282
  21:15:32 STEP 1683 H=1683:00 +CPU= 47.566
  21:16:03 STEP 1684 H=1684:00 +CPU= 31.636
  21:16:34 STEP 1685 H=1685:00 +CPU= 30.454
  21:17:24 STEP 1686 H=1686:00 +CPU= 49.146
  21:17:54 STEP 1687 H=1687:00 +CPU= 30.209
  21:18:26 STEP 1688 H=1688:00 +CPU= 31.514
  21:19:15 STEP 1689 H=1689:00 +CPU= 49.077
  21:19:47 STEP 1690 H=1690:00 +CPU= 31.683
  21:20:18 STEP 1691 H=1691:00 +CPU= 31.193
  21:21:11 STEP 1692 H=1692:00 +CPU= 52.694
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001692
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001692
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001692
  21:21:41 STEP 1693 H=1693:00 +CPU= 29.768
  21:22:11 STEP 1694 H=1694:00 +CPU= 30.059
  21:22:59 STEP 1695 H=1695:00 +CPU= 47.485
  21:23:31 STEP 1696 H=1696:00 +CPU= 32.123
  21:24:03 STEP 1697 H=1697:00 +CPU= 31.302
  21:24:51 STEP 1698 H=1698:00 +CPU= 48.271
  21:25:22 STEP 1699 H=1699:00 +CPU= 30.379
  21:25:53 STEP 1700 H=1700:00 +CPU= 30.583
  21:26:42 STEP 1701 H=1701:00 +CPU= 49.348
  21:27:15 STEP 1702 H=1702:00 +CPU= 32.085
  21:27:47 STEP 1703 H=1703:00 +CPU= 31.685
  21:28:44 STEP 1704 H=1704:00 +CPU= 56.907
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001704
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001704
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001704
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001680
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001680
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001680
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001692
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001692
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001692
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_70.zip
Uploading the intermediate file: upload_file_70.zip
Uploading trickle at timestep: 6134400
  21:29:15 STEP 1705 H=1705:00 +CPU= 30.681
  21:29:48 STEP 1706 H=1706:00 +CPU= 32.268
  21:30:36 STEP 1707 H=1707:00 +CPU= 47.692
  21:31:08 STEP 1708 H=1708:00 +CPU= 32.182
  21:31:40 STEP 1709 H=1709:00 +CPU= 31.096
  21:32:29 STEP 1710 H=1710:00 +CPU= 49.083
  21:33:01 STEP 1711 H=1711:00 +CPU= 31.213
  21:33:33 STEP 1712 H=1712:00 +CPU= 32.232
  21:34:21 STEP 1713 H=1713:00 +CPU= 47.435
  21:34:53 STEP 1714 H=1714:00 +CPU= 31.813
  21:35:23 STEP 1715 H=1715:00 +CPU= 30.724
  21:36:18 STEP 1716 H=1716:00 +CPU= 54.041
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001716
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001716
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001716
  21:36:49 STEP 1717 H=1717:00 +CPU= 31.088
  21:37:21 STEP 1718 H=1718:00 +CPU= 32.057
  21:38:11 STEP 1719 H=1719:00 +CPU= 48.971
  21:38:43 STEP 1720 H=1720:00 +CPU= 32.061
  21:39:15 STEP 1721 H=1721:00 +CPU= 31.971
  21:40:04 STEP 1722 H=1722:00 +CPU= 48.205
  21:40:33 STEP 1723 H=1723:00 +CPU= 29.264
  21:41:04 STEP 1724 H=1724:00 +CPU= 30.862
  21:41:52 STEP 1725 H=1725:00 +CPU= 47.577
  21:42:23 STEP 1726 H=1726:00 +CPU= 31.384
  21:42:56 STEP 1727 H=1727:00 +CPU= 32.635
  21:43:53 STEP 1728 H=1728:00 +CPU= 56.143
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001728
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001728
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001728
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001704
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001704
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001704
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001716
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001716
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001716
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_71.zip
Uploading the intermediate file: upload_file_71.zip
  21:44:25 STEP 1729 H=1729:00 +CPU= 31.354
  21:44:57 STEP 1730 H=1730:00 +CPU= 32.379
  21:45:46 STEP 1731 H=1731:00 +CPU= 48.156
  21:46:17 STEP 1732 H=1732:00 +CPU= 31.359
  21:46:48 STEP 1733 H=1733:00 +CPU= 31.045
  21:47:38 STEP 1734 H=1734:00 +CPU= 49.026
  21:48:08 STEP 1735 H=1735:00 +CPU= 30.293
  21:48:39 STEP 1736 H=1736:00 +CPU= 31.104
  21:49:29 STEP 1737 H=1737:00 +CPU= 48.895
  21:50:00 STEP 1738 H=1738:00 +CPU= 31.644
  21:50:31 STEP 1739 H=1739:00 +CPU= 30.332
  21:51:24 STEP 1740 H=1740:00 +CPU= 52.692
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001740
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001740
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001740
  21:51:55 STEP 1741 H=1741:00 +CPU= 30.793
  21:52:26 STEP 1742 H=1742:00 +CPU= 30.668
  21:53:15 STEP 1743 H=1743:00 +CPU= 48.530
  21:53:46 STEP 1744 H=1744:00 +CPU= 31.048
  21:54:17 STEP 1745 H=1745:00 +CPU= 30.761
  21:55:06 STEP 1746 H=1746:00 +CPU= 48.673
  21:55:37 STEP 1747 H=1747:00 +CPU= 30.954
  21:56:10 STEP 1748 H=1748:00 +CPU= 32.678
  21:57:00 STEP 1749 H=1749:00 +CPU= 50.196
  21:57:32 STEP 1750 H=1750:00 +CPU= 31.767
  21:58:04 STEP 1751 H=1751:00 +CPU= 31.401
  21:59:02 STEP 1752 H=1752:00 +CPU= 57.785
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001752
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001752
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001752
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001728
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001728
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001728
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001740
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001740
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001740
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_72.zip
Uploading the intermediate file: upload_file_72.zip
  21:59:34 STEP 1753 H=1753:00 +CPU= 31.750
  22:00:06 STEP 1754 H=1754:00 +CPU= 31.708
  22:00:55 STEP 1755 H=1755:00 +CPU= 48.690
  22:01:29 STEP 1756 H=1756:00 +CPU= 33.077
  22:02:01 STEP 1757 H=1757:00 +CPU= 31.627
  22:02:48 STEP 1758 H=1758:00 +CPU= 47.600
  22:03:20 STEP 1759 H=1759:00 +CPU= 31.276
  22:03:52 STEP 1760 H=1760:00 +CPU= 32.122
  22:04:40 STEP 1761 H=1761:00 +CPU= 47.590
  22:05:13 STEP 1762 H=1762:00 +CPU= 32.445
  22:05:44 STEP 1763 H=1763:00 +CPU= 31.266
  22:06:37 STEP 1764 H=1764:00 +CPU= 52.467
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001764
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001764
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001764
  22:07:08 STEP 1765 H=1765:00 +CPU= 30.466
  22:07:41 STEP 1766 H=1766:00 +CPU= 33.303
  22:08:29 STEP 1767 H=1767:00 +CPU= 47.429
  22:09:02 STEP 1768 H=1768:00 +CPU= 32.685
  22:09:33 STEP 1769 H=1769:00 +CPU= 31.263
  22:10:22 STEP 1770 H=1770:00 +CPU= 48.328
  22:10:52 STEP 1771 H=1771:00 +CPU= 30.358
  22:11:24 STEP 1772 H=1772:00 +CPU= 31.440
  22:12:13 STEP 1773 H=1773:00 +CPU= 48.813
  22:12:45 STEP 1774 H=1774:00 +CPU= 31.207
  22:13:17 STEP 1775 H=1775:00 +CPU= 31.963
  22:14:17 STEP 1776 H=1776:00 +CPU= 58.993
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001776
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001776
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001776
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001752
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001752
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001752
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001764
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001764
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001764
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_73.zip
Uploading the intermediate file: upload_file_73.zip
  22:14:49 STEP 1777 H=1777:00 +CPU= 31.508
  22:15:20 STEP 1778 H=1778:00 +CPU= 31.258
  22:16:08 STEP 1779 H=1779:00 +CPU= 47.134
  22:16:40 STEP 1780 H=1780:00 +CPU= 31.852
  22:17:11 STEP 1781 H=1781:00 +CPU= 30.826
  22:18:00 STEP 1782 H=1782:00 +CPU= 48.747
  22:18:31 STEP 1783 H=1783:00 +CPU= 31.040
  22:19:02 STEP 1784 H=1784:00 +CPU= 31.051
  22:19:52 STEP 1785 H=1785:00 +CPU= 49.863
  22:20:24 STEP 1786 H=1786:00 +CPU= 31.440
  22:20:56 STEP 1787 H=1787:00 +CPU= 31.313
  22:21:48 STEP 1788 H=1788:00 +CPU= 51.709
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001788
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001788
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001788
  22:22:17 STEP 1789 H=1789:00 +CPU= 29.279
  22:22:48 STEP 1790 H=1790:00 +CPU= 30.917
  22:23:35 STEP 1791 H=1791:00 +CPU= 46.723
  22:24:07 STEP 1792 H=1792:00 +CPU= 31.363
  22:24:36 STEP 1793 H=1793:00 +CPU= 29.506
  22:25:25 STEP 1794 H=1794:00 +CPU= 47.910
  22:25:55 STEP 1795 H=1795:00 +CPU= 29.872
  22:26:26 STEP 1796 H=1796:00 +CPU= 31.431
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:34:02 STEP 1797 H=1797:00 +CPU= 46.319
  22:34:33 STEP 1798 H=1798:00 +CPU= 31.514
  22:35:03 STEP 1799 H=1799:00 +CPU= 29.410
  22:35:59 STEP 1800 H=1800:00 +CPU= 55.727
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001800
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001800
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001800
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001776
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001776
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001776
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001788
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001788
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001788
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_74.zip
Uploading the intermediate file: upload_file_74.zip
  22:36:29 STEP 1801 H=1801:00 +CPU= 29.626
  22:37:00 STEP 1802 H=1802:00 +CPU= 30.540
  22:37:46 STEP 1803 H=1803:00 +CPU= 46.280
  22:38:18 STEP 1804 H=1804:00 +CPU= 31.517
  22:38:49 STEP 1805 H=1805:00 +CPU= 30.505
  22:39:37 STEP 1806 H=1806:00 +CPU= 47.759
  22:40:06 STEP 1807 H=1807:00 +CPU= 29.327
  22:40:39 STEP 1808 H=1808:00 +CPU= 31.993
  22:41:26 STEP 1809 H=1809:00 +CPU= 46.803
  22:41:57 STEP 1810 H=1810:00 +CPU= 31.270
  22:42:29 STEP 1811 H=1811:00 +CPU= 31.384
  22:43:21 STEP 1812 H=1812:00 +CPU= 51.779
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001812
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001812
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001812
  22:43:50 STEP 1813 H=1813:00 +CPU= 29.087
  22:44:22 STEP 1814 H=1814:00 +CPU= 31.929
  22:45:11 STEP 1815 H=1815:00 +CPU= 48.354
  22:45:42 STEP 1816 H=1816:00 +CPU= 30.470
  22:46:12 STEP 1817 H=1817:00 +CPU= 30.554
  22:47:00 STEP 1818 H=1818:00 +CPU= 47.509
  22:47:30 STEP 1819 H=1819:00 +CPU= 29.866
  22:48:02 STEP 1820 H=1820:00 +CPU= 32.160
  22:48:50 STEP 1821 H=1821:00 +CPU= 47.932
  22:49:21 STEP 1822 H=1822:00 +CPU= 30.543
  22:49:53 STEP 1823 H=1823:00 +CPU= 31.329
  22:50:49 STEP 1824 H=1824:00 +CPU= 55.798
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001824
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001824
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001824
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001800
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001800
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001800
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001812
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001812
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001812
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_75.zip
Uploading the intermediate file: upload_file_75.zip
  22:51:19 STEP 1825 H=1825:00 +CPU= 29.927
  22:51:50 STEP 1826 H=1826:00 +CPU= 31.173
  22:52:38 STEP 1827 H=1827:00 +CPU= 47.005
  22:53:09 STEP 1828 H=1828:00 +CPU= 31.357
  22:53:40 STEP 1829 H=1829:00 +CPU= 30.722
  22:54:27 STEP 1830 H=1830:00 +CPU= 47.045
  22:54:57 STEP 1831 H=1831:00 +CPU= 29.968
  22:55:29 STEP 1832 H=1832:00 +CPU= 31.520
  22:56:17 STEP 1833 H=1833:00 +CPU= 47.694
  22:56:47 STEP 1834 H=1834:00 +CPU= 29.981
  22:57:17 STEP 1835 H=1835:00 +CPU= 30.173
  22:58:10 STEP 1836 H=1836:00 +CPU= 52.075
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001836
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001836
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001836
  22:58:41 STEP 1837 H=1837:00 +CPU= 30.957
  22:59:12 STEP 1838 H=1838:00 +CPU= 31.049
  23:00:00 STEP 1839 H=1839:00 +CPU= 47.599
  23:00:30 STEP 1840 H=1840:00 +CPU= 30.236
  23:01:00 STEP 1841 H=1841:00 +CPU= 29.877
  23:01:50 STEP 1842 H=1842:00 +CPU= 49.066
  23:02:20 STEP 1843 H=1843:00 +CPU= 30.631
  23:02:51 STEP 1844 H=1844:00 +CPU= 30.569
  23:03:40 STEP 1845 H=1845:00 +CPU= 48.294
  23:04:13 STEP 1846 H=1846:00 +CPU= 33.522
  23:04:45 STEP 1847 H=1847:00 +CPU= 31.195
  23:05:42 STEP 1848 H=1848:00 +CPU= 56.422
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001848
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001848
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001848
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001824
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001824
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001824
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001836
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001836
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001836
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_76.zip
Uploading the intermediate file: upload_file_76.zip
  23:06:12 STEP 1849 H=1849:00 +CPU= 30.174
  23:06:42 STEP 1850 H=1850:00 +CPU= 29.908
  23:07:29 STEP 1851 H=1851:00 +CPU= 46.830
  23:08:01 STEP 1852 H=1852:00 +CPU= 31.208
  23:08:32 STEP 1853 H=1853:00 +CPU= 31.090
  23:09:21 STEP 1854 H=1854:00 +CPU= 48.951
  23:09:52 STEP 1855 H=1855:00 +CPU= 30.527
  23:10:24 STEP 1856 H=1856:00 +CPU= 32.186
  23:11:12 STEP 1857 H=1857:00 +CPU= 47.843
  23:11:43 STEP 1858 H=1858:00 +CPU= 30.210
  23:12:13 STEP 1859 H=1859:00 +CPU= 30.397
  23:13:07 STEP 1860 H=1860:00 +CPU= 53.517
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001860
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001860
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001860
  23:13:38 STEP 1861 H=1861:00 +CPU= 30.951
  23:14:10 STEP 1862 H=1862:00 +CPU= 31.383
  23:14:58 STEP 1863 H=1863:00 +CPU= 48.291
  23:15:30 STEP 1864 H=1864:00 +CPU= 31.553
  23:16:01 STEP 1865 H=1865:00 +CPU= 30.735
  23:16:49 STEP 1866 H=1866:00 +CPU= 48.500
  23:17:21 STEP 1867 H=1867:00 +CPU= 30.913
  23:17:51 STEP 1868 H=1868:00 +CPU= 30.476
  23:18:39 STEP 1869 H=1869:00 +CPU= 47.212
  23:19:10 STEP 1870 H=1870:00 +CPU= 30.850
  23:19:41 STEP 1871 H=1871:00 +CPU= 31.034
  23:20:39 STEP 1872 H=1872:00 +CPU= 57.234
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001872
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001872
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001872
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001848
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001848
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001848
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001860
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001860
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001860
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_77.zip
Uploading the intermediate file: upload_file_77.zip
  23:21:09 STEP 1873 H=1873:00 +CPU= 29.873
  23:21:39 STEP 1874 H=1874:00 +CPU= 29.823
  23:22:27 STEP 1875 H=1875:00 +CPU= 47.746
  23:22:57 STEP 1876 H=1876:00 +CPU= 29.601
  23:23:28 STEP 1877 H=1877:00 +CPU= 30.927
  23:24:16 STEP 1878 H=1878:00 +CPU= 47.836
  23:24:46 STEP 1879 H=1879:00 +CPU= 29.903
  23:25:17 STEP 1880 H=1880:00 +CPU= 31.107
  23:26:05 STEP 1881 H=1881:00 +CPU= 47.033
  23:26:36 STEP 1882 H=1882:00 +CPU= 31.108
  23:27:07 STEP 1883 H=1883:00 +CPU= 31.294
  23:28:01 STEP 1884 H=1884:00 +CPU= 53.352
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001884
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001884
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001884
  23:28:32 STEP 1885 H=1885:00 +CPU= 30.397
  23:29:04 STEP 1886 H=1886:00 +CPU= 32.282
  23:29:53 STEP 1887 H=1887:00 +CPU= 48.624
  23:30:25 STEP 1888 H=1888:00 +CPU= 31.733
  23:30:55 STEP 1889 H=1889:00 +CPU= 29.545
  23:31:43 STEP 1890 H=1890:00 +CPU= 48.143
  23:32:13 STEP 1891 H=1891:00 +CPU= 29.546
  23:32:45 STEP 1892 H=1892:00 +CPU= 31.835
  23:33:33 STEP 1893 H=1893:00 +CPU= 47.858
  23:34:04 STEP 1894 H=1894:00 +CPU= 30.853
  23:34:35 STEP 1895 H=1895:00 +CPU= 30.856
  23:35:31 STEP 1896 H=1896:00 +CPU= 56.046
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001896
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001896
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001896
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001872
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001872
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001872
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001884
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001884
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001884
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_78.zip
Uploading the intermediate file: upload_file_78.zip
  23:36:02 STEP 1897 H=1897:00 +CPU= 30.834
  23:36:34 STEP 1898 H=1898:00 +CPU= 31.963
  23:37:22 STEP 1899 H=1899:00 +CPU= 47.720
  23:37:54 STEP 1900 H=1900:00 +CPU= 31.174
  23:38:24 STEP 1901 H=1901:00 +CPU= 30.071
  23:39:14 STEP 1902 H=1902:00 +CPU= 49.939
  23:39:45 STEP 1903 H=1903:00 +CPU= 30.461
  23:40:16 STEP 1904 H=1904:00 +CPU= 31.232
  23:41:04 STEP 1905 H=1905:00 +CPU= 47.246
  23:41:36 STEP 1906 H=1906:00 +CPU= 31.790
  23:42:07 STEP 1907 H=1907:00 +CPU= 30.462
  23:43:00 STEP 1908 H=1908:00 +CPU= 52.875
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001908
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001908
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001908
  23:43:31 STEP 1909 H=1909:00 +CPU= 30.897
  23:44:02 STEP 1910 H=1910:00 +CPU= 31.380
  23:44:50 STEP 1911 H=1911:00 +CPU= 47.610
  23:45:21 STEP 1912 H=1912:00 +CPU= 31.396
  23:45:51 STEP 1913 H=1913:00 +CPU= 29.726
  23:46:40 STEP 1914 H=1914:00 +CPU= 48.856
  23:47:11 STEP 1915 H=1915:00 +CPU= 30.010
  23:47:41 STEP 1916 H=1916:00 +CPU= 30.434
  23:48:30 STEP 1917 H=1917:00 +CPU= 48.140
  23:49:00 STEP 1918 H=1918:00 +CPU= 30.257
  23:49:31 STEP 1919 H=1919:00 +CPU= 31.274
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235016:1671925816:27782.920] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 2639MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.359] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.359] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.359] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.359] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.359] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 2639 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.359] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        DYNFPOS 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         STEPO_FPOS 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          SCAN2M_VPOS 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           >OMP-SCAN2M VPOS          (1435) 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            VPOS 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             POS 
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:60409:60409] [20221224:235017:1671925817:27783.370] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              PPINIT 
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1985050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0661
(argv4) batchid: 954
(argv5) wuid: 12171305
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /data/boinc/boinc-client/slots/21
Project directory is: /data/boinc/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305
..mkdir for temp folder for results failed
Copying: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_926d27fbf10b779bf8022ae26edf4e85 to: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_0661_1985050100_123_954_12171305.zip
Unzipping the namelist zip file: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_0661_1985050100_123_954_12171305.zip
ic_ancil_file: ic_ancil_12171305
ifsdata_file: ifsdata_12171305
climate_data_file: clim_data_12171305
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_2008536d4f46f1862c81cb8a8f6ce4c4 to: /data/boinc/boinc-client/slots/21/ic_ancil_12171305.zip
Unzipping the IC ancils zip file: /data/boinc/boinc-client/slots/21/ic_ancil_12171305.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5da8b5231eb158d51b13d22bbbd35f14 to: /data/boinc/boinc-client/slots/21/ifsdata/ifsdata_12171305.zip
Unzipping the ifsdata_zip file: /data/boinc/boinc-client/slots/21/ifsdata/ifsdata_12171305.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /data/boinc/boinc-client/slots/21/159l_2/clim_data_12171305.zip
Unzipping the climate data zip file: /data/boinc/boinc-client/slots/21/159l_2/clim_data_12171305.zip
Checking for progress XML file: /data/boinc/boinc-client/slots/21/progress_file_12171305.xml
Opened progress file ok : /data/boinc/boinc-client/slots/21/progress_file_12171305.xml
-- Model is restarting --
Adjusting last_iter, 1920, to previous model restart step.
Creating progress file: /data/boinc/boinc-client/slots/21/progress_file_12171305.xml
last_cpu_time: 73193
upload_file_number: 79
last_iter: 1919
last_upload: 6825600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035
The child process has been launched with process id: 8847
Executing the command: /data/boinc/boinc-client/slots/21/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:8847:8847] [20221225:054107:1671946867:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:43:18 STEP 1920 H=1920:00 +CPU= 59.577
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001920
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001896
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001896
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001896
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001908
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001908
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001908
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_79.zip
Uploading the intermediate file: upload_file_79.zip
  05:43:55 STEP 1921 H=1921:00 +CPU= 34.748
  05:44:32 STEP 1922 H=1922:00 +CPU= 35.803
  05:45:28 STEP 1923 H=1923:00 +CPU= 53.461
  05:46:06 STEP 1924 H=1924:00 +CPU= 35.758
  05:46:41 STEP 1925 H=1925:00 +CPU= 34.428
  05:47:39 STEP 1926 H=1926:00 +CPU= 54.533
  05:48:15 STEP 1927 H=1927:00 +CPU= 34.725
  05:48:53 STEP 1928 H=1928:00 +CPU= 35.653
  05:49:48 STEP 1929 H=1929:00 +CPU= 53.603
  05:50:25 STEP 1930 H=1930:00 +CPU= 35.646
  05:51:01 STEP 1931 H=1931:00 +CPU= 34.611
  05:52:03 STEP 1932 H=1932:00 +CPU= 59.057
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001932
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001932
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001932
  05:52:40 STEP 1933 H=1933:00 +CPU= 34.652
  05:53:17 STEP 1934 H=1934:00 +CPU= 35.469
  05:54:13 STEP 1935 H=1935:00 +CPU= 53.224
  05:54:50 STEP 1936 H=1936:00 +CPU= 35.574
  05:55:26 STEP 1937 H=1937:00 +CPU= 34.328
  05:56:24 STEP 1938 H=1938:00 +CPU= 55.059
  05:57:00 STEP 1939 H=1939:00 +CPU= 34.893
  05:57:38 STEP 1940 H=1940:00 +CPU= 36.202
  05:58:32 STEP 1941 H=1941:00 +CPU= 53.877
  05:59:08 STEP 1942 H=1942:00 +CPU= 35.503
  05:59:43 STEP 1943 H=1943:00 +CPU= 34.972
  06:00:48 STEP 1944 H=1944:00 +CPU= 62.458
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001944
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001944
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001944
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001920
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001932
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001932
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001932
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_80.zip
Uploading the intermediate file: upload_file_80.zip
  06:01:24 STEP 1945 H=1945:00 +CPU= 34.953
  06:02:00 STEP 1946 H=1946:00 +CPU= 35.495
  06:02:49 STEP 1947 H=1947:00 +CPU= 49.094
  06:03:21 STEP 1948 H=1948:00 +CPU= 31.873
  06:03:53 STEP 1949 H=1949:00 +CPU= 31.288
  06:04:44 STEP 1950 H=1950:00 +CPU= 50.621
  06:05:17 STEP 1951 H=1951:00 +CPU= 31.847
  06:05:48 STEP 1952 H=1952:00 +CPU= 31.370
  06:06:34 STEP 1953 H=1953:00 +CPU= 46.336
  06:07:06 STEP 1954 H=1954:00 +CPU= 31.397
  06:07:35 STEP 1955 H=1955:00 +CPU= 29.448
  06:08:28 STEP 1956 H=1956:00 +CPU= 52.085
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001956
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001956
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001956
  06:08:58 STEP 1957 H=1957:00 +CPU= 30.528
  06:09:30 STEP 1958 H=1958:00 +CPU= 31.617
  06:10:17 STEP 1959 H=1959:00 +CPU= 47.004
  06:10:48 STEP 1960 H=1960:00 +CPU= 30.837
  06:11:19 STEP 1961 H=1961:00 +CPU= 30.375
  06:12:05 STEP 1962 H=1962:00 +CPU= 46.124
  06:12:34 STEP 1963 H=1963:00 +CPU= 29.382
  06:13:04 STEP 1964 H=1964:00 +CPU= 29.497
  06:13:51 STEP 1965 H=1965:00 +CPU= 46.890
  06:14:21 STEP 1966 H=1966:00 +CPU= 29.753
  06:14:52 STEP 1967 H=1967:00 +CPU= 30.554
  06:15:49 STEP 1968 H=1968:00 +CPU= 56.870
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001968
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001968
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001968
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001944
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001944
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001944
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001956
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001956
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001956
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_81.zip
Uploading the intermediate file: upload_file_81.zip
  06:16:21 STEP 1969 H=1969:00 +CPU= 31.511
  06:16:52 STEP 1970 H=1970:00 +CPU= 31.413
  06:17:40 STEP 1971 H=1971:00 +CPU= 47.403
  06:18:09 STEP 1972 H=1972:00 +CPU= 29.521
  06:18:40 STEP 1973 H=1973:00 +CPU= 30.267
  06:19:25 STEP 1974 H=1974:00 +CPU= 44.721
  06:19:55 STEP 1975 H=1975:00 +CPU= 30.487
  06:20:26 STEP 1976 H=1976:00 +CPU= 30.882
  06:21:14 STEP 1977 H=1977:00 +CPU= 47.840
  06:21:46 STEP 1978 H=1978:00 +CPU= 31.747
  06:22:14 STEP 1979 H=1979:00 +CPU= 28.434
  06:23:06 STEP 1980 H=1980:00 +CPU= 50.930
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001980
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001980
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001980
  06:23:34 STEP 1981 H=1981:00 +CPU= 28.060
  06:24:06 STEP 1982 H=1982:00 +CPU= 31.821
  06:24:52 STEP 1983 H=1983:00 +CPU= 46.154
  06:25:23 STEP 1984 H=1984:00 +CPU= 30.884
  06:25:54 STEP 1985 H=1985:00 +CPU= 30.644
  06:26:42 STEP 1986 H=1986:00 +CPU= 47.730
  06:27:13 STEP 1987 H=1987:00 +CPU= 31.481
  06:27:44 STEP 1988 H=1988:00 +CPU= 30.511
  06:28:32 STEP 1989 H=1989:00 +CPU= 47.664
  06:29:02 STEP 1990 H=1990:00 +CPU= 30.667
  06:29:33 STEP 1991 H=1991:00 +CPU= 30.102
  06:30:29 STEP 1992 H=1992:00 +CPU= 56.113
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+001992
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+001992
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+001992
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001968
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001968
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001968
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001980
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001980
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001980
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_82.zip
Uploading the intermediate file: upload_file_82.zip
  06:30:59 STEP 1993 H=1993:00 +CPU= 30.197
  06:31:30 STEP 1994 H=1994:00 +CPU= 30.755
  06:32:18 STEP 1995 H=1995:00 +CPU= 47.344
  06:32:50 STEP 1996 H=1996:00 +CPU= 31.779
  06:33:21 STEP 1997 H=1997:00 +CPU= 31.089
  06:34:11 STEP 1998 H=1998:00 +CPU= 49.605
  06:34:41 STEP 1999 H=1999:00 +CPU= 30.617
  06:35:12 STEP 2000 H=2000:00 +CPU= 31.019
  06:35:59 STEP 2001 H=2001:00 +CPU= 46.338
  06:36:30 STEP 2002 H=2002:00 +CPU= 30.944
  06:36:59 STEP 2003 H=2003:00 +CPU= 29.172
  06:37:51 STEP 2004 H=2004:00 +CPU= 51.514
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002004
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002004
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002004
  06:38:22 STEP 2005 H=2005:00 +CPU= 30.602
  06:38:52 STEP 2006 H=2006:00 +CPU= 30.047
  06:39:40 STEP 2007 H=2007:00 +CPU= 48.167
  06:40:12 STEP 2008 H=2008:00 +CPU= 31.496
  06:40:42 STEP 2009 H=2009:00 +CPU= 29.960
  06:41:30 STEP 2010 H=2010:00 +CPU= 48.011
  06:42:00 STEP 2011 H=2011:00 +CPU= 29.726
  06:42:30 STEP 2012 H=2012:00 +CPU= 30.363
  06:43:17 STEP 2013 H=2013:00 +CPU= 46.098
  06:43:46 STEP 2014 H=2014:00 +CPU= 29.518
  06:44:17 STEP 2015 H=2015:00 +CPU= 30.573
  06:45:11 STEP 2016 H=2016:00 +CPU= 54.055
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002016
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002016
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002016
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+001992
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+001992
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+001992
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMGGhq0f+002004
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMSHhq0f+002004
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171305/ICMUAhq0f+002004
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0661_1985050100_123_954_12171305_0_r702651035_83.zip
Uploading the intermediate file: upload_file_83.zip
  06:45:41 STEP 2017 H=2017:00 +CPU= 29.147
  06:46:11 STEP 2018 H=2018:00 +CPU= 30.387
  06:46:59 STEP 2019 H=2019:00 +CPU= 47.565
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:53:21 1478188 22255662 oifs_43r3_ps_0661_1985050100_123_954_12171305_0 6,134,400 65,201 0.0106
09 May 2023 15:50:35 1478188 22255662 oifs_43r3_ps_0661_1985050100_123_954_12171305_0 4,579,200 48,418 0.0106
09 May 2023 15:48:37 1478188 22255662 oifs_43r3_ps_0661_1985050100_123_954_12171305_0 3,283,200 34,786 0.0106
09 May 2023 15:46:26 1478188 22255662 oifs_43r3_ps_0661_1985050100_123_954_12171305_0 1,123,200 11,411 0.0102


©2024 cpdn.org