Name | oifs_43r3_ps_0239_2004050100_123_973_12189883_0 |
Workunit | 12189883 |
Created | 23 Dec 2022, 7:27:11 UTC |
Sent | 23 Dec 2022, 10:46:15 UTC |
Report deadline | 22 Jan 2023, 10:46:15 UTC |
Received | 24 Dec 2022, 6:26:48 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1520317 |
Run time | 12 hours 58 min 37 sec |
CPU time | 12 hours 24 min 12 sec |
Validate state | Invalid |
Credit | 8,709.12 |
Device peak FLOPS | 8.55 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,406.27 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,682.10 MB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> s_43r3_ps_12189883/ICMUAhq0f+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001572 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001572 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001572 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_65.zip Uploading the intermediate file: upload_file_65.zip 09:21:55 STEP 1585 H=1585:00 +CPU= 15.028 09:22:16 STEP 1586 H=1586:00 +CPU= 19.519 09:22:47 STEP 1587 H=1587:00 +CPU= 29.807 09:23:08 STEP 1588 H=1588:00 +CPU= 19.952 09:23:29 STEP 1589 H=1589:00 +CPU= 20.129 09:24:02 STEP 1590 H=1590:00 +CPU= 31.360 09:24:24 STEP 1591 H=1591:00 +CPU= 19.893 09:24:44 STEP 1592 H=1592:00 +CPU= 20.048 09:25:18 STEP 1593 H=1593:00 +CPU= 30.927 09:25:35 STEP 1594 H=1594:00 +CPU= 17.353 09:25:56 STEP 1595 H=1595:00 +CPU= 19.402 09:26:30 STEP 1596 H=1596:00 +CPU= 32.005 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001596 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001596 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001596 09:26:50 STEP 1597 H=1597:00 +CPU= 18.907 09:27:11 STEP 1598 H=1598:00 +CPU= 20.400 09:27:43 STEP 1599 H=1599:00 +CPU= 30.377 09:28:04 STEP 1600 H=1600:00 +CPU= 20.022 09:28:24 STEP 1601 H=1601:00 +CPU= 18.946 09:28:56 STEP 1602 H=1602:00 +CPU= 30.224 09:29:17 STEP 1603 H=1603:00 +CPU= 19.287 09:29:39 STEP 1604 H=1604:00 +CPU= 20.032 09:30:11 STEP 1605 H=1605:00 +CPU= 29.679 09:30:32 STEP 1606 H=1606:00 +CPU= 20.315 09:30:54 STEP 1607 H=1607:00 +CPU= 20.360 09:31:31 STEP 1608 H=1608:00 +CPU= 35.113 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001608 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001608 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001596 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001596 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001596 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_66.zip Uploading the intermediate file: upload_file_66.zip 09:31:49 STEP 1609 H=1609:00 +CPU= 17.628 09:32:08 STEP 1610 H=1610:00 +CPU= 18.363 09:32:35 STEP 1611 H=1611:00 +CPU= 26.869 09:32:50 STEP 1612 H=1612:00 +CPU= 14.275 09:33:06 STEP 1613 H=1613:00 +CPU= 15.747 09:33:38 STEP 1614 H=1614:00 +CPU= 30.080 09:33:58 STEP 1615 H=1615:00 +CPU= 19.117 09:34:19 STEP 1616 H=1616:00 +CPU= 20.042 09:34:51 STEP 1617 H=1617:00 +CPU= 29.121 09:35:12 STEP 1618 H=1618:00 +CPU= 19.551 09:35:32 STEP 1619 H=1619:00 +CPU= 18.737 09:36:07 STEP 1620 H=1620:00 +CPU= 32.848 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001620 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001620 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001620 09:36:28 STEP 1621 H=1621:00 +CPU= 19.300 09:36:49 STEP 1622 H=1622:00 +CPU= 21.233 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:39:52 STEP 1623 H=1623:00 +CPU= 30.544 09:40:13 STEP 1624 H=1624:00 +CPU= 19.857 09:40:32 STEP 1625 H=1625:00 +CPU= 18.716 09:41:04 STEP 1626 H=1626:00 +CPU= 29.975 09:41:25 STEP 1627 H=1627:00 +CPU= 19.134 09:41:47 STEP 1628 H=1628:00 +CPU= 19.674 09:42:17 STEP 1629 H=1629:00 +CPU= 27.480 09:42:38 STEP 1630 H=1630:00 +CPU= 20.194 09:43:00 STEP 1631 H=1631:00 +CPU= 19.965 09:43:39 STEP 1632 H=1632:00 +CPU= 36.041 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001632 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001632 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001620 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001620 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001620 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_67.zip Uploading the intermediate file: upload_file_67.zip 09:44:01 STEP 1633 H=1633:00 +CPU= 20.176 09:44:24 STEP 1634 H=1634:00 +CPU= 20.495 09:44:56 STEP 1635 H=1635:00 +CPU= 30.000 09:45:18 STEP 1636 H=1636:00 +CPU= 19.883 09:45:38 STEP 1637 H=1637:00 +CPU= 19.431 09:46:13 STEP 1638 H=1638:00 +CPU= 32.330 09:46:35 STEP 1639 H=1639:00 +CPU= 20.002 09:46:58 STEP 1640 H=1640:00 +CPU= 20.940 09:47:31 STEP 1641 H=1641:00 +CPU= 30.535 09:47:53 STEP 1642 H=1642:00 +CPU= 20.298 09:48:14 STEP 1643 H=1643:00 +CPU= 19.493 09:48:51 STEP 1644 H=1644:00 +CPU= 33.898 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001644 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001644 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001644 09:49:13 STEP 1645 H=1645:00 +CPU= 19.694 09:49:35 STEP 1646 H=1646:00 +CPU= 20.335 09:50:08 STEP 1647 H=1647:00 +CPU= 30.402 09:50:29 STEP 1648 H=1648:00 +CPU= 19.927 09:50:52 STEP 1649 H=1649:00 +CPU= 19.863 09:51:25 STEP 1650 H=1650:00 +CPU= 31.395 09:51:46 STEP 1651 H=1651:00 +CPU= 19.001 09:52:08 STEP 1652 H=1652:00 +CPU= 20.819 09:52:39 STEP 1653 H=1653:00 +CPU= 29.948 09:53:02 STEP 1654 H=1654:00 +CPU= 21.663 09:53:26 STEP 1655 H=1655:00 +CPU= 21.998 09:54:06 STEP 1656 H=1656:00 +CPU= 36.688 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001656 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001656 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001644 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001644 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001644 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_68.zip Uploading the intermediate file: upload_file_68.zip 09:54:28 STEP 1657 H=1657:00 +CPU= 19.861 09:54:50 STEP 1658 H=1658:00 +CPU= 20.491 09:55:24 STEP 1659 H=1659:00 +CPU= 30.869 09:55:46 STEP 1660 H=1660:00 +CPU= 21.117 09:56:06 STEP 1661 H=1661:00 +CPU= 19.388 09:56:40 STEP 1662 H=1662:00 +CPU= 31.647 09:56:58 STEP 1663 H=1663:00 +CPU= 16.695 09:57:22 STEP 1664 H=1664:00 +CPU= 21.202 09:57:57 STEP 1665 H=1665:00 +CPU= 32.591 09:58:22 STEP 1666 H=1666:00 +CPU= 21.450 09:58:40 STEP 1667 H=1667:00 +CPU= 18.292 09:59:14 STEP 1668 H=1668:00 +CPU= 31.063 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001668 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001668 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001668 09:59:31 STEP 1669 H=1669:00 +CPU= 17.466 09:59:52 STEP 1670 H=1670:00 +CPU= 19.865 10:00:25 STEP 1671 H=1671:00 +CPU= 31.375 10:00:47 STEP 1672 H=1672:00 +CPU= 20.026 10:01:07 STEP 1673 H=1673:00 +CPU= 18.986 10:01:39 STEP 1674 H=1674:00 +CPU= 30.409 10:02:01 STEP 1675 H=1675:00 +CPU= 20.624 10:02:24 STEP 1676 H=1676:00 +CPU= 21.886 10:02:56 STEP 1677 H=1677:00 +CPU= 30.335 10:03:18 STEP 1678 H=1678:00 +CPU= 19.973 10:03:41 STEP 1679 H=1679:00 +CPU= 21.121 10:04:21 STEP 1680 H=1680:00 +CPU= 37.479 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001680 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001680 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001668 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001668 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001668 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_69.zip Uploading the intermediate file: upload_file_69.zip 10:04:44 STEP 1681 H=1681:00 +CPU= 20.844 10:05:06 STEP 1682 H=1682:00 +CPU= 20.915 10:05:41 STEP 1683 H=1683:00 +CPU= 32.385 10:06:04 STEP 1684 H=1684:00 +CPU= 21.055 10:06:26 STEP 1685 H=1685:00 +CPU= 20.410 10:06:59 STEP 1686 H=1686:00 +CPU= 30.566 10:07:21 STEP 1687 H=1687:00 +CPU= 21.164 10:07:44 STEP 1688 H=1688:00 +CPU= 21.389 10:08:17 STEP 1689 H=1689:00 +CPU= 31.509 10:08:40 STEP 1690 H=1690:00 +CPU= 20.796 10:09:02 STEP 1691 H=1691:00 +CPU= 20.426 10:09:42 STEP 1692 H=1692:00 +CPU= 35.452 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001692 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001692 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001692 10:10:00 STEP 1693 H=1693:00 +CPU= 16.978 10:10:25 STEP 1694 H=1694:00 +CPU= 22.382 10:10:58 STEP 1695 H=1695:00 +CPU= 31.836 10:11:20 STEP 1696 H=1696:00 +CPU= 20.511 10:11:42 STEP 1697 H=1697:00 +CPU= 20.504 10:12:17 STEP 1698 H=1698:00 +CPU= 32.628 10:12:38 STEP 1699 H=1699:00 +CPU= 19.754 10:12:58 STEP 1700 H=1700:00 +CPU= 19.109 10:13:27 STEP 1701 H=1701:00 +CPU= 27.271 10:13:48 STEP 1702 H=1702:00 +CPU= 19.915 10:14:08 STEP 1703 H=1703:00 +CPU= 19.094 10:14:48 STEP 1704 H=1704:00 +CPU= 36.116 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001704 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001704 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001692 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001692 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001692 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_70.zip Uploading the intermediate file: upload_file_70.zip 10:15:09 STEP 1705 H=1705:00 +CPU= 19.544 10:15:31 STEP 1706 H=1706:00 +CPU= 20.341 10:16:05 STEP 1707 H=1707:00 +CPU= 31.118 10:16:28 STEP 1708 H=1708:00 +CPU= 20.892 10:16:50 STEP 1709 H=1709:00 +CPU= 20.355 10:17:25 STEP 1710 H=1710:00 +CPU= 32.143 10:17:47 STEP 1711 H=1711:00 +CPU= 20.306 10:18:10 STEP 1712 H=1712:00 +CPU= 20.710 10:18:42 STEP 1713 H=1713:00 +CPU= 30.327 10:19:06 STEP 1714 H=1714:00 +CPU= 20.688 10:19:28 STEP 1715 H=1715:00 +CPU= 19.785 10:20:04 STEP 1716 H=1716:00 +CPU= 33.546 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001716 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001716 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001716 10:20:25 STEP 1717 H=1717:00 +CPU= 19.528 10:20:48 STEP 1718 H=1718:00 +CPU= 20.659 10:21:17 STEP 1719 H=1719:00 +CPU= 27.681 10:21:39 STEP 1720 H=1720:00 +CPU= 21.248 10:22:00 STEP 1721 H=1721:00 +CPU= 20.033 10:22:34 STEP 1722 H=1722:00 +CPU= 30.980 10:22:55 STEP 1723 H=1723:00 +CPU= 19.642 10:23:17 STEP 1724 H=1724:00 +CPU= 20.083 10:23:49 STEP 1725 H=1725:00 +CPU= 29.942 10:24:11 STEP 1726 H=1726:00 +CPU= 19.998 10:24:32 STEP 1727 H=1727:00 +CPU= 19.858 10:25:11 STEP 1728 H=1728:00 +CPU= 35.759 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001728 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001728 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001716 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001716 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001716 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_71.zip Uploading the intermediate file: upload_file_71.zip 10:25:32 STEP 1729 H=1729:00 +CPU= 19.062 Uploading trickle at timestep: 6220800 10:25:53 STEP 1730 H=1730:00 +CPU= 19.671 10:26:27 STEP 1731 H=1731:00 +CPU= 29.808 10:26:48 STEP 1732 H=1732:00 +CPU= 19.564 10:27:09 STEP 1733 H=1733:00 +CPU= 19.219 10:27:44 STEP 1734 H=1734:00 +CPU= 31.264 10:28:06 STEP 1735 H=1735:00 +CPU= 19.515 10:28:27 STEP 1736 H=1736:00 +CPU= 19.879 10:29:00 STEP 1737 H=1737:00 +CPU= 30.808 10:29:22 STEP 1738 H=1738:00 +CPU= 20.599 10:29:43 STEP 1739 H=1739:00 +CPU= 19.364 10:30:20 STEP 1740 H=1740:00 +CPU= 33.287 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001740 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001740 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001740 10:30:41 STEP 1741 H=1741:00 +CPU= 19.239 10:31:02 STEP 1742 H=1742:00 +CPU= 19.397 10:31:29 STEP 1743 H=1743:00 +CPU= 26.091 10:31:46 STEP 1744 H=1744:00 +CPU= 17.347 10:32:04 STEP 1745 H=1745:00 +CPU= 17.087 10:32:45 STEP 1746 H=1746:00 +CPU= 32.899 10:33:07 STEP 1747 H=1747:00 +CPU= 20.573 10:33:29 STEP 1748 H=1748:00 +CPU= 21.307 10:34:03 STEP 1749 H=1749:00 +CPU= 31.719 10:34:27 STEP 1750 H=1750:00 +CPU= 22.154 10:34:51 STEP 1751 H=1751:00 +CPU= 21.692 10:35:31 STEP 1752 H=1752:00 +CPU= 37.461 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001752 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001752 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001740 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001740 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001740 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_72.zip Uploading the intermediate file: upload_file_72.zip 10:35:54 STEP 1753 H=1753:00 +CPU= 20.489 10:36:16 STEP 1754 H=1754:00 +CPU= 20.729 10:36:50 STEP 1755 H=1755:00 +CPU= 30.834 10:37:13 STEP 1756 H=1756:00 +CPU= 21.435 10:37:37 STEP 1757 H=1757:00 +CPU= 21.273 10:38:14 STEP 1758 H=1758:00 +CPU= 33.208 10:38:36 STEP 1759 H=1759:00 +CPU= 20.765 10:38:59 STEP 1760 H=1760:00 +CPU= 21.010 10:39:35 STEP 1761 H=1761:00 +CPU= 32.196 10:39:58 STEP 1762 H=1762:00 +CPU= 22.238 10:40:20 STEP 1763 H=1763:00 +CPU= 20.275 10:40:57 STEP 1764 H=1764:00 +CPU= 35.332 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001764 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001764 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001764 10:41:19 STEP 1765 H=1765:00 +CPU= 20.523 10:41:42 STEP 1766 H=1766:00 +CPU= 20.700 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2004050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0239 (argv4) batchid: 973 (argv5) wuid: 12189883 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/6 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip ic_ancil_file: ic_ancil_12189883 ifsdata_file: ifsdata_12189883 climate_data_file: clim_data_12189883 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml -- Model is restarting -- Adjusting last_iter, 1767, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml last_cpu_time: 42170 upload_file_number: 73 last_iter: 1751 last_upload: 6307200 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587 The child process has been launched with process id: 674170 Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 10:44:17 STEP 1752 H=1752:00 +CPU= 35.661 10:44:39 STEP 1753 H=1753:00 +CPU= 20.533 10:45:02 STEP 1754 H=1754:00 +CPU= 21.158 10:45:36 STEP 1755 H=1755:00 +CPU= 31.821 10:46:00 STEP 1756 H=1756:00 +CPU= 21.834 10:46:23 STEP 1757 H=1757:00 +CPU= 20.724 10:46:58 STEP 1758 H=1758:00 +CPU= 32.225 10:47:20 STEP 1759 H=1759:00 +CPU= 20.415 10:47:43 STEP 1760 H=1760:00 +CPU= 20.590 10:48:16 STEP 1761 H=1761:00 +CPU= 30.136 10:48:38 STEP 1762 H=1762:00 +CPU= 19.973 10:48:58 STEP 1763 H=1763:00 +CPU= 19.255 10:49:29 STEP 1764 H=1764:00 +CPU= 29.111 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001764 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001764 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001764 10:49:51 STEP 1765 H=1765:00 +CPU= 19.913 10:50:13 STEP 1766 H=1766:00 +CPU= 19.990 10:50:46 STEP 1767 H=1767:00 +CPU= 30.319 10:51:03 STEP 1768 H=1768:00 +CPU= 16.523 10:51:25 STEP 1769 H=1769:00 +CPU= 20.201 10:51:55 STEP 1770 H=1770:00 +CPU= 28.789 10:52:13 STEP 1771 H=1771:00 +CPU= 17.045 10:52:35 STEP 1772 H=1772:00 +CPU= 20.218 10:53:09 STEP 1773 H=1773:00 +CPU= 31.001 10:53:32 STEP 1774 H=1774:00 +CPU= 20.840 10:53:53 STEP 1775 H=1775:00 +CPU= 19.825 10:54:34 STEP 1776 H=1776:00 +CPU= 35.903 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001776 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001776 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001764 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001764 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001764 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_73.zip Uploading the intermediate file: upload_file_73.zip 10:54:55 STEP 1777 H=1777:00 +CPU= 19.046 10:55:17 STEP 1778 H=1778:00 +CPU= 20.056 10:55:51 STEP 1779 H=1779:00 +CPU= 30.779 10:56:13 STEP 1780 H=1780:00 +CPU= 19.883 10:56:35 STEP 1781 H=1781:00 +CPU= 19.696 10:57:09 STEP 1782 H=1782:00 +CPU= 31.065 10:57:27 STEP 1783 H=1783:00 +CPU= 17.457 10:57:49 STEP 1784 H=1784:00 +CPU= 20.325 10:58:22 STEP 1785 H=1785:00 +CPU= 30.866 10:58:44 STEP 1786 H=1786:00 +CPU= 20.033 10:59:04 STEP 1787 H=1787:00 +CPU= 19.359 10:59:40 STEP 1788 H=1788:00 +CPU= 34.024 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001788 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001788 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001788 11:00:01 STEP 1789 H=1789:00 +CPU= 19.388 11:00:20 STEP 1790 H=1790:00 +CPU= 17.990 11:00:53 STEP 1791 H=1791:00 +CPU= 30.847 11:01:15 STEP 1792 H=1792:00 +CPU= 20.359 11:01:33 STEP 1793 H=1793:00 +CPU= 17.003 11:02:04 STEP 1794 H=1794:00 +CPU= 30.077 11:02:27 STEP 1795 H=1795:00 +CPU= 20.136 11:02:49 STEP 1796 H=1796:00 +CPU= 20.363 11:03:20 STEP 1797 H=1797:00 +CPU= 30.201 11:03:39 STEP 1798 H=1798:00 +CPU= 17.606 Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2004050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0239 (argv4) batchid: 973 (argv5) wuid: 12189883 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/6 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip ic_ancil_file: ic_ancil_12189883 ifsdata_file: ifsdata_12189883 climate_data_file: clim_data_12189883 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml -- Model is restarting -- Adjusting last_iter, 1799, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml last_cpu_time: 43293 upload_file_number: 74 last_iter: 1775 last_upload: 6393600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587 The child process has been launched with process id: 690803 Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 14:00:22 STEP 1776 H=1776:00 +CPU= 35.259 14:00:44 STEP 1777 H=1777:00 +CPU= 21.047 14:01:06 STEP 1778 H=1778:00 +CPU= 21.100 14:01:39 STEP 1779 H=1779:00 +CPU= 31.945 14:01:59 STEP 1780 H=1780:00 +CPU= 19.588 14:02:19 STEP 1781 H=1781:00 +CPU= 19.234 14:02:51 STEP 1782 H=1782:00 +CPU= 30.088 14:03:08 STEP 1783 H=1783:00 +CPU= 16.839 14:03:26 STEP 1784 H=1784:00 +CPU= 18.065 14:03:51 STEP 1785 H=1785:00 +CPU= 23.989 14:04:04 STEP 1786 H=1786:00 +CPU= 12.621 14:04:16 STEP 1787 H=1787:00 +CPU= 12.374 14:04:38 STEP 1788 H=1788:00 +CPU= 21.406 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001788 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001788 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001788 14:04:56 STEP 1789 H=1789:00 +CPU= 16.966 14:05:15 STEP 1790 H=1790:00 +CPU= 19.090 14:05:47 STEP 1791 H=1791:00 +CPU= 30.480 14:06:07 STEP 1792 H=1792:00 +CPU= 19.973 14:06:25 STEP 1793 H=1793:00 +CPU= 17.286 14:06:59 STEP 1794 H=1794:00 +CPU= 32.078 14:07:18 STEP 1795 H=1795:00 +CPU= 18.925 14:07:38 STEP 1796 H=1796:00 +CPU= 19.377 14:08:10 STEP 1797 H=1797:00 +CPU= 29.466 14:08:31 STEP 1798 H=1798:00 +CPU= 19.540 14:08:52 STEP 1799 H=1799:00 +CPU= 19.583 14:09:30 STEP 1800 H=1800:00 +CPU= 34.814 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001800 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001800 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001788 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001788 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001788 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_74.zip Uploading the intermediate file: upload_file_74.zip 14:09:41 STEP 1801 H=1801:00 +CPU= 11.200 14:09:50 STEP 1802 H=1802:00 +CPU= 9.302 Suspend request received from the BOINC client, suspending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2004050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0239 (argv4) batchid: 973 (argv5) wuid: 12189883 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/6 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip ic_ancil_file: ic_ancil_12189883 ifsdata_file: ifsdata_12189883 climate_data_file: clim_data_12189883 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml -- Model is restarting -- Adjusting last_iter, 1801, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml last_cpu_time: 43860 upload_file_number: 75 last_iter: 1799 last_upload: 6480000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587 The child process has been launched with process id: 9824 Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 16:11:43 STEP 1800 H=1800:00 +CPU= 40.175 16:12:05 STEP 1801 H=1801:00 +CPU= 20.500 16:12:26 STEP 1802 H=1802:00 +CPU= 20.026 16:12:58 STEP 1803 H=1803:00 +CPU= 30.809 16:13:19 STEP 1804 H=1804:00 +CPU= 20.347 16:13:39 STEP 1805 H=1805:00 +CPU= 19.897 16:14:11 STEP 1806 H=1806:00 +CPU= 30.219 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2004050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0239 (argv4) batchid: 973 (argv5) wuid: 12189883 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/6 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip ic_ancil_file: ic_ancil_12189883 ifsdata_file: ifsdata_12189883 climate_data_file: clim_data_12189883 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml -- Model is restarting -- Adjusting last_iter, 1807, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml last_cpu_time: 44060 upload_file_number: 75 last_iter: 1799 last_upload: 6480000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587 The child process has been launched with process id: 10080 Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 16:16:09 STEP 1800 H=1800:00 +CPU= 33.404 16:16:28 STEP 1801 H=1801:00 +CPU= 18.157 16:16:50 STEP 1802 H=1802:00 +CPU= 20.955 16:17:21 STEP 1803 H=1803:00 +CPU= 30.149 16:17:40 STEP 1804 H=1804:00 +CPU= 18.587 16:18:03 STEP 1805 H=1805:00 +CPU= 20.694 16:18:37 STEP 1806 H=1806:00 +CPU= 33.125 16:18:57 STEP 1807 H=1807:00 +CPU= 18.589 16:19:20 STEP 1808 H=1808:00 +CPU= 22.840 16:19:52 STEP 1809 H=1809:00 +CPU= 30.987 16:20:12 STEP 1810 H=1810:00 +CPU= 19.286 16:20:33 STEP 1811 H=1811:00 +CPU= 19.727 16:21:09 STEP 1812 H=1812:00 +CPU= 34.428 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001812 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001812 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001812 16:21:29 STEP 1813 H=1813:00 +CPU= 18.801 16:21:50 STEP 1814 H=1814:00 +CPU= 19.232 16:22:21 STEP 1815 H=1815:00 +CPU= 29.261 16:22:40 STEP 1816 H=1816:00 +CPU= 19.228 16:23:00 STEP 1817 H=1817:00 +CPU= 19.091 16:23:33 STEP 1818 H=1818:00 +CPU= 31.284 16:23:53 STEP 1819 H=1819:00 +CPU= 19.052 16:24:14 STEP 1820 H=1820:00 +CPU= 19.827 16:24:49 STEP 1821 H=1821:00 +CPU= 30.755 16:25:09 STEP 1822 H=1822:00 +CPU= 19.224 16:25:32 STEP 1823 H=1823:00 +CPU= 20.471 Abort request received from BOINC client, ending the child process ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 15:51:14 | 1520317 | 22275446 | oifs_43r3_ps_0239_2004050100_123_973_12189883_0 | 6,220,800 | 41,246 | 0.0066 |
09 May 2023 15:51:04 | 1520317 | 22275446 | oifs_43r3_ps_0239_2004050100_123_973_12189883_0 | 5,184,000 | 34,377 | 0.0066 |
09 May 2023 15:50:51 | 1520317 | 22275446 | oifs_43r3_ps_0239_2004050100_123_973_12189883_0 | 4,060,800 | 26,886 | 0.0066 |
09 May 2023 15:50:39 | 1520317 | 22275446 | oifs_43r3_ps_0239_2004050100_123_973_12189883_0 | 3,024,000 | 19,578 | 0.0065 |
09 May 2023 15:50:27 | 1520317 | 22275446 | oifs_43r3_ps_0239_2004050100_123_973_12189883_0 | 1,900,800 | 12,360 | 0.0065 |
09 May 2023 15:50:15 | 1520317 | 22275446 | oifs_43r3_ps_0239_2004050100_123_973_12189883_0 | 864,000 | 5,994 | 0.0069 |
©2024 cpdn.org