Name | oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1 |
Workunit | 12294180 |
Created | 25 Jun 2024, 12:48:40 UTC |
Sent | 25 Jun 2024, 12:49:32 UTC |
Report deadline | 24 Aug 2024, 12:49:32 UTC |
Received | 2 Jul 2024, 14:57:37 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 1496581 |
Run time | 21 hours 40 min 21 sec |
CPU time | 21 hours 40 min 21 sec |
Validate state | Valid |
Credit | 2,635.55 |
Device peak FLOPS | 4.07 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.13 x86_64-pc-linux-gnu |
Peak working set size | 5,022.63 MB |
Peak swap size | 5,981.07 MB |
Peak disk usage | 1,316.98 MB |
Stderr | <core_client_version>7.24.1</core_client_version> <![CDATA[ <stderr_txt> ateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001632 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001644 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001644 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001644 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001656 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001656 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001656 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001668 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001668 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001668 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001680 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001680 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001680 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001692 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001692 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001692 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001704 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001704 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001704 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001716 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001716 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001716 Zipping up the intermediate file: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1_r1823311562_17.zip Uploading the intermediate file: upload_file_17.zip 13:40:15 STEP 1729 H= 432:15 +CPU= 36.512 13:40:53 STEP 1730 H= 432:30 +CPU= 36.588 13:41:31 STEP 1731 H= 432:45 +CPU= 36.516 13:42:10 STEP 1732 H= 433:00 +CPU= 37.227 13:42:48 STEP 1733 H= 433:15 +CPU= 36.482 13:43:26 STEP 1734 H= 433:30 +CPU= 36.507 13:44:03 STEP 1735 H= 433:45 +CPU= 36.532 13:44:42 STEP 1736 H= 434:00 +CPU= 37.215 13:45:21 STEP 1737 H= 434:15 +CPU= 36.564 13:45:58 STEP 1738 H= 434:30 +CPU= 36.527 13:46:36 STEP 1739 H= 434:45 +CPU= 36.532 13:47:24 STEP 1740 H= 435:00 +CPU= 45.679 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001740 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001740 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001740 13:48:02 STEP 1741 H= 435:15 +CPU= 36.470 13:48:40 STEP 1742 H= 435:30 +CPU= 36.476 13:49:17 STEP 1743 H= 435:45 +CPU= 36.545 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:50:17 STEP 1744 H= 436:00 +CPU= 36.807 13:50:54 STEP 1745 H= 436:15 +CPU= 36.619 13:51:33 STEP 1746 H= 436:30 +CPU= 36.513 13:52:11 STEP 1747 H= 436:45 +CPU= 36.579 13:52:50 STEP 1748 H= 437:00 +CPU= 37.286 13:53:28 STEP 1749 H= 437:15 +CPU= 36.576 13:54:06 STEP 1750 H= 437:30 +CPU= 36.603 13:54:47 STEP 1751 H= 437:45 +CPU= 36.662 13:55:37 STEP 1752 H= 438:00 +CPU= 45.990 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001752 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001752 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001752 13:56:15 STEP 1753 H= 438:15 +CPU= 36.675 13:56:54 STEP 1754 H= 438:30 +CPU= 36.636 13:57:32 STEP 1755 H= 438:45 +CPU= 36.584 13:58:11 STEP 1756 H= 439:00 +CPU= 37.414 13:58:49 STEP 1757 H= 439:15 +CPU= 36.551 13:59:27 STEP 1758 H= 439:30 +CPU= 36.594 14:00:06 STEP 1759 H= 439:45 +CPU= 36.615 14:00:46 STEP 1760 H= 440:00 +CPU= 37.367 14:01:24 STEP 1761 H= 440:15 +CPU= 36.608 14:02:02 STEP 1762 H= 440:30 +CPU= 36.592 14:02:40 STEP 1763 H= 440:45 +CPU= 36.620 14:03:28 STEP 1764 H= 441:00 +CPU= 45.933 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001764 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001764 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001764 14:04:06 STEP 1765 H= 441:15 +CPU= 36.545 14:04:44 STEP 1766 H= 441:30 +CPU= 36.579 14:05:23 STEP 1767 H= 441:45 +CPU= 36.584 14:06:02 STEP 1768 H= 442:00 +CPU= 37.495 14:06:41 STEP 1769 H= 442:15 +CPU= 36.896 14:07:20 STEP 1770 H= 442:30 +CPU= 37.182 14:07:59 STEP 1771 H= 442:45 +CPU= 37.096 14:08:38 STEP 1772 H= 443:00 +CPU= 37.541 14:09:16 STEP 1773 H= 443:15 +CPU= 36.641 14:09:54 STEP 1774 H= 443:30 +CPU= 36.476 14:10:33 STEP 1775 H= 443:45 +CPU= 36.600 14:11:24 STEP 1776 H= 444:00 +CPU= 49.487 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001776 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001776 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001776 14:12:03 STEP 1777 H= 444:15 +CPU= 36.653 14:12:41 STEP 1778 H= 444:30 +CPU= 36.552 14:13:19 STEP 1779 H= 444:45 +CPU= 36.576 14:13:57 STEP 1780 H= 445:00 +CPU= 37.402 14:14:35 STEP 1781 H= 445:15 +CPU= 36.586 14:15:14 STEP 1782 H= 445:30 +CPU= 36.644 14:15:52 STEP 1783 H= 445:45 +CPU= 36.547 14:16:31 STEP 1784 H= 446:00 +CPU= 37.357 14:17:09 STEP 1785 H= 446:15 +CPU= 36.596 14:17:47 STEP 1786 H= 446:30 +CPU= 36.657 14:18:26 STEP 1787 H= 446:45 +CPU= 36.670 14:19:13 STEP 1788 H= 447:00 +CPU= 45.956 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001788 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001788 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001788 14:19:51 STEP 1789 H= 447:15 +CPU= 36.589 14:20:31 STEP 1790 H= 447:30 +CPU= 36.682 14:21:09 STEP 1791 H= 447:45 +CPU= 36.560 14:21:47 STEP 1792 H= 448:00 +CPU= 37.371 14:22:25 STEP 1793 H= 448:15 +CPU= 36.642 14:23:03 STEP 1794 H= 448:30 +CPU= 36.614 14:23:42 STEP 1795 H= 448:45 +CPU= 36.558 14:24:21 STEP 1796 H= 449:00 +CPU= 37.405 14:24:59 STEP 1797 H= 449:15 +CPU= 36.601 14:25:38 STEP 1798 H= 449:30 +CPU= 36.667 14:26:16 STEP 1799 H= 449:45 +CPU= 36.647 14:27:04 STEP 1800 H= 450:00 +CPU= 45.981 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001800 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001800 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001800 14:27:42 STEP 1801 H= 450:15 +CPU= 36.664 14:28:20 STEP 1802 H= 450:30 +CPU= 36.599 14:28:58 STEP 1803 H= 450:45 +CPU= 36.637 14:29:37 STEP 1804 H= 451:00 +CPU= 37.373 14:30:16 STEP 1805 H= 451:15 +CPU= 36.691 14:30:54 STEP 1806 H= 451:30 +CPU= 36.658 14:31:33 STEP 1807 H= 451:45 +CPU= 36.697 14:32:12 STEP 1808 H= 452:00 +CPU= 37.485 14:32:50 STEP 1809 H= 452:15 +CPU= 36.641 14:33:28 STEP 1810 H= 452:30 +CPU= 36.641 14:34:06 STEP 1811 H= 452:45 +CPU= 36.622 14:34:54 STEP 1812 H= 453:00 +CPU= 45.938 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001812 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001812 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001812 14:35:41 STEP 1813 H= 453:15 +CPU= 37.179 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:36:40 STEP 1814 H= 453:30 +CPU= 36.367 14:37:18 STEP 1815 H= 453:45 +CPU= 36.614 14:37:57 STEP 1816 H= 454:00 +CPU= 37.431 14:38:36 STEP 1817 H= 454:15 +CPU= 36.620 14:39:14 STEP 1818 H= 454:30 +CPU= 36.700 14:39:52 STEP 1819 H= 454:45 +CPU= 36.647 14:40:32 STEP 1820 H= 455:00 +CPU= 37.448 14:41:10 STEP 1821 H= 455:15 +CPU= 36.542 14:41:48 STEP 1822 H= 455:30 +CPU= 36.639 14:42:26 STEP 1823 H= 455:45 +CPU= 36.657 14:43:18 STEP 1824 H= 456:00 +CPU= 49.636 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001824 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001824 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001824 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001728 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001728 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001728 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001740 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001740 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001740 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001752 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001752 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001752 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001764 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001764 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001764 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001776 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001776 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001776 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001788 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001788 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001788 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001800 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001800 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001800 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001812 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001812 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001812 Zipping up the intermediate file: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1_r1823311562_18.zip Uploading the intermediate file: upload_file_18.zip 14:44:00 STEP 1825 H= 456:15 +CPU= 36.602 14:44:38 STEP 1826 H= 456:30 +CPU= 36.772 14:45:17 STEP 1827 H= 456:45 +CPU= 36.695 14:45:56 STEP 1828 H= 457:00 +CPU= 37.330 14:46:35 STEP 1829 H= 457:15 +CPU= 36.670 14:47:13 STEP 1830 H= 457:30 +CPU= 36.679 14:47:51 STEP 1831 H= 457:45 +CPU= 36.574 14:48:30 STEP 1832 H= 458:00 +CPU= 37.414 14:49:08 STEP 1833 H= 458:15 +CPU= 36.640 14:49:46 STEP 1834 H= 458:30 +CPU= 36.677 14:50:25 STEP 1835 H= 458:45 +CPU= 36.723 14:51:13 STEP 1836 H= 459:00 +CPU= 45.926 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001836 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001836 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001836 14:51:51 STEP 1837 H= 459:15 +CPU= 36.709 14:52:29 STEP 1838 H= 459:30 +CPU= 36.608 14:53:08 STEP 1839 H= 459:45 +CPU= 36.645 14:53:47 STEP 1840 H= 460:00 +CPU= 37.411 14:54:25 STEP 1841 H= 460:15 +CPU= 36.645 14:55:04 STEP 1842 H= 460:30 +CPU= 36.621 14:55:42 STEP 1843 H= 460:45 +CPU= 36.656 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:59:24 STEP 1844 H= 461:00 +CPU= 37.566 15:00:05 STEP 1845 H= 461:15 +CPU= 37.001 15:00:46 STEP 1846 H= 461:30 +CPU= 36.972 15:01:25 STEP 1847 H= 461:45 +CPU= 36.784 15:02:13 STEP 1848 H= 462:00 +CPU= 45.985 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001848 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001848 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001848 15:02:51 STEP 1849 H= 462:15 +CPU= 36.667 15:03:29 STEP 1850 H= 462:30 +CPU= 36.651 15:04:07 STEP 1851 H= 462:45 +CPU= 36.160 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:05:07 STEP 1852 H= 463:00 +CPU= 37.007 15:05:45 STEP 1853 H= 463:15 +CPU= 36.606 15:06:23 STEP 1854 H= 463:30 +CPU= 36.624 15:07:01 STEP 1855 H= 463:45 +CPU= 36.652 15:07:40 STEP 1856 H= 464:00 +CPU= 37.381 15:08:18 STEP 1857 H= 464:15 +CPU= 36.625 15:08:56 STEP 1858 H= 464:30 +CPU= 36.608 15:09:35 STEP 1859 H= 464:45 +CPU= 36.609 15:10:23 STEP 1860 H= 465:00 +CPU= 45.952 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001860 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001860 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001860 15:11:02 STEP 1861 H= 465:15 +CPU= 36.643 15:11:39 STEP 1862 H= 465:30 +CPU= 36.578 15:12:18 STEP 1863 H= 465:45 +CPU= 36.643 15:12:57 STEP 1864 H= 466:00 +CPU= 37.446 15:13:35 STEP 1865 H= 466:15 +CPU= 36.618 15:14:13 STEP 1866 H= 466:30 +CPU= 36.760 15:14:51 STEP 1867 H= 466:45 +CPU= 36.586 15:15:31 STEP 1868 H= 467:00 +CPU= 37.411 15:16:09 STEP 1869 H= 467:15 +CPU= 36.595 15:16:47 STEP 1870 H= 467:30 +CPU= 36.611 15:17:25 STEP 1871 H= 467:45 +CPU= 36.675 15:18:16 STEP 1872 H= 468:00 +CPU= 49.530 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001872 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001872 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001872 15:18:54 STEP 1873 H= 468:15 +CPU= 36.684 15:19:33 STEP 1874 H= 468:30 +CPU= 36.703 15:20:06 STEP 1875 H= 468:45 +CPU= 31.593 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:20:59 STEP 1876 H= 469:00 +CPU= 29.403 15:21:28 STEP 1877 H= 469:15 +CPU= 28.469 15:21:59 STEP 1878 H= 469:30 +CPU= 29.333 15:22:31 STEP 1879 H= 469:45 +CPU= 30.020 15:23:01 STEP 1880 H= 470:00 +CPU= 29.365 15:23:32 STEP 1881 H= 470:15 +CPU= 28.781 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:24:49 STEP 1882 H= 470:30 +CPU= 31.824 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:25:41 STEP 1883 H= 470:45 +CPU= 29.589 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:26:49 STEP 1884 H= 471:00 +CPU= 40.845 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001884 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001884 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001884 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:27:47 STEP 1885 H= 471:15 +CPU= 31.523 15:28:17 STEP 1886 H= 471:30 +CPU= 28.357 15:28:48 STEP 1887 H= 471:45 +CPU= 29.434 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.518] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3536MB (heap), 4796MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.518] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 3536 MB (maxheap), 4796 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CPG_DRV [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : >OMP-CPG (1025) [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CPG [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CPG_GP [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GPXX [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152850:1719926930:8755.521] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : VERDER [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152851:1719926931:8756.522] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:124350:124350] [20240702:152851:1719926931:8756.522] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CF935B Unknown Unknown Unknown oifs_43r3_model.e 00000000008214C1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DD5490 Unknown Unknown Unknown oifs_43r3_model.e 0000000001783F91 Unknown Unknown Unknown oifs_43r3_model.e 0000000000C7047C verder_ 106 verder.F90 oifs_43r3_model.e 0000000000D25971 gpxx_ 162 gpxx.F90 oifs_43r3_model.e 0000000000D18EA1 cpg_gp_ 923 cpg_gp.F90 oifs_43r3_model.e 0000000000C2432E cpg_ 627 cpg.F90 oifs_43r3_model.e 0000000000C22238 cpg_drv_ 351 cpg_drv.F90 oifs_43r3_model.e 0000000000BCC914 gp_model_ 419 gp_model.F90 oifs_43r3_model.e 00000000012C5002 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BACBAE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 0000000000688153 stepo_ 327 stepo.F90 oifs_43r3_model.e 00000000004CD1DC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000004B5D85 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 00000000004B4F52 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 000000000049D638 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 0000000000406FBD cnt0_ 146 cnt0.F90 oifs_43r3_model.e 0000000000401E7F MAIN__ 96 master.F90 oifs_43r3_model.e 0000000000401E12 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCC34A Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCDBE7 Unknown Unknown Unknown oifs_43r3_model.e 0000000000401CD5 Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a4il (argv4) batchid: 1018 (argv5) wuid: 12294180 (argv6) fclen: 20 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /mnt/WDred/boinc/slots/1 Project directory is: /mnt/WDred/boinc/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.13 Location of temp folder: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180 ..mkdir for temp folder for results failed Copying: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_46114a206059e4bf26792c7816dc4100 to: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_a4il_2016092300_20_1018_12294180.zip Unzipping the namelist zip file: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_a4il_2016092300_20_1018_12294180.zip ic_ancil_file: ic_ancil_12294180 ifsdata_file: ifsdata_12294180 climate_data_file: clim_data_12294180 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /mnt/WDred/boinc/slots/1/ic_ancil_12294180.zip Unzipping the IC ancils zip file: /mnt/WDred/boinc/slots/1/ic_ancil_12294180.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7beb3f16a3760c83bc16c85ffa32515e to: /mnt/WDred/boinc/slots/1/ifsdata/ifsdata_12294180.zip Unzipping the ifsdata_zip file: /mnt/WDred/boinc/slots/1/ifsdata/ifsdata_12294180.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/WDred/boinc/slots/1/159l_2/clim_data_12294180.zip Unzipping the climate data zip file: /mnt/WDred/boinc/slots/1/159l_2/clim_data_12294180.zip Checking for progress XML file: /mnt/WDred/boinc/slots/1/progress_file_12294180.xml Opened progress file ok : /mnt/WDred/boinc/slots/1/progress_file_12294180.xml -- Model is restarting -- Adjusting last_iter, 1887, to previous model restart step. Creating progress file: /mnt/WDred/boinc/slots/1/progress_file_12294180.xml last_cpu_time: 76259 upload_file_number: 19 last_iter: 1873 last_upload: 1641600 model_completed: 0 total_length_of_simulation: 1728000 result_base_name: oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1_r1823311562 The child process has been launched with process id: 7080 Executing the command: /mnt/WDred/boinc/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153016:1719927016:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:31:25 STEP 1872 H= 468:00 +CPU= 27.817 15:31:52 STEP 1873 H= 468:15 +CPU= 26.749 15:32:19 STEP 1874 H= 468:30 +CPU= 26.657 15:32:46 STEP 1875 H= 468:45 +CPU= 26.341 15:33:14 STEP 1876 H= 469:00 +CPU= 27.230 15:33:41 STEP 1877 H= 469:15 +CPU= 27.030 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:36:17 STEP 1878 H= 469:30 +CPU= 31.147 15:36:46 STEP 1879 H= 469:45 +CPU= 27.592 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:37:38 STEP 1880 H= 470:00 +CPU= 29.793 15:38:07 STEP 1881 H= 470:15 +CPU= 28.083 15:38:37 STEP 1882 H= 470:30 +CPU= 28.816 Suspend request received from the BOINC client, suspending the child process Resuming the child process double free or corruption (top) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3933MB (heap), 4749MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 3933 MB (maxheap), 4749 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001) [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153924:1719927564:548.002] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CLOUDSC [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153925:1719927565:549.002] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:svarten.tribun:1:1:7080:7080] [20240702:153925:1719927565:549.003] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CF935B Unknown Unknown Unknown oifs_43r3_model.e 00000000008214C1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DD5490 Unknown Unknown Unknown oifs_43r3_model.e 000000000116F1C2 cloudsc_ 1970 cloudsc.F90 oifs_43r3_model.e 000000000115C847 cloud_layer_ 131 cloud_layer.F90 oifs_43r3_model.e 0000000000FBB064 callpar_ 1223 callpar.F90 oifs_43r3_model.e 0000000000F5FA6B ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000F4DBC4 ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BD7131 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012C5002 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BACBAE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 0000000000688153 stepo_ 327 stepo.F90 oifs_43r3_model.e 00000000004CD1DC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000004B5D85 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 00000000004B4F52 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 000000000049D638 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 0000000000406FBD cnt0_ 146 cnt0.F90 oifs_43r3_model.e 0000000000401E7F MAIN__ 96 master.F90 oifs_43r3_model.e 0000000000401E12 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCC34A Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCDBE7 Unknown Unknown Unknown oifs_43r3_model.e 0000000000401CD5 Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a4il (argv4) batchid: 1018 (argv5) wuid: 12294180 (argv6) fclen: 20 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /mnt/WDred/boinc/slots/1 Project directory is: /mnt/WDred/boinc/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.13 Location of temp folder: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180 ..mkdir for temp folder for results failed Copying: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_46114a206059e4bf26792c7816dc4100 to: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_a4il_2016092300_20_1018_12294180.zip Unzipping the namelist zip file: /mnt/WDred/boinc/slots/1/oifs_43r3_bl_a4il_2016092300_20_1018_12294180.zip ic_ancil_file: ic_ancil_12294180 ifsdata_file: ifsdata_12294180 climate_data_file: clim_data_12294180 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /mnt/WDred/boinc/slots/1/ic_ancil_12294180.zip Unzipping the IC ancils zip file: /mnt/WDred/boinc/slots/1/ic_ancil_12294180.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7beb3f16a3760c83bc16c85ffa32515e to: /mnt/WDred/boinc/slots/1/ifsdata/ifsdata_12294180.zip Unzipping the ifsdata_zip file: /mnt/WDred/boinc/slots/1/ifsdata/ifsdata_12294180.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/WDred/boinc/slots/1/159l_2/clim_data_12294180.zip Unzipping the climate data zip file: /mnt/WDred/boinc/slots/1/159l_2/clim_data_12294180.zip Checking for progress XML file: /mnt/WDred/boinc/slots/1/progress_file_12294180.xml Opened progress file ok : /mnt/WDred/boinc/slots/1/progress_file_12294180.xml -- Model is restarting -- Adjusting last_iter, 1883, to previous model restart step. Creating progress file: /mnt/WDred/boinc/slots/1/progress_file_12294180.xml last_cpu_time: 76589 upload_file_number: 19 last_iter: 1873 last_upload: 1641600 model_completed: 0 total_length_of_simulation: 1728000 result_base_name: oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1_r1823311562 The child process has been launched with process id: 7587 Executing the command: /mnt/WDred/boinc/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:svarten.tribun:1:1:7587:7587] [20240702:155706:1719928626:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:58:53 STEP 1872 H= 468:00 +CPU= 28.205 15:59:21 STEP 1873 H= 468:15 +CPU= 27.142 15:59:49 STEP 1874 H= 468:30 +CPU= 27.654 16:00:19 STEP 1875 H= 468:45 +CPU= 29.382 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:01:40 STEP 1876 H= 469:00 +CPU= 31.236 16:02:10 STEP 1877 H= 469:15 +CPU= 28.628 16:02:39 STEP 1878 H= 469:30 +CPU= 27.894 16:03:06 STEP 1879 H= 469:45 +CPU= 27.013 16:03:35 STEP 1880 H= 470:00 +CPU= 27.917 16:04:03 STEP 1881 H= 470:15 +CPU= 27.229 16:04:32 STEP 1882 H= 470:30 +CPU= 27.971 16:05:03 STEP 1883 H= 470:45 +CPU= 29.828 16:05:39 STEP 1884 H= 471:00 +CPU= 34.434 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001884 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001884 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001884 16:06:06 STEP 1885 H= 471:15 +CPU= 27.049 16:06:34 STEP 1886 H= 471:30 +CPU= 27.173 16:07:04 STEP 1887 H= 471:45 +CPU= 28.899 16:07:37 STEP 1888 H= 472:00 +CPU= 29.969 16:08:07 STEP 1889 H= 472:15 +CPU= 28.540 16:08:37 STEP 1890 H= 472:30 +CPU= 28.614 16:09:08 STEP 1891 H= 472:45 +CPU= 29.139 16:09:39 STEP 1892 H= 473:00 +CPU= 29.653 16:10:10 STEP 1893 H= 473:15 +CPU= 29.080 16:10:41 STEP 1894 H= 473:30 +CPU= 29.134 16:11:10 STEP 1895 H= 473:45 +CPU= 27.813 16:11:49 STEP 1896 H= 474:00 +CPU= 36.338 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001896 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001896 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001896 16:12:18 STEP 1897 H= 474:15 +CPU= 28.552 16:12:49 STEP 1898 H= 474:30 +CPU= 29.181 16:13:20 STEP 1899 H= 474:45 +CPU= 28.555 16:13:50 STEP 1900 H= 475:00 +CPU= 29.378 16:14:21 STEP 1901 H= 475:15 +CPU= 28.842 16:14:50 STEP 1902 H= 475:30 +CPU= 28.288 16:15:20 STEP 1903 H= 475:45 +CPU= 28.135 16:15:50 STEP 1904 H= 476:00 +CPU= 29.137 16:16:21 STEP 1905 H= 476:15 +CPU= 29.409 16:16:52 STEP 1906 H= 476:30 +CPU= 28.904 16:17:25 STEP 1907 H= 476:45 +CPU= 29.761 16:18:01 STEP 1908 H= 477:00 +CPU= 34.985 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001908 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001908 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001908 16:18:30 STEP 1909 H= 477:15 +CPU= 28.080 16:19:00 STEP 1910 H= 477:30 +CPU= 28.148 16:19:28 STEP 1911 H= 477:45 +CPU= 27.627 16:19:58 STEP 1912 H= 478:00 +CPU= 29.047 16:20:29 STEP 1913 H= 478:15 +CPU= 28.888 16:20:59 STEP 1914 H= 478:30 +CPU= 28.408 16:21:28 STEP 1915 H= 478:45 +CPU= 27.808 16:22:00 STEP 1916 H= 479:00 +CPU= 30.053 16:22:32 STEP 1917 H= 479:15 +CPU= 29.503 16:23:02 STEP 1918 H= 479:30 +CPU= 28.502 16:23:30 STEP 1919 H= 479:45 +CPU= 27.799 16:24:09 STEP 1920 H= 480:00 +CPU= 35.555 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 DDH------------------------------------------------------------------------ I I I TIME = 480.000H DDH * Diagnostic Var., AREA GLOBAL I I I I-------------------------------------------------------------------------I I VARIABLE I Initial value I Current value I I-------------------------------------------------------------------------I I OMEGA (P/S) I 0.1704779057E-21 I 0.5137918827E-05 I ICloud fractionI 0.0000000000E+00 I 0.7547029314E-01 I I Relat. Hum. I 0.1865626431E+00 I 0.3576947473E+00 I I PASS 01 I 0.0000000000E+00 I 0.8770310322E+03 I DDH------------------------------------------------------------------------ MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 1920 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+001920 MODE=a IO-STREAM CLOSED - ICMGGh7zg+001920 MPI-TASK: 1 - 327213092 BYTES IN 7 RECORDS TRANSFERRED IN 0.0003 SECONDS******** Mbytes/s, TOTAL TIME= 0.0115( 2.6%) 16:24:09 STEP 1920 H= 480:00 +CPU= 35.555 END CNT3 NSTEP = 1920 CNT0 000000000 26.508 26.508 27.407 0.967 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 46/ 1069125708 WRITTEN 1430/************ -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 28 0 46 1044068 1430 ************ 15.8 13.8 2.0 14.6 4.9 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 1622876.2 1622876.2 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 1544331.2 1544331.2 95.16 0.00 8 SCAN2M - GRID-POINT DYNAMICS 53 4145.3 4145.3 13.54 0.00 9 SPCM - SPECTRAL COMP. 48 889.7 889.7 2.63 0.00 10 SCAN2M - PHYSICS 49 16703.9 16703.9 50.43 0.00 11 IOPACK - OUTPUT P.P. RESULTS 4 1035.3 1035.3 0.26 0.00 12 SPNORM - SPECTRAL NORM COMP. 3 52.3 52.3 0.01 0.00 17 GRIDFPOS IN CNT4 4 28.9 28.9 0.01 0.00 27 RESTART FILES - WRITING 1 4141.0 4141.0 0.26 0.00 28 RESTART FILES - READING 1 15314.5 15314.5 0.94 0.00 29 SU4FPOS IN CNT4 4 0.0 0.0 0.00 0.00 30 DYNFPOS IN CNT4 4 6645.6 6645.6 1.64 0.00 31 POSDDH IN STEPO 1 0.8 0.8 0.00 0.00 37 CPGLAG - SL COMPUTATIONS 49 6077.4 6077.4 18.35 0.00 39 SU0YOMB 1 62675.2 62675.2 3.86 0.00 51 SCAN2M - SL COMM. PART 1 49 30.0 30.0 0.09 0.00 54 SPCM - M TO S/S TO M TRANSP. 48 312.2 312.2 0.92 0.00 55 SPCIMPF - S TO M/M TO S TRANSP. 48 49.7 49.7 0.15 0.00 56 SPNORM - SPECTRAL NORM COMM. 3 0.2 0.2 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 108 426.9 426.9 2.84 0.00 103 LTDIR_CTL - DIRECT LEGENDRE TRANSFORM 100 269.1 269.1 1.66 0.00 106 FTDIR_CTL - DIRECT FOURIER TRANSFORM 100 96.2 96.2 0.59 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 108 159.1 159.1 1.06 0.00 140 SULEG - COMP. OF LEGENDRE POL. 1 136.7 136.7 0.01 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 108 46.9 46.9 0.31 0.00 153 LTDIR_CTL - L TO M TRANSPOSITION 100 27.2 27.2 0.17 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 108 82.1 82.1 0.55 0.00 158 FTDIR_CTL - G TO L TRANSPOSITION 100 31.4 31.4 0.19 0.00 400 GSTATS 6667 0.0 0.0 0.00 0.00 401 GSTATS HOOK 6149 0.0 0.0 0.00 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 1622.9 CPU TIME 1432.2 VECTOR TIME 1432.2 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 1118.9 *** END CNT0 *** ------------------------------------------------ Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMGGh7zg+001920 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMSHh7zg+001920 Moving to projects directory: /mnt/WDred/boinc/slots/1/ICMUAh7zg+001920 Adding to the zip: /mnt/WDred/boinc/slots/1/NODE.001_01 Adding to the zip: /mnt/WDred/boinc/slots/1/ifs.stat Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001848 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001884 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001860 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001824 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001908 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001884 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001824 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001896 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001836 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001860 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001848 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001920 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001872 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001872 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001836 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001824 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001860 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001836 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001884 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001920 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001848 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMUAh7zg+001896 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001908 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001872 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001920 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMSHh7zg+001896 Adding to the zip: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_12294180/ICMGGh7zg+001908 Zipping up the final file: /mnt/WDred/boinc/projects/climateprediction.net/oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1_r1823311562_19.zip Uploading the final file: upload_file_19.zip Uploading trickle at timestep: 1727100 16:27:42 (7087): called boinc_finish(0) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
02 Jul 2024 14:58:52 | 1496581 | 22457277 | oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1 | 1,727,100 | 78,021 | 0.0452 |
26 Jun 2024 14:15:23 | 1496581 | 22457277 | oifs_43r3_bl_a4il_2016092300_20_1018_12294180_1 | 1,036,800 | 42,207 | 0.0407 |
©2024 cpdn.org