climateprediction.net home page
Task 22287360

Task 22287360

Name oifs_43r3_ps_0444_2015050100_123_984_12201088_0
Workunit 12201088
Created 24 Dec 2022, 12:48:17 UTC
Sent 24 Dec 2022, 13:00:57 UTC
Report deadline 23 Jan 2023, 13:00:57 UTC
Received 25 Dec 2022, 4:22:22 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1478928
Run time 12 hours 19 min 26 sec
CPU time 12 hours 14 min 53 sec
Validate state Invalid
Credit 9,918.72
Device peak FLOPS 8.02 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,770.56 MB
Peak swap size 4,978.17 MB
Peak disk usage 7,351.90 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
8:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1560275:1560275] [20221224:212308:1671931388:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:23:34 STEP 1752 H=1752:00 +CPU= 23.510
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001740
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_72.zip
Uploading the intermediate file: upload_file_72.zip
  21:23:47 STEP 1753 H=1753:00 +CPU= 13.088
  21:24:01 STEP 1754 H=1754:00 +CPU= 13.633
  21:24:21 STEP 1755 H=1755:00 +CPU= 20.375
  21:24:36 STEP 1756 H=1756:00 +CPU= 14.333
  21:24:49 STEP 1757 H=1757:00 +CPU= 12.983
  21:25:10 STEP 1758 H=1758:00 +CPU= 20.522
  21:25:23 STEP 1759 H=1759:00 +CPU= 13.251
  21:25:37 STEP 1760 H=1760:00 +CPU= 13.834
  21:25:58 STEP 1761 H=1761:00 +CPU= 21.056
  21:26:12 STEP 1762 H=1762:00 +CPU= 13.475
  21:26:25 STEP 1763 H=1763:00 +CPU= 13.492
  21:26:47 STEP 1764 H=1764:00 +CPU= 22.152
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001764
  21:27:01 STEP 1765 H=1765:00 +CPU= 13.180
  21:27:14 STEP 1766 H=1766:00 +CPU= 13.540
  21:27:35 STEP 1767 H=1767:00 +CPU= 20.473
  21:27:48 STEP 1768 H=1768:00 +CPU= 13.140
  21:28:02 STEP 1769 H=1769:00 +CPU= 13.337
  21:28:22 STEP 1770 H=1770:00 +CPU= 20.732
  21:28:36 STEP 1771 H=1771:00 +CPU= 13.409
  21:28:49 STEP 1772 H=1772:00 +CPU= 13.592
  21:29:11 STEP 1773 H=1773:00 +CPU= 21.430
  21:29:24 STEP 1774 H=1774:00 +CPU= 13.037
  21:29:38 STEP 1775 H=1775:00 +CPU= 13.826
  21:30:01 STEP 1776 H=1776:00 +CPU= 23.427
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001764
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_73.zip
Uploading the intermediate file: upload_file_73.zip
  21:30:15 STEP 1777 H=1777:00 +CPU= 13.659
  21:30:29 STEP 1778 H=1778:00 +CPU= 13.833
  21:30:49 STEP 1779 H=1779:00 +CPU= 20.317
  21:31:03 STEP 1780 H=1780:00 +CPU= 13.583
  21:31:16 STEP 1781 H=1781:00 +CPU= 13.400
  21:31:38 STEP 1782 H=1782:00 +CPU= 21.367
  21:31:51 STEP 1783 H=1783:00 +CPU= 13.814
  21:32:05 STEP 1784 H=1784:00 +CPU= 13.760
  21:32:26 STEP 1785 H=1785:00 +CPU= 20.529
  21:32:39 STEP 1786 H=1786:00 +CPU= 13.334
  21:32:53 STEP 1787 H=1787:00 +CPU= 13.637
  21:33:17 STEP 1788 H=1788:00 +CPU= 24.204
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001788
  21:33:31 STEP 1789 H=1789:00 +CPU= 14.008
  21:33:46 STEP 1790 H=1790:00 +CPU= 14.558
  21:34:06 STEP 1791 H=1791:00 +CPU= 20.659
  21:34:20 STEP 1792 H=1792:00 +CPU= 13.579
  21:34:34 STEP 1793 H=1793:00 +CPU= 13.747
  21:34:56 STEP 1794 H=1794:00 +CPU= 21.850
  21:35:10 STEP 1795 H=1795:00 +CPU= 14.485
  21:35:24 STEP 1796 H=1796:00 +CPU= 13.488
  21:35:44 STEP 1797 H=1797:00 +CPU= 20.197
  21:35:57 STEP 1798 H=1798:00 +CPU= 13.368
  21:36:11 STEP 1799 H=1799:00 +CPU= 13.793
  21:36:34 STEP 1800 H=1800:00 +CPU= 22.515
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001788
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_74.zip
Uploading the intermediate file: upload_file_74.zip
  21:36:47 STEP 1801 H=1801:00 +CPU= 13.026
  21:37:00 STEP 1802 H=1802:00 +CPU= 13.567
  21:37:21 STEP 1803 H=1803:00 +CPU= 20.916
  21:37:36 STEP 1804 H=1804:00 +CPU= 14.413
  21:37:49 STEP 1805 H=1805:00 +CPU= 13.569
  21:38:10 STEP 1806 H=1806:00 +CPU= 20.402
  21:38:23 STEP 1807 H=1807:00 +CPU= 12.999
  21:38:37 STEP 1808 H=1808:00 +CPU= 13.940
  21:38:58 STEP 1809 H=1809:00 +CPU= 21.119
  21:39:12 STEP 1810 H=1810:00 +CPU= 13.606
  21:39:25 STEP 1811 H=1811:00 +CPU= 13.539
  21:39:48 STEP 1812 H=1812:00 +CPU= 22.547
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001812
  21:40:03 STEP 1813 H=1813:00 +CPU= 14.742
  21:40:17 STEP 1814 H=1814:00 +CPU= 14.390
  21:40:38 STEP 1815 H=1815:00 +CPU= 20.380
  21:40:51 STEP 1816 H=1816:00 +CPU= 13.058
  21:41:04 STEP 1817 H=1817:00 +CPU= 13.186
  21:41:24 STEP 1818 H=1818:00 +CPU= 20.537
  21:41:38 STEP 1819 H=1819:00 +CPU= 13.833
  21:41:53 STEP 1820 H=1820:00 +CPU= 14.753
  21:42:13 STEP 1821 H=1821:00 +CPU= 20.080
  21:42:26 STEP 1822 H=1822:00 +CPU= 13.171
  21:42:40 STEP 1823 H=1823:00 +CPU= 13.669
  21:43:07 STEP 1824 H=1824:00 +CPU= 24.435
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_75.zip
Uploading the intermediate file: upload_file_75.zip
  21:43:21 STEP 1825 H=1825:00 +CPU= 14.476
  21:43:37 STEP 1826 H=1826:00 +CPU= 15.119
  21:43:58 STEP 1827 H=1827:00 +CPU= 21.317
  21:44:11 STEP 1828 H=1828:00 +CPU= 13.326
  21:44:24 STEP 1829 H=1829:00 +CPU= 12.923
  21:44:45 STEP 1830 H=1830:00 +CPU= 20.912
  21:44:59 STEP 1831 H=1831:00 +CPU= 13.948
  21:45:13 STEP 1832 H=1832:00 +CPU= 13.730
  21:45:34 STEP 1833 H=1833:00 +CPU= 21.338
  21:45:48 STEP 1834 H=1834:00 +CPU= 13.391
  21:46:01 STEP 1835 H=1835:00 +CPU= 12.770
  21:46:24 STEP 1836 H=1836:00 +CPU= 23.654
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001836
  21:46:39 STEP 1837 H=1837:00 +CPU= 14.579
  21:46:54 STEP 1838 H=1838:00 +CPU= 15.089
  21:47:16 STEP 1839 H=1839:00 +CPU= 21.478
  21:47:29 STEP 1840 H=1840:00 +CPU= 13.493
  21:47:42 STEP 1841 H=1841:00 +CPU= 13.335
  21:48:07 STEP 1842 H=1842:00 +CPU= 21.920
  21:48:21 STEP 1843 H=1843:00 +CPU= 14.159
  21:48:35 STEP 1844 H=1844:00 +CPU= 13.996
  21:48:56 STEP 1845 H=1845:00 +CPU= 20.157
  21:49:10 STEP 1846 H=1846:00 +CPU= 13.903
  21:49:24 STEP 1847 H=1847:00 +CPU= 14.809
  21:49:47 STEP 1848 H=1848:00 +CPU= 22.891
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_76.zip
Uploading the intermediate file: upload_file_76.zip
  21:49:59 STEP 1849 H=1849:00 +CPU= 11.733
  21:50:11 STEP 1850 H=1850:00 +CPU= 12.119
  21:50:30 STEP 1851 H=1851:00 +CPU= 18.969
  21:50:44 STEP 1852 H=1852:00 +CPU= 14.188
  21:50:59 STEP 1853 H=1853:00 +CPU= 14.634
  21:51:21 STEP 1854 H=1854:00 +CPU= 21.378
  21:51:34 STEP 1855 H=1855:00 +CPU= 13.108
  21:51:47 STEP 1856 H=1856:00 +CPU= 13.513
  21:52:07 STEP 1857 H=1857:00 +CPU= 20.044
  21:52:21 STEP 1858 H=1858:00 +CPU= 13.901
  21:52:35 STEP 1859 H=1859:00 +CPU= 13.235
  21:52:58 STEP 1860 H=1860:00 +CPU= 22.904
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001860
  21:53:11 STEP 1861 H=1861:00 +CPU= 13.258
  21:53:25 STEP 1862 H=1862:00 +CPU= 14.411
  21:53:48 STEP 1863 H=1863:00 +CPU= 20.952
  21:54:01 STEP 1864 H=1864:00 +CPU= 13.702
  21:54:15 STEP 1865 H=1865:00 +CPU= 13.676
  21:54:37 STEP 1866 H=1866:00 +CPU= 21.872
  21:54:51 STEP 1867 H=1867:00 +CPU= 13.654
  21:55:04 STEP 1868 H=1868:00 +CPU= 13.307
  21:55:24 STEP 1869 H=1869:00 +CPU= 19.887
  21:55:38 STEP 1870 H=1870:00 +CPU= 13.651
  21:55:52 STEP 1871 H=1871:00 +CPU= 14.237
  21:56:21 STEP 1872 H=1872:00 +CPU= 24.614
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_77.zip
Uploading the intermediate file: upload_file_77.zip
  21:56:34 STEP 1873 H=1873:00 +CPU= 13.225
  21:56:47 STEP 1874 H=1874:00 +CPU= 13.174
  21:57:07 STEP 1875 H=1875:00 +CPU= 19.840
  21:57:20 STEP 1876 H=1876:00 +CPU= 13.195
  21:57:36 STEP 1877 H=1877:00 +CPU= 12.946
  21:57:57 STEP 1878 H=1878:00 +CPU= 20.594
  21:58:10 STEP 1879 H=1879:00 +CPU= 13.340
  21:58:31 STEP 1880 H=1880:00 +CPU= 14.450
  21:58:52 STEP 1881 H=1881:00 +CPU= 21.312
  21:59:06 STEP 1882 H=1882:00 +CPU= 13.663
  21:59:20 STEP 1883 H=1883:00 +CPU= 13.556
  21:59:42 STEP 1884 H=1884:00 +CPU= 22.311
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001884
  21:59:57 STEP 1885 H=1885:00 +CPU= 14.570
  22:00:12 STEP 1886 H=1886:00 +CPU= 14.785
  22:00:32 STEP 1887 H=1887:00 +CPU= 19.829
  22:00:45 STEP 1888 H=1888:00 +CPU= 13.052
  22:00:58 STEP 1889 H=1889:00 +CPU= 12.952
  22:01:20 STEP 1890 H=1890:00 +CPU= 21.811
  22:01:34 STEP 1891 H=1891:00 +CPU= 13.634
  22:01:49 STEP 1892 H=1892:00 +CPU= 15.017
  22:02:10 STEP 1893 H=1893:00 +CPU= 21.290
  22:02:24 STEP 1894 H=1894:00 +CPU= 13.868
  22:02:38 STEP 1895 H=1895:00 +CPU= 13.809
  22:03:02 STEP 1896 H=1896:00 +CPU= 23.567
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_78.zip
Uploading the intermediate file: upload_file_78.zip
  22:03:16 STEP 1897 H=1897:00 +CPU= 14.393
  22:03:30 STEP 1898 H=1898:00 +CPU= 13.385
  22:03:50 STEP 1899 H=1899:00 +CPU= 19.998
  22:04:04 STEP 1900 H=1900:00 +CPU= 13.751
  22:04:17 STEP 1901 H=1901:00 +CPU= 13.352
  22:04:38 STEP 1902 H=1902:00 +CPU= 21.187
  22:04:52 STEP 1903 H=1903:00 +CPU= 13.494
  22:05:05 STEP 1904 H=1904:00 +CPU= 13.607
  22:05:26 STEP 1905 H=1905:00 +CPU= 20.177
  22:05:39 STEP 1906 H=1906:00 +CPU= 13.362
  22:05:53 STEP 1907 H=1907:00 +CPU= 14.110
  22:06:17 STEP 1908 H=1908:00 +CPU= 23.562
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001908
  22:06:30 STEP 1909 H=1909:00 +CPU= 13.338
  22:06:45 STEP 1910 H=1910:00 +CPU= 14.395
  22:07:05 STEP 1911 H=1911:00 +CPU= 20.681
  22:07:19 STEP 1912 H=1912:00 +CPU= 13.480
  22:07:32 STEP 1913 H=1913:00 +CPU= 12.926
  22:07:52 STEP 1914 H=1914:00 +CPU= 20.585
  22:08:06 STEP 1915 H=1915:00 +CPU= 14.114
  22:08:21 STEP 1916 H=1916:00 +CPU= 14.220
  22:08:43 STEP 1917 H=1917:00 +CPU= 22.404
  22:08:58 STEP 1918 H=1918:00 +CPU= 14.522
  22:09:12 STEP 1919 H=1919:00 +CPU= 14.550
  22:09:37 STEP 1920 H=1920:00 +CPU= 23.724
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_79.zip
Uploading the intermediate file: upload_file_79.zip
  22:09:50 STEP 1921 H=1921:00 +CPU= 13.100
  22:10:04 STEP 1922 H=1922:00 +CPU= 14.385
  22:10:25 STEP 1923 H=1923:00 +CPU= 21.139
  22:10:40 STEP 1924 H=1924:00 +CPU= 14.629
  22:10:54 STEP 1925 H=1925:00 +CPU= 13.926
  22:11:15 STEP 1926 H=1926:00 +CPU= 20.301
  22:11:28 STEP 1927 H=1927:00 +CPU= 13.618
  22:11:42 STEP 1928 H=1928:00 +CPU= 13.506
  22:12:02 STEP 1929 H=1929:00 +CPU= 20.482
  22:12:16 STEP 1930 H=1930:00 +CPU= 13.139
  22:12:29 STEP 1931 H=1931:00 +CPU= 13.494
  22:12:53 STEP 1932 H=1932:00 +CPU= 23.274
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001932
  22:13:07 STEP 1933 H=1933:00 +CPU= 14.379
  22:13:21 STEP 1934 H=1934:00 +CPU= 14.097
  22:13:41 STEP 1935 H=1935:00 +CPU= 19.992
  22:13:55 STEP 1936 H=1936:00 +CPU= 14.186
  22:14:09 STEP 1937 H=1937:00 +CPU= 13.167
  22:14:30 STEP 1938 H=1938:00 +CPU= 21.712
  22:14:44 STEP 1939 H=1939:00 +CPU= 13.250
  22:14:56 STEP 1940 H=1940:00 +CPU= 12.793
  22:15:16 STEP 1941 H=1941:00 +CPU= 20.023
  22:15:30 STEP 1942 H=1942:00 +CPU= 13.818
  22:15:45 STEP 1943 H=1943:00 +CPU= 15.161
  22:16:15 STEP 1944 H=1944:00 +CPU= 24.021
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_80.zip
Uploading the intermediate file: upload_file_80.zip
  22:16:28 STEP 1945 H=1945:00 +CPU= 13.180
  22:16:41 STEP 1946 H=1946:00 +CPU= 13.179
  22:17:02 STEP 1947 H=1947:00 +CPU= 20.886
  22:17:16 STEP 1948 H=1948:00 +CPU= 13.766
  22:17:29 STEP 1949 H=1949:00 +CPU= 13.097
  22:17:50 STEP 1950 H=1950:00 +CPU= 20.573
  22:18:03 STEP 1951 H=1951:00 +CPU= 12.866
  22:18:17 STEP 1952 H=1952:00 +CPU= 13.847
  22:18:43 STEP 1953 H=1953:00 +CPU= 22.321
  22:18:56 STEP 1954 H=1954:00 +CPU= 13.476
  22:19:09 STEP 1955 H=1955:00 +CPU= 13.121
  22:19:32 STEP 1956 H=1956:00 +CPU= 22.517
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001956
  22:19:46 STEP 1957 H=1957:00 +CPU= 13.586
  22:20:04 STEP 1958 H=1958:00 +CPU= 13.547
  22:20:28 STEP 1959 H=1959:00 +CPU= 23.242
  22:20:42 STEP 1960 H=1960:00 +CPU= 14.547
  22:20:56 STEP 1961 H=1961:00 +CPU= 13.509
  22:21:17 STEP 1962 H=1962:00 +CPU= 20.639
  22:21:30 STEP 1963 H=1963:00 +CPU= 13.075
  22:21:43 STEP 1964 H=1964:00 +CPU= 13.508
  22:22:05 STEP 1965 H=1965:00 +CPU= 21.257
  22:22:18 STEP 1966 H=1966:00 +CPU= 13.266
  22:22:31 STEP 1967 H=1967:00 +CPU= 13.424
  22:23:03 STEP 1968 H=1968:00 +CPU= 24.746
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_81.zip
Uploading the intermediate file: upload_file_81.zip
  22:23:17 STEP 1969 H=1969:00 +CPU= 13.973
Uploading trickle at timestep: 7084800
  22:23:31 STEP 1970 H=1970:00 +CPU= 14.140
  22:23:53 STEP 1971 H=1971:00 +CPU= 21.664
  22:24:06 STEP 1972 H=1972:00 +CPU= 13.443
  22:24:20 STEP 1973 H=1973:00 +CPU= 13.351
  22:24:41 STEP 1974 H=1974:00 +CPU= 21.184
  22:24:54 STEP 1975 H=1975:00 +CPU= 13.114
  22:25:09 STEP 1976 H=1976:00 +CPU= 14.717
  22:25:30 STEP 1977 H=1977:00 +CPU= 21.323
  22:25:45 STEP 1978 H=1978:00 +CPU= 14.910
  22:25:59 STEP 1979 H=1979:00 +CPU= 14.048
  22:26:23 STEP 1980 H=1980:00 +CPU= 23.385
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001980
  22:26:37 STEP 1981 H=1981:00 +CPU= 14.048
  22:26:50 STEP 1982 H=1982:00 +CPU= 13.028
  22:27:12 STEP 1983 H=1983:00 +CPU= 21.723
  22:27:26 STEP 1984 H=1984:00 +CPU= 13.912
  22:27:39 STEP 1985 H=1985:00 +CPU= 13.171
  22:28:01 STEP 1986 H=1986:00 +CPU= 22.515
  22:28:16 STEP 1987 H=1987:00 +CPU= 14.385
  22:28:31 STEP 1988 H=1988:00 +CPU= 14.586
  22:28:51 STEP 1989 H=1989:00 +CPU= 20.177
  22:29:04 STEP 1990 H=1990:00 +CPU= 12.987
  22:29:17 STEP 1991 H=1991:00 +CPU= 13.456
  22:29:42 STEP 1992 H=1992:00 +CPU= 24.598
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            9  MAX ETADOT VERTICAL VEL.=   2.549346426186874E-007
  LON  =    128.400000000000       degrees
  LAT  =   -30.8409554950018       degrees
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_82.zip
Uploading the intermediate file: upload_file_82.zip
  22:29:56 STEP 1993 H=1993:00 +CPU= 14.204
  22:30:10 STEP 1994 H=1994:00 +CPU= 13.675
  22:30:30 STEP 1995 H=1995:00 +CPU= 19.873
  22:30:43 STEP 1996 H=1996:00 +CPU= 13.426
  22:30:56 STEP 1997 H=1997:00 +CPU= 12.887
  22:31:19 STEP 1998 H=1998:00 +CPU= 18.967
  22:31:33 STEP 1999 H=1999:00 +CPU= 14.657
  22:31:48 STEP 2000 H=2000:00 +CPU= 14.400
  22:32:09 STEP 2001 H=2001:00 +CPU= 20.842
  22:32:22 STEP 2002 H=2002:00 +CPU= 13.502
  22:32:35 STEP 2003 H=2003:00 +CPU= 13.032
  22:32:59 STEP 2004 H=2004:00 +CPU= 23.496
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002004
  22:33:12 STEP 2005 H=2005:00 +CPU= 13.063
  22:33:25 STEP 2006 H=2006:00 +CPU= 13.122
  22:33:45 STEP 2007 H=2007:00 +CPU= 19.925
  22:33:59 STEP 2008 H=2008:00 +CPU= 13.711
  22:34:13 STEP 2009 H=2009:00 +CPU= 14.533
  22:34:34 STEP 2010 H=2010:00 +CPU= 20.918
  22:34:47 STEP 2011 H=2011:00 +CPU= 12.901
  22:35:01 STEP 2012 H=2012:00 +CPU= 13.360
  22:35:21 STEP 2013 H=2013:00 +CPU= 20.565
  22:35:36 STEP 2014 H=2014:00 +CPU= 14.595
  22:35:50 STEP 2015 H=2015:00 +CPU= 14.386
  22:36:25 STEP 2016 H=2016:00 +CPU= 24.700
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_83.zip
Uploading the intermediate file: upload_file_83.zip
  22:36:39 STEP 2017 H=2017:00 +CPU= 13.613
  22:36:53 STEP 2018 H=2018:00 +CPU= 13.880
  22:37:13 STEP 2019 H=2019:00 +CPU= 20.155
  22:37:27 STEP 2020 H=2020:00 +CPU= 13.545
  22:37:40 STEP 2021 H=2021:00 +CPU= 13.025
  22:38:07 STEP 2022 H=2022:00 +CPU= 21.775
  22:38:22 STEP 2023 H=2023:00 +CPU= 14.762
  22:38:35 STEP 2024 H=2024:00 +CPU= 13.128
  22:38:55 STEP 2025 H=2025:00 +CPU= 20.411
  22:39:09 STEP 2026 H=2026:00 +CPU= 13.423
  22:39:22 STEP 2027 H=2027:00 +CPU= 13.083
  22:39:45 STEP 2028 H=2028:00 +CPU= 23.380
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002028
  22:39:58 STEP 2029 H=2029:00 +CPU= 12.928
  22:40:11 STEP 2030 H=2030:00 +CPU= 13.088
  22:40:32 STEP 2031 H=2031:00 +CPU= 20.399
  22:40:46 STEP 2032 H=2032:00 +CPU= 14.216
  22:41:01 STEP 2033 H=2033:00 +CPU= 14.297
  22:41:22 STEP 2034 H=2034:00 +CPU= 20.968
  22:41:34 STEP 2035 H=2035:00 +CPU= 12.890
  22:41:48 STEP 2036 H=2036:00 +CPU= 13.354
  22:42:08 STEP 2037 H=2037:00 +CPU= 20.566
  22:42:21 STEP 2038 H=2038:00 +CPU= 12.749
  22:42:35 STEP 2039 H=2039:00 +CPU= 13.498
  22:42:59 STEP 2040 H=2040:00 +CPU= 23.878
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_84.zip
Uploading the intermediate file: upload_file_84.zip
  22:43:12 STEP 2041 H=2041:00 +CPU= 13.380
  22:43:25 STEP 2042 H=2042:00 +CPU= 13.272
  22:43:46 STEP 2043 H=2043:00 +CPU= 20.638
  22:43:59 STEP 2044 H=2044:00 +CPU= 13.357
  22:44:12 STEP 2045 H=2045:00 +CPU= 12.993
  22:44:36 STEP 2046 H=2046:00 +CPU= 21.744
  22:44:50 STEP 2047 H=2047:00 +CPU= 14.177
  22:45:03 STEP 2048 H=2048:00 +CPU= 12.905
  22:45:23 STEP 2049 H=2049:00 +CPU= 20.174
  22:45:37 STEP 2050 H=2050:00 +CPU= 13.546
  22:45:50 STEP 2051 H=2051:00 +CPU= 12.980
  22:46:14 STEP 2052 H=2052:00 +CPU= 24.320
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002052
  22:46:27 STEP 2053 H=2053:00 +CPU= 12.874
  22:46:40 STEP 2054 H=2054:00 +CPU= 13.586
  22:47:01 STEP 2055 H=2055:00 +CPU= 20.584
  22:47:14 STEP 2056 H=2056:00 +CPU= 13.198
  22:47:28 STEP 2057 H=2057:00 +CPU= 13.938
  22:47:49 STEP 2058 H=2058:00 +CPU= 20.862
  22:48:02 STEP 2059 H=2059:00 +CPU= 13.119
  22:48:16 STEP 2060 H=2060:00 +CPU= 13.280
  22:48:37 STEP 2061 H=2061:00 +CPU= 21.622
  22:48:51 STEP 2062 H=2062:00 +CPU= 14.022
  22:49:05 STEP 2063 H=2063:00 +CPU= 13.614
  22:49:29 STEP 2064 H=2064:00 +CPU= 24.077
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_85.zip
Uploading the intermediate file: upload_file_85.zip
  22:49:42 STEP 2065 H=2065:00 +CPU= 12.694
  22:49:56 STEP 2066 H=2066:00 +CPU= 13.735
  22:50:17 STEP 2067 H=2067:00 +CPU= 21.672
  22:50:31 STEP 2068 H=2068:00 +CPU= 13.405
  22:50:44 STEP 2069 H=2069:00 +CPU= 13.203
  22:51:05 STEP 2070 H=2070:00 +CPU= 21.095
  22:51:19 STEP 2071 H=2071:00 +CPU= 14.036
  22:51:33 STEP 2072 H=2072:00 +CPU= 13.357
  22:51:53 STEP 2073 H=2073:00 +CPU= 20.263
  22:52:06 STEP 2074 H=2074:00 +CPU= 13.180
  22:52:19 STEP 2075 H=2075:00 +CPU= 12.924
  22:52:44 STEP 2076 H=2076:00 +CPU= 24.625
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002076
  22:52:58 STEP 2077 H=2077:00 +CPU= 14.081
  22:53:11 STEP 2078 H=2078:00 +CPU= 13.323
  22:53:32 STEP 2079 H=2079:00 +CPU= 20.369
  22:53:45 STEP 2080 H=2080:00 +CPU= 13.140
  22:53:58 STEP 2081 H=2081:00 +CPU= 12.973
  22:54:21 STEP 2082 H=2082:00 +CPU= 23.000
  22:54:34 STEP 2083 H=2083:00 +CPU= 13.193
  22:54:48 STEP 2084 H=2084:00 +CPU= 13.318
  22:55:08 STEP 2085 H=2085:00 +CPU= 20.279
  22:55:22 STEP 2086 H=2086:00 +CPU= 13.560
  22:55:36 STEP 2087 H=2087:00 +CPU= 14.581
  22:56:01 STEP 2088 H=2088:00 +CPU= 24.288
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_86.zip
Uploading the intermediate file: upload_file_86.zip
  22:56:14 STEP 2089 H=2089:00 +CPU= 12.986
  22:56:27 STEP 2090 H=2090:00 +CPU= 12.976
  22:56:48 STEP 2091 H=2091:00 +CPU= 21.529
  22:57:02 STEP 2092 H=2092:00 +CPU= 13.859
  22:57:16 STEP 2093 H=2093:00 +CPU= 13.941
  22:57:37 STEP 2094 H=2094:00 +CPU= 20.856
  22:57:50 STEP 2095 H=2095:00 +CPU= 13.195
  22:58:05 STEP 2096 H=2096:00 +CPU= 14.593
  22:58:27 STEP 2097 H=2097:00 +CPU= 21.590
  22:58:40 STEP 2098 H=2098:00 +CPU= 13.436
  22:58:53 STEP 2099 H=2099:00 +CPU= 13.241
  22:59:17 STEP 2100 H=2100:00 +CPU= 23.795
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002100
  22:59:31 STEP 2101 H=2101:00 +CPU= 14.116
  22:59:45 STEP 2102 H=2102:00 +CPU= 13.464
  23:00:05 STEP 2103 H=2103:00 +CPU= 20.183
  23:00:19 STEP 2104 H=2104:00 +CPU= 13.514
  23:00:32 STEP 2105 H=2105:00 +CPU= 13.064
  23:00:54 STEP 2106 H=2106:00 +CPU= 22.559
  23:01:08 STEP 2107 H=2107:00 +CPU= 13.173
  23:01:21 STEP 2108 H=2108:00 +CPU= 13.638
  23:01:42 STEP 2109 H=2109:00 +CPU= 20.334
  23:01:55 STEP 2110 H=2110:00 +CPU= 13.350
  23:02:09 STEP 2111 H=2111:00 +CPU= 14.422
  23:02:43 STEP 2112 H=2112:00 +CPU= 26.401
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_87.zip
Uploading the intermediate file: upload_file_87.zip
  23:02:57 STEP 2113 H=2113:00 +CPU= 14.096
  23:03:10 STEP 2114 H=2114:00 +CPU= 12.694
  23:03:29 STEP 2115 H=2115:00 +CPU= 18.682
  23:03:42 STEP 2116 H=2116:00 +CPU= 12.649
  23:03:54 STEP 2117 H=2117:00 +CPU= 12.416
  23:04:16 STEP 2118 H=2118:00 +CPU= 21.478
  23:04:31 STEP 2119 H=2119:00 +CPU= 14.989
  23:04:44 STEP 2120 H=2120:00 +CPU= 13.387
  23:05:04 STEP 2121 H=2121:00 +CPU= 20.289
  23:05:18 STEP 2122 H=2122:00 +CPU= 13.533
  23:05:31 STEP 2123 H=2123:00 +CPU= 12.796
  23:05:55 STEP 2124 H=2124:00 +CPU= 23.721
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002124
  23:06:08 STEP 2125 H=2125:00 +CPU= 13.242
  23:06:22 STEP 2126 H=2126:00 +CPU= 13.234
  23:06:42 STEP 2127 H=2127:00 +CPU= 20.274
  23:06:56 STEP 2128 H=2128:00 +CPU= 13.968
  23:07:09 STEP 2129 H=2129:00 +CPU= 13.233
  23:07:30 STEP 2130 H=2130:00 +CPU= 20.671
  23:07:43 STEP 2131 H=2131:00 +CPU= 12.794
  23:07:56 STEP 2132 H=2132:00 +CPU= 13.075
  23:08:17 STEP 2133 H=2133:00 +CPU= 21.492
  23:08:31 STEP 2134 H=2134:00 +CPU= 13.770
  23:08:47 STEP 2135 H=2135:00 +CPU= 13.563
  23:09:11 STEP 2136 H=2136:00 +CPU= 24.202
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_88.zip
Uploading the intermediate file: upload_file_88.zip
  23:09:25 STEP 2137 H=2137:00 +CPU= 13.451
  23:09:40 STEP 2138 H=2138:00 +CPU= 14.255
  23:10:02 STEP 2139 H=2139:00 +CPU= 21.909
  23:10:17 STEP 2140 H=2140:00 +CPU= 14.740
  23:10:30 STEP 2141 H=2141:00 +CPU= 13.115
  23:10:52 STEP 2142 H=2142:00 +CPU= 21.884
  23:11:05 STEP 2143 H=2143:00 +CPU= 13.508
  23:11:19 STEP 2144 H=2144:00 +CPU= 13.848
  23:11:41 STEP 2145 H=2145:00 +CPU= 21.431
  23:11:54 STEP 2146 H=2146:00 +CPU= 13.338
  23:12:07 STEP 2147 H=2147:00 +CPU= 13.260
  23:12:30 STEP 2148 H=2148:00 +CPU= 22.237
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002148
  23:12:51 STEP 2149 H=2149:00 +CPU= 13.772
  23:13:06 STEP 2150 H=2150:00 +CPU= 15.265
  23:13:28 STEP 2151 H=2151:00 +CPU= 21.457
  23:13:41 STEP 2152 H=2152:00 +CPU= 13.126
  23:13:54 STEP 2153 H=2153:00 +CPU= 13.240
  23:14:15 STEP 2154 H=2154:00 +CPU= 21.148
  23:14:29 STEP 2155 H=2155:00 +CPU= 13.620
  23:14:42 STEP 2156 H=2156:00 +CPU= 13.327
  23:15:03 STEP 2157 H=2157:00 +CPU= 20.628
  23:15:16 STEP 2158 H=2158:00 +CPU= 13.000
  23:15:29 STEP 2159 H=2159:00 +CPU= 13.257
  23:15:57 STEP 2160 H=2160:00 +CPU= 26.641
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_89.zip
Uploading the intermediate file: upload_file_89.zip
  23:16:10 STEP 2161 H=2161:00 +CPU= 13.443
  23:16:24 STEP 2162 H=2162:00 +CPU= 13.886
  23:16:44 STEP 2163 H=2163:00 +CPU= 20.046
  23:16:58 STEP 2164 H=2164:00 +CPU= 13.281
  23:17:11 STEP 2165 H=2165:00 +CPU= 13.720
  23:17:33 STEP 2166 H=2166:00 +CPU= 21.583
  23:17:46 STEP 2167 H=2167:00 +CPU= 12.988
  23:17:59 STEP 2168 H=2168:00 +CPU= 13.348
  23:18:20 STEP 2169 H=2169:00 +CPU= 20.298
  23:18:33 STEP 2170 H=2170:00 +CPU= 13.262
  23:18:47 STEP 2171 H=2171:00 +CPU= 14.113
  23:19:10 STEP 2172 H=2172:00 +CPU= 22.299
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002172
  23:19:23 STEP 2173 H=2173:00 +CPU= 13.162
  23:19:36 STEP 2174 H=2174:00 +CPU= 13.210
  23:19:57 STEP 2175 H=2175:00 +CPU= 20.959
  23:20:11 STEP 2176 H=2176:00 +CPU= 14.258
  23:20:25 STEP 2177 H=2177:00 +CPU= 13.258
  23:20:46 STEP 2178 H=2178:00 +CPU= 21.301
  23:20:59 STEP 2179 H=2179:00 +CPU= 13.269
  23:21:13 STEP 2180 H=2180:00 +CPU= 13.655
  23:21:35 STEP 2181 H=2181:00 +CPU= 22.373
  23:21:49 STEP 2182 H=2182:00 +CPU= 13.966
  23:22:05 STEP 2183 H=2183:00 +CPU= 14.457
  23:22:27 STEP 2184 H=2184:00 +CPU= 22.184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_90.zip
Uploading the intermediate file: upload_file_90.zip
  23:22:38 STEP 2185 H=2185:00 +CPU= 11.525
  23:22:51 STEP 2186 H=2186:00 +CPU= 12.968
  23:23:12 STEP 2187 H=2187:00 +CPU= 20.942
  23:23:26 STEP 2188 H=2188:00 +CPU= 13.946
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0444
(argv4) batchid: 984
(argv5) wuid: 12201088
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_57282bf191950f83447550d9c5d9ab8f to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0444_2015050100_123_984_12201088.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0444_2015050100_123_984_12201088.zip
ic_ancil_file: ic_ancil_12201088
ifsdata_file: ifsdata_12201088
climate_data_file: clim_data_12201088
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8b487ec9251f9a452bc70c13afc0eb07 to: /var/lib/boinc-client/slots/2/ic_ancil_12201088.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12201088.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_207f6b7639f3caec7b5f5b0e232ce83f to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12201088.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12201088.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12201088.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12201088.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12201088.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12201088.xml
-- Model is restarting --
Adjusting last_iter, 2189, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12201088.xml
last_cpu_time: 43466
upload_file_number: 91
last_iter: 2183
last_upload: 7862400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580
The child process has been launched with process id: 1715344
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:saint-marie:1:1:1715344:1715344] [20221224:232539:1671938739:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  23:26:05 STEP 2184 H=2184:00 +CPU= 23.183
  23:26:18 STEP 2185 H=2185:00 +CPU= 12.742
  23:26:31 STEP 2186 H=2186:00 +CPU= 12.929
  23:26:53 STEP 2187 H=2187:00 +CPU= 21.891
  23:27:06 STEP 2188 H=2188:00 +CPU= 12.970
  23:27:18 STEP 2189 H=2189:00 +CPU= 12.619
  23:27:40 STEP 2190 H=2190:00 +CPU= 21.282
  23:27:53 STEP 2191 H=2191:00 +CPU= 12.777
  23:28:07 STEP 2192 H=2192:00 +CPU= 14.089
  23:28:27 STEP 2193 H=2193:00 +CPU= 20.507
  23:28:41 STEP 2194 H=2194:00 +CPU= 13.243
  23:28:53 STEP 2195 H=2195:00 +CPU= 12.652
  23:29:17 STEP 2196 H=2196:00 +CPU= 23.115
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002196
  23:29:31 STEP 2197 H=2197:00 +CPU= 14.001
  23:29:44 STEP 2198 H=2198:00 +CPU= 12.879
  23:30:05 STEP 2199 H=2199:00 +CPU= 20.958
  23:30:18 STEP 2200 H=2200:00 +CPU= 13.015
  23:30:31 STEP 2201 H=2201:00 +CPU= 12.665
  23:30:51 STEP 2202 H=2202:00 +CPU= 20.512
  23:31:04 STEP 2203 H=2203:00 +CPU= 12.758
  23:31:17 STEP 2204 H=2204:00 +CPU= 12.999
  23:31:38 STEP 2205 H=2205:00 +CPU= 20.585
  23:31:50 STEP 2206 H=2206:00 +CPU= 12.859
  23:32:04 STEP 2207 H=2207:00 +CPU= 13.116
  23:32:27 STEP 2208 H=2208:00 +CPU= 23.007
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201088/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0444_2015050100_123_984_12201088_0_r1306360580_91.zip
Uploading the intermediate file: upload_file_91.zip
  23:32:40 STEP 2209 H=2209:00 +CPU= 12.928
  23:32:54 STEP 2210 H=2210:00 +CPU= 14.360
  23:33:15 STEP 2211 H=2211:00 +CPU= 20.959
  23:33:30 STEP 2212 H=2212:00 +CPU= 14.358
  23:33:42 STEP 2213 H=2213:00 +CPU= 12.836
  23:34:04 STEP 2214 H=2214:00 +CPU= 21.118
  23:34:17 STEP 2215 H=2215:00 +CPU= 13.807
  23:34:30 STEP 2216 H=2216:00 +CPU= 12.089
  23:34:50 STEP 2217 H=2217:00 +CPU= 20.160
  23:35:03 STEP 2218 H=2218:00 +CPU= 13.355
  23:35:17 STEP 2219 H=2219:00 +CPU= 13.380
  23:35:41 STEP 2220 H=2220:00 +CPU= 23.394
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002220
  23:35:54 STEP 2221 H=2221:00 +CPU= 12.865
  23:36:07 STEP 2222 H=2222:00 +CPU= 13.537
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:53:56 1478928 22287360 oifs_43r3_ps_0444_2015050100_123_984_12201088_0 7,084,800 39,893 0.0056
09 May 2023 15:53:42 1478928 22287360 oifs_43r3_ps_0444_2015050100_123_984_12201088_0 5,616,000 33,222 0.0059
09 May 2023 15:53:28 1478928 22287360 oifs_43r3_ps_0444_2015050100_123_984_12201088_0 4,060,800 26,032 0.0064
09 May 2023 15:53:14 1478928 22287360 oifs_43r3_ps_0444_2015050100_123_984_12201088_0 2,505,600 18,891 0.0075
09 May 2023 15:53:01 1478928 22287360 oifs_43r3_ps_0444_2015050100_123_984_12201088_0 950,400 11,773 0.0124


©2024 cpdn.org