climateprediction.net home page
Task 22442361

Task 22442361

Name oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0
Workunit 12288796
Created 12 Jun 2024, 20:39:21 UTC
Sent 14 Jun 2024, 0:14:03 UTC
Report deadline 13 Aug 2024, 0:14:03 UTC
Received 15 Jun 2024, 1:14:45 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1533065
Run time 22 hours 16 min 32 sec
CPU time 22 hours 16 min 32 sec
Validate state Valid
Credit 2,638.30
Device peak FLOPS 5.37 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,020.89 MB
Peak swap size 5,981.12 MB
Peak disk usage 1,374.29 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6394:6394] [20240614:231712:1718399832:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0_r1615134544_16.zip
Uploading the intermediate file: upload_file_16.zip
  23:18:18 STEP 1632 H= 408:00 +CPU= 46.158
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001632
  23:18:57 STEP 1633 H= 408:15 +CPU= 38.557
  23:19:34 STEP 1634 H= 408:30 +CPU= 36.980
  23:19:57 STEP 1635 H= 408:45 +CPU= 22.749
  23:20:20 STEP 1636 H= 409:00 +CPU= 23.080
  23:21:00 STEP 1637 H= 409:15 +CPU= 39.876
  23:21:46 STEP 1638 H= 409:30 +CPU= 45.846
  23:22:31 STEP 1639 H= 409:45 +CPU= 44.662
  23:22:57 STEP 1640 H= 410:00 +CPU= 25.834
  23:23:40 STEP 1641 H= 410:15 +CPU= 43.341
  23:24:28 STEP 1642 H= 410:30 +CPU= 47.019
  23:25:13 STEP 1643 H= 410:45 +CPU= 33.662
  23:25:48 STEP 1644 H= 411:00 +CPU= 33.251
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001644
  23:26:28 STEP 1645 H= 411:15 +CPU= 39.147
  23:27:08 STEP 1646 H= 411:30 +CPU= 40.189
  23:27:44 STEP 1647 H= 411:45 +CPU= 36.131
  23:28:14 STEP 1648 H= 412:00 +CPU= 29.598
  23:28:42 STEP 1649 H= 412:15 +CPU= 28.130
  23:29:15 STEP 1650 H= 412:30 +CPU= 32.851
  23:29:55 STEP 1651 H= 412:45 +CPU= 39.164
  23:30:27 STEP 1652 H= 413:00 +CPU= 32.711
  23:31:01 STEP 1653 H= 413:15 +CPU= 33.240
  23:31:31 STEP 1654 H= 413:30 +CPU= 30.010
  23:31:56 STEP 1655 H= 413:45 +CPU= 24.672
  23:32:22 STEP 1656 H= 414:00 +CPU= 26.632
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001656
  23:32:43 STEP 1657 H= 414:15 +CPU= 20.397
  23:33:03 STEP 1658 H= 414:30 +CPU= 20.247
  23:33:26 STEP 1659 H= 414:45 +CPU= 23.493
  23:33:58 STEP 1660 H= 415:00 +CPU= 31.448
  23:34:29 STEP 1661 H= 415:15 +CPU= 31.484
  23:35:03 STEP 1662 H= 415:30 +CPU= 33.456
  23:35:34 STEP 1663 H= 415:45 +CPU= 31.386
  23:36:06 STEP 1664 H= 416:00 +CPU= 31.931
  23:36:36 STEP 1665 H= 416:15 +CPU= 29.220
  23:37:05 STEP 1666 H= 416:30 +CPU= 29.090
  23:37:33 STEP 1667 H= 416:45 +CPU= 28.196
  23:38:01 STEP 1668 H= 417:00 +CPU= 28.170
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001668
  23:38:24 STEP 1669 H= 417:15 +CPU= 22.680
  23:38:55 STEP 1670 H= 417:30 +CPU= 30.766
  23:39:23 STEP 1671 H= 417:45 +CPU= 27.655
  23:39:50 STEP 1672 H= 418:00 +CPU= 27.690
  23:40:16 STEP 1673 H= 418:15 +CPU= 26.209
  23:40:40 STEP 1674 H= 418:30 +CPU= 23.641
  23:41:12 STEP 1675 H= 418:45 +CPU= 31.443
  23:41:54 STEP 1676 H= 419:00 +CPU= 42.654
  23:42:42 STEP 1677 H= 419:15 +CPU= 46.393
  23:43:18 STEP 1678 H= 419:30 +CPU= 36.275
  23:43:52 STEP 1679 H= 419:45 +CPU= 33.116
  23:44:48 STEP 1680 H= 420:00 +CPU= 49.069
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001680
  23:45:19 STEP 1681 H= 420:15 +CPU= 31.116
  23:45:40 STEP 1682 H= 420:30 +CPU= 20.592
  23:46:00 STEP 1683 H= 420:45 +CPU= 20.441
  23:46:32 STEP 1684 H= 421:00 +CPU= 32.120
  23:47:15 STEP 1685 H= 421:15 +CPU= 42.414
  23:48:09 STEP 1686 H= 421:30 +CPU= 52.826
  23:48:55 STEP 1687 H= 421:45 +CPU= 45.721
  23:49:30 STEP 1688 H= 422:00 +CPU= 34.789
  23:50:08 STEP 1689 H= 422:15 +CPU= 38.409
  23:50:54 STEP 1690 H= 422:30 +CPU= 45.181
  23:51:43 STEP 1691 H= 422:45 +CPU= 47.674
  23:52:29 STEP 1692 H= 423:00 +CPU= 45.969
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001692
  23:53:00 STEP 1693 H= 423:15 +CPU= 31.270
  23:53:20 STEP 1694 H= 423:30 +CPU= 20.095
  23:53:41 STEP 1695 H= 423:45 +CPU= 20.315
  23:54:04 STEP 1696 H= 424:00 +CPU= 22.679
  23:54:38 STEP 1697 H= 424:15 +CPU= 34.063
  23:55:06 STEP 1698 H= 424:30 +CPU= 27.900
  23:55:32 STEP 1699 H= 424:45 +CPU= 25.818
  23:55:54 STEP 1700 H= 425:00 +CPU= 22.789
  23:56:37 STEP 1701 H= 425:15 +CPU= 42.974
  23:57:27 STEP 1702 H= 425:30 +CPU= 48.382
  23:58:14 STEP 1703 H= 425:45 +CPU= 45.847
  23:58:54 STEP 1704 H= 426:00 +CPU= 40.576
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001704
  23:59:15 STEP 1705 H= 426:15 +CPU= 20.838
  23:59:36 STEP 1706 H= 426:30 +CPU= 20.583
  23:59:58 STEP 1707 H= 426:45 +CPU= 22.013
  00:00:40 STEP 1708 H= 427:00 +CPU= 40.926
  00:01:07 STEP 1709 H= 427:15 +CPU= 27.301
  00:01:35 STEP 1710 H= 427:30 +CPU= 27.962
  00:02:07 STEP 1711 H= 427:45 +CPU= 31.711
  00:02:35 STEP 1712 H= 428:00 +CPU= 27.909
  00:03:06 STEP 1713 H= 428:15 +CPU= 30.739
  00:03:52 STEP 1714 H= 428:30 +CPU= 45.900
  00:04:38 STEP 1715 H= 428:45 +CPU= 46.403
  00:05:27 STEP 1716 H= 429:00 +CPU= 48.507
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001716
  00:06:08 STEP 1717 H= 429:15 +CPU= 40.175
  00:06:38 STEP 1718 H= 429:30 +CPU= 30.682
  00:07:25 STEP 1719 H= 429:45 +CPU= 46.028
  00:08:11 STEP 1720 H= 430:00 +CPU= 45.938
  00:08:34 STEP 1721 H= 430:15 +CPU= 22.748
  00:08:57 STEP 1722 H= 430:30 +CPU= 22.690
  00:09:24 STEP 1723 H= 430:45 +CPU= 27.289
  00:10:11 STEP 1724 H= 431:00 +CPU= 46.330
  00:10:59 STEP 1725 H= 431:15 +CPU= 47.409
  00:11:38 STEP 1726 H= 431:30 +CPU= 38.667
  00:12:02 STEP 1727 H= 431:45 +CPU= 23.836
  00:12:38 STEP 1728 H= 432:00 +CPU= 31.976
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001728
  00:13:18 STEP 1729 H= 432:15 +CPU= 39.325
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001728
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0_r1615134544_17.zip
Uploading the intermediate file: upload_file_17.zip
  00:13:51 STEP 1730 H= 432:30 +CPU= 33.722
  00:14:26 STEP 1731 H= 432:45 +CPU= 34.111
  00:14:54 STEP 1732 H= 433:00 +CPU= 27.326
  00:15:23 STEP 1733 H= 433:15 +CPU= 29.621
  00:15:59 STEP 1734 H= 433:30 +CPU= 35.978
  00:16:45 STEP 1735 H= 433:45 +CPU= 45.795
  00:17:28 STEP 1736 H= 434:00 +CPU= 42.649
  00:17:59 STEP 1737 H= 434:15 +CPU= 29.215
  00:18:30 STEP 1738 H= 434:30 +CPU= 30.836
  00:19:14 STEP 1739 H= 434:45 +CPU= 43.664
  00:20:06 STEP 1740 H= 435:00 +CPU= 50.491
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001740
  00:20:39 STEP 1741 H= 435:15 +CPU= 33.340
  00:21:13 STEP 1742 H= 435:30 +CPU= 33.587
  00:21:48 STEP 1743 H= 435:45 +CPU= 34.576
  00:22:13 STEP 1744 H= 436:00 +CPU= 24.975
  00:22:45 STEP 1745 H= 436:15 +CPU= 32.600
  00:23:33 STEP 1746 H= 436:30 +CPU= 47.722
  00:24:18 STEP 1747 H= 436:45 +CPU= 44.674
  00:24:56 STEP 1748 H= 437:00 +CPU= 37.771
  00:25:30 STEP 1749 H= 437:15 +CPU= 33.501
  00:26:07 STEP 1750 H= 437:30 +CPU= 36.648
  00:26:52 STEP 1751 H= 437:45 +CPU= 44.978
  00:28:03 STEP 1752 H= 438:00 +CPU= 58.211
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001752
  00:28:50 STEP 1753 H= 438:15 +CPU= 46.591
  00:29:36 STEP 1754 H= 438:30 +CPU= 45.437
  00:30:11 STEP 1755 H= 438:45 +CPU= 35.158
  00:30:44 STEP 1756 H= 439:00 +CPU= 32.367
  00:31:19 STEP 1757 H= 439:15 +CPU= 34.866
  00:31:53 STEP 1758 H= 439:30 +CPU= 34.457
  00:32:25 STEP 1759 H= 439:45 +CPU= 31.805
  00:33:00 STEP 1760 H= 440:00 +CPU= 34.453
  00:33:29 STEP 1761 H= 440:15 +CPU= 28.976
  00:34:14 STEP 1762 H= 440:30 +CPU= 44.866
  00:35:03 STEP 1763 H= 440:45 +CPU= 48.597
  00:35:37 STEP 1764 H= 441:00 +CPU= 33.172
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001764
  00:36:07 STEP 1765 H= 441:15 +CPU= 29.844
  00:36:39 STEP 1766 H= 441:30 +CPU= 31.415
  00:37:21 STEP 1767 H= 441:45 +CPU= 42.263
  00:38:08 STEP 1768 H= 442:00 +CPU= 45.582
  00:38:44 STEP 1769 H= 442:15 +CPU= 35.619
  00:39:23 STEP 1770 H= 442:30 +CPU= 39.324
  00:40:01 STEP 1771 H= 442:45 +CPU= 37.605
  00:40:39 STEP 1772 H= 443:00 +CPU= 38.632
  00:41:14 STEP 1773 H= 443:15 +CPU= 34.886
  00:41:45 STEP 1774 H= 443:30 +CPU= 30.595
  00:42:20 STEP 1775 H= 443:45 +CPU= 34.709
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0d1
(argv4) batchid: 1018
(argv5) wuid: 12288796
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_da9aa4b370735f792e78df47e71e01f5 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796.zip
ic_ancil_file: ic_ancil_12288796
ifsdata_file: ifsdata_12288796
climate_data_file: clim_data_12288796
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12288796.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12288796.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_82fed58ed8b846cea48834c57543b7df to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12288796.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12288796.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12288796.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12288796.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12288796.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12288796.xml
-- Model is restarting --
Adjusting last_iter, 1776, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12288796.xml
last_cpu_time: 75561
upload_file_number: 18
last_iter: 1777
last_upload: 1556100
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0_r1615134544
The child process has been launched with process id: 6931
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6931:6931] [20240615:004424:1718405064:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  00:45:18 STEP 1776 H= 444:00 +CPU= 46.473
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001776
  00:45:52 STEP 1777 H= 444:15 +CPU= 34.194
  00:46:21 STEP 1778 H= 444:30 +CPU= 28.811
  00:46:45 STEP 1779 H= 444:45 +CPU= 23.759
  00:47:08 STEP 1780 H= 445:00 +CPU= 23.399
  00:47:32 STEP 1781 H= 445:15 +CPU= 23.759
  00:48:16 STEP 1782 H= 445:30 +CPU= 43.143
  00:49:01 STEP 1783 H= 445:45 +CPU= 44.686
  00:49:46 STEP 1784 H= 446:00 +CPU= 43.601
  00:50:20 STEP 1785 H= 446:15 +CPU= 34.091
  00:50:52 STEP 1786 H= 446:30 +CPU= 31.876
  00:51:22 STEP 1787 H= 446:45 +CPU= 29.714
  00:52:09 STEP 1788 H= 447:00 +CPU= 46.428
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001788
  00:52:32 STEP 1789 H= 447:15 +CPU= 22.877
  00:52:55 STEP 1790 H= 447:30 +CPU= 22.964
  00:53:16 STEP 1791 H= 447:45 +CPU= 21.427
  00:53:46 STEP 1792 H= 448:00 +CPU= 30.237
  00:54:31 STEP 1793 H= 448:15 +CPU= 44.038
  00:55:19 STEP 1794 H= 448:30 +CPU= 47.360
  00:55:46 STEP 1795 H= 448:45 +CPU= 26.906
  00:56:15 STEP 1796 H= 449:00 +CPU= 28.690
  00:56:53 STEP 1797 H= 449:15 +CPU= 38.671
  00:57:24 STEP 1798 H= 449:30 +CPU= 30.683
  00:57:53 STEP 1799 H= 449:45 +CPU= 29.078
  00:58:29 STEP 1800 H= 450:00 +CPU= 35.905
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001800
  00:59:16 STEP 1801 H= 450:15 +CPU= 47.003
  00:59:57 STEP 1802 H= 450:30 +CPU= 40.788
  01:00:21 STEP 1803 H= 450:45 +CPU= 23.226
  01:00:45 STEP 1804 H= 451:00 +CPU= 23.921
  01:01:09 STEP 1805 H= 451:15 +CPU= 23.752
  01:01:32 STEP 1806 H= 451:30 +CPU= 23.561
  01:01:55 STEP 1807 H= 451:45 +CPU= 23.140
  01:02:24 STEP 1808 H= 452:00 +CPU= 28.948
  01:02:48 STEP 1809 H= 452:15 +CPU= 24.113
  01:03:12 STEP 1810 H= 452:30 +CPU= 23.255
  01:03:35 STEP 1811 H= 452:45 +CPU= 23.330
  01:04:26 STEP 1812 H= 453:00 +CPU= 50.150
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001812
  01:04:56 STEP 1813 H= 453:15 +CPU= 29.790
  01:05:27 STEP 1814 H= 453:30 +CPU= 31.330
  01:06:09 STEP 1815 H= 453:45 +CPU= 41.261
  01:06:57 STEP 1816 H= 454:00 +CPU= 47.512
  01:07:35 STEP 1817 H= 454:15 +CPU= 37.340
  01:08:08 STEP 1818 H= 454:30 +CPU= 33.244
  01:08:28 STEP 1819 H= 454:45 +CPU= 20.475
  01:08:49 STEP 1820 H= 455:00 +CPU= 20.946
  01:09:10 STEP 1821 H= 455:15 +CPU= 20.835
  01:09:43 STEP 1822 H= 455:30 +CPU= 32.388
  01:10:14 STEP 1823 H= 455:45 +CPU= 31.562
  01:11:09 STEP 1824 H= 456:00 +CPU= 51.135
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001824
  01:11:42 STEP 1825 H= 456:15 +CPU= 33.562
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0_r1615134544_18.zip
Uploading the intermediate file: upload_file_18.zip
  01:12:05 STEP 1826 H= 456:30 +CPU= 22.715
  01:12:26 STEP 1827 H= 456:45 +CPU= 20.701
  01:12:47 STEP 1828 H= 457:00 +CPU= 20.809
  01:13:07 STEP 1829 H= 457:15 +CPU= 20.372
  01:13:27 STEP 1830 H= 457:30 +CPU= 19.957
  01:13:48 STEP 1831 H= 457:45 +CPU= 20.651
  01:14:21 STEP 1832 H= 458:00 +CPU= 33.392
  01:14:59 STEP 1833 H= 458:15 +CPU= 37.355
  01:15:35 STEP 1834 H= 458:30 +CPU= 36.323
  01:16:06 STEP 1835 H= 458:45 +CPU= 30.241
  01:16:44 STEP 1836 H= 459:00 +CPU= 38.263
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001836
  01:17:08 STEP 1837 H= 459:15 +CPU= 23.458
  01:17:31 STEP 1838 H= 459:30 +CPU= 23.610
  01:18:05 STEP 1839 H= 459:45 +CPU= 33.665
  01:18:36 STEP 1840 H= 460:00 +CPU= 30.777
  01:19:21 STEP 1841 H= 460:15 +CPU= 45.139
  01:20:07 STEP 1842 H= 460:30 +CPU= 45.138
  01:20:47 STEP 1843 H= 460:45 +CPU= 40.186
  01:21:09 STEP 1844 H= 461:00 +CPU= 21.376
  01:21:29 STEP 1845 H= 461:15 +CPU= 20.308
  01:21:52 STEP 1846 H= 461:30 +CPU= 23.472
  01:22:29 STEP 1847 H= 461:45 +CPU= 36.279
  01:23:12 STEP 1848 H= 462:00 +CPU= 42.646
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001848
  01:23:44 STEP 1849 H= 462:15 +CPU= 31.911
  01:24:12 STEP 1850 H= 462:30 +CPU= 28.511
  01:24:39 STEP 1851 H= 462:45 +CPU= 26.695
  01:25:02 STEP 1852 H= 463:00 +CPU= 23.415
  01:25:25 STEP 1853 H= 463:15 +CPU= 22.478
  01:25:52 STEP 1854 H= 463:30 +CPU= 26.853
  01:26:19 STEP 1855 H= 463:45 +CPU= 27.618
  01:26:47 STEP 1856 H= 464:00 +CPU= 28.082
  01:27:15 STEP 1857 H= 464:15 +CPU= 27.777
  01:27:43 STEP 1858 H= 464:30 +CPU= 27.779
  01:28:15 STEP 1859 H= 464:45 +CPU= 31.449
  01:28:43 STEP 1860 H= 465:00 +CPU= 27.609
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001860
  01:29:05 STEP 1861 H= 465:15 +CPU= 22.804
  01:29:35 STEP 1862 H= 465:30 +CPU= 29.288
  01:30:03 STEP 1863 H= 465:45 +CPU= 27.867
  01:30:35 STEP 1864 H= 466:00 +CPU= 31.906
  01:31:14 STEP 1865 H= 466:15 +CPU= 38.596
  01:31:39 STEP 1866 H= 466:30 +CPU= 25.490
  01:31:59 STEP 1867 H= 466:45 +CPU= 20.249
  01:32:20 STEP 1868 H= 467:00 +CPU= 20.968
  01:32:43 STEP 1869 H= 467:15 +CPU= 22.754
  01:33:09 STEP 1870 H= 467:30 +CPU= 26.012
  01:33:40 STEP 1871 H= 467:45 +CPU= 31.084
  01:34:29 STEP 1872 H= 468:00 +CPU= 47.237
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001872
  01:34:55 STEP 1873 H= 468:15 +CPU= 25.978
  01:35:25 STEP 1874 H= 468:30 +CPU= 29.003
  01:35:57 STEP 1875 H= 468:45 +CPU= 32.673
  01:36:29 STEP 1876 H= 469:00 +CPU= 31.487
  01:36:55 STEP 1877 H= 469:15 +CPU= 26.048
  01:37:28 STEP 1878 H= 469:30 +CPU= 32.840
  01:38:12 STEP 1879 H= 469:45 +CPU= 43.832
  01:38:56 STEP 1880 H= 470:00 +CPU= 43.495
  01:39:27 STEP 1881 H= 470:15 +CPU= 31.056
  01:39:50 STEP 1882 H= 470:30 +CPU= 22.263
  01:40:13 STEP 1883 H= 470:45 +CPU= 23.090
  01:40:49 STEP 1884 H= 471:00 +CPU= 35.354
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001884
  01:41:11 STEP 1885 H= 471:15 +CPU= 22.626
  01:41:56 STEP 1886 H= 471:30 +CPU= 43.964
  01:42:44 STEP 1887 H= 471:45 +CPU= 47.544
  01:43:26 STEP 1888 H= 472:00 +CPU= 41.872
  01:43:56 STEP 1889 H= 472:15 +CPU= 30.326
  01:44:26 STEP 1890 H= 472:30 +CPU= 29.318
  01:45:01 STEP 1891 H= 472:45 +CPU= 35.305
  01:45:40 STEP 1892 H= 473:00 +CPU= 38.674
  01:46:04 STEP 1893 H= 473:15 +CPU= 24.074
  01:46:46 STEP 1894 H= 473:30 +CPU= 41.487
  01:47:31 STEP 1895 H= 473:45 +CPU= 45.290
  01:48:18 STEP 1896 H= 474:00 +CPU= 46.067
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001896
  01:48:45 STEP 1897 H= 474:15 +CPU= 26.722
  01:49:15 STEP 1898 H= 474:30 +CPU= 29.885
  01:49:47 STEP 1899 H= 474:45 +CPU= 31.884
  01:50:18 STEP 1900 H= 475:00 +CPU= 31.325
  01:50:44 STEP 1901 H= 475:15 +CPU= 26.441
  01:51:09 STEP 1902 H= 475:30 +CPU= 24.142
  01:51:29 STEP 1903 H= 475:45 +CPU= 20.719
  01:51:50 STEP 1904 H= 476:00 +CPU= 20.790
  01:52:10 STEP 1905 H= 476:15 +CPU= 20.179
  01:52:45 STEP 1906 H= 476:30 +CPU= 34.801
  01:53:30 STEP 1907 H= 476:45 +CPU= 44.286
  01:54:22 STEP 1908 H= 477:00 +CPU= 51.237
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001908
  01:54:49 STEP 1909 H= 477:15 +CPU= 27.020
  01:55:18 STEP 1910 H= 477:30 +CPU= 29.184
  01:56:25 STEP 1911 H= 477:45 +CPU= 45.462
  01:57:10 STEP 1912 H= 478:00 +CPU= 45.176
  01:57:49 STEP 1913 H= 478:15 +CPU= 38.273
  01:58:22 STEP 1914 H= 478:30 +CPU= 32.821
  01:58:57 STEP 1915 H= 478:45 +CPU= 34.339
  01:59:29 STEP 1916 H= 479:00 +CPU= 32.424
  02:00:00 STEP 1917 H= 479:15 +CPU= 30.641
  02:00:29 STEP 1918 H= 479:30 +CPU= 29.327
  02:00:57 STEP 1919 H= 479:45 +CPU= 28.218
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0d1
(argv4) batchid: 1018
(argv5) wuid: 12288796
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_da9aa4b370735f792e78df47e71e01f5 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796.zip
ic_ancil_file: ic_ancil_12288796
ifsdata_file: ifsdata_12288796
climate_data_file: clim_data_12288796
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12288796.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12288796.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_82fed58ed8b846cea48834c57543b7df to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12288796.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12288796.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12288796.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12288796.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12288796.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12288796.xml
-- Model is restarting --
Adjusting last_iter, 1920, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12288796.xml
last_cpu_time: 80142
upload_file_number: 19
last_iter: 1921
last_upload: 1642500
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0_r1615134544
The child process has been launched with process id: 7339
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7339:7339] [20240615:020303:1718409783:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  02:03:55 STEP 1920 H= 480:00 +CPU= 45.414
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I      I Hor. Flux of PEII     -0.00000 I              I              I              I               I
I      I Vert.Flux of PEII      0.00000 I              I              I              I               I
I      I Work Pres.G.F  II              I      2.90723 I              I              I               I
I      I Conv. Ad.-Int. II     -2.90767 I              I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I DYN- Horizont. Flux   II      0.34355 I     -0.19921 I      3.15341 I     -0.01407 I     -0.04727  I
I    I Vertical Flux    II      0.00000 I      0.00000 I      0.00000 I      0.00000 I      0.00000  I
DDH--------------------------------------------------------------------------------------------------I
  
DDH----------------------------------------------------------------------------------------------------------------------------------
  I   - Mass balance    (10**(-6)  KG/M**2) 
  
  I INITIAL      I HORIZ. FLUX  I DYN. FLUX    I DYN. FLUX    I PHYS. FLUX   I PHYS. FLUX   I CURRENT      I I I RESIDUAL     I
  I MASS         I              I TOA          I SURFACE      I TOA          I SURFACE      I MASS         I I I              I
  ---------------------------------------------------------------------------------------------------------- I ----------------
  -              -              -              -              -              -              -              - I -              -
  - 10341776742. I           3. .           0. .           0. .           0. .           0. I 10344982358. - I -     3205612. -
  -              -              -              -              -              -              -              - I -              -
  ---------------------------------------------------------------------------------------------------------- I ----------------
 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1445459646E-21      I       0.4115257823E-05      I
ICloud fractionI      0.0000000000E+00      I       0.7554810325E-01      I
I  Relat. Hum. I      0.5487651595E+00      I       0.4721375151E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1137356369E+04      I
DDH------------------------------------------------------------------------
  
 02:03:55 STEP 1920 H= 480:00 +CPU= 45.414
 END CNT3
 NSTEP =  1920 CNT0   000000000  45.097  45.097  46.086   0.979               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN            0/           0
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1      3    0       46      1044068        0            0     2.0     2.0     0.0     3.2     0.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1   51565.1   51565.1    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1   46406.1   46406.1     90.00      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                    1    8949.0    8949.0     17.35      0.00
  10 SCAN2M - PHYSICS                                1   24088.7   24088.7     46.72      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  1     100.5     100.5      0.19      0.00
  28 RESTART FILES - READING                         1    4507.6    4507.6      8.74      0.00
  31 POSDDH IN STEPO                                 1       1.3       1.3      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                      1   10326.5   10326.5     20.03      0.00
  39 SU0YOMB                                         1     415.0     415.0      0.80      0.00
  51 SCAN2M   - SL COMM. PART 1                      1      45.7      45.7      0.09      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  1       0.8       0.8      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM        4     335.6     335.6      2.60      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM         4     158.6     158.6      1.23      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      60.4      60.4      0.12      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION              4      30.6      30.6      0.24      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION              4     106.5     106.5      0.83      0.00
 400 GSTATS                                       1119       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                   987       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME       51.6 CPU TIME      50.5 VECTOR TIME       50.5


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY    37236.5
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/14/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/14/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288796/ICMGGh7zg+000384
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0_r1615134544_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1728900
02:07:32 (7336): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 00:12:17 1533065 22442361 oifs_43r3_bl_a0d1_2016092300_20_1018_12288796_0 1,728,900 80,192 0.0464


©2024 cpdn.org