Name | oifs_43r3_ps_0056_1991050100_123_960_12176700_0 |
Workunit | 12176700 |
Created | 21 Dec 2022, 20:48:37 UTC |
Sent | 20 Jan 2023, 18:18:50 UTC |
Report deadline | 21 Mar 2023, 18:18:50 UTC |
Received | 25 Jan 2023, 12:38:26 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1509861 |
Run time | 7 hours 38 min 9 sec |
CPU time | 7 hours 11 min 14 sec |
Validate state | Invalid |
Credit | 1,209.60 |
Device peak FLOPS | 5.68 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4.32 GB |
Peak swap size | 4.86 GB |
Peak disk usage | 7.70 GB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:223729:1674509849:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:39:18 STEP 888 H= 888:00 +CPU= 37.416 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:40:09 STEP 889 H= 889:00 +CPU= 18.938 22:40:28 STEP 890 H= 890:00 +CPU= 17.546 22:40:56 STEP 891 H= 891:00 +CPU= 27.297 22:41:15 STEP 892 H= 892:00 +CPU= 18.393 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:41:47 STEP 893 H= 893:00 +CPU= 18.857 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:42:31 STEP 894 H= 894:00 +CPU= 30.163 22:42:51 STEP 895 H= 895:00 +CPU= 18.056 22:43:11 STEP 896 H= 896:00 +CPU= 18.549 22:43:40 STEP 897 H= 897:00 +CPU= 27.831 22:43:59 STEP 898 H= 898:00 +CPU= 17.639 22:44:16 STEP 899 H= 899:00 +CPU= 17.017 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:45:23 STEP 900 H= 900:00 +CPU= 32.106 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000900 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000900 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000900 22:45:44 STEP 901 H= 901:00 +CPU= 18.741 22:46:03 STEP 902 H= 902:00 +CPU= 18.684 22:46:33 STEP 903 H= 903:00 +CPU= 28.174 22:46:53 STEP 904 H= 904:00 +CPU= 18.535 22:47:12 STEP 905 H= 905:00 +CPU= 17.928 22:47:43 STEP 906 H= 906:00 +CPU= 28.976 22:48:02 STEP 907 H= 907:00 +CPU= 18.052 22:48:22 STEP 908 H= 908:00 +CPU= 18.441 22:48:52 STEP 909 H= 909:00 +CPU= 28.191 22:49:11 STEP 910 H= 910:00 +CPU= 18.604 22:49:31 STEP 911 H= 911:00 +CPU= 18.587 22:50:05 STEP 912 H= 912:00 +CPU= 32.614 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000912 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000912 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000900 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000900 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000900 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_37.zip Uploading the intermediate file: upload_file_37.zip 22:50:23 STEP 913 H= 913:00 +CPU= 17.161 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:50:55 STEP 914 H= 914:00 +CPU= 19.939 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:51:38 STEP 915 H= 915:00 +CPU= 29.893 22:51:59 STEP 916 H= 916:00 +CPU= 19.214 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:52:29 STEP 917 H= 917:00 +CPU= 18.479 22:53:01 STEP 918 H= 918:00 +CPU= 29.724 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:53:45 STEP 919 H= 919:00 +CPU= 20.775 22:54:05 STEP 920 H= 920:00 +CPU= 18.867 22:54:36 STEP 921 H= 921:00 +CPU= 28.509 22:54:54 STEP 922 H= 922:00 +CPU= 17.668 22:55:13 STEP 923 H= 923:00 +CPU= 18.013 22:55:44 STEP 924 H= 924:00 +CPU= 30.367 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000924 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000924 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000924 22:56:03 STEP 925 H= 925:00 +CPU= 18.177 22:56:23 STEP 926 H= 926:00 +CPU= 18.324 22:56:52 STEP 927 H= 927:00 +CPU= 27.626 22:57:12 STEP 928 H= 928:00 +CPU= 18.050 22:57:31 STEP 929 H= 929:00 +CPU= 17.968 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:58:16 STEP 930 H= 930:00 +CPU= 31.892 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:58:47 STEP 931 H= 931:00 +CPU= 18.812 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:59:21 STEP 932 H= 932:00 +CPU= 20.229 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:00:04 STEP 933 H= 933:00 +CPU= 29.326 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:00:45 STEP 934 H= 934:00 +CPU= 18.554 23:01:04 STEP 935 H= 935:00 +CPU= 18.165 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:01:52 STEP 936 H= 936:00 +CPU= 35.251 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000936 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000936 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000936 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000924 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000924 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000924 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_38.zip Uploading the intermediate file: upload_file_38.zip 23:02:11 STEP 937 H= 937:00 +CPU= 17.724 23:02:31 STEP 938 H= 938:00 +CPU= 18.203 23:03:01 STEP 939 H= 939:00 +CPU= 27.728 23:03:19 STEP 940 H= 940:00 +CPU= 17.542 23:03:36 STEP 941 H= 941:00 +CPU= 17.125 23:04:05 STEP 942 H= 942:00 +CPU= 27.652 23:04:24 STEP 943 H= 943:00 +CPU= 17.797 23:04:44 STEP 944 H= 944:00 +CPU= 19.235 23:05:15 STEP 945 H= 945:00 +CPU= 28.670 23:05:35 STEP 946 H= 946:00 +CPU= 18.468 23:05:55 STEP 947 H= 947:00 +CPU= 18.095 23:06:29 STEP 948 H= 948:00 +CPU= 31.774 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000948 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000948 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000948 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:07:12 STEP 949 H= 949:00 +CPU= 19.341 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:08:07 STEP 950 H= 950:00 +CPU= 21.055 23:08:37 STEP 951 H= 951:00 +CPU= 28.636 23:08:57 STEP 952 H= 952:00 +CPU= 18.353 23:09:17 STEP 953 H= 953:00 +CPU= 18.391 23:09:48 STEP 954 H= 954:00 +CPU= 29.546 23:10:07 STEP 955 H= 955:00 +CPU= 18.101 23:10:28 STEP 956 H= 956:00 +CPU= 18.829 23:10:59 STEP 957 H= 957:00 +CPU= 29.068 23:11:19 STEP 958 H= 958:00 +CPU= 18.649 23:11:39 STEP 959 H= 959:00 +CPU= 18.517 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:12:27 STEP 960 H= 960:00 +CPU= 34.463 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000960 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000960 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000936 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000936 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000936 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000948 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000948 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000948 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_39.zip Uploading the intermediate file: upload_file_39.zip 23:12:47 STEP 961 H= 961:00 +CPU= 18.849 23:13:07 STEP 962 H= 962:00 +CPU= 18.918 23:13:37 STEP 963 H= 963:00 +CPU= 28.179 23:13:57 STEP 964 H= 964:00 +CPU= 18.314 23:14:16 STEP 965 H= 965:00 +CPU= 17.687 23:14:47 STEP 966 H= 966:00 +CPU= 28.815 23:15:06 STEP 967 H= 967:00 +CPU= 18.400 23:15:25 STEP 968 H= 968:00 +CPU= 18.083 23:15:55 STEP 969 H= 969:00 +CPU= 28.269 23:16:14 STEP 970 H= 970:00 +CPU= 18.683 23:16:33 STEP 971 H= 971:00 +CPU= 17.860 23:17:05 STEP 972 H= 972:00 +CPU= 30.554 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000972 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000972 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000972 23:17:24 STEP 973 H= 973:00 +CPU= 17.884 23:17:43 STEP 974 H= 974:00 +CPU= 18.304 23:18:13 STEP 975 H= 975:00 +CPU= 28.339 23:18:32 STEP 976 H= 976:00 +CPU= 18.422 23:18:51 STEP 977 H= 977:00 +CPU= 17.746 23:19:21 STEP 978 H= 978:00 +CPU= 28.687 23:19:39 STEP 979 H= 979:00 +CPU= 17.678 23:19:58 STEP 980 H= 980:00 +CPU= 18.412 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:21:12 STEP 981 H= 981:00 +CPU= 30.442 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:22:54 STEP 982 H= 982:00 +CPU= 19.873 23:23:14 STEP 983 H= 983:00 +CPU= 18.181 23:23:49 STEP 984 H= 984:00 +CPU= 32.651 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000984 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000984 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000972 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_40.zip Uploading the intermediate file: upload_file_40.zip 23:24:08 STEP 985 H= 985:00 +CPU= 17.678 23:24:28 STEP 986 H= 986:00 +CPU= 18.150 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:25:21 STEP 987 H= 987:00 +CPU= 29.815 23:25:42 STEP 988 H= 988:00 +CPU= 20.110 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:26:13 STEP 989 H= 989:00 +CPU= 17.989 23:26:43 STEP 990 H= 990:00 +CPU= 28.785 23:27:02 STEP 991 H= 991:00 +CPU= 17.722 23:27:23 STEP 992 H= 992:00 +CPU= 19.019 23:27:53 STEP 993 H= 993:00 +CPU= 28.585 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:28:26 STEP 994 H= 994:00 +CPU= 20.798 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:29:54 STEP 995 H= 995:00 +CPU= 20.456 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:30:38 STEP 996 H= 996:00 +CPU= 31.123 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000996 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000996 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000996 23:30:57 STEP 997 H= 997:00 +CPU= 17.675 23:31:17 STEP 998 H= 998:00 +CPU= 18.886 23:31:47 STEP 999 H= 999:00 +CPU= 28.359 23:32:07 STEP 1000 H=1000:00 +CPU= 18.996 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:32:49 STEP 1001 H=1001:00 +CPU= 20.027 23:33:19 STEP 1002 H=1002:00 +CPU= 28.167 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3547MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3547 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG_DRV [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CPG (1025) [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG_GP [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GPCTY [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VERINT [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233322:1674513202:3352.670] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VERINT_DGEMM_2 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233323:1674513203:3353.670] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:Olymp:1:1:2119:2119] [20230123:233323:1674513203:3353.671] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001711658 Unknown Unknown Unknown oifs_43r3_model.e 0000000000A3E48D verint_ 168 verint.F90 oifs_43r3_model.e 0000000000C6255C gpcty_ 278 gpcty.F90 oifs_43r3_model.e 0000000000C393EA cpg_gp_ 788 cpg_gp.F90 oifs_43r3_model.e 0000000000C2E72E cpg_ 627 cpg.F90 oifs_43r3_model.e 0000000000C2C638 cpg_drv_ 351 cpg_drv.F90 oifs_43r3_model.e 0000000000BF17C8 gp_model_ 419 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0056 (argv4) batchid: 960 (argv5) wuid: 12176700 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_07c619bc40a95c5326a942a5be913f56 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0056_1991050100_123_960_12176700.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0056_1991050100_123_960_12176700.zip ic_ancil_file: ic_ancil_12176700 ifsdata_file: ifsdata_12176700 climate_data_file: clim_data_12176700 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8d32566c8244c3ab6d6c331a922f27b4 to: /var/lib/boinc-client/slots/0/ic_ancil_12176700.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12176700.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_eaa1b009539d1fa9bae812c9228497ec to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12176700.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12176700.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12176700.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12176700.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12176700.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12176700.xml -- Model is restarting -- Adjusting last_iter, 1002, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12176700.xml last_cpu_time: 23364 upload_file_number: 41 last_iter: 983 last_upload: 3542400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028 The child process has been launched with process id: 2084 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:222615:1674595575:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:27:26 STEP 984 H= 984:00 +CPU= 32.497 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:28:56 STEP 985 H= 985:00 +CPU= 18.437 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:29:28 STEP 986 H= 986:00 +CPU= 19.533 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:30:11 STEP 987 H= 987:00 +CPU= 29.698 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:30:45 STEP 988 H= 988:00 +CPU= 20.413 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:31:15 STEP 989 H= 989:00 +CPU= 18.876 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:33:12 STEP 990 H= 990:00 +CPU= 31.286 22:33:32 STEP 991 H= 991:00 +CPU= 19.032 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:34:04 STEP 992 H= 992:00 +CPU= 19.884 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:34:58 STEP 993 H= 993:00 +CPU= 30.251 22:35:19 STEP 994 H= 994:00 +CPU= 19.299 22:35:39 STEP 995 H= 995:00 +CPU= 18.744 22:36:15 STEP 996 H= 996:00 +CPU= 33.752 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000996 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000996 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000996 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:36:46 STEP 997 H= 997:00 +CPU= 18.489 22:37:04 STEP 998 H= 998:00 +CPU= 17.779 22:37:34 STEP 999 H= 999:00 +CPU= 28.238 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:38:07 STEP 1000 H=1000:00 +CPU= 20.694 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:38:50 STEP 1001 H=1001:00 +CPU= 19.481 22:39:23 STEP 1002 H=1002:00 +CPU= 30.816 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:39:54 STEP 1003 H=1003:00 +CPU= 19.151 22:40:16 STEP 1004 H=1004:00 +CPU= 19.826 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:40:59 STEP 1005 H=1005:00 +CPU= 30.267 22:41:20 STEP 1006 H=1006:00 +CPU= 19.835 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:42:04 STEP 1007 H=1007:00 +CPU= 20.529 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:42:52 STEP 1008 H=1008:00 +CPU= 34.103 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001008 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001008 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+000996 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_41.zip Uploading the intermediate file: upload_file_41.zip 22:43:13 STEP 1009 H=1009:00 +CPU= 19.328 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:44:05 STEP 1010 H=1010:00 +CPU= 19.286 22:44:35 STEP 1011 H=1011:00 +CPU= 28.334 22:44:55 STEP 1012 H=1012:00 +CPU= 18.380 22:45:15 STEP 1013 H=1013:00 +CPU= 18.150 22:45:46 STEP 1014 H=1014:00 +CPU= 29.023 22:46:05 STEP 1015 H=1015:00 +CPU= 18.090 22:46:25 STEP 1016 H=1016:00 +CPU= 18.658 22:46:56 STEP 1017 H=1017:00 +CPU= 28.313 22:47:15 STEP 1018 H=1018:00 +CPU= 18.511 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:48:18 STEP 1019 H=1019:00 +CPU= 19.089 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:49:36 STEP 1020 H=1020:00 +CPU= 33.078 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001020 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001020 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001020 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:50:08 STEP 1021 H=1021:00 +CPU= 19.664 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:50:40 STEP 1022 H=1022:00 +CPU= 19.212 22:51:12 STEP 1023 H=1023:00 +CPU= 30.011 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:52:06 STEP 1024 H=1024:00 +CPU= 20.352 22:52:26 STEP 1025 H=1025:00 +CPU= 19.057 22:52:59 STEP 1026 H=1026:00 +CPU= 30.077 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:53:30 STEP 1027 H=1027:00 +CPU= 19.087 22:53:51 STEP 1028 H=1028:00 +CPU= 19.397 22:54:22 STEP 1029 H=1029:00 +CPU= 28.977 22:54:43 STEP 1030 H=1030:00 +CPU= 19.215 22:55:04 STEP 1031 H=1031:00 +CPU= 19.758 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:55:53 STEP 1032 H=1032:00 +CPU= 35.046 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001032 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001032 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+001020 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_42.zip Uploading the intermediate file: upload_file_42.zip 22:56:25 STEP 1033 H=1033:00 +CPU= 19.204 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:57:09 STEP 1034 H=1034:00 +CPU= 19.989 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:57:53 STEP 1035 H=1035:00 +CPU= 30.235 22:58:14 STEP 1036 H=1036:00 +CPU= 20.010 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:58:56 STEP 1037 H=1037:00 +CPU= 18.892 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:59:52 STEP 1038 H=1038:00 +CPU= 31.452 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:00:24 STEP 1039 H=1039:00 +CPU= 19.532 23:00:45 STEP 1040 H=1040:00 +CPU= 20.053 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:03:10 STEP 1041 H=1041:00 +CPU= 29.504 23:03:28 STEP 1042 H=1042:00 +CPU= 17.853 23:03:47 STEP 1043 H=1043:00 +CPU= 17.459 23:04:18 STEP 1044 H=1044:00 +CPU= 30.362 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001044 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001044 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001044 23:04:38 STEP 1045 H=1045:00 +CPU= 17.892 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:05:21 STEP 1046 H=1046:00 +CPU= 19.067 23:05:51 STEP 1047 H=1047:00 +CPU= 27.552 23:06:09 STEP 1048 H=1048:00 +CPU= 17.788 23:06:27 STEP 1049 H=1049:00 +CPU= 17.091 23:06:56 STEP 1050 H=1050:00 +CPU= 27.960 23:07:14 STEP 1051 H=1051:00 +CPU= 17.338 23:07:33 STEP 1052 H=1052:00 +CPU= 18.006 23:08:02 STEP 1053 H=1053:00 +CPU= 27.247 23:08:20 STEP 1054 H=1054:00 +CPU= 17.864 23:08:39 STEP 1055 H=1055:00 +CPU= 18.006 23:09:13 STEP 1056 H=1056:00 +CPU= 32.259 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001056 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001056 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+001044 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+001044 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+001044 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_43.zip Uploading the intermediate file: upload_file_43.zip 23:09:32 STEP 1057 H=1057:00 +CPU= 18.256 23:09:51 STEP 1058 H=1058:00 +CPU= 18.204 23:10:20 STEP 1059 H=1059:00 +CPU= 27.752 23:10:38 STEP 1060 H=1060:00 +CPU= 17.756 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.255] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 2550MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.258] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.258] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.258] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.258] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.258] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 2598 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.258] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WVCOUPLE [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WAVEMDL [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WAMODEL [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231056:1674598256:2681.282] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-WAMODEL 4 (1439) [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231057:1674598257:2682.282] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:Olymp:1:1:2084:2084] [20230124:231057:1674598257:2682.282] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DAFCE2 Unknown Unknown Unknown oifs_43r3_model.e 000000000168DC46 wamodel_ 1115 wamodel.F oifs_43r3_model.e 0000000000872295 wavemdl_ 584 wavemdl.F oifs_43r3_model.e 00000000007E0165 wvcouple_ 260 wvcouple.F90 oifs_43r3_model.e 000000000055E4AD cnt4_ 1283 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0056 (argv4) batchid: 960 (argv5) wuid: 12176700 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_07c619bc40a95c5326a942a5be913f56 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0056_1991050100_123_960_12176700.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0056_1991050100_123_960_12176700.zip ic_ancil_file: ic_ancil_12176700 ifsdata_file: ifsdata_12176700 climate_data_file: clim_data_12176700 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8d32566c8244c3ab6d6c331a922f27b4 to: /var/lib/boinc-client/slots/0/ic_ancil_12176700.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12176700.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_eaa1b009539d1fa9bae812c9228497ec to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12176700.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12176700.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12176700.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12176700.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12176700.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12176700.xml -- Model is restarting -- Adjusting last_iter, 1061, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12176700.xml last_cpu_time: 25128 upload_file_number: 44 last_iter: 1055 last_upload: 3801600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028 The child process has been launched with process id: 2070 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Olymp:1:1:2070:2070] [20230125:132144:1674649304:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:23:09 STEP 1056 H=1056:00 +CPU= 34.512 13:23:26 STEP 1057 H=1057:00 +CPU= 17.063 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:24:50 STEP 1058 H=1058:00 +CPU= 19.907 13:25:18 STEP 1059 H=1059:00 +CPU= 26.984 13:25:36 STEP 1060 H=1060:00 +CPU= 17.789 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:26:08 STEP 1061 H=1061:00 +CPU= 18.991 13:26:38 STEP 1062 H=1062:00 +CPU= 28.917 13:26:56 STEP 1063 H=1063:00 +CPU= 17.203 13:27:14 STEP 1064 H=1064:00 +CPU= 17.586 13:27:42 STEP 1065 H=1065:00 +CPU= 27.133 13:28:00 STEP 1066 H=1066:00 +CPU= 17.601 13:28:18 STEP 1067 H=1067:00 +CPU= 17.136 13:28:49 STEP 1068 H=1068:00 +CPU= 29.863 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001068 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001068 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001068 13:29:07 STEP 1069 H=1069:00 +CPU= 17.144 13:29:25 STEP 1070 H=1070:00 +CPU= 17.647 13:29:53 STEP 1071 H=1071:00 +CPU= 27.062 13:30:12 STEP 1072 H=1072:00 +CPU= 18.111 13:30:31 STEP 1073 H=1073:00 +CPU= 17.817 13:31:01 STEP 1074 H=1074:00 +CPU= 28.312 13:31:19 STEP 1075 H=1075:00 +CPU= 17.426 13:31:38 STEP 1076 H=1076:00 +CPU= 17.983 13:32:07 STEP 1077 H=1077:00 +CPU= 27.688 13:32:26 STEP 1078 H=1078:00 +CPU= 18.310 13:32:45 STEP 1079 H=1079:00 +CPU= 18.028 13:33:19 STEP 1080 H=1080:00 +CPU= 32.216 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001080 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001080 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001080 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMGGhq0f+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMSHhq0f+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176700/ICMUAhq0f+001068 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0056_1991050100_123_960_12176700_0_r936815028_44.zip Uploading the intermediate file: upload_file_44.zip 13:33:37 STEP 1081 H=1081:00 +CPU= 17.459 13:33:55 STEP 1082 H=1082:00 +CPU= 17.864 13:34:24 STEP 1083 H=1083:00 +CPU= 27.541 13:34:43 STEP 1084 H=1084:00 +CPU= 17.816 13:35:01 STEP 1085 H=1085:00 +CPU= 17.485 13:35:31 STEP 1086 H=1086:00 +CPU= 28.350 13:35:49 STEP 1087 H=1087:00 +CPU= 17.312 13:36:07 STEP 1088 H=1088:00 +CPU= 17.870 13:36:36 STEP 1089 H=1089:00 +CPU= 27.341 Abort request received from BOINC client, ending the child process </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:31:34 | 1509861 | 22261460 | oifs_43r3_ps_0056_1991050100_123_960_12176700_0 | 864,000 | 5,275 | 0.0061 |
©2024 cpdn.org