Name | oifs_43r3_bl_a008_2016092300_15_949_12166402_0 |
Workunit | 12166402 |
Created | 14 Dec 2022, 13:58:46 UTC |
Sent | 14 Dec 2022, 13:59:30 UTC |
Report deadline | 13 Jan 2023, 13:59:30 UTC |
Received | 16 Dec 2022, 1:17:41 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1520317 |
Run time | 12 hours 53 min 9 sec |
CPU time | 12 hours 17 min 15 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 8.61 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.07 x86_64-pc-linux-gnu |
Peak working set size | 5,056.30 MB |
Peak swap size | 5,930.81 MB |
Peak disk usage | 6,465.32 MB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> n: 1.07 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_1cac76e188de84702b879efd14204636 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip ic_ancil_file: ic_ancil_12166402 ifsdata_file: ifsdata_12166402 climate_data_file: clim_data_12166402 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_30f947d694b92b83f2821852fc0c9fec to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12166402.xml -- Model is restarting -- Adjusting last_iter, 1010, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml last_cpu_time: 35720 upload_file_number: 10 last_iter: 1007 last_upload: 864000 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904 The child process has been launched with process id: 187185 Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 16:32:26 STEP 1008 H= 252:00 +CPU= 36.416 16:33:15 STEP 1009 H= 252:15 +CPU= 46.352 16:34:04 STEP 1010 H= 252:30 +CPU= 47.020 16:34:51 STEP 1011 H= 252:45 +CPU= 45.055 16:35:43 STEP 1012 H= 253:00 +CPU= 49.732 16:36:36 STEP 1013 H= 253:15 +CPU= 49.812 16:37:28 STEP 1014 H= 253:30 +CPU= 50.926 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:47:53 STEP 1015 H= 253:45 +CPU= 48.297 16:48:51 STEP 1016 H= 254:00 +CPU= 53.513 16:49:39 STEP 1017 H= 254:15 +CPU= 46.482 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:58:51 STEP 1018 H= 254:30 +CPU= 49.067 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:12:23 STEP 1019 H= 254:45 +CPU= 51.237 17:13:37 STEP 1020 H= 255:00 +CPU= 62.792 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001020 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001020 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001020 17:14:38 STEP 1021 H= 255:15 +CPU= 52.881 17:15:36 STEP 1022 H= 255:30 +CPU= 51.228 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:20:37 STEP 1023 H= 255:45 +CPU= 50.107 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:27:23 STEP 1024 H= 256:00 +CPU= 46.475 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:36:18 STEP 1025 H= 256:15 +CPU= 44.773 17:36:54 STEP 1026 H= 256:30 +CPU= 30.196 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:49:26 STEP 1027 H= 256:45 +CPU= 30.050 17:50:03 STEP 1028 H= 257:00 +CPU= 31.744 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:56:54 STEP 1029 H= 257:15 +CPU= 30.563 17:57:31 STEP 1030 H= 257:30 +CPU= 34.956 17:57:58 STEP 1031 H= 257:45 +CPU= 26.439 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:00:36 STEP 1032 H= 258:00 +CPU= 34.927 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001032 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001032 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001032 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:06:30 STEP 1033 H= 258:15 +CPU= 33.637 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:20:40 STEP 1034 H= 258:30 +CPU= 29.706 18:21:15 STEP 1035 H= 258:45 +CPU= 31.989 18:21:52 STEP 1036 H= 259:00 +CPU= 33.872 18:22:28 STEP 1037 H= 259:15 +CPU= 32.003 18:23:01 STEP 1038 H= 259:30 +CPU= 31.253 18:23:40 STEP 1039 H= 259:45 +CPU= 34.279 18:24:16 STEP 1040 H= 260:00 +CPU= 32.858 18:24:51 STEP 1041 H= 260:15 +CPU= 31.399 18:25:30 STEP 1042 H= 260:30 +CPU= 35.976 18:26:10 STEP 1043 H= 260:45 +CPU= 36.070 18:26:59 STEP 1044 H= 261:00 +CPU= 45.400 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001044 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001044 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001044 18:27:36 STEP 1045 H= 261:15 +CPU= 34.628 18:28:18 STEP 1046 H= 261:30 +CPU= 38.237 18:29:01 STEP 1047 H= 261:45 +CPU= 38.090 18:29:44 STEP 1048 H= 262:00 +CPU= 38.646 18:30:23 STEP 1049 H= 262:15 +CPU= 35.717 18:31:04 STEP 1050 H= 262:30 +CPU= 36.131 18:31:46 STEP 1051 H= 262:45 +CPU= 39.254 18:32:31 STEP 1052 H= 263:00 +CPU= 39.858 18:33:12 STEP 1053 H= 263:15 +CPU= 36.541 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:35:56 STEP 1054 H= 263:30 +CPU= 35.263 18:36:42 STEP 1055 H= 263:45 +CPU= 40.312 18:37:37 STEP 1056 H= 264:00 +CPU= 50.133 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001056 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001056 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001044 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001044 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001044 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904_10.zip Uploading the intermediate file: upload_file_10.zip 18:38:17 STEP 1057 H= 264:15 +CPU= 35.476 18:38:54 STEP 1058 H= 264:30 +CPU= 34.090 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:49:54 STEP 1059 H= 264:45 +CPU= 39.229 18:50:39 STEP 1060 H= 265:00 +CPU= 41.500 18:51:17 STEP 1061 H= 265:15 +CPU= 35.331 18:51:55 STEP 1062 H= 265:30 +CPU= 35.425 18:52:34 STEP 1063 H= 265:45 +CPU= 36.086 18:53:13 STEP 1064 H= 266:00 +CPU= 36.277 18:53:53 STEP 1065 H= 266:15 +CPU= 37.718 18:54:35 STEP 1066 H= 266:30 +CPU= 39.196 18:55:13 STEP 1067 H= 266:45 +CPU= 35.156 18:55:58 STEP 1068 H= 267:00 +CPU= 41.956 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001068 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001068 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001068 18:56:41 STEP 1069 H= 267:15 +CPU= 40.242 18:57:20 STEP 1070 H= 267:30 +CPU= 37.710 18:58:03 STEP 1071 H= 267:45 +CPU= 40.159 18:58:43 STEP 1072 H= 268:00 +CPU= 37.856 18:59:24 STEP 1073 H= 268:15 +CPU= 38.339 19:00:07 STEP 1074 H= 268:30 +CPU= 41.378 19:00:46 STEP 1075 H= 268:45 +CPU= 36.402 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a008 (argv4) batchid: 949 (argv5) wuid: 12166402 (argv6) fclen: 15 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/14 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.07 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_1cac76e188de84702b879efd14204636 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip ic_ancil_file: ic_ancil_12166402 ifsdata_file: ifsdata_12166402 climate_data_file: clim_data_12166402 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_30f947d694b92b83f2821852fc0c9fec to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12166402.xml -- Model is restarting -- Adjusting last_iter, 1076, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml last_cpu_time: 38437 upload_file_number: 11 last_iter: 1055 last_upload: 950400 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904 The child process has been launched with process id: 219518 Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 19:03:05 STEP 1056 H= 264:00 +CPU= 37.603 19:03:43 STEP 1057 H= 264:15 +CPU= 35.370 19:04:19 STEP 1058 H= 264:30 +CPU= 35.362 19:04:52 STEP 1059 H= 264:45 +CPU= 31.617 19:05:28 STEP 1060 H= 265:00 +CPU= 34.301 19:06:00 STEP 1061 H= 265:15 +CPU= 30.511 19:06:40 STEP 1062 H= 265:30 +CPU= 38.444 19:07:14 STEP 1063 H= 265:45 +CPU= 32.350 19:07:50 STEP 1064 H= 266:00 +CPU= 33.931 19:08:24 STEP 1065 H= 266:15 +CPU= 32.182 19:09:04 STEP 1066 H= 266:30 +CPU= 37.531 19:09:35 STEP 1067 H= 266:45 +CPU= 29.264 19:10:17 STEP 1068 H= 267:00 +CPU= 39.307 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001068 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001068 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001068 19:10:52 STEP 1069 H= 267:15 +CPU= 33.840 19:11:29 STEP 1070 H= 267:30 +CPU= 35.318 19:12:00 STEP 1071 H= 267:45 +CPU= 30.048 19:12:28 STEP 1072 H= 268:00 +CPU= 27.664 19:13:02 STEP 1073 H= 268:15 +CPU= 31.973 19:13:32 STEP 1074 H= 268:30 +CPU= 28.005 19:14:07 STEP 1075 H= 268:45 +CPU= 33.745 19:14:35 STEP 1076 H= 269:00 +CPU= 26.518 19:15:02 STEP 1077 H= 269:15 +CPU= 25.226 19:15:27 STEP 1078 H= 269:30 +CPU= 23.679 19:15:51 STEP 1079 H= 269:45 +CPU= 23.365 19:16:21 STEP 1080 H= 270:00 +CPU= 28.869 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001080 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001080 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001080 19:16:44 STEP 1081 H= 270:15 +CPU= 23.018 19:17:07 STEP 1082 H= 270:30 +CPU= 21.875 19:17:27 STEP 1083 H= 270:45 +CPU= 19.757 19:17:50 STEP 1084 H= 271:00 +CPU= 23.352 19:18:13 STEP 1085 H= 271:15 +CPU= 22.534 19:18:39 STEP 1086 H= 271:30 +CPU= 23.827 19:19:01 STEP 1087 H= 271:45 +CPU= 22.286 19:19:28 STEP 1088 H= 272:00 +CPU= 25.286 19:19:51 STEP 1089 H= 272:15 +CPU= 22.232 19:20:19 STEP 1090 H= 272:30 +CPU= 25.991 19:20:44 STEP 1091 H= 272:45 +CPU= 24.328 19:21:11 STEP 1092 H= 273:00 +CPU= 26.456 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001092 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001092 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001092 19:21:36 STEP 1093 H= 273:15 +CPU= 24.960 19:21:57 STEP 1094 H= 273:30 +CPU= 20.102 19:22:20 STEP 1095 H= 273:45 +CPU= 21.939 19:22:43 STEP 1096 H= 274:00 +CPU= 21.951 19:23:09 STEP 1097 H= 274:15 +CPU= 23.530 19:23:33 STEP 1098 H= 274:30 +CPU= 22.770 19:24:00 STEP 1099 H= 274:45 +CPU= 24.606 19:24:22 STEP 1100 H= 275:00 +CPU= 22.046 19:24:47 STEP 1101 H= 275:15 +CPU= 23.635 19:25:09 STEP 1102 H= 275:30 +CPU= 21.640 19:25:29 STEP 1103 H= 275:45 +CPU= 20.270 19:25:58 STEP 1104 H= 276:00 +CPU= 28.423 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001104 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001104 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001104 19:26:21 STEP 1105 H= 276:15 +CPU= 21.993 19:26:43 STEP 1106 H= 276:30 +CPU= 21.062 19:27:03 STEP 1107 H= 276:45 +CPU= 20.109 19:27:26 STEP 1108 H= 277:00 +CPU= 22.208 19:27:47 STEP 1109 H= 277:15 +CPU= 21.112 19:28:09 STEP 1110 H= 277:30 +CPU= 20.962 19:28:30 STEP 1111 H= 277:45 +CPU= 19.897 19:28:52 STEP 1112 H= 278:00 +CPU= 21.512 19:29:13 STEP 1113 H= 278:15 +CPU= 21.302 19:29:39 STEP 1114 H= 278:30 +CPU= 24.702 19:30:10 STEP 1115 H= 278:45 +CPU= 28.683 19:30:40 STEP 1116 H= 279:00 +CPU= 30.151 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001116 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001116 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001116 19:31:10 STEP 1117 H= 279:15 +CPU= 28.324 19:31:36 STEP 1118 H= 279:30 +CPU= 25.078 19:31:58 STEP 1119 H= 279:45 +CPU= 21.917 19:32:20 STEP 1120 H= 280:00 +CPU= 22.178 19:32:49 STEP 1121 H= 280:15 +CPU= 28.621 19:33:15 STEP 1122 H= 280:30 +CPU= 25.494 19:33:41 STEP 1123 H= 280:45 +CPU= 25.778 19:34:07 STEP 1124 H= 281:00 +CPU= 25.450 19:34:31 STEP 1125 H= 281:15 +CPU= 23.630 19:34:55 STEP 1126 H= 281:30 +CPU= 23.572 19:35:18 STEP 1127 H= 281:45 +CPU= 23.263 19:35:56 STEP 1128 H= 282:00 +CPU= 35.671 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001128 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001128 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001128 19:36:24 STEP 1129 H= 282:15 +CPU= 26.441 19:36:56 STEP 1130 H= 282:30 +CPU= 31.054 19:37:25 STEP 1131 H= 282:45 +CPU= 27.181 19:37:54 STEP 1132 H= 283:00 +CPU= 27.775 19:38:19 STEP 1133 H= 283:15 +CPU= 25.302 19:38:44 STEP 1134 H= 283:30 +CPU= 24.235 19:39:15 STEP 1135 H= 283:45 +CPU= 30.780 19:39:44 STEP 1136 H= 284:00 +CPU= 27.912 19:40:12 STEP 1137 H= 284:15 +CPU= 26.938 19:40:38 STEP 1138 H= 284:30 +CPU= 26.704 19:41:02 STEP 1139 H= 284:45 +CPU= 23.178 19:41:35 STEP 1140 H= 285:00 +CPU= 32.572 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001140 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001140 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001140 19:42:06 STEP 1141 H= 285:15 +CPU= 30.367 19:42:35 STEP 1142 H= 285:30 +CPU= 28.185 19:43:03 STEP 1143 H= 285:45 +CPU= 26.619 19:43:35 STEP 1144 H= 286:00 +CPU= 30.954 19:44:03 STEP 1145 H= 286:15 +CPU= 27.920 19:44:30 STEP 1146 H= 286:30 +CPU= 26.499 19:44:58 STEP 1147 H= 286:45 +CPU= 27.891 19:45:29 STEP 1148 H= 287:00 +CPU= 30.128 19:45:57 STEP 1149 H= 287:15 +CPU= 27.971 19:46:25 STEP 1150 H= 287:30 +CPU= 26.921 19:46:51 STEP 1151 H= 287:45 +CPU= 26.014 19:47:34 STEP 1152 H= 288:00 +CPU= 39.684 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001152 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001152 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001152 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001080 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001080 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001080 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001092 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001092 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001092 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001104 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001104 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001104 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001116 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001116 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001116 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001128 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001128 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001128 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001140 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001140 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001140 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904_11.zip Uploading the intermediate file: upload_file_11.zip 19:48:01 STEP 1153 H= 288:15 +CPU= 25.080 19:48:24 STEP 1154 H= 288:30 +CPU= 22.347 19:48:47 STEP 1155 H= 288:45 +CPU= 22.582 19:49:16 STEP 1156 H= 289:00 +CPU= 26.877 19:49:41 STEP 1157 H= 289:15 +CPU= 23.725 19:50:11 STEP 1158 H= 289:30 +CPU= 28.334 19:50:38 STEP 1159 H= 289:45 +CPU= 26.421 19:51:07 STEP 1160 H= 290:00 +CPU= 28.888 19:51:35 STEP 1161 H= 290:15 +CPU= 25.890 19:52:02 STEP 1162 H= 290:30 +CPU= 26.053 19:52:31 STEP 1163 H= 290:45 +CPU= 27.335 19:53:08 STEP 1164 H= 291:00 +CPU= 36.416 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001164 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001164 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001164 19:53:36 STEP 1165 H= 291:15 +CPU= 26.980 19:53:59 STEP 1166 H= 291:30 +CPU= 22.545 19:54:29 STEP 1167 H= 291:45 +CPU= 28.477 19:54:56 STEP 1168 H= 292:00 +CPU= 25.582 19:55:23 STEP 1169 H= 292:15 +CPU= 25.863 19:55:51 STEP 1170 H= 292:30 +CPU= 26.960 19:56:19 STEP 1171 H= 292:45 +CPU= 26.339 19:56:46 STEP 1172 H= 293:00 +CPU= 25.902 19:57:16 STEP 1173 H= 293:15 +CPU= 28.525 19:57:45 STEP 1174 H= 293:30 +CPU= 27.383 19:58:12 STEP 1175 H= 293:45 +CPU= 26.244 19:58:52 STEP 1176 H= 294:00 +CPU= 37.871 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001176 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001176 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001176 19:59:21 STEP 1177 H= 294:15 +CPU= 28.198 19:59:48 STEP 1178 H= 294:30 +CPU= 25.592 20:00:19 STEP 1179 H= 294:45 +CPU= 29.990 20:00:45 STEP 1180 H= 295:00 +CPU= 25.977 20:01:20 STEP 1181 H= 295:15 +CPU= 35.180 20:01:49 STEP 1182 H= 295:30 +CPU= 27.236 20:02:15 STEP 1183 H= 295:45 +CPU= 25.332 20:02:40 STEP 1184 H= 296:00 +CPU= 24.082 20:03:04 STEP 1185 H= 296:15 +CPU= 23.008 20:03:30 STEP 1186 H= 296:30 +CPU= 25.009 20:03:55 STEP 1187 H= 296:45 +CPU= 24.104 20:04:26 STEP 1188 H= 297:00 +CPU= 29.009 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001188 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001188 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001188 20:04:49 STEP 1189 H= 297:15 +CPU= 22.031 20:05:12 STEP 1190 H= 297:30 +CPU= 22.081 20:05:38 STEP 1191 H= 297:45 +CPU= 26.369 20:06:05 STEP 1192 H= 298:00 +CPU= 25.223 20:06:30 STEP 1193 H= 298:15 +CPU= 24.024 20:06:55 STEP 1194 H= 298:30 +CPU= 24.053 20:07:21 STEP 1195 H= 298:45 +CPU= 24.215 20:07:48 STEP 1196 H= 299:00 +CPU= 25.540 20:08:13 STEP 1197 H= 299:15 +CPU= 23.387 20:08:40 STEP 1198 H= 299:30 +CPU= 25.916 20:09:04 STEP 1199 H= 299:45 +CPU= 23.634 20:09:40 STEP 1200 H= 300:00 +CPU= 33.804 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001200 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001200 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001200 20:10:05 STEP 1201 H= 300:15 +CPU= 24.364 20:10:31 STEP 1202 H= 300:30 +CPU= 24.549 20:10:56 STEP 1203 H= 300:45 +CPU= 24.690 20:11:22 STEP 1204 H= 301:00 +CPU= 25.743 20:11:48 STEP 1205 H= 301:15 +CPU= 25.018 20:12:15 STEP 1206 H= 301:30 +CPU= 25.210 20:12:42 STEP 1207 H= 301:45 +CPU= 26.296 Suspend request received from the BOINC client, suspending the child process [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 5397MB (heap), 4897MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x80f380 (old at (nil)) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3843] 5397 MB (maxheap), 4897 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CALL_SL [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 1 (1004) [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LAPINEA [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LARCINA [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LASCAW [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201454:1671095694:4357.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:roo:1:1:219518:219518] [20221215:201454:1671095694:4357.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 000000000080FFB1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001D8CDC0 Unknown Unknown Unknown oifs_43r3_model.e 00000000011D2146 lascaw_ 611 lascaw.F90 oifs_43r3_model.e 00000000011CB5C5 larcina_ 319 larcina.F90 oifs_43r3_model.e 00000000011C416D lapinea_ 325 lapinea.F90 oifs_43r3_model.e 00000000011BB1CC call_sl_ 300 call_sl.F90 oifs_43r3_model.e 000000000139E65D call_sl_heap_ 94 call_sl_heap.F90 oifs_43r3_model.e 0000000000D53278 gp_model_ 533 gp_model.F90 oifs_43r3_model.e 0000000000D3FB32 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 00000000014A1C3E scan2m_ 535 scan2m.F90 oifs_43r3_model.e 00000000006AD494 stepo_ 327 stepo.F90 oifs_43r3_model.e 00000000005750DC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 000000000055E289 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 000000000055D452 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000546AC8 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 00000000004078AD cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040276F MAIN__ 96 master.F90 oifs_43r3_model.e 000000000040270D Unknown Unknown Unknown oifs_43r3_model.e 0000000001D83B1A Unknown Unknown Unknown oifs_43r3_model.e 0000000001D85377 Unknown Unknown Unknown oifs_43r3_model.e 00000000004025E5 Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a008 (argv4) batchid: 949 (argv5) wuid: 12166402 (argv6) fclen: 15 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/14 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.07 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_1cac76e188de84702b879efd14204636 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip ic_ancil_file: ic_ancil_12166402 ifsdata_file: ifsdata_12166402 climate_data_file: clim_data_12166402 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_30f947d694b92b83f2821852fc0c9fec to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12166402.xml -- Model is restarting -- Adjusting last_iter, 1208, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml last_cpu_time: 42506 upload_file_number: 12 last_iter: 1199 last_upload: 1036800 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904 The child process has been launched with process id: 6291 Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil)) [EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 11:20:40 STEP 1200 H= 300:00 +CPU= 23.917 11:21:19 STEP 1201 H= 300:15 +CPU= 36.826 11:21:51 STEP 1202 H= 300:30 +CPU= 31.651 11:22:29 STEP 1203 H= 300:45 +CPU= 36.471 11:23:02 STEP 1204 H= 301:00 +CPU= 30.760 11:23:27 STEP 1205 H= 301:15 +CPU= 24.166 11:23:53 STEP 1206 H= 301:30 +CPU= 25.251 11:24:26 STEP 1207 H= 301:45 +CPU= 32.429 11:25:01 STEP 1208 H= 302:00 +CPU= 34.462 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:32:09 STEP 1209 H= 302:15 +CPU= 27.921 11:32:46 STEP 1210 H= 302:30 +CPU= 33.579 11:33:22 STEP 1211 H= 302:45 +CPU= 32.640 11:34:07 STEP 1212 H= 303:00 +CPU= 40.928 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001212 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001212 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001212 11:34:46 STEP 1213 H= 303:15 +CPU= 35.514 11:35:22 STEP 1214 H= 303:30 +CPU= 33.107 11:35:57 STEP 1215 H= 303:45 +CPU= 33.149 11:36:36 STEP 1216 H= 304:00 +CPU= 35.608 11:37:10 STEP 1217 H= 304:15 +CPU= 31.033 11:37:45 STEP 1218 H= 304:30 +CPU= 32.983 11:38:22 STEP 1219 H= 304:45 +CPU= 34.865 11:39:04 STEP 1220 H= 305:00 +CPU= 37.816 11:39:38 STEP 1221 H= 305:15 +CPU= 31.693 11:40:15 STEP 1222 H= 305:30 +CPU= 34.608 11:40:53 STEP 1223 H= 305:45 +CPU= 35.185 11:41:35 STEP 1224 H= 306:00 +CPU= 39.223 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001224 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001224 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001224 11:42:12 STEP 1225 H= 306:15 +CPU= 35.144 11:42:48 STEP 1226 H= 306:30 +CPU= 34.565 11:43:24 STEP 1227 H= 306:45 +CPU= 34.730 11:44:01 STEP 1228 H= 307:00 +CPU= 35.589 11:44:37 STEP 1229 H= 307:15 +CPU= 34.759 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:47:25 STEP 1230 H= 307:30 +CPU= 30.013 11:47:58 STEP 1231 H= 307:45 +CPU= 30.191 11:48:36 STEP 1232 H= 308:00 +CPU= 35.253 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:51:12 STEP 1233 H= 308:15 +CPU= 33.937 11:51:46 STEP 1234 H= 308:30 +CPU= 31.864 11:52:22 STEP 1235 H= 308:45 +CPU= 34.058 11:53:07 STEP 1236 H= 309:00 +CPU= 41.385 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001236 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001236 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001236 11:53:45 STEP 1237 H= 309:15 +CPU= 35.418 11:54:19 STEP 1238 H= 309:30 +CPU= 31.224 11:54:55 STEP 1239 H= 309:45 +CPU= 33.714 11:55:32 STEP 1240 H= 310:00 +CPU= 34.576 11:56:07 STEP 1241 H= 310:15 +CPU= 32.961 11:56:45 STEP 1242 H= 310:30 +CPU= 35.197 11:57:25 STEP 1243 H= 310:45 +CPU= 36.260 11:58:01 STEP 1244 H= 311:00 +CPU= 32.523 11:58:39 STEP 1245 H= 311:15 +CPU= 34.647 11:59:16 STEP 1246 H= 311:30 +CPU= 34.262 11:59:50 STEP 1247 H= 311:45 +CPU= 32.474 12:00:38 STEP 1248 H= 312:00 +CPU= 43.586 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001248 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001248 Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001248 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001152 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001152 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001152 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001164 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001164 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001164 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001176 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001176 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001176 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001188 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001188 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001188 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001200 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001200 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001200 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001212 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001212 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001212 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001224 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001224 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001224 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001236 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001236 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001236 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904_12.zip Uploading the intermediate file: upload_file_12.zip 12:01:19 STEP 1249 H= 312:15 +CPU= 36.569 12:01:54 STEP 1250 H= 312:30 +CPU= 33.945 Abort request received from BOINC client, ending the child process ..Failed, model did not complete successfully </stderr_txt> ]]> |
No trickles! |
---|
©2024 cpdn.org