Name | oifs_43r3_ps_0785_2006050100_123_975_12192429_1 |
Workunit | 12192429 |
Created | 24 Jan 2023, 8:57:43 UTC |
Sent | 24 Jan 2023, 9:18:48 UTC |
Report deadline | 25 Mar 2023, 9:18:48 UTC |
Received | 26 Jan 2023, 7:58:44 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1499379 |
Run time | 7 hours 40 min 10 sec |
CPU time | 51 min 47 sec |
Validate state | Invalid |
Credit | 9,192.96 |
Device peak FLOPS | 7.81 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,788.25 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,810.55 MB |
Stderr | <core_client_version>7.20.4</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> e_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG_DRV [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CPG (1025) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG_DIA [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082153:1674717713:225.423] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPCUDDH [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082154:1674717714:226.423] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4617:4617] [20230126:082154:1674717714:226.423] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000000CD52FD cpcuddh_ 293 cpcuddh.F90 oifs_43r3_model.e 0000000000C8B2A6 cpg_dia_ 723 cpg_dia.F90 oifs_43r3_model.e 0000000000C2F59F cpg_ 718 cpg.F90 oifs_43r3_model.e 0000000000C2C638 cpg_drv_ 351 cpg_drv.F90 oifs_43r3_model.e 0000000000BF17C8 gp_model_ 419 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2006050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0785 (argv4) batchid: 975 (argv5) wuid: 12192429 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /mnt/nvme0n1p7/Boinc/data/slots/5 Project directory is: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429 ..mkdir for temp folder for results failed Copying: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_cee3df04c6ee9d6801ddeeebe7239575 to: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_0785_2006050100_123_975_12192429.zip Unzipping the namelist zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_0785_2006050100_123_975_12192429.zip ic_ancil_file: ic_ancil_12192429 ifsdata_file: ifsdata_12192429 climate_data_file: clim_data_12192429 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8cd49e9033a8c67853faa2bb255fcad4 to: /mnt/nvme0n1p7/Boinc/data/slots/5/ic_ancil_12192429.zip Unzipping the IC ancils zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/ic_ancil_12192429.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8b30c0ac1169461b32faa88f01e1adef to: /mnt/nvme0n1p7/Boinc/data/slots/5/ifsdata/ifsdata_12192429.zip Unzipping the ifsdata_zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/ifsdata/ifsdata_12192429.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/nvme0n1p7/Boinc/data/slots/5/159l_2/clim_data_12192429.zip Unzipping the climate data zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/159l_2/clim_data_12192429.zip Checking for progress XML file: /mnt/nvme0n1p7/Boinc/data/slots/5/progress_file_12192429.xml Opened progress file ok : /mnt/nvme0n1p7/Boinc/data/slots/5/progress_file_12192429.xml -- Model is restarting -- Adjusting last_iter, 2168, to previous model restart step. Creating progress file: /mnt/nvme0n1p7/Boinc/data/slots/5/progress_file_12192429.xml last_cpu_time: 2317 upload_file_number: 7 last_iter: 2159 last_upload: 7776000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0785_2006050100_123_975_12192429_1_r221742427 The child process has been launched with process id: 10464 Executing the command: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082359:1674717839:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:24:37 STEP 2160 H=2160:00 +CPU= 21.078 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:25:15 STEP 2161 H=2161:00 +CPU= 10.404 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:25:31 STEP 2162 H=2162:00 +CPU= 9.950 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:25:57 STEP 2163 H=2163:00 +CPU= 15.607 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:26:15 STEP 2164 H=2164:00 +CPU= 9.607 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:26:32 STEP 2165 H=2165:00 +CPU= 9.440 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process double free or corruption (top) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3993MB (heap), 4783MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3993 MB (maxheap), 4783 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS CLDPP T/S (1002) [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TURBULENCE_LAYER [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFOUTER [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFMAIN [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFDIFH [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SURFSEB [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082649:1674718009:170.399] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SURFSEB_CTL_MOD:SURFSEB_CTL [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082650:1674718010:171.399] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:10464:10464] [20230126:082650:1674718010:171.399] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001CBC54D Unknown Unknown Unknown oifs_43r3_model.e 0000000001739466 surfseb_ctl_mod_m 473 surfseb_ctl_mod.F90 oifs_43r3_model.e 00000000015CDE79 surfseb_ 391 surfseb.F90 oifs_43r3_model.e 0000000000FC04A6 vdfdifh_ 444 vdfdifh.F90 oifs_43r3_model.e 0000000000FADFE1 vdfmain_ 1171 vdfmain.F90 oifs_43r3_model.e 0000000000F92699 vdfouter_ 739 vdfouter.F90 oifs_43r3_model.e 0000000000F88512 turbulence_layer_ 158 turbulence_layer.F90 oifs_43r3_model.e 0000000000F0B683 callpar_ 869 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2006050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0785 (argv4) batchid: 975 (argv5) wuid: 12192429 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /mnt/nvme0n1p7/Boinc/data/slots/5 Project directory is: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429 ..mkdir for temp folder for results failed Copying: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_cee3df04c6ee9d6801ddeeebe7239575 to: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_0785_2006050100_123_975_12192429.zip Unzipping the namelist zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_ps_0785_2006050100_123_975_12192429.zip ic_ancil_file: ic_ancil_12192429 ifsdata_file: ifsdata_12192429 climate_data_file: clim_data_12192429 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8cd49e9033a8c67853faa2bb255fcad4 to: /mnt/nvme0n1p7/Boinc/data/slots/5/ic_ancil_12192429.zip Unzipping the IC ancils zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/ic_ancil_12192429.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8b30c0ac1169461b32faa88f01e1adef to: /mnt/nvme0n1p7/Boinc/data/slots/5/ifsdata/ifsdata_12192429.zip Unzipping the ifsdata_zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/ifsdata/ifsdata_12192429.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/nvme0n1p7/Boinc/data/slots/5/159l_2/clim_data_12192429.zip Unzipping the climate data zip file: /mnt/nvme0n1p7/Boinc/data/slots/5/159l_2/clim_data_12192429.zip Checking for progress XML file: /mnt/nvme0n1p7/Boinc/data/slots/5/progress_file_12192429.xml Opened progress file ok : /mnt/nvme0n1p7/Boinc/data/slots/5/progress_file_12192429.xml -- Model is restarting -- Adjusting last_iter, 2166, to previous model restart step. Creating progress file: /mnt/nvme0n1p7/Boinc/data/slots/5/progress_file_12192429.xml last_cpu_time: 2404 upload_file_number: 7 last_iter: 2159 last_upload: 7776000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0785_2006050100_123_975_12192429_1_r221742427 The child process has been launched with process id: 4715 Executing the command: /mnt/nvme0n1p7/Boinc/data/slots/5/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:RyzenPC-Manjaro:1:1:4715:4715] [20230126:082832:1674718112:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:29:21 STEP 2160 H=2160:00 +CPU= 20.325 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:30:03 STEP 2161 H=2161:00 +CPU= 10.913 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:30:19 STEP 2162 H=2162:00 +CPU= 9.926 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:30:44 STEP 2163 H=2163:00 +CPU= 15.557 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:31:03 STEP 2164 H=2164:00 +CPU= 10.080 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:31:18 STEP 2165 H=2165:00 +CPU= 9.571 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:31:45 STEP 2166 H=2166:00 +CPU= 15.171 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:32:02 STEP 2167 H=2167:00 +CPU= 10.485 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:32:21 STEP 2168 H=2168:00 +CPU= 10.510 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:32:47 STEP 2169 H=2169:00 +CPU= 16.127 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:33:05 STEP 2170 H=2170:00 +CPU= 10.091 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:33:19 STEP 2171 H=2171:00 +CPU= 9.842 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:33:49 STEP 2172 H=2172:00 +CPU= 18.196 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMGGhq0f+002172 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMSHhq0f+002172 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMUAhq0f+002172 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:34:08 STEP 2173 H=2173:00 +CPU= 10.440 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:34:25 STEP 2174 H=2174:00 +CPU= 10.825 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:34:54 STEP 2175 H=2175:00 +CPU= 15.484 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:35:11 STEP 2176 H=2176:00 +CPU= 10.655 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:35:29 STEP 2177 H=2177:00 +CPU= 9.739 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:35:55 STEP 2178 H=2178:00 +CPU= 15.648 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:36:08 STEP 2179 H=2179:00 +CPU= 9.700 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:36:28 STEP 2180 H=2180:00 +CPU= 9.509 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:36:57 STEP 2181 H=2181:00 +CPU= 14.704 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:37:13 STEP 2182 H=2182:00 +CPU= 9.978 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:37:29 STEP 2183 H=2183:00 +CPU= 9.980 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:38:01 STEP 2184 H=2184:00 +CPU= 18.246 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMGGhq0f+002184 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMSHhq0f+002184 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMUAhq0f+002184 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMGGhq0f+002160 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMSHhq0f+002160 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMUAhq0f+002160 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMGGhq0f+002172 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMSHhq0f+002172 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMUAhq0f+002172 Zipping up the intermediate file: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_0785_2006050100_123_975_12192429_1_r221742427_7.zip Uploading the intermediate file: upload_file_7.zip 08:38:13 STEP 2185 H=2185:00 +CPU= 9.624 08:38:23 STEP 2186 H=2186:00 +CPU= 10.360 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:38:47 STEP 2187 H=2187:00 +CPU= 15.970 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:39:06 STEP 2188 H=2188:00 +CPU= 10.350 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:39:22 STEP 2189 H=2189:00 +CPU= 9.976 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:39:48 STEP 2190 H=2190:00 +CPU= 15.810 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:40:03 STEP 2191 H=2191:00 +CPU= 9.372 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:40:20 STEP 2192 H=2192:00 +CPU= 10.642 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:40:48 STEP 2193 H=2193:00 +CPU= 15.716 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:41:05 STEP 2194 H=2194:00 +CPU= 10.375 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:41:23 STEP 2195 H=2195:00 +CPU= 10.124 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:41:55 STEP 2196 H=2196:00 +CPU= 18.046 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMGGhq0f+002196 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMSHhq0f+002196 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMUAhq0f+002196 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:42:12 STEP 2197 H=2197:00 +CPU= 10.688 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:42:30 STEP 2198 H=2198:00 +CPU= 10.524 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:42:57 STEP 2199 H=2199:00 +CPU= 16.727 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:43:15 STEP 2200 H=2200:00 +CPU= 9.928 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:43:33 STEP 2201 H=2201:00 +CPU= 10.056 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:44:02 STEP 2202 H=2202:00 +CPU= 16.529 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:44:19 STEP 2203 H=2203:00 +CPU= 10.436 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:44:37 STEP 2204 H=2204:00 +CPU= 10.034 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:44:59 STEP 2205 H=2205:00 +CPU= 14.589 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:45:19 STEP 2206 H=2206:00 +CPU= 9.777 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:45:35 STEP 2207 H=2207:00 +CPU= 10.030 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:46:06 STEP 2208 H=2208:00 +CPU= 18.148 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMGGhq0f+002208 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMSHhq0f+002208 Moving to projects directory: /mnt/nvme0n1p7/Boinc/data/slots/5/ICMUAhq0f+002208 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMGGhq0f+002184 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMSHhq0f+002184 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMUAhq0f+002184 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMGGhq0f+002196 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMSHhq0f+002196 Adding to the zip: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_12192429/ICMUAhq0f+002196 Zipping up the intermediate file: /mnt/nvme0n1p7/Boinc/data/projects/climateprediction.net/oifs_43r3_ps_0785_2006050100_123_975_12192429_1_r221742427_8.zip Uploading the intermediate file: upload_file_8.zip 08:46:21 STEP 2209 H=2209:00 +CPU= 9.831 08:46:32 STEP 2210 H=2210:00 +CPU= 10.067 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:46:53 STEP 2211 H=2211:00 +CPU= 15.771 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:47:10 STEP 2212 H=2212:00 +CPU= 10.199 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:47:26 STEP 2213 H=2213:00 +CPU= 9.822 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:47:54 STEP 2214 H=2214:00 +CPU= 15.941 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:48:10 STEP 2215 H=2215:00 +CPU= 10.000 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:48:25 STEP 2216 H=2216:00 +CPU= 9.597 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Abort request received from BOINC client, ending the child process ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:40:49 | 1499379 | 22302813 | oifs_43r3_ps_0785_2006050100_123_975_12192429_1 | 6,566,400 | 16,770 | 0.0026 |
09 May 2023 16:40:46 | 1499379 | 22302813 | oifs_43r3_ps_0785_2006050100_123_975_12192429_1 | 5,702,400 | 14,001 | 0.0025 |
09 May 2023 16:40:37 | 1499379 | 22302813 | oifs_43r3_ps_0785_2006050100_123_975_12192429_1 | 4,838,400 | 11,275 | 0.0023 |
09 May 2023 16:40:29 | 1499379 | 22302813 | oifs_43r3_ps_0785_2006050100_123_975_12192429_1 | 3,974,400 | 8,353 | 0.0021 |
09 May 2023 16:39:26 | 1499379 | 22302813 | oifs_43r3_ps_0785_2006050100_123_975_12192429_1 | 2,592,000 | 3,661 | 0.0014 |
09 May 2023 16:39:13 | 1499379 | 22302813 | oifs_43r3_ps_0785_2006050100_123_975_12192429_1 | 864,000 | 2,774 | 0.0032 |
©2024 cpdn.org