Name | oifs_43r3_ps_0843_1993050100_123_962_12179487_0 |
Workunit | 12179487 |
Created | 22 Dec 2022, 4:10:49 UTC |
Sent | 22 Dec 2022, 4:33:06 UTC |
Report deadline | 21 Jan 2023, 4:33:06 UTC |
Received | 29 Dec 2022, 22:34:51 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1527792 |
Run time | 16 hours 7 min 41 sec |
CPU time | 9 hours 57 min 57 sec |
Validate state | Invalid |
Credit | 7,373.52 |
Device peak FLOPS | 6.83 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,417.14 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,643.00 MB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> t/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3993 MB (maxheap), 4403 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001) [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005047:1672185047:185.658] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDSC [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005048:1672185048:186.658] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:philip-All-Series:1:1:2877:2877] [20221228:005048:1672185048:186.658] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 000000000101A2B9 cloudsc_ 774 cloudsc.F90 oifs_43r3_model.e 0000000001014D57 cloud_layer_ 131 cloud_layer.F90 oifs_43r3_model.e 0000000000F1A790 callpar_ 1223 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1993050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0843 (argv4) batchid: 962 (argv5) wuid: 12179487 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/6 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8c72e395c84d8b696f8cbf6c940e9d29 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0843_1993050100_123_962_12179487.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0843_1993050100_123_962_12179487.zip ic_ancil_file: ic_ancil_12179487 ifsdata_file: ifsdata_12179487 climate_data_file: clim_data_12179487 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_c6d64edd46c61c4c21f456a6325a1196 to: /var/lib/boinc-client/slots/6/ic_ancil_12179487.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12179487.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_9883cbe597c1ef7f39205eb0f3321282 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12179487.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12179487.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12179487.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12179487.zip Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12179487.xml Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12179487.xml -- Model is restarting -- Adjusting last_iter, 1228, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12179487.xml last_cpu_time: 27048 upload_file_number: 43 last_iter: 1223 last_upload: 4402800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728 The child process has been launched with process id: 2851 Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:005223:1672185143:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:53:23 STEP 1224 H=1224:00 +CPU= 37.695 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:54:12 STEP 1225 H=1225:00 +CPU= 22.899 00:54:39 STEP 1226 H=1226:00 +CPU= 24.478 00:55:19 STEP 1227 H=1227:00 +CPU= 37.105 00:55:44 STEP 1228 H=1228:00 +CPU= 22.610 00:56:07 STEP 1229 H=1229:00 +CPU= 21.563 00:56:42 STEP 1230 H=1230:00 +CPU= 33.880 00:57:06 STEP 1231 H=1231:00 +CPU= 22.279 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:57:53 STEP 1232 H=1232:00 +CPU= 22.606 00:58:29 STEP 1233 H=1233:00 +CPU= 33.302 00:58:54 STEP 1234 H=1234:00 +CPU= 22.346 00:59:18 STEP 1235 H=1235:00 +CPU= 22.021 00:59:59 STEP 1236 H=1236:00 +CPU= 37.353 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001236 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001236 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001236 01:00:24 STEP 1237 H=1237:00 +CPU= 22.052 01:00:49 STEP 1238 H=1238:00 +CPU= 22.538 01:01:25 STEP 1239 H=1239:00 +CPU= 33.206 01:01:50 STEP 1240 H=1240:00 +CPU= 22.218 01:02:14 STEP 1241 H=1241:00 +CPU= 21.898 01:02:51 STEP 1242 H=1242:00 +CPU= 34.038 01:03:16 STEP 1243 H=1243:00 +CPU= 21.982 01:03:40 STEP 1244 H=1244:00 +CPU= 22.692 01:04:18 STEP 1245 H=1245:00 +CPU= 33.950 01:04:42 STEP 1246 H=1246:00 +CPU= 21.974 01:05:07 STEP 1247 H=1247:00 +CPU= 22.485 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001224 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001224 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001224 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001236 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001236 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001236 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_43.zip Uploading the intermediate file: upload_file_43.zip 01:05:51 STEP 1248 H=1248:00 +CPU= 39.257 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001248 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001248 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001248 01:06:15 STEP 1249 H=1249:00 +CPU= 21.853 01:06:40 STEP 1250 H=1250:00 +CPU= 22.800 01:07:17 STEP 1251 H=1251:00 +CPU= 33.805 01:07:42 STEP 1252 H=1252:00 +CPU= 22.725 01:08:07 STEP 1253 H=1253:00 +CPU= 22.415 01:08:45 STEP 1254 H=1254:00 +CPU= 34.317 01:09:09 STEP 1255 H=1255:00 +CPU= 21.791 01:09:33 STEP 1256 H=1256:00 +CPU= 22.037 01:10:07 STEP 1257 H=1257:00 +CPU= 32.144 01:10:31 STEP 1258 H=1258:00 +CPU= 22.118 01:10:54 STEP 1259 H=1259:00 +CPU= 21.345 01:11:32 STEP 1260 H=1260:00 +CPU= 36.303 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001260 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001260 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001260 01:11:55 STEP 1261 H=1261:00 +CPU= 21.382 01:12:19 STEP 1262 H=1262:00 +CPU= 22.137 01:12:53 STEP 1263 H=1263:00 +CPU= 32.271 01:13:16 STEP 1264 H=1264:00 +CPU= 21.970 01:13:41 STEP 1265 H=1265:00 +CPU= 21.858 01:14:19 STEP 1266 H=1266:00 +CPU= 33.988 01:14:43 STEP 1267 H=1267:00 +CPU= 22.397 01:15:08 STEP 1268 H=1268:00 +CPU= 22.905 01:15:45 STEP 1269 H=1269:00 +CPU= 33.470 01:16:10 STEP 1270 H=1270:00 +CPU= 22.785 01:16:36 STEP 1271 H=1271:00 +CPU= 22.684 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001248 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001248 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001248 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001260 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001260 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001260 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_44.zip Uploading the intermediate file: upload_file_44.zip 01:17:18 STEP 1272 H=1272:00 +CPU= 38.223 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001272 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001272 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001272 01:17:42 STEP 1273 H=1273:00 +CPU= 21.751 01:18:08 STEP 1274 H=1274:00 +CPU= 22.705 01:18:45 STEP 1275 H=1275:00 +CPU= 33.912 01:19:11 STEP 1276 H=1276:00 +CPU= 23.197 01:19:35 STEP 1277 H=1277:00 +CPU= 22.025 01:20:14 STEP 1278 H=1278:00 +CPU= 34.494 01:20:38 STEP 1279 H=1279:00 +CPU= 22.392 01:21:04 STEP 1280 H=1280:00 +CPU= 22.841 01:21:41 STEP 1281 H=1281:00 +CPU= 33.937 01:22:06 STEP 1282 H=1282:00 +CPU= 22.406 01:22:31 STEP 1283 H=1283:00 +CPU= 22.121 01:23:13 STEP 1284 H=1284:00 +CPU= 37.386 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001284 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001284 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001284 01:23:38 STEP 1285 H=1285:00 +CPU= 22.660 01:24:04 STEP 1286 H=1286:00 +CPU= 23.285 01:24:41 STEP 1287 H=1287:00 +CPU= 33.908 01:25:07 STEP 1288 H=1288:00 +CPU= 23.073 01:25:32 STEP 1289 H=1289:00 +CPU= 22.310 01:26:10 STEP 1290 H=1290:00 +CPU= 34.269 01:26:35 STEP 1291 H=1291:00 +CPU= 22.779 01:27:01 STEP 1292 H=1292:00 +CPU= 23.160 01:27:38 STEP 1293 H=1293:00 +CPU= 33.536 01:28:03 STEP 1294 H=1294:00 +CPU= 22.920 01:28:29 STEP 1295 H=1295:00 +CPU= 23.117 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001272 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001272 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001272 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001284 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001284 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001284 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_45.zip Uploading the intermediate file: upload_file_45.zip 01:29:13 STEP 1296 H=1296:00 +CPU= 39.944 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001296 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001296 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001296 01:29:38 STEP 1297 H=1297:00 +CPU= 22.719 01:30:04 STEP 1298 H=1298:00 +CPU= 23.133 01:30:42 STEP 1299 H=1299:00 +CPU= 34.503 01:31:08 STEP 1300 H=1300:00 +CPU= 23.252 01:31:34 STEP 1301 H=1301:00 +CPU= 23.274 01:32:12 STEP 1302 H=1302:00 +CPU= 34.260 01:32:36 STEP 1303 H=1303:00 +CPU= 22.468 01:33:02 STEP 1304 H=1304:00 +CPU= 22.887 01:33:39 STEP 1305 H=1305:00 +CPU= 33.621 01:34:05 STEP 1306 H=1306:00 +CPU= 22.987 01:34:29 STEP 1307 H=1307:00 +CPU= 22.308 01:35:08 STEP 1308 H=1308:00 +CPU= 36.211 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001308 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001308 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001308 01:35:34 STEP 1309 H=1309:00 +CPU= 22.808 01:36:00 STEP 1310 H=1310:00 +CPU= 23.436 01:36:38 STEP 1311 H=1311:00 +CPU= 34.493 01:37:04 STEP 1312 H=1312:00 +CPU= 23.554 01:37:30 STEP 1313 H=1313:00 +CPU= 22.907 01:38:06 STEP 1314 H=1314:00 +CPU= 33.934 01:38:31 STEP 1315 H=1315:00 +CPU= 22.317 01:38:57 STEP 1316 H=1316:00 +CPU= 23.370 01:39:35 STEP 1317 H=1317:00 +CPU= 33.885 01:40:01 STEP 1318 H=1318:00 +CPU= 23.050 01:40:27 STEP 1319 H=1319:00 +CPU= 23.265 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001296 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001296 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001296 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001308 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001308 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001308 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_46.zip Uploading the intermediate file: upload_file_46.zip 01:41:10 STEP 1320 H=1320:00 +CPU= 39.377 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001320 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001320 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001320 01:41:35 STEP 1321 H=1321:00 +CPU= 22.082 01:42:01 STEP 1322 H=1322:00 +CPU= 23.283 01:42:39 STEP 1323 H=1323:00 +CPU= 34.232 01:43:03 STEP 1324 H=1324:00 +CPU= 22.706 01:43:29 STEP 1325 H=1325:00 +CPU= 22.821 01:44:08 STEP 1326 H=1326:00 +CPU= 35.258 01:44:33 STEP 1327 H=1327:00 +CPU= 22.443 01:44:59 STEP 1328 H=1328:00 +CPU= 23.275 01:45:37 STEP 1329 H=1329:00 +CPU= 33.833 01:46:02 STEP 1330 H=1330:00 +CPU= 22.848 01:46:28 STEP 1331 H=1331:00 +CPU= 23.050 01:47:10 STEP 1332 H=1332:00 +CPU= 38.239 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001332 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001332 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001332 01:47:35 STEP 1333 H=1333:00 +CPU= 22.590 01:47:59 STEP 1334 H=1334:00 +CPU= 22.774 01:48:36 STEP 1335 H=1335:00 +CPU= 33.664 01:49:03 STEP 1336 H=1336:00 +CPU= 23.553 01:49:28 STEP 1337 H=1337:00 +CPU= 23.036 01:50:07 STEP 1338 H=1338:00 +CPU= 34.323 01:50:32 STEP 1339 H=1339:00 +CPU= 22.167 01:50:58 STEP 1340 H=1340:00 +CPU= 23.306 01:51:36 STEP 1341 H=1341:00 +CPU= 34.754 01:52:02 STEP 1342 H=1342:00 +CPU= 23.382 01:52:28 STEP 1343 H=1343:00 +CPU= 23.682 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001332 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001332 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001332 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_47.zip Uploading the intermediate file: upload_file_47.zip 01:53:06 STEP 1344 H=1344:00 +CPU= 33.945 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001344 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001344 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001344 01:53:31 STEP 1345 H=1345:00 +CPU= 21.827 01:53:56 STEP 1346 H=1346:00 +CPU= 23.198 01:54:36 STEP 1347 H=1347:00 +CPU= 35.416 01:55:02 STEP 1348 H=1348:00 +CPU= 22.959 01:55:28 STEP 1349 H=1349:00 +CPU= 23.122 01:56:08 STEP 1350 H=1350:00 +CPU= 36.189 01:56:32 STEP 1351 H=1351:00 +CPU= 21.856 01:56:57 STEP 1352 H=1352:00 +CPU= 23.420 01:57:36 STEP 1353 H=1353:00 +CPU= 34.882 01:58:01 STEP 1354 H=1354:00 +CPU= 22.269 01:58:26 STEP 1355 H=1355:00 +CPU= 22.708 01:59:09 STEP 1356 H=1356:00 +CPU= 38.312 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001356 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001356 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001356 01:59:34 STEP 1357 H=1357:00 +CPU= 22.511 02:00:02 STEP 1358 H=1358:00 +CPU= 24.879 02:00:43 STEP 1359 H=1359:00 +CPU= 37.111 02:01:09 STEP 1360 H=1360:00 +CPU= 24.322 02:01:36 STEP 1361 H=1361:00 +CPU= 24.565 02:02:18 STEP 1362 H=1362:00 +CPU= 37.923 02:02:45 STEP 1363 H=1363:00 +CPU= 24.504 02:03:14 STEP 1364 H=1364:00 +CPU= 25.881 02:03:55 STEP 1365 H=1365:00 +CPU= 37.155 02:04:22 STEP 1366 H=1366:00 +CPU= 23.984 02:04:49 STEP 1367 H=1367:00 +CPU= 24.625 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001356 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001356 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001356 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_48.zip Uploading the intermediate file: upload_file_48.zip Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:05:48 STEP 1368 H=1368:00 +CPU= 42.996 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001368 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001368 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001368 02:06:14 STEP 1369 H=1369:00 +CPU= 23.702 02:06:42 STEP 1370 H=1370:00 +CPU= 25.037 02:07:22 STEP 1371 H=1371:00 +CPU= 36.011 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:08:03 STEP 1372 H=1372:00 +CPU= 26.423 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:08:43 STEP 1373 H=1373:00 +CPU= 25.878 02:09:27 STEP 1374 H=1374:00 +CPU= 39.264 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:10:17 STEP 1375 H=1375:00 +CPU= 24.340 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:10:57 STEP 1376 H=1376:00 +CPU= 26.030 02:11:38 STEP 1377 H=1377:00 +CPU= 37.561 02:12:07 STEP 1378 H=1378:00 +CPU= 26.037 02:12:36 STEP 1379 H=1379:00 +CPU= 25.824 02:13:22 STEP 1380 H=1380:00 +CPU= 41.548 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001380 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001380 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001380 02:13:49 STEP 1381 H=1381:00 +CPU= 24.580 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:14:31 STEP 1382 H=1382:00 +CPU= 26.867 02:15:13 STEP 1383 H=1383:00 +CPU= 37.851 02:15:42 STEP 1384 H=1384:00 +CPU= 25.783 02:16:09 STEP 1385 H=1385:00 +CPU= 25.033 02:16:51 STEP 1386 H=1386:00 +CPU= 37.906 02:17:19 STEP 1387 H=1387:00 +CPU= 25.135 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:17:58 STEP 1388 H=1388:00 +CPU= 25.121 02:18:40 STEP 1389 H=1389:00 +CPU= 37.770 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:19:19 STEP 1390 H=1390:00 +CPU= 25.143 02:19:48 STEP 1391 H=1391:00 +CPU= 25.619 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001380 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001380 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001380 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_49.zip Uploading the intermediate file: upload_file_49.zip 02:20:35 STEP 1392 H=1392:00 +CPU= 42.879 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001392 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001392 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001392 02:21:03 STEP 1393 H=1393:00 +CPU= 25.050 02:21:32 STEP 1394 H=1394:00 +CPU= 26.289 02:22:14 STEP 1395 H=1395:00 +CPU= 37.598 02:22:41 STEP 1396 H=1396:00 +CPU= 24.786 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:23:21 STEP 1397 H=1397:00 +CPU= 25.273 02:24:03 STEP 1398 H=1398:00 +CPU= 38.341 02:24:30 STEP 1399 H=1399:00 +CPU= 24.176 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:25:10 STEP 1400 H=1400:00 +CPU= 25.925 02:25:52 STEP 1401 H=1401:00 +CPU= 37.494 02:26:20 STEP 1402 H=1402:00 +CPU= 25.040 02:26:47 STEP 1403 H=1403:00 +CPU= 24.800 02:27:32 STEP 1404 H=1404:00 +CPU= 40.806 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001404 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001404 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001404 02:27:59 STEP 1405 H=1405:00 +CPU= 24.886 02:28:28 STEP 1406 H=1406:00 +CPU= 25.850 02:29:08 STEP 1407 H=1407:00 +CPU= 36.532 02:29:38 STEP 1408 H=1408:00 +CPU= 26.463 02:30:07 STEP 1409 H=1409:00 +CPU= 25.365 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:31:10 STEP 1410 H=1410:00 +CPU= 37.126 02:31:37 STEP 1411 H=1411:00 +CPU= 24.927 02:32:05 STEP 1412 H=1412:00 +CPU= 25.367 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:32:56 STEP 1413 H=1413:00 +CPU= 36.621 02:33:23 STEP 1414 H=1414:00 +CPU= 25.353 02:33:52 STEP 1415 H=1415:00 +CPU= 26.008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001404 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001404 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001404 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_50.zip Uploading the intermediate file: upload_file_50.zip 02:34:41 STEP 1416 H=1416:00 +CPU= 43.134 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001416 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001416 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001416 02:35:09 STEP 1417 H=1417:00 +CPU= 25.098 02:35:38 STEP 1418 H=1418:00 +CPU= 25.328 02:36:17 STEP 1419 H=1419:00 +CPU= 35.606 02:36:45 STEP 1420 H=1420:00 +CPU= 25.353 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:37:25 STEP 1421 H=1421:00 +CPU= 25.908 02:38:06 STEP 1422 H=1422:00 +CPU= 37.572 02:38:33 STEP 1423 H=1423:00 +CPU= 24.399 02:39:03 STEP 1424 H=1424:00 +CPU= 26.310 02:39:44 STEP 1425 H=1425:00 +CPU= 36.926 02:40:14 STEP 1426 H=1426:00 +CPU= 26.112 02:40:43 STEP 1427 H=1427:00 +CPU= 25.695 02:41:28 STEP 1428 H=1428:00 +CPU= 40.903 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001428 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001428 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001428 02:41:56 STEP 1429 H=1429:00 +CPU= 25.165 02:42:24 STEP 1430 H=1430:00 +CPU= 25.438 02:43:03 STEP 1431 H=1431:00 +CPU= 34.993 02:43:31 STEP 1432 H=1432:00 +CPU= 25.141 02:44:00 STEP 1433 H=1433:00 +CPU= 25.654 02:44:41 STEP 1434 H=1434:00 +CPU= 37.278 02:45:06 STEP 1435 H=1435:00 +CPU= 22.999 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:45:45 STEP 1436 H=1436:00 +CPU= 24.708 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:46:36 STEP 1437 H=1437:00 +CPU= 36.634 02:47:04 STEP 1438 H=1438:00 +CPU= 25.174 02:47:33 STEP 1439 H=1439:00 +CPU= 26.230 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001428 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001428 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001428 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_51.zip Uploading the intermediate file: upload_file_51.zip 02:48:18 STEP 1440 H=1440:00 +CPU= 42.117 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001440 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001440 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001440 02:48:46 STEP 1441 H=1441:00 +CPU= 25.843 02:49:15 STEP 1442 H=1442:00 +CPU= 26.618 02:49:59 STEP 1443 H=1443:00 +CPU= 39.075 02:50:28 STEP 1444 H=1444:00 +CPU= 27.105 02:50:58 STEP 1445 H=1445:00 +CPU= 26.964 02:51:42 STEP 1446 H=1446:00 +CPU= 39.833 02:52:11 STEP 1447 H=1447:00 +CPU= 27.058 02:52:42 STEP 1448 H=1448:00 +CPU= 27.724 02:53:26 STEP 1449 H=1449:00 +CPU= 39.895 02:53:58 STEP 1450 H=1450:00 +CPU= 27.714 02:54:29 STEP 1451 H=1451:00 +CPU= 26.886 02:55:18 STEP 1452 H=1452:00 +CPU= 43.888 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001452 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001452 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001452 02:55:47 STEP 1453 H=1453:00 +CPU= 26.573 02:56:18 STEP 1454 H=1454:00 +CPU= 28.226 02:57:02 STEP 1455 H=1455:00 +CPU= 39.877 02:57:32 STEP 1456 H=1456:00 +CPU= 27.613 02:58:03 STEP 1457 H=1457:00 +CPU= 27.371 02:58:46 STEP 1458 H=1458:00 +CPU= 40.921 02:59:15 STEP 1459 H=1459:00 +CPU= 26.728 02:59:47 STEP 1460 H=1460:00 +CPU= 28.110 03:00:32 STEP 1461 H=1461:00 +CPU= 40.344 03:01:02 STEP 1462 H=1462:00 +CPU= 27.605 03:01:33 STEP 1463 H=1463:00 +CPU= 28.214 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001452 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001452 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001452 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_52.zip Uploading the intermediate file: upload_file_52.zip Uploading trickle at timestep: 5266800 03:02:22 STEP 1464 H=1464:00 +CPU= 45.486 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001464 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001464 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001464 03:02:52 STEP 1465 H=1465:00 +CPU= 26.687 03:03:23 STEP 1466 H=1466:00 +CPU= 27.985 03:04:06 STEP 1467 H=1467:00 +CPU= 39.579 03:04:37 STEP 1468 H=1468:00 +CPU= 27.414 03:05:07 STEP 1469 H=1469:00 +CPU= 26.726 03:05:53 STEP 1470 H=1470:00 +CPU= 41.348 03:06:22 STEP 1471 H=1471:00 +CPU= 26.556 03:06:53 STEP 1472 H=1472:00 +CPU= 28.461 03:07:38 STEP 1473 H=1473:00 +CPU= 40.920 03:08:10 STEP 1474 H=1474:00 +CPU= 28.641 03:08:41 STEP 1475 H=1475:00 +CPU= 27.500 03:09:32 STEP 1476 H=1476:00 +CPU= 45.361 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001476 03:10:02 STEP 1477 H=1477:00 +CPU= 27.080 03:10:31 STEP 1478 H=1478:00 +CPU= 27.321 03:11:15 STEP 1479 H=1479:00 +CPU= 39.077 03:11:46 STEP 1480 H=1480:00 +CPU= 27.464 03:12:16 STEP 1481 H=1481:00 +CPU= 27.202 03:13:02 STEP 1482 H=1482:00 +CPU= 40.998 03:13:32 STEP 1483 H=1483:00 +CPU= 27.089 03:14:03 STEP 1484 H=1484:00 +CPU= 27.409 03:14:47 STEP 1485 H=1485:00 +CPU= 39.898 03:15:18 STEP 1486 H=1486:00 +CPU= 27.628 03:15:49 STEP 1487 H=1487:00 +CPU= 28.166 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001476 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001476 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001476 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_53.zip Uploading the intermediate file: upload_file_53.zip 03:16:40 STEP 1488 H=1488:00 +CPU= 46.201 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001488 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001488 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001488 03:17:09 STEP 1489 H=1489:00 +CPU= 26.766 03:17:39 STEP 1490 H=1490:00 +CPU= 27.810 03:18:23 STEP 1491 H=1491:00 +CPU= 40.451 03:18:55 STEP 1492 H=1492:00 +CPU= 28.446 03:19:27 STEP 1493 H=1493:00 +CPU= 27.548 03:20:14 STEP 1494 H=1494:00 +CPU= 40.938 03:20:44 STEP 1495 H=1495:00 +CPU= 27.121 03:21:15 STEP 1496 H=1496:00 +CPU= 28.151 03:21:59 STEP 1497 H=1497:00 +CPU= 39.969 03:22:29 STEP 1498 H=1498:00 +CPU= 27.783 03:22:59 STEP 1499 H=1499:00 +CPU= 27.246 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.640] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS CLDPP T/S (1002) [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.656] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.657] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STOCHPERT_LAYER [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032339:1672194219:9076.657] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SPPTEN [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032340:1672194220:9077.657] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:philip-All-Series:1:1:2851:2851] [20221228:032340:1672194220:9077.657] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 00000000010C3C24 sppten_ 384 sppten.F90 oifs_43r3_model.e 00000000010BD01C stochpert_layer_ 159 stochpert_layer.F90 oifs_43r3_model.e 0000000000F251B4 callpar_ 1554 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1993050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0843 (argv4) batchid: 962 (argv5) wuid: 12179487 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/6 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8c72e395c84d8b696f8cbf6c940e9d29 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0843_1993050100_123_962_12179487.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0843_1993050100_123_962_12179487.zip ic_ancil_file: ic_ancil_12179487 ifsdata_file: ifsdata_12179487 climate_data_file: clim_data_12179487 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_c6d64edd46c61c4c21f456a6325a1196 to: /var/lib/boinc-client/slots/6/ic_ancil_12179487.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12179487.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_9883cbe597c1ef7f39205eb0f3321282 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12179487.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12179487.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12179487.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12179487.zip Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12179487.xml Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12179487.xml -- Model is restarting -- Adjusting last_iter, 1500, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12179487.xml last_cpu_time: 35022 upload_file_number: 54 last_iter: 1487 last_upload: 5353200 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728 The child process has been launched with process id: 2937 Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:philip-All-Series:1:1:2937:2937] [20221229:221449:1672348489:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 22:15:40 STEP 1488 H=1488:00 +CPU= 39.587 22:16:02 STEP 1489 H=1489:00 +CPU= 21.954 22:16:26 STEP 1490 H=1490:00 +CPU= 22.797 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:17:50 STEP 1491 H=1491:00 +CPU= 39.021 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:18:32 STEP 1492 H=1492:00 +CPU= 26.942 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:19:13 STEP 1493 H=1493:00 +CPU= 26.316 22:19:57 STEP 1494 H=1494:00 +CPU= 40.014 22:20:27 STEP 1495 H=1495:00 +CPU= 26.364 22:21:01 STEP 1496 H=1496:00 +CPU= 26.961 22:21:44 STEP 1497 H=1497:00 +CPU= 38.862 22:22:14 STEP 1498 H=1498:00 +CPU= 27.164 22:22:42 STEP 1499 H=1499:00 +CPU= 26.130 22:23:29 STEP 1500 H=1500:00 +CPU= 43.078 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001500 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001500 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001500 22:23:59 STEP 1501 H=1501:00 +CPU= 26.900 22:24:28 STEP 1502 H=1502:00 +CPU= 26.874 22:25:15 STEP 1503 H=1503:00 +CPU= 38.246 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:26:01 STEP 1504 H=1504:00 +CPU= 26.331 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:27:34 STEP 1505 H=1505:00 +CPU= 26.133 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:28:34 STEP 1506 H=1506:00 +CPU= 40.495 22:29:04 STEP 1507 H=1507:00 +CPU= 26.368 22:29:35 STEP 1508 H=1508:00 +CPU= 27.037 22:30:45 STEP 1509 H=1509:00 +CPU= 40.171 22:31:35 STEP 1510 H=1510:00 +CPU= 27.816 22:32:08 STEP 1511 H=1511:00 +CPU= 27.600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMGGhq0f+001500 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMSHhq0f+001500 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12179487/ICMUAhq0f+001500 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0843_1993050100_123_962_12179487_0_r998197728_54.zip Uploading the intermediate file: upload_file_54.zip 22:32:55 STEP 1512 H=1512:00 +CPU= 41.436 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001512 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001512 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001512 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:33:52 STEP 1513 H=1513:00 +CPU= 27.505 22:34:28 STEP 1514 H=1514:00 +CPU= 29.147 Abort request received from BOINC client, ending the child process </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 15:59:28 | 1527792 | 22264399 | oifs_43r3_ps_0843_1993050100_123_962_12179487_0 | 5,266,800 | 33,835 | 0.0064 |
09 May 2023 15:59:18 | 1527792 | 22264399 | oifs_43r3_ps_0843_1993050100_123_962_12179487_0 | 4,230,000 | 24,885 | 0.0059 |
09 May 2023 15:59:10 | 1527792 | 22264399 | oifs_43r3_ps_0843_1993050100_123_962_12179487_0 | 3,366,000 | 18,392 | 0.0055 |
09 May 2023 15:57:57 | 1527792 | 22264399 | oifs_43r3_ps_0843_1993050100_123_962_12179487_0 | 2,502,000 | 11,950 | 0.0048 |
09 May 2023 15:57:53 | 1527792 | 22264399 | oifs_43r3_ps_0843_1993050100_123_962_12179487_0 | 1,638,000 | 5,824 | 0.0036 |
©2024 cpdn.org