climateprediction.net home page
Task 22269830

Task 22269830

Name oifs_43r3_ps_0926_1998050100_123_967_12184570_0
Workunit 12184570
Created 22 Dec 2022, 17:28:30 UTC
Sent 6 Jan 2023, 10:38:31 UTC
Report deadline 5 Feb 2023, 10:38:31 UTC
Received 10 Jan 2023, 13:53:52 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1534826
Run time 9 hours 12 min 40 sec
CPU time 8 hours 11 min 16 sec
Validate state Invalid
Credit 8,588.16
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,779.78 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,182.16 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_032c134fb25d69d2ae7039b335d9962f to: /root/slots/1/ic_ancil_12184570.zip
Unzipping the IC ancils zip file: /root/slots/1/ic_ancil_12184570.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1824c6f7e3b30cd2779bf30ff73bea95 to: /root/slots/1/ifsdata/ifsdata_12184570.zip
Unzipping the ifsdata_zip file: /root/slots/1/ifsdata/ifsdata_12184570.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /root/slots/1/159l_2/clim_data_12184570.zip
Unzipping the climate data zip file: /root/slots/1/159l_2/clim_data_12184570.zip
Checking for progress XML file: /root/slots/1/progress_file_12184570.xml
Opened progress file ok : /root/slots/1/progress_file_12184570.xml
-- Model is restarting --
Adjusting last_iter, 1731, to previous model restart step.
Creating progress file: /root/slots/1/progress_file_12184570.xml
last_cpu_time: 26940
upload_file_number: 72
last_iter: 1727
last_upload: 6220800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185
The child process has been launched with process id: 5336
Executing the command: /root/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5336:5336] [20230110:134828:1673354908:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:48:53 STEP 1728 H=1728:00 +CPU= 19.924
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:49:08 STEP 1729 H=1729:00 +CPU= 11.001
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:49:21 STEP 1730 H=1730:00 +CPU= 11.312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:49:46 STEP 1731 H=1731:00 +CPU= 18.058
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:50:00 STEP 1732 H=1732:00 +CPU= 11.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:50:13 STEP 1733 H=1733:00 +CPU= 11.497
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:50:38 STEP 1734 H=1734:00 +CPU= 18.270
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:50:51 STEP 1735 H=1735:00 +CPU= 11.157
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1998050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0926
(argv4) batchid: 967
(argv5) wuid: 12184570
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /root/slots/1
Project directory is: /root/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /root/projects/climateprediction.net/oifs_43r3_ps_12184570
..mkdir for temp folder for results failed
Copying: /root/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /root/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /root/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_98e2671caa2f949ccd02a4b58a63a1fc to: /root/slots/1/oifs_43r3_ps_0926_1998050100_123_967_12184570.zip
Unzipping the namelist zip file: /root/slots/1/oifs_43r3_ps_0926_1998050100_123_967_12184570.zip
ic_ancil_file: ic_ancil_12184570
ifsdata_file: ifsdata_12184570
climate_data_file: clim_data_12184570
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_032c134fb25d69d2ae7039b335d9962f to: /root/slots/1/ic_ancil_12184570.zip
Unzipping the IC ancils zip file: /root/slots/1/ic_ancil_12184570.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1824c6f7e3b30cd2779bf30ff73bea95 to: /root/slots/1/ifsdata/ifsdata_12184570.zip
Unzipping the ifsdata_zip file: /root/slots/1/ifsdata/ifsdata_12184570.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /root/slots/1/159l_2/clim_data_12184570.zip
Unzipping the climate data zip file: /root/slots/1/159l_2/clim_data_12184570.zip
Checking for progress XML file: /root/slots/1/progress_file_12184570.xml
Opened progress file ok : /root/slots/1/progress_file_12184570.xml
-- Model is restarting --
Adjusting last_iter, 1735, to previous model restart step.
Creating progress file: /root/slots/1/progress_file_12184570.xml
last_cpu_time: 27048
upload_file_number: 72
last_iter: 1727
last_upload: 6220800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185
The child process has been launched with process id: 5409
Executing the command: /root/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:5409:5409] [20230110:135310:1673355190:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:53:38 STEP 1728 H=1728:00 +CPU= 20.591
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:53:52 STEP 1729 H=1729:00 +CPU= 11.668
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:54:09 STEP 1730 H=1730:00 +CPU= 12.931
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:54:36 STEP 1731 H=1731:00 +CPU= 20.374
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:54:51 STEP 1732 H=1732:00 +CPU= 12.947
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:55:07 STEP 1733 H=1733:00 +CPU= 12.422
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:55:32 STEP 1734 H=1734:00 +CPU= 20.915
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:55:49 STEP 1735 H=1735:00 +CPU= 12.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:56:03 STEP 1736 H=1736:00 +CPU= 12.718
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:56:30 STEP 1737 H=1737:00 +CPU= 20.167
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:56:46 STEP 1738 H=1738:00 +CPU= 12.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:57:01 STEP 1739 H=1739:00 +CPU= 12.567
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:57:29 STEP 1740 H=1740:00 +CPU= 22.235
Moving to projects directory: /root/slots/1/ICMGGhq0f+001740
Moving to projects directory: /root/slots/1/ICMSHhq0f+001740
Moving to projects directory: /root/slots/1/ICMUAhq0f+001740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:57:43 STEP 1741 H=1741:00 +CPU= 12.414
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:58:00 STEP 1742 H=1742:00 +CPU= 12.862
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:58:27 STEP 1743 H=1743:00 +CPU= 20.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:58:42 STEP 1744 H=1744:00 +CPU= 12.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:58:58 STEP 1745 H=1745:00 +CPU= 12.633
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:59:23 STEP 1746 H=1746:00 +CPU= 21.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:59:40 STEP 1747 H=1747:00 +CPU= 12.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:59:57 STEP 1748 H=1748:00 +CPU= 12.719
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:00:21 STEP 1749 H=1749:00 +CPU= 19.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:00:37 STEP 1750 H=1750:00 +CPU= 12.580
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:00:53 STEP 1751 H=1751:00 +CPU= 13.356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:01:22 STEP 1752 H=1752:00 +CPU= 23.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001752
Moving to projects directory: /root/slots/1/ICMSHhq0f+001752
Moving to projects directory: /root/slots/1/ICMUAhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001728
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001728
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001728
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001740
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001740
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001740
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185_72.zip
Uploading the intermediate file: upload_file_72.zip
  14:01:36 STEP 1753 H=1753:00 +CPU= 12.090
  14:01:49 STEP 1754 H=1754:00 +CPU= 12.447
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:02:13 STEP 1755 H=1755:00 +CPU= 20.083
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:02:30 STEP 1756 H=1756:00 +CPU= 12.525
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:02:46 STEP 1757 H=1757:00 +CPU= 12.558
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:03:11 STEP 1758 H=1758:00 +CPU= 20.629
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:03:27 STEP 1759 H=1759:00 +CPU= 11.932
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:03:41 STEP 1760 H=1760:00 +CPU= 12.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:07 STEP 1761 H=1761:00 +CPU= 19.604
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:22 STEP 1762 H=1762:00 +CPU= 13.089
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:38 STEP 1763 H=1763:00 +CPU= 12.164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:05:07 STEP 1764 H=1764:00 +CPU= 22.295
Moving to projects directory: /root/slots/1/ICMGGhq0f+001764
Moving to projects directory: /root/slots/1/ICMSHhq0f+001764
Moving to projects directory: /root/slots/1/ICMUAhq0f+001764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:05:21 STEP 1765 H=1765:00 +CPU= 12.178
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:05:38 STEP 1766 H=1766:00 +CPU= 12.848
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:06:02 STEP 1767 H=1767:00 +CPU= 20.108
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:06:19 STEP 1768 H=1768:00 +CPU= 12.469
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:06:33 STEP 1769 H=1769:00 +CPU= 12.639
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:06:59 STEP 1770 H=1770:00 +CPU= 19.984
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:07:14 STEP 1771 H=1771:00 +CPU= 12.484
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:07:31 STEP 1772 H=1772:00 +CPU= 13.053
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:07:57 STEP 1773 H=1773:00 +CPU= 19.674
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:08:11 STEP 1774 H=1774:00 +CPU= 12.306
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:08:28 STEP 1775 H=1775:00 +CPU= 12.936
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:08:58 STEP 1776 H=1776:00 +CPU= 24.095
Moving to projects directory: /root/slots/1/ICMGGhq0f+001776
Moving to projects directory: /root/slots/1/ICMSHhq0f+001776
Moving to projects directory: /root/slots/1/ICMUAhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001764
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001764
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001764
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185_73.zip
Uploading the intermediate file: upload_file_73.zip
  14:09:10 STEP 1777 H=1777:00 +CPU= 11.958
  14:09:23 STEP 1778 H=1778:00 +CPU= 12.760
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:09:48 STEP 1779 H=1779:00 +CPU= 19.088
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:10:02 STEP 1780 H=1780:00 +CPU= 11.706
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:10:17 STEP 1781 H=1781:00 +CPU= 11.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:10:40 STEP 1782 H=1782:00 +CPU= 18.460
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:10:53 STEP 1783 H=1783:00 +CPU= 11.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:11:09 STEP 1784 H=1784:00 +CPU= 11.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:11:31 STEP 1785 H=1785:00 +CPU= 18.140
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:11:44 STEP 1786 H=1786:00 +CPU= 11.386
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:12:00 STEP 1787 H=1787:00 +CPU= 11.273
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:12:24 STEP 1788 H=1788:00 +CPU= 20.005
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001788
Moving to projects directory: /root/slots/1/ICMSHhq0f+001788
Moving to projects directory: /root/slots/1/ICMUAhq0f+001788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:12:39 STEP 1789 H=1789:00 +CPU= 11.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:12:52 STEP 1790 H=1790:00 +CPU= 11.444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:13:17 STEP 1791 H=1791:00 +CPU= 18.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:13:30 STEP 1792 H=1792:00 +CPU= 11.648
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:13:46 STEP 1793 H=1793:00 +CPU= 11.349
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:14:08 STEP 1794 H=1794:00 +CPU= 18.513
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:14:22 STEP 1795 H=1795:00 +CPU= 11.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:14:37 STEP 1796 H=1796:00 +CPU= 11.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:15:00 STEP 1797 H=1797:00 +CPU= 18.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:15:13 STEP 1798 H=1798:00 +CPU= 11.484
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:15:29 STEP 1799 H=1799:00 +CPU= 11.586
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:15:56 STEP 1800 H=1800:00 +CPU= 21.224
Moving to projects directory: /root/slots/1/ICMGGhq0f+001800
Moving to projects directory: /root/slots/1/ICMSHhq0f+001800
Moving to projects directory: /root/slots/1/ICMUAhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001788
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001788
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001788
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185_74.zip
Uploading the intermediate file: upload_file_74.zip
  14:16:07 STEP 1801 H=1801:00 +CPU= 10.928
  14:16:19 STEP 1802 H=1802:00 +CPU= 11.453
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:16:41 STEP 1803 H=1803:00 +CPU= 18.014
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:16:54 STEP 1804 H=1804:00 +CPU= 11.738
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:17:10 STEP 1805 H=1805:00 +CPU= 11.179
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:17:32 STEP 1806 H=1806:00 +CPU= 18.614
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:17:48 STEP 1807 H=1807:00 +CPU= 11.322
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:18:01 STEP 1808 H=1808:00 +CPU= 11.625
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:18:25 STEP 1809 H=1809:00 +CPU= 18.286
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:18:39 STEP 1810 H=1810:00 +CPU= 11.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:18:52 STEP 1811 H=1811:00 +CPU= 11.114
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:19:19 STEP 1812 H=1812:00 +CPU= 20.449
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001812
Moving to projects directory: /root/slots/1/ICMSHhq0f+001812
Moving to projects directory: /root/slots/1/ICMUAhq0f+001812
  14:19:32 STEP 1813 H=1813:00 +CPU= 11.235
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:19:47 STEP 1814 H=1814:00 +CPU= 11.566
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:20:09 STEP 1815 H=1815:00 +CPU= 17.933
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:20:23 STEP 1816 H=1816:00 +CPU= 11.194
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:20:38 STEP 1817 H=1817:00 +CPU= 10.911
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:20:59 STEP 1818 H=1818:00 +CPU= 17.801
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:21:12 STEP 1819 H=1819:00 +CPU= 10.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:21:27 STEP 1820 H=1820:00 +CPU= 11.141
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:21:49 STEP 1821 H=1821:00 +CPU= 17.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:22:02 STEP 1822 H=1822:00 +CPU= 10.983
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:22:17 STEP 1823 H=1823:00 +CPU= 10.963
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:22:41 STEP 1824 H=1824:00 +CPU= 20.305
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001824
Moving to projects directory: /root/slots/1/ICMSHhq0f+001824
Moving to projects directory: /root/slots/1/ICMUAhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001812
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001812
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001812
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185_75.zip
Uploading the intermediate file: upload_file_75.zip
  14:22:54 STEP 1825 H=1825:00 +CPU= 10.838
  14:23:05 STEP 1826 H=1826:00 +CPU= 10.828
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:23:27 STEP 1827 H=1827:00 +CPU= 17.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:23:40 STEP 1828 H=1828:00 +CPU= 10.938
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:23:52 STEP 1829 H=1829:00 +CPU= 10.654
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:24:16 STEP 1830 H=1830:00 +CPU= 17.788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:24:29 STEP 1831 H=1831:00 +CPU= 10.788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:24:42 STEP 1832 H=1832:00 +CPU= 10.939
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:25:06 STEP 1833 H=1833:00 +CPU= 17.632
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:25:19 STEP 1834 H=1834:00 +CPU= 11.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:25:31 STEP 1835 H=1835:00 +CPU= 10.879
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:25:57 STEP 1836 H=1836:00 +CPU= 19.665
Moving to projects directory: /root/slots/1/ICMGGhq0f+001836
Moving to projects directory: /root/slots/1/ICMSHhq0f+001836
Moving to projects directory: /root/slots/1/ICMUAhq0f+001836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:26:10 STEP 1837 H=1837:00 +CPU= 10.835
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:26:23 STEP 1838 H=1838:00 +CPU= 11.128
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:26:47 STEP 1839 H=1839:00 +CPU= 17.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:27:00 STEP 1840 H=1840:00 +CPU= 10.988
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:27:13 STEP 1841 H=1841:00 +CPU= 10.774
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:27:36 STEP 1842 H=1842:00 +CPU= 17.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:27:49 STEP 1843 H=1843:00 +CPU= 10.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:02 STEP 1844 H=1844:00 +CPU= 11.139
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:24 STEP 1845 H=1845:00 +CPU= 17.268
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:39 STEP 1846 H=1846:00 +CPU= 11.136
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:52 STEP 1847 H=1847:00 +CPU= 10.911
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:29:18 STEP 1848 H=1848:00 +CPU= 20.068
Moving to projects directory: /root/slots/1/ICMGGhq0f+001848
Moving to projects directory: /root/slots/1/ICMSHhq0f+001848
Moving to projects directory: /root/slots/1/ICMUAhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001836
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001836
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001836
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185_76.zip
Uploading the intermediate file: upload_file_76.zip
  14:29:28 STEP 1849 H=1849:00 +CPU= 10.406
  14:29:39 STEP 1850 H=1850:00 +CPU= 10.842
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:29:58 STEP 1851 H=1851:00 +CPU= 17.004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:30:11 STEP 1852 H=1852:00 +CPU= 10.970
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:30:25 STEP 1853 H=1853:00 +CPU= 10.304
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:30:47 STEP 1854 H=1854:00 +CPU= 17.468
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:30:59 STEP 1855 H=1855:00 +CPU= 10.325
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:31:12 STEP 1856 H=1856:00 +CPU= 10.535
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:31:32 STEP 1857 H=1857:00 +CPU= 16.317
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:31:47 STEP 1858 H=1858:00 +CPU= 10.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:31:59 STEP 1859 H=1859:00 +CPU= 10.268
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:32:22 STEP 1860 H=1860:00 +CPU= 18.976
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001860
Moving to projects directory: /root/slots/1/ICMSHhq0f+001860
Moving to projects directory: /root/slots/1/ICMUAhq0f+001860
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:32:36 STEP 1861 H=1861:00 +CPU=  9.981
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:32:49 STEP 1862 H=1862:00 +CPU= 10.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:33:09 STEP 1863 H=1863:00 +CPU= 16.903
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:33:22 STEP 1864 H=1864:00 +CPU= 10.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:33:37 STEP 1865 H=1865:00 +CPU= 10.608
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:33:58 STEP 1866 H=1866:00 +CPU= 17.185
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:34:10 STEP 1867 H=1867:00 +CPU= 10.014
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:34:23 STEP 1868 H=1868:00 +CPU= 10.782
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:34:46 STEP 1869 H=1869:00 +CPU= 16.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:34:59 STEP 1870 H=1870:00 +CPU= 10.577
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:35:11 STEP 1871 H=1871:00 +CPU= 10.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:35:37 STEP 1872 H=1872:00 +CPU= 19.691
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001872
Moving to projects directory: /root/slots/1/ICMSHhq0f+001872
Moving to projects directory: /root/slots/1/ICMUAhq0f+001872
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001860
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001860
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001860
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185_77.zip
Uploading the intermediate file: upload_file_77.zip
  14:35:49 STEP 1873 H=1873:00 +CPU= 10.383
  14:36:00 STEP 1874 H=1874:00 +CPU= 10.733
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:36:18 STEP 1875 H=1875:00 +CPU= 16.345
  14:36:29 STEP 1876 H=1876:00 +CPU= 10.501
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:36:41 STEP 1877 H=1877:00 +CPU= 10.343
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:37:02 STEP 1878 H=1878:00 +CPU= 17.542
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:37:17 STEP 1879 H=1879:00 +CPU= 10.537
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:37:30 STEP 1880 H=1880:00 +CPU= 10.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:37:51 STEP 1881 H=1881:00 +CPU= 16.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:38:06 STEP 1882 H=1882:00 +CPU= 10.877
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:38:18 STEP 1883 H=1883:00 +CPU= 10.730
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:38:41 STEP 1884 H=1884:00 +CPU= 18.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001884
Moving to projects directory: /root/slots/1/ICMSHhq0f+001884
Moving to projects directory: /root/slots/1/ICMUAhq0f+001884
  14:38:53 STEP 1885 H=1885:00 +CPU= 10.200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:39:08 STEP 1886 H=1886:00 +CPU= 10.904
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:39:29 STEP 1887 H=1887:00 +CPU= 16.779
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:39:42 STEP 1888 H=1888:00 +CPU= 10.646
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:39:56 STEP 1889 H=1889:00 +CPU= 10.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:40:18 STEP 1890 H=1890:00 +CPU= 17.178
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:40:30 STEP 1891 H=1891:00 +CPU= 10.683
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:40:43 STEP 1892 H=1892:00 +CPU= 11.021
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:41:06 STEP 1893 H=1893:00 +CPU= 16.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:41:19 STEP 1894 H=1894:00 +CPU= 10.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:41:32 STEP 1895 H=1895:00 +CPU= 10.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:41:57 STEP 1896 H=1896:00 +CPU= 19.418
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001896
Moving to projects directory: /root/slots/1/ICMSHhq0f+001896
Moving to projects directory: /root/slots/1/ICMUAhq0f+001896
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001872
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001872
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001872
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMGGhq0f+001884
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMSHhq0f+001884
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12184570/ICMUAhq0f+001884
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0926_1998050100_123_967_12184570_0_r157605185_78.zip
Uploading the intermediate file: upload_file_78.zip
  14:42:10 STEP 1897 H=1897:00 +CPU= 10.295
  14:42:20 STEP 1898 H=1898:00 +CPU= 10.776
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:42:39 STEP 1899 H=1899:00 +CPU= 16.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:42:52 STEP 1900 H=1900:00 +CPU= 10.868
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:43:04 STEP 1901 H=1901:00 +CPU= 10.533
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:11:49 1534826 22269830 oifs_43r3_ps_0926_1998050100_123_967_12184570_0 6,134,400 25,985 0.0042
09 May 2023 16:11:43 1534826 22269830 oifs_43r3_ps_0926_1998050100_123_967_12184570_0 4,492,800 19,370 0.0043
09 May 2023 16:11:40 1534826 22269830 oifs_43r3_ps_0926_1998050100_123_967_12184570_0 3,628,800 16,404 0.0045
09 May 2023 16:10:34 1534826 22269830 oifs_43r3_ps_0926_1998050100_123_967_12184570_0 1,987,200 9,758 0.0049
09 May 2023 16:10:26 1534826 22269830 oifs_43r3_ps_0926_1998050100_123_967_12184570_0 1,123,200 6,108 0.0054


©2024 cpdn.org