<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
nt, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:31:03 STEP 1849 H= 462:15 +CPU= 22.632
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:31:29 STEP 1850 H= 462:30 +CPU= 22.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:31:56 STEP 1851 H= 462:45 +CPU= 22.843
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:32:26 STEP 1852 H= 463:00 +CPU= 23.548
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:32:54 STEP 1853 H= 463:15 +CPU= 22.087
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:33:23 STEP 1854 H= 463:30 +CPU= 22.808
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:33:52 STEP 1855 H= 463:45 +CPU= 22.656
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:34:21 STEP 1856 H= 464:00 +CPU= 23.218
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:34:48 STEP 1857 H= 464:15 +CPU= 22.765
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:35:17 STEP 1858 H= 464:30 +CPU= 22.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:35:46 STEP 1859 H= 464:45 +CPU= 22.525
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:36:18 STEP 1860 H= 465:00 +CPU= 27.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001860
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:36:46 STEP 1861 H= 465:15 +CPU= 22.460
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:37:15 STEP 1862 H= 465:30 +CPU= 22.436
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:37:43 STEP 1863 H= 465:45 +CPU= 22.130
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:38:13 STEP 1864 H= 466:00 +CPU= 23.398
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:38:41 STEP 1865 H= 466:15 +CPU= 22.759
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:39:09 STEP 1866 H= 466:30 +CPU= 22.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:39:38 STEP 1867 H= 466:45 +CPU= 22.995
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:40:07 STEP 1868 H= 467:00 +CPU= 23.319
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:40:35 STEP 1869 H= 467:15 +CPU= 22.484
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:41:03 STEP 1870 H= 467:30 +CPU= 22.652
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:41:32 STEP 1871 H= 467:45 +CPU= 22.702
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:42:08 STEP 1872 H= 468:00 +CPU= 30.754
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001872
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:42:37 STEP 1873 H= 468:15 +CPU= 22.348
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a00f
(argv4) batchid: 1018
(argv5) wuid: 12288342
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9782cf2ef8927b3801e1bd4b65b3b3f9 to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a00f_2016092300_20_1018_12288342.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a00f_2016092300_20_1018_12288342.zip
ic_ancil_file: ic_ancil_12288342
ifsdata_file: ifsdata_12288342
climate_data_file: clim_data_12288342
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/3/ic_ancil_12288342.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12288342.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1ebfb56415a28f2da5865377f8421999 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288342.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288342.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288342.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288342.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12288342.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12288342.xml
-- Model is restarting --
Adjusting last_iter, 1874, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12288342.xml
last_cpu_time: 44712
upload_file_number: 19
last_iter: 1873
last_upload: 1642500
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a00f_2016092300_20_1018_12288342_0_r862536504
The child process has been launched with process id: 9759
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9759:9759] [20240617:114401:1718617441:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:44:35 STEP 1872 H= 468:00 +CPU= 24.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:45:04 STEP 1873 H= 468:15 +CPU= 23.238
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:45:33 STEP 1874 H= 468:30 +CPU= 23.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:46:02 STEP 1875 H= 468:45 +CPU= 23.205
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:46:30 STEP 1876 H= 469:00 +CPU= 23.504
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:46:58 STEP 1877 H= 469:15 +CPU= 22.435
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:47:28 STEP 1878 H= 469:30 +CPU= 23.509
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:47:56 STEP 1879 H= 469:45 +CPU= 22.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:48:26 STEP 1880 H= 470:00 +CPU= 23.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:48:55 STEP 1881 H= 470:15 +CPU= 23.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:49:24 STEP 1882 H= 470:30 +CPU= 22.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:49:50 STEP 1883 H= 470:45 +CPU= 21.945
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:50:27 STEP 1884 H= 471:00 +CPU= 28.545
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:50:55 STEP 1885 H= 471:15 +CPU= 22.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:51:24 STEP 1886 H= 471:30 +CPU= 22.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:51:50 STEP 1887 H= 471:45 +CPU= 22.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:52:20 STEP 1888 H= 472:00 +CPU= 23.639
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:52:48 STEP 1889 H= 472:15 +CPU= 22.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:53:17 STEP 1890 H= 472:30 +CPU= 22.535
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:53:46 STEP 1891 H= 472:45 +CPU= 23.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:54:16 STEP 1892 H= 473:00 +CPU= 22.980
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:54:45 STEP 1893 H= 473:15 +CPU= 23.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:55:11 STEP 1894 H= 473:30 +CPU= 22.570
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:55:40 STEP 1895 H= 473:45 +CPU= 22.578
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:56:16 STEP 1896 H= 474:00 +CPU= 28.065
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001896
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:56:45 STEP 1897 H= 474:15 +CPU= 23.061
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:57:12 STEP 1898 H= 474:30 +CPU= 22.175
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:57:42 STEP 1899 H= 474:45 +CPU= 22.086
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:58:11 STEP 1900 H= 475:00 +CPU= 23.132
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:58:39 STEP 1901 H= 475:15 +CPU= 22.130
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:59:07 STEP 1902 H= 475:30 +CPU= 22.074
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:59:36 STEP 1903 H= 475:45 +CPU= 22.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:00:05 STEP 1904 H= 476:00 +CPU= 23.062
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:00:31 STEP 1905 H= 476:15 +CPU= 22.141
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:01:00 STEP 1906 H= 476:30 +CPU= 22.520
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:01:28 STEP 1907 H= 476:45 +CPU= 22.509
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:02:03 STEP 1908 H= 477:00 +CPU= 28.262
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:02:31 STEP 1909 H= 477:15 +CPU= 22.285
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:03:00 STEP 1910 H= 477:30 +CPU= 22.528
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:03:29 STEP 1911 H= 477:45 +CPU= 23.113
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:03:58 STEP 1912 H= 478:00 +CPU= 23.169
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:04:27 STEP 1913 H= 478:15 +CPU= 22.577
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:04:54 STEP 1914 H= 478:30 +CPU= 23.041
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:05:23 STEP 1915 H= 478:45 +CPU= 22.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:05:52 STEP 1916 H= 479:00 +CPU= 23.243
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:06:21 STEP 1917 H= 479:15 +CPU= 22.816
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:06:50 STEP 1918 H= 479:30 +CPU= 23.000
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:07:19 STEP 1919 H= 479:45 +CPU= 22.928
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01
DATE= 2016 10 12
NORMS AT NSTEP CNT4 1920
SPECTRAL NORMS - LOG(PREHYDS) 0.115260975534699E+02
LEV VORTICITY DIVERGENCE TEMPERATURE KINETIC ENERGY
AVE 0.572855563981801E-05 0.759011572647401E-06 0.230886440867204E+03 0.233068752093305E+02
GPNORM HUMIDITY AVERAGE MINIMUM MAXIMUM
AVE 0.372261193740200E-02 0.100000000000000E-07 0.293235138931416E-01
GPNORM LIQUID WATER AVERAGE MINIMUM MAXIMUM
AVE 0.135839446778665E-05 0.000000000000000E+00 0.629028616269086E-03
GPNORM ICE WATER AVERAGE MINIMUM MAXIMUM
AVE 0.353951193224852E-07 0.000000000000000E+00 0.980299231182383E-04
GPNORM SNOW AVERAGE MINIMUM MAXIMUM
AVE 0.558406809804776E-07 0.000000000000000E+00 0.230225999510748E-03
GPNORM RAIN AVERAGE MINIMUM MAXIMUM
AVE 0.685882695427918E-07 0.000000000000000E+00 0.134963958531650E-03
GPNORM CLOUD FRACTION AVERAGE MINIMUM MAXIMUM
AVE 0.898110211752190E-02 0.000000000000000E+00 0.100000000000000E+01
GPNORM CLW AVERAGE MINIMUM MAXIMUM
AVE 0.135602413612857E-05 0.000000000000000E+00 0.729345226942093E-03
GPNORM CIW AVERAGE MINIMUM MAXIMUM
AVE 0.353069946225370E-07 0.000000000000000E+00 0.991027360059992E-04
GPNORM CC AVERAGE MINIMUM MAXIMUM
AVE 0.896516108415205E-02 0.000000000000000E+00 0.100000000000000E+01
GPNORM RFL AVERAGE MINIMUM MAXIMUM
AVE 0.287457908835212E-06 0.000000000000000E+00 0.509808330776699E-03
GPNORM SFL AVERAGE MINIMUM MAXIMUM
AVE 0.549487166904828E-07 0.000000000000000E+00 0.155365863898049E-03
GPNORM CRFL AVERAGE MINIMUM MAXIMUM
AVE 0.645516294412868E-07 0.000000000000000E+00 0.499019697056798E-03
GPNORM CSFL AVERAGE MINIMUM MAXIMUM
AVE 0.143941575086704E-08 0.000000000000000E+00 0.122856323703772E-03
GPNORM PFRC AVERAGE MINIMUM MAXIMUM
AVE 0.191486283775413E-01 0.000000000000000E+00 0.100000000000000E+01
NSTEP = 1920 STEPO F00000000
CREATE RESTART FILE ./srf00200000.0001
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./srf00200000.0001 MODE=w
IO-STREAM CLOSED - ./srf00200000.0001
MPI-TASK: 1 - ************ BYTES IN 46 RECORDS TRANSFERRED IN 1.7830 SECONDS******** Mbytes/s, TOTAL TIME= 2.2648(78.7%)
CREATE RESTART CONTROL FILE ./rcf
WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00191200.0001
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1920 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001920 MODE=w
IO-STREAM CLOSED - ICMGGh7zg+001920
MPI-TASK: 1 - 306700548 BYTES IN 6 RECORDS TRANSFERRED IN 0.0003 SECONDS******** Mbytes/s, TOTAL TIME= 0.0105( 2.9%)
NSTEP = 1920 STEPO 0AA000000
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMSHh7zg+001920 MODE=w
IO-STREAM CLOSED - ./ICMSHh7zg+001920
MPI-TASK: 1 - 707399250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0071 SECONDS 99633.7 Mbytes/s, TOTAL TIME= 2.2303( 0.3%)
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001920 MODE=w
IO-STREAM CLOSED - ./ICMUAh7zg+001920
MPI-TASK: 1 - 4907208768 BYTES IN 125 RECORDS TRANSFERRED IN 0.0053 SECONDS925888.4 Mbytes/s, TOTAL TIME= 0.1879( 2.8%)
STEPO_FPOS WAS : B00000F00E
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001920 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001920
MPI-TASK: 1 - 10 BYTES IN 2 RECORDS TRANSFERRED IN 0.0001 SECONDS 0.1 Mbytes/s, TOTAL TIME= 0.0034( 2.9%)
STEPO_FPOS WAS : V00000000Y
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001920 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001920
MPI-TASK: 1 - 298120819 BYTES IN 4 RECORDS TRANSFERRED IN 0.0003 SECONDS993736.0 Mbytes/s, TOTAL TIME= 0.0079( 3.8%)
STEPO_FPOS WAS : T00000000M
NSTEP = 1920 STEPO 0AAA00000
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat
12:06:50 0AAA00AAA STEPO 1919 23.000 23.000 29.058 18:12 22:48 0.76458896040099E-06 2GB 0MB
12:07:19 F00000000 STEPO 1920 23.049 23.049 29.113 18:35 23:18 0.75901157264740E-06 2GB 0MB
12:07:22 0AA000000 STEPO 1920 2.564 2.564 2.565 18:37 23:20 0.75901157264740E-06 2GB 0MB
12:07:23 FULLPOS-B DYNFPOS 1920 1.006 1.006 1.006 18:38 23:21 0.75901157264740E-06 2GB 0MB
12:07:27 FULLPOS-V DYNFPOS 1920 2.626 2.626 4.638 18:41 23:26 0.75901157264740E-06 2GB 0MB
12:07:28 FULLPOS-T DYNFPOS 1920 0.566 0.566 0.566 18:41 23:26 0.75901157264740E-06 2GB 0MB
12:07:29 0AAA00000 STEPO 1920 0.570 0.570 0.570 18:42 23:27 0.75901157264740E-06 2GB 0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf
CSTEP = " 1920",
CTIME = "00200000 ",
NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 1908, 2*0,
IPRGPNSRES = 1,
IPRGPEWRES = 1,
IPRTRWRES = 1,
IPRTRVRES = 1,
GMASS0 = 0.000000000000000E+000,
GMASSI = 0.000000000000000E+000
/
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/3/progress_file_12288342.xml
<running_values>
<last_cpu_time>45845.160000</last_cpu_time>
<upload_file_number>19</upload_file_number>
<last_iter>1920</last_iter>
<last_upload>1642500</last_upload>
<model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a00f
(argv4) batchid: 1018
(argv5) wuid: 12288342
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9782cf2ef8927b3801e1bd4b65b3b3f9 to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a00f_2016092300_20_1018_12288342.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a00f_2016092300_20_1018_12288342.zip
ic_ancil_file: ic_ancil_12288342
ifsdata_file: ifsdata_12288342
climate_data_file: clim_data_12288342
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/3/ic_ancil_12288342.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12288342.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1ebfb56415a28f2da5865377f8421999 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288342.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288342.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288342.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288342.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12288342.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12288342.xml
-- Model is restarting --
Adjusting last_iter, 1920, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12288342.xml
last_cpu_time: 45845
upload_file_number: 19
last_iter: 1921
last_upload: 1642500
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a00f_2016092300_20_1018_12288342_0_r862536504
The child process has been launched with process id: 9906
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:facebook-V:1:1:9906:9906] [20240617:120856:1718618936:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:09:26 STEP 1920 H= 480:00 +CPU= 22.632
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01
I I Hor. Flux of PEII -0.00000 I I I I I
I I Vert.Flux of PEII 0.00000 I I I I I
I I Work Pres.G.F II I 0.08843 I I I I
I I Conv. Ad.-Int. II -0.08842 I I I I I
DDH--------------------------------------------------------------------------------------------------I
I DYN- Horizont. Flux II 0.02779 I -0.00321 I 0.36919 I 0.00083 I -0.00034 I
I I Vertical Flux II 0.00000 I 0.00000 I 0.00000 I 0.00000 I 0.00000 I
DDH--------------------------------------------------------------------------------------------------I
DDH----------------------------------------------------------------------------------------------------------------------------------
I - Mass balance (10**(-6) KG/M**2)
I INITIAL I HORIZ. FLUX I DYN. FLUX I DYN. FLUX I PHYS. FLUX I PHYS. FLUX I CURRENT I I I RESIDUAL I
I MASS I I TOA I SURFACE I TOA I SURFACE I MASS I I I I
---------------------------------------------------------------------------------------------------------- I ----------------
- - - - - - - - I - -
- 10341776742. I 0. . 0. . 0. . 0. . 0. I 10341960692. - I - 183950. -
- - - - - - - - I - -
---------------------------------------------------------------------------------------------------------- I ----------------
DDH------------------------------------------------------------------------
I I
I TIME = 480.000H DDH * Diagnostic Var., AREA GLOBAL I
I I
I-------------------------------------------------------------------------I
I VARIABLE I Initial value I Current value I
I-------------------------------------------------------------------------I
I OMEGA (P/S) I -0.1302002124E-21 I 0.1348790033E-05 I
ICloud fractionI 0.0000000000E+00 I 0.1386638756E-01 I
I Relat. Hum. I 0.4186141375E+00 I 0.4419105362E+00 I
I PASS 01 I 0.0000000000E+00 I 0.2330203769E+03 I
DDH------------------------------------------------------------------------
12:09:26 STEP 1920 H= 480:00 +CPU= 22.632
END CNT3
NSTEP = 1920 CNT0 000000000 22.494 22.494 26.576 0.846 0 0 0 0 0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 46/ 1069125708 WRITTEN 0/ 0
-TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL
OU-TOT IN -TOT OUT
1 3 0 46 1044068 0 0 1.3 1.3 0.0 1.9 0.0
===-=== START OF TIMING STATISTICS ===-===
STATS FOR ALL TASKS
NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%)
0 CNT0 - COMPLETE EXECUTION 1 29901.2 29901.2 100.00 0.00
1 CNT4 - FORWARD INTEGRATION 1 26713.7 26713.7 89.34 0.00
8 SCAN2M - GRID-POINT DYNAMICS 1 3648.0 3648.0 12.20 0.00
10 SCAN2M - PHYSICS 1 15114.7 15114.7 50.55 0.00
12 SPNORM - SPECTRAL NORM COMP. 1 47.8 47.8 0.16 0.00
28 RESTART FILES - READING 1 2654.4 2654.4 8.88 0.00
31 POSDDH IN STEPO 1 0.8 0.8 0.00 0.00
37 CPGLAG - SL COMPUTATIONS 1 6769.4 6769.4 22.64 0.00
39 SU0YOMB 1 356.9 356.9 1.19 0.00
51 SCAN2M - SL COMM. PART 1 1 15.6 15.6 0.05 0.00
56 SPNORM - SPECTRAL NORM COMM. 1 0.4 0.4 0.00 0.00
102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 4 129.4 129.4 1.73 0.00
107 FTINV_CTL - INVERSE FOURIER TRANSFORM 4 42.3 42.3 0.57 0.00
140 SULEG - COMP. OF LEGENDRE POL. 1 48.6 48.6 0.16 0.00
152 LTINV_CTL - M TO L TRANSPOSITION 4 22.3 22.3 0.30 0.00
157 FTINV_CTL - L TO G TRANSPOSITION 4 39.5 39.5 0.53 0.00
400 GSTATS 1119 0.0 0.0 0.00 0.00
401 GSTATS HOOK 987 0.0 0.0 0.00 0.00
TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT
TOTAL WALLCLOCK TIME 29.9 CPU TIME 25.8 VECTOR TIME 25.8
===-=== END OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY 64685.9
*** END CNT0 ***
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/3/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/3/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288342/ICMSHh7zg+001836
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a00f_2016092300_20_1018_12288342_0_r862536504_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1728900
12:12:58 (9903): called boinc_finish(0)
</stderr_txt>
]]>
