climateprediction.net home page
Task 22315496

Task 22315496

Name oifs_43r3_00jh_2019110100_123_993_12214139_0
Workunit 12214139
Created 24 Feb 2023, 13:57:13 UTC
Sent 24 Feb 2023, 22:05:13 UTC
Report deadline 25 Apr 2023, 22:05:13 UTC
Received 25 Feb 2023, 0:06:35 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1539226
Run time 15 min 50 sec
CPU time 15 min 50 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,273.49 MB
Peak swap size 4,974.79 MB
Peak disk usage 1,246.66 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 00jh
(argv4) batchid: 993
(argv5) wuid: 12214139
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /home/nemo/slots/0
Project directory is: /home/nemo/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139
Copying: /home/nemo/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /home/nemo/slots/0/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/nemo/slots/0/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_87e39447a7f3f371a70c39a429e92715 to: /home/nemo/slots/0/oifs_43r3_00jh_2019110100_123_993_12214139.zip
Unzipping the namelist zip file: /home/nemo/slots/0/oifs_43r3_00jh_2019110100_123_993_12214139.zip
ic_ancil_file: ic_ancil_12214139
ifsdata_file: ifsdata_12214139
climate_data_file: clim_data_12214139
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_3af9485b0bdce9a135b04d4d74730fc4 to: /home/nemo/slots/0/ic_ancil_12214139.zip
Unzipping the IC ancils zip file: /home/nemo/slots/0/ic_ancil_12214139.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3eff4565c012db74fae70595a8169cbf to: /home/nemo/slots/0/ifsdata/ifsdata_12214139.zip
Unzipping the ifsdata_zip file: /home/nemo/slots/0/ifsdata/ifsdata_12214139.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/nemo/slots/0/159l_2/clim_data_12214139.zip
Unzipping the climate data zip file: /home/nemo/slots/0/159l_2/clim_data_12214139.zip
Checking for progress XML file: /home/nemo/slots/0/progress_file_12214139.xml
Creating progress file: /home/nemo/slots/0/progress_file_12214139.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_00jh_2019110100_123_993_12214139_0_r716976217
The child process has been launched with process id: 91
Executing the command: /home/nemo/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:91:91] [20230225:005432:1677282872:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  00:55:09 STEP    0 H=   0:00 +CPU= 29.530
Moving to projects directory: /home/nemo/slots/0/ICMGGhnw4+000000
Moving to projects directory: /home/nemo/slots/0/ICMSHhnw4+000000
Moving to projects directory: /home/nemo/slots/0/ICMUAhnw4+000000
  00:55:31 STEP    1 H=   1:00 +CPU= 20.543
  00:55:50 STEP    2 H=   2:00 +CPU= 18.085
  00:56:18 STEP    3 H=   3:00 +CPU= 27.253
  00:56:37 STEP    4 H=   4:00 +CPU= 17.810
  00:56:55 STEP    5 H=   5:00 +CPU= 17.303
  00:57:24 STEP    6 H=   6:00 +CPU= 27.773
  00:57:44 STEP    7 H=   7:00 +CPU= 17.921
  00:58:03 STEP    8 H=   8:00 +CPU= 18.195
  00:58:31 STEP    9 H=   9:00 +CPU= 27.146
  00:58:50 STEP   10 H=  10:00 +CPU= 17.832
  00:59:11 STEP   11 H=  11:00 +CPU= 18.260
  00:59:43 STEP   12 H=  12:00 +CPU= 29.935
Moving to projects directory: /home/nemo/slots/0/ICMGGhnw4+000012
Moving to projects directory: /home/nemo/slots/0/ICMSHhnw4+000012
Moving to projects directory: /home/nemo/slots/0/ICMUAhnw4+000012
  01:00:02 STEP   13 H=  13:00 +CPU= 17.929
  01:00:22 STEP   14 H=  14:00 +CPU= 18.212
  01:00:51 STEP   15 H=  15:00 +CPU= 27.456
  01:01:10 STEP   16 H=  16:00 +CPU= 17.916
  01:01:30 STEP   17 H=  17:00 +CPU= 17.729
  01:02:06 STEP   18 H=  18:00 +CPU= 30.503
  01:02:29 STEP   19 H=  19:00 +CPU= 19.725
  01:02:50 STEP   20 H=  20:00 +CPU= 19.329
  01:03:25 STEP   21 H=  21:00 +CPU= 29.747
  01:03:46 STEP   22 H=  22:00 +CPU= 19.322
  01:04:09 STEP   23 H=  23:00 +CPU= 19.852
  01:04:51 STEP   24 H=  24:00 +CPU= 34.008
Moving to projects directory: /home/nemo/slots/0/ICMGGhnw4+000024
Moving to projects directory: /home/nemo/slots/0/ICMSHhnw4+000024
Moving to projects directory: /home/nemo/slots/0/ICMUAhnw4+000024
Adding to the zip: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139/ICMGGhnw4+000000
Adding to the zip: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139/ICMSHhnw4+000000
Adding to the zip: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139/ICMUAhnw4+000000
Adding to the zip: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139/ICMGGhnw4+000012
Adding to the zip: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139/ICMSHhnw4+000012
Adding to the zip: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139/ICMUAhnw4+000012
Zipping up the intermediate file: /home/nemo/projects/climateprediction.net/oifs_43r3_00jh_2019110100_123_993_12214139_0_r716976217_0.zip
Uploading the intermediate file: upload_file_0.zip
  01:05:11 STEP   25 H=  25:00 +CPU= 18.580
  01:05:32 STEP   26 H=  26:00 +CPU= 19.068
  01:06:02 STEP   27 H=  27:00 +CPU= 28.085
  01:06:21 STEP   28 H=  28:00 +CPU= 18.051
  01:06:39 STEP   29 H=  29:00 +CPU= 17.569
  01:07:09 STEP   30 H=  30:00 +CPU= 27.997
  01:07:28 STEP   31 H=  31:00 +CPU= 17.571
  01:07:47 STEP   32 H=  32:00 +CPU= 18.113
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 00jh
(argv4) batchid: 993
(argv5) wuid: 12214139
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /home/nemo/slots/0
Project directory is: /home/nemo/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /home/nemo/projects/climateprediction.net/oifs_43r3_12214139
..mkdir for temp folder for results failed
Copying: /home/nemo/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /home/nemo/slots/0/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/nemo/slots/0/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_87e39447a7f3f371a70c39a429e92715 to: /home/nemo/slots/0/oifs_43r3_00jh_2019110100_123_993_12214139.zip
Unzipping the namelist zip file: /home/nemo/slots/0/oifs_43r3_00jh_2019110100_123_993_12214139.zip
ic_ancil_file: ic_ancil_12214139
ifsdata_file: ifsdata_12214139
climate_data_file: clim_data_12214139
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_3af9485b0bdce9a135b04d4d74730fc4 to: /home/nemo/slots/0/ic_ancil_12214139.zip
Unzipping the IC ancils zip file: /home/nemo/slots/0/ic_ancil_12214139.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3eff4565c012db74fae70595a8169cbf to: /home/nemo/slots/0/ifsdata/ifsdata_12214139.zip
Unzipping the ifsdata_zip file: /home/nemo/slots/0/ifsdata/ifsdata_12214139.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/nemo/slots/0/159l_2/clim_data_12214139.zip
Unzipping the climate data zip file: /home/nemo/slots/0/159l_2/clim_data_12214139.zip
Checking for progress XML file: /home/nemo/slots/0/progress_file_12214139.xml
Opened progress file ok : /home/nemo/slots/0/progress_file_12214139.xml
-- Model is restarting --
Adjusting last_iter, 33, to previous model restart step.
Creating progress file: /home/nemo/slots/0/progress_file_12214139.xml
last_cpu_time: 732
upload_file_number: 1
last_iter: 25
last_upload: 86400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_00jh_2019110100_123_993_12214139_0_r716976217
The child process has been launched with process id: 103
Executing the command: /home/nemo/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:debian-nemo:1:1:103:103] [20230225:010908:1677283748:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:10:33 STEP   24 H=  24:00 +CPU= 32.437
  01:10:51 STEP   25 H=  25:00 +CPU= 17.687
  01:11:11 STEP   26 H=  26:00 +CPU= 18.296
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:12:01 STEP   27 H=  27:00 +CPU= 27.763
  01:12:19 STEP   28 H=  28:00 +CPU= 17.902
  01:12:38 STEP   29 H=  29:00 +CPU= 17.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:13:29 STEP   30 H=  30:00 +CPU= 28.233
  01:13:47 STEP   31 H=  31:00 +CPU= 17.387
  01:14:06 STEP   32 H=  32:00 +CPU= 17.938
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01

SUECSO4: USE AVAILABLE FILES FOR DECADES:   2010 2020
 SUBROUTINE SUECSO4:, UNWEIGHTED AVERAGE:   2.318548376519975E-003
 NSTEP =    30 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.144857052483E-01 1.970660721775E-01 7.076838461015E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.386752660891E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  2.51138327333197E-02 -1.92457801565357E+00  2.44238543946940E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  2.33467789179043E-02 -1.00000000000000E+00  1.00000000000000E+00
  STEPO: evolving YSPSDT_AR1(  1)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102051500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102053000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102054500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102060000  !!!!!!!!!!!! 
 01:13:29 STEP   30 H=  30:00 +CPU= 28.233
 NSTEP =    31 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.084500113128E-01 1.980482619460E-01 7.122677076310E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.438213547025E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  2.06557470877039E-02 -2.09731620211884E+00  2.54658802906826E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  1.81995385339754E-02 -1.00000000000000E+00  1.00000000000000E+00
  STEPO: evolving YSPSDT_AR1(  1)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102061500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102063000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102064500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102070000  !!!!!!!!!!!! 
 01:13:47 STEP   31 H=  31:00 +CPU= 17.387
 NSTEP =    32 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.038584850031E-01 1.997784290946E-01 7.166055116459E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.407455128998E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  1.62608230814749E-02 -1.70187077442784E+00  2.16281079374697E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  1.09995925774064E-02 -1.00000000000000E+00  1.00000000000000E+00
  STEPO: evolving YSPSDT_AR1(  1)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102071500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102073000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102074500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191102080000  !!!!!!!!!!!! 
 01:14:06 STEP   32 H=  32:00 +CPU= 17.938
  DATE= 2019   11    2
ISTEP=      33: Reset RSOLINC= 1361.0304 RIP0= 1382.5914
 SU_GHGCLIM, RCARDI,RCH4,RN2O,RCFC11,RCFC12,RNO2:   6.240006412635001E-004
  9.628461857022168E-007  4.996611132045250E-007  9.607432727722499E-010
  2.027017283006666E-009  7.940283000000001E-011
 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT)     20191101
        2070  9.999999999999998E-005
SUECSO4: USE AVAILABLE FILES FOR DECADES:   2010 2020
 SUBROUTINE SUECSO4:, UNWEIGHTED AVERAGE:   2.317845668331562E-003
 NSTEP =    33 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.195230082755E-01 2.000494845981E-01 7.065658213727E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.521246070460E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  2.63251806811208E-02 -2.11606331949582E+00  2.08596285575537E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  2.25410725564848E-02 -1.00000000000000E+00  1.00000000000000E+00
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 01:11:11 0AAA00AAA STEPO       27    18.297   18.297   19.374      1:11      2:03 0.11661541053724E-04       2GB       0MB
 01:12:01 0AAA00AAA STEPO       28    27.762   27.762   50.063      1:39      2:53 0.11661541053724E-04       2GB       0MB
 01:12:19 0AAA00AAA STEPO       29    17.903   17.903   18.666      1:57      3:11 0.11661541053724E-04       2GB       0MB
 01:12:38 0AAA00AAA STEPO       30    17.653   17.653   18.621      2:14      3:30 0.11661541053724E-04       2GB       0MB
 01:13:29 0AAA00AAA STEPO       31    28.171   28.171   50.575      2:43      4:21 0.11661541053724E-04       2GB       0MB
 01:13:47 0AAA00AAA STEPO       32    17.387   17.387   18.232      3:00      4:39 0.11661541053724E-04       2GB       0MB
 01:14:06 0AAA00AAA STEPO       33    17.938   17.938   18.860      3:18      4:58 0.11661541053724E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "      24",
 CTIME   = "00010000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 5*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =   98326.5171593782     ,
 GMASSI  =   98326.1993223684     
 /
------------------------------------------------
>>> Printing last 17 lines from file: waminfo

PARAM=10U/10V,                                                          
REPRES=GG,                                                              
ANALYSIS PERIOD =       0                                               
FORECAST PERIOD =   10544400                                            
WIND TIME STEP IN SECONDS =    3600                                     
ANALYSIS FROM 20191101000000 TO 20191101000000                          
ASSIMILATION NO                                                         
NEW FORECAST NO                                                         
STATUS STORM1= UNFINISHED                                               
STATUS STORM2= UNFINISHED                                               
STATUS STORM3= UNFINISHED                                               
DATE FOR OUTPUT OF BOTH RESTART FILES = 00000000000000                  
LAST DATE FOR SPECTRA FILE OUTPUT = 20200303000000                      
BEGIN DATE FOR USING SURFACE CURRENT = 20191101000000                   
PARALLEL RESTART = YES                                                  
NUMBER OF MPI TASKS USED =          1                                   
------------------------------------------------
>>> Printing last 8 lines from file: /home/nemo/slots/0/progress_file_12214139.xml

<running_values>
  <last_cpu_time>948.840000</last_cpu_time>
  <upload_file_number>1</upload_file_number>
  <last_iter>33</last_iter>
  <last_upload>86400</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org