climateprediction.net home page
Task 22309693

Task 22309693

Name oifs_43r3_bl_a238_2016092300_15_991_12209495_0
Workunit 12209495
Created 13 Feb 2023, 16:59:47 UTC
Sent 13 Feb 2023, 17:09:45 UTC
Report deadline 14 Apr 2023, 17:09:45 UTC
Received 13 Feb 2023, 18:19:48 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1537623
Run time 25 min 37 sec
CPU time 24 min 25 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 7.68 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,034.68 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,291.03 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a238
(argv4) batchid: 991
(argv5) wuid: 12209495
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /home/galfridian/slots/33
Project directory is: /home/galfridian/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /home/galfridian/projects/climateprediction.net/oifs_43r3_bl_12209495
Copying: /home/galfridian/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /home/galfridian/slots/33/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/galfridian/slots/33/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_84b1fffdb1474127a842b38d4c562fcd to: /home/galfridian/slots/33/oifs_43r3_bl_a238_2016092300_15_991_12209495.zip
Unzipping the namelist zip file: /home/galfridian/slots/33/oifs_43r3_bl_a238_2016092300_15_991_12209495.zip
ic_ancil_file: ic_ancil_12209495
ifsdata_file: ifsdata_12209495
climate_data_file: clim_data_12209495
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /home/galfridian/slots/33/ic_ancil_12209495.zip
Unzipping the IC ancils zip file: /home/galfridian/slots/33/ic_ancil_12209495.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_d7f575bc085a18c685465bcafa879fa7 to: /home/galfridian/slots/33/ifsdata/ifsdata_12209495.zip
Unzipping the ifsdata_zip file: /home/galfridian/slots/33/ifsdata/ifsdata_12209495.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/galfridian/slots/33/159l_2/clim_data_12209495.zip
Unzipping the climate data zip file: /home/galfridian/slots/33/159l_2/clim_data_12209495.zip
Checking for progress XML file: /home/galfridian/slots/33/progress_file_12209495.xml
Creating progress file: /home/galfridian/slots/33/progress_file_12209495.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a238_2016092300_15_991_12209495_0_r71207560
The child process has been launched with process id: 4342
Executing the command: /home/galfridian/slots/33/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:4342:4342] [20230213:181052:1676308252:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  18:11:13 STEP    0 H=   0:00 +CPU= 18.867
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000000
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000000
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000000
  18:11:28 STEP    1 H=   0:15 +CPU= 15.226
  18:11:44 STEP    2 H=   0:30 +CPU= 14.933
  18:11:59 STEP    3 H=   0:45 +CPU= 15.327
  18:12:15 STEP    4 H=   1:00 +CPU= 15.408
  18:12:30 STEP    5 H=   1:15 +CPU= 15.402
  18:12:45 STEP    6 H=   1:30 +CPU= 14.902
  18:13:00 STEP    7 H=   1:45 +CPU= 15.063
  18:13:16 STEP    8 H=   2:00 +CPU= 15.720
  18:13:31 STEP    9 H=   2:15 +CPU= 14.871
  18:13:46 STEP   10 H=   2:30 +CPU= 14.861
  18:14:01 STEP   11 H=   2:45 +CPU= 14.844
  18:14:21 STEP   12 H=   3:00 +CPU= 19.966
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000012
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000012
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000012
  18:14:36 STEP   13 H=   3:15 +CPU= 15.118
  18:14:51 STEP   14 H=   3:30 +CPU= 15.078
  18:15:07 STEP   15 H=   3:45 +CPU= 15.128
  18:15:22 STEP   16 H=   4:00 +CPU= 15.368
  18:15:37 STEP   17 H=   4:15 +CPU= 14.588
  18:15:52 STEP   18 H=   4:30 +CPU= 15.052
  18:16:07 STEP   19 H=   4:45 +CPU= 15.015
  18:16:22 STEP   20 H=   5:00 +CPU= 15.115
  18:16:37 STEP   21 H=   5:15 +CPU= 14.718
  18:16:52 STEP   22 H=   5:30 +CPU= 14.905
  18:17:06 STEP   23 H=   5:45 +CPU= 14.541
  18:17:26 STEP   24 H=   6:00 +CPU= 19.824
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000024
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000024
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000024
  18:17:42 STEP   25 H=   6:15 +CPU= 15.659
  18:17:58 STEP   26 H=   6:30 +CPU= 15.503
  18:18:13 STEP   27 H=   6:45 +CPU= 15.505
  18:18:29 STEP   28 H=   7:00 +CPU= 15.315
  18:18:44 STEP   29 H=   7:15 +CPU= 15.250
  18:18:59 STEP   30 H=   7:30 +CPU= 15.029
  18:19:15 STEP   31 H=   7:45 +CPU= 15.753
  18:19:31 STEP   32 H=   8:00 +CPU= 15.817
  18:19:46 STEP   33 H=   8:15 +CPU= 15.037
  18:20:02 STEP   34 H=   8:30 +CPU= 15.421
  18:20:16 STEP   35 H=   8:45 +CPU= 14.838
  18:20:35 STEP   36 H=   9:00 +CPU= 18.234
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000036
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000036
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000036
  18:20:49 STEP   37 H=   9:15 +CPU= 14.120
  18:21:03 STEP   38 H=   9:30 +CPU= 13.859
  18:21:17 STEP   39 H=   9:45 +CPU= 13.773
  18:21:30 STEP   40 H=  10:00 +CPU= 13.416
  18:21:44 STEP   41 H=  10:15 +CPU= 14.096
  18:21:58 STEP   42 H=  10:30 +CPU= 13.748
  18:22:12 STEP   43 H=  10:45 +CPU= 13.889
  18:22:27 STEP   44 H=  11:00 +CPU= 15.065
  18:22:41 STEP   45 H=  11:15 +CPU= 14.222
  18:22:54 STEP   46 H=  11:30 +CPU= 13.254
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:23:38 STEP   47 H=  11:45 +CPU= 13.889
  18:24:00 STEP   48 H=  12:00 +CPU= 21.702
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000048
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000048
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000048
  18:24:15 STEP   49 H=  12:15 +CPU= 14.420
  18:24:30 STEP   50 H=  12:30 +CPU= 14.998
  18:24:45 STEP   51 H=  12:45 +CPU= 15.033
  18:25:00 STEP   52 H=  13:00 +CPU= 14.979
  18:25:15 STEP   53 H=  13:15 +CPU= 14.676
  18:25:30 STEP   54 H=  13:30 +CPU= 14.675
  18:25:45 STEP   55 H=  13:45 +CPU= 14.950
  18:26:00 STEP   56 H=  14:00 +CPU= 15.433
  18:26:15 STEP   57 H=  14:15 +CPU= 15.167
  18:26:31 STEP   58 H=  14:30 +CPU= 15.171
  18:26:46 STEP   59 H=  14:45 +CPU= 15.047
  18:27:05 STEP   60 H=  15:00 +CPU= 18.685
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000060
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000060
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000060
  18:27:19 STEP   61 H=  15:15 +CPU= 14.375
  18:27:33 STEP   62 H=  15:30 +CPU= 14.208
  18:27:48 STEP   63 H=  15:45 +CPU= 14.154
  18:28:02 STEP   64 H=  16:00 +CPU= 14.843
  18:28:17 STEP   65 H=  16:15 +CPU= 13.927
  18:28:31 STEP   66 H=  16:30 +CPU= 14.330
  18:28:46 STEP   67 H=  16:45 +CPU= 14.613
  18:29:01 STEP   68 H=  17:00 +CPU= 14.990
  18:29:15 STEP   69 H=  17:15 +CPU= 14.527
  18:29:29 STEP   70 H=  17:30 +CPU= 14.133
  18:29:44 STEP   71 H=  17:45 +CPU= 14.677
  18:30:03 STEP   72 H=  18:00 +CPU= 18.536
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000072
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000072
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000072
  18:30:17 STEP   73 H=  18:15 +CPU= 14.291
  18:30:32 STEP   74 H=  18:30 +CPU= 14.628
  18:30:46 STEP   75 H=  18:45 +CPU= 14.392
  18:31:01 STEP   76 H=  19:00 +CPU= 15.222
  18:31:16 STEP   77 H=  19:15 +CPU= 14.787
  18:31:32 STEP   78 H=  19:30 +CPU= 15.157
  18:31:47 STEP   79 H=  19:45 +CPU= 15.210
  18:32:04 STEP   80 H=  20:00 +CPU= 16.447
  18:32:19 STEP   81 H=  20:15 +CPU= 15.654
  18:32:36 STEP   82 H=  20:30 +CPU= 16.056
  18:32:52 STEP   83 H=  20:45 +CPU= 15.729
  18:33:12 STEP   84 H=  21:00 +CPU= 19.348
Moving to projects directory: /home/galfridian/slots/33/ICMGGh7zg+000084
Moving to projects directory: /home/galfridian/slots/33/ICMSHh7zg+000084
Moving to projects directory: /home/galfridian/slots/33/ICMUAh7zg+000084
  18:33:26 STEP   85 H=  21:15 +CPU= 14.509
  18:33:42 STEP   86 H=  21:30 +CPU= 15.320
  18:33:57 STEP   87 H=  21:45 +CPU= 15.079
  18:34:13 STEP   88 H=  22:00 +CPU= 16.067
  18:34:30 STEP   89 H=  22:15 +CPU= 16.271
  18:34:47 STEP   90 H=  22:30 +CPU= 16.328
  18:35:03 STEP   91 H=  22:45 +CPU= 15.750
  18:35:20 STEP   92 H=  23:00 +CPU= 16.516
  18:35:35 STEP   93 H=  23:15 +CPU= 15.323
  18:35:51 STEP   94 H=  23:30 +CPU= 16.060
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.820687192122988E-02 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =    84 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =    84 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000084 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000084
 MPI-TASK:   1 -    328085514 BYTES IN      6 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0146( 0.7%)
 NSTEP =    84 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000084 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000084
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0072 SECONDS 98300.7 Mbytes/s, TOTAL TIME=    0.1879( 3.8%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000084 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000084
 MPI-TASK:   1 -   5249368215 BYTES IN    125 RECORDS TRANSFERRED IN    0.0031 SECONDS******** Mbytes/s, TOTAL TIME=    0.1009( 3.1%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000084 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000084
 MPI-TASK:   1 -    319511881 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0028( 3.6%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000084 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000084
 MPI-TASK:   1 -    328085514 BYTES IN      4 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0041( 4.9%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =    84 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =    84 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000084 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000084
 MPI-TASK:   1 -    328085505 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS820213.7 Mbytes/s, TOTAL TIME=    0.0068( 5.9%)
 18:33:12 STEP   84 H=  21:00 +CPU= 19.348
ISTEP=      85 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=      85 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =    85 STEPO   0AAA00AAA
 18:33:26 STEP   85 H=  21:15 +CPU= 14.509
 NSTEP =    86 STEPO   0AAA00AAA
 18:33:42 STEP   86 H=  21:30 +CPU= 15.320
 NSTEP =    87 STEPO   0AAA00AAA
 18:33:57 STEP   87 H=  21:45 +CPU= 15.079
 NSTEP =    88 STEPO   0AAA00AAA
 18:34:13 STEP   88 H=  22:00 +CPU= 16.067
 NSTEP =    89 STEPO   0AAA00AAA
 18:34:30 STEP   89 H=  22:15 +CPU= 16.271
ISTEP=      90 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=      90 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =    90 STEPO   0AAA00AAA
 18:34:47 STEP   90 H=  22:30 +CPU= 16.328
 NSTEP =    91 STEPO   0AAA00AAA
 18:35:03 STEP   91 H=  22:45 +CPU= 15.750
 NSTEP =    92 STEPO   0AAA00AAA
 18:35:20 STEP   92 H=  23:00 +CPU= 16.516
 NSTEP =    93 STEPO   0AAA00AAA
 18:35:35 STEP   93 H=  23:15 +CPU= 15.323
 NSTEP =    94 STEPO   0AAA00AAA
 18:35:51 STEP   94 H=  23:30 +CPU= 16.060
ISTEP=      95 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=      95 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =    95 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 18:34:13 0AAA00AAA STEPO       89    16.068   16.068   16.116     22:43     23:21 0.34199894110531E-07       2GB       0MB
 18:34:30 0AAA00AAA STEPO       90    16.272   16.272   16.333     22:59     23:37 0.34199894110531E-07       2GB       0MB
 18:34:47 0AAA00AAA STEPO       91    16.329   16.329   17.790     23:15     23:55 0.34199894110531E-07       2GB       0MB
 18:35:03 0AAA00AAA STEPO       92    15.750   15.750   15.966     23:31     24:11 0.34199894110531E-07       2GB       0MB
 18:35:20 0AAA00AAA STEPO       93    16.517   16.517   16.567     23:47     24:28 0.34199894110531E-07       2GB       0MB
 18:35:35 0AAA00AAA STEPO       94    15.323   15.323   15.377     24:03     24:43 0.34199894110531E-07       2GB       0MB
 18:35:51 0AAA00AAA STEPO       95    16.060   16.060   16.066     24:19     24:59 0.34199894110531E-07       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "      48",
 CTIME   = "00001200      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,          36, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /home/galfridian/slots/33/progress_file_12209495.xml

<running_values>
  <last_cpu_time>1458.680000</last_cpu_time>
  <upload_file_number>0</upload_file_number>
  <last_iter>95</last_iter>
  <last_upload>0</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net