climateprediction.net home page
Task 22254452

Task 22254452

Name oifs_43r3_ps_0493_1984050100_123_953_12170137_0
Workunit 12170137
Created 21 Dec 2022, 3:32:40 UTC
Sent 16 Jan 2023, 2:42:56 UTC
Report deadline 15 Feb 2023, 2:42:56 UTC
Received 5 Feb 2023, 16:43:57 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1509518
Run time 7 hours 21 min 55 sec
CPU time 7 hours 0 min 54 sec
Validate state Invalid
Credit 1,572.48
Device peak FLOPS 4.11 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,417.99 MB
Peak swap size 4,978.07 MB
Peak disk usage 5,258.04 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
1675575364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1932:1932] [20230204:213604:1675575364:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:37:49 STEP  360 H= 360:00 +CPU= 62.546
  05:38:28 STEP  361 H= 361:00 +CPU= 36.582
  05:38:59 STEP  362 H= 362:00 +CPU= 30.648
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:33 STEP  363 H= 363:00 +CPU= 50.649
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0493
(argv4) batchid: 953
(argv5) wuid: 12170137
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_e69ff5d6a2ec073e5c749bcec0ce939d to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0493_1984050100_123_953_12170137.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0493_1984050100_123_953_12170137.zip
ic_ancil_file: ic_ancil_12170137
ifsdata_file: ifsdata_12170137
climate_data_file: clim_data_12170137
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a98c5b63311a9716a767d357dc5b3a5a to: /var/lib/boinc-client/slots/2/ic_ancil_12170137.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12170137.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_843ae2c75d0c38ccf14ccec253bbb42e to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12170137.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12170137.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12170137.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12170137.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12170137.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12170137.xml
-- Model is restarting --
Adjusting last_iter, 364, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12170137.xml
last_cpu_time: 17240
upload_file_number: 15
last_iter: 359
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903
The child process has been launched with process id: 5565
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5565:5565] [20230205:054245:1675604565:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:49 STEP  360 H= 360:00 +CPU= 61.628
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:46:53 STEP  361 H= 361:00 +CPU= 34.202
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:47:41 STEP  362 H= 362:00 +CPU= 33.113
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:48:40 STEP  363 H= 363:00 +CPU= 44.279
  05:49:08 STEP  364 H= 364:00 +CPU= 26.134
  05:49:33 STEP  365 H= 365:00 +CPU= 24.004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:51:08 STEP  366 H= 366:00 +CPU= 54.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:52:30 STEP  367 H= 367:00 +CPU= 34.792
  05:53:05 STEP  368 H= 368:00 +CPU= 30.990
  05:53:50 STEP  369 H= 369:00 +CPU= 42.248
  05:54:25 STEP  370 H= 370:00 +CPU= 32.199
  05:54:51 STEP  371 H= 371:00 +CPU= 24.866
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:56:02 STEP  372 H= 372:00 +CPU= 53.767
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000372
  05:57:38 STEP  373 H= 373:00 +CPU= 31.258
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:52 STEP  374 H= 374:00 +CPU= 36.464
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:00:44 STEP  375 H= 375:00 +CPU= 51.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:01:46 STEP  376 H= 376:00 +CPU= 27.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:03:40 STEP  377 H= 377:00 +CPU= 34.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:04:45 STEP  378 H= 378:00 +CPU= 49.585
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:05:30 STEP  379 H= 379:00 +CPU= 30.560
  06:05:59 STEP  380 H= 380:00 +CPU= 27.870
  06:06:47 STEP  381 H= 381:00 +CPU= 45.093
  06:07:21 STEP  382 H= 382:00 +CPU= 30.976
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:08:39 STEP  383 H= 383:00 +CPU= 36.229
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:11:01 STEP  384 H= 384:00 +CPU= 52.620
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_15.zip
Uploading the intermediate file: upload_file_15.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:12:04 STEP  385 H= 385:00 +CPU= 33.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:13:17 STEP  386 H= 386:00 +CPU= 34.572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:15:25 STEP  387 H= 387:00 +CPU= 54.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:16:11 STEP  388 H= 388:00 +CPU= 31.507
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:18:12 STEP  389 H= 389:00 +CPU= 33.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:19:30 STEP  390 H= 390:00 +CPU= 50.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:21:01 STEP  391 H= 391:00 +CPU= 32.917
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:22:07 STEP  392 H= 392:00 +CPU= 36.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:23:33 STEP  393 H= 393:00 +CPU= 48.635
  06:24:04 STEP  394 H= 394:00 +CPU= 28.628
  06:24:36 STEP  395 H= 395:00 +CPU= 29.573
  06:25:27 STEP  396 H= 396:00 +CPU= 47.887
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:26:09 STEP  397 H= 397:00 +CPU= 29.694
  06:26:40 STEP  398 H= 398:00 +CPU= 28.498
  06:27:27 STEP  399 H= 399:00 +CPU= 44.328
  06:27:55 STEP  400 H= 400:00 +CPU= 27.014
  06:28:24 STEP  401 H= 401:00 +CPU= 27.290
  06:29:14 STEP  402 H= 402:00 +CPU= 46.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:29:59 STEP  403 H= 403:00 +CPU= 30.608
  06:30:24 STEP  404 H= 404:00 +CPU= 24.524
  06:31:01 STEP  405 H= 405:00 +CPU= 36.559
  06:31:29 STEP  406 H= 406:00 +CPU= 27.617
  06:31:53 STEP  407 H= 407:00 +CPU= 23.693
  06:32:36 STEP  408 H= 408:00 +CPU= 43.134
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_16.zip
Uploading the intermediate file: upload_file_16.zip
  06:32:59 STEP  409 H= 409:00 +CPU= 22.681
  06:33:22 STEP  410 H= 410:00 +CPU= 23.045
  06:34:02 STEP  411 H= 411:00 +CPU= 39.689
  06:34:25 STEP  412 H= 412:00 +CPU= 23.009
  06:34:48 STEP  413 H= 413:00 +CPU= 22.514
  06:35:29 STEP  414 H= 414:00 +CPU= 40.431
  06:35:51 STEP  415 H= 415:00 +CPU= 22.519
  06:36:16 STEP  416 H= 416:00 +CPU= 24.736
  06:36:55 STEP  417 H= 417:00 +CPU= 38.471
  06:37:18 STEP  418 H= 418:00 +CPU= 23.098
  06:37:43 STEP  419 H= 419:00 +CPU= 25.144
  06:38:24 STEP  420 H= 420:00 +CPU= 40.603
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000420
  06:38:47 STEP  421 H= 421:00 +CPU= 22.443
  06:39:15 STEP  422 H= 422:00 +CPU= 27.903
  06:39:50 STEP  423 H= 423:00 +CPU= 34.625
  06:40:13 STEP  424 H= 424:00 +CPU= 23.214
  06:40:41 STEP  425 H= 425:00 +CPU= 27.721
  06:41:17 STEP  426 H= 426:00 +CPU= 35.293
  06:41:40 STEP  427 H= 427:00 +CPU= 23.651
  06:42:07 STEP  428 H= 428:00 +CPU= 26.265
  06:42:41 STEP  429 H= 429:00 +CPU= 34.268
  06:43:09 STEP  430 H= 430:00 +CPU= 27.117
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0493
(argv4) batchid: 953
(argv5) wuid: 12170137
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_e69ff5d6a2ec073e5c749bcec0ce939d to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0493_1984050100_123_953_12170137.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0493_1984050100_123_953_12170137.zip
ic_ancil_file: ic_ancil_12170137
ifsdata_file: ifsdata_12170137
climate_data_file: clim_data_12170137
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a98c5b63311a9716a767d357dc5b3a5a to: /var/lib/boinc-client/slots/2/ic_ancil_12170137.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12170137.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_843ae2c75d0c38ccf14ccec253bbb42e to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12170137.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12170137.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12170137.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12170137.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12170137.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12170137.xml
-- Model is restarting --
Adjusting last_iter, 430, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12170137.xml
last_cpu_time: 19670
upload_file_number: 17
last_iter: 407
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903
The child process has been launched with process id: 11489
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:11489:11489] [20230205:064420:1675608260:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  06:45:08 STEP  408 H= 408:00 +CPU= 42.903
  06:45:30 STEP  409 H= 409:00 +CPU= 22.577
  06:45:57 STEP  410 H= 410:00 +CPU= 26.098
  06:46:32 STEP  411 H= 411:00 +CPU= 35.187
  06:46:58 STEP  412 H= 412:00 +CPU= 25.483
  06:47:23 STEP  413 H= 413:00 +CPU= 25.403
  06:48:00 STEP  414 H= 414:00 +CPU= 36.065
  06:48:27 STEP  415 H= 415:00 +CPU= 27.346
  06:48:54 STEP  416 H= 416:00 +CPU= 26.852
  06:49:30 STEP  417 H= 417:00 +CPU= 35.249
  06:49:59 STEP  418 H= 418:00 +CPU= 28.470
  06:50:21 STEP  419 H= 419:00 +CPU= 22.615
  06:51:02 STEP  420 H= 420:00 +CPU= 40.191
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000420
  06:51:27 STEP  421 H= 421:00 +CPU= 25.089
  06:51:50 STEP  422 H= 422:00 +CPU= 23.511
  06:52:28 STEP  423 H= 423:00 +CPU= 37.420
  06:52:51 STEP  424 H= 424:00 +CPU= 23.307
  06:53:14 STEP  425 H= 425:00 +CPU= 22.662
  06:53:55 STEP  426 H= 426:00 +CPU= 40.861
  06:54:18 STEP  427 H= 427:00 +CPU= 22.654
  06:54:41 STEP  428 H= 428:00 +CPU= 23.322
  06:55:21 STEP  429 H= 429:00 +CPU= 39.619
  06:55:44 STEP  430 H= 430:00 +CPU= 23.240
  06:56:10 STEP  431 H= 431:00 +CPU= 25.779
  06:56:55 STEP  432 H= 432:00 +CPU= 44.383
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_17.zip
Uploading the intermediate file: upload_file_17.zip
  06:57:18 STEP  433 H= 433:00 +CPU= 22.666
  06:57:46 STEP  434 H= 434:00 +CPU= 28.121
  06:58:22 STEP  435 H= 435:00 +CPU= 35.196
  06:58:45 STEP  436 H= 436:00 +CPU= 23.607
  06:59:12 STEP  437 H= 437:00 +CPU= 26.837
  06:59:49 STEP  438 H= 438:00 +CPU= 36.107
  07:00:13 STEP  439 H= 439:00 +CPU= 24.737
  07:00:41 STEP  440 H= 440:00 +CPU= 26.982
  07:01:16 STEP  441 H= 441:00 +CPU= 35.040
  07:01:44 STEP  442 H= 442:00 +CPU= 28.105
  07:02:07 STEP  443 H= 443:00 +CPU= 22.791
  07:02:46 STEP  444 H= 444:00 +CPU= 38.829
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000444
  07:03:13 STEP  445 H= 445:00 +CPU= 26.484
  07:03:36 STEP  446 H= 446:00 +CPU= 23.248
  07:04:13 STEP  447 H= 447:00 +CPU= 36.637
  07:04:39 STEP  448 H= 448:00 +CPU= 25.942
  07:05:01 STEP  449 H= 449:00 +CPU= 22.228
  07:05:41 STEP  450 H= 450:00 +CPU= 39.119
  07:06:05 STEP  451 H= 451:00 +CPU= 24.142
  07:06:28 STEP  452 H= 452:00 +CPU= 23.152
  07:07:06 STEP  453 H= 453:00 +CPU= 37.633
  07:07:34 STEP  454 H= 454:00 +CPU= 26.636
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:08:45 STEP  455 H= 455:00 +CPU= 34.970
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:10:28 STEP  456 H= 456:00 +CPU= 54.805
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_18.zip
Uploading the intermediate file: upload_file_18.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:11:11 STEP  457 H= 457:00 +CPU= 29.477
  07:11:39 STEP  458 H= 458:00 +CPU= 26.895
  07:12:22 STEP  459 H= 459:00 +CPU= 41.186
  07:12:51 STEP  460 H= 460:00 +CPU= 28.347
  07:13:13 STEP  461 H= 461:00 +CPU= 22.438
  07:13:59 STEP  462 H= 462:00 +CPU= 43.837
  07:14:30 STEP  463 H= 463:00 +CPU= 28.446
  07:15:05 STEP  464 H= 464:00 +CPU= 31.038
  07:15:53 STEP  465 H= 465:00 +CPU= 45.421
  07:16:20 STEP  466 H= 466:00 +CPU= 25.650
  07:16:51 STEP  467 H= 467:00 +CPU= 29.476
  07:17:38 STEP  468 H= 468:00 +CPU= 45.278
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000468
  07:18:10 STEP  469 H= 469:00 +CPU= 29.607
  07:18:39 STEP  470 H= 470:00 +CPU= 27.279
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:19:51 STEP  471 H= 471:00 +CPU= 45.915
  07:20:18 STEP  472 H= 472:00 +CPU= 25.698
  07:20:52 STEP  473 H= 473:00 +CPU= 31.371
  07:21:33 STEP  474 H= 474:00 +CPU= 39.268
  07:22:03 STEP  475 H= 475:00 +CPU= 29.003
  07:22:32 STEP  476 H= 476:00 +CPU= 27.508
  07:23:23 STEP  477 H= 477:00 +CPU= 46.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:24:07 STEP  478 H= 478:00 +CPU= 30.154
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:24:42 STEP  479 H= 479:00 +CPU= 23.792
  07:25:35 STEP  480 H= 480:00 +CPU= 50.560
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_19.zip
Uploading the intermediate file: upload_file_19.zip
  07:26:02 STEP  481 H= 481:00 +CPU= 25.827
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:27:11 STEP  482 H= 482:00 +CPU= 32.790
  07:27:59 STEP  483 H= 483:00 +CPU= 45.205
  07:28:33 STEP  484 H= 484:00 +CPU= 30.703
  07:29:03 STEP  485 H= 485:00 +CPU= 28.100
  07:29:46 STEP  486 H= 486:00 +CPU= 42.820
  07:30:09 STEP  487 H= 487:00 +CPU= 22.648
  07:30:33 STEP  488 H= 488:00 +CPU= 23.489
  07:31:14 STEP  489 H= 489:00 +CPU= 40.146
  07:31:37 STEP  490 H= 490:00 +CPU= 23.573
  07:32:03 STEP  491 H= 491:00 +CPU= 25.337
  07:32:45 STEP  492 H= 492:00 +CPU= 41.912
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000492
  07:33:08 STEP  493 H= 493:00 +CPU= 22.539
  07:33:34 STEP  494 H= 494:00 +CPU= 25.822
  07:34:10 STEP  495 H= 495:00 +CPU= 35.877
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:34:55 STEP  496 H= 496:00 +CPU= 30.354
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:35:42 STEP  497 H= 497:00 +CPU= 31.811
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:36:52 STEP  498 H= 498:00 +CPU= 51.754
  07:37:21 STEP  499 H= 499:00 +CPU= 28.066
  07:38:01 STEP  500 H= 500:00 +CPU= 35.489
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:39:03 STEP  501 H= 501:00 +CPU= 46.788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:39:47 STEP  502 H= 502:00 +CPU= 30.356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:41:38 STEP  503 H= 503:00 +CPU= 34.708
  07:42:29 STEP  504 H= 504:00 +CPU= 48.258
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_20.zip
Uploading the intermediate file: upload_file_20.zip
  07:42:58 STEP  505 H= 505:00 +CPU= 26.788
  07:43:24 STEP  506 H= 506:00 +CPU= 25.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:44:45 STEP  507 H= 507:00 +CPU= 45.316
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:45:26 STEP  508 H= 508:00 +CPU= 28.590
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:46:13 STEP  509 H= 509:00 +CPU= 30.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0493
(argv4) batchid: 953
(argv5) wuid: 12170137
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_e69ff5d6a2ec073e5c749bcec0ce939d to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0493_1984050100_123_953_12170137.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0493_1984050100_123_953_12170137.zip
ic_ancil_file: ic_ancil_12170137
ifsdata_file: ifsdata_12170137
climate_data_file: clim_data_12170137
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a98c5b63311a9716a767d357dc5b3a5a to: /var/lib/boinc-client/slots/2/ic_ancil_12170137.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12170137.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_843ae2c75d0c38ccf14ccec253bbb42e to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12170137.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12170137.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12170137.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12170137.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12170137.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12170137.xml
-- Model is restarting --
Adjusting last_iter, 510, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12170137.xml
last_cpu_time: 22929
upload_file_number: 21
last_iter: 503
last_upload: 1814400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903
The child process has been launched with process id: 13615
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:13615:13615] [20230205:074808:1675612088:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:49:22 STEP  504 H= 504:00 +CPU= 53.132
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:50:16 STEP  505 H= 505:00 +CPU= 29.589
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:51:02 STEP  506 H= 506:00 +CPU= 31.551
  07:51:48 STEP  507 H= 507:00 +CPU= 42.745
  07:52:21 STEP  508 H= 508:00 +CPU= 29.894
  07:52:49 STEP  509 H= 509:00 +CPU= 26.889
  07:53:35 STEP  510 H= 510:00 +CPU= 43.357
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:54:23 STEP  511 H= 511:00 +CPU= 32.436
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:55:26 STEP  512 H= 512:00 +CPU= 27.486
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:57:13 STEP  513 H= 513:00 +CPU= 47.687
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:58:16 STEP  514 H= 514:00 +CPU= 28.746
  07:58:39 STEP  515 H= 515:00 +CPU= 22.725
  07:59:19 STEP  516 H= 516:00 +CPU= 39.415
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000516
  07:59:42 STEP  517 H= 517:00 +CPU= 22.222
  08:00:05 STEP  518 H= 518:00 +CPU= 23.377
  08:00:41 STEP  519 H= 519:00 +CPU= 35.317
  08:01:05 STEP  520 H= 520:00 +CPU= 23.423
  08:01:29 STEP  521 H= 521:00 +CPU= 23.254
  08:02:05 STEP  522 H= 522:00 +CPU= 36.036
  08:02:28 STEP  523 H= 523:00 +CPU= 22.452
  08:02:51 STEP  524 H= 524:00 +CPU= 23.268
  08:03:27 STEP  525 H= 525:00 +CPU= 35.006
  08:03:50 STEP  526 H= 526:00 +CPU= 23.219
  08:04:14 STEP  527 H= 527:00 +CPU= 23.364
  08:04:58 STEP  528 H= 528:00 +CPU= 42.840
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_21.zip
Uploading the intermediate file: upload_file_21.zip
  08:05:32 STEP  529 H= 529:00 +CPU= 22.595
  08:05:57 STEP  530 H= 530:00 +CPU= 24.469
  08:06:33 STEP  531 H= 531:00 +CPU= 35.309
  08:06:56 STEP  532 H= 532:00 +CPU= 22.939
  08:07:19 STEP  533 H= 533:00 +CPU= 22.575
  08:07:55 STEP  534 H= 534:00 +CPU= 35.837
  08:08:18 STEP  535 H= 535:00 +CPU= 22.576
  08:08:42 STEP  536 H= 536:00 +CPU= 23.455
  08:09:17 STEP  537 H= 537:00 +CPU= 35.093
  08:09:41 STEP  538 H= 538:00 +CPU= 23.158
  08:10:04 STEP  539 H= 539:00 +CPU= 22.721
  08:10:44 STEP  540 H= 540:00 +CPU= 39.619
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000540
  08:11:06 STEP  541 H= 541:00 +CPU= 22.195
  08:11:30 STEP  542 H= 542:00 +CPU= 23.297
  08:12:05 STEP  543 H= 543:00 +CPU= 34.993
  08:12:28 STEP  544 H= 544:00 +CPU= 23.181
  08:12:51 STEP  545 H= 545:00 +CPU= 22.434
  08:13:27 STEP  546 H= 546:00 +CPU= 35.699
  08:13:50 STEP  547 H= 547:00 +CPU= 22.724
  08:14:13 STEP  548 H= 548:00 +CPU= 23.010
  08:14:48 STEP  549 H= 549:00 +CPU= 34.806
  08:15:12 STEP  550 H= 550:00 +CPU= 22.964
  08:15:35 STEP  551 H= 551:00 +CPU= 23.166
  08:16:17 STEP  552 H= 552:00 +CPU= 41.362
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_22.zip
Uploading the intermediate file: upload_file_22.zip
  08:16:39 STEP  553 H= 553:00 +CPU= 22.341
  08:17:03 STEP  554 H= 554:00 +CPU= 23.492
  08:17:40 STEP  555 H= 555:00 +CPU= 36.540
  08:18:09 STEP  556 H= 556:00 +CPU= 27.172
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:18:49 STEP  557 H= 557:00 +CPU= 27.255
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:20:15 STEP  558 H= 558:00 +CPU= 47.633
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:21:19 STEP  559 H= 559:00 +CPU= 29.816
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:22:16 STEP  560 H= 560:00 +CPU= 31.670
  08:23:02 STEP  561 H= 561:00 +CPU= 43.198
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:23:55 STEP  562 H= 562:00 +CPU= 29.009
  08:24:21 STEP  563 H= 563:00 +CPU= 25.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:25:22 STEP  564 H= 564:00 +CPU= 46.764
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000564
  08:25:50 STEP  565 H= 565:00 +CPU= 26.467
  08:26:21 STEP  566 H= 566:00 +CPU= 28.349
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:27:19 STEP  567 H= 567:00 +CPU= 44.183
  08:27:50 STEP  568 H= 568:00 +CPU= 28.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:28:47 STEP  569 H= 569:00 +CPU= 31.234
  08:29:36 STEP  570 H= 570:00 +CPU= 45.730
  08:30:08 STEP  571 H= 571:00 +CPU= 28.584
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:30:59 STEP  572 H= 572:00 +CPU= 27.803
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:32:00 STEP  573 H= 573:00 +CPU= 46.093
  08:32:26 STEP  574 H= 574:00 +CPU= 25.470
  08:32:55 STEP  575 H= 575:00 +CPU= 27.154
  08:33:52 STEP  576 H= 576:00 +CPU= 52.648
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170137/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0493_1984050100_123_953_12170137_0_r2086723903_23.zip
Uploading the intermediate file: upload_file_23.zip
  08:34:14 STEP  577 H= 577:00 +CPU= 22.462
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:35:12 STEP  578 H= 578:00 +CPU= 31.666
Suspend request received from the BOINC client, suspending the child process
double free or corruption (out)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:47:10 1509518 22254452 oifs_43r3_ps_0493_1984050100_123_953_12170137_0 1,123,200 14,118 0.0126


©2024 cpdn.org