climateprediction.net home page
Task 22256233

Task 22256233

Name oifs_43r3_ps_0206_1986050100_123_955_12171850_0
Workunit 12171850
Created 21 Dec 2022, 8:01:50 UTC
Sent 11 Jan 2023, 21:13:25 UTC
Report deadline 10 Feb 2023, 21:13:25 UTC
Received 14 Jan 2023, 1:14:17 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1519251
Run time 5 hours 50 min 32 sec
CPU time 4 hours 35 min 40 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.94 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,417.30 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,675.97 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
358:00 +CPU= 20.518
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:25:43 STEP  359 H= 359:00 +CPU= 21.525
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:28:34 STEP  360 H= 360:00 +CPU= 39.053
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_14.zip
Uploading the intermediate file: upload_file_14.zip
  03:28:53 STEP  361 H= 361:00 +CPU= 19.136
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:31:16 STEP  362 H= 362:00 +CPU= 19.530
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:34:09 STEP  363 H= 363:00 +CPU= 32.584
  03:34:31 STEP  364 H= 364:00 +CPU= 21.381
  03:34:51 STEP  365 H= 365:00 +CPU= 20.128
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:38:16 STEP  366 H= 366:00 +CPU= 33.622
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:40:58 STEP  367 H= 367:00 +CPU= 20.389
  03:41:19 STEP  368 H= 368:00 +CPU= 21.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:44:33 STEP  369 H= 369:00 +CPU= 32.264
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:46:55 STEP  370 H= 370:00 +CPU= 21.099
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:49:17 STEP  371 H= 371:00 +CPU= 21.101
  03:49:53 STEP  372 H= 372:00 +CPU= 35.669
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000372
  03:52:35 STEP  373 H= 373:00 +CPU= 20.957
  03:52:56 STEP  374 H= 374:00 +CPU= 20.939
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:57:11 STEP  375 H= 375:00 +CPU= 33.582
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:59:23 STEP  376 H= 376:00 +CPU= 21.291
  03:59:43 STEP  377 H= 377:00 +CPU= 20.251
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:02:47 STEP  378 H= 378:00 +CPU= 32.880
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:05:30 STEP  379 H= 379:00 +CPU= 20.999
  04:05:51 STEP  380 H= 380:00 +CPU= 21.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:08:55 STEP  381 H= 381:00 +CPU= 32.872
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:11:59 STEP  382 H= 382:00 +CPU= 21.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:16:51 STEP  383 H= 383:00 +CPU= 20.745
  04:17:28 STEP  384 H= 384:00 +CPU= 37.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_15.zip
Uploading the intermediate file: upload_file_15.zip
  04:20:20 STEP  385 H= 385:00 +CPU= 20.777
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:23:02 STEP  386 H= 386:00 +CPU= 20.354
  04:23:35 STEP  387 H= 387:00 +CPU= 33.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:26:48 STEP  388 H= 388:00 +CPU= 21.215
  04:27:09 STEP  389 H= 389:00 +CPU= 20.885
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:30:25 STEP  390 H= 390:00 +CPU= 34.310
  04:30:46 STEP  391 H= 391:00 +CPU= 20.662
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:34:08 STEP  392 H= 392:00 +CPU= 20.874
  04:34:41 STEP  393 H= 393:00 +CPU= 32.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:37:34 STEP  394 H= 394:00 +CPU= 21.703
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:39:56 STEP  395 H= 395:00 +CPU= 20.231
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:43:04 STEP  396 H= 396:00 +CPU= 36.588
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000396
  04:45:26 STEP  397 H= 397:00 +CPU= 21.187
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0206
(argv4) batchid: 955
(argv5) wuid: 12171850
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_020fb844f8c1143203e3588395280d66 to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0206_1986050100_123_955_12171850.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0206_1986050100_123_955_12171850.zip
ic_ancil_file: ic_ancil_12171850
ifsdata_file: ifsdata_12171850
climate_data_file: clim_data_12171850
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_13a3a0fa5a2f939d2cb61020d3ed6e9e to: /var/lib/boinc-client/slots/14/ic_ancil_12171850.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12171850.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f4a2f97172d9c677f628562351eb1ce8 to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12171850.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12171850.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12171850.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12171850.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12171850.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12171850.xml
-- Model is restarting --
Adjusting last_iter, 397, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12171850.xml
last_cpu_time: 12020
upload_file_number: 16
last_iter: 383
last_upload: 1382400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900
The child process has been launched with process id: 1931069
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1931069:1931069] [20230114:044806:1673639286:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:50:37 STEP  384 H= 384:00 +CPU= 36.847
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:53:18 STEP  385 H= 385:00 +CPU= 20.407
  04:53:40 STEP  386 H= 386:00 +CPU= 21.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:56:42 STEP  387 H= 387:00 +CPU= 32.392
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:59:25 STEP  388 H= 388:00 +CPU= 21.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:02:48 STEP  389 H= 389:00 +CPU= 21.475
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:05:12 STEP  390 H= 390:00 +CPU= 33.419
  05:05:33 STEP  391 H= 391:00 +CPU= 20.485
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:07:57 STEP  392 H= 392:00 +CPU= 21.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:11:54 STEP  393 H= 393:00 +CPU= 33.224
  05:12:14 STEP  394 H= 394:00 +CPU= 20.733
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:14:57 STEP  395 H= 395:00 +CPU= 20.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:17:53 STEP  396 H= 396:00 +CPU= 36.099
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000396
  05:18:14 STEP  397 H= 397:00 +CPU= 20.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:20:47 STEP  398 H= 398:00 +CPU= 21.006
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:23:21 STEP  399 H= 399:00 +CPU= 31.952
  05:23:42 STEP  400 H= 400:00 +CPU= 21.652
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:26:14 STEP  401 H= 401:00 +CPU= 20.718
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:29:10 STEP  402 H= 402:00 +CPU= 33.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:31:54 STEP  403 H= 403:00 +CPU= 21.676
  05:32:15 STEP  404 H= 404:00 +CPU= 21.257
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:35:39 STEP  405 H= 405:00 +CPU= 32.831
  05:36:01 STEP  406 H= 406:00 +CPU= 21.050
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:38:53 STEP  407 H= 407:00 +CPU= 20.956
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:53 STEP  408 H= 408:00 +CPU= 39.130
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_16.zip
Uploading the intermediate file: upload_file_16.zip
  05:42:14 STEP  409 H= 409:00 +CPU= 20.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:51 STEP  410 H= 410:00 +CPU= 20.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:47:55 STEP  411 H= 411:00 +CPU= 32.689
  05:48:17 STEP  412 H= 412:00 +CPU= 21.697
  05:48:37 STEP  413 H= 413:00 +CPU= 20.403
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:52:43 STEP  414 H= 414:00 +CPU= 33.899
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:56:25 STEP  415 H= 415:00 +CPU= 20.427
  05:56:47 STEP  416 H= 416:00 +CPU= 21.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:41 STEP  417 H= 417:00 +CPU= 32.824
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:02:04 STEP  418 H= 418:00 +CPU= 21.413
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:03:46 STEP  419 H= 419:00 +CPU= 20.706
  06:04:22 STEP  420 H= 420:00 +CPU= 35.398
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000420
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:06:55 STEP  421 H= 421:00 +CPU= 20.692
  06:07:15 STEP  422 H= 422:00 +CPU= 20.552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:09:49 STEP  423 H= 423:00 +CPU= 32.780
  06:10:10 STEP  424 H= 424:00 +CPU= 20.935
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:12:42 STEP  425 H= 425:00 +CPU= 21.074
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:14:57 STEP  426 H= 426:00 +CPU= 33.151
  06:15:17 STEP  427 H= 427:00 +CPU= 20.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:17:40 STEP  428 H= 428:00 +CPU= 21.522
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:21:06 STEP  429 H= 429:00 +CPU= 32.400
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:22:48 STEP  430 H= 430:00 +CPU= 21.302
  06:23:10 STEP  431 H= 431:00 +CPU= 21.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:27:02 STEP  432 H= 432:00 +CPU= 39.314
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_17.zip
Uploading the intermediate file: upload_file_17.zip
  06:27:23 STEP  433 H= 433:00 +CPU= 21.132
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:29:56 STEP  434 H= 434:00 +CPU= 20.911
  06:30:27 STEP  435 H= 435:00 +CPU= 31.747
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:33:10 STEP  436 H= 436:00 +CPU= 21.268
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:35:01 STEP  437 H= 437:00 +CPU= 20.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:36:46 STEP  438 H= 438:00 +CPU= 33.129
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:38:47 STEP  439 H= 439:00 +CPU= 20.289
  06:39:09 STEP  440 H= 440:00 +CPU= 21.620
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:42:22 STEP  441 H= 441:00 +CPU= 32.400
  06:42:44 STEP  442 H= 442:00 +CPU= 21.100
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:45:05 STEP  443 H= 443:00 +CPU= 20.561
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:48:02 STEP  444 H= 444:00 +CPU= 35.483
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000444
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0206
(argv4) batchid: 955
(argv5) wuid: 12171850
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_020fb844f8c1143203e3588395280d66 to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0206_1986050100_123_955_12171850.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0206_1986050100_123_955_12171850.zip
ic_ancil_file: ic_ancil_12171850
ifsdata_file: ifsdata_12171850
climate_data_file: clim_data_12171850
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_13a3a0fa5a2f939d2cb61020d3ed6e9e to: /var/lib/boinc-client/slots/14/ic_ancil_12171850.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12171850.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f4a2f97172d9c677f628562351eb1ce8 to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12171850.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12171850.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12171850.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12171850.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12171850.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12171850.xml
-- Model is restarting --
Adjusting last_iter, 445, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12171850.xml
last_cpu_time: 13576
upload_file_number: 18
last_iter: 431
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900
The child process has been launched with process id: 1954759
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1954759:1954759] [20230114:065031:1673646631:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  06:51:10 STEP  432 H= 432:00 +CPU= 36.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:53:31 STEP  433 H= 433:00 +CPU= 20.215
  06:53:52 STEP  434 H= 434:00 +CPU= 20.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:56:26 STEP  435 H= 435:00 +CPU= 32.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:58:38 STEP  436 H= 436:00 +CPU= 20.806
  06:58:58 STEP  437 H= 437:00 +CPU= 19.805
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:01:53 STEP  438 H= 438:00 +CPU= 33.126
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:05:25 STEP  439 H= 439:00 +CPU= 20.511
  07:05:47 STEP  440 H= 440:00 +CPU= 21.233
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:08:32 STEP  441 H= 441:00 +CPU= 33.038
  07:08:53 STEP  442 H= 442:00 +CPU= 20.955
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:12:26 STEP  443 H= 443:00 +CPU= 20.555
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:14:54 STEP  444 H= 444:00 +CPU= 36.131
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:17:07 STEP  445 H= 445:00 +CPU= 21.579
  07:17:29 STEP  446 H= 446:00 +CPU= 21.625
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:20:34 STEP  447 H= 447:00 +CPU= 33.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:23:07 STEP  448 H= 448:00 +CPU= 21.365
  07:23:28 STEP  449 H= 449:00 +CPU= 20.477
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:26:13 STEP  450 H= 450:00 +CPU= 33.556
  07:26:34 STEP  451 H= 451:00 +CPU= 20.825
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:29:16 STEP  452 H= 452:00 +CPU= 20.685
  07:29:49 STEP  453 H= 453:00 +CPU= 32.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:31:52 STEP  454 H= 454:00 +CPU= 21.527
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:33:55 STEP  455 H= 455:00 +CPU= 20.932
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:37:07 STEP  456 H= 456:00 +CPU= 39.416
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_18.zip
Uploading the intermediate file: upload_file_18.zip
  07:37:28 STEP  457 H= 457:00 +CPU= 20.852
  07:37:48 STEP  458 H= 458:00 +CPU= 19.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:40:22 STEP  459 H= 459:00 +CPU= 32.833
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:43:04 STEP  460 H= 460:00 +CPU= 20.606
  07:43:24 STEP  461 H= 461:00 +CPU= 20.468
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:45:50 STEP  462 H= 462:00 +CPU= 33.195
  07:46:10 STEP  463 H= 463:00 +CPU= 20.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:49:03 STEP  464 H= 464:00 +CPU= 20.882
  07:49:35 STEP  465 H= 465:00 +CPU= 32.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:51:57 STEP  466 H= 466:00 +CPU= 21.179
  07:52:18 STEP  467 H= 467:00 +CPU= 20.729
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:54:36 STEP  468 H= 468:00 +CPU= 36.103
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000468
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:57:27 STEP  469 H= 469:00 +CPU= 20.047
  07:57:48 STEP  470 H= 470:00 +CPU= 21.256
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:00:33 STEP  471 H= 471:00 +CPU= 32.842
  08:00:55 STEP  472 H= 472:00 +CPU= 21.299
  08:01:14 STEP  473 H= 473:00 +CPU= 19.439
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:04:09 STEP  474 H= 474:00 +CPU= 33.546
  08:04:29 STEP  475 H= 475:00 +CPU= 19.975
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:07:01 STEP  476 H= 476:00 +CPU= 21.038
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:09:49 STEP  477 H= 477:00 +CPU= 31.922
  08:10:10 STEP  478 H= 478:00 +CPU= 21.154
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:12:23 STEP  479 H= 479:00 +CPU= 20.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:14:53 STEP  480 H= 480:00 +CPU= 38.369
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_19.zip
Uploading the intermediate file: upload_file_19.zip
  08:15:13 STEP  481 H= 481:00 +CPU= 20.007
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:17:02 STEP  482 H= 482:00 +CPU= 19.104
  08:17:34 STEP  483 H= 483:00 +CPU= 31.734
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:19:56 STEP  484 H= 484:00 +CPU= 20.974
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:22:10 STEP  485 H= 485:00 +CPU= 22.088
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:24:04 STEP  486 H= 486:00 +CPU= 33.204
  08:24:26 STEP  487 H= 487:00 +CPU= 21.583
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:26:38 STEP  488 H= 488:00 +CPU= 21.042
  08:27:10 STEP  489 H= 489:00 +CPU= 31.712
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:29:52 STEP  490 H= 490:00 +CPU= 21.102
  08:30:12 STEP  491 H= 491:00 +CPU= 20.283
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:34:02 STEP  492 H= 492:00 +CPU= 36.766
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000492
  08:34:22 STEP  493 H= 493:00 +CPU= 20.260
  08:34:42 STEP  494 H= 494:00 +CPU= 20.492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:37:05 STEP  495 H= 495:00 +CPU= 32.467
  08:37:26 STEP  496 H= 496:00 +CPU= 20.914
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:40:09 STEP  497 H= 497:00 +CPU= 20.233
  08:40:41 STEP  498 H= 498:00 +CPU= 32.762
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:43:24 STEP  499 H= 499:00 +CPU= 20.357
  08:43:45 STEP  500 H= 500:00 +CPU= 21.305
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:45:59 STEP  501 H= 501:00 +CPU= 32.564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:48:31 STEP  502 H= 502:00 +CPU= 20.616
  08:48:52 STEP  503 H= 503:00 +CPU= 21.459
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0206
(argv4) batchid: 955
(argv5) wuid: 12171850
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_020fb844f8c1143203e3588395280d66 to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0206_1986050100_123_955_12171850.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0206_1986050100_123_955_12171850.zip
ic_ancil_file: ic_ancil_12171850
ifsdata_file: ifsdata_12171850
climate_data_file: clim_data_12171850
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_13a3a0fa5a2f939d2cb61020d3ed6e9e to: /var/lib/boinc-client/slots/14/ic_ancil_12171850.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12171850.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f4a2f97172d9c677f628562351eb1ce8 to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12171850.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12171850.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12171850.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12171850.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12171850.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12171850.xml
-- Model is restarting --
Adjusting last_iter, 504, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12171850.xml
last_cpu_time: 15401
upload_file_number: 20
last_iter: 503
last_upload: 1728000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900
The child process has been launched with process id: 1972948
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1972948:1972948] [20230114:085436:1673654076:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:17 STEP  504 H= 504:00 +CPU= 37.386
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_20.zip
Uploading the intermediate file: upload_file_20.zip
  08:57:38 STEP  505 H= 505:00 +CPU= 20.786
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:59:19 STEP  506 H= 506:00 +CPU= 19.166
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:02:23 STEP  507 H= 507:00 +CPU= 32.004
  09:02:45 STEP  508 H= 508:00 +CPU= 22.024
  09:03:05 STEP  509 H= 509:00 +CPU= 20.135
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:06:30 STEP  510 H= 510:00 +CPU= 33.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:08:52 STEP  511 H= 511:00 +CPU= 21.052
  09:09:14 STEP  512 H= 512:00 +CPU= 21.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:11:27 STEP  513 H= 513:00 +CPU= 32.367
  09:11:48 STEP  514 H= 514:00 +CPU= 20.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:14:10 STEP  515 H= 515:00 +CPU= 21.027
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:16:29 STEP  516 H= 516:00 +CPU= 36.221
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:18:22 STEP  517 H= 517:00 +CPU= 20.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:20:45 STEP  518 H= 518:00 +CPU= 21.156
  09:21:18 STEP  519 H= 519:00 +CPU= 32.690
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:23:51 STEP  520 H= 520:00 +CPU= 21.085
  09:24:12 STEP  521 H= 521:00 +CPU= 20.951
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:26:47 STEP  522 H= 522:00 +CPU= 33.430
  09:27:07 STEP  523 H= 523:00 +CPU= 20.525
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:29:20 STEP  524 H= 524:00 +CPU= 21.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:31:54 STEP  525 H= 525:00 +CPU= 31.851
  09:32:15 STEP  526 H= 526:00 +CPU= 21.600
  09:32:36 STEP  527 H= 527:00 +CPU= 20.509
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:35:25 STEP  528 H= 528:00 +CPU= 38.074
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12171850/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0206_1986050100_123_955_12171850_0_r836332900_21.zip
Uploading the intermediate file: upload_file_21.zip
  09:35:44 STEP  529 H= 529:00 +CPU= 18.527
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:37:48 STEP  530 H= 530:00 +CPU= 20.463
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:39:51 STEP  531 H= 531:00 +CPU= 32.631
  09:40:12 STEP  532 H= 532:00 +CPU= 20.620
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:42:15 STEP  533 H= 533:00 +CPU= 20.572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:44:30 STEP  534 H= 534:00 +CPU= 33.690
  09:44:51 STEP  535 H= 535:00 +CPU= 20.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:47:03 STEP  536 H= 536:00 +CPU= 21.827
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:49:46 STEP  537 H= 537:00 +CPU= 32.126
  09:50:08 STEP  538 H= 538:00 +CPU= 21.603
  09:50:28 STEP  539 H= 539:00 +CPU= 19.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:54:37 STEP  540 H= 540:00 +CPU= 36.003
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000540
  09:54:58 STEP  541 H= 541:00 +CPU= 20.670
  09:55:19 STEP  542 H= 542:00 +CPU= 21.312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:57:43 STEP  543 H= 543:00 +CPU= 32.218
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:00:05 STEP  544 H= 544:00 +CPU= 20.802
  10:00:26 STEP  545 H= 545:00 +CPU= 20.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:05:30 STEP  546 H= 546:00 +CPU= 33.172
  10:05:50 STEP  547 H= 547:00 +CPU= 20.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:08:22 STEP  548 H= 548:00 +CPU= 21.013
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net