climateprediction.net home page
Task 22276432

Task 22276432

Name oifs_43r3_ps_0137_2005050100_123_974_12190781_0
Workunit 12190781
Created 23 Dec 2022, 9:47:12 UTC
Sent 11 Jan 2023, 18:38:14 UTC
Report deadline 10 Feb 2023, 18:38:14 UTC
Received 12 Jan 2023, 23:10:54 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1519251
Run time 4 hours 42 min 43 sec
CPU time 4 hours 34 min 17 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.94 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,416.89 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,816.93 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
 upload_file_14.zip
  00:51:32 STEP  361 H= 361:00 +CPU= 19.388
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:53:14 STEP  362 H= 362:00 +CPU= 20.371
  00:53:47 STEP  363 H= 363:00 +CPU= 32.713
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:11 STEP  364 H= 364:00 +CPU= 21.060
  00:57:32 STEP  365 H= 365:00 +CPU= 20.116
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:59:37 STEP  366 H= 366:00 +CPU= 33.895
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:01:38 STEP  367 H= 367:00 +CPU= 20.817
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:19 STEP  368 H= 368:00 +CPU= 20.676
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:05:13 STEP  369 H= 369:00 +CPU= 31.846
  01:05:35 STEP  370 H= 370:00 +CPU= 21.748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:15 STEP  371 H= 371:00 +CPU= 20.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:09:23 STEP  372 H= 372:00 +CPU= 36.075
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:11:04 STEP  373 H= 373:00 +CPU= 20.508
  01:11:25 STEP  374 H= 374:00 +CPU= 21.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:14:30 STEP  375 H= 375:00 +CPU= 32.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:16:04 STEP  376 H= 376:00 +CPU= 21.488
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:17:25 STEP  377 H= 377:00 +CPU= 20.152
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2005050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0137
(argv4) batchid: 974
(argv5) wuid: 12190781
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5e0e83c7051f5d56a647034d48ad33ef to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0137_2005050100_123_974_12190781.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0137_2005050100_123_974_12190781.zip
ic_ancil_file: ic_ancil_12190781
ifsdata_file: ifsdata_12190781
climate_data_file: clim_data_12190781
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e0907031a15d50175cb6f939dfc2cd39 to: /var/lib/boinc-client/slots/4/ic_ancil_12190781.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12190781.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6ba621602dbf45f1f230bff246f86a5f to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12190781.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12190781.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12190781.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12190781.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12190781.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12190781.xml
-- Model is restarting --
Adjusting last_iter, 378, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12190781.xml
last_cpu_time: 11490
upload_file_number: 15
last_iter: 359
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417
The child process has been launched with process id: 1632920
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632920:1632920] [20230113:011933:1673540373:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:21:44 STEP  360 H= 360:00 +CPU= 37.035
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:23:46 STEP  361 H= 361:00 +CPU= 20.406
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:25:09 STEP  362 H= 362:00 +CPU= 21.584
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:52 STEP  363 H= 363:00 +CPU= 32.583
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:28:44 STEP  364 H= 364:00 +CPU= 20.751
  01:29:05 STEP  365 H= 365:00 +CPU= 20.297
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:31:07 STEP  366 H= 366:00 +CPU= 32.461
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:33:00 STEP  367 H= 367:00 +CPU= 20.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:52 STEP  368 H= 368:00 +CPU= 21.071
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:36:46 STEP  369 H= 369:00 +CPU= 32.539
  01:37:07 STEP  370 H= 370:00 +CPU= 20.785
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:38 STEP  371 H= 371:00 +CPU= 20.201
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:25 STEP  372 H= 372:00 +CPU= 35.781
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000372
  01:40:46 STEP  373 H= 373:00 +CPU= 20.895
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:42:39 STEP  374 H= 374:00 +CPU= 21.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:44:32 STEP  375 H= 375:00 +CPU= 31.673
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:46:14 STEP  376 H= 376:00 +CPU= 21.487
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:47:55 STEP  377 H= 377:00 +CPU= 20.259
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:49:50 STEP  378 H= 378:00 +CPU= 33.101
  01:50:10 STEP  379 H= 379:00 +CPU= 20.257
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:52:03 STEP  380 H= 380:00 +CPU= 21.621
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:54:05 STEP  381 H= 381:00 +CPU= 32.293
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:55:48 STEP  382 H= 382:00 +CPU= 20.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:57:31 STEP  383 H= 383:00 +CPU= 21.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:59:41 STEP  384 H= 384:00 +CPU= 37.330
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417_15.zip
Uploading the intermediate file: upload_file_15.zip
  02:00:01 STEP  385 H= 385:00 +CPU= 20.456
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:02:10 STEP  386 H= 386:00 +CPU= 20.832
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:04:13 STEP  387 H= 387:00 +CPU= 31.892
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:05:35 STEP  388 H= 388:00 +CPU= 21.825
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:07:38 STEP  389 H= 389:00 +CPU= 20.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:09:23 STEP  390 H= 390:00 +CPU= 33.179
  02:09:44 STEP  391 H= 391:00 +CPU= 20.400
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:11:07 STEP  392 H= 392:00 +CPU= 21.098
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:13:10 STEP  393 H= 393:00 +CPU= 32.536
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:15:02 STEP  394 H= 394:00 +CPU= 21.385
  02:15:23 STEP  395 H= 395:00 +CPU= 20.541
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:17:39 STEP  396 H= 396:00 +CPU= 35.331
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:19:32 STEP  397 H= 397:00 +CPU= 20.938
  02:19:53 STEP  398 H= 398:00 +CPU= 21.418
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:21:55 STEP  399 H= 399:00 +CPU= 32.349
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:23:18 STEP  400 H= 400:00 +CPU= 21.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:25:11 STEP  401 H= 401:00 +CPU= 20.821
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:27:06 STEP  402 H= 402:00 +CPU= 33.235
  02:27:27 STEP  403 H= 403:00 +CPU= 20.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:29:19 STEP  404 H= 404:00 +CPU= 20.936
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:31:22 STEP  405 H= 405:00 +CPU= 31.441
  02:31:43 STEP  406 H= 406:00 +CPU= 21.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:34:06 STEP  407 H= 407:00 +CPU= 21.048
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:36:25 STEP  408 H= 408:00 +CPU= 37.746
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417_16.zip
Uploading the intermediate file: upload_file_16.zip
  02:36:44 STEP  409 H= 409:00 +CPU= 19.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:38:16 STEP  410 H= 410:00 +CPU= 20.131
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:40:08 STEP  411 H= 411:00 +CPU= 31.445
  02:40:29 STEP  412 H= 412:00 +CPU= 20.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:42:40 STEP  413 H= 413:00 +CPU= 20.160
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:44:45 STEP  414 H= 414:00 +CPU= 32.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:46:28 STEP  415 H= 415:00 +CPU= 21.173
  02:46:49 STEP  416 H= 416:00 +CPU= 21.016
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:48:42 STEP  417 H= 417:00 +CPU= 32.227
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:50:25 STEP  418 H= 418:00 +CPU= 21.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:52:06 STEP  419 H= 419:00 +CPU= 20.304
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:54:23 STEP  420 H= 420:00 +CPU= 35.454
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000420
  02:54:43 STEP  421 H= 421:00 +CPU= 19.983
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:56:34 STEP  422 H= 422:00 +CPU= 20.643
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:58:28 STEP  423 H= 423:00 +CPU= 31.456
  02:58:49 STEP  424 H= 424:00 +CPU= 20.743
  02:59:09 STEP  425 H= 425:00 +CPU= 20.020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:01:22 STEP  426 H= 426:00 +CPU= 32.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:03:34 STEP  427 H= 427:00 +CPU= 19.542
  03:03:55 STEP  428 H= 428:00 +CPU= 20.990
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:05:48 STEP  429 H= 429:00 +CPU= 31.688
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:08:10 STEP  430 H= 430:00 +CPU= 21.502
  03:08:31 STEP  431 H= 431:00 +CPU= 20.926
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:10:39 STEP  432 H= 432:00 +CPU= 37.112
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417_17.zip
Uploading the intermediate file: upload_file_17.zip
  03:10:58 STEP  433 H= 433:00 +CPU= 18.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:13:01 STEP  434 H= 434:00 +CPU= 18.430
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:14:54 STEP  435 H= 435:00 +CPU= 32.509
  03:15:16 STEP  436 H= 436:00 +CPU= 21.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:17:07 STEP  437 H= 437:00 +CPU= 19.982
  03:17:40 STEP  438 H= 438:00 +CPU= 33.280
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2005050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0137
(argv4) batchid: 974
(argv5) wuid: 12190781
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5e0e83c7051f5d56a647034d48ad33ef to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0137_2005050100_123_974_12190781.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0137_2005050100_123_974_12190781.zip
ic_ancil_file: ic_ancil_12190781
ifsdata_file: ifsdata_12190781
climate_data_file: clim_data_12190781
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e0907031a15d50175cb6f939dfc2cd39 to: /var/lib/boinc-client/slots/4/ic_ancil_12190781.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12190781.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6ba621602dbf45f1f230bff246f86a5f to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12190781.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12190781.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12190781.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12190781.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12190781.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12190781.xml
-- Model is restarting --
Adjusting last_iter, 439, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12190781.xml
last_cpu_time: 13478
upload_file_number: 18
last_iter: 431
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417
The child process has been launched with process id: 1650336
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650336:1650336] [20230113:032107:1673547667:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:23:07 STEP  432 H= 432:00 +CPU= 36.195
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:24:19 STEP  433 H= 433:00 +CPU= 20.688
  03:24:40 STEP  434 H= 434:00 +CPU= 20.769
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:26:44 STEP  435 H= 435:00 +CPU= 33.096
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:28:25 STEP  436 H= 436:00 +CPU= 20.886
  03:28:47 STEP  437 H= 437:00 +CPU= 21.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:30:52 STEP  438 H= 438:00 +CPU= 33.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:32:24 STEP  439 H= 439:00 +CPU= 20.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:33:46 STEP  440 H= 440:00 +CPU= 21.388
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:35:30 STEP  441 H= 441:00 +CPU= 32.831
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:37:03 STEP  442 H= 442:00 +CPU= 21.792
  03:37:23 STEP  443 H= 443:00 +CPU= 20.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:39:29 STEP  444 H= 444:00 +CPU= 35.632
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:41:10 STEP  445 H= 445:00 +CPU= 20.043
  03:41:31 STEP  446 H= 446:00 +CPU= 20.209
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:43:33 STEP  447 H= 447:00 +CPU= 31.640
  03:43:55 STEP  448 H= 448:00 +CPU= 21.418
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:45:47 STEP  449 H= 449:00 +CPU= 21.214
  03:46:20 STEP  450 H= 450:00 +CPU= 33.029
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:48:12 STEP  451 H= 451:00 +CPU= 20.422
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:49:52 STEP  452 H= 452:00 +CPU= 20.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:51:36 STEP  453 H= 453:00 +CPU= 32.089
  03:51:57 STEP  454 H= 454:00 +CPU= 21.049
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:54:20 STEP  455 H= 455:00 +CPU= 20.571
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:56:39 STEP  456 H= 456:00 +CPU= 38.343
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417_18.zip
Uploading the intermediate file: upload_file_18.zip
  03:57:00 STEP  457 H= 457:00 +CPU= 20.579
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:58:43 STEP  458 H= 458:00 +CPU= 21.328
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:00:37 STEP  459 H= 459:00 +CPU= 32.261
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:02:19 STEP  460 H= 460:00 +CPU= 21.586
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:04:01 STEP  461 H= 461:00 +CPU= 20.510
  04:04:34 STEP  462 H= 462:00 +CPU= 33.070
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:06:36 STEP  463 H= 463:00 +CPU= 20.600
  04:06:57 STEP  464 H= 464:00 +CPU= 21.189
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:09:02 STEP  465 H= 465:00 +CPU= 32.850
  04:09:24 STEP  466 H= 466:00 +CPU= 21.373
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:11:25 STEP  467 H= 467:00 +CPU= 20.524
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:13:21 STEP  468 H= 468:00 +CPU= 34.893
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000468
  04:13:43 STEP  469 H= 469:00 +CPU= 21.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:15:35 STEP  470 H= 470:00 +CPU= 20.699
  04:16:07 STEP  471 H= 471:00 +CPU= 31.936
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:18:00 STEP  472 H= 472:00 +CPU= 21.984
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:20:02 STEP  473 H= 473:00 +CPU= 20.911
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:21:46 STEP  474 H= 474:00 +CPU= 33.134
  04:22:07 STEP  475 H= 475:00 +CPU= 20.769
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:24:00 STEP  476 H= 476:00 +CPU= 21.872
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:25:44 STEP  477 H= 477:00 +CPU= 32.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:27:16 STEP  478 H= 478:00 +CPU= 21.562
  04:27:38 STEP  479 H= 479:00 +CPU= 21.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:30:28 STEP  480 H= 480:00 +CPU= 37.364
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417_19.zip
Uploading the intermediate file: upload_file_19.zip
  04:30:47 STEP  481 H= 481:00 +CPU= 19.136
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:31:59 STEP  482 H= 482:00 +CPU= 18.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:33:33 STEP  483 H= 483:00 +CPU= 32.130
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:35:16 STEP  484 H= 484:00 +CPU= 21.029
  04:35:36 STEP  485 H= 485:00 +CPU= 20.719
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:37:51 STEP  486 H= 486:00 +CPU= 33.575
  04:38:12 STEP  487 H= 487:00 +CPU= 20.831
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:40:56 STEP  488 H= 488:00 +CPU= 21.842
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:43:02 STEP  489 H= 489:00 +CPU= 34.016
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:47:06 STEP  490 H= 490:00 +CPU= 22.177
  04:47:27 STEP  491 H= 491:00 +CPU= 20.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:49:56 STEP  492 H= 492:00 +CPU= 37.220
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:51:38 STEP  493 H= 493:00 +CPU= 20.805
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:54:10 STEP  494 H= 494:00 +CPU= 21.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:56:56 STEP  495 H= 495:00 +CPU= 33.813
  04:57:17 STEP  496 H= 496:00 +CPU= 21.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:59:51 STEP  497 H= 497:00 +CPU= 21.370
  05:00:25 STEP  498 H= 498:00 +CPU= 33.742
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:02:57 STEP  499 H= 499:00 +CPU= 20.772
  05:03:19 STEP  500 H= 500:00 +CPU= 21.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:06:23 STEP  501 H= 501:00 +CPU= 32.736
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:09:07 STEP  502 H= 502:00 +CPU= 21.611
  05:09:28 STEP  503 H= 503:00 +CPU= 21.585
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:12:30 STEP  504 H= 504:00 +CPU= 39.666
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417_20.zip
Uploading the intermediate file: upload_file_20.zip
  05:12:50 STEP  505 H= 505:00 +CPU= 19.954
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:02 STEP  506 H= 506:00 +CPU= 21.188
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:17:47 STEP  507 H= 507:00 +CPU= 33.747
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:20:21 STEP  508 H= 508:00 +CPU= 22.760
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:22:25 STEP  509 H= 509:00 +CPU= 21.915
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2005050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0137
(argv4) batchid: 974
(argv5) wuid: 12190781
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5e0e83c7051f5d56a647034d48ad33ef to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0137_2005050100_123_974_12190781.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0137_2005050100_123_974_12190781.zip
ic_ancil_file: ic_ancil_12190781
ifsdata_file: ifsdata_12190781
climate_data_file: clim_data_12190781
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e0907031a15d50175cb6f939dfc2cd39 to: /var/lib/boinc-client/slots/4/ic_ancil_12190781.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12190781.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6ba621602dbf45f1f230bff246f86a5f to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12190781.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12190781.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12190781.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12190781.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12190781.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12190781.xml
-- Model is restarting --
Adjusting last_iter, 510, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12190781.xml
last_cpu_time: 15472
upload_file_number: 21
last_iter: 503
last_upload: 1814400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417
The child process has been launched with process id: 1671299
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1671299:1671299] [20230113:052431:1673555071:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:26:52 STEP  504 H= 504:00 +CPU= 37.789
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:28:55 STEP  505 H= 505:00 +CPU= 20.731
  05:29:17 STEP  506 H= 506:00 +CPU= 22.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:31:32 STEP  507 H= 507:00 +CPU= 33.270
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:33:15 STEP  508 H= 508:00 +CPU= 21.752
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:34:46 STEP  509 H= 509:00 +CPU= 20.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:36:40 STEP  510 H= 510:00 +CPU= 33.850
  05:37:01 STEP  511 H= 511:00 +CPU= 21.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:39:06 STEP  512 H= 512:00 +CPU= 22.047
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:41 STEP  513 H= 513:00 +CPU= 33.887
  05:42:02 STEP  514 H= 514:00 +CPU= 21.271
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:25 STEP  515 H= 515:00 +CPU= 20.945
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:46:33 STEP  516 H= 516:00 +CPU= 36.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000516
  05:48:16 STEP  517 H= 517:00 +CPU= 20.579
  05:48:37 STEP  518 H= 518:00 +CPU= 21.427
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:51:33 STEP  519 H= 519:00 +CPU= 32.953
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:53:17 STEP  520 H= 520:00 +CPU= 21.045
  05:53:38 STEP  521 H= 521:00 +CPU= 20.935
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:42 STEP  522 H= 522:00 +CPU= 33.639
  05:56:02 STEP  523 H= 523:00 +CPU= 19.911
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:57:54 STEP  524 H= 524:00 +CPU= 20.024
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:39 STEP  525 H= 525:00 +CPU= 33.607
  06:00:01 STEP  526 H= 526:00 +CPU= 21.477
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:01:54 STEP  527 H= 527:00 +CPU= 21.831
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:04:03 STEP  528 H= 528:00 +CPU= 38.059
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190781/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0137_2005050100_123_974_12190781_0_r659660417_21.zip
Uploading the intermediate file: upload_file_21.zip
  06:04:24 STEP  529 H= 529:00 +CPU= 20.723
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:05:46 STEP  530 H= 530:00 +CPU= 21.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:07:41 STEP  531 H= 531:00 +CPU= 33.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:09:34 STEP  532 H= 532:00 +CPU= 21.986
  06:09:55 STEP  533 H= 533:00 +CPU= 20.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:12:00 STEP  534 H= 534:00 +CPU= 33.398
  06:12:21 STEP  535 H= 535:00 +CPU= 20.217
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:14:23 STEP  536 H= 536:00 +CPU= 20.953
  06:14:56 STEP  537 H= 537:00 +CPU= 33.109
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:16:49 STEP  538 H= 538:00 +CPU= 21.112
  06:17:10 STEP  539 H= 539:00 +CPU= 21.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:19:28 STEP  540 H= 540:00 +CPU= 36.691
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000540
  06:19:48 STEP  541 H= 541:00 +CPU= 20.023
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net