climateprediction.net home page
Task 22279297

Task 22279297

Name oifs_43r3_ps_0860_2007050100_123_976_12193504_0
Workunit 12193504
Created 23 Dec 2022, 16:54:32 UTC
Sent 11 Jan 2023, 18:38:14 UTC
Report deadline 10 Feb 2023, 18:38:14 UTC
Received 12 Jan 2023, 23:10:54 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1519251
Run time 4 hours 39 min 59 sec
CPU time 4 hours 32 min 32 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.94 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,416.02 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,816.90 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ild process
  00:51:14 STEP  362 H= 362:00 +CPU= 21.130
  00:51:46 STEP  363 H= 363:00 +CPU= 31.971
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:53:29 STEP  364 H= 364:00 +CPU= 21.820
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:42 STEP  365 H= 365:00 +CPU= 20.066
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:27 STEP  366 H= 366:00 +CPU= 33.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:59:19 STEP  367 H= 367:00 +CPU= 20.364
  00:59:40 STEP  368 H= 368:00 +CPU= 20.929
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:14 STEP  369 H= 369:00 +CPU= 31.898
  01:03:34 STEP  370 H= 370:00 +CPU= 20.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:05:16 STEP  371 H= 371:00 +CPU= 19.797
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:12 STEP  372 H= 372:00 +CPU= 35.863
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:09:04 STEP  373 H= 373:00 +CPU= 20.688
  01:09:26 STEP  374 H= 374:00 +CPU= 21.777
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:11:20 STEP  375 H= 375:00 +CPU= 33.000
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:13:12 STEP  376 H= 376:00 +CPU= 21.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0860
(argv4) batchid: 976
(argv5) wuid: 12193504
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_433a1d4781e6ffdd650f6cc1999278a7 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0860_2007050100_123_976_12193504.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0860_2007050100_123_976_12193504.zip
ic_ancil_file: ic_ancil_12193504
ifsdata_file: ifsdata_12193504
climate_data_file: clim_data_12193504
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_53213087289a46b97fc08cb940c8a5cc to: /var/lib/boinc-client/slots/8/ic_ancil_12193504.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12193504.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0a76f42707729ce1be0c534688d6b886 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12193504.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12193504.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12193504.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12193504.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12193504.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12193504.xml
-- Model is restarting --
Adjusting last_iter, 377, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12193504.xml
last_cpu_time: 11316
upload_file_number: 15
last_iter: 359
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013
The child process has been launched with process id: 1632407
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1632407:1632407] [20230113:011554:1673540154:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:17:34 STEP  360 H= 360:00 +CPU= 36.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:19:35 STEP  361 H= 361:00 +CPU= 20.383
  01:19:57 STEP  362 H= 362:00 +CPU= 22.011
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:23:42 STEP  363 H= 363:00 +CPU= 32.762
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:25:06 STEP  364 H= 364:00 +CPU= 21.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:39 STEP  365 H= 365:00 +CPU= 20.930
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:28:43 STEP  366 H= 366:00 +CPU= 33.554
  01:29:05 STEP  367 H= 367:00 +CPU= 21.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:57 STEP  368 H= 368:00 +CPU= 21.234
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:33:02 STEP  369 H= 369:00 +CPU= 32.646
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:55 STEP  370 H= 370:00 +CPU= 21.828
  01:35:16 STEP  371 H= 371:00 +CPU= 21.166
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:37:13 STEP  372 H= 372:00 +CPU= 36.165
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:45 STEP  373 H= 373:00 +CPU= 20.999
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:16 STEP  374 H= 374:00 +CPU= 20.790
  01:40:48 STEP  375 H= 375:00 +CPU= 31.930
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:42:40 STEP  376 H= 376:00 +CPU= 21.088
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:44:22 STEP  377 H= 377:00 +CPU= 20.571
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:46:18 STEP  378 H= 378:00 +CPU= 33.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:47:59 STEP  379 H= 379:00 +CPU= 20.279
  01:48:20 STEP  380 H= 380:00 +CPU= 20.954
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:50:14 STEP  381 H= 381:00 +CPU= 32.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:52:06 STEP  382 H= 382:00 +CPU= 21.381
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:53:58 STEP  383 H= 383:00 +CPU= 20.666
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:55:58 STEP  384 H= 384:00 +CPU= 38.555
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013_15.zip
Uploading the intermediate file: upload_file_15.zip
  01:56:16 STEP  385 H= 385:00 +CPU= 17.803
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:57:42 STEP  386 H= 386:00 +CPU= 20.455
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:59:46 STEP  387 H= 387:00 +CPU= 31.907
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:01:59 STEP  388 H= 388:00 +CPU= 21.341
  02:02:19 STEP  389 H= 389:00 +CPU= 20.412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:04:24 STEP  390 H= 390:00 +CPU= 33.204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:05:46 STEP  391 H= 391:00 +CPU= 21.227
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:07:49 STEP  392 H= 392:00 +CPU= 21.261
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:09:32 STEP  393 H= 393:00 +CPU= 32.293
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:10:54 STEP  394 H= 394:00 +CPU= 21.128
  02:11:16 STEP  395 H= 395:00 +CPU= 21.377
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:13:23 STEP  396 H= 396:00 +CPU= 36.750
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:15:15 STEP  397 H= 397:00 +CPU= 20.283
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:17:17 STEP  398 H= 398:00 +CPU= 21.007
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:19:20 STEP  399 H= 399:00 +CPU= 31.583
  02:19:42 STEP  400 H= 400:00 +CPU= 21.709
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:21:34 STEP  401 H= 401:00 +CPU= 20.918
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:23:09 STEP  402 H= 402:00 +CPU= 33.650
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:25:02 STEP  403 H= 403:00 +CPU= 20.752
  02:25:23 STEP  404 H= 404:00 +CPU= 20.775
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:27:18 STEP  405 H= 405:00 +CPU= 32.252
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:29:10 STEP  406 H= 406:00 +CPU= 21.100
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:31:02 STEP  407 H= 407:00 +CPU= 21.233
  02:31:40 STEP  408 H= 408:00 +CPU= 37.648
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013_16.zip
Uploading the intermediate file: upload_file_16.zip
  02:31:58 STEP  409 H= 409:00 +CPU= 18.273
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:33:56 STEP  410 H= 410:00 +CPU= 18.459
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:36:10 STEP  411 H= 411:00 +CPU= 32.787
  02:36:31 STEP  412 H= 412:00 +CPU= 21.754
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:38:13 STEP  413 H= 413:00 +CPU= 20.886
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:40:07 STEP  414 H= 414:00 +CPU= 32.241
  02:40:26 STEP  415 H= 415:00 +CPU= 19.518
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:42:38 STEP  416 H= 416:00 +CPU= 20.759
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:44:43 STEP  417 H= 417:00 +CPU= 32.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:46:28 STEP  418 H= 418:00 +CPU= 22.337
  02:46:48 STEP  419 H= 419:00 +CPU= 20.527
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:48:45 STEP  420 H= 420:00 +CPU= 35.360
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000420
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:50:26 STEP  421 H= 421:00 +CPU= 19.991
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:52:08 STEP  422 H= 422:00 +CPU= 20.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:54:21 STEP  423 H= 423:00 +CPU= 31.913
  02:54:42 STEP  424 H= 424:00 +CPU= 21.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:56:33 STEP  425 H= 425:00 +CPU= 20.120
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:58:28 STEP  426 H= 426:00 +CPU= 32.480
  02:58:48 STEP  427 H= 427:00 +CPU= 20.291
  02:59:09 STEP  428 H= 428:00 +CPU= 20.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:01:22 STEP  429 H= 429:00 +CPU= 32.643
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:03:34 STEP  430 H= 430:00 +CPU= 20.409
  03:03:56 STEP  431 H= 431:00 +CPU= 21.712
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:05:54 STEP  432 H= 432:00 +CPU= 37.216
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013_17.zip
Uploading the intermediate file: upload_file_17.zip
  03:06:13 STEP  433 H= 433:00 +CPU= 18.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:08:16 STEP  434 H= 434:00 +CPU= 19.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:10:19 STEP  435 H= 435:00 +CPU= 32.151
  03:10:40 STEP  436 H= 436:00 +CPU= 20.504
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:13:02 STEP  437 H= 437:00 +CPU= 20.819
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:14:56 STEP  438 H= 438:00 +CPU= 33.068
  03:15:16 STEP  439 H= 439:00 +CPU= 19.892
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:17:08 STEP  440 H= 440:00 +CPU= 21.260
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0860
(argv4) batchid: 976
(argv5) wuid: 12193504
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_433a1d4781e6ffdd650f6cc1999278a7 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0860_2007050100_123_976_12193504.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0860_2007050100_123_976_12193504.zip
ic_ancil_file: ic_ancil_12193504
ifsdata_file: ifsdata_12193504
climate_data_file: clim_data_12193504
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_53213087289a46b97fc08cb940c8a5cc to: /var/lib/boinc-client/slots/8/ic_ancil_12193504.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12193504.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0a76f42707729ce1be0c534688d6b886 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12193504.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12193504.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12193504.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12193504.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12193504.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12193504.xml
-- Model is restarting --
Adjusting last_iter, 440, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12193504.xml
last_cpu_time: 13344
upload_file_number: 18
last_iter: 431
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013
The child process has been launched with process id: 1650103
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1650103:1650103] [20230113:031927:1673547567:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:21:18 STEP  432 H= 432:00 +CPU= 37.291
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:23:01 STEP  433 H= 433:00 +CPU= 20.307
  03:23:22 STEP  434 H= 434:00 +CPU= 20.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:24:45 STEP  435 H= 435:00 +CPU= 32.332
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:26:38 STEP  436 H= 436:00 +CPU= 22.023
  03:26:59 STEP  437 H= 437:00 +CPU= 20.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:28:52 STEP  438 H= 438:00 +CPU= 33.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:30:44 STEP  439 H= 439:00 +CPU= 21.046
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:32:17 STEP  440 H= 440:00 +CPU= 21.324
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:33:49 STEP  441 H= 441:00 +CPU= 32.434
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:35:21 STEP  442 H= 442:00 +CPU= 20.871
  03:35:42 STEP  443 H= 443:00 +CPU= 20.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:37:29 STEP  444 H= 444:00 +CPU= 35.725
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:39:20 STEP  445 H= 445:00 +CPU= 19.886
  03:39:41 STEP  446 H= 446:00 +CPU= 21.626
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:41:34 STEP  447 H= 447:00 +CPU= 32.236
  03:41:55 STEP  448 H= 448:00 +CPU= 20.759
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:43:46 STEP  449 H= 449:00 +CPU= 20.230
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:45:50 STEP  450 H= 450:00 +CPU= 32.951
  03:46:11 STEP  451 H= 451:00 +CPU= 21.027
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:48:04 STEP  452 H= 452:00 +CPU= 21.572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:49:58 STEP  453 H= 453:00 +CPU= 32.720
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:51:30 STEP  454 H= 454:00 +CPU= 20.755
  03:51:50 STEP  455 H= 455:00 +CPU= 20.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:54:30 STEP  456 H= 456:00 +CPU= 38.106
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013_18.zip
Uploading the intermediate file: upload_file_18.zip
  03:54:50 STEP  457 H= 457:00 +CPU= 20.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:56:41 STEP  458 H= 458:00 +CPU= 19.811
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:58:35 STEP  459 H= 459:00 +CPU= 32.699
  03:58:57 STEP  460 H= 460:00 +CPU= 21.863
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:00:38 STEP  461 H= 461:00 +CPU= 20.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:02:33 STEP  462 H= 462:00 +CPU= 33.839
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:04:14 STEP  463 H= 463:00 +CPU= 20.899
  04:04:35 STEP  464 H= 464:00 +CPU= 21.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:06:50 STEP  465 H= 465:00 +CPU= 33.198
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:08:43 STEP  466 H= 466:00 +CPU= 20.842
  04:09:04 STEP  467 H= 467:00 +CPU= 21.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:11:22 STEP  468 H= 468:00 +CPU= 36.453
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000468
  04:11:42 STEP  469 H= 469:00 +CPU= 19.967
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:13:22 STEP  470 H= 470:00 +CPU= 20.788
  04:13:55 STEP  471 H= 471:00 +CPU= 32.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:15:47 STEP  472 H= 472:00 +CPU= 20.995
  04:16:08 STEP  473 H= 473:00 +CPU= 20.354
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:18:13 STEP  474 H= 474:00 +CPU= 33.718
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:20:14 STEP  475 H= 475:00 +CPU= 20.513
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:21:46 STEP  476 H= 476:00 +CPU= 21.077
  04:22:20 STEP  477 H= 477:00 +CPU= 33.025
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:24:11 STEP  478 H= 478:00 +CPU= 20.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:25:44 STEP  479 H= 479:00 +CPU= 20.597
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:27:33 STEP  480 H= 480:00 +CPU= 37.787
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013_19.zip
Uploading the intermediate file: upload_file_19.zip
  04:27:51 STEP  481 H= 481:00 +CPU= 18.062
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:29:08 STEP  482 H= 482:00 +CPU= 20.000
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:31:54 STEP  483 H= 483:00 +CPU= 31.892
  04:32:15 STEP  484 H= 484:00 +CPU= 20.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:33:36 STEP  485 H= 485:00 +CPU= 19.930
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:35:31 STEP  486 H= 486:00 +CPU= 33.344
  04:35:52 STEP  487 H= 487:00 +CPU= 21.154
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:37:54 STEP  488 H= 488:00 +CPU= 21.140
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:40:48 STEP  489 H= 489:00 +CPU= 32.792
  04:41:12 STEP  490 H= 490:00 +CPU= 23.047
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:43:03 STEP  491 H= 491:00 +CPU= 20.403
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:47:23 STEP  492 H= 492:00 +CPU= 37.196
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000492
  04:47:45 STEP  493 H= 493:00 +CPU= 21.800
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:49:59 STEP  494 H= 494:00 +CPU= 21.925
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:54:07 STEP  495 H= 495:00 +CPU= 33.298
  04:54:30 STEP  496 H= 496:00 +CPU= 22.373
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:57:03 STEP  497 H= 497:00 +CPU= 21.552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:59:48 STEP  498 H= 498:00 +CPU= 33.962
  05:00:10 STEP  499 H= 499:00 +CPU= 21.459
  05:00:31 STEP  500 H= 500:00 +CPU= 21.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:03:16 STEP  501 H= 501:00 +CPU= 32.903
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:06:10 STEP  502 H= 502:00 +CPU= 21.363
  05:06:31 STEP  503 H= 503:00 +CPU= 21.181
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:09:32 STEP  504 H= 504:00 +CPU= 39.875
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013_20.zip
Uploading the intermediate file: upload_file_20.zip
  05:09:54 STEP  505 H= 505:00 +CPU= 20.944
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:12:21 STEP  506 H= 506:00 +CPU= 20.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:06 STEP  507 H= 507:00 +CPU= 33.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:17:39 STEP  508 H= 508:00 +CPU= 22.087
  05:18:02 STEP  509 H= 509:00 +CPU= 22.354
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0860
(argv4) batchid: 976
(argv5) wuid: 12193504
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_433a1d4781e6ffdd650f6cc1999278a7 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0860_2007050100_123_976_12193504.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0860_2007050100_123_976_12193504.zip
ic_ancil_file: ic_ancil_12193504
ifsdata_file: ifsdata_12193504
climate_data_file: clim_data_12193504
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_53213087289a46b97fc08cb940c8a5cc to: /var/lib/boinc-client/slots/8/ic_ancil_12193504.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12193504.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0a76f42707729ce1be0c534688d6b886 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12193504.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12193504.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12193504.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12193504.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12193504.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12193504.xml
-- Model is restarting --
Adjusting last_iter, 510, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12193504.xml
last_cpu_time: 15339
upload_file_number: 21
last_iter: 503
last_upload: 1814400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013
The child process has been launched with process id: 1670974
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1670974:1670974] [20230113:052219:1673554939:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:24:42 STEP  504 H= 504:00 +CPU= 38.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:26:44 STEP  505 H= 505:00 +CPU= 20.726
  05:27:06 STEP  506 H= 506:00 +CPU= 21.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:29:22 STEP  507 H= 507:00 +CPU= 33.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:31:26 STEP  508 H= 508:00 +CPU= 21.639
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:33:09 STEP  509 H= 509:00 +CPU= 21.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:34:54 STEP  510 H= 510:00 +CPU= 33.866
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:36:34 STEP  511 H= 511:00 +CPU= 20.299
  05:36:56 STEP  512 H= 512:00 +CPU= 21.449
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:39:13 STEP  513 H= 513:00 +CPU= 34.259
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:35 STEP  514 H= 514:00 +CPU= 21.457
  05:41:56 STEP  515 H= 515:00 +CPU= 20.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:35 STEP  516 H= 516:00 +CPU= 37.446
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000516
  05:46:26 STEP  517 H= 517:00 +CPU= 20.393
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:48:09 STEP  518 H= 518:00 +CPU= 20.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:03 STEP  519 H= 519:00 +CPU= 32.935
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:51:27 STEP  520 H= 520:00 +CPU= 21.437
  05:51:48 STEP  521 H= 521:00 +CPU= 20.860
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:53:44 STEP  522 H= 522:00 +CPU= 34.207
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:36 STEP  523 H= 523:00 +CPU= 20.483
  05:55:57 STEP  524 H= 524:00 +CPU= 20.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:00 STEP  525 H= 525:00 +CPU= 32.183
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:34 STEP  526 H= 526:00 +CPU= 21.513
  05:59:56 STEP  527 H= 527:00 +CPU= 21.970
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:02:06 STEP  528 H= 528:00 +CPU= 38.945
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193504/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0860_2007050100_123_976_12193504_0_r1526689013_21.zip
Uploading the intermediate file: upload_file_21.zip
  06:02:27 STEP  529 H= 529:00 +CPU= 20.579
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:04:19 STEP  530 H= 530:00 +CPU= 21.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:05:53 STEP  531 H= 531:00 +CPU= 33.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:07:35 STEP  532 H= 532:00 +CPU= 20.898
  06:07:56 STEP  533 H= 533:00 +CPU= 20.596
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:10:01 STEP  534 H= 534:00 +CPU= 33.763
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:11:52 STEP  535 H= 535:00 +CPU= 20.034
  06:12:13 STEP  536 H= 536:00 +CPU= 20.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:14:26 STEP  537 H= 537:00 +CPU= 32.326
  06:14:47 STEP  538 H= 538:00 +CPU= 20.794
  06:15:07 STEP  539 H= 539:00 +CPU= 19.867
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:17:15 STEP  540 H= 540:00 +CPU= 35.963
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000540
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:19:18 STEP  541 H= 541:00 +CPU= 20.669
  06:19:39 STEP  542 H= 542:00 +CPU= 21.059
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net