climateprediction.net home page
Task 22294133

Task 22294133

Name oifs_43r3_ps_0064_2001050100_123_970_12186708_1
Workunit 12186708
Created 28 Dec 2022, 22:55:12 UTC
Sent 11 Jan 2023, 20:12:22 UTC
Report deadline 10 Feb 2023, 20:12:22 UTC
Received 14 Jan 2023, 4:25:37 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1519251
Run time 7 hours 25 min 32 sec
CPU time 5 hours 2 min 44 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.94 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,414.77 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,690.73 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
44
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:29:08 STEP  445 H= 445:00 +CPU= 20.177
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:31:54 STEP  446 H= 446:00 +CPU= 21.800
  05:32:27 STEP  447 H= 447:00 +CPU= 33.267
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:35:41 STEP  448 H= 448:00 +CPU= 21.426
  05:36:01 STEP  449 H= 449:00 +CPU= 20.489
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:39:07 STEP  450 H= 450:00 +CPU= 33.012
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:50 STEP  451 H= 451:00 +CPU= 20.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:44 STEP  452 H= 452:00 +CPU= 22.050
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:47:48 STEP  453 H= 453:00 +CPU= 32.829
  05:48:10 STEP  454 H= 454:00 +CPU= 21.757
  05:48:32 STEP  455 H= 455:00 +CPU= 21.704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:52:42 STEP  456 H= 456:00 +CPU= 38.446
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_18.zip
Uploading the intermediate file: upload_file_18.zip
  05:53:03 STEP  457 H= 457:00 +CPU= 20.539
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:56:32 STEP  458 H= 458:00 +CPU= 20.301
  05:57:04 STEP  459 H= 459:00 +CPU= 32.061
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:47 STEP  460 H= 460:00 +CPU= 21.170
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:02:10 STEP  461 H= 461:00 +CPU= 20.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:04:03 STEP  462 H= 462:00 +CPU= 32.716
  06:04:23 STEP  463 H= 463:00 +CPU= 19.862
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:06:57 STEP  464 H= 464:00 +CPU= 21.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:09:31 STEP  465 H= 465:00 +CPU= 32.887
  06:09:52 STEP  466 H= 466:00 +CPU= 21.142
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:12:23 STEP  467 H= 467:00 +CPU= 19.949
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:14:41 STEP  468 H= 468:00 +CPU= 36.356
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000468
  06:15:01 STEP  469 H= 469:00 +CPU= 19.938
  06:15:22 STEP  470 H= 470:00 +CPU= 20.669
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:19:27 STEP  471 H= 471:00 +CPU= 32.291
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:21:09 STEP  472 H= 472:00 +CPU= 20.791
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:22:51 STEP  473 H= 473:00 +CPU= 20.912
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:25:07 STEP  474 H= 474:00 +CPU= 33.634
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:26:59 STEP  475 H= 475:00 +CPU= 20.491
  06:27:21 STEP  476 H= 476:00 +CPU= 22.071
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:30:06 STEP  477 H= 477:00 +CPU= 32.689
  06:30:26 STEP  478 H= 478:00 +CPU= 20.210
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:33:09 STEP  479 H= 479:00 +CPU= 21.314
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:36:29 STEP  480 H= 480:00 +CPU= 38.206
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_19.zip
Uploading the intermediate file: upload_file_19.zip
  06:36:49 STEP  481 H= 481:00 +CPU= 20.029
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:38:51 STEP  482 H= 482:00 +CPU= 20.919
  06:39:24 STEP  483 H= 483:00 +CPU= 32.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:42:27 STEP  484 H= 484:00 +CPU= 21.646
  06:42:48 STEP  485 H= 485:00 +CPU= 20.892
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:45:22 STEP  486 H= 486:00 +CPU= 33.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:48:04 STEP  487 H= 487:00 +CPU= 20.589
  06:48:25 STEP  488 H= 488:00 +CPU= 20.850
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:50:50 STEP  489 H= 489:00 +CPU= 33.139
  06:51:11 STEP  490 H= 490:00 +CPU= 20.640
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:53:33 STEP  491 H= 491:00 +CPU= 20.149
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:56:09 STEP  492 H= 492:00 +CPU= 35.691
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000492
  06:56:30 STEP  493 H= 493:00 +CPU= 20.663
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:58:43 STEP  494 H= 494:00 +CPU= 21.227
  06:59:14 STEP  495 H= 495:00 +CPU= 31.820
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2001050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0064
(argv4) batchid: 970
(argv5) wuid: 12186708
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/11
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_64d65e4c5ec2cee4991fff136ab45783 to: /var/lib/boinc-client/slots/11/oifs_43r3_ps_0064_2001050100_123_970_12186708.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/11/oifs_43r3_ps_0064_2001050100_123_970_12186708.zip
ic_ancil_file: ic_ancil_12186708
ifsdata_file: ifsdata_12186708
climate_data_file: clim_data_12186708
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0145cebe58acd4ae715703032ba35e11 to: /var/lib/boinc-client/slots/11/ic_ancil_12186708.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/11/ic_ancil_12186708.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ec77d1c4c08c678a99fd40e2ee57d9da to: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12186708.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12186708.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/11/159l_2/clim_data_12186708.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/11/159l_2/clim_data_12186708.zip
Checking for progress XML file: /var/lib/boinc-client/slots/11/progress_file_12186708.xml
Opened progress file ok : /var/lib/boinc-client/slots/11/progress_file_12186708.xml
-- Model is restarting --
Adjusting last_iter, 496, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/11/progress_file_12186708.xml
last_cpu_time: 13928
upload_file_number: 20
last_iter: 479
last_upload: 1728000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461
The child process has been launched with process id: 1956865
Executing the command: /var/lib/boinc-client/slots/11/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1956865:1956865] [20230114:070522:1673647522:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  07:06:02 STEP  480 H= 480:00 +CPU= 37.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:08:34 STEP  481 H= 481:00 +CPU= 20.817
  07:08:55 STEP  482 H= 482:00 +CPU= 20.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:12:39 STEP  483 H= 483:00 +CPU= 32.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:14:50 STEP  484 H= 484:00 +CPU= 20.325
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:17:02 STEP  485 H= 485:00 +CPU= 20.643
  07:17:36 STEP  486 H= 486:00 +CPU= 33.790
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:20:29 STEP  487 H= 487:00 +CPU= 21.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:23:01 STEP  488 H= 488:00 +CPU= 20.974
  07:23:34 STEP  489 H= 489:00 +CPU= 32.153
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:26:07 STEP  490 H= 490:00 +CPU= 20.900
  07:26:28 STEP  491 H= 491:00 +CPU= 20.928
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:29:25 STEP  492 H= 492:00 +CPU= 36.080
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000492
  07:29:46 STEP  493 H= 493:00 +CPU= 20.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:31:48 STEP  494 H= 494:00 +CPU= 21.264
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:34:02 STEP  495 H= 495:00 +CPU= 32.761
  07:34:24 STEP  496 H= 496:00 +CPU= 21.406
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:37:16 STEP  497 H= 497:00 +CPU= 20.944
  07:37:49 STEP  498 H= 498:00 +CPU= 32.580
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:40:10 STEP  499 H= 499:00 +CPU= 20.191
  07:40:31 STEP  500 H= 500:00 +CPU= 20.834
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:43:25 STEP  501 H= 501:00 +CPU= 32.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:45:38 STEP  502 H= 502:00 +CPU= 21.222
  07:45:59 STEP  503 H= 503:00 +CPU= 20.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:49:10 STEP  504 H= 504:00 +CPU= 38.316
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_20.zip
Uploading the intermediate file: upload_file_20.zip
  07:49:30 STEP  505 H= 505:00 +CPU= 20.019
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:51:52 STEP  506 H= 506:00 +CPU= 20.646
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:54:05 STEP  507 H= 507:00 +CPU= 32.499
  07:54:26 STEP  508 H= 508:00 +CPU= 21.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:57:19 STEP  509 H= 509:00 +CPU= 20.402
  07:57:52 STEP  510 H= 510:00 +CPU= 33.139
  07:58:12 STEP  511 H= 511:00 +CPU= 20.173
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:00:46 STEP  512 H= 512:00 +CPU= 21.314
  08:01:18 STEP  513 H= 513:00 +CPU= 32.233
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:04:01 STEP  514 H= 514:00 +CPU= 21.336
  08:04:21 STEP  515 H= 515:00 +CPU= 20.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:07:09 STEP  516 H= 516:00 +CPU= 35.733
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:09:47 STEP  517 H= 517:00 +CPU= 20.322
  08:10:09 STEP  518 H= 518:00 +CPU= 21.700
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:12:33 STEP  519 H= 519:00 +CPU= 32.649
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:14:46 STEP  520 H= 520:00 +CPU= 20.922
  08:15:06 STEP  521 H= 521:00 +CPU= 20.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:17:23 STEP  522 H= 522:00 +CPU= 33.799
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:19:44 STEP  523 H= 523:00 +CPU= 20.008
  08:20:07 STEP  524 H= 524:00 +CPU= 22.074
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:22:31 STEP  525 H= 525:00 +CPU= 32.764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:24:12 STEP  526 H= 526:00 +CPU= 20.988
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:26:25 STEP  527 H= 527:00 +CPU= 21.125
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:29:37 STEP  528 H= 528:00 +CPU= 37.744
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_21.zip
Uploading the intermediate file: upload_file_21.zip
  08:29:58 STEP  529 H= 529:00 +CPU= 20.929
  08:30:19 STEP  530 H= 530:00 +CPU= 20.971
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:34:04 STEP  531 H= 531:00 +CPU= 32.546
  08:34:24 STEP  532 H= 532:00 +CPU= 20.697
  08:34:44 STEP  533 H= 533:00 +CPU= 19.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:37:10 STEP  534 H= 534:00 +CPU= 33.910
  08:37:30 STEP  535 H= 535:00 +CPU= 20.501
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:40:12 STEP  536 H= 536:00 +CPU= 20.752
  08:40:44 STEP  537 H= 537:00 +CPU= 32.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:43:28 STEP  538 H= 538:00 +CPU= 21.628
  08:43:48 STEP  539 H= 539:00 +CPU= 20.100
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:46:06 STEP  540 H= 540:00 +CPU= 36.159
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000540
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:48:37 STEP  541 H= 541:00 +CPU= 20.784
  08:48:58 STEP  542 H= 542:00 +CPU= 21.443
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:51:34 STEP  543 H= 543:00 +CPU= 32.673
  08:51:55 STEP  544 H= 544:00 +CPU= 20.770
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:54:37 STEP  545 H= 545:00 +CPU= 20.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:12 STEP  546 H= 546:00 +CPU= 33.182
  08:57:33 STEP  547 H= 547:00 +CPU= 20.931
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:59:26 STEP  548 H= 548:00 +CPU= 21.515
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:02:31 STEP  549 H= 549:00 +CPU= 32.114
  09:02:52 STEP  550 H= 550:00 +CPU= 21.399
  09:03:13 STEP  551 H= 551:00 +CPU= 20.302
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2001050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0064
(argv4) batchid: 970
(argv5) wuid: 12186708
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/11
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_64d65e4c5ec2cee4991fff136ab45783 to: /var/lib/boinc-client/slots/11/oifs_43r3_ps_0064_2001050100_123_970_12186708.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/11/oifs_43r3_ps_0064_2001050100_123_970_12186708.zip
ic_ancil_file: ic_ancil_12186708
ifsdata_file: ifsdata_12186708
climate_data_file: clim_data_12186708
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0145cebe58acd4ae715703032ba35e11 to: /var/lib/boinc-client/slots/11/ic_ancil_12186708.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/11/ic_ancil_12186708.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ec77d1c4c08c678a99fd40e2ee57d9da to: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12186708.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12186708.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/11/159l_2/clim_data_12186708.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/11/159l_2/clim_data_12186708.zip
Checking for progress XML file: /var/lib/boinc-client/slots/11/progress_file_12186708.xml
Opened progress file ok : /var/lib/boinc-client/slots/11/progress_file_12186708.xml
-- Model is restarting --
Adjusting last_iter, 552, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/11/progress_file_12186708.xml
last_cpu_time: 15761
upload_file_number: 22
last_iter: 551
last_upload: 1900800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461
The child process has been launched with process id: 1974970
Executing the command: /var/lib/boinc-client/slots/11/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1974970:1974970] [20230114:090855:1673654935:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:11:19 STEP  552 H= 552:00 +CPU= 36.961
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_22.zip
Uploading the intermediate file: upload_file_22.zip
  09:11:40 STEP  553 H= 553:00 +CPU= 20.793
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:14:03 STEP  554 H= 554:00 +CPU= 21.188
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:16:18 STEP  555 H= 555:00 +CPU= 32.866
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:18:12 STEP  556 H= 556:00 +CPU= 21.507
  09:18:33 STEP  557 H= 557:00 +CPU= 20.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:21:07 STEP  558 H= 558:00 +CPU= 33.546
  09:21:28 STEP  559 H= 559:00 +CPU= 20.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:24:00 STEP  560 H= 560:00 +CPU= 21.053
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:26:34 STEP  561 H= 561:00 +CPU= 32.448
  09:26:56 STEP  562 H= 562:00 +CPU= 22.015
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:29:07 STEP  563 H= 563:00 +CPU= 20.330
  09:29:43 STEP  564 H= 564:00 +CPU= 35.939
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:32:05 STEP  565 H= 565:00 +CPU= 20.593
  09:32:26 STEP  566 H= 566:00 +CPU= 21.186
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:35:10 STEP  567 H= 567:00 +CPU= 32.552
  09:35:31 STEP  568 H= 568:00 +CPU= 20.757
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:37:53 STEP  569 H= 569:00 +CPU= 21.109
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:39:58 STEP  570 H= 570:00 +CPU= 33.314
  09:40:18 STEP  571 H= 571:00 +CPU= 20.094
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:42:21 STEP  572 H= 572:00 +CPU= 21.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:44:35 STEP  573 H= 573:00 +CPU= 32.511
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:46:46 STEP  574 H= 574:00 +CPU= 21.097
  09:47:08 STEP  575 H= 575:00 +CPU= 21.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:49:56 STEP  576 H= 576:00 +CPU= 37.518
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_23.zip
Uploading the intermediate file: upload_file_23.zip
  09:50:17 STEP  577 H= 577:00 +CPU= 20.763
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:52:29 STEP  578 H= 578:00 +CPU= 20.390
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:54:44 STEP  579 H= 579:00 +CPU= 33.160
  09:55:05 STEP  580 H= 580:00 +CPU= 20.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:57:16 STEP  581 H= 581:00 +CPU= 20.157
  09:57:50 STEP  582 H= 582:00 +CPU= 33.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:00:12 STEP  583 H= 583:00 +CPU= 21.253
  10:00:33 STEP  584 H= 584:00 +CPU= 20.944
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:05:37 STEP  585 H= 585:00 +CPU= 32.873
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:08:09 STEP  586 H= 586:00 +CPU= 20.569
  10:08:30 STEP  587 H= 587:00 +CPU= 20.850
  10:09:05 STEP  588 H= 588:00 +CPU= 34.874
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000588
  10:09:25 STEP  589 H= 589:00 +CPU= 19.681
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:11:59 STEP  590 H= 590:00 +CPU= 21.752
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:14:32 STEP  591 H= 591:00 +CPU= 32.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:16:45 STEP  592 H= 592:00 +CPU= 21.496
  10:17:05 STEP  593 H= 593:00 +CPU= 20.369
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:20:20 STEP  594 H= 594:00 +CPU= 33.826
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:22:52 STEP  595 H= 595:00 +CPU= 20.510
  10:23:13 STEP  596 H= 596:00 +CPU= 21.363
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:26:18 STEP  597 H= 597:00 +CPU= 32.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:29:20 STEP  598 H= 598:00 +CPU= 20.917
  10:29:42 STEP  599 H= 599:00 +CPU= 21.885
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:32:22 STEP  600 H= 600:00 +CPU= 38.248
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_24.zip
Uploading the intermediate file: upload_file_24.zip
  10:32:42 STEP  601 H= 601:00 +CPU= 20.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:36:24 STEP  602 H= 602:00 +CPU= 21.641
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:39:37 STEP  603 H= 603:00 +CPU= 32.025
  10:39:59 STEP  604 H= 604:00 +CPU= 21.046
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:45:02 STEP  605 H= 605:00 +CPU= 20.797
  10:45:36 STEP  606 H= 606:00 +CPU= 34.222
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:48:18 STEP  607 H= 607:00 +CPU= 20.813
  10:48:40 STEP  608 H= 608:00 +CPU= 21.045
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:52:25 STEP  609 H= 609:00 +CPU= 32.600
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:55:07 STEP  610 H= 610:00 +CPU= 21.627
  10:55:27 STEP  611 H= 611:00 +CPU= 20.466
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:58:55 STEP  612 H= 612:00 +CPU= 36.068
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000612
  11:01:28 STEP  613 H= 613:00 +CPU= 20.932
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:04:10 STEP  614 H= 614:00 +CPU= 21.339
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:07:54 STEP  615 H= 615:00 +CPU= 32.880
  11:08:16 STEP  616 H= 616:00 +CPU= 21.395
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:11:18 STEP  617 H= 617:00 +CPU= 20.058
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2001050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0064
(argv4) batchid: 970
(argv5) wuid: 12186708
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/11
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_64d65e4c5ec2cee4991fff136ab45783 to: /var/lib/boinc-client/slots/11/oifs_43r3_ps_0064_2001050100_123_970_12186708.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/11/oifs_43r3_ps_0064_2001050100_123_970_12186708.zip
ic_ancil_file: ic_ancil_12186708
ifsdata_file: ifsdata_12186708
climate_data_file: clim_data_12186708
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0145cebe58acd4ae715703032ba35e11 to: /var/lib/boinc-client/slots/11/ic_ancil_12186708.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/11/ic_ancil_12186708.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ec77d1c4c08c678a99fd40e2ee57d9da to: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12186708.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12186708.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/11/159l_2/clim_data_12186708.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/11/159l_2/clim_data_12186708.zip
Checking for progress XML file: /var/lib/boinc-client/slots/11/progress_file_12186708.xml
Opened progress file ok : /var/lib/boinc-client/slots/11/progress_file_12186708.xml
-- Model is restarting --
Adjusting last_iter, 618, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/11/progress_file_12186708.xml
last_cpu_time: 17432
upload_file_number: 25
last_iter: 599
last_upload: 2160000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461
The child process has been launched with process id: 1994614
Executing the command: /var/lib/boinc-client/slots/11/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1994614:1994614] [20230114:111427:1673662467:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:17:31 STEP  600 H= 600:00 +CPU= 37.766
  11:17:52 STEP  601 H= 601:00 +CPU= 20.492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:21:06 STEP  602 H= 602:00 +CPU= 22.403
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:27:01 STEP  603 H= 603:00 +CPU= 32.543
  11:27:22 STEP  604 H= 604:00 +CPU= 21.001
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:30:14 STEP  605 H= 605:00 +CPU= 20.535
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:33:59 STEP  606 H= 606:00 +CPU= 33.416
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:36:30 STEP  607 H= 607:00 +CPU= 20.382
  11:36:52 STEP  608 H= 608:00 +CPU= 21.565
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:40:07 STEP  609 H= 609:00 +CPU= 32.564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:51 STEP  610 H= 610:00 +CPU= 22.360
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:45:54 STEP  611 H= 611:00 +CPU= 20.934
  11:46:30 STEP  612 H= 612:00 +CPU= 36.103
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000612
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:49:32 STEP  613 H= 613:00 +CPU= 20.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:52:34 STEP  614 H= 614:00 +CPU= 21.268
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:55:19 STEP  615 H= 615:00 +CPU= 33.300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:58:12 STEP  616 H= 616:00 +CPU= 21.848
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:24 STEP  617 H= 617:00 +CPU= 20.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:20 STEP  618 H= 618:00 +CPU= 34.113
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:10:14 STEP  619 H= 619:00 +CPU= 20.951
  12:10:35 STEP  620 H= 620:00 +CPU= 21.441
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:14:30 STEP  621 H= 621:00 +CPU= 33.483
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:17:03 STEP  622 H= 622:00 +CPU= 21.520
  12:17:25 STEP  623 H= 623:00 +CPU= 22.144
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:20:15 STEP  624 H= 624:00 +CPU= 38.321
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12186708/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0064_2001050100_123_970_12186708_1_r1942573461_25.zip
Uploading the intermediate file: upload_file_25.zip
  12:20:34 STEP  625 H= 625:00 +CPU= 19.035
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:24:39 STEP  626 H= 626:00 +CPU= 20.439
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:28:32 STEP  627 H= 627:00 +CPU= 32.238
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net