<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ild process
Resuming the child process
22:05:06 STEP 564 H= 564:00 +CPU= 40.610
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000564
22:05:30 STEP 565 H= 565:00 +CPU= 23.396
22:05:54 STEP 566 H= 566:00 +CPU= 23.356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:10:53 STEP 567 H= 567:00 +CPU= 36.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:14:28 STEP 568 H= 568:00 +CPU= 23.767
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0239
(argv4) batchid: 958
(argv5) wuid: 12174883
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_32bb48023a4543316632a4506cbe0d85 to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0239_1989050100_123_958_12174883.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0239_1989050100_123_958_12174883.zip
ic_ancil_file: ic_ancil_12174883
ifsdata_file: ifsdata_12174883
climate_data_file: clim_data_12174883
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_2026e556d9d01653854501cb6456e937 to: /var/lib/boinc-client/slots/13/ic_ancil_12174883.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12174883.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e4407feb319799f71b1e6cf014e455fc to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174883.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174883.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174883.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174883.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12174883.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12174883.xml
-- Model is restarting --
Adjusting last_iter, 569, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12174883.xml
last_cpu_time: 16268
upload_file_number: 23
last_iter: 551
last_upload: 1987200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818
The child process has been launched with process id: 333887
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333887:333887] [20230120:222330:1674221010:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
22:24:15 STEP 552 H= 552:00 +CPU= 40.746
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:28:50 STEP 553 H= 553:00 +CPU= 22.747
22:29:14 STEP 554 H= 554:00 +CPU= 23.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:04 STEP 555 H= 555:00 +CPU= 37.160
22:35:28 STEP 556 H= 556:00 +CPU= 23.606
22:35:51 STEP 557 H= 557:00 +CPU= 22.590
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:04 STEP 558 H= 558:00 +CPU= 37.514
22:51:24 STEP 559 H= 559:00 +CPU= 20.299
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:08 STEP 560 H= 560:00 +CPU= 21.860
22:53:39 STEP 561 H= 561:00 +CPU= 31.770
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:55:21 STEP 562 H= 562:00 +CPU= 21.301
22:55:40 STEP 563 H= 563:00 +CPU= 19.529
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:57:48 STEP 564 H= 564:00 +CPU= 35.177
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:59:19 STEP 565 H= 565:00 +CPU= 19.887
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:00:42 STEP 566 H= 566:00 +CPU= 21.136
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:02:06 STEP 567 H= 567:00 +CPU= 31.621
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:03:27 STEP 568 H= 568:00 +CPU= 21.043
23:03:46 STEP 569 H= 569:00 +CPU= 19.576
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:05:10 STEP 570 H= 570:00 +CPU= 31.959
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:06:41 STEP 571 H= 571:00 +CPU= 19.921
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:08:02 STEP 572 H= 572:00 +CPU= 20.836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:09:45 STEP 573 H= 573:00 +CPU= 31.432
23:10:06 STEP 574 H= 574:00 +CPU= 20.798
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:11:27 STEP 575 H= 575:00 +CPU= 20.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:13:06 STEP 576 H= 576:00 +CPU= 37.529
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_23.zip
Uploading the intermediate file: upload_file_23.zip
23:13:25 STEP 577 H= 577:00 +CPU= 19.196
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:14:43 STEP 578 H= 578:00 +CPU= 19.663
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:16:47 STEP 579 H= 579:00 +CPU= 31.976
23:17:08 STEP 580 H= 580:00 +CPU= 20.288
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:18:37 STEP 581 H= 581:00 +CPU= 19.328
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:20:21 STEP 582 H= 582:00 +CPU= 32.906
23:20:41 STEP 583 H= 583:00 +CPU= 20.009
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:22:03 STEP 584 H= 584:00 +CPU= 20.931
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:23:37 STEP 585 H= 585:00 +CPU= 32.722
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:25:19 STEP 586 H= 586:00 +CPU= 20.839
23:25:39 STEP 587 H= 587:00 +CPU= 20.457
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:27:36 STEP 588 H= 588:00 +CPU= 35.006
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000588
23:27:56 STEP 589 H= 589:00 +CPU= 19.863
23:28:16 STEP 590 H= 590:00 +CPU= 20.498
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:30:18 STEP 591 H= 591:00 +CPU= 31.852
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:31:38 STEP 592 H= 592:00 +CPU= 20.282
23:31:59 STEP 593 H= 593:00 +CPU= 20.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:33:43 STEP 594 H= 594:00 +CPU= 33.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:35:13 STEP 595 H= 595:00 +CPU= 19.790
23:35:34 STEP 596 H= 596:00 +CPU= 20.902
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:37:29 STEP 597 H= 597:00 +CPU= 32.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:38:40 STEP 598 H= 598:00 +CPU= 20.416
23:39:00 STEP 599 H= 599:00 +CPU= 20.246
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:41:19 STEP 600 H= 600:00 +CPU= 37.695
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_24.zip
Uploading the intermediate file: upload_file_24.zip
23:41:39 STEP 601 H= 601:00 +CPU= 19.227
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:43:07 STEP 602 H= 602:00 +CPU= 19.921
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:44:50 STEP 603 H= 603:00 +CPU= 31.876
23:45:10 STEP 604 H= 604:00 +CPU= 20.518
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:46:50 STEP 605 H= 605:00 +CPU= 19.982
23:47:24 STEP 606 H= 606:00 +CPU= 33.368
23:47:44 STEP 607 H= 607:00 +CPU= 19.893
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:27 STEP 608 H= 608:00 +CPU= 21.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:51:10 STEP 609 H= 609:00 +CPU= 31.660
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:33 STEP 610 H= 610:00 +CPU= 20.818
23:52:53 STEP 611 H= 611:00 +CPU= 19.883
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:48 STEP 612 H= 612:00 +CPU= 35.472
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000612
23:55:08 STEP 613 H= 613:00 +CPU= 20.053
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:56:40 STEP 614 H= 614:00 +CPU= 20.455
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:58:22 STEP 615 H= 615:00 +CPU= 31.902
23:58:43 STEP 616 H= 616:00 +CPU= 20.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:00:04 STEP 617 H= 617:00 +CPU= 19.762
00:00:36 STEP 618 H= 618:00 +CPU= 32.499
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:02:08 STEP 619 H= 619:00 +CPU= 20.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:03:20 STEP 620 H= 620:00 +CPU= 20.817
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:05:03 STEP 621 H= 621:00 +CPU= 32.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:06:34 STEP 622 H= 622:00 +CPU= 20.990
00:06:55 STEP 623 H= 623:00 +CPU= 20.595
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:09:14 STEP 624 H= 624:00 +CPU= 37.650
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_25.zip
Uploading the intermediate file: upload_file_25.zip
00:09:34 STEP 625 H= 625:00 +CPU= 19.498
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:11:26 STEP 626 H= 626:00 +CPU= 20.914
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:13:31 STEP 627 H= 627:00 +CPU= 32.833
00:13:52 STEP 628 H= 628:00 +CPU= 20.704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:15:22 STEP 629 H= 629:00 +CPU= 20.084
00:15:55 STEP 630 H= 630:00 +CPU= 32.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:17:36 STEP 631 H= 631:00 +CPU= 20.424
00:17:58 STEP 632 H= 632:00 +CPU= 21.356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:19:50 STEP 633 H= 633:00 +CPU= 31.518
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:21:12 STEP 634 H= 634:00 +CPU= 20.254
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:22:13 STEP 635 H= 635:00 +CPU= 20.636
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:23:40 STEP 636 H= 636:00 +CPU= 35.396
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000636
00:24:00 STEP 637 H= 637:00 +CPU= 19.989
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0239
(argv4) batchid: 958
(argv5) wuid: 12174883
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_32bb48023a4543316632a4506cbe0d85 to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0239_1989050100_123_958_12174883.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0239_1989050100_123_958_12174883.zip
ic_ancil_file: ic_ancil_12174883
ifsdata_file: ifsdata_12174883
climate_data_file: clim_data_12174883
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_2026e556d9d01653854501cb6456e937 to: /var/lib/boinc-client/slots/13/ic_ancil_12174883.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12174883.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e4407feb319799f71b1e6cf014e455fc to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174883.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174883.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174883.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174883.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12174883.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12174883.xml
-- Model is restarting --
Adjusting last_iter, 638, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12174883.xml
last_cpu_time: 18432
upload_file_number: 26
last_iter: 623
last_upload: 2246400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818
The child process has been launched with process id: 351940
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351940:351940] [20230121:002743:1674228463:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
00:28:22 STEP 624 H= 624:00 +CPU= 36.296
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:30:24 STEP 625 H= 625:00 +CPU= 20.501
00:30:45 STEP 626 H= 626:00 +CPU= 20.532
00:31:17 STEP 627 H= 627:00 +CPU= 31.638
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:33:08 STEP 628 H= 628:00 +CPU= 21.066
00:33:28 STEP 629 H= 629:00 +CPU= 19.664
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:35:11 STEP 630 H= 630:00 +CPU= 32.431
00:35:31 STEP 631 H= 631:00 +CPU= 20.282
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:37:04 STEP 632 H= 632:00 +CPU= 20.444
00:37:36 STEP 633 H= 633:00 +CPU= 32.297
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:39:27 STEP 634 H= 634:00 +CPU= 20.568
00:39:47 STEP 635 H= 635:00 +CPU= 19.916
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:41:33 STEP 636 H= 636:00 +CPU= 34.527
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000636
00:41:54 STEP 637 H= 637:00 +CPU= 21.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:43:26 STEP 638 H= 638:00 +CPU= 20.801
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:45:19 STEP 639 H= 639:00 +CPU= 32.071
00:45:39 STEP 640 H= 640:00 +CPU= 20.257
00:46:00 STEP 641 H= 641:00 +CPU= 20.357
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:47:44 STEP 642 H= 642:00 +CPU= 33.060
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:49:06 STEP 643 H= 643:00 +CPU= 20.612
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:50:36 STEP 644 H= 644:00 +CPU= 20.694
00:51:08 STEP 645 H= 645:00 +CPU= 31.866
00:51:29 STEP 646 H= 646:00 +CPU= 20.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:53:12 STEP 647 H= 647:00 +CPU= 21.023
00:53:50 STEP 648 H= 648:00 +CPU= 37.585
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_26.zip
Uploading the intermediate file: upload_file_26.zip
00:54:09 STEP 649 H= 649:00 +CPU= 19.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:55:54 STEP 650 H= 650:00 +CPU= 18.752
00:56:26 STEP 651 H= 651:00 +CPU= 31.610
00:56:46 STEP 652 H= 652:00 +CPU= 20.087
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:58:38 STEP 653 H= 653:00 +CPU= 20.471
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:00:01 STEP 654 H= 654:00 +CPU= 33.017
01:00:20 STEP 655 H= 655:00 +CPU= 18.906
01:00:40 STEP 656 H= 656:00 +CPU= 20.628
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:02:35 STEP 657 H= 657:00 +CPU= 32.002
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:03:56 STEP 658 H= 658:00 +CPU= 20.193
01:04:17 STEP 659 H= 659:00 +CPU= 20.638
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:06:04 STEP 660 H= 660:00 +CPU= 35.189
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000660
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:07:25 STEP 661 H= 661:00 +CPU= 19.751
01:07:45 STEP 662 H= 662:00 +CPU= 20.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:09:08 STEP 663 H= 663:00 +CPU= 31.736
01:09:29 STEP 664 H= 664:00 +CPU= 20.473
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:11:09 STEP 665 H= 665:00 +CPU= 19.614
01:11:41 STEP 666 H= 666:00 +CPU= 32.113
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:13:13 STEP 667 H= 667:00 +CPU= 20.218
01:13:34 STEP 668 H= 668:00 +CPU= 20.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:15:27 STEP 669 H= 669:00 +CPU= 31.911
01:15:47 STEP 670 H= 670:00 +CPU= 20.555
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:17:19 STEP 671 H= 671:00 +CPU= 20.112
01:17:55 STEP 672 H= 672:00 +CPU= 36.773
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_27.zip
Uploading the intermediate file: upload_file_27.zip
01:18:14 STEP 673 H= 673:00 +CPU= 18.076
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:19:30 STEP 674 H= 674:00 +CPU= 18.926
01:20:01 STEP 675 H= 675:00 +CPU= 31.372
01:20:21 STEP 676 H= 676:00 +CPU= 20.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:22:03 STEP 677 H= 677:00 +CPU= 19.665
01:22:35 STEP 678 H= 678:00 +CPU= 32.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:24:07 STEP 679 H= 679:00 +CPU= 19.632
01:24:27 STEP 680 H= 680:00 +CPU= 20.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:26:10 STEP 681 H= 681:00 +CPU= 31.841
01:26:31 STEP 682 H= 682:00 +CPU= 20.240
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:28:21 STEP 683 H= 683:00 +CPU= 19.503
01:28:56 STEP 684 H= 684:00 +CPU= 35.025
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:30:17 STEP 685 H= 685:00 +CPU= 19.595
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:31:38 STEP 686 H= 686:00 +CPU= 20.214
01:32:10 STEP 687 H= 687:00 +CPU= 32.017
01:32:31 STEP 688 H= 688:00 +CPU= 20.674
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:34:02 STEP 689 H= 689:00 +CPU= 19.612
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:35:44 STEP 690 H= 690:00 +CPU= 32.027
01:36:04 STEP 691 H= 691:00 +CPU= 20.103
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:37:36 STEP 692 H= 692:00 +CPU= 20.128
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:39:10 STEP 693 H= 693:00 +CPU= 31.789
01:39:31 STEP 694 H= 694:00 +CPU= 20.483
01:39:51 STEP 695 H= 695:00 +CPU= 20.408
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:41:48 STEP 696 H= 696:00 +CPU= 36.684
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_28.zip
Uploading the intermediate file: upload_file_28.zip
01:42:08 STEP 697 H= 697:00 +CPU= 19.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:43:39 STEP 698 H= 698:00 +CPU= 20.206
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:45:22 STEP 699 H= 699:00 +CPU= 31.673
01:45:42 STEP 700 H= 700:00 +CPU= 20.048
01:46:03 STEP 701 H= 701:00 +CPU= 20.670
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:47:46 STEP 702 H= 702:00 +CPU= 32.433
01:48:06 STEP 703 H= 703:00 +CPU= 19.856
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:49:28 STEP 704 H= 704:00 +CPU= 20.649
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:51:10 STEP 705 H= 705:00 +CPU= 31.619
01:51:30 STEP 706 H= 706:00 +CPU= 20.336
01:51:50 STEP 707 H= 707:00 +CPU= 19.830
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:53:55 STEP 708 H= 708:00 +CPU= 34.838
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000708
01:54:15 STEP 709 H= 709:00 +CPU= 19.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:56:07 STEP 710 H= 710:00 +CPU= 21.253
01:56:39 STEP 711 H= 711:00 +CPU= 31.716
01:56:59 STEP 712 H= 712:00 +CPU= 19.668
01:57:18 STEP 713 H= 713:00 +CPU= 19.581
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:59:12 STEP 714 H= 714:00 +CPU= 32.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:00:33 STEP 715 H= 715:00 +CPU= 20.078
02:00:54 STEP 716 H= 716:00 +CPU= 20.547
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:02:38 STEP 717 H= 717:00 +CPU= 31.481
02:02:58 STEP 718 H= 718:00 +CPU= 20.141
02:03:18 STEP 719 H= 719:00 +CPU= 20.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:05:17 STEP 720 H= 720:00 +CPU= 37.028
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_29.zip
Uploading the intermediate file: upload_file_29.zip
02:06:47 STEP 721 H= 721:00 +CPU= 19.596
02:07:08 STEP 722 H= 722:00 +CPU= 20.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:08:39 STEP 723 H= 723:00 +CPU= 31.520
02:09:00 STEP 724 H= 724:00 +CPU= 20.343
02:09:19 STEP 725 H= 725:00 +CPU= 19.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:11:03 STEP 726 H= 726:00 +CPU= 32.460
02:11:23 STEP 727 H= 727:00 +CPU= 20.227
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:12:46 STEP 728 H= 728:00 +CPU= 20.390
02:13:17 STEP 729 H= 729:00 +CPU= 31.306
02:13:37 STEP 730 H= 730:00 +CPU= 20.060
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:15:19 STEP 731 H= 731:00 +CPU= 19.542
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:16:46 STEP 732 H= 732:00 +CPU= 35.325
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000732
02:17:06 STEP 733 H= 733:00 +CPU= 19.722
02:17:27 STEP 734 H= 734:00 +CPU= 20.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:19:21 STEP 735 H= 735:00 +CPU= 32.490
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:20:31 STEP 736 H= 736:00 +CPU= 20.131
02:20:52 STEP 737 H= 737:00 +CPU= 20.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:22:15 STEP 738 H= 738:00 +CPU= 32.169
02:22:36 STEP 739 H= 739:00 +CPU= 20.459
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:24:16 STEP 740 H= 740:00 +CPU= 20.614
02:24:49 STEP 741 H= 741:00 +CPU= 32.317
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:26:31 STEP 742 H= 742:00 +CPU= 20.669
02:26:52 STEP 743 H= 743:00 +CPU= 20.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:28:21 STEP 744 H= 744:00 +CPU= 38.088
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818_30.zip
Uploading the intermediate file: upload_file_30.zip
02:28:41 STEP 745 H= 745:00 +CPU= 19.743
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:30:05 STEP 746 H= 746:00 +CPU= 19.744
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0239
(argv4) batchid: 958
(argv5) wuid: 12174883
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174883
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_32bb48023a4543316632a4506cbe0d85 to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0239_1989050100_123_958_12174883.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0239_1989050100_123_958_12174883.zip
ic_ancil_file: ic_ancil_12174883
ifsdata_file: ifsdata_12174883
climate_data_file: clim_data_12174883
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_2026e556d9d01653854501cb6456e937 to: /var/lib/boinc-client/slots/13/ic_ancil_12174883.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12174883.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e4407feb319799f71b1e6cf014e455fc to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174883.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12174883.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174883.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12174883.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12174883.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12174883.xml
-- Model is restarting --
Adjusting last_iter, 746, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12174883.xml
last_cpu_time: 21442
upload_file_number: 31
last_iter: 743
last_upload: 2678400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0239_1989050100_123_958_12174883_0_r364497818
The child process has been launched with process id: 369299
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369299:369299] [20230121:023156:1674235916:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:33:36 STEP 744 H= 744:00 +CPU= 36.491
02:33:55 STEP 745 H= 745:00 +CPU= 19.292
02:34:16 STEP 746 H= 746:00 +CPU= 20.059
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:36:09 STEP 747 H= 747:00 +CPU= 31.588
02:36:29 STEP 748 H= 748:00 +CPU= 20.146
02:36:49 STEP 749 H= 749:00 +CPU= 19.706
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:38:22 STEP 750 H= 750:00 +CPU= 32.150
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:39:33 STEP 751 H= 751:00 +CPU= 19.811
02:39:54 STEP 752 H= 752:00 +CPU= 20.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:41:26 STEP 753 H= 753:00 +CPU= 31.470
02:41:47 STEP 754 H= 754:00 +CPU= 20.448
02:42:06 STEP 755 H= 755:00 +CPU= 19.614
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:44:01 STEP 756 H= 756:00 +CPU= 34.549
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+000756
02:44:21 STEP 757 H= 757:00 +CPU= 20.111
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:46:04 STEP 758 H= 758:00 +CPU= 20.406
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:47:26 STEP 759 H= 759:00 +CPU= 31.619
02:47:47 STEP 760 H= 760:00 +CPU= 20.336
02:48:07 STEP 761 H= 761:00 +CPU= 19.947
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:49:51 STEP 762 H= 762:00 +CPU= 32.368
02:50:11 STEP 763 H= 763:00 +CPU= 19.891
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:51:43 STEP 764 H= 764:00 +CPU= 20.536
02:52:16 STEP 765 H= 765:00 +CPU= 32.635
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:53:36 STEP 766 H= 766:00 +CPU= 20.135
02:53:56 STEP 767 H= 767:00 +CPU= 20.618
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
