<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
0038
(argv4) batchid: 987
(argv5) wuid: 12203682
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_18fbf5ec97d9b7828343c698c679730a to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0038_2018050100_123_987_12203682.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0038_2018050100_123_987_12203682.zip
ic_ancil_file: ic_ancil_12203682
ifsdata_file: ifsdata_12203682
climate_data_file: clim_data_12203682
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_18b57870a823b5ee819840efd729c583 to: /var/lib/boinc-client/slots/14/ic_ancil_12203682.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12203682.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6cc3e41314c38acd13a06b32be76eebf to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12203682.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12203682.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12203682.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12203682.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12203682.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12203682.xml
-- Model is restarting --
Adjusting last_iter, 571, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12203682.xml
last_cpu_time: 15756
upload_file_number: 23
last_iter: 551
last_upload: 1987200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692
The child process has been launched with process id: 333115
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:333115:333115] [20230120:221751:1674220671:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
22:18:37 STEP 552 H= 552:00 +CPU= 41.392
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:23:42 STEP 553 H= 553:00 +CPU= 23.086
22:24:06 STEP 554 H= 554:00 +CPU= 23.330
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:28:55 STEP 555 H= 555:00 +CPU= 36.709
22:29:20 STEP 556 H= 556:00 +CPU= 24.165
22:29:43 STEP 557 H= 557:00 +CPU= 22.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:31 STEP 558 H= 558:00 +CPU= 36.645
22:35:55 STEP 559 H= 559:00 +CPU= 22.772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:42 STEP 560 H= 560:00 +CPU= 23.797
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:27 STEP 561 H= 561:00 +CPU= 33.460
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:09 STEP 562 H= 562:00 +CPU= 21.599
22:53:29 STEP 563 H= 563:00 +CPU= 20.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:55:26 STEP 564 H= 564:00 +CPU= 35.624
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000564
22:55:46 STEP 565 H= 565:00 +CPU= 19.471
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:57:38 STEP 566 H= 566:00 +CPU= 20.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:59:20 STEP 567 H= 567:00 +CPU= 31.373
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:00:41 STEP 568 H= 568:00 +CPU= 20.731
23:01:01 STEP 569 H= 569:00 +CPU= 19.876
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:03:26 STEP 570 H= 570:00 +CPU= 32.142
23:03:45 STEP 571 H= 571:00 +CPU= 19.518
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:04:58 STEP 572 H= 572:00 +CPU= 20.499
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:06:41 STEP 573 H= 573:00 +CPU= 32.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:08:04 STEP 574 H= 574:00 +CPU= 20.607
23:08:24 STEP 575 H= 575:00 +CPU= 20.061
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:10:14 STEP 576 H= 576:00 +CPU= 37.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_23.zip
Uploading the intermediate file: upload_file_23.zip
23:11:34 STEP 577 H= 577:00 +CPU= 19.963
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:12:52 STEP 578 H= 578:00 +CPU= 19.806
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:14:34 STEP 579 H= 579:00 +CPU= 31.404
23:14:55 STEP 580 H= 580:00 +CPU= 21.147
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:16:47 STEP 581 H= 581:00 +CPU= 20.476
23:17:20 STEP 582 H= 582:00 +CPU= 33.030
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:18:52 STEP 583 H= 583:00 +CPU= 20.858
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:20:24 STEP 584 H= 584:00 +CPU= 20.323
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:21:56 STEP 585 H= 585:00 +CPU= 31.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:23:19 STEP 586 H= 586:00 +CPU= 21.346
23:23:39 STEP 587 H= 587:00 +CPU= 19.472
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:25:36 STEP 588 H= 588:00 +CPU= 35.850
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000588
23:25:56 STEP 589 H= 589:00 +CPU= 19.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:27:37 STEP 590 H= 590:00 +CPU= 20.652
23:28:09 STEP 591 H= 591:00 +CPU= 31.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:30:01 STEP 592 H= 592:00 +CPU= 20.505
23:30:21 STEP 593 H= 593:00 +CPU= 20.130
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:31:55 STEP 594 H= 594:00 +CPU= 32.811
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:33:28 STEP 595 H= 595:00 +CPU= 20.516
23:33:49 STEP 596 H= 596:00 +CPU= 21.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:35:32 STEP 597 H= 597:00 +CPU= 31.580
23:35:53 STEP 598 H= 598:00 +CPU= 20.742
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:38:27 STEP 599 H= 599:00 +CPU= 21.102
23:39:05 STEP 600 H= 600:00 +CPU= 37.533
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_24.zip
Uploading the intermediate file: upload_file_24.zip
23:39:25 STEP 601 H= 601:00 +CPU= 19.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:41:18 STEP 602 H= 602:00 +CPU= 19.185
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:43:11 STEP 603 H= 603:00 +CPU= 32.113
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:44:43 STEP 604 H= 604:00 +CPU= 20.518
23:45:03 STEP 605 H= 605:00 +CPU= 20.290
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:46:56 STEP 606 H= 606:00 +CPU= 32.695
23:47:16 STEP 607 H= 607:00 +CPU= 19.537
23:47:36 STEP 608 H= 608:00 +CPU= 20.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:30 STEP 609 H= 609:00 +CPU= 31.761
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:51:00 STEP 610 H= 610:00 +CPU= 20.021
23:51:20 STEP 611 H= 611:00 +CPU= 20.379
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:56 STEP 612 H= 612:00 +CPU= 35.104
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000612
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:54:36 STEP 613 H= 613:00 +CPU= 19.534
23:54:57 STEP 614 H= 614:00 +CPU= 21.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:56:42 STEP 615 H= 615:00 +CPU= 32.374
23:57:03 STEP 616 H= 616:00 +CPU= 20.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:58:34 STEP 617 H= 617:00 +CPU= 20.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:00:08 STEP 618 H= 618:00 +CPU= 32.497
00:00:28 STEP 619 H= 619:00 +CPU= 20.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:02:00 STEP 620 H= 620:00 +CPU= 20.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:03:24 STEP 621 H= 621:00 +CPU= 32.291
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:04:55 STEP 622 H= 622:00 +CPU= 19.748
00:05:16 STEP 623 H= 623:00 +CPU= 20.565
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:07:04 STEP 624 H= 624:00 +CPU= 37.458
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_25.zip
Uploading the intermediate file: upload_file_25.zip
00:07:23 STEP 625 H= 625:00 +CPU= 18.576
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:09:08 STEP 626 H= 626:00 +CPU= 18.584
00:09:40 STEP 627 H= 627:00 +CPU= 31.929
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:11:31 STEP 628 H= 628:00 +CPU= 20.691
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:13:22 STEP 629 H= 629:00 +CPU= 19.606
00:13:55 STEP 630 H= 630:00 +CPU= 32.380
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:15:25 STEP 631 H= 631:00 +CPU= 19.624
00:15:46 STEP 632 H= 632:00 +CPU= 20.485
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:17:40 STEP 633 H= 633:00 +CPU= 32.976
00:18:01 STEP 634 H= 634:00 +CPU= 20.530
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2018050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0038
(argv4) batchid: 987
(argv5) wuid: 12203682
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_18fbf5ec97d9b7828343c698c679730a to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0038_2018050100_123_987_12203682.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0038_2018050100_123_987_12203682.zip
ic_ancil_file: ic_ancil_12203682
ifsdata_file: ifsdata_12203682
climate_data_file: clim_data_12203682
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_18b57870a823b5ee819840efd729c583 to: /var/lib/boinc-client/slots/14/ic_ancil_12203682.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12203682.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6cc3e41314c38acd13a06b32be76eebf to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12203682.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12203682.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12203682.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12203682.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12203682.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12203682.xml
-- Model is restarting --
Adjusting last_iter, 635, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12203682.xml
last_cpu_time: 17848
upload_file_number: 26
last_iter: 623
last_upload: 2246400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692
The child process has been launched with process id: 351057
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:351057:351057] [20230121:002113:1674228073:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:22:32 STEP 624 H= 624:00 +CPU= 35.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:23:43 STEP 625 H= 625:00 +CPU= 20.331
00:24:04 STEP 626 H= 626:00 +CPU= 20.879
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:25:56 STEP 627 H= 627:00 +CPU= 31.952
00:26:16 STEP 628 H= 628:00 +CPU= 20.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:27:57 STEP 629 H= 629:00 +CPU= 19.891
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:30:11 STEP 630 H= 630:00 +CPU= 31.792
00:30:32 STEP 631 H= 631:00 +CPU= 20.727
00:30:52 STEP 632 H= 632:00 +CPU= 19.958
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:32:54 STEP 633 H= 633:00 +CPU= 31.188
00:33:16 STEP 634 H= 634:00 +CPU= 21.167
00:33:36 STEP 635 H= 635:00 +CPU= 20.044
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:35:23 STEP 636 H= 636:00 +CPU= 35.783
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000636
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:36:53 STEP 637 H= 637:00 +CPU= 19.487
00:37:14 STEP 638 H= 638:00 +CPU= 20.396
00:37:45 STEP 639 H= 639:00 +CPU= 30.914
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:39:37 STEP 640 H= 640:00 +CPU= 20.878
00:39:57 STEP 641 H= 641:00 +CPU= 19.844
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:41:39 STEP 642 H= 642:00 +CPU= 32.326
00:42:00 STEP 643 H= 643:00 +CPU= 20.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:43:31 STEP 644 H= 644:00 +CPU= 20.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:45:24 STEP 645 H= 645:00 +CPU= 31.620
00:45:44 STEP 646 H= 646:00 +CPU= 20.455
00:46:06 STEP 647 H= 647:00 +CPU= 21.300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:48:56 STEP 648 H= 648:00 +CPU= 37.023
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_26.zip
Uploading the intermediate file: upload_file_26.zip
00:49:17 STEP 649 H= 649:00 +CPU= 20.479
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:50:49 STEP 650 H= 650:00 +CPU= 20.912
00:51:20 STEP 651 H= 651:00 +CPU= 31.346
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:53:02 STEP 652 H= 652:00 +CPU= 20.585
00:53:22 STEP 653 H= 653:00 +CPU= 20.416
00:53:54 STEP 654 H= 654:00 +CPU= 32.098
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:55:56 STEP 655 H= 655:00 +CPU= 20.080
00:56:16 STEP 656 H= 656:00 +CPU= 20.630
00:56:48 STEP 657 H= 657:00 +CPU= 31.974
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:58:41 STEP 658 H= 658:00 +CPU= 21.669
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:59:51 STEP 659 H= 659:00 +CPU= 19.368
01:00:26 STEP 660 H= 660:00 +CPU= 34.173
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000660
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:02:06 STEP 661 H= 661:00 +CPU= 19.579
01:02:27 STEP 662 H= 662:00 +CPU= 20.941
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:04:01 STEP 663 H= 663:00 +CPU= 31.913
01:04:22 STEP 664 H= 664:00 +CPU= 20.677
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:05:54 STEP 665 H= 665:00 +CPU= 19.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:07:29 STEP 666 H= 666:00 +CPU= 32.361
01:07:49 STEP 667 H= 667:00 +CPU= 20.278
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:09:02 STEP 668 H= 668:00 +CPU= 20.511
01:09:33 STEP 669 H= 669:00 +CPU= 31.709
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:11:15 STEP 670 H= 670:00 +CPU= 20.751
01:11:36 STEP 671 H= 671:00 +CPU= 21.014
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:13:26 STEP 672 H= 672:00 +CPU= 37.913
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_27.zip
Uploading the intermediate file: upload_file_27.zip
01:13:46 STEP 673 H= 673:00 +CPU= 19.880
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:15:28 STEP 674 H= 674:00 +CPU= 20.138
01:15:59 STEP 675 H= 675:00 +CPU= 31.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:17:31 STEP 676 H= 676:00 +CPU= 20.257
01:17:50 STEP 677 H= 677:00 +CPU= 19.302
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:19:33 STEP 678 H= 678:00 +CPU= 32.252
01:19:53 STEP 679 H= 679:00 +CPU= 19.383
01:20:13 STEP 680 H= 680:00 +CPU= 20.174
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:22:05 STEP 681 H= 681:00 +CPU= 31.106
01:22:26 STEP 682 H= 682:00 +CPU= 20.325
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:23:58 STEP 683 H= 683:00 +CPU= 19.948
01:24:33 STEP 684 H= 684:00 +CPU= 35.264
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:26:04 STEP 685 H= 685:00 +CPU= 19.238
01:26:25 STEP 686 H= 686:00 +CPU= 20.693
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:28:27 STEP 687 H= 687:00 +CPU= 32.151
01:28:48 STEP 688 H= 688:00 +CPU= 20.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:30:07 STEP 689 H= 689:00 +CPU= 19.688
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:31:39 STEP 690 H= 690:00 +CPU= 31.899
01:32:00 STEP 691 H= 691:00 +CPU= 20.275
01:32:20 STEP 692 H= 692:00 +CPU= 20.463
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:34:03 STEP 693 H= 693:00 +CPU= 31.554
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:35:35 STEP 694 H= 694:00 +CPU= 20.819
01:35:56 STEP 695 H= 695:00 +CPU= 20.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:37:44 STEP 696 H= 696:00 +CPU= 37.048
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_28.zip
Uploading the intermediate file: upload_file_28.zip
01:38:03 STEP 697 H= 697:00 +CPU= 19.201
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:39:20 STEP 698 H= 698:00 +CPU= 19.876
01:39:51 STEP 699 H= 699:00 +CPU= 31.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:41:33 STEP 700 H= 700:00 +CPU= 20.156
01:41:53 STEP 701 H= 701:00 +CPU= 19.813
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:43:34 STEP 702 H= 702:00 +CPU= 31.573
01:43:54 STEP 703 H= 703:00 +CPU= 19.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:45:24 STEP 704 H= 704:00 +CPU= 20.152
01:45:56 STEP 705 H= 705:00 +CPU= 31.395
01:46:17 STEP 706 H= 706:00 +CPU= 20.732
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:47:48 STEP 707 H= 707:00 +CPU= 20.296
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:49:23 STEP 708 H= 708:00 +CPU= 34.635
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000708
01:49:43 STEP 709 H= 709:00 +CPU= 20.300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:51:15 STEP 710 H= 710:00 +CPU= 21.253
01:51:47 STEP 711 H= 711:00 +CPU= 31.315
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:53:38 STEP 712 H= 712:00 +CPU= 20.423
01:53:58 STEP 713 H= 713:00 +CPU= 19.855
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:56:02 STEP 714 H= 714:00 +CPU= 32.274
01:56:21 STEP 715 H= 715:00 +CPU= 19.708
01:56:42 STEP 716 H= 716:00 +CPU= 20.451
01:57:13 STEP 717 H= 717:00 +CPU= 31.371
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:58:54 STEP 718 H= 718:00 +CPU= 20.849
01:59:15 STEP 719 H= 719:00 +CPU= 20.320
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:00:53 STEP 720 H= 720:00 +CPU= 36.671
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_29.zip
Uploading the intermediate file: upload_file_29.zip
02:01:13 STEP 721 H= 721:00 +CPU= 19.857
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:02:35 STEP 722 H= 722:00 +CPU= 19.034
02:03:07 STEP 723 H= 723:00 +CPU= 31.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:04:47 STEP 724 H= 724:00 +CPU= 20.312
02:05:07 STEP 725 H= 725:00 +CPU= 19.469
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:06:49 STEP 726 H= 726:00 +CPU= 32.239
02:07:10 STEP 727 H= 727:00 +CPU= 20.419
02:07:30 STEP 728 H= 728:00 +CPU= 20.485
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:09:03 STEP 729 H= 729:00 +CPU= 31.425
02:09:23 STEP 730 H= 730:00 +CPU= 19.999
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:10:54 STEP 731 H= 731:00 +CPU= 19.596
02:11:30 STEP 732 H= 732:00 +CPU= 35.335
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000732
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:12:51 STEP 733 H= 733:00 +CPU= 20.008
02:13:11 STEP 734 H= 734:00 +CPU= 20.138
02:13:42 STEP 735 H= 735:00 +CPU= 31.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:15:26 STEP 736 H= 736:00 +CPU= 20.922
02:15:45 STEP 737 H= 737:00 +CPU= 19.216
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:17:08 STEP 738 H= 738:00 +CPU= 32.467
02:17:28 STEP 739 H= 739:00 +CPU= 19.797
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:19:11 STEP 740 H= 740:00 +CPU= 20.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:20:33 STEP 741 H= 741:00 +CPU= 31.541
02:20:53 STEP 742 H= 742:00 +CPU= 20.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:22:06 STEP 743 H= 743:00 +CPU= 20.144
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:24:04 STEP 744 H= 744:00 +CPU= 37.404
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_30.zip
Uploading the intermediate file: upload_file_30.zip
02:24:25 STEP 745 H= 745:00 +CPU= 20.468
02:24:45 STEP 746 H= 746:00 +CPU= 20.446
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:26:38 STEP 747 H= 747:00 +CPU= 31.560
02:26:59 STEP 748 H= 748:00 +CPU= 20.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2018050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0038
(argv4) batchid: 987
(argv5) wuid: 12203682
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_18fbf5ec97d9b7828343c698c679730a to: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0038_2018050100_123_987_12203682.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_ps_0038_2018050100_123_987_12203682.zip
ic_ancil_file: ic_ancil_12203682
ifsdata_file: ifsdata_12203682
climate_data_file: clim_data_12203682
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_18b57870a823b5ee819840efd729c583 to: /var/lib/boinc-client/slots/14/ic_ancil_12203682.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12203682.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6cc3e41314c38acd13a06b32be76eebf to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12203682.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12203682.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12203682.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12203682.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12203682.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12203682.xml
-- Model is restarting --
Adjusting last_iter, 748, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12203682.xml
last_cpu_time: 20908
upload_file_number: 31
last_iter: 743
last_upload: 2678400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692
The child process has been launched with process id: 369034
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:369034:369034] [20230121:022956:1674235796:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
02:30:34 STEP 744 H= 744:00 +CPU= 35.753
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:31:55 STEP 745 H= 745:00 +CPU= 19.469
02:32:16 STEP 746 H= 746:00 +CPU= 21.106
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:33:49 STEP 747 H= 747:00 +CPU= 31.575
02:34:10 STEP 748 H= 748:00 +CPU= 20.616
02:34:30 STEP 749 H= 749:00 +CPU= 20.464
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:36:24 STEP 750 H= 750:00 +CPU= 32.226
02:36:44 STEP 751 H= 751:00 +CPU= 20.144
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:38:06 STEP 752 H= 752:00 +CPU= 20.508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:39:29 STEP 753 H= 753:00 +CPU= 31.189
02:39:49 STEP 754 H= 754:00 +CPU= 20.212
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:41:09 STEP 755 H= 755:00 +CPU= 19.591
02:41:43 STEP 756 H= 756:00 +CPU= 34.455
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000756
02:42:03 STEP 757 H= 757:00 +CPU= 19.924
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:43:45 STEP 758 H= 758:00 +CPU= 20.986
02:44:16 STEP 759 H= 759:00 +CPU= 31.175
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:45:59 STEP 760 H= 760:00 +CPU= 20.378
02:46:19 STEP 761 H= 761:00 +CPU= 20.328
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:47:44 STEP 762 H= 762:00 +CPU= 32.371
02:48:04 STEP 763 H= 763:00 +CPU= 19.840
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:49:36 STEP 764 H= 764:00 +CPU= 20.425
02:50:07 STEP 765 H= 765:00 +CPU= 31.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:51:39 STEP 766 H= 766:00 +CPU= 20.157
02:51:59 STEP 767 H= 767:00 +CPU= 20.422
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:53:37 STEP 768 H= 768:00 +CPU= 36.970
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203682/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0038_2018050100_123_987_12203682_0_r1125810692_31.zip
Uploading the intermediate file: upload_file_31.zip
02:53:57 STEP 769 H= 769:00 +CPU= 20.313
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
