<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2ms_2016092300_15_991_12210199_0_r1455937353_7.zip
Uploading the intermediate file: upload_file_7.zip
23:22:01 STEP 768 H= 192:00 +CPU= 26.720
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000768
23:22:27 STEP 769 H= 192:15 +CPU= 25.540
23:22:51 STEP 770 H= 192:30 +CPU= 24.083
23:23:19 STEP 771 H= 192:45 +CPU= 27.179
23:23:45 STEP 772 H= 193:00 +CPU= 26.116
23:24:11 STEP 773 H= 193:15 +CPU= 25.775
23:24:36 STEP 774 H= 193:30 +CPU= 24.056
23:25:03 STEP 775 H= 193:45 +CPU= 26.812
23:25:27 STEP 776 H= 194:00 +CPU= 24.039
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:26:06 STEP 777 H= 194:15 +CPU= 27.257
23:26:32 STEP 778 H= 194:30 +CPU= 25.909
23:26:53 STEP 779 H= 194:45 +CPU= 20.381
23:27:24 STEP 780 H= 195:00 +CPU= 30.869
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000780
23:27:49 STEP 781 H= 195:15 +CPU= 24.694
23:28:12 STEP 782 H= 195:30 +CPU= 22.696
23:28:38 STEP 783 H= 195:45 +CPU= 25.531
23:29:05 STEP 784 H= 196:00 +CPU= 26.250
23:29:27 STEP 785 H= 196:15 +CPU= 21.762
23:29:52 STEP 786 H= 196:30 +CPU= 25.427
23:30:17 STEP 787 H= 196:45 +CPU= 24.312
23:30:43 STEP 788 H= 197:00 +CPU= 25.696
23:31:08 STEP 789 H= 197:15 +CPU= 24.462
23:31:33 STEP 790 H= 197:30 +CPU= 24.693
23:32:00 STEP 791 H= 197:45 +CPU= 26.415
23:32:31 STEP 792 H= 198:00 +CPU= 31.043
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000792
23:32:56 STEP 793 H= 198:15 +CPU= 24.581
23:33:23 STEP 794 H= 198:30 +CPU= 26.226
23:33:49 STEP 795 H= 198:45 +CPU= 24.948
23:34:15 STEP 796 H= 199:00 +CPU= 26.413
23:34:40 STEP 797 H= 199:15 +CPU= 24.607
23:35:07 STEP 798 H= 199:30 +CPU= 26.030
23:35:32 STEP 799 H= 199:45 +CPU= 24.939
23:36:00 STEP 800 H= 200:00 +CPU= 27.860
23:36:25 STEP 801 H= 200:15 +CPU= 24.737
23:36:50 STEP 802 H= 200:30 +CPU= 24.929
23:37:15 STEP 803 H= 200:45 +CPU= 24.696
23:37:48 STEP 804 H= 201:00 +CPU= 32.020
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000804
23:38:13 STEP 805 H= 201:15 +CPU= 24.847
23:38:37 STEP 806 H= 201:30 +CPU= 24.056
23:39:04 STEP 807 H= 201:45 +CPU= 26.235
23:39:29 STEP 808 H= 202:00 +CPU= 25.036
23:39:55 STEP 809 H= 202:15 +CPU= 25.326
23:40:20 STEP 810 H= 202:30 +CPU= 24.868
23:40:46 STEP 811 H= 202:45 +CPU= 25.092
23:41:13 STEP 812 H= 203:00 +CPU= 26.352
23:41:36 STEP 813 H= 203:15 +CPU= 23.109
23:42:02 STEP 814 H= 203:30 +CPU= 25.266
23:42:26 STEP 815 H= 203:45 +CPU= 24.002
23:43:01 STEP 816 H= 204:00 +CPU= 34.264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000816
23:43:25 STEP 817 H= 204:15 +CPU= 24.184
23:43:50 STEP 818 H= 204:30 +CPU= 24.659
23:44:16 STEP 819 H= 204:45 +CPU= 25.422
23:44:42 STEP 820 H= 205:00 +CPU= 24.865
23:45:09 STEP 821 H= 205:15 +CPU= 27.423
23:45:35 STEP 822 H= 205:30 +CPU= 25.474
23:45:57 STEP 823 H= 205:45 +CPU= 21.602
23:46:22 STEP 824 H= 206:00 +CPU= 24.513
23:46:48 STEP 825 H= 206:15 +CPU= 25.319
23:47:13 STEP 826 H= 206:30 +CPU= 25.436
23:47:37 STEP 827 H= 206:45 +CPU= 23.703
23:48:08 STEP 828 H= 207:00 +CPU= 30.111
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000828
23:48:31 STEP 829 H= 207:15 +CPU= 23.389
23:48:56 STEP 830 H= 207:30 +CPU= 24.743
23:49:21 STEP 831 H= 207:45 +CPU= 23.909
23:49:46 STEP 832 H= 208:00 +CPU= 25.450
23:50:10 STEP 833 H= 208:15 +CPU= 23.436
23:50:37 STEP 834 H= 208:30 +CPU= 26.247
23:51:02 STEP 835 H= 208:45 +CPU= 25.465
23:51:26 STEP 836 H= 209:00 +CPU= 23.752
23:51:51 STEP 837 H= 209:15 +CPU= 24.204
23:52:16 STEP 838 H= 209:30 +CPU= 25.225
23:52:41 STEP 839 H= 209:45 +CPU= 24.556
23:53:13 STEP 840 H= 210:00 +CPU= 31.637
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000840
23:53:41 STEP 841 H= 210:15 +CPU= 27.251
23:54:07 STEP 842 H= 210:30 +CPU= 25.912
23:54:31 STEP 843 H= 210:45 +CPU= 22.953
23:54:55 STEP 844 H= 211:00 +CPU= 24.531
23:55:21 STEP 845 H= 211:15 +CPU= 25.471
23:55:47 STEP 846 H= 211:30 +CPU= 25.382
23:56:12 STEP 847 H= 211:45 +CPU= 24.769
23:56:37 STEP 848 H= 212:00 +CPU= 24.862
23:57:03 STEP 849 H= 212:15 +CPU= 24.953
23:57:28 STEP 850 H= 212:30 +CPU= 25.382
23:57:54 STEP 851 H= 212:45 +CPU= 24.756
23:58:25 STEP 852 H= 213:00 +CPU= 31.350
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000852
23:58:52 STEP 853 H= 213:15 +CPU= 26.409
23:59:19 STEP 854 H= 213:30 +CPU= 26.317
23:59:44 STEP 855 H= 213:45 +CPU= 25.274
00:00:11 STEP 856 H= 214:00 +CPU= 25.934
00:00:38 STEP 857 H= 214:15 +CPU= 26.502
00:01:02 STEP 858 H= 214:30 +CPU= 24.445
00:01:28 STEP 859 H= 214:45 +CPU= 25.253
00:01:56 STEP 860 H= 215:00 +CPU= 27.516
00:02:21 STEP 861 H= 215:15 +CPU= 24.221
00:02:47 STEP 862 H= 215:30 +CPU= 25.823
00:03:13 STEP 863 H= 215:45 +CPU= 25.522
00:03:47 STEP 864 H= 216:00 +CPU= 33.656
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000864
00:04:12 STEP 865 H= 216:15 +CPU= 25.153
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000864
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2ms_2016092300_15_991_12210199_0_r1455937353_8.zip
Uploading the intermediate file: upload_file_8.zip
00:04:37 STEP 866 H= 216:30 +CPU= 24.440
00:05:02 STEP 867 H= 216:45 +CPU= 24.968
00:05:27 STEP 868 H= 217:00 +CPU= 24.044
00:05:55 STEP 869 H= 217:15 +CPU= 27.325
00:06:20 STEP 870 H= 217:30 +CPU= 24.943
00:06:43 STEP 871 H= 217:45 +CPU= 23.093
00:07:10 STEP 872 H= 218:00 +CPU= 26.733
00:07:37 STEP 873 H= 218:15 +CPU= 26.436
00:08:00 STEP 874 H= 218:30 +CPU= 22.498
00:08:24 STEP 875 H= 218:45 +CPU= 24.003
00:08:56 STEP 876 H= 219:00 +CPU= 31.965
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000876
00:09:21 STEP 877 H= 219:15 +CPU= 24.808
00:09:45 STEP 878 H= 219:30 +CPU= 23.390
00:10:10 STEP 879 H= 219:45 +CPU= 24.853
00:10:36 STEP 880 H= 220:00 +CPU= 25.374
00:11:00 STEP 881 H= 220:15 +CPU= 23.715
00:11:26 STEP 882 H= 220:30 +CPU= 25.583
00:11:51 STEP 883 H= 220:45 +CPU= 25.127
00:12:17 STEP 884 H= 221:00 +CPU= 25.308
00:12:43 STEP 885 H= 221:15 +CPU= 26.110
00:13:10 STEP 886 H= 221:30 +CPU= 25.953
00:13:35 STEP 887 H= 221:45 +CPU= 24.870
00:14:06 STEP 888 H= 222:00 +CPU= 31.229
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000888
00:14:32 STEP 889 H= 222:15 +CPU= 25.128
00:14:55 STEP 890 H= 222:30 +CPU= 23.077
00:15:21 STEP 891 H= 222:45 +CPU= 25.469
00:15:48 STEP 892 H= 223:00 +CPU= 26.434
00:16:12 STEP 893 H= 223:15 +CPU= 23.714
00:16:37 STEP 894 H= 223:30 +CPU= 24.907
00:17:02 STEP 895 H= 223:45 +CPU= 23.710
00:17:27 STEP 896 H= 224:00 +CPU= 25.117
00:17:53 STEP 897 H= 224:15 +CPU= 25.368
00:18:18 STEP 898 H= 224:30 +CPU= 25.040
00:18:44 STEP 899 H= 224:45 +CPU= 26.038
00:19:15 STEP 900 H= 225:00 +CPU= 29.919
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000900
00:19:41 STEP 901 H= 225:15 +CPU= 26.184
00:20:07 STEP 902 H= 225:30 +CPU= 25.260
00:20:32 STEP 903 H= 225:45 +CPU= 24.943
00:20:59 STEP 904 H= 226:00 +CPU= 26.548
00:21:30 STEP 905 H= 226:15 +CPU= 26.651
00:21:55 STEP 906 H= 226:30 +CPU= 25.206
00:22:22 STEP 907 H= 226:45 +CPU= 26.299
00:22:50 STEP 908 H= 227:00 +CPU= 27.665
00:23:17 STEP 909 H= 227:15 +CPU= 26.095
00:23:44 STEP 910 H= 227:30 +CPU= 26.904
00:24:11 STEP 911 H= 227:45 +CPU= 26.592
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2ms
(argv4) batchid: 991
(argv5) wuid: 12210199
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6a63b302896c8c93cbf26299d22069f9 to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2ms_2016092300_15_991_12210199.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2ms_2016092300_15_991_12210199.zip
ic_ancil_file: ic_ancil_12210199
ifsdata_file: ifsdata_12210199
climate_data_file: clim_data_12210199
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/1/ic_ancil_12210199.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12210199.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a56e012b4a1ff3d40bb48216f0ad8b87 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12210199.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12210199.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12210199.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12210199.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12210199.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12210199.xml
-- Model is restarting --
Adjusting last_iter, 912, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12210199.xml
last_cpu_time: 23041
upload_file_number: 9
last_iter: 913
last_upload: 778500
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a2ms_2016092300_15_991_12210199_0_r1455937353
The child process has been launched with process id: 3797184
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3797184:3797184] [20230214:002535:1676359535:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
00:26:06 STEP 912 H= 228:00 +CPU= 26.568
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000912
00:26:32 STEP 913 H= 228:15 +CPU= 25.464
00:26:56 STEP 914 H= 228:30 +CPU= 23.657
00:27:20 STEP 915 H= 228:45 +CPU= 24.011
00:27:46 STEP 916 H= 229:00 +CPU= 25.897
00:28:12 STEP 917 H= 229:15 +CPU= 25.633
00:28:38 STEP 918 H= 229:30 +CPU= 24.968
00:29:02 STEP 919 H= 229:45 +CPU= 23.822
00:29:26 STEP 920 H= 230:00 +CPU= 24.253
00:29:48 STEP 921 H= 230:15 +CPU= 21.589
00:30:13 STEP 922 H= 230:30 +CPU= 24.382
00:30:39 STEP 923 H= 230:45 +CPU= 25.865
00:31:12 STEP 924 H= 231:00 +CPU= 32.533
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000924
00:31:35 STEP 925 H= 231:15 +CPU= 22.817
00:32:01 STEP 926 H= 231:30 +CPU= 25.459
00:32:27 STEP 927 H= 231:45 +CPU= 25.588
00:32:51 STEP 928 H= 232:00 +CPU= 24.054
00:33:15 STEP 929 H= 232:15 +CPU= 24.051
00:33:39 STEP 930 H= 232:30 +CPU= 23.967
00:34:05 STEP 931 H= 232:45 +CPU= 24.992
00:34:32 STEP 932 H= 233:00 +CPU= 26.602
00:34:57 STEP 933 H= 233:15 +CPU= 25.305
00:35:19 STEP 934 H= 233:30 +CPU= 21.491
00:35:44 STEP 935 H= 233:45 +CPU= 24.094
00:36:16 STEP 936 H= 234:00 +CPU= 31.527
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000936
00:36:41 STEP 937 H= 234:15 +CPU= 24.554
00:37:06 STEP 938 H= 234:30 +CPU= 25.437
00:37:31 STEP 939 H= 234:45 +CPU= 24.731
00:37:59 STEP 940 H= 235:00 +CPU= 27.135
00:38:25 STEP 941 H= 235:15 +CPU= 25.334
00:38:52 STEP 942 H= 235:30 +CPU= 26.516
00:39:18 STEP 943 H= 235:45 +CPU= 25.769
00:39:45 STEP 944 H= 236:00 +CPU= 27.389
00:40:11 STEP 945 H= 236:15 +CPU= 25.264
00:40:36 STEP 946 H= 236:30 +CPU= 24.468
00:41:00 STEP 947 H= 236:45 +CPU= 24.173
00:41:33 STEP 948 H= 237:00 +CPU= 32.310
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000948
00:42:00 STEP 949 H= 237:15 +CPU= 26.079
00:42:24 STEP 950 H= 237:30 +CPU= 24.544
00:42:51 STEP 951 H= 237:45 +CPU= 26.717
00:43:18 STEP 952 H= 238:00 +CPU= 26.212
00:43:44 STEP 953 H= 238:15 +CPU= 26.208
00:44:11 STEP 954 H= 238:30 +CPU= 25.732
00:44:37 STEP 955 H= 238:45 +CPU= 25.590
00:45:05 STEP 956 H= 239:00 +CPU= 27.528
00:45:33 STEP 957 H= 239:15 +CPU= 27.674
00:46:00 STEP 958 H= 239:30 +CPU= 26.436
00:46:24 STEP 959 H= 239:45 +CPU= 24.328
00:46:57 STEP 960 H= 240:00 +CPU= 32.140
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000960
00:47:23 STEP 961 H= 240:15 +CPU= 25.285
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000960
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2ms_2016092300_15_991_12210199_0_r1455937353_9.zip
Uploading the intermediate file: upload_file_9.zip
00:47:49 STEP 962 H= 240:30 +CPU= 25.879
00:48:14 STEP 963 H= 240:45 +CPU= 24.524
00:48:39 STEP 964 H= 241:00 +CPU= 24.898
00:49:07 STEP 965 H= 241:15 +CPU= 26.792
00:49:32 STEP 966 H= 241:30 +CPU= 25.251
00:49:57 STEP 967 H= 241:45 +CPU= 24.703
00:50:24 STEP 968 H= 242:00 +CPU= 26.704
00:50:50 STEP 969 H= 242:15 +CPU= 25.731
00:51:14 STEP 970 H= 242:30 +CPU= 23.730
00:51:40 STEP 971 H= 242:45 +CPU= 25.459
00:52:11 STEP 972 H= 243:00 +CPU= 30.582
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000972
00:52:35 STEP 973 H= 243:15 +CPU= 23.889
00:52:59 STEP 974 H= 243:30 +CPU= 23.745
00:53:26 STEP 975 H= 243:45 +CPU= 26.757
00:53:52 STEP 976 H= 244:00 +CPU= 24.893
00:54:17 STEP 977 H= 244:15 +CPU= 24.931
00:54:41 STEP 978 H= 244:30 +CPU= 23.894
00:55:06 STEP 979 H= 244:45 +CPU= 24.639
00:55:33 STEP 980 H= 245:00 +CPU= 26.382
00:55:58 STEP 981 H= 245:15 +CPU= 25.226
00:56:23 STEP 982 H= 245:30 +CPU= 24.537
00:56:48 STEP 983 H= 245:45 +CPU= 24.454
00:57:20 STEP 984 H= 246:00 +CPU= 31.744
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000984
00:57:44 STEP 985 H= 246:15 +CPU= 23.880
00:58:08 STEP 986 H= 246:30 +CPU= 23.556
00:58:33 STEP 987 H= 246:45 +CPU= 24.905
00:58:59 STEP 988 H= 247:00 +CPU= 25.801
00:59:23 STEP 989 H= 247:15 +CPU= 23.527
00:59:49 STEP 990 H= 247:30 +CPU= 25.997
01:00:15 STEP 991 H= 247:45 +CPU= 25.611
01:00:42 STEP 992 H= 248:00 +CPU= 26.441
01:01:08 STEP 993 H= 248:15 +CPU= 25.882
01:01:34 STEP 994 H= 248:30 +CPU= 25.463
01:02:01 STEP 995 H= 248:45 +CPU= 26.161
01:02:33 STEP 996 H= 249:00 +CPU= 31.423
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+000996
01:02:59 STEP 997 H= 249:15 +CPU= 26.008
01:03:24 STEP 998 H= 249:30 +CPU= 24.315
01:03:49 STEP 999 H= 249:45 +CPU= 24.212
01:04:13 STEP 1000 H= 250:00 +CPU= 24.699
01:04:38 STEP 1001 H= 250:15 +CPU= 24.397
01:05:04 STEP 1002 H= 250:30 +CPU= 25.400
01:05:29 STEP 1003 H= 250:45 +CPU= 24.981
01:05:54 STEP 1004 H= 251:00 +CPU= 24.515
01:06:18 STEP 1005 H= 251:15 +CPU= 23.422
01:06:42 STEP 1006 H= 251:30 +CPU= 23.808
01:07:08 STEP 1007 H= 251:45 +CPU= 25.027
01:07:41 STEP 1008 H= 252:00 +CPU= 33.094
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001008
01:08:05 STEP 1009 H= 252:15 +CPU= 23.050
01:08:29 STEP 1010 H= 252:30 +CPU= 23.930
01:08:55 STEP 1011 H= 252:45 +CPU= 26.066
01:09:22 STEP 1012 H= 253:00 +CPU= 25.697
01:09:48 STEP 1013 H= 253:15 +CPU= 26.330
01:10:14 STEP 1014 H= 253:30 +CPU= 25.472
01:10:39 STEP 1015 H= 253:45 +CPU= 24.538
01:11:04 STEP 1016 H= 254:00 +CPU= 25.352
01:11:31 STEP 1017 H= 254:15 +CPU= 26.573
01:11:54 STEP 1018 H= 254:30 +CPU= 22.928
01:12:18 STEP 1019 H= 254:45 +CPU= 23.547
01:12:49 STEP 1020 H= 255:00 +CPU= 30.151
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001020
01:13:14 STEP 1021 H= 255:15 +CPU= 25.135
01:13:39 STEP 1022 H= 255:30 +CPU= 24.619
01:14:03 STEP 1023 H= 255:45 +CPU= 24.188
01:14:31 STEP 1024 H= 256:00 +CPU= 26.810
01:14:58 STEP 1025 H= 256:15 +CPU= 27.163
01:15:22 STEP 1026 H= 256:30 +CPU= 23.908
01:15:47 STEP 1027 H= 256:45 +CPU= 24.715
01:16:14 STEP 1028 H= 257:00 +CPU= 25.950
01:16:39 STEP 1029 H= 257:15 +CPU= 25.278
01:17:03 STEP 1030 H= 257:30 +CPU= 23.113
01:17:27 STEP 1031 H= 257:45 +CPU= 23.816
01:17:59 STEP 1032 H= 258:00 +CPU= 31.828
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001032
01:18:24 STEP 1033 H= 258:15 +CPU= 24.603
01:18:48 STEP 1034 H= 258:30 +CPU= 23.914
01:19:14 STEP 1035 H= 258:45 +CPU= 25.637
01:19:41 STEP 1036 H= 259:00 +CPU= 26.598
01:20:06 STEP 1037 H= 259:15 +CPU= 24.436
01:20:31 STEP 1038 H= 259:30 +CPU= 25.054
01:20:57 STEP 1039 H= 259:45 +CPU= 25.224
01:21:23 STEP 1040 H= 260:00 +CPU= 25.522
01:21:47 STEP 1041 H= 260:15 +CPU= 24.287
01:22:14 STEP 1042 H= 260:30 +CPU= 26.251
01:22:39 STEP 1043 H= 260:45 +CPU= 24.182
01:23:11 STEP 1044 H= 261:00 +CPU= 31.716
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001044
01:23:35 STEP 1045 H= 261:15 +CPU= 23.715
01:24:02 STEP 1046 H= 261:30 +CPU= 26.562
01:24:26 STEP 1047 H= 261:45 +CPU= 23.458
01:24:52 STEP 1048 H= 262:00 +CPU= 26.216
01:25:18 STEP 1049 H= 262:15 +CPU= 25.242
01:25:44 STEP 1050 H= 262:30 +CPU= 25.257
01:26:08 STEP 1051 H= 262:45 +CPU= 24.015
01:26:33 STEP 1052 H= 263:00 +CPU= 25.255
01:26:59 STEP 1053 H= 263:15 +CPU= 25.429
01:27:26 STEP 1054 H= 263:30 +CPU= 26.514
01:27:53 STEP 1055 H= 263:45 +CPU= 27.034
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01
DATE= 2016 10 2
NORMS AT NSTEP CNT4 1056
SPECTRAL NORMS - LOG(PREHYDS) 0.115129362210273E+02
LEV VORTICITY DIVERGENCE TEMPERATURE KINETIC ENERGY
AVE 0.135259792832067E-04 0.215874178586156E-05 0.190173376279423E+03 0.856430014388153E+02
GPNORM HUMIDITY AVERAGE MINIMUM MAXIMUM
AVE 0.551734083633418E-03 0.999999999999999E-08 0.801187575839040E-02
GPNORM LIQUID WATER AVERAGE MINIMUM MAXIMUM
AVE 0.528999100817509E-06 0.000000000000000E+00 0.628740135624847E-03
GPNORM ICE WATER AVERAGE MINIMUM MAXIMUM
AVE 0.253389540188168E-06 0.000000000000000E+00 0.107108194007350E-03
GPNORM SNOW AVERAGE MINIMUM MAXIMUM
AVE 0.311968535203261E-06 0.000000000000000E+00 0.229362864325504E-03
GPNORM RAIN AVERAGE MINIMUM MAXIMUM
AVE 0.632224580009906E-08 0.000000000000000E+00 0.459533594037791E-04
GPNORM CLOUD FRACTION AVERAGE MINIMUM MAXIMUM
AVE 0.201949639070413E+00 0.000000000000000E+00 0.100000000000000E+01
GPNORM CLW AVERAGE MINIMUM MAXIMUM
AVE 0.526249290425953E-06 0.000000000000000E+00 0.632216782831756E-03
GPNORM CIW AVERAGE MINIMUM MAXIMUM
AVE 0.253083377929088E-06 0.000000000000000E+00 0.106701630865289E-03
GPNORM CC AVERAGE MINIMUM MAXIMUM
AVE 0.201964368140139E+00 0.000000000000000E+00 0.100000000000000E+01
GPNORM RFL AVERAGE MINIMUM MAXIMUM
AVE 0.293052882291238E-07 0.000000000000000E+00 0.228676292419397E-03
GPNORM SFL AVERAGE MINIMUM MAXIMUM
AVE 0.380634619534445E-06 0.000000000000000E+00 0.276881775761104E-03
GPNORM CRFL AVERAGE MINIMUM MAXIMUM
AVE 0.354039981757167E-07 0.000000000000000E+00 0.109114502738038E-03
GPNORM CSFL AVERAGE MINIMUM MAXIMUM
AVE 0.594882338855640E-07 0.000000000000000E+00 0.243326430134275E-03
GPNORM PFRC AVERAGE MINIMUM MAXIMUM
AVE 0.500897990125013E-01 0.000000000000000E+00 0.100000000000000E+01
NSTEP = 1056 STEPO F00000000
CREATE RESTART FILE ./srf00110000.0001
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./srf00110000.0001 MODE=w
IO-STREAM CLOSED - ./srf00110000.0001
MPI-TASK: 1 - ************ BYTES IN 34 RECORDS TRANSFERRED IN 2.1174 SECONDS******** Mbytes/s, TOTAL TIME= 2.8767(73.6%)
CREATE RESTART CONTROL FILE ./rcf
WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00101200.0001
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1056 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001056 MODE=w
IO-STREAM CLOSED - ICMGGh7zg+001056
MPI-TASK: 1 - 315229658 BYTES IN 6 RECORDS TRANSFERRED IN 0.0004 SECONDS788074.2 Mbytes/s, TOTAL TIME= 0.0094( 4.3%)
NSTEP = 1056 STEPO 0AA000000
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMSHh7zg+001056 MODE=a
IO-STREAM CLOSED - ./ICMSHh7zg+001056
MPI-TASK: 1 - 707765250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0079 SECONDS 89590.5 Mbytes/s, TOTAL TIME= 0.4403( 1.8%)
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001056 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001056
MPI-TASK: 1 - 5043674519 BYTES IN 125 RECORDS TRANSFERRED IN 0.0059 SECONDS854860.1 Mbytes/s, TOTAL TIME= 0.1911( 3.1%)
STEPO_FPOS WAS : B00000F00E
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001056 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001056
MPI-TASK: 1 - 306656025 BYTES IN 2 RECORDS TRANSFERRED IN 0.0002 SECONDS******** Mbytes/s, TOTAL TIME= 0.0059( 3.4%)
STEPO_FPOS WAS : V00000000Y
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001056 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001056
MPI-TASK: 1 - 315229658 BYTES IN 4 RECORDS TRANSFERRED IN 0.0004 SECONDS788074.1 Mbytes/s, TOTAL TIME= 0.0067( 6.0%)
STEPO_FPOS WAS : T00000000M
NSTEP = 1056 STEPO 0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat
01:27:26 0AAA00AAA STEPO 1055 26.514 26.514 26.844 61:01 61:51 0.21016645397331E-05 2GB 0MB
01:27:54 F00000000 STEPO 1056 27.230 27.230 27.594 61:28 62:18 0.21587417858616E-05 2GB 0MB
01:27:57 0AA000000 STEPO 1056 2.943 2.943 3.176 61:31 62:22 0.21587417858616E-05 2GB 0MB
01:27:58 FULLPOS-B DYNFPOS 1056 1.418 1.418 1.424 61:32 62:23 0.21587417858616E-05 2GB 0MB
01:28:01 FULLPOS-V DYNFPOS 1056 2.903 2.903 3.173 61:35 62:26 0.21587417858616E-05 2GB 0MB
01:28:02 FULLPOS-T DYNFPOS 1056 0.446 0.446 0.447 61:36 62:27 0.21587417858616E-05 2GB 0MB
01:28:02 0AAA00AAA STEPO 1056 0.483 0.483 0.484 61:36 62:27 0.21587417858616E-05 2GB 0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf
CSTEP = " 1056",
CTIME = "00110000 ",
NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 1044, 2*0,
IPRGPNSRES = 1,
IPRGPEWRES = 1,
IPRTRWRES = 1,
IPRTRVRES = 1,
GMASS0 = 0.000000000000000E+000,
GMASSI = 0.000000000000000E+000
/
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/1/progress_file_12210199.xml
<running_values>
<last_cpu_time>26734.910000</last_cpu_time>
<upload_file_number>10</upload_file_number>
<last_iter>1056</last_iter>
<last_upload>864900</last_upload>
<model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2ms
(argv4) batchid: 991
(argv5) wuid: 12210199
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6a63b302896c8c93cbf26299d22069f9 to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2ms_2016092300_15_991_12210199.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2ms_2016092300_15_991_12210199.zip
ic_ancil_file: ic_ancil_12210199
ifsdata_file: ifsdata_12210199
climate_data_file: clim_data_12210199
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/1/ic_ancil_12210199.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12210199.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a56e012b4a1ff3d40bb48216f0ad8b87 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12210199.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12210199.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12210199.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12210199.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12210199.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12210199.xml
-- Model is restarting --
Adjusting last_iter, 1056, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12210199.xml
last_cpu_time: 26734
upload_file_number: 10
last_iter: 1057
last_upload: 864900
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a2ms_2016092300_15_991_12210199_0_r1455937353
The child process has been launched with process id: 3808307
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:wkst-01.kalnet.xyz:1:1:3808307:3808307] [20230214:012909:1676363349:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210199/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2ms_2016092300_15_991_12210199_0_r1455937353_10.zip
Uploading the intermediate file: upload_file_10.zip
01:29:40 STEP 1056 H= 264:00 +CPU= 26.675
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001056
01:30:07 STEP 1057 H= 264:15 +CPU= 26.272
01:30:34 STEP 1058 H= 264:30 +CPU= 27.233
01:30:58 STEP 1059 H= 264:45 +CPU= 23.269
01:31:25 STEP 1060 H= 265:00 +CPU= 26.925
01:31:50 STEP 1061 H= 265:15 +CPU= 24.779
01:32:15 STEP 1062 H= 265:30 +CPU= 24.248
01:32:41 STEP 1063 H= 265:45 +CPU= 25.594
01:33:06 STEP 1064 H= 266:00 +CPU= 25.301
01:33:31 STEP 1065 H= 266:15 +CPU= 24.187
01:33:55 STEP 1066 H= 266:30 +CPU= 24.558
01:34:22 STEP 1067 H= 266:45 +CPU= 25.825
01:34:51 STEP 1068 H= 267:00 +CPU= 29.468
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001068
01:35:18 STEP 1069 H= 267:15 +CPU= 26.003
01:35:43 STEP 1070 H= 267:30 +CPU= 24.703
01:36:07 STEP 1071 H= 267:45 +CPU= 24.461
01:36:33 STEP 1072 H= 268:00 +CPU= 24.793
01:36:57 STEP 1073 H= 268:15 +CPU= 24.583
01:37:23 STEP 1074 H= 268:30 +CPU= 25.274
01:37:47 STEP 1075 H= 268:45 +CPU= 23.820
01:38:14 STEP 1076 H= 269:00 +CPU= 27.123
01:38:40 STEP 1077 H= 269:15 +CPU= 25.365
01:39:04 STEP 1078 H= 269:30 +CPU= 23.587
01:39:28 STEP 1079 H= 269:45 +CPU= 23.613
01:39:58 STEP 1080 H= 270:00 +CPU= 29.705
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001080
01:40:22 STEP 1081 H= 270:15 +CPU= 23.105
01:40:48 STEP 1082 H= 270:30 +CPU= 25.564
01:41:14 STEP 1083 H= 270:45 +CPU= 26.546
01:41:39 STEP 1084 H= 271:00 +CPU= 23.985
01:42:03 STEP 1085 H= 271:15 +CPU= 24.552
01:42:28 STEP 1086 H= 271:30 +CPU= 23.867
01:42:53 STEP 1087 H= 271:45 +CPU= 25.184
01:43:18 STEP 1088 H= 272:00 +CPU= 24.432
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01
AVE 0.259687309148965E-06 0.000000000000000E+00 0.119005997858927E-03
GPNORM CC AVERAGE MINIMUM MAXIMUM
AVE 0.202602973311982E+00 0.000000000000000E+00 0.100000000000000E+01
GPNORM RFL AVERAGE MINIMUM MAXIMUM
AVE 0.335002051002487E-07 0.000000000000000E+00 0.220967652196355E-03
GPNORM SFL AVERAGE MINIMUM MAXIMUM
AVE 0.396757970084676E-06 0.000000000000000E+00 0.249375947909941E-03
GPNORM CRFL AVERAGE MINIMUM MAXIMUM
AVE 0.443363614758195E-07 0.000000000000000E+00 0.375323390723471E-03
GPNORM CSFL AVERAGE MINIMUM MAXIMUM
AVE 0.827237276050807E-07 0.000000000000000E+00 0.375691442472723E-03
GPNORM PFRC AVERAGE MINIMUM MAXIMUM
AVE 0.487866136535750E-01 0.000000000000000E+00 0.100000000000000E+01
NSTEP = 1080 STEPO A00000000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1080 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001080 MODE=w
IO-STREAM CLOSED - ICMGGh7zg+001080
MPI-TASK: 1 - 319350234 BYTES IN 6 RECORDS TRANSFERRED IN 0.0005 SECONDS638700.5 Mbytes/s, TOTAL TIME= 0.0092( 5.4%)
NSTEP = 1080 STEPO 0AA000000
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMSHh7zg+001080 MODE=a
IO-STREAM CLOSED - ./ICMSHh7zg+001080
MPI-TASK: 1 - 707765250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0101 SECONDS 70075.8 Mbytes/s, TOTAL TIME= 0.2108( 4.8%)
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001080 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001080
MPI-TASK: 1 - 5109603735 BYTES IN 125 RECORDS TRANSFERRED IN 0.0071 SECONDS719662.5 Mbytes/s, TOTAL TIME= 0.2247( 3.2%)
STEPO_FPOS WAS : B00000F00E
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001080 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001080
MPI-TASK: 1 - 310776601 BYTES IN 2 RECORDS TRANSFERRED IN 0.0001 SECONDS******** Mbytes/s, TOTAL TIME= 0.0043( 2.3%)
STEPO_FPOS WAS : V00000000Y
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001080 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001080
MPI-TASK: 1 - 319350234 BYTES IN 4 RECORDS TRANSFERRED IN 0.0003 SECONDS******** Mbytes/s, TOTAL TIME= 0.0053( 5.7%)
STEPO_FPOS WAS : T00000000M
NSTEP = 1080 STEPO 0AAA00AAA
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1080 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001080 MODE=a
IO-STREAM CLOSED - ICMGGh7zg+001080
MPI-TASK: 1 - 319350225 BYTES IN 7 RECORDS TRANSFERRED IN 0.0006 SECONDS532250.4 Mbytes/s, TOTAL TIME= 0.0104( 5.8%)
01:39:58 STEP 1080 H= 270:00 +CPU= 29.705
NSTEP = 1081 STEPO 0AAA00AAA
01:40:22 STEP 1081 H= 270:15 +CPU= 23.105
NSTEP = 1082 STEPO 0AAA00AAA
01:40:48 STEP 1082 H= 270:30 +CPU= 25.564
NSTEP = 1083 STEPO 0AAA00AAA
01:41:14 STEP 1083 H= 270:45 +CPU= 26.546
NSTEP = 1084 STEPO 0AAA00AAA
01:41:39 STEP 1084 H= 271:00 +CPU= 23.985
ISTEP= 1085 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP= 1085 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
NSTEP = 1085 STEPO 0AAA00AAA
01:42:03 STEP 1085 H= 271:15 +CPU= 24.552
NSTEP = 1086 STEPO 0AAA00AAA
01:42:28 STEP 1086 H= 271:30 +CPU= 23.867
NSTEP = 1087 STEPO 0AAA00AAA
01:42:53 STEP 1087 H= 271:45 +CPU= 25.184
NSTEP = 1088 STEPO 0AAA00AAA
01:43:18 STEP 1088 H= 272:00 +CPU= 24.432
NSTEP = 1089 STEPO 0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat
01:40:48 0AAA00AAA STEPO 1083 25.564 25.564 26.043 11:29 11:38 0.22241745332865E-05 2GB 0MB
01:41:14 0AAA00AAA STEPO 1084 26.546 26.546 26.812 11:56 12:05 0.22241745332865E-05 2GB 0MB
01:41:39 0AAA00AAA STEPO 1085 23.985 23.985 24.254 12:20 12:29 0.22241745332865E-05 2GB 0MB
01:42:03 0AAA00AAA STEPO 1086 24.552 24.552 24.695 12:44 12:54 0.22241745332865E-05 2GB 0MB
01:42:28 0AAA00AAA STEPO 1087 23.867 23.867 24.312 13:08 13:18 0.22241745332865E-05 2GB 0MB
01:42:53 0AAA00AAA STEPO 1088 25.184 25.184 25.485 13:34 13:44 0.22241745332865E-05 2GB 0MB
01:43:18 0AAA00AAA STEPO 1089 24.433 24.433 24.927 13:58 14:09 0.22241745332865E-05 2GB 0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf
CSTEP = " 1056",
CTIME = "00110000 ",
NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 1044, 2*0,
IPRGPNSRES = 1,
IPRGPEWRES = 1,
IPRTRWRES = 1,
IPRTRVRES = 1,
GMASS0 = 0.000000000000000E+000,
GMASSI = 0.000000000000000E+000
/
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/1/progress_file_12210199.xml
<running_values>
<last_cpu_time>27579.490000</last_cpu_time>
<upload_file_number>11</upload_file_number>
<last_iter>1089</last_iter>
<last_upload>951300</last_upload>
<model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
</stderr_txt>
]]>
