<core_client_version>7.22.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
var/lib/boinc/slots/2/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000780
08:22:17 STEP 781 H= 195:15 +CPU= 19.086
08:22:36 STEP 782 H= 195:30 +CPU= 18.775
08:22:54 STEP 783 H= 195:45 +CPU= 18.341
08:23:13 STEP 784 H= 196:00 +CPU= 19.281
08:23:32 STEP 785 H= 196:15 +CPU= 18.479
08:23:51 STEP 786 H= 196:30 +CPU= 18.616
08:24:09 STEP 787 H= 196:45 +CPU= 18.476
08:24:28 STEP 788 H= 197:00 +CPU= 19.298
08:24:47 STEP 789 H= 197:15 +CPU= 18.594
08:25:06 STEP 790 H= 197:30 +CPU= 18.636
08:25:24 STEP 791 H= 197:45 +CPU= 18.362
08:25:48 STEP 792 H= 198:00 +CPU= 24.137
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000792
08:26:08 STEP 793 H= 198:15 +CPU= 19.477
08:26:27 STEP 794 H= 198:30 +CPU= 18.648
08:26:45 STEP 795 H= 198:45 +CPU= 18.181
08:27:04 STEP 796 H= 199:00 +CPU= 19.187
08:30:35 STEP 797 H= 199:15 +CPU= 20.281
08:31:20 STEP 798 H= 199:30 +CPU= 19.381
08:31:38 STEP 799 H= 199:45 +CPU= 18.357
08:31:57 STEP 800 H= 200:00 +CPU= 18.828
08:32:15 STEP 801 H= 200:15 +CPU= 18.189
08:32:33 STEP 802 H= 200:30 +CPU= 18.148
08:32:52 STEP 803 H= 200:45 +CPU= 18.352
08:33:16 STEP 804 H= 201:00 +CPU= 24.189
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000804
08:33:35 STEP 805 H= 201:15 +CPU= 18.733
08:33:53 STEP 806 H= 201:30 +CPU= 18.638
08:34:12 STEP 807 H= 201:45 +CPU= 18.628
08:34:32 STEP 808 H= 202:00 +CPU= 19.433
08:34:51 STEP 809 H= 202:15 +CPU= 18.927
08:35:10 STEP 810 H= 202:30 +CPU= 18.925
08:35:28 STEP 811 H= 202:45 +CPU= 18.696
08:35:47 STEP 812 H= 203:00 +CPU= 19.221
08:36:06 STEP 813 H= 203:15 +CPU= 18.557
08:36:25 STEP 814 H= 203:30 +CPU= 18.696
08:36:44 STEP 815 H= 203:45 +CPU= 18.684
08:37:11 STEP 816 H= 204:00 +CPU= 26.927
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000816
08:37:29 STEP 817 H= 204:15 +CPU= 18.708
08:37:48 STEP 818 H= 204:30 +CPU= 18.607
08:38:06 STEP 819 H= 204:45 +CPU= 18.356
08:38:26 STEP 820 H= 205:00 +CPU= 19.219
08:38:44 STEP 821 H= 205:15 +CPU= 18.464
08:39:03 STEP 822 H= 205:30 +CPU= 18.480
08:39:22 STEP 823 H= 205:45 +CPU= 19.114
08:39:41 STEP 824 H= 206:00 +CPU= 19.279
08:39:59 STEP 825 H= 206:15 +CPU= 18.140
08:40:18 STEP 826 H= 206:30 +CPU= 18.885
08:40:37 STEP 827 H= 206:45 +CPU= 18.588
08:41:02 STEP 828 H= 207:00 +CPU= 25.089
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000828
08:41:20 STEP 829 H= 207:15 +CPU= 18.354
08:41:39 STEP 830 H= 207:30 +CPU= 18.761
08:41:57 STEP 831 H= 207:45 +CPU= 18.136
08:42:17 STEP 832 H= 208:00 +CPU= 19.825
08:42:36 STEP 833 H= 208:15 +CPU= 18.676
08:42:55 STEP 834 H= 208:30 +CPU= 18.683
08:43:13 STEP 835 H= 208:45 +CPU= 18.086
08:43:32 STEP 836 H= 209:00 +CPU= 19.306
08:43:51 STEP 837 H= 209:15 +CPU= 18.945
08:44:10 STEP 838 H= 209:30 +CPU= 19.076
08:44:29 STEP 839 H= 209:45 +CPU= 18.401
08:44:53 STEP 840 H= 210:00 +CPU= 24.138
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000840
08:45:11 STEP 841 H= 210:15 +CPU= 18.542
08:45:30 STEP 842 H= 210:30 +CPU= 19.032
08:45:49 STEP 843 H= 210:45 +CPU= 18.472
08:46:08 STEP 844 H= 211:00 +CPU= 19.477
08:46:27 STEP 845 H= 211:15 +CPU= 18.477
08:46:45 STEP 846 H= 211:30 +CPU= 18.254
08:47:04 STEP 847 H= 211:45 +CPU= 18.245
08:47:23 STEP 848 H= 212:00 +CPU= 19.478
08:47:42 STEP 849 H= 212:15 +CPU= 18.682
08:48:01 STEP 850 H= 212:30 +CPU= 19.032
08:48:20 STEP 851 H= 212:45 +CPU= 18.702
08:48:44 STEP 852 H= 213:00 +CPU= 23.993
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000852
08:49:03 STEP 853 H= 213:15 +CPU= 18.922
08:49:24 STEP 854 H= 213:30 +CPU= 20.279
08:49:42 STEP 855 H= 213:45 +CPU= 18.563
08:50:02 STEP 856 H= 214:00 +CPU= 19.506
08:50:21 STEP 857 H= 214:15 +CPU= 19.031
08:50:40 STEP 858 H= 214:30 +CPU= 18.851
08:50:58 STEP 859 H= 214:45 +CPU= 18.563
08:51:17 STEP 860 H= 215:00 +CPU= 18.762
08:51:35 STEP 861 H= 215:15 +CPU= 18.073
08:51:55 STEP 862 H= 215:30 +CPU= 19.119
08:52:13 STEP 863 H= 215:45 +CPU= 18.479
08:52:39 STEP 864 H= 216:00 +CPU= 26.158
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000864
08:52:57 STEP 865 H= 216:15 +CPU= 18.137
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000852
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000864
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194_0_r880167314_8.zip
08:53:16 STEP 866 H= 216:30 +CPU= 18.980
Uploading the intermediate file: upload_file_8.zip
08:53:35 STEP 867 H= 216:45 +CPU= 18.803
08:53:55 STEP 868 H= 217:00 +CPU= 19.869
08:54:14 STEP 869 H= 217:15 +CPU= 18.583
08:54:32 STEP 870 H= 217:30 +CPU= 18.525
08:54:51 STEP 871 H= 217:45 +CPU= 18.840
08:55:11 STEP 872 H= 218:00 +CPU= 19.220
08:55:29 STEP 873 H= 218:15 +CPU= 18.498
08:55:48 STEP 874 H= 218:30 +CPU= 19.030
08:56:06 STEP 875 H= 218:45 +CPU= 18.225
08:56:31 STEP 876 H= 219:00 +CPU= 24.471
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000876
08:56:49 STEP 877 H= 219:15 +CPU= 18.041
08:57:07 STEP 878 H= 219:30 +CPU= 18.458
08:57:26 STEP 879 H= 219:45 +CPU= 18.812
08:57:46 STEP 880 H= 220:00 +CPU= 19.725
08:58:05 STEP 881 H= 220:15 +CPU= 18.616
08:58:23 STEP 882 H= 220:30 +CPU= 18.426
08:58:42 STEP 883 H= 220:45 +CPU= 18.646
08:59:01 STEP 884 H= 221:00 +CPU= 19.244
08:59:20 STEP 885 H= 221:15 +CPU= 18.617
08:59:38 STEP 886 H= 221:30 +CPU= 18.428
08:59:57 STEP 887 H= 221:45 +CPU= 18.725
09:00:22 STEP 888 H= 222:00 +CPU= 24.803
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000888
09:00:41 STEP 889 H= 222:15 +CPU= 18.652
09:00:59 STEP 890 H= 222:30 +CPU= 18.464
09:01:18 STEP 891 H= 222:45 +CPU= 18.379
09:01:37 STEP 892 H= 223:00 +CPU= 19.112
09:01:55 STEP 893 H= 223:15 +CPU= 18.601
09:02:14 STEP 894 H= 223:30 +CPU= 18.465
09:02:32 STEP 895 H= 223:45 +CPU= 18.562
09:02:52 STEP 896 H= 224:00 +CPU= 19.135
09:03:10 STEP 897 H= 224:15 +CPU= 18.426
09:03:29 STEP 898 H= 224:30 +CPU= 19.025
09:03:48 STEP 899 H= 224:45 +CPU= 19.232
09:04:13 STEP 900 H= 225:00 +CPU= 24.691
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000900
09:04:32 STEP 901 H= 225:15 +CPU= 18.318
09:04:51 STEP 902 H= 225:30 +CPU= 18.954
09:05:09 STEP 903 H= 225:45 +CPU= 18.861
09:05:29 STEP 904 H= 226:00 +CPU= 19.300
09:05:47 STEP 905 H= 226:15 +CPU= 18.329
09:06:06 STEP 906 H= 226:30 +CPU= 18.608
09:06:24 STEP 907 H= 226:45 +CPU= 18.631
09:06:44 STEP 908 H= 227:00 +CPU= 19.134
09:07:02 STEP 909 H= 227:15 +CPU= 18.265
09:07:21 STEP 910 H= 227:30 +CPU= 18.686
09:07:39 STEP 911 H= 227:45 +CPU= 18.536
09:08:07 STEP 912 H= 228:00 +CPU= 27.379
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000912
09:08:26 STEP 913 H= 228:15 +CPU= 18.812
09:08:44 STEP 914 H= 228:30 +CPU= 18.629
09:09:03 STEP 915 H= 228:45 +CPU= 18.805
09:09:22 STEP 916 H= 229:00 +CPU= 19.078
09:09:41 STEP 917 H= 229:15 +CPU= 18.886
09:10:00 STEP 918 H= 229:30 +CPU= 18.656
09:10:18 STEP 919 H= 229:45 +CPU= 18.609
09:10:38 STEP 920 H= 230:00 +CPU= 19.613
09:10:57 STEP 921 H= 230:15 +CPU= 19.018
09:11:16 STEP 922 H= 230:30 +CPU= 18.655
09:11:35 STEP 923 H= 230:45 +CPU= 18.775
09:12:00 STEP 924 H= 231:00 +CPU= 25.106
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000924
09:12:19 STEP 925 H= 231:15 +CPU= 19.000
09:12:38 STEP 926 H= 231:30 +CPU= 18.925
09:12:56 STEP 927 H= 231:45 +CPU= 18.382
09:13:16 STEP 928 H= 232:00 +CPU= 19.613
09:13:35 STEP 929 H= 232:15 +CPU= 19.247
09:13:54 STEP 930 H= 232:30 +CPU= 18.689
09:14:13 STEP 931 H= 232:45 +CPU= 19.308
09:14:33 STEP 932 H= 233:00 +CPU= 20.132
09:14:53 STEP 933 H= 233:15 +CPU= 19.197
09:15:12 STEP 934 H= 233:30 +CPU= 18.858
09:15:30 STEP 935 H= 233:45 +CPU= 18.308
09:15:54 STEP 936 H= 234:00 +CPU= 24.492
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000936
09:16:14 STEP 937 H= 234:15 +CPU= 19.432
09:16:37 STEP 938 H= 234:30 +CPU= 21.598
09:16:55 STEP 939 H= 234:45 +CPU= 18.463
09:17:15 STEP 940 H= 235:00 +CPU= 19.312
09:17:34 STEP 941 H= 235:15 +CPU= 19.108
09:17:53 STEP 942 H= 235:30 +CPU= 18.914
09:18:11 STEP 943 H= 235:45 +CPU= 18.381
09:18:31 STEP 944 H= 236:00 +CPU= 19.849
09:18:51 STEP 945 H= 236:15 +CPU= 19.246
09:19:09 STEP 946 H= 236:30 +CPU= 18.590
09:19:28 STEP 947 H= 236:45 +CPU= 18.901
09:19:53 STEP 948 H= 237:00 +CPU= 25.120
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000948
09:20:13 STEP 949 H= 237:15 +CPU= 19.226
09:20:32 STEP 950 H= 237:30 +CPU= 19.383
09:20:51 STEP 951 H= 237:45 +CPU= 19.015
09:21:11 STEP 952 H= 238:00 +CPU= 19.733
09:21:30 STEP 953 H= 238:15 +CPU= 19.187
09:21:49 STEP 954 H= 238:30 +CPU= 18.779
09:22:07 STEP 955 H= 238:45 +CPU= 18.473
09:22:27 STEP 956 H= 239:00 +CPU= 19.354
09:22:46 STEP 957 H= 239:15 +CPU= 19.006
09:23:05 STEP 958 H= 239:30 +CPU= 19.060
09:23:25 STEP 959 H= 239:45 +CPU= 20.148
09:23:52 STEP 960 H= 240:00 +CPU= 26.650
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000960
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2zf
(argv4) batchid: 1018
(argv5) wuid: 12292194
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/2
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_b4fab48d0af4fac93eb28590e3937875 to: /var/lib/boinc/slots/2/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194.zip
ic_ancil_file: ic_ancil_12292194
ifsdata_file: ifsdata_12292194
climate_data_file: clim_data_12292194
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/2/ic_ancil_12292194.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/2/ic_ancil_12292194.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f62e56b85a35fac0226008a6de9ff649 to: /var/lib/boinc/slots/2/ifsdata/ifsdata_12292194.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/2/ifsdata/ifsdata_12292194.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/2/159l_2/clim_data_12292194.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/2/159l_2/clim_data_12292194.zip
Checking for progress XML file: /var/lib/boinc/slots/2/progress_file_12292194.xml
Opened progress file ok : /var/lib/boinc/slots/2/progress_file_12292194.xml
-- Model is restarting --
Adjusting last_iter, 961, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/2/progress_file_12292194.xml
last_cpu_time: 18500
upload_file_number: 9
last_iter: 961
last_upload: 778500
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2zf_2016092300_20_1018_12292194_0_r880167314
The child process has been launched with process id: 590441
Executing the command: /var/lib/boinc/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:590441:590441] [20240614:092508:1718349908:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000960
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194_0_r880167314_9.zip
Uploading the intermediate file: upload_file_9.zip
09:25:31 STEP 960 H= 240:00 +CPU= 19.762
09:25:50 STEP 961 H= 240:15 +CPU= 19.115
09:26:09 STEP 962 H= 240:30 +CPU= 18.756
09:26:28 STEP 963 H= 240:45 +CPU= 18.729
09:26:47 STEP 964 H= 241:00 +CPU= 19.315
09:27:06 STEP 965 H= 241:15 +CPU= 18.913
09:27:24 STEP 966 H= 241:30 +CPU= 18.308
09:27:44 STEP 967 H= 241:45 +CPU= 19.262
09:28:04 STEP 968 H= 242:00 +CPU= 20.022
09:28:44 STEP 969 H= 242:15 +CPU= 19.645
09:29:03 STEP 970 H= 242:30 +CPU= 19.439
09:29:23 STEP 971 H= 242:45 +CPU= 19.020
09:29:48 STEP 972 H= 243:00 +CPU= 24.930
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000972
09:30:06 STEP 973 H= 243:15 +CPU= 18.678
09:30:25 STEP 974 H= 243:30 +CPU= 18.867
09:30:44 STEP 975 H= 243:45 +CPU= 18.619
09:31:04 STEP 976 H= 244:00 +CPU= 19.688
09:31:22 STEP 977 H= 244:15 +CPU= 18.668
09:31:41 STEP 978 H= 244:30 +CPU= 18.889
09:32:00 STEP 979 H= 244:45 +CPU= 18.793
09:32:19 STEP 980 H= 245:00 +CPU= 19.406
09:32:39 STEP 981 H= 245:15 +CPU= 19.224
09:32:58 STEP 982 H= 245:30 +CPU= 18.950
09:33:17 STEP 983 H= 245:45 +CPU= 18.993
09:33:41 STEP 984 H= 246:00 +CPU= 24.670
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000984
09:34:00 STEP 985 H= 246:15 +CPU= 18.936
09:34:20 STEP 986 H= 246:30 +CPU= 19.252
09:34:38 STEP 987 H= 246:45 +CPU= 18.598
09:34:58 STEP 988 H= 247:00 +CPU= 19.226
09:35:16 STEP 989 H= 247:15 +CPU= 18.568
09:35:35 STEP 990 H= 247:30 +CPU= 19.035
09:35:54 STEP 991 H= 247:45 +CPU= 19.002
09:36:14 STEP 992 H= 248:00 +CPU= 19.515
09:36:33 STEP 993 H= 248:15 +CPU= 18.832
09:36:52 STEP 994 H= 248:30 +CPU= 19.056
09:37:11 STEP 995 H= 248:45 +CPU= 18.825
09:37:35 STEP 996 H= 249:00 +CPU= 24.411
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000996
09:37:54 STEP 997 H= 249:15 +CPU= 18.274
09:38:13 STEP 998 H= 249:30 +CPU= 19.142
09:39:54 STEP 999 H= 249:45 +CPU= 19.830
09:40:13 STEP 1000 H= 250:00 +CPU= 19.472
09:40:32 STEP 1001 H= 250:15 +CPU= 18.302
09:40:50 STEP 1002 H= 250:30 +CPU= 18.299
09:41:09 STEP 1003 H= 250:45 +CPU= 18.890
09:42:22 STEP 1004 H= 251:00 +CPU= 22.754
09:42:41 STEP 1005 H= 251:15 +CPU= 18.403
09:42:59 STEP 1006 H= 251:30 +CPU= 18.406
09:43:18 STEP 1007 H= 251:45 +CPU= 18.446
09:43:44 STEP 1008 H= 252:00 +CPU= 26.675
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001008
09:44:03 STEP 1009 H= 252:15 +CPU= 18.528
09:44:22 STEP 1010 H= 252:30 +CPU= 19.001
09:44:41 STEP 1011 H= 252:45 +CPU= 19.257
09:45:02 STEP 1012 H= 253:00 +CPU= 20.225
09:45:20 STEP 1013 H= 253:15 +CPU= 18.706
09:45:39 STEP 1014 H= 253:30 +CPU= 18.672
09:45:58 STEP 1015 H= 253:45 +CPU= 18.695
09:46:17 STEP 1016 H= 254:00 +CPU= 19.639
09:46:36 STEP 1017 H= 254:15 +CPU= 18.910
09:46:55 STEP 1018 H= 254:30 +CPU= 19.063
09:47:14 STEP 1019 H= 254:45 +CPU= 18.934
09:47:39 STEP 1020 H= 255:00 +CPU= 24.475
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001020
09:47:57 STEP 1021 H= 255:15 +CPU= 18.269
09:48:16 STEP 1022 H= 255:30 +CPU= 18.804
09:48:35 STEP 1023 H= 255:45 +CPU= 18.996
09:48:55 STEP 1024 H= 256:00 +CPU= 19.574
09:49:13 STEP 1025 H= 256:15 +CPU= 18.475
09:49:32 STEP 1026 H= 256:30 +CPU= 18.569
09:49:54 STEP 1027 H= 256:45 +CPU= 19.434
09:50:14 STEP 1028 H= 257:00 +CPU= 19.521
09:50:32 STEP 1029 H= 257:15 +CPU= 18.481
09:50:51 STEP 1030 H= 257:30 +CPU= 18.557
09:51:10 STEP 1031 H= 257:45 +CPU= 18.966
09:51:34 STEP 1032 H= 258:00 +CPU= 24.456
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001032
09:51:53 STEP 1033 H= 258:15 +CPU= 18.666
09:52:12 STEP 1034 H= 258:30 +CPU= 18.481
09:53:35 STEP 1035 H= 258:45 +CPU= 20.167
09:53:54 STEP 1036 H= 259:00 +CPU= 19.300
09:54:13 STEP 1037 H= 259:15 +CPU= 18.642
09:54:32 STEP 1038 H= 259:30 +CPU= 18.703
09:54:50 STEP 1039 H= 259:45 +CPU= 18.901
09:55:10 STEP 1040 H= 260:00 +CPU= 19.177
09:55:28 STEP 1041 H= 260:15 +CPU= 18.628
09:55:47 STEP 1042 H= 260:30 +CPU= 18.701
09:56:06 STEP 1043 H= 260:45 +CPU= 18.800
09:56:31 STEP 1044 H= 261:00 +CPU= 25.328
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001044
09:56:50 STEP 1045 H= 261:15 +CPU= 18.500
09:57:09 STEP 1046 H= 261:30 +CPU= 18.756
09:57:27 STEP 1047 H= 261:45 +CPU= 18.751
09:57:47 STEP 1048 H= 262:00 +CPU= 19.357
09:58:05 STEP 1049 H= 262:15 +CPU= 18.484
09:58:24 STEP 1050 H= 262:30 +CPU= 18.559
09:58:43 STEP 1051 H= 262:45 +CPU= 18.929
09:59:03 STEP 1052 H= 263:00 +CPU= 19.975
09:59:21 STEP 1053 H= 263:15 +CPU= 18.318
09:59:40 STEP 1054 H= 263:30 +CPU= 18.784
09:59:59 STEP 1055 H= 263:45 +CPU= 18.967
10:00:26 STEP 1056 H= 264:00 +CPU= 26.639
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001056
10:00:44 STEP 1057 H= 264:15 +CPU= 18.328
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001056
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194_0_r880167314_10.zip
Uploading the intermediate file: upload_file_10.zip
10:01:03 STEP 1058 H= 264:30 +CPU= 18.856
10:01:22 STEP 1059 H= 264:45 +CPU= 18.938
10:01:42 STEP 1060 H= 265:00 +CPU= 20.179
10:02:01 STEP 1061 H= 265:15 +CPU= 18.534
10:02:19 STEP 1062 H= 265:30 +CPU= 18.333
10:02:38 STEP 1063 H= 265:45 +CPU= 18.612
10:02:57 STEP 1064 H= 266:00 +CPU= 19.466
10:03:16 STEP 1065 H= 266:15 +CPU= 18.651
10:03:34 STEP 1066 H= 266:30 +CPU= 18.230
10:03:53 STEP 1067 H= 266:45 +CPU= 18.380
10:04:17 STEP 1068 H= 267:00 +CPU= 24.466
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001068
10:04:36 STEP 1069 H= 267:15 +CPU= 19.150
10:04:55 STEP 1070 H= 267:30 +CPU= 18.859
10:05:14 STEP 1071 H= 267:45 +CPU= 18.470
10:05:33 STEP 1072 H= 268:00 +CPU= 19.336
10:05:52 STEP 1073 H= 268:15 +CPU= 18.579
10:06:11 STEP 1074 H= 268:30 +CPU= 18.797
10:06:29 STEP 1075 H= 268:45 +CPU= 18.623
10:06:49 STEP 1076 H= 269:00 +CPU= 19.505
10:07:08 STEP 1077 H= 269:15 +CPU= 18.840
10:07:26 STEP 1078 H= 269:30 +CPU= 18.525
10:07:45 STEP 1079 H= 269:45 +CPU= 18.982
10:08:10 STEP 1080 H= 270:00 +CPU= 24.696
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001080
10:08:29 STEP 1081 H= 270:15 +CPU= 18.773
10:08:48 STEP 1082 H= 270:30 +CPU= 18.873
10:09:06 STEP 1083 H= 270:45 +CPU= 18.335
10:09:26 STEP 1084 H= 271:00 +CPU= 19.384
10:09:45 STEP 1085 H= 271:15 +CPU= 18.931
10:10:03 STEP 1086 H= 271:30 +CPU= 18.553
10:10:22 STEP 1087 H= 271:45 +CPU= 18.689
10:10:42 STEP 1088 H= 272:00 +CPU= 19.665
10:11:00 STEP 1089 H= 272:15 +CPU= 18.389
10:11:18 STEP 1090 H= 272:30 +CPU= 18.439
10:11:37 STEP 1091 H= 272:45 +CPU= 18.814
10:12:03 STEP 1092 H= 273:00 +CPU= 25.247
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001092
10:12:22 STEP 1093 H= 273:15 +CPU= 19.076
10:12:40 STEP 1094 H= 273:30 +CPU= 18.714
10:12:59 STEP 1095 H= 273:45 +CPU= 18.597
10:13:19 STEP 1096 H= 274:00 +CPU= 19.543
10:13:38 STEP 1097 H= 274:15 +CPU= 19.084
10:13:57 STEP 1098 H= 274:30 +CPU= 18.684
10:14:15 STEP 1099 H= 274:45 +CPU= 18.331
10:14:35 STEP 1100 H= 275:00 +CPU= 19.686
10:14:53 STEP 1101 H= 275:15 +CPU= 18.674
10:15:12 STEP 1102 H= 275:30 +CPU= 18.602
10:15:31 STEP 1103 H= 275:45 +CPU= 18.545
10:15:58 STEP 1104 H= 276:00 +CPU= 27.254
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001104
10:16:17 STEP 1105 H= 276:15 +CPU= 19.190
10:16:36 STEP 1106 H= 276:30 +CPU= 18.861
10:16:55 STEP 1107 H= 276:45 +CPU= 18.833
10:17:14 STEP 1108 H= 277:00 +CPU= 19.521
10:17:33 STEP 1109 H= 277:15 +CPU= 18.984
10:17:52 STEP 1110 H= 277:30 +CPU= 18.695
10:18:11 STEP 1111 H= 277:45 +CPU= 18.592
10:18:31 STEP 1112 H= 278:00 +CPU= 19.675
10:19:28 STEP 1113 H= 278:15 +CPU= 22.614
10:19:47 STEP 1114 H= 278:30 +CPU= 18.874
10:20:06 STEP 1115 H= 278:45 +CPU= 19.095
10:20:31 STEP 1116 H= 279:00 +CPU= 24.934
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001116
10:20:50 STEP 1117 H= 279:15 +CPU= 19.085
10:21:10 STEP 1118 H= 279:30 +CPU= 19.805
10:21:29 STEP 1119 H= 279:45 +CPU= 19.149
10:21:48 STEP 1120 H= 280:00 +CPU= 19.176
10:22:06 STEP 1121 H= 280:15 +CPU= 18.360
10:22:25 STEP 1122 H= 280:30 +CPU= 18.601
10:22:45 STEP 1123 H= 280:45 +CPU= 19.666
10:23:04 STEP 1124 H= 281:00 +CPU= 19.600
10:23:23 STEP 1125 H= 281:15 +CPU= 18.875
10:23:42 STEP 1126 H= 281:30 +CPU= 19.030
10:24:02 STEP 1127 H= 281:45 +CPU= 19.035
10:24:27 STEP 1128 H= 282:00 +CPU= 24.976
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001128
10:24:46 STEP 1129 H= 282:15 +CPU= 18.978
10:25:04 STEP 1130 H= 282:30 +CPU= 18.750
10:25:24 STEP 1131 H= 282:45 +CPU= 19.240
10:25:43 STEP 1132 H= 283:00 +CPU= 19.319
10:26:02 STEP 1133 H= 283:15 +CPU= 18.778
10:26:21 STEP 1134 H= 283:30 +CPU= 19.146
10:26:40 STEP 1135 H= 283:45 +CPU= 19.196
10:27:00 STEP 1136 H= 284:00 +CPU= 20.147
10:27:19 STEP 1137 H= 284:15 +CPU= 18.658
10:27:38 STEP 1138 H= 284:30 +CPU= 18.961
10:27:57 STEP 1139 H= 284:45 +CPU= 19.338
10:28:23 STEP 1140 H= 285:00 +CPU= 25.168
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001140
10:28:42 STEP 1141 H= 285:15 +CPU= 19.038
10:29:00 STEP 1142 H= 285:30 +CPU= 18.615
10:29:19 STEP 1143 H= 285:45 +CPU= 18.860
10:29:40 STEP 1144 H= 286:00 +CPU= 20.264
10:29:59 STEP 1145 H= 286:15 +CPU= 18.994
10:30:17 STEP 1146 H= 286:30 +CPU= 18.639
10:30:36 STEP 1147 H= 286:45 +CPU= 18.931
10:30:56 STEP 1148 H= 287:00 +CPU= 19.865
10:31:15 STEP 1149 H= 287:15 +CPU= 18.845
10:31:34 STEP 1150 H= 287:30 +CPU= 19.135
10:31:54 STEP 1151 H= 287:45 +CPU= 19.448
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2zf
(argv4) batchid: 1018
(argv5) wuid: 12292194
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/2
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_b4fab48d0af4fac93eb28590e3937875 to: /var/lib/boinc/slots/2/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194.zip
ic_ancil_file: ic_ancil_12292194
ifsdata_file: ifsdata_12292194
climate_data_file: clim_data_12292194
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/2/ic_ancil_12292194.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/2/ic_ancil_12292194.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f62e56b85a35fac0226008a6de9ff649 to: /var/lib/boinc/slots/2/ifsdata/ifsdata_12292194.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/2/ifsdata/ifsdata_12292194.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/2/159l_2/clim_data_12292194.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/2/159l_2/clim_data_12292194.zip
Checking for progress XML file: /var/lib/boinc/slots/2/progress_file_12292194.xml
Opened progress file ok : /var/lib/boinc/slots/2/progress_file_12292194.xml
-- Model is restarting --
Adjusting last_iter, 1152, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/2/progress_file_12292194.xml
last_cpu_time: 22251
upload_file_number: 11
last_iter: 1153
last_upload: 951300
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2zf_2016092300_20_1018_12292194_0_r880167314
The child process has been launched with process id: 593797
Executing the command: /var/lib/boinc/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Happywetter11:1:1:593797:593797] [20240614:103309:1718353989:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292194/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zf_2016092300_20_1018_12292194_0_r880167314_11.zip
Uploading the intermediate file: upload_file_11.zip
10:33:31 STEP 1152 H= 288:00 +CPU= 19.916
10:33:51 STEP 1153 H= 288:15 +CPU= 19.024
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001152
10:34:09 STEP 1154 H= 288:30 +CPU= 18.936
10:34:28 STEP 1155 H= 288:45 +CPU= 18.804
10:34:48 STEP 1156 H= 289:00 +CPU= 19.705
10:35:07 STEP 1157 H= 289:15 +CPU= 19.056
10:35:27 STEP 1158 H= 289:30 +CPU= 19.449
10:35:46 STEP 1159 H= 289:45 +CPU= 18.977
10:36:05 STEP 1160 H= 290:00 +CPU= 19.673
10:36:24 STEP 1161 H= 290:15 +CPU= 18.939
10:36:44 STEP 1162 H= 290:30 +CPU= 19.305
10:37:02 STEP 1163 H= 290:45 +CPU= 18.308
10:37:27 STEP 1164 H= 291:00 +CPU= 24.516
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001164
10:37:45 STEP 1165 H= 291:15 +CPU= 18.545
10:38:04 STEP 1166 H= 291:30 +CPU= 18.902
10:38:23 STEP 1167 H= 291:45 +CPU= 18.905
10:38:43 STEP 1168 H= 292:00 +CPU= 19.721
10:39:02 STEP 1169 H= 292:15 +CPU= 18.881
10:39:21 STEP 1170 H= 292:30 +CPU= 18.722
10:39:39 STEP 1171 H= 292:45 +CPU= 18.546
10:39:59 STEP 1172 H= 293:00 +CPU= 19.791
10:40:18 STEP 1173 H= 293:15 +CPU= 18.764
10:40:37 STEP 1174 H= 293:30 +CPU= 18.721
10:40:55 STEP 1175 H= 293:45 +CPU= 18.845
10:41:20 STEP 1176 H= 294:00 +CPU= 24.654
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001176
10:41:39 STEP 1177 H= 294:15 +CPU= 18.991
10:41:59 STEP 1178 H= 294:30 +CPU= 19.313
10:42:18 STEP 1179 H= 294:45 +CPU= 19.220
10:42:38 STEP 1180 H= 295:00 +CPU= 19.882
10:42:56 STEP 1181 H= 295:15 +CPU= 18.751
10:43:15 STEP 1182 H= 295:30 +CPU= 18.843
10:43:34 STEP 1183 H= 295:45 +CPU= 19.018
10:43:54 STEP 1184 H= 296:00 +CPU= 19.889
10:44:13 STEP 1185 H= 296:15 +CPU= 18.495
10:44:32 STEP 1186 H= 296:30 +CPU= 18.901
10:44:51 STEP 1187 H= 296:45 +CPU= 18.953
10:45:16 STEP 1188 H= 297:00 +CPU= 24.719
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001188
10:45:34 STEP 1189 H= 297:15 +CPU= 18.596
10:45:53 STEP 1190 H= 297:30 +CPU= 18.789
10:46:12 STEP 1191 H= 297:45 +CPU= 18.902
10:46:31 STEP 1192 H= 298:00 +CPU= 19.388
10:46:50 STEP 1193 H= 298:15 +CPU= 18.760
10:47:09 STEP 1194 H= 298:30 +CPU= 18.892
10:47:28 STEP 1195 H= 298:45 +CPU= 19.071
10:47:48 STEP 1196 H= 299:00 +CPU= 19.472
10:48:06 STEP 1197 H= 299:15 +CPU= 18.531
10:48:25 STEP 1198 H= 299:30 +CPU= 18.694
10:48:44 STEP 1199 H= 299:45 +CPU= 18.873
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01
810 9.999999999999998E-005
NORMS AT NSTEP CNT4 1200
SPECTRAL NORMS - LOG(PREHYDS) 0.115261303199841E+02
LEV VORTICITY DIVERGENCE TEMPERATURE KINETIC ENERGY
AVE 0.107097432259928E-04 0.215245679921585E-05 0.202864884089001E+03 0.534930933837795E+02
GPNORM HUMIDITY AVERAGE MINIMUM MAXIMUM
AVE 0.951287424410026E-03 0.999999999999999E-08 0.158894975740778E-01
GPNORM LIQUID WATER AVERAGE MINIMUM MAXIMUM
AVE 0.801589982703956E-06 0.000000000000000E+00 0.683556104009700E-03
GPNORM ICE WATER AVERAGE MINIMUM MAXIMUM
AVE 0.162476975806725E-06 0.000000000000000E+00 0.110330312809653E-03
GPNORM SNOW AVERAGE MINIMUM MAXIMUM
AVE 0.225691659351560E-06 0.000000000000000E+00 0.191827373970299E-03
GPNORM RAIN AVERAGE MINIMUM MAXIMUM
AVE 0.291227062094363E-07 0.000000000000000E+00 0.965376218533307E-04
GPNORM CLOUD FRACTION AVERAGE MINIMUM MAXIMUM
AVE 0.124209672156206E+00 0.000000000000000E+00 0.100000000000000E+01
GPNORM CLW AVERAGE MINIMUM MAXIMUM
AVE 0.804024911845162E-06 0.000000000000000E+00 0.683502794855748E-03
GPNORM CIW AVERAGE MINIMUM MAXIMUM
AVE 0.162311550557835E-06 0.000000000000000E+00 0.107439845002224E-03
GPNORM CC AVERAGE MINIMUM MAXIMUM
AVE 0.124176569446076E+00 0.000000000000000E+00 0.100000000000000E+01
GPNORM RFL AVERAGE MINIMUM MAXIMUM
AVE 0.133184942912574E-06 0.000000000000000E+00 0.452580050174546E-03
GPNORM SFL AVERAGE MINIMUM MAXIMUM
AVE 0.282946364479928E-06 0.000000000000000E+00 0.239528882051131E-03
GPNORM CRFL AVERAGE MINIMUM MAXIMUM
AVE 0.139583930493029E-06 0.000000000000000E+00 0.127219776092346E-02
GPNORM CSFL AVERAGE MINIMUM MAXIMUM
AVE 0.134249997658918E-07 0.000000000000000E+00 0.354741883185118E-03
GPNORM PFRC AVERAGE MINIMUM MAXIMUM
AVE 0.487179082271310E-01 0.000000000000000E+00 0.100000000000000E+01
NSTEP = 1200 STEPO F00000000
CREATE RESTART FILE ./srf00121200.0001
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./srf00121200.0001 MODE=w
IO-STREAM CLOSED - ./srf00121200.0001
MPI-TASK: 1 - ************ BYTES IN 46 RECORDS TRANSFERRED IN 1.8345 SECONDS******** Mbytes/s, TOTAL TIME= 2.0634(88.9%)
CREATE RESTART CONTROL FILE ./rcf
WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00120000.0001
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1200 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001200 MODE=w
IO-STREAM CLOSED - ICMGGh7zg+001200
MPI-TASK: 1 - 299711604 BYTES IN 6 RECORDS TRANSFERRED IN 0.0005 SECONDS599423.2 Mbytes/s, TOTAL TIME= 0.0096( 5.2%)
NSTEP = 1200 STEPO 0AA000000
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMSHh7zg+001200 MODE=w
IO-STREAM CLOSED - ./ICMSHh7zg+001200
MPI-TASK: 1 - 707399250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0082 SECONDS 86268.2 Mbytes/s, TOTAL TIME= 0.2223( 3.7%)
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001200 MODE=w
IO-STREAM CLOSED - ./ICMUAh7zg+001200
MPI-TASK: 1 - 4795385664 BYTES IN 125 RECORDS TRANSFERRED IN 0.0053 SECONDS904789.8 Mbytes/s, TOTAL TIME= 0.1753( 3.0%)
STEPO_FPOS WAS : B00000F00E
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001200 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001200
MPI-TASK: 1 - 10 BYTES IN 2 RECORDS TRANSFERRED IN 0.0002 SECONDS 0.0 Mbytes/s, TOTAL TIME= 0.0034( 5.9%)
STEPO_FPOS WAS : V00000000Y
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001200 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001200
MPI-TASK: 1 - 291131875 BYTES IN 4 RECORDS TRANSFERRED IN 0.0003 SECONDS970439.5 Mbytes/s, TOTAL TIME= 0.0054( 5.6%)
STEPO_FPOS WAS : T00000000M
NSTEP = 1200 STEPO 0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat
10:48:25 0AAA00AAA STEPO 1199 18.694 18.694 18.730 15:14 15:16 0.21452618840136E-05 2GB 0MB
10:48:44 F00000000 STEPO 1200 19.020 19.020 19.066 15:33 15:35 0.21524567992158E-05 2GB 0MB
10:48:46 0AA000000 STEPO 1200 2.183 2.183 2.211 15:35 15:37 0.21524567992158E-05 2GB 0MB
10:48:47 FULLPOS-B DYNFPOS 1200 0.936 0.936 0.938 15:36 15:38 0.21524567992158E-05 2GB 0MB
10:48:50 FULLPOS-V DYNFPOS 1200 2.762 2.762 2.770 15:39 15:41 0.21524567992158E-05 2GB 0MB
10:48:51 FULLPOS-T DYNFPOS 1200 0.653 0.653 0.654 15:39 15:41 0.21524567992158E-05 2GB 0MB
10:48:51 0AAA00AAA STEPO 1200 0.636 0.636 0.637 15:40 15:42 0.21524567992158E-05 2GB 0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf
CSTEP = " 1200",
CTIME = "00121200 ",
NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 1188, 2*0,
IPRGPNSRES = 1,
IPRGPEWRES = 1,
IPRTRWRES = 1,
IPRTRVRES = 1,
GMASS0 = 0.000000000000000E+000,
GMASSI = 0.000000000000000E+000
/
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc/slots/2/progress_file_12292194.xml
<running_values>
<last_cpu_time>23198.250000</last_cpu_time>
<upload_file_number>12</upload_file_number>
<last_iter>1200</last_iter>
<last_upload>1037700</last_upload>
<model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
</stderr_txt>
]]>
