climateprediction.net home page
Task 22314210

Task 22314210

Name oifs_43r3_bl_a29h_2016092300_15_991_12209720_2
Workunit 12209720
Created 18 Feb 2023, 18:16:38 UTC
Sent 18 Feb 2023, 18:17:52 UTC
Report deadline 19 Apr 2023, 18:17:52 UTC
Received 19 Feb 2023, 17:35:23 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1514203
Run time 14 hours 49 min 18 sec
CPU time 14 hours 49 min 18 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 2.06 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,074.31 MB
Peak swap size 5,930.79 MB
Peak disk usage 2,031.83 MB
Stderr
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29h_2016092300_15_991_12209720_2_r486837674_8.zip
Uploading the intermediate file: upload_file_8.zip
  14:01:51 STEP  865 H= 216:15 +CPU= 35.088
  14:02:27 STEP  866 H= 216:30 +CPU= 35.325
  14:03:02 STEP  867 H= 216:45 +CPU= 35.050
  14:03:39 STEP  868 H= 217:00 +CPU= 35.914
  14:04:14 STEP  869 H= 217:15 +CPU= 35.030
  14:04:50 STEP  870 H= 217:30 +CPU= 35.121
  14:05:26 STEP  871 H= 217:45 +CPU= 35.113
  14:06:02 STEP  872 H= 218:00 +CPU= 36.314
  14:06:38 STEP  873 H= 218:15 +CPU= 34.861
  14:07:13 STEP  874 H= 218:30 +CPU= 35.411
  14:07:49 STEP  875 H= 218:45 +CPU= 35.198
  14:08:34 STEP  876 H= 219:00 +CPU= 44.247
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000876
  14:09:10 STEP  877 H= 219:15 +CPU= 36.033
  14:09:47 STEP  878 H= 219:30 +CPU= 35.611
  14:10:23 STEP  879 H= 219:45 +CPU= 35.934
  14:11:01 STEP  880 H= 220:00 +CPU= 36.878
  14:11:38 STEP  881 H= 220:15 +CPU= 35.972
  14:12:14 STEP  882 H= 220:30 +CPU= 35.998
  14:12:50 STEP  883 H= 220:45 +CPU= 35.911
  14:13:28 STEP  884 H= 221:00 +CPU= 36.906
  14:14:04 STEP  885 H= 221:15 +CPU= 36.133
  14:14:41 STEP  886 H= 221:30 +CPU= 35.822
  14:15:17 STEP  887 H= 221:45 +CPU= 35.873
  14:16:03 STEP  888 H= 222:00 +CPU= 44.901
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000888
  14:16:40 STEP  889 H= 222:15 +CPU= 36.224
  14:17:17 STEP  890 H= 222:30 +CPU= 36.207
  14:17:53 STEP  891 H= 222:45 +CPU= 36.296
  14:18:31 STEP  892 H= 223:00 +CPU= 36.847
  14:19:07 STEP  893 H= 223:15 +CPU= 36.023
  14:19:44 STEP  894 H= 223:30 +CPU= 36.057
  14:20:20 STEP  895 H= 223:45 +CPU= 36.035
  14:20:58 STEP  896 H= 224:00 +CPU= 36.868
  14:21:35 STEP  897 H= 224:15 +CPU= 36.011
  14:22:12 STEP  898 H= 224:30 +CPU= 36.333
  14:22:48 STEP  899 H= 224:45 +CPU= 36.154
  14:23:34 STEP  900 H= 225:00 +CPU= 44.891
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000900
  14:24:11 STEP  901 H= 225:15 +CPU= 36.122
  14:24:48 STEP  902 H= 225:30 +CPU= 36.121
  14:25:25 STEP  903 H= 225:45 +CPU= 36.061
  14:26:03 STEP  904 H= 226:00 +CPU= 36.933
  14:26:40 STEP  905 H= 226:15 +CPU= 36.097
  14:27:17 STEP  906 H= 226:30 +CPU= 36.521
  14:27:54 STEP  907 H= 226:45 +CPU= 36.364
  14:28:31 STEP  908 H= 227:00 +CPU= 36.972
  14:29:08 STEP  909 H= 227:15 +CPU= 36.248
  14:29:45 STEP  910 H= 227:30 +CPU= 36.154
  14:30:21 STEP  911 H= 227:45 +CPU= 35.937
  14:31:11 STEP  912 H= 228:00 +CPU= 49.462
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000912
  14:31:47 STEP  913 H= 228:15 +CPU= 36.014
  14:32:24 STEP  914 H= 228:30 +CPU= 36.090
  14:33:01 STEP  915 H= 228:45 +CPU= 36.036
  14:33:38 STEP  916 H= 229:00 +CPU= 37.116
  14:34:15 STEP  917 H= 229:15 +CPU= 36.032
  14:34:52 STEP  918 H= 229:30 +CPU= 36.090
  14:35:29 STEP  919 H= 229:45 +CPU= 36.041
  14:36:06 STEP  920 H= 230:00 +CPU= 37.017
  14:36:44 STEP  921 H= 230:15 +CPU= 36.391
  14:37:20 STEP  922 H= 230:30 +CPU= 36.052
  14:37:58 STEP  923 H= 230:45 +CPU= 36.114
  14:38:44 STEP  924 H= 231:00 +CPU= 45.002
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000924
  14:39:20 STEP  925 H= 231:15 +CPU= 36.328
  14:39:57 STEP  926 H= 231:30 +CPU= 36.059
  14:40:34 STEP  927 H= 231:45 +CPU= 36.124
  14:41:11 STEP  928 H= 232:00 +CPU= 37.272
  14:41:48 STEP  929 H= 232:15 +CPU= 36.104
  14:42:25 STEP  930 H= 232:30 +CPU= 36.295
  14:43:02 STEP  931 H= 232:45 +CPU= 36.226
  14:43:40 STEP  932 H= 233:00 +CPU= 37.172
  14:44:16 STEP  933 H= 233:15 +CPU= 36.097
  14:44:53 STEP  934 H= 233:30 +CPU= 36.144
  14:45:30 STEP  935 H= 233:45 +CPU= 36.090
  14:46:16 STEP  936 H= 234:00 +CPU= 45.227
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000936
  14:46:52 STEP  937 H= 234:15 +CPU= 36.168
  14:47:29 STEP  938 H= 234:30 +CPU= 36.182
  14:48:05 STEP  939 H= 234:45 +CPU= 36.051
  14:48:43 STEP  940 H= 235:00 +CPU= 36.943
  14:49:19 STEP  941 H= 235:15 +CPU= 36.091
  14:49:56 STEP  942 H= 235:30 +CPU= 36.099
  14:50:33 STEP  943 H= 235:45 +CPU= 36.147
  14:51:11 STEP  944 H= 236:00 +CPU= 37.028
  14:51:47 STEP  945 H= 236:15 +CPU= 36.088
  14:52:25 STEP  946 H= 236:30 +CPU= 36.250
  14:53:01 STEP  947 H= 236:45 +CPU= 36.161
  14:53:47 STEP  948 H= 237:00 +CPU= 45.124
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000948
  14:54:23 STEP  949 H= 237:15 +CPU= 36.388
  14:55:01 STEP  950 H= 237:30 +CPU= 36.333
  14:55:38 STEP  951 H= 237:45 +CPU= 36.230
  14:56:15 STEP  952 H= 238:00 +CPU= 36.981
  14:56:52 STEP  953 H= 238:15 +CPU= 36.124
  14:57:28 STEP  954 H= 238:30 +CPU= 36.078
  14:58:05 STEP  955 H= 238:45 +CPU= 36.242
  14:58:42 STEP  956 H= 239:00 +CPU= 36.909
  14:59:19 STEP  957 H= 239:15 +CPU= 36.249
  14:59:56 STEP  958 H= 239:30 +CPU= 36.185
  15:00:33 STEP  959 H= 239:45 +CPU= 36.021
  15:01:23 STEP  960 H= 240:00 +CPU= 49.529
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29h_2016092300_15_991_12209720_2_r486837674_9.zip
Uploading the intermediate file: upload_file_9.zip
  15:02:01 STEP  961 H= 240:15 +CPU= 36.319
  15:02:37 STEP  962 H= 240:30 +CPU= 36.154
  15:03:14 STEP  963 H= 240:45 +CPU= 36.097
  15:03:52 STEP  964 H= 241:00 +CPU= 37.052
  15:04:29 STEP  965 H= 241:15 +CPU= 36.087
  15:05:06 STEP  966 H= 241:30 +CPU= 36.459
  15:05:43 STEP  967 H= 241:45 +CPU= 36.087
  15:06:20 STEP  968 H= 242:00 +CPU= 37.127
  15:06:57 STEP  969 H= 242:15 +CPU= 36.171
  15:07:34 STEP  970 H= 242:30 +CPU= 36.087
  15:08:10 STEP  971 H= 242:45 +CPU= 36.137
  15:08:57 STEP  972 H= 243:00 +CPU= 45.151
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000972
  15:09:34 STEP  973 H= 243:15 +CPU= 36.284
  15:10:11 STEP  974 H= 243:30 +CPU= 36.381
  15:10:47 STEP  975 H= 243:45 +CPU= 35.982
  15:11:25 STEP  976 H= 244:00 +CPU= 36.955
  15:12:02 STEP  977 H= 244:15 +CPU= 36.678
  15:12:38 STEP  978 H= 244:30 +CPU= 36.196
  15:13:15 STEP  979 H= 244:45 +CPU= 36.180
  15:13:52 STEP  980 H= 245:00 +CPU= 36.934
  15:14:29 STEP  981 H= 245:15 +CPU= 36.183
  15:15:07 STEP  982 H= 245:30 +CPU= 36.319
  15:15:44 STEP  983 H= 245:45 +CPU= 36.274
  15:16:31 STEP  984 H= 246:00 +CPU= 45.171
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000984
  15:17:07 STEP  985 H= 246:15 +CPU= 36.311
  15:17:44 STEP  986 H= 246:30 +CPU= 36.024
  15:18:21 STEP  987 H= 246:45 +CPU= 36.150
  15:18:58 STEP  988 H= 247:00 +CPU= 36.998
  15:19:35 STEP  989 H= 247:15 +CPU= 36.204
  15:20:12 STEP  990 H= 247:30 +CPU= 36.142
  15:20:49 STEP  991 H= 247:45 +CPU= 36.283
  15:21:27 STEP  992 H= 248:00 +CPU= 37.094
  15:22:04 STEP  993 H= 248:15 +CPU= 36.266
  15:22:41 STEP  994 H= 248:30 +CPU= 36.313
  15:23:18 STEP  995 H= 248:45 +CPU= 36.098
  15:24:03 STEP  996 H= 249:00 +CPU= 44.849
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000996
  15:24:40 STEP  997 H= 249:15 +CPU= 36.157
  15:25:17 STEP  998 H= 249:30 +CPU= 36.192
  15:25:54 STEP  999 H= 249:45 +CPU= 36.223
  15:26:32 STEP 1000 H= 250:00 +CPU= 37.025
  15:27:09 STEP 1001 H= 250:15 +CPU= 36.343
  15:27:46 STEP 1002 H= 250:30 +CPU= 36.067
  15:28:22 STEP 1003 H= 250:45 +CPU= 36.096
  15:29:00 STEP 1004 H= 251:00 +CPU= 37.100
  15:29:37 STEP 1005 H= 251:15 +CPU= 36.222
  15:30:13 STEP 1006 H= 251:30 +CPU= 36.091
  15:30:50 STEP 1007 H= 251:45 +CPU= 36.177
  15:31:41 STEP 1008 H= 252:00 +CPU= 49.590
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001008
  15:32:18 STEP 1009 H= 252:15 +CPU= 36.398
  15:32:54 STEP 1010 H= 252:30 +CPU= 36.307
  15:33:31 STEP 1011 H= 252:45 +CPU= 36.232
  15:34:08 STEP 1012 H= 253:00 +CPU= 37.034
  15:34:45 STEP 1013 H= 253:15 +CPU= 36.051
  15:35:22 STEP 1014 H= 253:30 +CPU= 36.008
  15:35:58 STEP 1015 H= 253:45 +CPU= 36.123
  15:36:36 STEP 1016 H= 254:00 +CPU= 37.082
  15:37:13 STEP 1017 H= 254:15 +CPU= 36.087
  15:37:50 STEP 1018 H= 254:30 +CPU= 36.230
  15:38:26 STEP 1019 H= 254:45 +CPU= 36.071
  15:39:12 STEP 1020 H= 255:00 +CPU= 45.209
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001020
  15:39:49 STEP 1021 H= 255:15 +CPU= 36.232
  15:40:26 STEP 1022 H= 255:30 +CPU= 36.257
  15:41:03 STEP 1023 H= 255:45 +CPU= 36.278
  15:41:41 STEP 1024 H= 256:00 +CPU= 37.071
  15:42:18 STEP 1025 H= 256:15 +CPU= 36.211
  15:42:55 STEP 1026 H= 256:30 +CPU= 36.208
  15:43:32 STEP 1027 H= 256:45 +CPU= 36.346
  15:44:11 STEP 1028 H= 257:00 +CPU= 37.197
  15:44:48 STEP 1029 H= 257:15 +CPU= 36.300
  15:45:24 STEP 1030 H= 257:30 +CPU= 36.153
  15:46:01 STEP 1031 H= 257:45 +CPU= 36.288
  15:46:48 STEP 1032 H= 258:00 +CPU= 45.353
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001032
  15:47:25 STEP 1033 H= 258:15 +CPU= 36.294
  15:48:01 STEP 1034 H= 258:30 +CPU= 36.137
  15:48:38 STEP 1035 H= 258:45 +CPU= 36.172
  15:49:16 STEP 1036 H= 259:00 +CPU= 36.984
  15:49:52 STEP 1037 H= 259:15 +CPU= 36.083
  15:50:29 STEP 1038 H= 259:30 +CPU= 36.308
  15:51:05 STEP 1039 H= 259:45 +CPU= 36.160
  15:51:43 STEP 1040 H= 260:00 +CPU= 37.160
  15:52:20 STEP 1041 H= 260:15 +CPU= 36.331
  15:52:57 STEP 1042 H= 260:30 +CPU= 36.152
  15:53:34 STEP 1043 H= 260:45 +CPU= 36.418
  15:54:20 STEP 1044 H= 261:00 +CPU= 45.143
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001044
  15:54:57 STEP 1045 H= 261:15 +CPU= 36.285
  15:55:34 STEP 1046 H= 261:30 +CPU= 36.254
  15:56:10 STEP 1047 H= 261:45 +CPU= 36.119
  15:56:48 STEP 1048 H= 262:00 +CPU= 37.102
  15:57:25 STEP 1049 H= 262:15 +CPU= 36.202
  15:58:02 STEP 1050 H= 262:30 +CPU= 36.191
  15:58:38 STEP 1051 H= 262:45 +CPU= 36.222
  15:59:16 STEP 1052 H= 263:00 +CPU= 37.110
  15:59:52 STEP 1053 H= 263:15 +CPU= 36.200
  16:00:30 STEP 1054 H= 263:30 +CPU= 36.283
  16:01:07 STEP 1055 H= 263:45 +CPU= 36.189
  16:01:57 STEP 1056 H= 264:00 +CPU= 49.481
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29h_2016092300_15_991_12209720_2_r486837674_10.zip
Uploading the intermediate file: upload_file_10.zip
  16:02:34 STEP 1057 H= 264:15 +CPU= 36.084
  16:03:11 STEP 1058 H= 264:30 +CPU= 36.325
  16:03:47 STEP 1059 H= 264:45 +CPU= 36.195
  16:04:24 STEP 1060 H= 265:00 +CPU= 36.837
  16:05:02 STEP 1061 H= 265:15 +CPU= 36.361
  16:05:38 STEP 1062 H= 265:30 +CPU= 36.175
  16:06:15 STEP 1063 H= 265:45 +CPU= 36.257
  16:06:52 STEP 1064 H= 266:00 +CPU= 37.161
  16:07:29 STEP 1065 H= 266:15 +CPU= 36.346
  16:08:06 STEP 1066 H= 266:30 +CPU= 36.202
  16:08:43 STEP 1067 H= 266:45 +CPU= 36.265
  16:09:29 STEP 1068 H= 267:00 +CPU= 45.116
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001068
  16:10:06 STEP 1069 H= 267:15 +CPU= 36.141
  16:10:42 STEP 1070 H= 267:30 +CPU= 36.214
  16:11:19 STEP 1071 H= 267:45 +CPU= 36.137
  16:11:57 STEP 1072 H= 268:00 +CPU= 36.992
  16:12:33 STEP 1073 H= 268:15 +CPU= 36.112
  16:13:10 STEP 1074 H= 268:30 +CPU= 36.225
  16:13:47 STEP 1075 H= 268:45 +CPU= 36.600
  16:14:25 STEP 1076 H= 269:00 +CPU= 37.247
  16:15:02 STEP 1077 H= 269:15 +CPU= 36.299
  16:15:39 STEP 1078 H= 269:30 +CPU= 36.196
  16:16:16 STEP 1079 H= 269:45 +CPU= 36.381
  16:17:02 STEP 1080 H= 270:00 +CPU= 45.258
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001080
  16:17:39 STEP 1081 H= 270:15 +CPU= 36.263
  16:18:16 STEP 1082 H= 270:30 +CPU= 36.210
  16:18:53 STEP 1083 H= 270:45 +CPU= 36.181
  16:19:31 STEP 1084 H= 271:00 +CPU= 37.134
  16:20:08 STEP 1085 H= 271:15 +CPU= 36.284
  16:20:45 STEP 1086 H= 271:30 +CPU= 36.177
  16:21:22 STEP 1087 H= 271:45 +CPU= 36.148
  16:22:00 STEP 1088 H= 272:00 +CPU= 37.021
  16:22:37 STEP 1089 H= 272:15 +CPU= 36.216
  16:23:14 STEP 1090 H= 272:30 +CPU= 36.257
  16:23:50 STEP 1091 H= 272:45 +CPU= 36.181
  16:24:36 STEP 1092 H= 273:00 +CPU= 44.998
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001092
  16:25:13 STEP 1093 H= 273:15 +CPU= 36.156
  16:25:49 STEP 1094 H= 273:30 +CPU= 36.052
  16:26:26 STEP 1095 H= 273:45 +CPU= 36.239
  16:27:04 STEP 1096 H= 274:00 +CPU= 37.097
  16:27:41 STEP 1097 H= 274:15 +CPU= 36.261
  16:28:18 STEP 1098 H= 274:30 +CPU= 36.510
  16:28:55 STEP 1099 H= 274:45 +CPU= 36.375
  16:29:33 STEP 1100 H= 275:00 +CPU= 37.003
  16:30:09 STEP 1101 H= 275:15 +CPU= 36.220
  16:30:47 STEP 1102 H= 275:30 +CPU= 36.475
  16:31:23 STEP 1103 H= 275:45 +CPU= 36.272
  16:32:13 STEP 1104 H= 276:00 +CPU= 49.699
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001104
  16:32:50 STEP 1105 H= 276:15 +CPU= 36.168
  16:33:27 STEP 1106 H= 276:30 +CPU= 36.381
  16:34:04 STEP 1107 H= 276:45 +CPU= 36.154
  16:34:42 STEP 1108 H= 277:00 +CPU= 37.197
  16:35:19 STEP 1109 H= 277:15 +CPU= 36.082
  16:35:55 STEP 1110 H= 277:30 +CPU= 36.171
  16:36:33 STEP 1111 H= 277:45 +CPU= 36.150
  16:37:10 STEP 1112 H= 278:00 +CPU= 36.951
  16:37:48 STEP 1113 H= 278:15 +CPU= 36.405
  16:38:25 STEP 1114 H= 278:30 +CPU= 36.207
  16:39:01 STEP 1115 H= 278:45 +CPU= 36.232
  16:39:47 STEP 1116 H= 279:00 +CPU= 44.939
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001116
  16:40:24 STEP 1117 H= 279:15 +CPU= 36.318
  16:41:00 STEP 1118 H= 279:30 +CPU= 36.119
  16:41:38 STEP 1119 H= 279:45 +CPU= 36.447
  16:42:15 STEP 1120 H= 280:00 +CPU= 37.012
  16:42:52 STEP 1121 H= 280:15 +CPU= 36.205
  16:43:29 STEP 1122 H= 280:30 +CPU= 36.172
  16:44:05 STEP 1123 H= 280:45 +CPU= 35.562
  16:44:42 STEP 1124 H= 281:00 +CPU= 36.989
  16:45:19 STEP 1125 H= 281:15 +CPU= 36.112
  16:45:56 STEP 1126 H= 281:30 +CPU= 36.185
  16:46:33 STEP 1127 H= 281:45 +CPU= 36.264
  16:47:18 STEP 1128 H= 282:00 +CPU= 44.901
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001128
  16:47:55 STEP 1129 H= 282:15 +CPU= 36.136
  16:48:32 STEP 1130 H= 282:30 +CPU= 36.307
  16:49:09 STEP 1131 H= 282:45 +CPU= 36.102
  16:49:46 STEP 1132 H= 283:00 +CPU= 37.114
  16:50:23 STEP 1133 H= 283:15 +CPU= 36.511
  16:51:00 STEP 1134 H= 283:30 +CPU= 36.205
  16:51:37 STEP 1135 H= 283:45 +CPU= 36.157
  16:52:15 STEP 1136 H= 284:00 +CPU= 37.027
  16:52:52 STEP 1137 H= 284:15 +CPU= 36.211
  16:53:29 STEP 1138 H= 284:30 +CPU= 36.046
  16:54:05 STEP 1139 H= 284:45 +CPU= 35.977
  16:54:51 STEP 1140 H= 285:00 +CPU= 44.794
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001140
  16:55:27 STEP 1141 H= 285:15 +CPU= 36.193
  16:56:04 STEP 1142 H= 285:30 +CPU= 36.298
  16:56:40 STEP 1143 H= 285:45 +CPU= 36.000
  16:57:18 STEP 1144 H= 286:00 +CPU= 36.894
  16:57:55 STEP 1145 H= 286:15 +CPU= 36.114
  16:58:32 STEP 1146 H= 286:30 +CPU= 36.208
  16:59:09 STEP 1147 H= 286:45 +CPU= 36.234
  16:59:46 STEP 1148 H= 287:00 +CPU= 36.986
  17:00:23 STEP 1149 H= 287:15 +CPU= 36.176
  17:01:00 STEP 1150 H= 287:30 +CPU= 36.283
  17:01:37 STEP 1151 H= 287:45 +CPU= 36.302
  17:02:27 STEP 1152 H= 288:00 +CPU= 49.333
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29h_2016092300_15_991_12209720_2_r486837674_11.zip
Uploading the intermediate file: upload_file_11.zip
  17:03:04 STEP 1153 H= 288:15 +CPU= 36.018
  17:03:40 STEP 1154 H= 288:30 +CPU= 36.031
  17:04:17 STEP 1155 H= 288:45 +CPU= 36.148
  17:04:55 STEP 1156 H= 289:00 +CPU= 36.919
  17:05:31 STEP 1157 H= 289:15 +CPU= 36.015
  17:06:08 STEP 1158 H= 289:30 +CPU= 35.964
  17:06:44 STEP 1159 H= 289:45 +CPU= 35.075
  17:07:21 STEP 1160 H= 290:00 +CPU= 36.766
  17:07:57 STEP 1161 H= 290:15 +CPU= 35.685
  17:08:33 STEP 1162 H= 290:30 +CPU= 35.920
  17:09:11 STEP 1163 H= 290:45 +CPU= 36.158
  17:09:56 STEP 1164 H= 291:00 +CPU= 44.997
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001164
  17:10:32 STEP 1165 H= 291:15 +CPU= 35.548
  17:11:08 STEP 1166 H= 291:30 +CPU= 34.954
  17:11:44 STEP 1167 H= 291:45 +CPU= 35.888
  17:12:21 STEP 1168 H= 292:00 +CPU= 37.112
  17:12:58 STEP 1169 H= 292:15 +CPU= 36.300
  17:13:35 STEP 1170 H= 292:30 +CPU= 36.238
  17:14:11 STEP 1171 H= 292:45 +CPU= 35.980
  17:14:49 STEP 1172 H= 293:00 +CPU= 36.930
  17:15:25 STEP 1173 H= 293:15 +CPU= 36.016
  17:16:02 STEP 1174 H= 293:30 +CPU= 36.032
  17:16:38 STEP 1175 H= 293:45 +CPU= 36.174
  17:17:24 STEP 1176 H= 294:00 +CPU= 45.290
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001176
  17:18:01 STEP 1177 H= 294:15 +CPU= 36.297
  17:18:39 STEP 1178 H= 294:30 +CPU= 36.552
  17:19:16 STEP 1179 H= 294:45 +CPU= 36.300
  17:19:54 STEP 1180 H= 295:00 +CPU= 37.196
  17:20:31 STEP 1181 H= 295:15 +CPU= 36.530
  17:21:08 STEP 1182 H= 295:30 +CPU= 36.191
  17:21:45 STEP 1183 H= 295:45 +CPU= 36.120
  17:22:22 STEP 1184 H= 296:00 +CPU= 36.970
  17:22:59 STEP 1185 H= 296:15 +CPU= 36.292
  17:23:36 STEP 1186 H= 296:30 +CPU= 36.237
  17:24:13 STEP 1187 H= 296:45 +CPU= 36.167
  17:24:59 STEP 1188 H= 297:00 +CPU= 45.219
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001188
  17:25:36 STEP 1189 H= 297:15 +CPU= 36.315
  17:26:13 STEP 1190 H= 297:30 +CPU= 36.382
  17:26:49 STEP 1191 H= 297:45 +CPU= 36.402
  17:27:28 STEP 1192 H= 298:00 +CPU= 37.154
  17:28:04 STEP 1193 H= 298:15 +CPU= 36.134
  17:28:41 STEP 1194 H= 298:30 +CPU= 36.318
  17:29:17 STEP 1195 H= 298:45 +CPU= 36.174
  17:29:55 STEP 1196 H= 299:00 +CPU= 37.018
  17:30:31 STEP 1197 H= 299:15 +CPU= 36.092
  17:31:08 STEP 1198 H= 299:30 +CPU= 36.174
  17:31:46 STEP 1199 H= 299:45 +CPU= 36.292
  17:32:37 STEP 1200 H= 300:00 +CPU= 49.658
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001200
  17:33:14 STEP 1201 H= 300:15 +CPU= 36.368
  17:33:50 STEP 1202 H= 300:30 +CPU= 35.979
  17:34:26 STEP 1203 H= 300:45 +CPU= 36.218
  17:35:04 STEP 1204 H= 301:00 +CPU= 36.839
  17:35:41 STEP 1205 H= 301:15 +CPU= 36.103
  17:36:17 STEP 1206 H= 301:30 +CPU= 35.974
  17:36:54 STEP 1207 H= 301:45 +CPU= 36.070
  17:37:32 STEP 1208 H= 302:00 +CPU= 36.894
  17:38:09 STEP 1209 H= 302:15 +CPU= 36.241
  17:38:45 STEP 1210 H= 302:30 +CPU= 35.977
  17:39:22 STEP 1211 H= 302:45 +CPU= 36.114
  17:40:08 STEP 1212 H= 303:00 +CPU= 44.920
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001212
  17:40:45 STEP 1213 H= 303:15 +CPU= 36.318
  17:41:22 STEP 1214 H= 303:30 +CPU= 36.002
  17:41:58 STEP 1215 H= 303:45 +CPU= 35.952
  17:42:36 STEP 1216 H= 304:00 +CPU= 36.994
  17:43:13 STEP 1217 H= 304:15 +CPU= 36.158
  17:43:49 STEP 1218 H= 304:30 +CPU= 36.043
  17:44:25 STEP 1219 H= 304:45 +CPU= 35.881
  17:45:03 STEP 1220 H= 305:00 +CPU= 37.053
  17:45:40 STEP 1221 H= 305:15 +CPU= 36.171
  17:46:16 STEP 1222 H= 305:30 +CPU= 35.961
  17:46:53 STEP 1223 H= 305:45 +CPU= 35.997
  17:47:39 STEP 1224 H= 306:00 +CPU= 45.174
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001224
  17:48:15 STEP 1225 H= 306:15 +CPU= 36.004
  17:48:51 STEP 1226 H= 306:30 +CPU= 35.995
  17:49:28 STEP 1227 H= 306:45 +CPU= 36.052
  17:50:05 STEP 1228 H= 307:00 +CPU= 36.840
  17:50:42 STEP 1229 H= 307:15 +CPU= 36.102
  17:51:19 STEP 1230 H= 307:30 +CPU= 36.142
  17:51:55 STEP 1231 H= 307:45 +CPU= 36.220
  17:52:33 STEP 1232 H= 308:00 +CPU= 36.801
  17:53:10 STEP 1233 H= 308:15 +CPU= 36.125
  17:53:47 STEP 1234 H= 308:30 +CPU= 36.469
  17:54:23 STEP 1235 H= 308:45 +CPU= 36.185
  17:55:09 STEP 1236 H= 309:00 +CPU= 44.979
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001236
  17:55:45 STEP 1237 H= 309:15 +CPU= 36.100
  17:56:22 STEP 1238 H= 309:30 +CPU= 36.007
  17:56:59 STEP 1239 H= 309:45 +CPU= 36.021
  17:57:37 STEP 1240 H= 310:00 +CPU= 36.912
  17:58:14 STEP 1241 H= 310:15 +CPU= 35.964
  17:58:50 STEP 1242 H= 310:30 +CPU= 35.978
  17:59:27 STEP 1243 H= 310:45 +CPU= 35.971
  18:00:04 STEP 1244 H= 311:00 +CPU= 36.964
  18:00:40 STEP 1245 H= 311:15 +CPU= 35.998
  18:01:17 STEP 1246 H= 311:30 +CPU= 36.144
  18:01:53 STEP 1247 H= 311:45 +CPU= 36.038
  18:02:44 STEP 1248 H= 312:00 +CPU= 49.403
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209720/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29h_2016092300_15_991_12209720_2_r486837674_12.zip
Uploading the intermediate file: upload_file_12.zip
  18:03:21 STEP 1249 H= 312:15 +CPU= 35.955
  18:03:58 STEP 1250 H= 312:30 +CPU= 36.296
  18:04:34 STEP 1251 H= 312:45 +CPU= 35.959
  18:05:12 STEP 1252 H= 313:00 +CPU= 36.865
  18:05:49 STEP 1253 H= 313:15 +CPU= 36.088
  18:06:25 STEP 1254 H= 313:30 +CPU= 35.801
  18:07:02 STEP 1255 H= 313:45 +CPU= 36.129
  18:07:39 STEP 1256 H= 314:00 +CPU= 36.723
  18:08:15 STEP 1257 H= 314:15 +CPU= 35.874
  18:08:51 STEP 1258 H= 314:30 +CPU= 35.841
  18:09:27 STEP 1259 H= 314:45 +CPU= 35.755
  18:10:12 STEP 1260 H= 315:00 +CPU= 44.653
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001260
  18:10:49 STEP 1261 H= 315:15 +CPU= 35.888
  18:11:25 STEP 1262 H= 315:30 +CPU= 36.118
  18:12:02 STEP 1263 H= 315:45 +CPU= 35.768
  18:12:39 STEP 1264 H= 316:00 +CPU= 36.686
  18:13:15 STEP 1265 H= 316:15 +CPU= 35.897
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4057MB (heap), 4897MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc4a8f], nsigs = 1
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 4057 MB (maxheap), 4897 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181339:1676826819:18638.095] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' : sigcontextptr=0x7ffe077512d0
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' 0x838803 0x838e09 0x1d8bb20 0x1cc4a8f 0x12d283b 0x12ba8d9 0x12b0762 0x1163fc3 0x10b682f 0x10a915c 0xe680c3 0x15c1e62 0x15ac40e 0x6620b4 0x4a0e7c 0x49dc29 0x49ce72 0x49cd68 0x4078ad 0x40276f 0x40270d 0x1d8287a 0x1d840d7 0x4025e5
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x838803] : LinuxTraceBack() at linuxtrbk.c:264
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x838e09] : LinuxTraceBack() at linuxtrbk.c:236
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1d8bb20] : __restore_rt() at libc_sigaction.o:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc4a8f] : __libm_exp_e7() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x12d283b] : vdfmain_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x12ba8d9] : vdfouter_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x12b0762] : turbulence_layer_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x1163fc3] : callpar_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0x10b682f] : ec_phys_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0x10a915c] : ec_phys_drv_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xe680c3] : gp_model_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x15c1e62] : gp_model_heap_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x15ac40e] : scan2m_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x6620b4] : stepo_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4a0e7c] : cnt4_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x49dc29] : cnt3_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x49ce72] : cnt2_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49cd68] : cnt1_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x4078ad] : cnt0_() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x40276f] : MAIN__() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x40270d] : main() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1d8287a] : __libc_start_call_main() at libc-start.o:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1d840d7] : __libc_start_main_impl() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x4025e5] : _start() at ??:?
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.103] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' : sigcontextptr=0x7ffe077511e0
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' 0x838803 0x838e09 0x820d70 0x1d8bb20 0x1cc4a8f 0x12d283b 0x12ba8d9 0x12b0762 0x1163fc3 0x10b682f 0x10a915c 0xe680c3 0x15c1e62 0x15ac40e 0x6620b4 0x4a0e7c 0x49dc29 0x49ce72 0x49cd68 0x4078ad 0x40276f 0x40270d 0x1d8287a 0x1d840d7 0x4025e5
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x838803] : LinuxTraceBack() at linuxtrbk.c:264
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x838e09] : LinuxTraceBack() at linuxtrbk.c:236
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x820d70] : signal_drhook() at drhook.c:1734
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1d8bb20] : __restore_rt() at libc_sigaction.o:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc4a8f] : __libm_exp_e7() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x12d283b] : vdfmain_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x12ba8d9] : vdfouter_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x12b0762] : turbulence_layer_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0x1163fc3] : callpar_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0x10b682f] : ec_phys_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0x10a915c] : ec_phys_drv_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xe680c3] : gp_model_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x15c1e62] : gp_model_heap_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x15ac40e] : scan2m_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x6620b4] : stepo_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4a0e7c] : cnt4_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x49dc29] : cnt3_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49ce72] : cnt2_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49cd68] : cnt1_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x4078ad] : cnt0_() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x40276f] : MAIN__() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x40270d] : main() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1d8287a] : __libc_start_call_main() at libc-start.o:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1d840d7] : __libc_start_main_impl() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x4025e5] : _start() at ??:?
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.387] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:Azrael:1:1:658479:658479] [20230219:181340:1676826820:18639.387] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC4A8F  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000012D283B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  00000000012BA8D9  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000012B0762  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000001163FC3  callpar_                  869  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.379715294639743E-05 0.000000000000000E+00 0.569225699498548E-02
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.239078405241109E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.256604154403894E-05 0.000000000000000E+00 0.826959432611370E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.811250239573753E-05 0.000000000000000E+00 0.856761527012323E-02
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.239002234237760E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.139822354586270E-04 0.000000000000000E+00 0.202177443892530E-01
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.922126966143626E-05 0.000000000000000E+00 0.631800151544764E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.483125646409411E-05 0.000000000000000E+00 0.892857611584886E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.183773560294204E-05 0.000000000000000E+00 0.664764747376628E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.143419142937916E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =  1260 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1260 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001260 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+001260
 MPI-TASK:   1 -    309503818 BYTES IN      6 RECORDS TRANSFERRED IN    0.0005 SECONDS619007.6 Mbytes/s, TOTAL TIME=    0.0120( 4.2%)
 NSTEP =  1260 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+001260 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+001260
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0124 SECONDS 57077.8 Mbytes/s, TOTAL TIME=    0.3787( 3.3%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001260 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001260
 MPI-TASK:   1 -   4952061079 BYTES IN    125 RECORDS TRANSFERRED IN    0.0079 SECONDS626843.2 Mbytes/s, TOTAL TIME=    0.2862( 2.8%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001260 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001260
 MPI-TASK:   1 -    300930185 BYTES IN      2 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0049( 6.1%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001260 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001260
 MPI-TASK:   1 -    309503818 BYTES IN      4 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0079( 3.8%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =  1260 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1260 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001260 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001260
 MPI-TASK:   1 -    309503809 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS442148.3 Mbytes/s, TOTAL TIME=    0.0130( 5.4%)
 18:10:12 STEP 1260 H= 315:00 +CPU= 44.653
 NSTEP =  1261 STEPO   0AAA00AAA
 18:10:49 STEP 1261 H= 315:15 +CPU= 35.888
 NSTEP =  1262 STEPO   0AAA00AAA
 18:11:25 STEP 1262 H= 315:30 +CPU= 36.118
 NSTEP =  1263 STEPO   0AAA00AAA
 18:12:02 STEP 1263 H= 315:45 +CPU= 35.768
 NSTEP =  1264 STEPO   0AAA00AAA
 18:12:39 STEP 1264 H= 316:00 +CPU= 36.686
ISTEP=    1265 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=    1265 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =  1265 STEPO   0AAA00AAA
 18:13:15 STEP 1265 H= 316:15 +CPU= 35.897
 NSTEP =  1266 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 18:09:35 0AAA00AAA STEPO     1260     0.759    0.759    0.765    301:50    306:34 0.26696297253670E-04       2GB       0MB
 18:10:12 0AAA00AAA STEPO     1261    36.762   36.762   37.052    302:27    307:11 0.26696297253670E-04       2GB       0MB
 18:10:49 0AAA00AAA STEPO     1262    35.888   35.888   36.406    303:03    307:48 0.26696297253670E-04       2GB       0MB
 18:11:25 0AAA00AAA STEPO     1263    36.118   36.118   36.477    303:39    308:24 0.26696297253670E-04       2GB       0MB
 18:12:02 0AAA00AAA STEPO     1264    35.768   35.768   36.180    304:15    309:00 0.26696297253670E-04       2GB       0MB
 18:12:39 0AAA00AAA STEPO     1265    36.686   36.686   37.078    304:51    309:38 0.26696297253670E-04       2GB       0MB
 18:13:15 0AAA00AAA STEPO     1266    35.897   35.897   36.534    305:27    310:14 0.26696297253670E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    1248",
 CTIME   = "00130000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,        1236, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/8/progress_file_12209720.xml

<running_values>
  <last_cpu_time>53350.350000</last_cpu_time>
  <upload_file_number>13</upload_file_number>
  <last_iter>1266</last_iter>
  <last_upload>1123200</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net