climateprediction.net home page
Task 22299108

Task 22299108

Name oifs_43r3_ps_0714_1999050100_123_968_12185358_1
Workunit 12185358
Created 17 Jan 2023, 17:56:41 UTC
Sent 17 Jan 2023, 18:33:19 UTC
Report deadline 18 Mar 2023, 18:33:19 UTC
Received 21 Jan 2023, 15:06:10 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1532028
Run time 12 hours 22 min 48 sec
CPU time 11 hours 15 min 21 sec
Validate state Invalid
Credit 7,252.56
Device peak FLOPS 8.61 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,779.99 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,688.99 MB
Stderr
<core_client_version>7.20.4</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
04 H=1104:00 +CPU= 41.002
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001104
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001104
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001104
  11:05:11 STEP 1105 H=1105:00 +CPU= 20.578
  11:05:32 STEP 1106 H=1106:00 +CPU= 20.874
  11:06:03 STEP 1107 H=1107:00 +CPU= 30.448
  11:06:24 STEP 1108 H=1108:00 +CPU= 21.052
  11:06:44 STEP 1109 H=1109:00 +CPU= 19.498
  11:07:14 STEP 1110 H=1110:00 +CPU= 29.717
  11:07:32 STEP 1111 H=1111:00 +CPU= 18.539
  11:07:53 STEP 1112 H=1112:00 +CPU= 20.188
  11:08:25 STEP 1113 H=1113:00 +CPU= 31.961
  11:08:46 STEP 1114 H=1114:00 +CPU= 20.861
  11:09:07 STEP 1115 H=1115:00 +CPU= 20.786
  11:09:43 STEP 1116 H=1116:00 +CPU= 36.109
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001116
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001116
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001116
  11:10:03 STEP 1117 H=1117:00 +CPU= 19.519
  11:10:23 STEP 1118 H=1118:00 +CPU= 20.533
  11:10:54 STEP 1119 H=1119:00 +CPU= 30.462
  11:11:16 STEP 1120 H=1120:00 +CPU= 21.846
  11:11:36 STEP 1121 H=1121:00 +CPU= 20.046
  11:12:07 STEP 1122 H=1122:00 +CPU= 30.109
  11:12:27 STEP 1123 H=1123:00 +CPU= 20.476
  11:12:48 STEP 1124 H=1124:00 +CPU= 20.410
  11:13:18 STEP 1125 H=1125:00 +CPU= 29.877
  11:13:43 STEP 1126 H=1126:00 +CPU= 24.982
  11:14:04 STEP 1127 H=1127:00 +CPU= 21.550
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001104
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001104
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001104
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001116
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001116
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001116
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_45.zip
Uploading the intermediate file: upload_file_45.zip
  11:14:40 STEP 1128 H=1128:00 +CPU= 35.086
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001128
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001128
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001128
  11:15:00 STEP 1129 H=1129:00 +CPU= 19.904
  11:15:20 STEP 1130 H=1130:00 +CPU= 19.829
  11:15:50 STEP 1131 H=1131:00 +CPU= 29.992
  11:16:10 STEP 1132 H=1132:00 +CPU= 19.760
  11:16:29 STEP 1133 H=1133:00 +CPU= 19.126
  11:17:00 STEP 1134 H=1134:00 +CPU= 30.230
  11:17:21 STEP 1135 H=1135:00 +CPU= 21.234
  11:17:41 STEP 1136 H=1136:00 +CPU= 19.482
  11:18:12 STEP 1137 H=1137:00 +CPU= 30.995
  11:18:32 STEP 1138 H=1138:00 +CPU= 20.426
  11:18:53 STEP 1139 H=1139:00 +CPU= 20.769
  11:19:26 STEP 1140 H=1140:00 +CPU= 32.436
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001140
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001140
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001140
  11:19:46 STEP 1141 H=1141:00 +CPU= 19.846
  11:20:07 STEP 1142 H=1142:00 +CPU= 20.563
  11:20:37 STEP 1143 H=1143:00 +CPU= 30.015
  11:20:59 STEP 1144 H=1144:00 +CPU= 21.940
  11:21:19 STEP 1145 H=1145:00 +CPU= 19.900
  11:21:49 STEP 1146 H=1146:00 +CPU= 30.263
  11:22:10 STEP 1147 H=1147:00 +CPU= 21.069
  11:22:32 STEP 1148 H=1148:00 +CPU= 21.246
  11:23:02 STEP 1149 H=1149:00 +CPU= 29.793
  11:23:23 STEP 1150 H=1150:00 +CPU= 20.593
  11:23:44 STEP 1151 H=1151:00 +CPU= 20.485
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001128
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001128
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001128
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001140
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001140
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001140
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_46.zip
Uploading the intermediate file: upload_file_46.zip
  11:24:20 STEP 1152 H=1152:00 +CPU= 35.783
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001152
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001152
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001152
  11:24:39 STEP 1153 H=1153:00 +CPU= 19.406
  11:25:00 STEP 1154 H=1154:00 +CPU= 20.294
  11:25:31 STEP 1155 H=1155:00 +CPU= 31.354
  11:25:51 STEP 1156 H=1156:00 +CPU= 19.205
  11:26:10 STEP 1157 H=1157:00 +CPU= 19.578
  11:26:42 STEP 1158 H=1158:00 +CPU= 31.352
  11:27:02 STEP 1159 H=1159:00 +CPU= 19.770
  11:27:23 STEP 1160 H=1160:00 +CPU= 21.035
  11:27:54 STEP 1161 H=1161:00 +CPU= 31.376
  11:28:15 STEP 1162 H=1162:00 +CPU= 20.154
  11:28:35 STEP 1163 H=1163:00 +CPU= 19.935
  11:29:09 STEP 1164 H=1164:00 +CPU= 34.089
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001164
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001164
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001164
  11:29:30 STEP 1165 H=1165:00 +CPU= 20.883
  11:29:50 STEP 1166 H=1166:00 +CPU= 19.588
  11:30:20 STEP 1167 H=1167:00 +CPU= 29.685
  11:30:40 STEP 1168 H=1168:00 +CPU= 20.667
  11:31:00 STEP 1169 H=1169:00 +CPU= 19.503
  11:31:32 STEP 1170 H=1170:00 +CPU= 31.527
  11:31:51 STEP 1171 H=1171:00 +CPU= 19.545
  11:32:12 STEP 1172 H=1172:00 +CPU= 20.418
  11:32:42 STEP 1173 H=1173:00 +CPU= 29.953
  11:33:02 STEP 1174 H=1174:00 +CPU= 19.144
  11:33:23 STEP 1175 H=1175:00 +CPU= 21.543
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001152
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001152
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001152
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001164
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001164
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001164
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_47.zip
Uploading the intermediate file: upload_file_47.zip
  11:33:59 STEP 1176 H=1176:00 +CPU= 35.683
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001176
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001176
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001176
  11:34:19 STEP 1177 H=1177:00 +CPU= 19.095
  11:34:40 STEP 1178 H=1178:00 +CPU= 21.269
  11:35:13 STEP 1179 H=1179:00 +CPU= 32.896
  11:35:35 STEP 1180 H=1180:00 +CPU= 21.589
  11:35:56 STEP 1181 H=1181:00 +CPU= 21.083
  11:36:30 STEP 1182 H=1182:00 +CPU= 33.332
  11:36:51 STEP 1183 H=1183:00 +CPU= 21.048
  11:37:13 STEP 1184 H=1184:00 +CPU= 21.893
  11:37:44 STEP 1185 H=1185:00 +CPU= 30.883
  11:38:04 STEP 1186 H=1186:00 +CPU= 20.011
  11:38:26 STEP 1187 H=1187:00 +CPU= 22.220
  11:39:04 STEP 1188 H=1188:00 +CPU= 37.226
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001188
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001188
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001188
  11:39:25 STEP 1189 H=1189:00 +CPU= 21.236
  11:39:47 STEP 1190 H=1190:00 +CPU= 21.629
  11:40:18 STEP 1191 H=1191:00 +CPU= 31.477
  11:40:39 STEP 1192 H=1192:00 +CPU= 20.069
  11:40:59 STEP 1193 H=1193:00 +CPU= 19.993
  11:41:30 STEP 1194 H=1194:00 +CPU= 30.351
  11:41:50 STEP 1195 H=1195:00 +CPU= 20.235
  11:42:11 STEP 1196 H=1196:00 +CPU= 20.350
  11:42:42 STEP 1197 H=1197:00 +CPU= 31.505
  11:43:02 STEP 1198 H=1198:00 +CPU= 19.477
  11:43:26 STEP 1199 H=1199:00 +CPU= 23.573
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001176
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001176
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001176
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001188
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001188
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001188
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_48.zip
Uploading the intermediate file: upload_file_48.zip
Uploading trickle at timestep: 4316400
  11:44:02 STEP 1200 H=1200:00 +CPU= 36.026
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001200
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001200
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001200
  11:44:22 STEP 1201 H=1201:00 +CPU= 19.383
  11:44:43 STEP 1202 H=1202:00 +CPU= 21.007
  11:45:16 STEP 1203 H=1203:00 +CPU= 32.616
  11:45:35 STEP 1204 H=1204:00 +CPU= 19.334
  11:45:56 STEP 1205 H=1205:00 +CPU= 21.210
  11:46:27 STEP 1206 H=1206:00 +CPU= 30.468
  11:46:48 STEP 1207 H=1207:00 +CPU= 20.292
  11:47:10 STEP 1208 H=1208:00 +CPU= 22.310
  11:47:43 STEP 1209 H=1209:00 +CPU= 32.755
  11:48:03 STEP 1210 H=1210:00 +CPU= 19.971
  11:48:25 STEP 1211 H=1211:00 +CPU= 21.811
  11:48:59 STEP 1212 H=1212:00 +CPU= 33.700
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001212
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001212
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001212
  11:49:18 STEP 1213 H=1213:00 +CPU= 18.714
  11:49:39 STEP 1214 H=1214:00 +CPU= 21.157
  11:50:11 STEP 1215 H=1215:00 +CPU= 31.106
  11:50:30 STEP 1216 H=1216:00 +CPU= 19.513
  11:50:51 STEP 1217 H=1217:00 +CPU= 20.424
  11:51:24 STEP 1218 H=1218:00 +CPU= 33.136
  11:51:44 STEP 1219 H=1219:00 +CPU= 19.936
  11:52:06 STEP 1220 H=1220:00 +CPU= 21.752
  11:52:37 STEP 1221 H=1221:00 +CPU= 30.065
  11:52:56 STEP 1222 H=1222:00 +CPU= 19.421
  11:53:18 STEP 1223 H=1223:00 +CPU= 20.994
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001200
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001200
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001200
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001212
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001212
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001212
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_49.zip
Uploading the intermediate file: upload_file_49.zip
  11:53:56 STEP 1224 H=1224:00 +CPU= 38.049
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001224
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001224
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001224
  11:54:16 STEP 1225 H=1225:00 +CPU= 19.420
  11:54:39 STEP 1226 H=1226:00 +CPU= 23.240
  11:55:10 STEP 1227 H=1227:00 +CPU= 31.098
  11:55:31 STEP 1228 H=1228:00 +CPU= 20.312
  11:55:53 STEP 1229 H=1229:00 +CPU= 22.414
  11:56:25 STEP 1230 H=1230:00 +CPU= 31.609
  11:56:45 STEP 1231 H=1231:00 +CPU= 19.793
  11:57:07 STEP 1232 H=1232:00 +CPU= 22.116
  11:57:38 STEP 1233 H=1233:00 +CPU= 30.919
  11:57:57 STEP 1234 H=1234:00 +CPU= 18.950
  11:58:18 STEP 1235 H=1235:00 +CPU= 20.685
  11:58:53 STEP 1236 H=1236:00 +CPU= 34.238
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001236
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001236
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001236
  11:59:12 STEP 1237 H=1237:00 +CPU= 19.019
  11:59:33 STEP 1238 H=1238:00 +CPU= 20.987
  12:00:05 STEP 1239 H=1239:00 +CPU= 31.775
  12:00:26 STEP 1240 H=1240:00 +CPU= 20.268
  12:00:47 STEP 1241 H=1241:00 +CPU= 21.103
  12:01:21 STEP 1242 H=1242:00 +CPU= 34.369
  12:01:42 STEP 1243 H=1243:00 +CPU= 20.269
  12:02:04 STEP 1244 H=1244:00 +CPU= 22.336
  12:02:37 STEP 1245 H=1245:00 +CPU= 32.983
  12:02:56 STEP 1246 H=1246:00 +CPU= 19.082
  12:03:18 STEP 1247 H=1247:00 +CPU= 21.359
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001224
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001224
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001224
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001236
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001236
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001236
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_50.zip
Uploading the intermediate file: upload_file_50.zip
  12:03:55 STEP 1248 H=1248:00 +CPU= 36.483
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001248
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001248
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001248
  12:04:14 STEP 1249 H=1249:00 +CPU= 19.104
  12:04:35 STEP 1250 H=1250:00 +CPU= 20.305
  12:05:08 STEP 1251 H=1251:00 +CPU= 33.011
  12:05:28 STEP 1252 H=1252:00 +CPU= 20.324
  12:05:49 STEP 1253 H=1253:00 +CPU= 20.854
  12:06:21 STEP 1254 H=1254:00 +CPU= 31.867
  12:06:41 STEP 1255 H=1255:00 +CPU= 19.619
  12:07:02 STEP 1256 H=1256:00 +CPU= 20.520
  12:07:34 STEP 1257 H=1257:00 +CPU= 31.442
  12:07:53 STEP 1258 H=1258:00 +CPU= 19.427
  12:08:14 STEP 1259 H=1259:00 +CPU= 20.247
  12:08:49 STEP 1260 H=1260:00 +CPU= 35.270
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001260
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001260
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001260
  12:09:08 STEP 1261 H=1261:00 +CPU= 18.448
  12:09:29 STEP 1262 H=1262:00 +CPU= 20.717
  12:10:02 STEP 1263 H=1263:00 +CPU= 33.016
  12:10:23 STEP 1264 H=1264:00 +CPU= 21.319
  12:10:45 STEP 1265 H=1265:00 +CPU= 21.463
  12:11:18 STEP 1266 H=1266:00 +CPU= 32.810
  12:11:38 STEP 1267 H=1267:00 +CPU= 19.776
  12:11:59 STEP 1268 H=1268:00 +CPU= 20.819
  12:12:32 STEP 1269 H=1269:00 +CPU= 32.764
  12:12:53 STEP 1270 H=1270:00 +CPU= 21.556
  12:13:15 STEP 1271 H=1271:00 +CPU= 21.753
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001248
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001248
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001248
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001260
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001260
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001260
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_51.zip
Uploading the intermediate file: upload_file_51.zip
  12:13:54 STEP 1272 H=1272:00 +CPU= 38.717
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001272
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001272
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001272
  12:14:14 STEP 1273 H=1273:00 +CPU= 19.433
  12:14:35 STEP 1274 H=1274:00 +CPU= 21.129
  12:15:06 STEP 1275 H=1275:00 +CPU= 30.605
  12:15:26 STEP 1276 H=1276:00 +CPU= 19.376
  12:15:46 STEP 1277 H=1277:00 +CPU= 20.478
  12:16:18 STEP 1278 H=1278:00 +CPU= 31.922
  12:16:38 STEP 1279 H=1279:00 +CPU= 19.625
  12:16:59 STEP 1280 H=1280:00 +CPU= 20.889
  12:17:31 STEP 1281 H=1281:00 +CPU= 32.060
  12:17:50 STEP 1282 H=1282:00 +CPU= 18.949
  12:18:12 STEP 1283 H=1283:00 +CPU= 21.345
  12:18:48 STEP 1284 H=1284:00 +CPU= 35.781
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001284
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001284
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001284
  12:19:08 STEP 1285 H=1285:00 +CPU= 20.205
  12:19:30 STEP 1286 H=1286:00 +CPU= 21.771
  12:20:03 STEP 1287 H=1287:00 +CPU= 32.477
  12:20:23 STEP 1288 H=1288:00 +CPU= 20.447
  12:20:45 STEP 1289 H=1289:00 +CPU= 21.168
  12:21:17 STEP 1290 H=1290:00 +CPU= 31.798
  12:21:35 STEP 1291 H=1291:00 +CPU= 18.837
  12:21:57 STEP 1292 H=1292:00 +CPU= 21.456
  12:22:31 STEP 1293 H=1293:00 +CPU= 33.335
  12:22:51 STEP 1294 H=1294:00 +CPU= 20.298
  12:23:11 STEP 1295 H=1295:00 +CPU= 19.675
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001272
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001272
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001272
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001284
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001284
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001284
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_52.zip
Uploading the intermediate file: upload_file_52.zip
  12:23:48 STEP 1296 H=1296:00 +CPU= 36.720
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001296
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001296
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001296
  12:24:07 STEP 1297 H=1297:00 +CPU= 18.917
  12:24:28 STEP 1298 H=1298:00 +CPU= 20.851
  12:25:00 STEP 1299 H=1299:00 +CPU= 31.732
  12:25:20 STEP 1300 H=1300:00 +CPU= 20.017
  12:25:42 STEP 1301 H=1301:00 +CPU= 21.018
  12:26:13 STEP 1302 H=1302:00 +CPU= 31.488
  12:26:33 STEP 1303 H=1303:00 +CPU= 20.136
  12:26:54 STEP 1304 H=1304:00 +CPU= 20.825
  12:27:28 STEP 1305 H=1305:00 +CPU= 33.063
  12:27:47 STEP 1306 H=1306:00 +CPU= 19.327
  12:28:08 STEP 1307 H=1307:00 +CPU= 21.151
  12:28:44 STEP 1308 H=1308:00 +CPU= 35.777
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001308
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001308
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001308
  12:29:05 STEP 1309 H=1309:00 +CPU= 20.362
  12:29:25 STEP 1310 H=1310:00 +CPU= 20.197
  12:29:56 STEP 1311 H=1311:00 +CPU= 30.439
  12:30:17 STEP 1312 H=1312:00 +CPU= 21.351
  12:30:38 STEP 1313 H=1313:00 +CPU= 20.931
  12:31:11 STEP 1314 H=1314:00 +CPU= 32.197
  12:31:30 STEP 1315 H=1315:00 +CPU= 19.443
  12:31:51 STEP 1316 H=1316:00 +CPU= 20.046
  12:32:22 STEP 1317 H=1317:00 +CPU= 31.428
  12:32:42 STEP 1318 H=1318:00 +CPU= 19.572
  12:33:03 STEP 1319 H=1319:00 +CPU= 20.862
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001296
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001296
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001296
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001308
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001308
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001308
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_53.zip
Uploading the intermediate file: upload_file_53.zip
  12:33:40 STEP 1320 H=1320:00 +CPU= 36.751
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001320
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001320
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001320
  12:34:00 STEP 1321 H=1321:00 +CPU= 19.573
  12:34:20 STEP 1322 H=1322:00 +CPU= 20.252
  12:34:53 STEP 1323 H=1323:00 +CPU= 32.732
  12:35:14 STEP 1324 H=1324:00 +CPU= 20.364
  12:35:34 STEP 1325 H=1325:00 +CPU= 19.991
  12:36:11 STEP 1326 H=1326:00 +CPU= 33.894
  12:36:32 STEP 1327 H=1327:00 +CPU= 21.453
  12:36:53 STEP 1328 H=1328:00 +CPU= 20.199
  12:37:29 STEP 1329 H=1329:00 +CPU= 33.358
  12:37:51 STEP 1330 H=1330:00 +CPU= 22.334
  12:38:13 STEP 1331 H=1331:00 +CPU= 21.258
  12:38:50 STEP 1332 H=1332:00 +CPU= 36.938
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001332
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001332
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001332
  12:39:12 STEP 1333 H=1333:00 +CPU= 21.530
  12:39:33 STEP 1334 H=1334:00 +CPU= 20.970
  12:40:06 STEP 1335 H=1335:00 +CPU= 33.042
  12:40:26 STEP 1336 H=1336:00 +CPU= 20.227
  12:40:46 STEP 1337 H=1337:00 +CPU= 19.738
  12:41:19 STEP 1338 H=1338:00 +CPU= 32.095
  12:41:39 STEP 1339 H=1339:00 +CPU= 19.853
  12:42:01 STEP 1340 H=1340:00 +CPU= 21.767
  12:42:32 STEP 1341 H=1341:00 +CPU= 31.617
  12:42:55 STEP 1342 H=1342:00 +CPU= 21.654
  12:43:17 STEP 1343 H=1343:00 +CPU= 21.993
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001320
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001320
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001320
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001332
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001332
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001332
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_54.zip
Uploading the intermediate file: upload_file_54.zip
  12:43:58 STEP 1344 H=1344:00 +CPU= 38.534
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001344
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001344
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001344
  12:44:19 STEP 1345 H=1345:00 +CPU= 20.946
  12:44:40 STEP 1346 H=1346:00 +CPU= 20.899
  12:45:14 STEP 1347 H=1347:00 +CPU= 32.955
  12:45:37 STEP 1348 H=1348:00 +CPU= 22.820
  12:45:58 STEP 1349 H=1349:00 +CPU= 21.205
  12:46:32 STEP 1350 H=1350:00 +CPU= 33.118
  12:46:52 STEP 1351 H=1351:00 +CPU= 20.203
  12:47:15 STEP 1352 H=1352:00 +CPU= 22.154
  12:47:47 STEP 1353 H=1353:00 +CPU= 32.363
  12:48:09 STEP 1354 H=1354:00 +CPU= 21.683
  12:48:31 STEP 1355 H=1355:00 +CPU= 22.049
  12:49:07 STEP 1356 H=1356:00 +CPU= 35.480
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001356
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001356
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001356
  12:49:28 STEP 1357 H=1357:00 +CPU= 19.732
  12:49:49 STEP 1358 H=1358:00 +CPU= 20.500
  12:50:21 STEP 1359 H=1359:00 +CPU= 31.585
  12:50:42 STEP 1360 H=1360:00 +CPU= 20.250
  12:51:02 STEP 1361 H=1361:00 +CPU= 20.621
  12:51:36 STEP 1362 H=1362:00 +CPU= 33.385
  12:51:56 STEP 1363 H=1363:00 +CPU= 19.826
  12:52:17 STEP 1364 H=1364:00 +CPU= 20.840
  12:52:45 STEP 1365 H=1365:00 +CPU= 27.807
  12:53:06 STEP 1366 H=1366:00 +CPU= 21.155
  12:53:35 STEP 1367 H=1367:00 +CPU= 23.365
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001344
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001344
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001344
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001356
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001356
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001356
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_55.zip
Uploading the intermediate file: upload_file_55.zip
  12:54:09 STEP 1368 H=1368:00 +CPU= 32.200
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001368
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001368
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001368
  12:54:27 STEP 1369 H=1369:00 +CPU= 18.056
  12:54:45 STEP 1370 H=1370:00 +CPU= 18.446
  12:55:16 STEP 1371 H=1371:00 +CPU= 30.831
  12:55:37 STEP 1372 H=1372:00 +CPU= 20.389
  12:55:56 STEP 1373 H=1373:00 +CPU= 19.638
  12:56:27 STEP 1374 H=1374:00 +CPU= 30.849
  12:56:45 STEP 1375 H=1375:00 +CPU= 17.425
  12:57:03 STEP 1376 H=1376:00 +CPU= 17.426
  12:57:30 STEP 1377 H=1377:00 +CPU= 26.931
  12:57:47 STEP 1378 H=1378:00 +CPU= 17.415
  12:58:04 STEP 1379 H=1379:00 +CPU= 17.187
  12:58:33 STEP 1380 H=1380:00 +CPU= 28.664
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001380
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001380
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001380
  12:58:50 STEP 1381 H=1381:00 +CPU= 16.932
  12:59:08 STEP 1382 H=1382:00 +CPU= 17.821
  12:59:37 STEP 1383 H=1383:00 +CPU= 28.269
  12:59:55 STEP 1384 H=1384:00 +CPU= 17.894
  13:00:11 STEP 1385 H=1385:00 +CPU= 16.553
  13:00:42 STEP 1386 H=1386:00 +CPU= 30.357
  13:01:01 STEP 1387 H=1387:00 +CPU= 19.315
  13:01:21 STEP 1388 H=1388:00 +CPU= 19.640
  13:01:53 STEP 1389 H=1389:00 +CPU= 31.321
  13:02:14 STEP 1390 H=1390:00 +CPU= 20.597
  13:02:34 STEP 1391 H=1391:00 +CPU= 20.087
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001368
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001368
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001368
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001380
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001380
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001380
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_56.zip
Uploading the intermediate file: upload_file_56.zip
  13:03:11 STEP 1392 H=1392:00 +CPU= 35.182
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001392
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001392
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001392
  13:03:30 STEP 1393 H=1393:00 +CPU= 19.199
  13:03:49 STEP 1394 H=1394:00 +CPU= 19.151
  13:04:20 STEP 1395 H=1395:00 +CPU= 30.546
  13:04:40 STEP 1396 H=1396:00 +CPU= 19.602
  13:04:59 STEP 1397 H=1397:00 +CPU= 18.895
  13:05:30 STEP 1398 H=1398:00 +CPU= 30.977
  13:05:49 STEP 1399 H=1399:00 +CPU= 19.345
  13:06:09 STEP 1400 H=1400:00 +CPU= 19.350
  13:06:39 STEP 1401 H=1401:00 +CPU= 30.036
  13:06:59 STEP 1402 H=1402:00 +CPU= 19.647
  13:07:18 STEP 1403 H=1403:00 +CPU= 18.992
  13:07:52 STEP 1404 H=1404:00 +CPU= 33.322
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001404
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001404
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001404
  13:08:11 STEP 1405 H=1405:00 +CPU= 19.208
  13:08:30 STEP 1406 H=1406:00 +CPU= 19.356
  13:09:02 STEP 1407 H=1407:00 +CPU= 31.428
  13:09:22 STEP 1408 H=1408:00 +CPU= 20.009
  13:09:41 STEP 1409 H=1409:00 +CPU= 18.751
  13:10:13 STEP 1410 H=1410:00 +CPU= 31.355
  13:10:32 STEP 1411 H=1411:00 +CPU= 19.452
  13:10:52 STEP 1412 H=1412:00 +CPU= 19.594
  13:11:22 STEP 1413 H=1413:00 +CPU= 30.411
  13:11:43 STEP 1414 H=1414:00 +CPU= 20.250
  13:12:02 STEP 1415 H=1415:00 +CPU= 19.330
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001392
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001392
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001392
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001404
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001404
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001404
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_57.zip
Uploading the intermediate file: upload_file_57.zip
  13:12:39 STEP 1416 H=1416:00 +CPU= 35.813
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001416
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001416
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001416
  13:12:59 STEP 1417 H=1417:00 +CPU= 19.700
  13:13:20 STEP 1418 H=1418:00 +CPU= 20.548
  13:13:51 STEP 1419 H=1419:00 +CPU= 31.379
  13:14:12 STEP 1420 H=1420:00 +CPU= 20.089
  13:14:31 STEP 1421 H=1421:00 +CPU= 19.291
  13:15:02 STEP 1422 H=1422:00 +CPU= 31.171
  13:15:22 STEP 1423 H=1423:00 +CPU= 19.552
  13:15:42 STEP 1424 H=1424:00 +CPU= 19.405
  13:16:13 STEP 1425 H=1425:00 +CPU= 30.854
  13:16:33 STEP 1426 H=1426:00 +CPU= 20.130
  13:16:53 STEP 1427 H=1427:00 +CPU= 19.878
  13:17:27 STEP 1428 H=1428:00 +CPU= 34.232
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001428
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001428
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001428
  13:17:47 STEP 1429 H=1429:00 +CPU= 19.456
  13:18:07 STEP 1430 H=1430:00 +CPU= 20.112
  13:18:37 STEP 1431 H=1431:00 +CPU= 29.651
  13:18:57 STEP 1432 H=1432:00 +CPU= 19.998
  13:19:17 STEP 1433 H=1433:00 +CPU= 19.665
  13:19:49 STEP 1434 H=1434:00 +CPU= 31.963
  13:20:08 STEP 1435 H=1435:00 +CPU= 18.900
  13:20:29 STEP 1436 H=1436:00 +CPU= 20.336
  13:20:59 STEP 1437 H=1437:00 +CPU= 30.614
  13:21:20 STEP 1438 H=1438:00 +CPU= 20.330
  13:21:42 STEP 1439 H=1439:00 +CPU= 20.899
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001416
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001416
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001416
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001428
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001428
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001428
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_58.zip
Uploading the intermediate file: upload_file_58.zip
Uploading trickle at timestep: 5180400
  13:22:18 STEP 1440 H=1440:00 +CPU= 35.754
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001440
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001440
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001440
  13:22:37 STEP 1441 H=1441:00 +CPU= 18.394
  13:22:57 STEP 1442 H=1442:00 +CPU= 20.131
  13:23:27 STEP 1443 H=1443:00 +CPU= 29.908
  13:23:47 STEP 1444 H=1444:00 +CPU= 19.582
  13:24:06 STEP 1445 H=1445:00 +CPU= 19.234
  13:24:38 STEP 1446 H=1446:00 +CPU= 31.433
  13:24:57 STEP 1447 H=1447:00 +CPU= 19.271
  13:25:18 STEP 1448 H=1448:00 +CPU= 20.385
  13:25:49 STEP 1449 H=1449:00 +CPU= 30.848
  13:26:09 STEP 1450 H=1450:00 +CPU= 20.189
  13:26:29 STEP 1451 H=1451:00 +CPU= 19.649
  13:27:02 STEP 1452 H=1452:00 +CPU= 33.329
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001452
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001452
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001452
  13:27:22 STEP 1453 H=1453:00 +CPU= 19.506
  13:27:42 STEP 1454 H=1454:00 +CPU= 19.682
  13:28:12 STEP 1455 H=1455:00 +CPU= 29.845
  13:28:32 STEP 1456 H=1456:00 +CPU= 20.243
  13:28:52 STEP 1457 H=1457:00 +CPU= 19.424
  13:29:23 STEP 1458 H=1458:00 +CPU= 30.516
  13:29:42 STEP 1459 H=1459:00 +CPU= 19.711
  13:30:02 STEP 1460 H=1460:00 +CPU= 19.330
  13:30:32 STEP 1461 H=1461:00 +CPU= 29.975
  13:30:51 STEP 1462 H=1462:00 +CPU= 19.029
  13:31:12 STEP 1463 H=1463:00 +CPU= 20.254
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001440
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001440
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001440
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001452
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001452
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001452
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_59.zip
Uploading the intermediate file: upload_file_59.zip
  13:31:48 STEP 1464 H=1464:00 +CPU= 36.589
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001464
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001464
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001464
  13:32:08 STEP 1465 H=1465:00 +CPU= 19.407
  13:32:27 STEP 1466 H=1466:00 +CPU= 19.207
  13:32:58 STEP 1467 H=1467:00 +CPU= 30.129
  13:33:17 STEP 1468 H=1468:00 +CPU= 19.195
  13:33:37 STEP 1469 H=1469:00 +CPU= 19.858
  13:34:07 STEP 1470 H=1470:00 +CPU= 30.016
  13:34:27 STEP 1471 H=1471:00 +CPU= 19.347
  13:34:47 STEP 1472 H=1472:00 +CPU= 19.892
  13:35:17 STEP 1473 H=1473:00 +CPU= 30.061
  13:35:37 STEP 1474 H=1474:00 +CPU= 19.486
  13:35:57 STEP 1475 H=1475:00 +CPU= 19.808
  13:36:29 STEP 1476 H=1476:00 +CPU= 32.510
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001476
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001476
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001476
  13:36:49 STEP 1477 H=1477:00 +CPU= 19.668
  13:37:10 STEP 1478 H=1478:00 +CPU= 20.499
  13:37:41 STEP 1479 H=1479:00 +CPU= 30.624
  13:38:01 STEP 1480 H=1480:00 +CPU= 20.149
  13:38:20 STEP 1481 H=1481:00 +CPU= 19.031
  13:38:51 STEP 1482 H=1482:00 +CPU= 30.786
  13:39:11 STEP 1483 H=1483:00 +CPU= 19.826
  13:39:32 STEP 1484 H=1484:00 +CPU= 20.676
  13:40:02 STEP 1485 H=1485:00 +CPU= 29.784
  13:40:22 STEP 1486 H=1486:00 +CPU= 19.912
  13:40:42 STEP 1487 H=1487:00 +CPU= 19.923
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001464
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001464
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001464
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001476
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001476
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001476
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_60.zip
Uploading the intermediate file: upload_file_60.zip
  13:41:18 STEP 1488 H=1488:00 +CPU= 35.198
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001488
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001488
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001488
  13:41:37 STEP 1489 H=1489:00 +CPU= 19.031
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:54:17 STEP 1490 H=1490:00 +CPU= 23.544
  13:54:52 STEP 1491 H=1491:00 +CPU= 31.492
  13:55:12 STEP 1492 H=1492:00 +CPU= 19.362
  13:55:31 STEP 1493 H=1493:00 +CPU= 18.690
  13:56:03 STEP 1494 H=1494:00 +CPU= 30.873
  13:56:24 STEP 1495 H=1495:00 +CPU= 20.354
  13:56:45 STEP 1496 H=1496:00 +CPU= 19.419
  13:57:15 STEP 1497 H=1497:00 +CPU= 29.636
  13:57:36 STEP 1498 H=1498:00 +CPU= 19.840
  13:57:55 STEP 1499 H=1499:00 +CPU= 18.659
  13:58:25 STEP 1500 H=1500:00 +CPU= 28.718
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001500
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001500
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001500
  13:58:44 STEP 1501 H=1501:00 +CPU= 18.098
  13:59:04 STEP 1502 H=1502:00 +CPU= 18.546
  13:59:32 STEP 1503 H=1503:00 +CPU= 27.145
  13:59:54 STEP 1504 H=1504:00 +CPU= 20.776
  14:00:13 STEP 1505 H=1505:00 +CPU= 18.435
  14:00:43 STEP 1506 H=1506:00 +CPU= 29.226
  14:01:02 STEP 1507 H=1507:00 +CPU= 18.681
  14:01:22 STEP 1508 H=1508:00 +CPU= 19.330
  14:01:54 STEP 1509 H=1509:00 +CPU= 28.896
  14:02:13 STEP 1510 H=1510:00 +CPU= 18.511
  14:02:32 STEP 1511 H=1511:00 +CPU= 18.268
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001488
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001488
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001488
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001500
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001500
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001500
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_61.zip
Uploading the intermediate file: upload_file_61.zip
  14:03:08 STEP 1512 H=1512:00 +CPU= 33.372
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001512
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001512
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001512
  14:03:23 STEP 1513 H=1513:00 +CPU= 15.210
  14:03:42 STEP 1514 H=1514:00 +CPU= 18.514
  14:04:10 STEP 1515 H=1515:00 +CPU= 27.401
  14:04:30 STEP 1516 H=1516:00 +CPU= 18.785
  14:04:48 STEP 1517 H=1517:00 +CPU= 15.997
  14:05:18 STEP 1518 H=1518:00 +CPU= 28.352
  14:05:38 STEP 1519 H=1519:00 +CPU= 19.115
  14:05:57 STEP 1520 H=1520:00 +CPU= 19.393
  14:06:25 STEP 1521 H=1521:00 +CPU= 27.065
  14:06:43 STEP 1522 H=1522:00 +CPU= 17.790
  14:07:01 STEP 1523 H=1523:00 +CPU= 18.018
  14:07:34 STEP 1524 H=1524:00 +CPU= 31.908
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001524
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001524
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001524
  14:07:50 STEP 1525 H=1525:00 +CPU= 15.285
  14:08:12 STEP 1526 H=1526:00 +CPU= 21.919
  14:08:41 STEP 1527 H=1527:00 +CPU= 28.704
  14:09:03 STEP 1528 H=1528:00 +CPU= 20.758
  14:09:23 STEP 1529 H=1529:00 +CPU= 20.252
  14:09:56 STEP 1530 H=1530:00 +CPU= 31.840
  14:10:16 STEP 1531 H=1531:00 +CPU= 19.671
  14:10:37 STEP 1532 H=1532:00 +CPU= 21.049
  14:11:09 STEP 1533 H=1533:00 +CPU= 30.906
  14:11:29 STEP 1534 H=1534:00 +CPU= 20.024
  14:11:51 STEP 1535 H=1535:00 +CPU= 21.162
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001512
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001512
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001512
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001524
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001524
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001524
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_62.zip
Uploading the intermediate file: upload_file_62.zip
  14:12:26 STEP 1536 H=1536:00 +CPU= 33.415
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001536
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001536
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001536
  14:12:45 STEP 1537 H=1537:00 +CPU= 18.502
  14:13:05 STEP 1538 H=1538:00 +CPU= 19.183
  14:13:32 STEP 1539 H=1539:00 +CPU= 26.527
  14:13:51 STEP 1540 H=1540:00 +CPU= 18.385
  14:14:10 STEP 1541 H=1541:00 +CPU= 19.102
  14:14:40 STEP 1542 H=1542:00 +CPU= 28.764
  14:14:58 STEP 1543 H=1543:00 +CPU= 18.597
  14:15:18 STEP 1544 H=1544:00 +CPU= 18.911
  14:15:47 STEP 1545 H=1545:00 +CPU= 28.331
  14:16:06 STEP 1546 H=1546:00 +CPU= 18.487
  14:16:26 STEP 1547 H=1547:00 +CPU= 19.808
  14:16:57 STEP 1548 H=1548:00 +CPU= 29.670
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001548
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001548
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001548
  14:17:14 STEP 1549 H=1549:00 +CPU= 17.061
  14:17:34 STEP 1550 H=1550:00 +CPU= 18.828
  14:18:03 STEP 1551 H=1551:00 +CPU= 28.935
  14:18:23 STEP 1552 H=1552:00 +CPU= 20.455
  14:18:41 STEP 1553 H=1553:00 +CPU= 17.529
  14:19:09 STEP 1554 H=1554:00 +CPU= 27.691
  14:19:28 STEP 1555 H=1555:00 +CPU= 19.085
  14:19:48 STEP 1556 H=1556:00 +CPU= 19.621
  14:20:20 STEP 1557 H=1557:00 +CPU= 30.795
  14:20:36 STEP 1558 H=1558:00 +CPU= 16.347
  14:20:54 STEP 1559 H=1559:00 +CPU= 17.564
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001536
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001536
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001536
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001548
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001548
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001548
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_63.zip
Uploading the intermediate file: upload_file_63.zip
  14:21:27 STEP 1560 H=1560:00 +CPU= 32.112
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001560
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001560
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001560
  14:21:46 STEP 1561 H=1561:00 +CPU= 18.618
  14:22:06 STEP 1562 H=1562:00 +CPU= 19.594
  14:22:37 STEP 1563 H=1563:00 +CPU= 30.010
  14:22:57 STEP 1564 H=1564:00 +CPU= 19.474
  14:23:16 STEP 1565 H=1565:00 +CPU= 18.794
  14:23:46 STEP 1566 H=1566:00 +CPU= 28.926
  14:24:05 STEP 1567 H=1567:00 +CPU= 19.019
  14:24:26 STEP 1568 H=1568:00 +CPU= 20.397
  14:24:58 STEP 1569 H=1569:00 +CPU= 30.711
  14:25:20 STEP 1570 H=1570:00 +CPU= 21.496
  14:25:40 STEP 1571 H=1571:00 +CPU= 19.566
  14:26:12 STEP 1572 H=1572:00 +CPU= 31.420
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001572
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001572
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001572
  14:26:32 STEP 1573 H=1573:00 +CPU= 18.562
  14:26:49 STEP 1574 H=1574:00 +CPU= 16.885
  14:27:12 STEP 1575 H=1575:00 +CPU= 22.749
  14:27:28 STEP 1576 H=1576:00 +CPU= 15.643
  14:27:42 STEP 1577 H=1577:00 +CPU= 13.842
  14:28:09 STEP 1578 H=1578:00 +CPU= 26.379
  14:28:27 STEP 1579 H=1579:00 +CPU= 17.028
  14:28:45 STEP 1580 H=1580:00 +CPU= 18.098
  14:29:17 STEP 1581 H=1581:00 +CPU= 28.878
  14:29:35 STEP 1582 H=1582:00 +CPU= 18.389
  14:29:51 STEP 1583 H=1583:00 +CPU= 15.652
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001560
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001560
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001560
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001572
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001572
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001572
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_64.zip
Uploading the intermediate file: upload_file_64.zip
  14:30:23 STEP 1584 H=1584:00 +CPU= 30.441
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001584
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001584
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001584
  14:30:43 STEP 1585 H=1585:00 +CPU= 19.527
  14:31:02 STEP 1586 H=1586:00 +CPU= 18.184
  14:31:33 STEP 1587 H=1587:00 +CPU= 29.691
  14:31:52 STEP 1588 H=1588:00 +CPU= 18.472
  14:32:12 STEP 1589 H=1589:00 +CPU= 19.438
  14:32:38 STEP 1590 H=1590:00 +CPU= 26.074
  14:32:56 STEP 1591 H=1591:00 +CPU= 17.567
  14:33:16 STEP 1592 H=1592:00 +CPU= 19.337
  14:33:42 STEP 1593 H=1593:00 +CPU= 25.080
  14:34:04 STEP 1594 H=1594:00 +CPU= 21.913
  14:34:26 STEP 1595 H=1595:00 +CPU= 20.570
  14:35:00 STEP 1596 H=1596:00 +CPU= 33.453
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001596
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001596
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001596
  14:35:20 STEP 1597 H=1597:00 +CPU= 19.115
  14:35:36 STEP 1598 H=1598:00 +CPU= 15.810
  14:36:01 STEP 1599 H=1599:00 +CPU= 23.886
  14:36:16 STEP 1600 H=1600:00 +CPU= 15.480
  14:36:36 STEP 1601 H=1601:00 +CPU= 18.986
  14:37:03 STEP 1602 H=1602:00 +CPU= 26.382
  14:37:21 STEP 1603 H=1603:00 +CPU= 17.405
  14:37:40 STEP 1604 H=1604:00 +CPU= 18.724
  14:38:10 STEP 1605 H=1605:00 +CPU= 29.095
  14:38:29 STEP 1606 H=1606:00 +CPU= 19.043
  14:38:44 STEP 1607 H=1607:00 +CPU= 14.912
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001584
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001584
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001584
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001596
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001596
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001596
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_65.zip
Uploading the intermediate file: upload_file_65.zip
  14:39:11 STEP 1608 H=1608:00 +CPU= 25.342
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001608
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001608
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001608
  14:39:26 STEP 1609 H=1609:00 +CPU= 14.685
  14:39:45 STEP 1610 H=1610:00 +CPU= 19.116
  14:40:17 STEP 1611 H=1611:00 +CPU= 31.165
  14:40:37 STEP 1612 H=1612:00 +CPU= 19.594
  14:40:56 STEP 1613 H=1613:00 +CPU= 18.820
  14:41:25 STEP 1614 H=1614:00 +CPU= 27.602
  14:41:42 STEP 1615 H=1615:00 +CPU= 17.373
  14:42:03 STEP 1616 H=1616:00 +CPU= 19.974
  14:42:34 STEP 1617 H=1617:00 +CPU= 30.983
  14:42:52 STEP 1618 H=1618:00 +CPU= 17.456
  14:43:06 STEP 1619 H=1619:00 +CPU= 14.300
  14:43:38 STEP 1620 H=1620:00 +CPU= 31.585
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001620
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001620
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001620
  14:43:56 STEP 1621 H=1621:00 +CPU= 18.042
  14:44:15 STEP 1622 H=1622:00 +CPU= 18.797
  14:44:41 STEP 1623 H=1623:00 +CPU= 25.610
  14:45:00 STEP 1624 H=1624:00 +CPU= 18.602
  14:45:18 STEP 1625 H=1625:00 +CPU= 18.213
  14:45:47 STEP 1626 H=1626:00 +CPU= 28.363
  14:46:04 STEP 1627 H=1627:00 +CPU= 16.757
  14:46:24 STEP 1628 H=1628:00 +CPU= 19.143
  14:46:55 STEP 1629 H=1629:00 +CPU= 29.654
  14:47:16 STEP 1630 H=1630:00 +CPU= 17.898
  14:47:34 STEP 1631 H=1631:00 +CPU= 17.263
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001608
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001608
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001608
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001620
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001620
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001620
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_66.zip
Uploading the intermediate file: upload_file_66.zip
  14:48:08 STEP 1632 H=1632:00 +CPU= 32.412
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001632
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001632
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001632
  14:48:28 STEP 1633 H=1633:00 +CPU= 19.654
  14:48:47 STEP 1634 H=1634:00 +CPU= 18.991
  14:49:17 STEP 1635 H=1635:00 +CPU= 29.313
  14:49:36 STEP 1636 H=1636:00 +CPU= 18.457
  14:49:52 STEP 1637 H=1637:00 +CPU= 15.526
  14:50:21 STEP 1638 H=1638:00 +CPU= 28.186
  14:50:37 STEP 1639 H=1639:00 +CPU= 15.716
  14:50:52 STEP 1640 H=1640:00 +CPU= 14.823
  14:51:18 STEP 1641 H=1641:00 +CPU= 25.431
  14:51:36 STEP 1642 H=1642:00 +CPU= 17.625
  14:51:56 STEP 1643 H=1643:00 +CPU= 18.801
  14:52:28 STEP 1644 H=1644:00 +CPU= 31.663
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001644
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001644
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001644
  14:52:49 STEP 1645 H=1645:00 +CPU= 20.024
  14:53:09 STEP 1646 H=1646:00 +CPU= 20.276
  14:53:39 STEP 1647 H=1647:00 +CPU= 28.836
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0714
(argv4) batchid: 968
(argv5) wuid: 12185358
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /mnt/home/boinc/slots/3
Project directory is: /mnt/home/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358
..mkdir for temp folder for results failed
Copying: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /mnt/home/boinc/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /mnt/home/boinc/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bd3d0fa52899abf2fa2b957edd896296 to: /mnt/home/boinc/slots/3/oifs_43r3_ps_0714_1999050100_123_968_12185358.zip
Unzipping the namelist zip file: /mnt/home/boinc/slots/3/oifs_43r3_ps_0714_1999050100_123_968_12185358.zip
ic_ancil_file: ic_ancil_12185358
ifsdata_file: ifsdata_12185358
climate_data_file: clim_data_12185358
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_ae18860f103ee5790e9b7db832619a54 to: /mnt/home/boinc/slots/3/ic_ancil_12185358.zip
Unzipping the IC ancils zip file: /mnt/home/boinc/slots/3/ic_ancil_12185358.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5b5ede742d033d2835567b32551249e9 to: /mnt/home/boinc/slots/3/ifsdata/ifsdata_12185358.zip
Unzipping the ifsdata_zip file: /mnt/home/boinc/slots/3/ifsdata/ifsdata_12185358.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/home/boinc/slots/3/159l_2/clim_data_12185358.zip
Unzipping the climate data zip file: /mnt/home/boinc/slots/3/159l_2/clim_data_12185358.zip
Checking for progress XML file: /mnt/home/boinc/slots/3/progress_file_12185358.xml
Opened progress file ok : /mnt/home/boinc/slots/3/progress_file_12185358.xml
-- Model is restarting --
Adjusting last_iter, 1648, to previous model restart step.
Creating progress file: /mnt/home/boinc/slots/3/progress_file_12185358.xml
last_cpu_time: 39879
upload_file_number: 67
last_iter: 1631
last_upload: 5871600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451
The child process has been launched with process id: 26344
Executing the command: /mnt/home/boinc/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:x570gamingx:1:1:26344:26344] [20230121:145455:1674309295:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  14:55:33 STEP 1632 H=1632:00 +CPU= 32.093
  14:55:49 STEP 1633 H=1633:00 +CPU= 15.192
  14:56:07 STEP 1634 H=1634:00 +CPU= 17.611
  14:56:34 STEP 1635 H=1635:00 +CPU= 26.759
  14:56:51 STEP 1636 H=1636:00 +CPU= 16.658
  14:57:11 STEP 1637 H=1637:00 +CPU= 18.157
  14:57:41 STEP 1638 H=1638:00 +CPU= 30.008
  14:58:00 STEP 1639 H=1639:00 +CPU= 18.341
  14:58:19 STEP 1640 H=1640:00 +CPU= 17.789
  14:58:46 STEP 1641 H=1641:00 +CPU= 26.279
  14:59:03 STEP 1642 H=1642:00 +CPU= 16.349
  14:59:20 STEP 1643 H=1643:00 +CPU= 17.064
  14:59:46 STEP 1644 H=1644:00 +CPU= 25.033
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001644
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001644
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001644
  15:00:01 STEP 1645 H=1645:00 +CPU= 15.116
  15:00:16 STEP 1646 H=1646:00 +CPU= 15.006
  15:00:41 STEP 1647 H=1647:00 +CPU= 24.756
  15:00:59 STEP 1648 H=1648:00 +CPU= 17.870
  15:01:17 STEP 1649 H=1649:00 +CPU= 18.514
  15:01:46 STEP 1650 H=1650:00 +CPU= 27.892
  15:02:03 STEP 1651 H=1651:00 +CPU= 17.768
  15:02:17 STEP 1652 H=1652:00 +CPU= 13.854
  15:02:42 STEP 1653 H=1653:00 +CPU= 24.536
  15:03:00 STEP 1654 H=1654:00 +CPU= 16.211
  15:03:19 STEP 1655 H=1655:00 +CPU= 18.900
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001632
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001632
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001632
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMGGhq0f+001644
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMSHhq0f+001644
Adding to the zip: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_12185358/ICMUAhq0f+001644
Zipping up the intermediate file: /mnt/home/boinc/projects/climateprediction.net/oifs_43r3_ps_0714_1999050100_123_968_12185358_1_r1253877451_67.zip
Uploading the intermediate file: upload_file_67.zip
  15:03:53 STEP 1656 H=1656:00 +CPU= 33.575
Moving to projects directory: /mnt/home/boinc/slots/3/ICMGGhq0f+001656
Moving to projects directory: /mnt/home/boinc/slots/3/ICMSHhq0f+001656
Moving to projects directory: /mnt/home/boinc/slots/3/ICMUAhq0f+001656
  15:04:13 STEP 1657 H=1657:00 +CPU= 19.638
  15:04:33 STEP 1658 H=1658:00 +CPU= 19.580
  15:05:03 STEP 1659 H=1659:00 +CPU= 30.062
  15:05:21 STEP 1660 H=1660:00 +CPU= 17.866
  15:05:37 STEP 1661 H=1661:00 +CPU= 15.926
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:33:11 1532028 22299108 oifs_43r3_ps_0714_1999050100_123_968_12185358_1 5,180,400 35,216 0.0068
09 May 2023 16:33:05 1532028 22299108 oifs_43r3_ps_0714_1999050100_123_968_12185358_1 4,316,400 29,376 0.0068
09 May 2023 16:31:36 1532028 22299108 oifs_43r3_ps_0714_1999050100_123_968_12185358_1 2,847,600 19,138 0.0067


©2024 cpdn.org