Name | oifs_43r3_ps_0572_2016050100_123_985_12202216_1 |
Workunit | 12202216 |
Created | 26 Dec 2022, 8:31:58 UTC |
Sent | 26 Dec 2022, 16:19:16 UTC |
Report deadline | 25 Jan 2023, 16:19:16 UTC |
Received | 27 Dec 2022, 14:39:17 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1515822 |
Run time | 3 hours 0 min 47 sec |
CPU time | 2 hours 43 min 5 sec |
Validate state | Invalid |
Credit | 1,209.60 |
Device peak FLOPS | 4.20 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,260.21 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 2,187.83 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> TEP 156 H= 156:00 +CPU= 42.012 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000156 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000156 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000156 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:55:50 STEP 157 H= 157:00 +CPU= 24.544 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:56:17 STEP 158 H= 158:00 +CPU= 23.993 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:57:02 STEP 159 H= 159:00 +CPU= 37.288 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:57:29 STEP 160 H= 160:00 +CPU= 23.231 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:57:54 STEP 161 H= 161:00 +CPU= 22.563 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:58:37 STEP 162 H= 162:00 +CPU= 37.561 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:59:05 STEP 163 H= 163:00 +CPU= 24.630 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:59:33 STEP 164 H= 164:00 +CPU= 24.740 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:00:16 STEP 165 H= 165:00 +CPU= 38.472 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:00:46 STEP 166 H= 166:00 +CPU= 24.967 13:01:12 STEP 167 H= 167:00 +CPU= 25.102 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:02:05 STEP 168 H= 168:00 +CPU= 44.471 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000168 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000168 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000144 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000144 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000144 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000156 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000156 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000156 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632_6.zip Uploading the intermediate file: upload_file_6.zip 13:02:30 STEP 169 H= 169:00 +CPU= 24.446 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:02:58 STEP 170 H= 170:00 +CPU= 24.943 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:03:44 STEP 171 H= 171:00 +CPU= 38.016 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:04:12 STEP 172 H= 172:00 +CPU= 24.933 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:04:42 STEP 173 H= 173:00 +CPU= 24.195 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:05:27 STEP 174 H= 174:00 +CPU= 39.048 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:05:54 STEP 175 H= 175:00 +CPU= 24.229 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:06:25 STEP 176 H= 176:00 +CPU= 25.250 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:07:08 STEP 177 H= 177:00 +CPU= 37.619 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:07:37 STEP 178 H= 178:00 +CPU= 25.354 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:08:06 STEP 179 H= 179:00 +CPU= 24.489 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:08:55 STEP 180 H= 180:00 +CPU= 42.457 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000180 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000180 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000180 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:09:24 STEP 181 H= 181:00 +CPU= 24.358 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:09:53 STEP 182 H= 182:00 +CPU= 25.051 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:10:37 STEP 183 H= 183:00 +CPU= 38.343 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:11:07 STEP 184 H= 184:00 +CPU= 24.602 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:11:34 STEP 185 H= 185:00 +CPU= 24.224 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:12:22 STEP 186 H= 186:00 +CPU= 39.611 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:12:49 STEP 187 H= 187:00 +CPU= 24.187 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:13:17 STEP 188 H= 188:00 +CPU= 25.016 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:14:03 STEP 189 H= 189:00 +CPU= 37.577 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:14:31 STEP 190 H= 190:00 +CPU= 25.267 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:14:59 STEP 191 H= 191:00 +CPU= 24.450 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:15:48 STEP 192 H= 192:00 +CPU= 41.572 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000192 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000192 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000180 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000180 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000180 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632_7.zip Uploading the intermediate file: upload_file_7.zip 13:16:12 STEP 193 H= 193:00 +CPU= 22.784 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:16:42 STEP 194 H= 194:00 +CPU= 24.691 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:17:26 STEP 195 H= 195:00 +CPU= 38.131 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:17:54 STEP 196 H= 196:00 +CPU= 25.188 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:18:24 STEP 197 H= 197:00 +CPU= 24.357 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:19:09 STEP 198 H= 198:00 +CPU= 39.370 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:19:37 STEP 199 H= 199:00 +CPU= 24.439 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:20:07 STEP 200 H= 200:00 +CPU= 25.205 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:20:52 STEP 201 H= 201:00 +CPU= 39.001 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:21:22 STEP 202 H= 202:00 +CPU= 25.113 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:21:50 STEP 203 H= 203:00 +CPU= 24.367 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:22:40 STEP 204 H= 204:00 +CPU= 42.563 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000204 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000204 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000204 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:23:08 STEP 205 H= 205:00 +CPU= 24.417 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:23:36 STEP 206 H= 206:00 +CPU= 25.092 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:24:22 STEP 207 H= 207:00 +CPU= 37.921 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:24:51 STEP 208 H= 208:00 +CPU= 25.341 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:25:18 STEP 209 H= 209:00 +CPU= 24.336 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:26:05 STEP 210 H= 210:00 +CPU= 39.266 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:26:33 STEP 211 H= 211:00 +CPU= 24.440 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:27:03 STEP 212 H= 212:00 +CPU= 25.227 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:27:48 STEP 213 H= 213:00 +CPU= 38.576 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:28:16 STEP 214 H= 214:00 +CPU= 25.136 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:28:46 STEP 215 H= 215:00 +CPU= 24.926 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:29:37 STEP 216 H= 216:00 +CPU= 44.709 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000216 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000216 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000204 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000204 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000204 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632_8.zip Uploading the intermediate file: upload_file_8.zip 13:30:05 STEP 217 H= 217:00 +CPU= 24.379 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:30:33 STEP 218 H= 218:00 +CPU= 25.147 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:31:17 STEP 219 H= 219:00 +CPU= 38.086 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:31:47 STEP 220 H= 220:00 +CPU= 25.090 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:32:15 STEP 221 H= 221:00 +CPU= 24.424 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:33:02 STEP 222 H= 222:00 +CPU= 38.775 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:33:28 STEP 223 H= 223:00 +CPU= 22.774 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:33:54 STEP 224 H= 224:00 +CPU= 23.220 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:34:35 STEP 225 H= 225:00 +CPU= 35.433 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:35:05 STEP 226 H= 226:00 +CPU= 24.643 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:35:32 STEP 227 H= 227:00 +CPU= 24.243 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:36:23 STEP 228 H= 228:00 +CPU= 43.156 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000228 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000228 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000228 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:36:51 STEP 229 H= 229:00 +CPU= 24.561 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:37:21 STEP 230 H= 230:00 +CPU= 24.954 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:38:05 STEP 231 H= 231:00 +CPU= 37.796 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:38:34 STEP 232 H= 232:00 +CPU= 25.492 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:39:03 STEP 233 H= 233:00 +CPU= 24.382 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:39:48 STEP 234 H= 234:00 +CPU= 39.040 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:40:16 STEP 235 H= 235:00 +CPU= 24.341 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:40:46 STEP 236 H= 236:00 +CPU= 24.996 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:41:30 STEP 237 H= 237:00 +CPU= 38.614 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:42:01 STEP 238 H= 238:00 +CPU= 25.120 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:42:29 STEP 239 H= 239:00 +CPU= 24.896 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:43:22 STEP 240 H= 240:00 +CPU= 45.151 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000240 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000240 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000228 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000228 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000228 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632_9.zip Uploading the intermediate file: upload_file_9.zip 13:43:48 STEP 241 H= 241:00 +CPU= 24.739 Uploading trickle at timestep: 864000 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:44:17 STEP 242 H= 242:00 +CPU= 25.133 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:45:03 STEP 243 H= 243:00 +CPU= 38.395 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:45:31 STEP 244 H= 244:00 +CPU= 25.288 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:46:01 STEP 245 H= 245:00 +CPU= 24.465 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:46:46 STEP 246 H= 246:00 +CPU= 39.129 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:47:14 STEP 247 H= 247:00 +CPU= 24.430 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:47:44 STEP 248 H= 248:00 +CPU= 25.156 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:48:29 STEP 249 H= 249:00 +CPU= 38.871 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:48:57 STEP 250 H= 250:00 +CPU= 24.957 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:49:26 STEP 251 H= 251:00 +CPU= 24.336 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:50:16 STEP 252 H= 252:00 +CPU= 43.026 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000252 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000252 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000252 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:50:45 STEP 253 H= 253:00 +CPU= 24.493 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:51:14 STEP 254 H= 254:00 +CPU= 25.043 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:51:54 STEP 255 H= 255:00 +CPU= 35.248 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:52:23 STEP 256 H= 256:00 +CPU= 23.138 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:52:48 STEP 257 H= 257:00 +CPU= 22.459 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:53:33 STEP 258 H= 258:00 +CPU= 39.216 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:54:03 STEP 259 H= 259:00 +CPU= 24.259 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:54:31 STEP 260 H= 260:00 +CPU= 25.061 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:55:15 STEP 261 H= 261:00 +CPU= 38.112 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:55:45 STEP 262 H= 262:00 +CPU= 25.026 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:56:14 STEP 263 H= 263:00 +CPU= 25.334 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:57:07 STEP 264 H= 264:00 +CPU= 44.705 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000264 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000264 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000252 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000252 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000252 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632_10.zip Uploading the intermediate file: upload_file_10.zip 13:57:32 STEP 265 H= 265:00 +CPU= 24.681 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:58:03 STEP 266 H= 266:00 +CPU= 25.044 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:58:45 STEP 267 H= 267:00 +CPU= 38.802 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:59:14 STEP 268 H= 268:00 +CPU= 25.013 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:59:43 STEP 269 H= 269:00 +CPU= 24.225 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:00:28 STEP 270 H= 270:00 +CPU= 39.356 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:00:56 STEP 271 H= 271:00 +CPU= 24.684 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:01:26 STEP 272 H= 272:00 +CPU= 24.933 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:02:10 STEP 273 H= 273:00 +CPU= 38.228 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:02:41 STEP 274 H= 274:00 +CPU= 25.136 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:03:08 STEP 275 H= 275:00 +CPU= 24.531 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:03:57 STEP 276 H= 276:00 +CPU= 42.370 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000276 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000276 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000276 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:04:27 STEP 277 H= 277:00 +CPU= 24.708 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:04:55 STEP 278 H= 278:00 +CPU= 25.343 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:05:42 STEP 279 H= 279:00 +CPU= 39.001 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:06:10 STEP 280 H= 280:00 +CPU= 24.885 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:06:38 STEP 281 H= 281:00 +CPU= 24.373 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:07:25 STEP 282 H= 282:00 +CPU= 39.272 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:07:53 STEP 283 H= 283:00 +CPU= 24.527 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:08:23 STEP 284 H= 284:00 +CPU= 25.083 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:09:06 STEP 285 H= 285:00 +CPU= 37.725 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:09:34 STEP 286 H= 286:00 +CPU= 24.438 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:10:03 STEP 287 H= 287:00 +CPU= 23.427 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:10:50 STEP 288 H= 288:00 +CPU= 41.340 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000288 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000288 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000276 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000276 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000276 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632_11.zip Uploading the intermediate file: upload_file_11.zip 14:11:14 STEP 289 H= 289:00 +CPU= 23.288 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:11:45 STEP 290 H= 290:00 +CPU= 25.159 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:12:29 STEP 291 H= 291:00 +CPU= 38.583 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:12:58 STEP 292 H= 292:00 +CPU= 25.347 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:13:27 STEP 293 H= 293:00 +CPU= 24.637 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:14:13 STEP 294 H= 294:00 +CPU= 39.836 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:14:43 STEP 295 H= 295:00 +CPU= 24.579 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:15:11 STEP 296 H= 296:00 +CPU= 25.150 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:15:56 STEP 297 H= 297:00 +CPU= 38.511 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:16:26 STEP 298 H= 298:00 +CPU= 25.328 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:16:54 STEP 299 H= 299:00 +CPU= 24.612 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:17:44 STEP 300 H= 300:00 +CPU= 41.973 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000300 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000300 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000300 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:18:12 STEP 301 H= 301:00 +CPU= 24.888 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:18:43 STEP 302 H= 302:00 +CPU= 25.445 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:19:27 STEP 303 H= 303:00 +CPU= 38.246 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:19:58 STEP 304 H= 304:00 +CPU= 26.350 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:20:29 STEP 305 H= 305:00 +CPU= 25.693 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:21:16 STEP 306 H= 306:00 +CPU= 39.661 Suspend request received from the BOINC client, suspending the child process Resuming the child process [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.008] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3992MB (heap), 4261MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.012] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.012] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.012] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.012] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.012] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3992 MB (maxheap), 4261 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001) [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TURBULENCE_LAYER [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFOUTER [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFMAIN [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142127:1672150887:13528.013] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDVAR [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142128:1672150888:13529.013] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:KestrelWilts:1:1:2663:2663] [20221227:142128:1672150888:13529.013] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000000FD029F cloudvar_ 214 cloudvar.F90 oifs_43r3_model.e 0000000000FB3DC7 vdfmain_ 890 vdfmain.F90 oifs_43r3_model.e 0000000000F92699 vdfouter_ 739 vdfouter.F90 oifs_43r3_model.e 0000000000F88512 turbulence_layer_ 158 turbulence_layer.F90 oifs_43r3_model.e 0000000000F0B683 callpar_ 869 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2016050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0572 (argv4) batchid: 985 (argv5) wuid: 12202216 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/2 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_f61bc2e8ca269f501a1a9252d3a7cbd0 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0572_2016050100_123_985_12202216.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0572_2016050100_123_985_12202216.zip ic_ancil_file: ic_ancil_12202216 ifsdata_file: ifsdata_12202216 climate_data_file: clim_data_12202216 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_52ff8b26380a4e98d24c4a5ae663072c to: /var/lib/boinc-client/slots/2/ic_ancil_12202216.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12202216.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1a59a0628dfff36be7ac79cf6a4b1916 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12202216.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12202216.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12202216.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12202216.zip Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12202216.xml Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12202216.xml -- Model is restarting -- Adjusting last_iter, 307, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12202216.xml last_cpu_time: 9024 upload_file_number: 12 last_iter: 287 last_upload: 1036800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632 The child process has been launched with process id: 1676 Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:KestrelWilts:1:1:1676:1676] [20221227:142203:1672150923:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:22:55 STEP 288 H= 288:00 +CPU= 41.857 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:23:23 STEP 289 H= 289:00 +CPU= 23.241 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:23:50 STEP 290 H= 290:00 +CPU= 23.174 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:24:26 STEP 291 H= 291:00 +CPU= 34.143 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:24:51 STEP 292 H= 292:00 +CPU= 22.164 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:25:14 STEP 293 H= 293:00 +CPU= 21.750 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:25:53 STEP 294 H= 294:00 +CPU= 34.029 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:26:16 STEP 295 H= 295:00 +CPU= 21.769 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:26:41 STEP 296 H= 296:00 +CPU= 22.082 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:27:16 STEP 297 H= 297:00 +CPU= 33.358 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:27:42 STEP 298 H= 298:00 +CPU= 22.062 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:28:08 STEP 299 H= 299:00 +CPU= 22.861 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:28:56 STEP 300 H= 300:00 +CPU= 41.013 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000300 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000300 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000300 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:29:24 STEP 301 H= 301:00 +CPU= 22.658 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:29:51 STEP 302 H= 302:00 +CPU= 23.337 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:30:31 STEP 303 H= 303:00 +CPU= 35.039 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:30:57 STEP 304 H= 304:00 +CPU= 22.874 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:31:25 STEP 305 H= 305:00 +CPU= 22.516 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:32:06 STEP 306 H= 306:00 +CPU= 35.404 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:32:31 STEP 307 H= 307:00 +CPU= 22.333 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:32:57 STEP 308 H= 308:00 +CPU= 22.912 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:33:37 STEP 309 H= 309:00 +CPU= 34.652 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:34:05 STEP 310 H= 310:00 +CPU= 22.919 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:34:32 STEP 311 H= 311:00 +CPU= 23.078 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:35:21 STEP 312 H= 312:00 +CPU= 40.537 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000312 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000312 Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMGGhq0f+000300 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMSHhq0f+000300 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12202216/ICMUAhq0f+000300 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0572_2016050100_123_985_12202216_1_r1278818632_12.zip Uploading the intermediate file: upload_file_12.zip 14:35:44 STEP 313 H= 313:00 +CPU= 22.684 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:36:10 STEP 314 H= 314:00 +CPU= 22.971 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process The child process terminated with status: 0 ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 15:58:44 | 1515822 | 22293592 | oifs_43r3_ps_0572_2016050100_123_985_12202216_1 | 864,000 | 7,074 | 0.0082 |
©2024 cpdn.org