climateprediction.net home page
Task 22284068

Task 22284068

Name oifs_43r3_ps_0352_2012050100_123_981_12197996_0
Workunit 12197996
Created 24 Dec 2022, 4:44:46 UTC
Sent 26 Dec 2022, 16:10:21 UTC
Report deadline 25 Jan 2023, 16:10:21 UTC
Received 27 Dec 2022, 11:42:38 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1514051
Run time 3 hours 35 min 28 sec
CPU time 3 hours 13 min 54 sec
Validate state Invalid
Credit 1,209.60
Device peak FLOPS 4.26 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,257.91 MB
Peak swap size 4,974.80 MB
Peak disk usage 1,393.99 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ing the child process
  00:48:08 STEP  190 H= 190:00 +CPU= 27.115
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:48:42 STEP  191 H= 191:00 +CPU= 27.566
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:49:40 STEP  192 H= 192:00 +CPU= 48.593
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000192
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000192
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000180
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630_7.zip
Uploading the intermediate file: upload_file_7.zip
  00:50:08 STEP  193 H= 193:00 +CPU= 26.632
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:50:41 STEP  194 H= 194:00 +CPU= 27.210
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:51:28 STEP  195 H= 195:00 +CPU= 41.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:52:01 STEP  196 H= 196:00 +CPU= 27.418
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:52:32 STEP  197 H= 197:00 +CPU= 26.887
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:53:23 STEP  198 H= 198:00 +CPU= 42.836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:53:54 STEP  199 H= 199:00 +CPU= 27.124
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:54:27 STEP  200 H= 200:00 +CPU= 27.365
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:15 STEP  201 H= 201:00 +CPU= 41.834
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:48 STEP  202 H= 202:00 +CPU= 27.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:56:20 STEP  203 H= 203:00 +CPU= 26.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:12 STEP  204 H= 204:00 +CPU= 45.986
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:45 STEP  205 H= 205:00 +CPU= 26.689
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:58:16 STEP  206 H= 206:00 +CPU= 27.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:59:06 STEP  207 H= 207:00 +CPU= 41.733
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:59:39 STEP  208 H= 208:00 +CPU= 27.822
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:00:10 STEP  209 H= 209:00 +CPU= 26.933
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:01:01 STEP  210 H= 210:00 +CPU= 42.772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:01:31 STEP  211 H= 211:00 +CPU= 26.499
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:02:04 STEP  212 H= 212:00 +CPU= 27.527
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:02:53 STEP  213 H= 213:00 +CPU= 41.904
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:26 STEP  214 H= 214:00 +CPU= 27.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:59 STEP  215 H= 215:00 +CPU= 27.433
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:04:56 STEP  216 H= 216:00 +CPU= 49.203
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000204
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630_8.zip
Uploading the intermediate file: upload_file_8.zip
  01:05:25 STEP  217 H= 217:00 +CPU= 26.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:05:56 STEP  218 H= 218:00 +CPU= 27.479
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:06:46 STEP  219 H= 219:00 +CPU= 41.706
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:17 STEP  220 H= 220:00 +CPU= 27.049
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:49 STEP  221 H= 221:00 +CPU= 26.443
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:08:37 STEP  222 H= 222:00 +CPU= 42.041
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:09:10 STEP  223 H= 223:00 +CPU= 26.895
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:09:43 STEP  224 H= 224:00 +CPU= 27.513
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:10:31 STEP  225 H= 225:00 +CPU= 41.650
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:11:04 STEP  226 H= 226:00 +CPU= 27.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:11:34 STEP  227 H= 227:00 +CPU= 26.925
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:12:30 STEP  228 H= 228:00 +CPU= 46.613
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000228
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:13:02 STEP  229 H= 229:00 +CPU= 26.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:13:34 STEP  230 H= 230:00 +CPU= 27.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:14:24 STEP  231 H= 231:00 +CPU= 41.824
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:14:55 STEP  232 H= 232:00 +CPU= 27.668
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:15:27 STEP  233 H= 233:00 +CPU= 26.517
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:16:13 STEP  234 H= 234:00 +CPU= 42.320
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:16:46 STEP  235 H= 235:00 +CPU= 26.709
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:17:17 STEP  236 H= 236:00 +CPU= 27.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:18:06 STEP  237 H= 237:00 +CPU= 41.433
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:18:37 STEP  238 H= 238:00 +CPU= 27.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:19:10 STEP  239 H= 239:00 +CPU= 27.133
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:20:08 STEP  240 H= 240:00 +CPU= 48.458
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000228
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630_9.zip
Uploading the intermediate file: upload_file_9.zip
  01:20:36 STEP  241 H= 241:00 +CPU= 26.388
Uploading trickle at timestep: 864000
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:21:08 STEP  242 H= 242:00 +CPU= 27.139
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:21:56 STEP  243 H= 243:00 +CPU= 41.875
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:22:30 STEP  244 H= 244:00 +CPU= 27.783
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:22:59 STEP  245 H= 245:00 +CPU= 26.101
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:23:49 STEP  246 H= 246:00 +CPU= 43.078
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:24:21 STEP  247 H= 247:00 +CPU= 26.916
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:24:52 STEP  248 H= 248:00 +CPU= 27.642
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:25:43 STEP  249 H= 249:00 +CPU= 42.281
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:15 STEP  250 H= 250:00 +CPU= 27.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:46 STEP  251 H= 251:00 +CPU= 26.466
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:27:42 STEP  252 H= 252:00 +CPU= 46.487
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000252
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:28:12 STEP  253 H= 253:00 +CPU= 26.997
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:28:45 STEP  254 H= 254:00 +CPU= 27.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:29:33 STEP  255 H= 255:00 +CPU= 42.008
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:06 STEP  256 H= 256:00 +CPU= 27.216
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:36 STEP  257 H= 257:00 +CPU= 26.736
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:31:27 STEP  258 H= 258:00 +CPU= 42.483
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:31:57 STEP  259 H= 259:00 +CPU= 26.458
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:32:30 STEP  260 H= 260:00 +CPU= 27.428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:33:17 STEP  261 H= 261:00 +CPU= 41.584
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:33:51 STEP  262 H= 262:00 +CPU= 27.537
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:24 STEP  263 H= 263:00 +CPU= 27.540
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:35:21 STEP  264 H= 264:00 +CPU= 48.592
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000252
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630_10.zip
Uploading the intermediate file: upload_file_10.zip
  01:35:49 STEP  265 H= 265:00 +CPU= 26.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:36:22 STEP  266 H= 266:00 +CPU= 26.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:37:09 STEP  267 H= 267:00 +CPU= 41.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:37:43 STEP  268 H= 268:00 +CPU= 27.562
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:13 STEP  269 H= 269:00 +CPU= 26.429
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:39:04 STEP  270 H= 270:00 +CPU= 43.419
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:39:35 STEP  271 H= 271:00 +CPU= 26.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:07 STEP  272 H= 272:00 +CPU= 27.122
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:56 STEP  273 H= 273:00 +CPU= 42.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:41:29 STEP  274 H= 274:00 +CPU= 27.434
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:42:01 STEP  275 H= 275:00 +CPU= 26.187
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:42:54 STEP  276 H= 276:00 +CPU= 46.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000276
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:43:26 STEP  277 H= 277:00 +CPU= 26.558
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:43:56 STEP  278 H= 278:00 +CPU= 27.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:44:46 STEP  279 H= 279:00 +CPU= 40.908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:45:16 STEP  280 H= 280:00 +CPU= 27.233
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:45:48 STEP  281 H= 281:00 +CPU= 26.345
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:46:39 STEP  282 H= 282:00 +CPU= 42.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:47:09 STEP  283 H= 283:00 +CPU= 26.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:47:42 STEP  284 H= 284:00 +CPU= 27.217
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:48:30 STEP  285 H= 285:00 +CPU= 41.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:49:03 STEP  286 H= 286:00 +CPU= 27.362
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:49:34 STEP  287 H= 287:00 +CPU= 27.790
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:50:33 STEP  288 H= 288:00 +CPU= 48.965
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630_11.zip
Uploading the intermediate file: upload_file_11.zip
  01:51:01 STEP  289 H= 289:00 +CPU= 26.391
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:51:31 STEP  290 H= 290:00 +CPU= 27.405
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:52:21 STEP  291 H= 291:00 +CPU= 41.662
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:52:52 STEP  292 H= 292:00 +CPU= 27.495
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:53:21 STEP  293 H= 293:00 +CPU= 25.569
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:54:11 STEP  294 H= 294:00 +CPU= 42.834
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:54:43 STEP  295 H= 295:00 +CPU= 27.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:55:15 STEP  296 H= 296:00 +CPU= 27.912
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:56:05 STEP  297 H= 297:00 +CPU= 41.775
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:56:36 STEP  298 H= 298:00 +CPU= 27.484
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:57:08 STEP  299 H= 299:00 +CPU= 27.181
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:58:03 STEP  300 H= 300:00 +CPU= 46.439
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:58:34 STEP  301 H= 301:00 +CPU= 27.097
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:59:07 STEP  302 H= 302:00 +CPU= 27.767
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:59:55 STEP  303 H= 303:00 +CPU= 41.824
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:00:29 STEP  304 H= 304:00 +CPU= 27.506
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:01:01 STEP  305 H= 305:00 +CPU= 26.842
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:01:49 STEP  306 H= 306:00 +CPU= 42.024
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:02:22 STEP  307 H= 307:00 +CPU= 26.751
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:02:53 STEP  308 H= 308:00 +CPU= 27.486
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:03:43 STEP  309 H= 309:00 +CPU= 41.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:04:14 STEP  310 H= 310:00 +CPU= 27.430
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:04:47 STEP  311 H= 311:00 +CPU= 27.338
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:05:45 STEP  312 H= 312:00 +CPU= 48.555
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630_12.zip
Uploading the intermediate file: upload_file_12.zip
  02:06:13 STEP  313 H= 313:00 +CPU= 26.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:06:46 STEP  314 H= 314:00 +CPU= 27.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:07:34 STEP  315 H= 315:00 +CPU= 41.601
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:08:04 STEP  316 H= 316:00 +CPU= 26.674
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:08:34 STEP  317 H= 317:00 +CPU= 26.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:09:22 STEP  318 H= 318:00 +CPU= 42.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:09:53 STEP  319 H= 319:00 +CPU= 26.960
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:10:23 STEP  320 H= 320:00 +CPU= 27.270
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:11:11 STEP  321 H= 321:00 +CPU= 41.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:11:45 STEP  322 H= 322:00 +CPU= 27.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:12:15 STEP  323 H= 323:00 +CPU= 26.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:13:10 STEP  324 H= 324:00 +CPU= 46.506
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000324
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:13:42 STEP  325 H= 325:00 +CPU= 26.419
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:14:13 STEP  326 H= 326:00 +CPU= 27.500
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:15:03 STEP  327 H= 327:00 +CPU= 41.864
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:15:34 STEP  328 H= 328:00 +CPU= 27.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:16:06 STEP  329 H= 329:00 +CPU= 26.916
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:16:55 STEP  330 H= 330:00 +CPU= 42.638
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:17:27 STEP  331 H= 331:00 +CPU= 26.427
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:18:00 STEP  332 H= 332:00 +CPU= 27.356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:18:48 STEP  333 H= 333:00 +CPU= 41.481
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:19:20 STEP  334 H= 334:00 +CPU= 27.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:19:51 STEP  335 H= 335:00 +CPU= 27.123
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:20:48 STEP  336 H= 336:00 +CPU= 48.146
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMGGhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMSHhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996/ICMUAhq0f+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630_13.zip
Uploading the intermediate file: upload_file_13.zip
  02:21:16 STEP  337 H= 337:00 +CPU= 26.063
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:21:49 STEP  338 H= 338:00 +CPU= 27.115
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:22:37 STEP  339 H= 339:00 +CPU= 41.873
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:23:10 STEP  340 H= 340:00 +CPU= 27.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:23:42 STEP  341 H= 341:00 +CPU= 26.578
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.721] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4745MB (heap), 4266MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.722] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.722] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.722] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.723] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4745 MB (maxheap), 4266 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            CALL_SL_HEAP 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             CALL_SL 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              >OMP-CALL_SL     2        (1005) 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               LAPINEB 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                LARCINB 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 LAITRE_GMV 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035258:1672113178:19350.729] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                  LAITRI 
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035259:1672113179:19351.729] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:FinnWilts:1:1:11511:11511] [20221227:035259:1672113179:19351.730] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000DD21F8  laitri_                   317  laitri.F90
oifs_43r3_model.e  0000000000DDBEE4  laitre_gmv_               143  laitre_gmv.F90
oifs_43r3_model.e  0000000000DBD8F4  larcinb_                  301  larcinb.F90
oifs_43r3_model.e  0000000000D7EBA8  lapineb_                  451  lapineb.F90
oifs_43r3_model.e  0000000000D7B935  call_sl_                  363  call_sl.F90
oifs_43r3_model.e  0000000000D784FD  call_sl_heap_              94  call_sl_heap.F90
oifs_43r3_model.e  0000000000BF9958  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2012050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0352
(argv4) batchid: 981
(argv5) wuid: 12197996
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197996
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c99fbe7ad139f4528aeeba54a248d940 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0352_2012050100_123_981_12197996.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0352_2012050100_123_981_12197996.zip
ic_ancil_file: ic_ancil_12197996
ifsdata_file: ifsdata_12197996
climate_data_file: clim_data_12197996
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_fe74f80180ad967f040aba5e9dfbc2b0 to: /var/lib/boinc-client/slots/0/ic_ancil_12197996.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12197996.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ef28437e5391d27cce99fa9408634638 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12197996.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12197996.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12197996.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12197996.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12197996.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12197996.xml
-- Model is restarting --
Adjusting last_iter, 342, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12197996.xml
last_cpu_time: 11216
upload_file_number: 14
last_iter: 335
last_upload: 1209600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0352_2012050100_123_981_12197996_0_r866069630
The child process has been launched with process id: 1889
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:FinnWilts:1:1:1889:1889] [20221227:103224:1672137144:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:33:30 STEP  336 H= 336:00 +CPU= 50.484
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:34:07 STEP  337 H= 337:00 +CPU= 28.743
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:34:41 STEP  338 H= 338:00 +CPU= 28.361
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:35:29 STEP  339 H= 339:00 +CPU= 41.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:36:01 STEP  340 H= 340:00 +CPU= 27.958
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:36:31 STEP  341 H= 341:00 +CPU= 26.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:37:20 STEP  342 H= 342:00 +CPU= 42.350
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:37:49 STEP  343 H= 343:00 +CPU= 26.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:38:21 STEP  344 H= 344:00 +CPU= 27.193
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:39:08 STEP  345 H= 345:00 +CPU= 41.338
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:39:43 STEP  346 H= 346:00 +CPU= 28.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:40:15 STEP  347 H= 347:00 +CPU= 27.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:57:58 1514051 22284068 oifs_43r3_ps_0352_2012050100_123_981_12197996_0 864,000 7,875 0.0091


©2024 climateprediction.net