climateprediction.net home page
Task 22445923

Task 22445923

Name oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0
Workunit 12292318
Created 13 Jun 2024, 1:48:23 UTC
Sent 13 Jun 2024, 16:56:02 UTC
Report deadline 12 Aug 2024, 16:56:02 UTC
Received 14 Jun 2024, 18:18:12 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1537687
Run time 16 hours 24 min 30 sec
CPU time 13 hours 18 min 23 sec
Validate state Invalid
Credit 2,635.55
Device peak FLOPS 7.20 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,002.23 MB
Peak swap size 5,981.07 MB
Peak disk usage 2,046.93 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
oving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001440
  12:05:20 STEP 1441 H= 360:15 +CPU= 28.541
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001440
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_12.zip
Uploading the intermediate file: upload_file_12.zip
  12:05:48 STEP 1442 H= 360:30 +CPU= 27.790
  12:06:17 STEP 1443 H= 360:45 +CPU= 28.566
  12:06:47 STEP 1444 H= 361:00 +CPU= 29.258
  12:07:17 STEP 1445 H= 361:15 +CPU= 28.599
  12:07:45 STEP 1446 H= 361:30 +CPU= 28.334
  12:08:15 STEP 1447 H= 361:45 +CPU= 28.343
  12:08:45 STEP 1448 H= 362:00 +CPU= 28.955
  12:09:13 STEP 1449 H= 362:15 +CPU= 28.522
  12:09:44 STEP 1450 H= 362:30 +CPU= 28.934
  12:10:13 STEP 1451 H= 362:45 +CPU= 28.549
  12:10:50 STEP 1452 H= 363:00 +CPU= 36.152
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001452
  12:11:19 STEP 1453 H= 363:15 +CPU= 28.179
  12:11:48 STEP 1454 H= 363:30 +CPU= 28.788
  12:12:16 STEP 1455 H= 363:45 +CPU= 27.776
  12:12:46 STEP 1456 H= 364:00 +CPU= 29.793
  12:13:15 STEP 1457 H= 364:15 +CPU= 28.077
  12:13:43 STEP 1458 H= 364:30 +CPU= 28.119
  12:14:11 STEP 1459 H= 364:45 +CPU= 28.534
  12:14:42 STEP 1460 H= 365:00 +CPU= 30.130
  12:15:10 STEP 1461 H= 365:15 +CPU= 27.610
  12:15:40 STEP 1462 H= 365:30 +CPU= 29.199
  12:16:09 STEP 1463 H= 365:45 +CPU= 28.958
  12:16:45 STEP 1464 H= 366:00 +CPU= 35.294
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001464
  12:17:14 STEP 1465 H= 366:15 +CPU= 28.989
  12:17:43 STEP 1466 H= 366:30 +CPU= 29.245
  12:18:12 STEP 1467 H= 366:45 +CPU= 28.532
  12:18:42 STEP 1468 H= 367:00 +CPU= 29.001
  12:19:11 STEP 1469 H= 367:15 +CPU= 28.360
  12:19:39 STEP 1470 H= 367:30 +CPU= 28.409
  12:20:09 STEP 1471 H= 367:45 +CPU= 28.505
  12:20:38 STEP 1472 H= 368:00 +CPU= 28.995
  12:21:08 STEP 1473 H= 368:15 +CPU= 29.439
  12:21:38 STEP 1474 H= 368:30 +CPU= 29.378
  12:22:06 STEP 1475 H= 368:45 +CPU= 28.303
  12:22:42 STEP 1476 H= 369:00 +CPU= 35.336
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001476
  12:23:12 STEP 1477 H= 369:15 +CPU= 28.645
  12:23:41 STEP 1478 H= 369:30 +CPU= 29.292
  12:24:11 STEP 1479 H= 369:45 +CPU= 28.953
  12:24:40 STEP 1480 H= 370:00 +CPU= 29.348
  12:25:09 STEP 1481 H= 370:15 +CPU= 28.807
  12:25:38 STEP 1482 H= 370:30 +CPU= 29.153
  12:26:07 STEP 1483 H= 370:45 +CPU= 28.544
  12:26:37 STEP 1484 H= 371:00 +CPU= 29.681
  12:27:06 STEP 1485 H= 371:15 +CPU= 29.005
  12:27:36 STEP 1486 H= 371:30 +CPU= 28.982
  12:28:05 STEP 1487 H= 371:45 +CPU= 28.708
  12:28:45 STEP 1488 H= 372:00 +CPU= 38.847
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001488
  12:29:14 STEP 1489 H= 372:15 +CPU= 28.612
  12:29:43 STEP 1490 H= 372:30 +CPU= 28.579
  12:30:11 STEP 1491 H= 372:45 +CPU= 28.136
  12:30:40 STEP 1492 H= 373:00 +CPU= 29.481
  12:31:10 STEP 1493 H= 373:15 +CPU= 28.831
  12:31:38 STEP 1494 H= 373:30 +CPU= 28.127
  12:32:06 STEP 1495 H= 373:45 +CPU= 27.929
  12:32:36 STEP 1496 H= 374:00 +CPU= 29.515
  12:33:05 STEP 1497 H= 374:15 +CPU= 29.172
  12:33:34 STEP 1498 H= 374:30 +CPU= 28.879
  12:34:02 STEP 1499 H= 374:45 +CPU= 27.953
  12:34:39 STEP 1500 H= 375:00 +CPU= 35.987
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001500
  12:35:08 STEP 1501 H= 375:15 +CPU= 28.623
  12:35:36 STEP 1502 H= 375:30 +CPU= 28.405
  12:36:05 STEP 1503 H= 375:45 +CPU= 28.672
  12:36:35 STEP 1504 H= 376:00 +CPU= 29.253
  12:37:04 STEP 1505 H= 376:15 +CPU= 28.588
  12:37:33 STEP 1506 H= 376:30 +CPU= 28.977
  12:38:03 STEP 1507 H= 376:45 +CPU= 28.769
  12:38:32 STEP 1508 H= 377:00 +CPU= 29.337
  12:39:02 STEP 1509 H= 377:15 +CPU= 28.774
  12:39:31 STEP 1510 H= 377:30 +CPU= 29.156
  12:40:00 STEP 1511 H= 377:45 +CPU= 28.747
  12:40:36 STEP 1512 H= 378:00 +CPU= 35.539
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001512
  12:41:07 STEP 1513 H= 378:15 +CPU= 28.581
  12:41:36 STEP 1514 H= 378:30 +CPU= 28.827
  12:42:05 STEP 1515 H= 378:45 +CPU= 28.343
  12:42:35 STEP 1516 H= 379:00 +CPU= 29.788
  12:43:06 STEP 1517 H= 379:15 +CPU= 29.648
  12:43:36 STEP 1518 H= 379:30 +CPU= 29.366
  12:44:05 STEP 1519 H= 379:45 +CPU= 28.012
  12:44:35 STEP 1520 H= 380:00 +CPU= 28.955
  12:45:05 STEP 1521 H= 380:15 +CPU= 28.833
  12:45:35 STEP 1522 H= 380:30 +CPU= 28.858
  12:46:04 STEP 1523 H= 380:45 +CPU= 28.782
  12:46:40 STEP 1524 H= 381:00 +CPU= 36.385
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001524
  12:47:09 STEP 1525 H= 381:15 +CPU= 28.736
  12:47:38 STEP 1526 H= 381:30 +CPU= 28.663
  12:48:07 STEP 1527 H= 381:45 +CPU= 28.023
  12:48:37 STEP 1528 H= 382:00 +CPU= 29.011
  12:49:05 STEP 1529 H= 382:15 +CPU= 28.369
  12:49:36 STEP 1530 H= 382:30 +CPU= 29.214
  12:50:05 STEP 1531 H= 382:45 +CPU= 28.600
  12:50:34 STEP 1532 H= 383:00 +CPU= 29.361
  12:51:04 STEP 1533 H= 383:15 +CPU= 29.079
  12:51:34 STEP 1534 H= 383:30 +CPU= 28.972
  12:52:03 STEP 1535 H= 383:45 +CPU= 28.536
  12:52:42 STEP 1536 H= 384:00 +CPU= 39.129
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001536
  12:53:11 STEP 1537 H= 384:15 +CPU= 28.731
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001536
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_13.zip
Uploading the intermediate file: upload_file_13.zip
  12:53:41 STEP 1538 H= 384:30 +CPU= 28.624
  12:54:09 STEP 1539 H= 384:45 +CPU= 28.253
  12:54:38 STEP 1540 H= 385:00 +CPU= 28.588
  12:55:08 STEP 1541 H= 385:15 +CPU= 29.204
  12:55:37 STEP 1542 H= 385:30 +CPU= 28.333
  12:56:05 STEP 1543 H= 385:45 +CPU= 27.853
  12:56:34 STEP 1544 H= 386:00 +CPU= 29.193
  12:57:03 STEP 1545 H= 386:15 +CPU= 28.345
  12:57:33 STEP 1546 H= 386:30 +CPU= 28.885
  12:58:01 STEP 1547 H= 386:45 +CPU= 28.262
  12:58:37 STEP 1548 H= 387:00 +CPU= 35.687
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001548
  12:59:05 STEP 1549 H= 387:15 +CPU= 28.232
  12:59:34 STEP 1550 H= 387:30 +CPU= 28.469
  13:00:02 STEP 1551 H= 387:45 +CPU= 28.150
  13:00:31 STEP 1552 H= 388:00 +CPU= 28.866
  13:01:00 STEP 1553 H= 388:15 +CPU= 28.496
  13:01:29 STEP 1554 H= 388:30 +CPU= 28.255
  13:01:57 STEP 1555 H= 388:45 +CPU= 28.386
  13:02:27 STEP 1556 H= 389:00 +CPU= 28.605
  13:02:56 STEP 1557 H= 389:15 +CPU= 28.814
  13:03:25 STEP 1558 H= 389:30 +CPU= 28.683
  13:03:53 STEP 1559 H= 389:45 +CPU= 28.368
  13:04:29 STEP 1560 H= 390:00 +CPU= 35.246
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001560
  13:04:59 STEP 1561 H= 390:15 +CPU= 28.720
  13:05:29 STEP 1562 H= 390:30 +CPU= 28.765
  13:05:57 STEP 1563 H= 390:45 +CPU= 28.235
  13:06:27 STEP 1564 H= 391:00 +CPU= 28.449
  13:06:57 STEP 1565 H= 391:15 +CPU= 28.832
  13:07:27 STEP 1566 H= 391:30 +CPU= 29.548
  13:07:56 STEP 1567 H= 391:45 +CPU= 28.387
  13:08:26 STEP 1568 H= 392:00 +CPU= 28.795
  13:08:56 STEP 1569 H= 392:15 +CPU= 29.281
  13:09:27 STEP 1570 H= 392:30 +CPU= 29.114
  13:09:57 STEP 1571 H= 392:45 +CPU= 29.139
  13:10:33 STEP 1572 H= 393:00 +CPU= 35.339
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001572
  13:11:03 STEP 1573 H= 393:15 +CPU= 29.319
  13:11:33 STEP 1574 H= 393:30 +CPU= 28.999
  13:12:02 STEP 1575 H= 393:45 +CPU= 28.400
  13:12:32 STEP 1576 H= 394:00 +CPU= 28.753
  13:13:01 STEP 1577 H= 394:15 +CPU= 28.884
  13:13:31 STEP 1578 H= 394:30 +CPU= 29.022
  13:14:00 STEP 1579 H= 394:45 +CPU= 28.985
  13:14:30 STEP 1580 H= 395:00 +CPU= 29.875
  13:15:00 STEP 1581 H= 395:15 +CPU= 29.269
  13:15:30 STEP 1582 H= 395:30 +CPU= 29.561
  13:16:00 STEP 1583 H= 395:45 +CPU= 29.105
  13:16:39 STEP 1584 H= 396:00 +CPU= 39.305
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001584
  13:17:09 STEP 1585 H= 396:15 +CPU= 28.386
  13:17:39 STEP 1586 H= 396:30 +CPU= 28.760
  13:18:07 STEP 1587 H= 396:45 +CPU= 28.227
  13:18:37 STEP 1588 H= 397:00 +CPU= 29.143
  13:19:06 STEP 1589 H= 397:15 +CPU= 28.780
  13:19:35 STEP 1590 H= 397:30 +CPU= 28.810
  13:20:04 STEP 1591 H= 397:45 +CPU= 29.150
  13:20:35 STEP 1592 H= 398:00 +CPU= 29.416
  13:21:04 STEP 1593 H= 398:15 +CPU= 28.763
  13:21:33 STEP 1594 H= 398:30 +CPU= 29.160
  13:22:02 STEP 1595 H= 398:45 +CPU= 28.160
  13:22:38 STEP 1596 H= 399:00 +CPU= 35.813
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001596
  13:23:06 STEP 1597 H= 399:15 +CPU= 28.342
  13:23:35 STEP 1598 H= 399:30 +CPU= 28.306
  13:24:03 STEP 1599 H= 399:45 +CPU= 28.356
  13:24:33 STEP 1600 H= 400:00 +CPU= 29.131
  13:25:02 STEP 1601 H= 400:15 +CPU= 28.791
  13:25:30 STEP 1602 H= 400:30 +CPU= 27.902
  13:26:00 STEP 1603 H= 400:45 +CPU= 28.526
  13:26:30 STEP 1604 H= 401:00 +CPU= 30.147
  13:26:59 STEP 1605 H= 401:15 +CPU= 28.486
  13:27:27 STEP 1606 H= 401:30 +CPU= 28.353
  13:27:56 STEP 1607 H= 401:45 +CPU= 28.518
  13:28:31 STEP 1608 H= 402:00 +CPU= 34.375
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001608
  13:28:57 STEP 1609 H= 402:15 +CPU= 26.005
  13:29:22 STEP 1610 H= 402:30 +CPU= 25.250
  13:29:49 STEP 1611 H= 402:45 +CPU= 26.880
  13:30:16 STEP 1612 H= 403:00 +CPU= 26.850
  13:30:40 STEP 1613 H= 403:15 +CPU= 24.293
  13:31:07 STEP 1614 H= 403:30 +CPU= 26.474
  13:31:34 STEP 1615 H= 403:45 +CPU= 27.253
  13:32:04 STEP 1616 H= 404:00 +CPU= 28.638
  13:32:33 STEP 1617 H= 404:15 +CPU= 29.145
  13:33:03 STEP 1618 H= 404:30 +CPU= 28.869
  13:33:32 STEP 1619 H= 404:45 +CPU= 28.991
  13:34:08 STEP 1620 H= 405:00 +CPU= 35.572
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001620
  13:34:37 STEP 1621 H= 405:15 +CPU= 29.010
  13:35:06 STEP 1622 H= 405:30 +CPU= 28.850
  13:35:34 STEP 1623 H= 405:45 +CPU= 28.107
  13:36:03 STEP 1624 H= 406:00 +CPU= 29.274
  13:36:31 STEP 1625 H= 406:15 +CPU= 28.004
  13:37:00 STEP 1626 H= 406:30 +CPU= 28.442
  13:37:29 STEP 1627 H= 406:45 +CPU= 28.745
  13:37:59 STEP 1628 H= 407:00 +CPU= 29.912
  13:38:28 STEP 1629 H= 407:15 +CPU= 29.152
  13:38:58 STEP 1630 H= 407:30 +CPU= 28.898
  13:39:27 STEP 1631 H= 407:45 +CPU= 28.599
  13:40:07 STEP 1632 H= 408:00 +CPU= 39.803
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001632
  13:40:35 STEP 1633 H= 408:15 +CPU= 28.699
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001632
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_14.zip
Uploading the intermediate file: upload_file_14.zip
  13:41:05 STEP 1634 H= 408:30 +CPU= 28.557
Uploading trickle at timestep: 1469700
  13:41:33 STEP 1635 H= 408:45 +CPU= 28.506
  13:42:02 STEP 1636 H= 409:00 +CPU= 28.614
  13:42:31 STEP 1637 H= 409:15 +CPU= 28.789
  13:43:00 STEP 1638 H= 409:30 +CPU= 28.352
  13:43:28 STEP 1639 H= 409:45 +CPU= 28.255
  13:43:59 STEP 1640 H= 410:00 +CPU= 29.465
  13:44:29 STEP 1641 H= 410:15 +CPU= 29.494
  13:44:58 STEP 1642 H= 410:30 +CPU= 28.327
  13:45:27 STEP 1643 H= 410:45 +CPU= 28.221
  13:46:02 STEP 1644 H= 411:00 +CPU= 34.535
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001644
  13:46:31 STEP 1645 H= 411:15 +CPU= 28.657
  13:47:00 STEP 1646 H= 411:30 +CPU= 28.704
  13:47:28 STEP 1647 H= 411:45 +CPU= 28.513
  13:47:58 STEP 1648 H= 412:00 +CPU= 29.551
  13:48:26 STEP 1649 H= 412:15 +CPU= 28.287
  13:48:55 STEP 1650 H= 412:30 +CPU= 28.874
  13:49:25 STEP 1651 H= 412:45 +CPU= 28.861
  13:49:56 STEP 1652 H= 413:00 +CPU= 29.531
  13:50:25 STEP 1653 H= 413:15 +CPU= 28.884
  13:50:54 STEP 1654 H= 413:30 +CPU= 29.078
  13:51:23 STEP 1655 H= 413:45 +CPU= 28.747
  13:51:59 STEP 1656 H= 414:00 +CPU= 36.057
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001656
  13:52:28 STEP 1657 H= 414:15 +CPU= 28.441
  13:52:56 STEP 1658 H= 414:30 +CPU= 28.558
  13:53:25 STEP 1659 H= 414:45 +CPU= 28.202
  13:53:54 STEP 1660 H= 415:00 +CPU= 29.311
  13:54:22 STEP 1661 H= 415:15 +CPU= 28.164
  13:54:51 STEP 1662 H= 415:30 +CPU= 28.522
  13:55:20 STEP 1663 H= 415:45 +CPU= 28.601
  13:55:50 STEP 1664 H= 416:00 +CPU= 28.820
  13:56:20 STEP 1665 H= 416:15 +CPU= 28.967
  13:56:49 STEP 1666 H= 416:30 +CPU= 28.552
  13:57:18 STEP 1667 H= 416:45 +CPU= 28.811
  13:57:55 STEP 1668 H= 417:00 +CPU= 36.547
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001668
  13:58:25 STEP 1669 H= 417:15 +CPU= 29.412
  13:58:54 STEP 1670 H= 417:30 +CPU= 28.501
  13:59:23 STEP 1671 H= 417:45 +CPU= 28.736
  13:59:52 STEP 1672 H= 418:00 +CPU= 29.470
  14:00:21 STEP 1673 H= 418:15 +CPU= 28.904
  14:00:50 STEP 1674 H= 418:30 +CPU= 28.574
  14:01:19 STEP 1675 H= 418:45 +CPU= 28.395
  14:01:49 STEP 1676 H= 419:00 +CPU= 29.485
  14:02:18 STEP 1677 H= 419:15 +CPU= 29.140
  14:02:47 STEP 1678 H= 419:30 +CPU= 28.576
  14:03:17 STEP 1679 H= 419:45 +CPU= 28.874
  14:03:57 STEP 1680 H= 420:00 +CPU= 40.006
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001680
  14:04:26 STEP 1681 H= 420:15 +CPU= 28.411
  14:04:50 STEP 1682 H= 420:30 +CPU= 23.799
  14:05:11 STEP 1683 H= 420:45 +CPU= 21.399
  14:05:33 STEP 1684 H= 421:00 +CPU= 21.621
  14:05:58 STEP 1685 H= 421:15 +CPU= 25.773
  14:06:25 STEP 1686 H= 421:30 +CPU= 26.178
  14:06:50 STEP 1687 H= 421:45 +CPU= 24.962
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a32v
(argv4) batchid: 1018
(argv5) wuid: 12292318
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6192e1ad94e725e828ca83b22cc5f5d6 to: /var/lib/boinc-client/slots/6/oifs_43r3_bl_a32v_2016092300_20_1018_12292318.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_bl_a32v_2016092300_20_1018_12292318.zip
ic_ancil_file: ic_ancil_12292318
ifsdata_file: ifsdata_12292318
climate_data_file: clim_data_12292318
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/6/ic_ancil_12292318.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12292318.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_14117ae2af5d095b25fc51a7881eaaa1 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12292318.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12292318.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12292318.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12292318.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12292318.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12292318.xml
-- Model is restarting --
Adjusting last_iter, 1688, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12292318.xml
last_cpu_time: 40837
upload_file_number: 15
last_iter: 1681
last_upload: 1469700
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482
The child process has been launched with process id: 854691
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854691:854691] [20240614:151646:1718378206:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:17:16 STEP 1680 H= 420:00 +CPU= 25.635
  15:17:41 STEP 1681 H= 420:15 +CPU= 25.229
  15:18:05 STEP 1682 H= 420:30 +CPU= 24.058
  15:18:29 STEP 1683 H= 420:45 +CPU= 24.223
  15:18:55 STEP 1684 H= 421:00 +CPU= 25.905
  15:19:24 STEP 1685 H= 421:15 +CPU= 27.996
  15:19:53 STEP 1686 H= 421:30 +CPU= 28.933
  15:20:22 STEP 1687 H= 421:45 +CPU= 27.919
  15:20:51 STEP 1688 H= 422:00 +CPU= 28.910
  15:21:20 STEP 1689 H= 422:15 +CPU= 28.767
  15:21:49 STEP 1690 H= 422:30 +CPU= 28.498
  15:22:18 STEP 1691 H= 422:45 +CPU= 28.575
  15:22:54 STEP 1692 H= 423:00 +CPU= 35.884
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001692
  15:23:24 STEP 1693 H= 423:15 +CPU= 28.965
  15:23:53 STEP 1694 H= 423:30 +CPU= 28.429
  15:24:22 STEP 1695 H= 423:45 +CPU= 28.603
  15:24:51 STEP 1696 H= 424:00 +CPU= 29.127
  15:25:20 STEP 1697 H= 424:15 +CPU= 28.586
  15:25:51 STEP 1698 H= 424:30 +CPU= 29.442
  15:26:20 STEP 1699 H= 424:45 +CPU= 28.687
  15:26:50 STEP 1700 H= 425:00 +CPU= 29.482
  15:27:19 STEP 1701 H= 425:15 +CPU= 29.046
  15:27:49 STEP 1702 H= 425:30 +CPU= 29.146
  15:28:17 STEP 1703 H= 425:45 +CPU= 28.578
  15:28:54 STEP 1704 H= 426:00 +CPU= 35.945
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001704
  15:29:24 STEP 1705 H= 426:15 +CPU= 29.029
  15:29:53 STEP 1706 H= 426:30 +CPU= 28.532
  15:30:21 STEP 1707 H= 426:45 +CPU= 28.446
  15:30:51 STEP 1708 H= 427:00 +CPU= 29.150
  15:31:20 STEP 1709 H= 427:15 +CPU= 28.840
  15:31:49 STEP 1710 H= 427:30 +CPU= 28.970
  15:32:18 STEP 1711 H= 427:45 +CPU= 28.838
  15:32:48 STEP 1712 H= 428:00 +CPU= 29.790
  15:33:17 STEP 1713 H= 428:15 +CPU= 28.725
  15:33:46 STEP 1714 H= 428:30 +CPU= 28.330
  15:34:14 STEP 1715 H= 428:45 +CPU= 28.420
  15:34:49 STEP 1716 H= 429:00 +CPU= 34.497
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001716
  15:35:19 STEP 1717 H= 429:15 +CPU= 28.984
  15:35:47 STEP 1718 H= 429:30 +CPU= 28.521
  15:36:17 STEP 1719 H= 429:45 +CPU= 28.642
  15:36:45 STEP 1720 H= 430:00 +CPU= 28.618
  15:37:15 STEP 1721 H= 430:15 +CPU= 29.182
  15:37:45 STEP 1722 H= 430:30 +CPU= 29.085
  15:38:15 STEP 1723 H= 430:45 +CPU= 29.262
  15:38:44 STEP 1724 H= 431:00 +CPU= 29.848
  15:39:14 STEP 1725 H= 431:15 +CPU= 28.806
  15:39:43 STEP 1726 H= 431:30 +CPU= 28.897
  15:40:13 STEP 1727 H= 431:45 +CPU= 29.265
  15:40:52 STEP 1728 H= 432:00 +CPU= 38.667
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001728
  15:41:21 STEP 1729 H= 432:15 +CPU= 28.660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001728
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_15.zip
Uploading the intermediate file: upload_file_15.zip
  15:41:49 STEP 1730 H= 432:30 +CPU= 27.782
  15:42:18 STEP 1731 H= 432:45 +CPU= 28.315
  15:42:47 STEP 1732 H= 433:00 +CPU= 28.660
  15:43:16 STEP 1733 H= 433:15 +CPU= 28.344
  15:43:45 STEP 1734 H= 433:30 +CPU= 28.932
  15:44:15 STEP 1735 H= 433:45 +CPU= 29.014
  15:44:43 STEP 1736 H= 434:00 +CPU= 28.488
  15:45:13 STEP 1737 H= 434:15 +CPU= 28.179
  15:45:41 STEP 1738 H= 434:30 +CPU= 28.589
  15:46:11 STEP 1739 H= 434:45 +CPU= 28.451
  15:46:46 STEP 1740 H= 435:00 +CPU= 34.456
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001740
  15:47:15 STEP 1741 H= 435:15 +CPU= 28.300
  15:47:43 STEP 1742 H= 435:30 +CPU= 27.928
  15:48:12 STEP 1743 H= 435:45 +CPU= 28.388
  15:48:42 STEP 1744 H= 436:00 +CPU= 28.876
  15:49:11 STEP 1745 H= 436:15 +CPU= 28.826
  15:49:40 STEP 1746 H= 436:30 +CPU= 28.777
  15:50:09 STEP 1747 H= 436:45 +CPU= 28.861
  15:50:39 STEP 1748 H= 437:00 +CPU= 29.049
  15:51:07 STEP 1749 H= 437:15 +CPU= 28.520
  15:51:37 STEP 1750 H= 437:30 +CPU= 29.153
  15:52:08 STEP 1751 H= 437:45 +CPU= 28.973
  15:52:45 STEP 1752 H= 438:00 +CPU= 35.946
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001752
  15:53:14 STEP 1753 H= 438:15 +CPU= 28.478
  15:53:43 STEP 1754 H= 438:30 +CPU= 28.815
  15:54:11 STEP 1755 H= 438:45 +CPU= 28.423
  15:54:40 STEP 1756 H= 439:00 +CPU= 29.029
  15:55:10 STEP 1757 H= 439:15 +CPU= 28.699
  15:55:38 STEP 1758 H= 439:30 +CPU= 28.634
  15:56:08 STEP 1759 H= 439:45 +CPU= 29.052
  15:56:38 STEP 1760 H= 440:00 +CPU= 29.335
  15:57:07 STEP 1761 H= 440:15 +CPU= 28.596
  15:57:37 STEP 1762 H= 440:30 +CPU= 29.227
  15:58:06 STEP 1763 H= 440:45 +CPU= 29.083
  15:58:42 STEP 1764 H= 441:00 +CPU= 35.312
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001764
  15:59:11 STEP 1765 H= 441:15 +CPU= 28.343
  15:59:40 STEP 1766 H= 441:30 +CPU= 28.823
  16:00:09 STEP 1767 H= 441:45 +CPU= 28.969
  16:00:39 STEP 1768 H= 442:00 +CPU= 29.377
  16:01:08 STEP 1769 H= 442:15 +CPU= 29.164
  16:01:37 STEP 1770 H= 442:30 +CPU= 29.163
  16:02:07 STEP 1771 H= 442:45 +CPU= 29.145
  16:02:37 STEP 1772 H= 443:00 +CPU= 29.844
  16:03:06 STEP 1773 H= 443:15 +CPU= 29.032
  16:03:35 STEP 1774 H= 443:30 +CPU= 28.707
  16:04:05 STEP 1775 H= 443:45 +CPU= 29.324
  16:04:44 STEP 1776 H= 444:00 +CPU= 39.358
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001776
  16:05:13 STEP 1777 H= 444:15 +CPU= 27.553
  16:05:43 STEP 1778 H= 444:30 +CPU= 28.806
  16:06:11 STEP 1779 H= 444:45 +CPU= 28.672
  16:06:41 STEP 1780 H= 445:00 +CPU= 29.035
  16:07:10 STEP 1781 H= 445:15 +CPU= 28.859
  16:07:40 STEP 1782 H= 445:30 +CPU= 27.733
  16:08:10 STEP 1783 H= 445:45 +CPU= 29.144
  16:08:39 STEP 1784 H= 446:00 +CPU= 28.720
  16:09:08 STEP 1785 H= 446:15 +CPU= 28.808
  16:09:38 STEP 1786 H= 446:30 +CPU= 29.147
  16:10:06 STEP 1787 H= 446:45 +CPU= 28.541
  16:10:42 STEP 1788 H= 447:00 +CPU= 35.496
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001788
  16:11:11 STEP 1789 H= 447:15 +CPU= 28.746
  16:11:40 STEP 1790 H= 447:30 +CPU= 28.786
  16:12:09 STEP 1791 H= 447:45 +CPU= 28.939
  16:12:38 STEP 1792 H= 448:00 +CPU= 28.481
  16:13:07 STEP 1793 H= 448:15 +CPU= 29.014
  16:13:37 STEP 1794 H= 448:30 +CPU= 29.320
  16:14:06 STEP 1795 H= 448:45 +CPU= 28.987
  16:14:36 STEP 1796 H= 449:00 +CPU= 29.500
  16:15:06 STEP 1797 H= 449:15 +CPU= 29.087
  16:15:35 STEP 1798 H= 449:30 +CPU= 28.817
  16:16:05 STEP 1799 H= 449:45 +CPU= 29.078
  16:16:40 STEP 1800 H= 450:00 +CPU= 35.626
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001800
  16:17:09 STEP 1801 H= 450:15 +CPU= 28.655
  16:17:39 STEP 1802 H= 450:30 +CPU= 29.152
  16:18:07 STEP 1803 H= 450:45 +CPU= 28.387
  16:18:37 STEP 1804 H= 451:00 +CPU= 29.362
  16:19:06 STEP 1805 H= 451:15 +CPU= 28.131
  16:19:36 STEP 1806 H= 451:30 +CPU= 29.550
  16:20:05 STEP 1807 H= 451:45 +CPU= 28.999
  16:20:34 STEP 1808 H= 452:00 +CPU= 28.929
  16:21:02 STEP 1809 H= 452:15 +CPU= 28.344
  16:21:31 STEP 1810 H= 452:30 +CPU= 28.655
  16:22:00 STEP 1811 H= 452:45 +CPU= 29.011
  16:22:35 STEP 1812 H= 453:00 +CPU= 35.390
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001812
  16:23:04 STEP 1813 H= 453:15 +CPU= 28.617
  16:23:34 STEP 1814 H= 453:30 +CPU= 29.383
  16:24:02 STEP 1815 H= 453:45 +CPU= 28.365
  16:24:32 STEP 1816 H= 454:00 +CPU= 29.436
  16:25:00 STEP 1817 H= 454:15 +CPU= 28.371
  16:25:29 STEP 1818 H= 454:30 +CPU= 28.921
  16:25:58 STEP 1819 H= 454:45 +CPU= 28.800
  16:26:28 STEP 1820 H= 455:00 +CPU= 29.296
  16:26:57 STEP 1821 H= 455:15 +CPU= 28.927
  16:27:26 STEP 1822 H= 455:30 +CPU= 28.795
  16:27:56 STEP 1823 H= 455:45 +CPU= 29.055
  16:28:35 STEP 1824 H= 456:00 +CPU= 38.696
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001824
  16:29:03 STEP 1825 H= 456:15 +CPU= 28.203
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_16.zip
Uploading the intermediate file: upload_file_16.zip
  16:29:32 STEP 1826 H= 456:30 +CPU= 27.862
  16:30:01 STEP 1827 H= 456:45 +CPU= 28.479
  16:30:30 STEP 1828 H= 457:00 +CPU= 27.887
  16:30:59 STEP 1829 H= 457:15 +CPU= 28.589
  16:31:28 STEP 1830 H= 457:30 +CPU= 28.591
  16:31:57 STEP 1831 H= 457:45 +CPU= 28.754
  16:32:27 STEP 1832 H= 458:00 +CPU= 29.171
  16:32:56 STEP 1833 H= 458:15 +CPU= 28.615
  16:33:24 STEP 1834 H= 458:30 +CPU= 28.316
  16:33:54 STEP 1835 H= 458:45 +CPU= 28.822
  16:34:29 STEP 1836 H= 459:00 +CPU= 34.873
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001836
  16:34:59 STEP 1837 H= 459:15 +CPU= 29.207
  16:35:27 STEP 1838 H= 459:30 +CPU= 28.212
  16:35:56 STEP 1839 H= 459:45 +CPU= 28.474
  16:36:25 STEP 1840 H= 460:00 +CPU= 29.511
  16:36:54 STEP 1841 H= 460:15 +CPU= 28.590
  16:37:23 STEP 1842 H= 460:30 +CPU= 28.806
  16:37:52 STEP 1843 H= 460:45 +CPU= 28.702
  16:38:22 STEP 1844 H= 461:00 +CPU= 29.601
  16:38:51 STEP 1845 H= 461:15 +CPU= 28.833
  16:39:19 STEP 1846 H= 461:30 +CPU= 28.317
  16:39:48 STEP 1847 H= 461:45 +CPU= 28.228
  16:40:23 STEP 1848 H= 462:00 +CPU= 35.416
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001848
  16:40:52 STEP 1849 H= 462:15 +CPU= 28.524
  16:41:20 STEP 1850 H= 462:30 +CPU= 28.520
  16:41:50 STEP 1851 H= 462:45 +CPU= 28.566
  16:42:19 STEP 1852 H= 463:00 +CPU= 28.888
  16:42:47 STEP 1853 H= 463:15 +CPU= 28.554
  16:43:17 STEP 1854 H= 463:30 +CPU= 29.336
  16:43:46 STEP 1855 H= 463:45 +CPU= 28.782
  16:44:15 STEP 1856 H= 464:00 +CPU= 29.192
  16:44:44 STEP 1857 H= 464:15 +CPU= 28.643
  16:45:13 STEP 1858 H= 464:30 +CPU= 28.675
  16:45:41 STEP 1859 H= 464:45 +CPU= 28.381
  16:46:16 STEP 1860 H= 465:00 +CPU= 35.008
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001860
  16:46:45 STEP 1861 H= 465:15 +CPU= 28.542
  16:47:14 STEP 1862 H= 465:30 +CPU= 28.713
  16:47:43 STEP 1863 H= 465:45 +CPU= 28.803
  16:48:12 STEP 1864 H= 466:00 +CPU= 29.300
  16:48:41 STEP 1865 H= 466:15 +CPU= 28.192
  16:49:11 STEP 1866 H= 466:30 +CPU= 28.931
  16:49:39 STEP 1867 H= 466:45 +CPU= 28.074
  16:50:09 STEP 1868 H= 467:00 +CPU= 29.111
  16:50:37 STEP 1869 H= 467:15 +CPU= 28.722
  16:51:06 STEP 1870 H= 467:30 +CPU= 28.345
  16:51:34 STEP 1871 H= 467:45 +CPU= 27.738
  16:52:13 STEP 1872 H= 468:00 +CPU= 38.768
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001872
  16:52:42 STEP 1873 H= 468:15 +CPU= 27.807
  16:53:11 STEP 1874 H= 468:30 +CPU= 28.505
  16:53:41 STEP 1875 H= 468:45 +CPU= 29.060
  16:54:11 STEP 1876 H= 469:00 +CPU= 28.464
  16:54:39 STEP 1877 H= 469:15 +CPU= 28.537
  16:55:08 STEP 1878 H= 469:30 +CPU= 27.762
  16:55:37 STEP 1879 H= 469:45 +CPU= 28.280
  16:56:07 STEP 1880 H= 470:00 +CPU= 29.571
  16:56:36 STEP 1881 H= 470:15 +CPU= 28.481
  16:57:05 STEP 1882 H= 470:30 +CPU= 28.375
  16:57:34 STEP 1883 H= 470:45 +CPU= 28.702
  16:58:10 STEP 1884 H= 471:00 +CPU= 35.721
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001884
  16:58:39 STEP 1885 H= 471:15 +CPU= 28.567
  16:59:08 STEP 1886 H= 471:30 +CPU= 28.932
  16:59:36 STEP 1887 H= 471:45 +CPU= 28.271
  17:00:05 STEP 1888 H= 472:00 +CPU= 28.943
  17:00:34 STEP 1889 H= 472:15 +CPU= 28.810
  17:01:03 STEP 1890 H= 472:30 +CPU= 28.722
  17:01:32 STEP 1891 H= 472:45 +CPU= 28.522
  17:02:01 STEP 1892 H= 473:00 +CPU= 28.905
  17:02:30 STEP 1893 H= 473:15 +CPU= 28.891
  17:02:59 STEP 1894 H= 473:30 +CPU= 28.911
  17:03:28 STEP 1895 H= 473:45 +CPU= 28.863
  17:04:03 STEP 1896 H= 474:00 +CPU= 35.127
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001896
  17:04:32 STEP 1897 H= 474:15 +CPU= 28.957
  17:05:01 STEP 1898 H= 474:30 +CPU= 28.982
  17:05:29 STEP 1899 H= 474:45 +CPU= 27.958
  17:05:58 STEP 1900 H= 475:00 +CPU= 28.970
  17:06:28 STEP 1901 H= 475:15 +CPU= 28.740
  17:06:57 STEP 1902 H= 475:30 +CPU= 29.136
  17:07:26 STEP 1903 H= 475:45 +CPU= 29.169
  17:07:55 STEP 1904 H= 476:00 +CPU= 29.015
  17:08:24 STEP 1905 H= 476:15 +CPU= 28.815
  17:08:53 STEP 1906 H= 476:30 +CPU= 28.715
  17:09:22 STEP 1907 H= 476:45 +CPU= 28.523
  17:09:57 STEP 1908 H= 477:00 +CPU= 35.140
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001908
  17:10:27 STEP 1909 H= 477:15 +CPU= 28.638
  17:10:56 STEP 1910 H= 477:30 +CPU= 28.677
  17:11:25 STEP 1911 H= 477:45 +CPU= 29.128
  17:11:55 STEP 1912 H= 478:00 +CPU= 29.203
  17:12:25 STEP 1913 H= 478:15 +CPU= 29.427
  17:12:54 STEP 1914 H= 478:30 +CPU= 29.351
  17:13:24 STEP 1915 H= 478:45 +CPU= 29.237
  17:13:54 STEP 1916 H= 479:00 +CPU= 29.611
  17:14:22 STEP 1917 H= 479:15 +CPU= 28.262
  17:14:52 STEP 1918 H= 479:30 +CPU= 28.962
  17:15:21 STEP 1919 H= 479:45 +CPU= 28.823
  17:15:58 STEP 1920 H= 480:00 +CPU= 36.177
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.4981314872E-22      I       0.1058659322E-06      I
ICloud fractionI      0.0000000000E+00      I       0.3122340498E-01      I
I  Relat. Hum. I      0.2437493897E+00      I       0.2889130582E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1373804365E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    293938244 BYTES IN      7 RECORDS TRANSFERRED IN    0.0008 SECONDS367422.8 Mbytes/s, TOTAL TIME=    0.0143( 5.6%)
 17:15:58 STEP 1920 H= 480:00 +CPU= 36.177
 END CNT3
 NSTEP =  1920 CNT0   000000000  26.678  26.678  27.378   0.974               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         7150/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    128    0       46      1044068     7150 ************    14.1     1.6    12.5     2.3    26.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 7151463.6 7151463.6    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 7147706.6 7147706.6     99.95      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  261    5684.4    5684.4     20.75      0.00
   9 SPCM     - SPECTRAL COMP.                     240    1011.5    1011.5      3.39      0.00
  10 SCAN2M - PHYSICS                              241   12623.6   12623.6     42.54      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 20     916.5     916.5      0.26      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 11      87.1      87.1      0.01      0.00
  17 GRIDFPOS IN CNT4                               20      33.4      33.4      0.01      0.00
  27 RESTART FILES - WRITING                         5    3665.7    3665.7      0.26      0.00
  28 RESTART FILES - READING                         1    3164.8    3164.8      0.04      0.00
  29 SU4FPOS IN CNT4                                20       0.5       0.5      0.00      0.00
  30 DYNFPOS IN CNT4                                20    5920.5    5920.5      1.66      0.00
  31 POSDDH IN STEPO                                 2      10.5      10.5      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    241    6017.4    6017.4     20.28      0.00
  39 SU0YOMB                                         1     318.0     318.0      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    241      31.4      31.4      0.11      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              240     623.8     623.8      2.09      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              240      78.6      78.6      0.26      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 11       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      524     432.3     432.3      3.17      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       500     249.5     249.5      1.74      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        500      83.4      83.4      0.58      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       524     115.9     115.9      0.85      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      47.7      47.7      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            524      70.7      70.7      0.52      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            500      40.3      40.3      0.28      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            524     148.9     148.9      1.09      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            500      65.5      65.5      0.46      0.00
 400 GSTATS                                      28864       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 26802       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     7151.5 CPU TIME    7065.9 VECTOR TIME     7065.9


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      241.8
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/6/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/6/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMSHh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292318/ICMUAh7zg+001848
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_17.zip
Uploading the final file: upload_file_17.zip
Uploading trickle at timestep: 1727100
17:19:27 (854683): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0_r1053615482_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 17:18:35 1537687 22445923 oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0 1,727,100 47,903 0.0277
14 Jun 2024 14:04:29 1537687 22445923 oifs_43r3_bl_a32v_2016092300_20_1018_12292318_0 1,469,700 39,276 0.0267


©2024 cpdn.org