climateprediction.net home page
Task 22442853

Task 22442853

Name oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0
Workunit 12289268
Created 12 Jun 2024, 21:19:55 UTC
Sent 13 Jun 2024, 21:55:19 UTC
Report deadline 12 Aug 2024, 21:55:19 UTC
Received 23 Jun 2024, 21:48:39 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1551031
Run time 17 hours 35 min 6 sec
CPU time 15 hours 25 min 47 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 4.04 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,025.38 MB
Peak swap size 5,981.12 MB
Peak disk usage 1,374.61 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
g+001512
  06:46:29 STEP 1513 H= 378:15 +CPU= 18.480
  06:46:48 STEP 1514 H= 378:30 +CPU= 18.416
  06:47:06 STEP 1515 H= 378:45 +CPU= 18.698
  06:47:25 STEP 1516 H= 379:00 +CPU= 18.986
  06:47:44 STEP 1517 H= 379:15 +CPU= 18.835
  06:48:03 STEP 1518 H= 379:30 +CPU= 18.770
  06:48:21 STEP 1519 H= 379:45 +CPU= 18.534
  06:48:40 STEP 1520 H= 380:00 +CPU= 18.986
  06:48:59 STEP 1521 H= 380:15 +CPU= 18.657
  06:49:18 STEP 1522 H= 380:30 +CPU= 18.798
  06:49:36 STEP 1523 H= 380:45 +CPU= 18.416
  06:49:59 STEP 1524 H= 381:00 +CPU= 23.091
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001524
  06:50:18 STEP 1525 H= 381:15 +CPU= 18.510
  06:50:36 STEP 1526 H= 381:30 +CPU= 18.470
  06:50:55 STEP 1527 H= 381:45 +CPU= 18.468
  06:51:14 STEP 1528 H= 382:00 +CPU= 18.809
  06:51:32 STEP 1529 H= 382:15 +CPU= 18.440
  06:51:51 STEP 1530 H= 382:30 +CPU= 18.844
  06:52:10 STEP 1531 H= 382:45 +CPU= 18.889
  06:52:29 STEP 1532 H= 383:00 +CPU= 19.406
  06:52:48 STEP 1533 H= 383:15 +CPU= 18.705
  06:53:07 STEP 1534 H= 383:30 +CPU= 18.702
  06:53:25 STEP 1535 H= 383:45 +CPU= 18.630
  06:53:51 STEP 1536 H= 384:00 +CPU= 25.623
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0_r884999839_15.zip
Uploading the intermediate file: upload_file_15.zip
  06:54:10 STEP 1537 H= 384:15 +CPU= 19.096
  06:54:29 STEP 1538 H= 384:30 +CPU= 18.864
  06:54:48 STEP 1539 H= 384:45 +CPU= 19.141
  06:55:07 STEP 1540 H= 385:00 +CPU= 19.379
  06:55:26 STEP 1541 H= 385:15 +CPU= 18.917
  06:55:45 STEP 1542 H= 385:30 +CPU= 18.789
  06:56:04 STEP 1543 H= 385:45 +CPU= 18.741
  06:56:23 STEP 1544 H= 386:00 +CPU= 19.197
  06:56:42 STEP 1545 H= 386:15 +CPU= 18.822
  06:57:01 STEP 1546 H= 386:30 +CPU= 19.005
  06:57:20 STEP 1547 H= 386:45 +CPU= 18.791
  06:57:43 STEP 1548 H= 387:00 +CPU= 23.365
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001548
  06:58:02 STEP 1549 H= 387:15 +CPU= 18.546
  06:58:20 STEP 1550 H= 387:30 +CPU= 18.522
  06:58:39 STEP 1551 H= 387:45 +CPU= 18.512
  06:58:57 STEP 1552 H= 388:00 +CPU= 18.881
  06:59:16 STEP 1553 H= 388:15 +CPU= 18.494
  06:59:34 STEP 1554 H= 388:30 +CPU= 18.497
  06:59:53 STEP 1555 H= 388:45 +CPU= 18.596
  07:00:12 STEP 1556 H= 389:00 +CPU= 19.275
  07:00:31 STEP 1557 H= 389:15 +CPU= 18.960
  07:00:51 STEP 1558 H= 389:30 +CPU= 19.304
  07:01:10 STEP 1559 H= 389:45 +CPU= 19.238
  07:01:34 STEP 1560 H= 390:00 +CPU= 23.977
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001560
  07:01:54 STEP 1561 H= 390:15 +CPU= 19.972
  07:02:13 STEP 1562 H= 390:30 +CPU= 19.219
  07:02:32 STEP 1563 H= 390:45 +CPU= 18.997
  07:02:51 STEP 1564 H= 391:00 +CPU= 19.376
  07:03:10 STEP 1565 H= 391:15 +CPU= 18.669
  07:03:29 STEP 1566 H= 391:30 +CPU= 18.891
  07:03:48 STEP 1567 H= 391:45 +CPU= 18.943
  07:04:08 STEP 1568 H= 392:00 +CPU= 19.756
  07:04:27 STEP 1569 H= 392:15 +CPU= 19.248
  07:04:46 STEP 1570 H= 392:30 +CPU= 18.983
  07:05:05 STEP 1571 H= 392:45 +CPU= 18.631
  07:05:28 STEP 1572 H= 393:00 +CPU= 23.378
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001572
  07:05:47 STEP 1573 H= 393:15 +CPU= 18.564
  07:06:05 STEP 1574 H= 393:30 +CPU= 18.506
  07:06:24 STEP 1575 H= 393:45 +CPU= 18.508
  07:06:43 STEP 1576 H= 394:00 +CPU= 19.531
  07:07:02 STEP 1577 H= 394:15 +CPU= 19.103
  07:07:21 STEP 1578 H= 394:30 +CPU= 18.722
  07:07:40 STEP 1579 H= 394:45 +CPU= 19.011
  07:07:59 STEP 1580 H= 395:00 +CPU= 18.749
  07:08:17 STEP 1581 H= 395:15 +CPU= 18.346
  07:08:36 STEP 1582 H= 395:30 +CPU= 18.923
  07:08:55 STEP 1583 H= 395:45 +CPU= 18.952
  07:09:21 STEP 1584 H= 396:00 +CPU= 26.108
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001584
  07:09:40 STEP 1585 H= 396:15 +CPU= 18.840
  07:09:59 STEP 1586 H= 396:30 +CPU= 18.813
  07:10:19 STEP 1587 H= 396:45 +CPU= 20.074
  07:10:38 STEP 1588 H= 397:00 +CPU= 19.374
  07:10:57 STEP 1589 H= 397:15 +CPU= 18.992
  07:11:16 STEP 1590 H= 397:30 +CPU= 18.670
  07:11:34 STEP 1591 H= 397:45 +CPU= 18.522
  07:11:53 STEP 1592 H= 398:00 +CPU= 18.820
  07:12:12 STEP 1593 H= 398:15 +CPU= 18.662
  07:12:31 STEP 1594 H= 398:30 +CPU= 18.946
  07:12:50 STEP 1595 H= 398:45 +CPU= 18.892
  07:13:13 STEP 1596 H= 399:00 +CPU= 23.423
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001596
  07:13:32 STEP 1597 H= 399:15 +CPU= 19.200
  07:13:51 STEP 1598 H= 399:30 +CPU= 18.806
  07:14:10 STEP 1599 H= 399:45 +CPU= 18.745
  07:14:29 STEP 1600 H= 400:00 +CPU= 19.164
  07:14:48 STEP 1601 H= 400:15 +CPU= 19.034
  07:15:07 STEP 1602 H= 400:30 +CPU= 18.878
  07:15:26 STEP 1603 H= 400:45 +CPU= 18.850
  07:15:45 STEP 1604 H= 401:00 +CPU= 19.244
  07:16:04 STEP 1605 H= 401:15 +CPU= 18.875
  07:16:23 STEP 1606 H= 401:30 +CPU= 18.963
  07:16:42 STEP 1607 H= 401:45 +CPU= 18.766
  07:17:05 STEP 1608 H= 402:00 +CPU= 23.487
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001608
  07:17:24 STEP 1609 H= 402:15 +CPU= 18.904
  07:17:43 STEP 1610 H= 402:30 +CPU= 19.064
  07:18:02 STEP 1611 H= 402:45 +CPU= 18.942
  07:18:21 STEP 1612 H= 403:00 +CPU= 19.101
  07:18:40 STEP 1613 H= 403:15 +CPU= 19.053
  07:18:59 STEP 1614 H= 403:30 +CPU= 18.757
  07:19:17 STEP 1615 H= 403:45 +CPU= 18.424
  07:19:36 STEP 1616 H= 404:00 +CPU= 19.015
  07:19:55 STEP 1617 H= 404:15 +CPU= 18.879
  07:20:14 STEP 1618 H= 404:30 +CPU= 18.743
  07:20:33 STEP 1619 H= 404:45 +CPU= 18.752
  07:20:56 STEP 1620 H= 405:00 +CPU= 23.453
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001620
  07:21:15 STEP 1621 H= 405:15 +CPU= 18.785
  07:21:34 STEP 1622 H= 405:30 +CPU= 18.761
  07:21:53 STEP 1623 H= 405:45 +CPU= 18.746
  07:22:12 STEP 1624 H= 406:00 +CPU= 19.143
  07:22:31 STEP 1625 H= 406:15 +CPU= 18.815
  07:22:49 STEP 1626 H= 406:30 +CPU= 18.773
  07:23:08 STEP 1627 H= 406:45 +CPU= 18.761
  07:23:27 STEP 1628 H= 407:00 +CPU= 19.153
  07:23:46 STEP 1629 H= 407:15 +CPU= 18.726
  07:24:05 STEP 1630 H= 407:30 +CPU= 18.762
  07:24:24 STEP 1631 H= 407:45 +CPU= 18.766
  07:24:49 STEP 1632 H= 408:00 +CPU= 25.856
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0_r884999839_16.zip
Uploading the intermediate file: upload_file_16.zip
  07:25:08 STEP 1633 H= 408:15 +CPU= 18.900
  07:25:27 STEP 1634 H= 408:30 +CPU= 18.816
  07:25:46 STEP 1635 H= 408:45 +CPU= 19.127
  07:26:06 STEP 1636 H= 409:00 +CPU= 19.505
  07:26:25 STEP 1637 H= 409:15 +CPU= 18.840
  07:26:43 STEP 1638 H= 409:30 +CPU= 18.779
  07:27:02 STEP 1639 H= 409:45 +CPU= 18.804
  07:27:21 STEP 1640 H= 410:00 +CPU= 19.010
  07:27:40 STEP 1641 H= 410:15 +CPU= 18.801
  07:27:59 STEP 1642 H= 410:30 +CPU= 18.833
  07:28:18 STEP 1643 H= 410:45 +CPU= 18.842
  07:28:41 STEP 1644 H= 411:00 +CPU= 23.527
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001644
  07:29:00 STEP 1645 H= 411:15 +CPU= 18.671
  07:29:19 STEP 1646 H= 411:30 +CPU= 18.714
  07:29:37 STEP 1647 H= 411:45 +CPU= 18.726
  07:29:56 STEP 1648 H= 412:00 +CPU= 19.183
  07:30:15 STEP 1649 H= 412:15 +CPU= 18.869
  07:30:34 STEP 1650 H= 412:30 +CPU= 18.892
  07:30:53 STEP 1651 H= 412:45 +CPU= 18.968
  07:31:13 STEP 1652 H= 413:00 +CPU= 19.360
  07:31:32 STEP 1653 H= 413:15 +CPU= 19.618
  07:31:51 STEP 1654 H= 413:30 +CPU= 18.853
  07:32:10 STEP 1655 H= 413:45 +CPU= 18.715
  07:32:33 STEP 1656 H= 414:00 +CPU= 23.543
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001656
  07:32:52 STEP 1657 H= 414:15 +CPU= 18.876
  07:33:11 STEP 1658 H= 414:30 +CPU= 18.821
  07:33:30 STEP 1659 H= 414:45 +CPU= 19.049
  07:33:50 STEP 1660 H= 415:00 +CPU= 19.561
  07:34:08 STEP 1661 H= 415:15 +CPU= 18.826
  07:34:27 STEP 1662 H= 415:30 +CPU= 18.659
  07:34:46 STEP 1663 H= 415:45 +CPU= 18.554
  07:35:05 STEP 1664 H= 416:00 +CPU= 19.110
  07:35:24 STEP 1665 H= 416:15 +CPU= 19.226
  07:35:43 STEP 1666 H= 416:30 +CPU= 18.829
  07:36:02 STEP 1667 H= 416:45 +CPU= 18.679
  07:36:25 STEP 1668 H= 417:00 +CPU= 23.386
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001668
  07:36:44 STEP 1669 H= 417:15 +CPU= 19.387
  07:37:03 STEP 1670 H= 417:30 +CPU= 18.855
  07:37:22 STEP 1671 H= 417:45 +CPU= 18.816
  07:37:42 STEP 1672 H= 418:00 +CPU= 19.724
  07:38:01 STEP 1673 H= 418:15 +CPU= 19.012
  07:38:20 STEP 1674 H= 418:30 +CPU= 19.047
  07:38:39 STEP 1675 H= 418:45 +CPU= 19.442
  07:38:58 STEP 1676 H= 419:00 +CPU= 19.230
  07:39:17 STEP 1677 H= 419:15 +CPU= 18.363
  07:39:35 STEP 1678 H= 419:30 +CPU= 18.376
  07:39:54 STEP 1679 H= 419:45 +CPU= 18.780
  07:40:20 STEP 1680 H= 420:00 +CPU= 26.052
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001680
  07:40:39 STEP 1681 H= 420:15 +CPU= 18.796
  07:40:58 STEP 1682 H= 420:30 +CPU= 18.901
  07:41:17 STEP 1683 H= 420:45 +CPU= 18.845
  07:41:36 STEP 1684 H= 421:00 +CPU= 19.215
  07:41:55 STEP 1685 H= 421:15 +CPU= 18.906
  07:42:14 STEP 1686 H= 421:30 +CPU= 18.940
  07:42:33 STEP 1687 H= 421:45 +CPU= 18.883
  07:42:52 STEP 1688 H= 422:00 +CPU= 19.286
  07:43:11 STEP 1689 H= 422:15 +CPU= 18.938
  07:43:29 STEP 1690 H= 422:30 +CPU= 18.608
  07:43:48 STEP 1691 H= 422:45 +CPU= 18.864
  07:44:12 STEP 1692 H= 423:00 +CPU= 23.632
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001692
  07:44:30 STEP 1693 H= 423:15 +CPU= 18.549
  07:44:49 STEP 1694 H= 423:30 +CPU= 18.570
  07:45:08 STEP 1695 H= 423:45 +CPU= 18.828
  07:45:27 STEP 1696 H= 424:00 +CPU= 19.230
  07:45:46 STEP 1697 H= 424:15 +CPU= 18.681
  07:46:05 STEP 1698 H= 424:30 +CPU= 18.744
  07:46:23 STEP 1699 H= 424:45 +CPU= 18.459
  07:46:42 STEP 1700 H= 425:00 +CPU= 18.882
  07:47:00 STEP 1701 H= 425:15 +CPU= 18.482
  07:47:19 STEP 1702 H= 425:30 +CPU= 18.488
  07:47:37 STEP 1703 H= 425:45 +CPU= 18.474
  07:48:01 STEP 1704 H= 426:00 +CPU= 23.195
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001704
  07:48:19 STEP 1705 H= 426:15 +CPU= 18.544
  07:48:38 STEP 1706 H= 426:30 +CPU= 18.648
  07:48:56 STEP 1707 H= 426:45 +CPU= 18.456
  07:49:15 STEP 1708 H= 427:00 +CPU= 19.184
  07:49:34 STEP 1709 H= 427:15 +CPU= 18.863
  07:49:53 STEP 1710 H= 427:30 +CPU= 18.865
  07:50:12 STEP 1711 H= 427:45 +CPU= 18.540
  07:50:31 STEP 1712 H= 428:00 +CPU= 18.954
  07:50:49 STEP 1713 H= 428:15 +CPU= 18.518
  07:51:08 STEP 1714 H= 428:30 +CPU= 18.503
  07:51:26 STEP 1715 H= 428:45 +CPU= 18.509
  07:51:49 STEP 1716 H= 429:00 +CPU= 23.164
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001716
  07:52:08 STEP 1717 H= 429:15 +CPU= 18.588
  07:52:27 STEP 1718 H= 429:30 +CPU= 18.753
  07:52:45 STEP 1719 H= 429:45 +CPU= 18.456
  07:53:04 STEP 1720 H= 430:00 +CPU= 18.853
  07:53:22 STEP 1721 H= 430:15 +CPU= 18.471
  07:53:41 STEP 1722 H= 430:30 +CPU= 18.455
  07:53:59 STEP 1723 H= 430:45 +CPU= 18.429
  07:54:18 STEP 1724 H= 431:00 +CPU= 18.867
  07:54:37 STEP 1725 H= 431:15 +CPU= 18.439
  07:54:55 STEP 1726 H= 431:30 +CPU= 18.460
  07:55:14 STEP 1727 H= 431:45 +CPU= 18.477
  07:55:39 STEP 1728 H= 432:00 +CPU= 25.587
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0_r884999839_17.zip
Uploading the intermediate file: upload_file_17.zip
  07:55:58 STEP 1729 H= 432:15 +CPU= 18.964
  07:56:17 STEP 1730 H= 432:30 +CPU= 19.040
  07:56:37 STEP 1731 H= 432:45 +CPU= 19.409
  07:56:56 STEP 1732 H= 433:00 +CPU= 19.562
  07:57:15 STEP 1733 H= 433:15 +CPU= 18.756
  07:57:34 STEP 1734 H= 433:30 +CPU= 18.810
  07:57:52 STEP 1735 H= 433:45 +CPU= 18.560
  07:58:11 STEP 1736 H= 434:00 +CPU= 18.935
  07:58:30 STEP 1737 H= 434:15 +CPU= 18.520
  07:58:48 STEP 1738 H= 434:30 +CPU= 18.517
  07:59:07 STEP 1739 H= 434:45 +CPU= 18.495
  07:59:30 STEP 1740 H= 435:00 +CPU= 23.036
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001740
  07:59:49 STEP 1741 H= 435:15 +CPU= 19.198
  08:00:08 STEP 1742 H= 435:30 +CPU= 19.460
  08:00:27 STEP 1743 H= 435:45 +CPU= 18.632
  08:00:46 STEP 1744 H= 436:00 +CPU= 18.946
  08:01:05 STEP 1745 H= 436:15 +CPU= 19.319
  08:01:24 STEP 1746 H= 436:30 +CPU= 18.582
  08:01:43 STEP 1747 H= 436:45 +CPU= 19.261
  08:02:03 STEP 1748 H= 437:00 +CPU= 19.231
  08:02:21 STEP 1749 H= 437:15 +CPU= 18.856
  08:02:40 STEP 1750 H= 437:30 +CPU= 18.431
  08:02:58 STEP 1751 H= 437:45 +CPU= 18.469
  08:03:21 STEP 1752 H= 438:00 +CPU= 23.186
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001752
  08:03:40 STEP 1753 H= 438:15 +CPU= 18.563
  08:03:59 STEP 1754 H= 438:30 +CPU= 18.545
  08:04:17 STEP 1755 H= 438:45 +CPU= 18.612
  08:04:37 STEP 1756 H= 439:00 +CPU= 19.375
  08:04:55 STEP 1757 H= 439:15 +CPU= 18.629
  08:05:14 STEP 1758 H= 439:30 +CPU= 18.738
  08:05:32 STEP 1759 H= 439:45 +CPU= 18.531
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.569759017341195E-07 0.000000000000000E+00 0.358813641111344E-04
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.159311058158000E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.121342029953769E-07 0.000000000000000E+00 0.811804105862748E-04
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.323203340947732E-07 0.000000000000000E+00 0.972965262799213E-04
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.120931196549975E-07 0.000000000000000E+00 0.678029700997252E-04
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.168987252967338E-08 0.000000000000000E+00 0.666410281248270E-04
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.371632224771447E-01 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =  1752 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1752 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001752 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+001752
 MPI-TASK:   1 -    323154036 BYTES IN      6 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0070( 4.3%)
 NSTEP =  1752 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+001752 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+001752
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0063 SECONDS112285.6 Mbytes/s, TOTAL TIME=    0.2144( 2.9%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001752 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+001752
 MPI-TASK:   1 -   5170464576 BYTES IN    125 RECORDS TRANSFERRED IN    0.0037 SECONDS******** Mbytes/s, TOTAL TIME=    0.1494( 2.5%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001752 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001752
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS     0.1 Mbytes/s, TOTAL TIME=    0.0029( 3.4%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001752 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001752
 MPI-TASK:   1 -    314574307 BYTES IN      4 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0048( 4.2%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =  1752 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1752 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001752 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001752
 MPI-TASK:   1 -    323154036 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0079( 3.8%)
 08:03:21 STEP 1752 H= 438:00 +CPU= 23.186
 NSTEP =  1753 STEPO   0AAA00AAA
 08:03:40 STEP 1753 H= 438:15 +CPU= 18.563
 NSTEP =  1754 STEPO   0AAA00AAA
 08:03:59 STEP 1754 H= 438:30 +CPU= 18.545
ISTEP=    1755 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=    1755 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =  1755 STEPO   0AAA00AAA
 08:04:17 STEP 1755 H= 438:45 +CPU= 18.612
 NSTEP =  1756 STEPO   0AAA00AAA
 08:04:37 STEP 1756 H= 439:00 +CPU= 19.375
 NSTEP =  1757 STEPO   0AAA00AAA
 08:04:55 STEP 1757 H= 439:15 +CPU= 18.629
 NSTEP =  1758 STEPO   0AAA00AAA
 08:05:14 STEP 1758 H= 439:30 +CPU= 18.738
 NSTEP =  1759 STEPO   0AAA00AAA
 08:05:32 STEP 1759 H= 439:45 +CPU= 18.531
ISTEP=    1760 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=    1760 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =  1760 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 08:03:40 0AAA00AAA STEPO     1754    18.563   18.563   18.564    803:31    808:19 0.34221912560851E-06       2GB       0MB
 08:03:59 0AAA00AAA STEPO     1755    18.545   18.545   18.545    803:50    808:38 0.34221912560851E-06       2GB       0MB
 08:04:17 0AAA00AAA STEPO     1756    18.612   18.612   18.615    804:08    808:57 0.34221912560851E-06       2GB       0MB
 08:04:37 0AAA00AAA STEPO     1757    19.375   19.375   19.382    804:28    809:16 0.34221912560851E-06       2GB       0MB
 08:04:55 0AAA00AAA STEPO     1758    18.629   18.629   18.630    804:46    809:35 0.34221912560851E-06       2GB       0MB
 08:05:14 0AAA00AAA STEPO     1759    18.738   18.738   18.738    805:05    809:53 0.34221912560851E-06       2GB       0MB
 08:05:32 0AAA00AAA STEPO     1760    18.531   18.531   18.532    805:23    810:12 0.34221912560851E-06       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    1728",
 CTIME   = "00180000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,        1716, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/30/progress_file_12289268.xml

<running_values>
  <last_cpu_time>48333.020000</last_cpu_time>
  <upload_file_number>18</upload_file_number>
  <last_iter>1760</last_iter>
  <last_upload>1555200</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0q5
(argv4) batchid: 1018
(argv5) wuid: 12289268
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/30
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/30/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/30/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f21ad3f734a733e9d933758fdd2bdf13 to: /var/lib/boinc-client/slots/30/oifs_43r3_bl_a0q5_2016092300_20_1018_12289268.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/30/oifs_43r3_bl_a0q5_2016092300_20_1018_12289268.zip
ic_ancil_file: ic_ancil_12289268
ifsdata_file: ifsdata_12289268
climate_data_file: clim_data_12289268
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/30/ic_ancil_12289268.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/30/ic_ancil_12289268.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c8baf680ec76c4963d2f81dbb5d80336 to: /var/lib/boinc-client/slots/30/ifsdata/ifsdata_12289268.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/30/ifsdata/ifsdata_12289268.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/30/159l_2/clim_data_12289268.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/30/159l_2/clim_data_12289268.zip
Checking for progress XML file: /var/lib/boinc-client/slots/30/progress_file_12289268.xml
Opened progress file ok : /var/lib/boinc-client/slots/30/progress_file_12289268.xml
-- Model is restarting --
Adjusting last_iter, 1760, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/30/progress_file_12289268.xml
last_cpu_time: 48333
upload_file_number: 18
last_iter: 1729
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0_r884999839
The child process has been launched with process id: 2636787
Executing the command: /var/lib/boinc-client/slots/30/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:P5810:1:1:2636787:2636787] [20240623:124240:1719168160:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  12:43:24 STEP 1728 H= 432:00 +CPU= 38.394
  12:44:03 STEP 1729 H= 432:15 +CPU= 37.975
  12:44:42 STEP 1730 H= 432:30 +CPU= 38.896
  12:45:20 STEP 1731 H= 432:45 +CPU= 36.421
  12:45:49 STEP 1732 H= 433:00 +CPU= 28.011
  12:46:19 STEP 1733 H= 433:15 +CPU= 29.586
  12:46:58 STEP 1734 H= 433:30 +CPU= 38.355
  12:47:36 STEP 1735 H= 433:45 +CPU= 37.337
  12:48:14 STEP 1736 H= 434:00 +CPU= 36.676
  12:48:52 STEP 1737 H= 434:15 +CPU= 36.416
  12:49:22 STEP 1738 H= 434:30 +CPU= 29.951
  12:49:52 STEP 1739 H= 434:45 +CPU= 29.480
  12:50:40 STEP 1740 H= 435:00 +CPU= 46.510
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001740
  12:51:19 STEP 1741 H= 435:15 +CPU= 38.334
  12:51:58 STEP 1742 H= 435:30 +CPU= 37.609
  12:52:35 STEP 1743 H= 435:45 +CPU= 35.978
  12:53:15 STEP 1744 H= 436:00 +CPU= 39.220
  12:53:55 STEP 1745 H= 436:15 +CPU= 38.530
  12:54:32 STEP 1746 H= 436:30 +CPU= 36.126
  12:55:03 STEP 1747 H= 436:45 +CPU= 31.087
  12:55:39 STEP 1748 H= 437:00 +CPU= 34.289
  12:56:17 STEP 1749 H= 437:15 +CPU= 37.264
  12:56:57 STEP 1750 H= 437:30 +CPU= 38.789
  12:57:33 STEP 1751 H= 437:45 +CPU= 35.193
  12:58:17 STEP 1752 H= 438:00 +CPU= 43.129
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001752
  12:58:53 STEP 1753 H= 438:15 +CPU= 35.369
  12:59:32 STEP 1754 H= 438:30 +CPU= 37.404
  13:00:10 STEP 1755 H= 438:45 +CPU= 36.738
  13:00:50 STEP 1756 H= 439:00 +CPU= 39.369
  13:01:27 STEP 1757 H= 439:15 +CPU= 36.443
  13:02:05 STEP 1758 H= 439:30 +CPU= 36.564
  13:02:41 STEP 1759 H= 439:45 +CPU= 35.696
  13:03:19 STEP 1760 H= 440:00 +CPU= 36.783
  13:03:52 STEP 1761 H= 440:15 +CPU= 32.000
  13:04:23 STEP 1762 H= 440:30 +CPU= 30.625
  13:05:00 STEP 1763 H= 440:45 +CPU= 35.829
  13:05:45 STEP 1764 H= 441:00 +CPU= 44.027
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001764
  13:06:24 STEP 1765 H= 441:15 +CPU= 37.475
  13:07:02 STEP 1766 H= 441:30 +CPU= 36.838
  13:07:38 STEP 1767 H= 441:45 +CPU= 35.669
  13:08:17 STEP 1768 H= 442:00 +CPU= 37.355
  13:08:55 STEP 1769 H= 442:15 +CPU= 37.636
  13:09:35 STEP 1770 H= 442:30 +CPU= 38.534
  13:10:14 STEP 1771 H= 442:45 +CPU= 38.630
  13:10:53 STEP 1772 H= 443:00 +CPU= 37.288
  13:11:31 STEP 1773 H= 443:15 +CPU= 37.633
  13:12:11 STEP 1774 H= 443:30 +CPU= 38.571
  13:12:49 STEP 1775 H= 443:45 +CPU= 36.905
  13:13:41 STEP 1776 H= 444:00 +CPU= 50.545
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001776
  13:14:19 STEP 1777 H= 444:15 +CPU= 37.022
  13:14:59 STEP 1778 H= 444:30 +CPU= 38.596
  13:15:36 STEP 1779 H= 444:45 +CPU= 35.838
  13:16:13 STEP 1780 H= 445:00 +CPU= 36.538
  13:16:52 STEP 1781 H= 445:15 +CPU= 37.814
  13:17:25 STEP 1782 H= 445:30 +CPU= 32.218
  13:17:59 STEP 1783 H= 445:45 +CPU= 32.719
  13:18:37 STEP 1784 H= 446:00 +CPU= 37.333
  13:19:14 STEP 1785 H= 446:15 +CPU= 36.476
  13:19:53 STEP 1786 H= 446:30 +CPU= 37.574
  13:20:32 STEP 1787 H= 446:45 +CPU= 37.830
  13:21:20 STEP 1788 H= 447:00 +CPU= 46.778
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001788
  13:21:57 STEP 1789 H= 447:15 +CPU= 36.148
  13:22:35 STEP 1790 H= 447:30 +CPU= 37.333
  13:23:13 STEP 1791 H= 447:45 +CPU= 36.249
  13:23:49 STEP 1792 H= 448:00 +CPU= 35.083
  13:24:29 STEP 1793 H= 448:15 +CPU= 38.859
  13:25:07 STEP 1794 H= 448:30 +CPU= 37.345
  13:25:42 STEP 1795 H= 448:45 +CPU= 34.179
  13:26:19 STEP 1796 H= 449:00 +CPU= 35.919
  13:26:55 STEP 1797 H= 449:15 +CPU= 35.063
  13:27:31 STEP 1798 H= 449:30 +CPU= 34.890
  13:28:08 STEP 1799 H= 449:45 +CPU= 36.509
  13:28:58 STEP 1800 H= 450:00 +CPU= 47.866
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001800
  13:29:34 STEP 1801 H= 450:15 +CPU= 35.080
  13:30:10 STEP 1802 H= 450:30 +CPU= 34.991
  13:30:48 STEP 1803 H= 450:45 +CPU= 36.944
  13:31:23 STEP 1804 H= 451:00 +CPU= 35.021
  13:31:54 STEP 1805 H= 451:15 +CPU= 29.425
  13:32:29 STEP 1806 H= 451:30 +CPU= 35.052
  13:33:07 STEP 1807 H= 451:45 +CPU= 36.965
  13:33:47 STEP 1808 H= 452:00 +CPU= 38.647
  13:34:26 STEP 1809 H= 452:15 +CPU= 37.918
  13:35:02 STEP 1810 H= 452:30 +CPU= 35.163
  13:35:39 STEP 1811 H= 452:45 +CPU= 36.265
  13:36:26 STEP 1812 H= 453:00 +CPU= 45.829
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001812
  13:37:05 STEP 1813 H= 453:15 +CPU= 37.698
  13:37:44 STEP 1814 H= 453:30 +CPU= 37.996
  13:38:20 STEP 1815 H= 453:45 +CPU= 35.075
  13:38:57 STEP 1816 H= 454:00 +CPU= 35.885
  13:39:34 STEP 1817 H= 454:15 +CPU= 35.677
  13:40:11 STEP 1818 H= 454:30 +CPU= 36.354
  13:40:49 STEP 1819 H= 454:45 +CPU= 36.904
  13:41:26 STEP 1820 H= 455:00 +CPU= 35.705
  13:42:06 STEP 1821 H= 455:15 +CPU= 39.055
  13:42:44 STEP 1822 H= 455:30 +CPU= 36.673
  13:43:24 STEP 1823 H= 455:45 +CPU= 39.087
  13:44:16 STEP 1824 H= 456:00 +CPU= 49.931
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0_r884999839_18.zip
Uploading the intermediate file: upload_file_18.zip
  13:44:54 STEP 1825 H= 456:15 +CPU= 37.180
  13:45:32 STEP 1826 H= 456:30 +CPU= 36.376
  13:46:09 STEP 1827 H= 456:45 +CPU= 36.322
  13:46:49 STEP 1828 H= 457:00 +CPU= 38.100
  13:47:26 STEP 1829 H= 457:15 +CPU= 36.091
  13:48:02 STEP 1830 H= 457:30 +CPU= 35.534
  13:48:38 STEP 1831 H= 457:45 +CPU= 34.675
  13:49:16 STEP 1832 H= 458:00 +CPU= 36.460
  13:49:53 STEP 1833 H= 458:15 +CPU= 36.128
  13:50:31 STEP 1834 H= 458:30 +CPU= 36.808
  13:51:07 STEP 1835 H= 458:45 +CPU= 34.321
  13:51:53 STEP 1836 H= 459:00 +CPU= 44.796
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001836
  13:52:30 STEP 1837 H= 459:15 +CPU= 35.767
  13:53:07 STEP 1838 H= 459:30 +CPU= 36.362
  13:53:45 STEP 1839 H= 459:45 +CPU= 36.640
  13:54:24 STEP 1840 H= 460:00 +CPU= 37.325
  13:55:02 STEP 1841 H= 460:15 +CPU= 37.654
  13:55:39 STEP 1842 H= 460:30 +CPU= 35.973
  13:56:16 STEP 1843 H= 460:45 +CPU= 35.307
  13:56:54 STEP 1844 H= 461:00 +CPU= 36.840
  13:57:31 STEP 1845 H= 461:15 +CPU= 36.307
  13:58:10 STEP 1846 H= 461:30 +CPU= 38.436
  13:58:48 STEP 1847 H= 461:45 +CPU= 36.543
  13:59:36 STEP 1848 H= 462:00 +CPU= 46.627
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001848
  14:00:14 STEP 1849 H= 462:15 +CPU= 37.102
  14:00:51 STEP 1850 H= 462:30 +CPU= 36.082
  14:01:28 STEP 1851 H= 462:45 +CPU= 36.254
  14:02:07 STEP 1852 H= 463:00 +CPU= 37.209
  14:02:45 STEP 1853 H= 463:15 +CPU= 37.650
  14:03:22 STEP 1854 H= 463:30 +CPU= 35.663
  14:03:59 STEP 1855 H= 463:45 +CPU= 35.566
  14:04:39 STEP 1856 H= 464:00 +CPU= 38.911
  14:05:16 STEP 1857 H= 464:15 +CPU= 36.822
  14:05:56 STEP 1858 H= 464:30 +CPU= 38.131
  14:06:34 STEP 1859 H= 464:45 +CPU= 37.544
  14:07:21 STEP 1860 H= 465:00 +CPU= 45.017
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001860
  14:07:58 STEP 1861 H= 465:15 +CPU= 35.989
  14:08:35 STEP 1862 H= 465:30 +CPU= 36.794
  14:09:12 STEP 1863 H= 465:45 +CPU= 35.820
  14:09:50 STEP 1864 H= 466:00 +CPU= 36.397
  14:10:28 STEP 1865 H= 466:15 +CPU= 36.872
  14:11:06 STEP 1866 H= 466:30 +CPU= 37.719
  14:11:42 STEP 1867 H= 466:45 +CPU= 34.704
  14:12:20 STEP 1868 H= 467:00 +CPU= 36.949
  14:12:58 STEP 1869 H= 467:15 +CPU= 36.710
  14:13:36 STEP 1870 H= 467:30 +CPU= 36.843
  14:14:13 STEP 1871 H= 467:45 +CPU= 36.397
  14:15:07 STEP 1872 H= 468:00 +CPU= 52.117
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001872
  14:15:46 STEP 1873 H= 468:15 +CPU= 37.475
  14:16:25 STEP 1874 H= 468:30 +CPU= 38.196
  14:17:03 STEP 1875 H= 468:45 +CPU= 37.497
  14:17:44 STEP 1876 H= 469:00 +CPU= 39.130
  14:18:20 STEP 1877 H= 469:15 +CPU= 35.070
  14:18:56 STEP 1878 H= 469:30 +CPU= 35.286
  14:19:35 STEP 1879 H= 469:45 +CPU= 37.613
  14:20:12 STEP 1880 H= 470:00 +CPU= 36.377
  14:20:49 STEP 1881 H= 470:15 +CPU= 36.066
  14:21:26 STEP 1882 H= 470:30 +CPU= 35.794
  14:22:03 STEP 1883 H= 470:45 +CPU= 35.962
  14:22:50 STEP 1884 H= 471:00 +CPU= 45.626
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001884
  14:23:29 STEP 1885 H= 471:15 +CPU= 37.309
  14:24:07 STEP 1886 H= 471:30 +CPU= 37.809
  14:24:45 STEP 1887 H= 471:45 +CPU= 37.074
  14:25:24 STEP 1888 H= 472:00 +CPU= 37.534
  14:26:02 STEP 1889 H= 472:15 +CPU= 36.662
  14:26:41 STEP 1890 H= 472:30 +CPU= 37.860
  14:27:18 STEP 1891 H= 472:45 +CPU= 36.190
  14:27:57 STEP 1892 H= 473:00 +CPU= 38.208
  14:28:37 STEP 1893 H= 473:15 +CPU= 38.208
  14:29:14 STEP 1894 H= 473:30 +CPU= 36.815
  14:29:54 STEP 1895 H= 473:45 +CPU= 38.338
  14:30:42 STEP 1896 H= 474:00 +CPU= 46.639
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001896
  14:31:18 STEP 1897 H= 474:15 +CPU= 35.748
  14:31:56 STEP 1898 H= 474:30 +CPU= 36.412
  14:32:34 STEP 1899 H= 474:45 +CPU= 36.797
  14:33:12 STEP 1900 H= 475:00 +CPU= 37.826
  14:33:52 STEP 1901 H= 475:15 +CPU= 39.041
  14:34:31 STEP 1902 H= 475:30 +CPU= 37.423
  14:35:09 STEP 1903 H= 475:45 +CPU= 37.101
  14:35:47 STEP 1904 H= 476:00 +CPU= 36.470
  14:36:25 STEP 1905 H= 476:15 +CPU= 37.534
  14:37:04 STEP 1906 H= 476:30 +CPU= 37.987
  14:37:41 STEP 1907 H= 476:45 +CPU= 35.189
  14:38:29 STEP 1908 H= 477:00 +CPU= 47.511
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001908
  14:39:07 STEP 1909 H= 477:15 +CPU= 36.446
  14:39:46 STEP 1910 H= 477:30 +CPU= 37.901
  14:40:24 STEP 1911 H= 477:45 +CPU= 36.814
  14:41:03 STEP 1912 H= 478:00 +CPU= 37.743
  14:41:40 STEP 1913 H= 478:15 +CPU= 36.176
  14:42:19 STEP 1914 H= 478:30 +CPU= 38.374
  14:42:57 STEP 1915 H= 478:45 +CPU= 36.621
  14:43:37 STEP 1916 H= 479:00 +CPU= 38.516
  14:44:14 STEP 1917 H= 479:15 +CPU= 36.566
  14:44:52 STEP 1918 H= 479:30 +CPU= 36.835
  14:45:28 STEP 1919 H= 479:45 +CPU= 35.111
  14:46:16 STEP 1920 H= 480:00 +CPU= 46.167
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1512058967E-21      I       0.3358593593E-06      I
ICloud fractionI      0.0000000000E+00      I       0.3268900588E-01      I
I  Relat. Hum. I      0.5005171387E+00      I       0.5365061850E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2258399594E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    317977236 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS454253.2 Mbytes/s, TOTAL TIME=    0.0168( 4.2%)
 14:46:16 STEP 1920 H= 480:00 +CPU= 46.167
 END CNT3
 NSTEP =  1920 CNT0   000000000  33.675  33.675  34.712   0.970               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         5720/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    103    0       46      1044068     5720 ************    15.4     2.0    13.4     2.8    28.7
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 7416065.0 7416065.0    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 7411667.5 7411667.5     99.94      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  209    5517.7    5517.7     15.55      0.00
   9 SPCM     - SPECTRAL COMP.                     192    1074.7    1074.7      2.78      0.00
  10 SCAN2M - PHYSICS                              193   19009.7   19009.7     49.47      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 16    1113.6    1113.6      0.24      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  9      64.5      64.5      0.01      0.00
  17 GRIDFPOS IN CNT4                               16      37.9      37.9      0.01      0.00
  27 RESTART FILES - WRITING                         4    4454.3    4454.3      0.24      0.00
  28 RESTART FILES - READING                         1    3687.7    3687.7      0.05      0.00
  29 SU4FPOS IN CNT4                                16       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                16    8551.9    8551.9      1.85      0.00
  31 POSDDH IN STEPO                                 1       1.6       1.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    193    8032.1    8032.1     20.90      0.00
  39 SU0YOMB                                         1     473.9     473.9      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    193      43.9      43.9      0.11      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              192     189.8     189.8      0.49      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              192      69.6      69.6      0.18      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  9       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      420     627.0     627.0      3.55      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       400     357.0     357.0      1.93      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        400     133.0     133.0      0.72      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       420     203.9     203.9      1.15      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      49.4      49.4      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            420      68.9      68.9      0.39      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            400      35.0      35.0      0.19      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            420      97.0      97.0      0.55      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            400      30.1      30.1      0.16      0.00
 400 GSTATS                                      23311       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 21635       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     7416.1 CPU TIME    7214.0 VECTOR TIME     7214.0


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      233.1
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/30/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/30/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/30/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289268/ICMUAh7zg+001896
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0_r884999839_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
14:49:53 (2635809): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 Jun 2024 20:48:15 1551031 22442853 oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0 1,727,100 55,547 0.0322
23 Jun 2024 09:49:54 1551031 22442853 oifs_43r3_bl_a0q5_2016092300_20_1018_12289268_0 864,000 31,778 0.0368


©2024 climateprediction.net