climateprediction.net home page
Task 22280390

Task 22280390

Name oifs_43r3_ps_0890_2008050100_123_977_12194534_0
Workunit 12194534
Created 23 Dec 2022, 19:37:03 UTC
Sent 27 Dec 2022, 13:50:44 UTC
Report deadline 26 Jan 2023, 13:50:44 UTC
Received 5 Jan 2023, 9:41:35 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1220995
Run time 12 hours 15 min 36 sec
CPU time 12 hours 2 min 59 sec
Validate state Invalid
Credit 8,225.28
Device peak FLOPS 3.21 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,802.18 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,181.48 MB
Stderr
<core_client_version>7.21.0</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_61.zip
Uploading the intermediate file: upload_file_61.zip
  12:13:08 STEP 1489 H=1489:00 +CPU= 17.406
  12:13:26 STEP 1490 H=1490:00 +CPU= 17.438
  12:13:54 STEP 1491 H=1491:00 +CPU= 27.883
  12:14:11 STEP 1492 H=1492:00 +CPU= 17.524
  12:14:28 STEP 1493 H=1493:00 +CPU= 16.861
  12:14:58 STEP 1494 H=1494:00 +CPU= 29.383
  12:15:15 STEP 1495 H=1495:00 +CPU= 17.096
  12:15:33 STEP 1496 H=1496:00 +CPU= 17.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:16:22 STEP 1497 H=1497:00 +CPU= 27.869
  12:16:39 STEP 1498 H=1498:00 +CPU= 17.555
  12:16:57 STEP 1499 H=1499:00 +CPU= 17.085
  12:17:29 STEP 1500 H=1500:00 +CPU= 31.932
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001500
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001500
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001500
  12:17:45 STEP 1501 H=1501:00 +CPU= 16.773
  12:18:03 STEP 1502 H=1502:00 +CPU= 17.912
  12:18:31 STEP 1503 H=1503:00 +CPU= 27.615
  12:18:49 STEP 1504 H=1504:00 +CPU= 18.207
  12:19:07 STEP 1505 H=1505:00 +CPU= 17.243
  12:19:36 STEP 1506 H=1506:00 +CPU= 29.192
  12:19:53 STEP 1507 H=1507:00 +CPU= 17.199
  12:20:11 STEP 1508 H=1508:00 +CPU= 17.822
  12:20:39 STEP 1509 H=1509:00 +CPU= 27.965
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:21:18 STEP 1510 H=1510:00 +CPU= 17.275
  12:21:35 STEP 1511 H=1511:00 +CPU= 17.677
  12:22:07 STEP 1512 H=1512:00 +CPU= 31.731
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001512
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001512
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001512
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001488
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001488
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001488
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001500
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001500
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001500
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_62.zip
Uploading the intermediate file: upload_file_62.zip
  12:22:24 STEP 1513 H=1513:00 +CPU= 16.890
  12:22:42 STEP 1514 H=1514:00 +CPU= 17.539
  12:23:09 STEP 1515 H=1515:00 +CPU= 27.441
  12:23:27 STEP 1516 H=1516:00 +CPU= 17.546
  12:23:44 STEP 1517 H=1517:00 +CPU= 16.803
  12:24:12 STEP 1518 H=1518:00 +CPU= 28.088
  12:24:29 STEP 1519 H=1519:00 +CPU= 16.665
  12:24:46 STEP 1520 H=1520:00 +CPU= 17.312
  12:25:13 STEP 1521 H=1521:00 +CPU= 27.295
  12:25:31 STEP 1522 H=1522:00 +CPU= 17.554
  12:25:48 STEP 1523 H=1523:00 +CPU= 16.580
  12:26:18 STEP 1524 H=1524:00 +CPU= 30.100
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001524
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001524
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001524
  12:26:35 STEP 1525 H=1525:00 +CPU= 17.144
  12:26:52 STEP 1526 H=1526:00 +CPU= 17.333
  12:27:20 STEP 1527 H=1527:00 +CPU= 27.191
  12:27:37 STEP 1528 H=1528:00 +CPU= 17.487
  12:27:54 STEP 1529 H=1529:00 +CPU= 16.600
  12:28:22 STEP 1530 H=1530:00 +CPU= 27.939
  12:28:39 STEP 1531 H=1531:00 +CPU= 16.743
  12:28:56 STEP 1532 H=1532:00 +CPU= 17.149
  12:29:23 STEP 1533 H=1533:00 +CPU= 27.205
  12:29:40 STEP 1534 H=1534:00 +CPU= 17.189
  12:29:57 STEP 1535 H=1535:00 +CPU= 16.985
  12:30:30 STEP 1536 H=1536:00 +CPU= 32.202
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001536
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001536
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001536
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001512
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001512
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001512
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001524
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001524
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001524
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_63.zip
Uploading the intermediate file: upload_file_63.zip
  12:30:47 STEP 1537 H=1537:00 +CPU= 17.237
  12:31:04 STEP 1538 H=1538:00 +CPU= 17.295
  12:31:32 STEP 1539 H=1539:00 +CPU= 27.732
  12:31:50 STEP 1540 H=1540:00 +CPU= 17.414
  12:32:07 STEP 1541 H=1541:00 +CPU= 17.447
  12:32:38 STEP 1542 H=1542:00 +CPU= 30.658
  12:32:56 STEP 1543 H=1543:00 +CPU= 18.304
  12:33:16 STEP 1544 H=1544:00 +CPU= 19.293
  12:33:46 STEP 1545 H=1545:00 +CPU= 30.668
  12:34:04 STEP 1546 H=1546:00 +CPU= 17.287
  12:34:20 STEP 1547 H=1547:00 +CPU= 16.629
  12:34:51 STEP 1548 H=1548:00 +CPU= 30.274
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001548
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001548
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001548
  12:35:08 STEP 1549 H=1549:00 +CPU= 16.725
  12:35:25 STEP 1550 H=1550:00 +CPU= 17.351
  12:35:53 STEP 1551 H=1551:00 +CPU= 27.498
  12:36:10 STEP 1552 H=1552:00 +CPU= 17.169
  12:36:26 STEP 1553 H=1553:00 +CPU= 16.627
  12:36:54 STEP 1554 H=1554:00 +CPU= 27.762
  12:37:11 STEP 1555 H=1555:00 +CPU= 17.064
  12:37:29 STEP 1556 H=1556:00 +CPU= 17.159
  12:37:56 STEP 1557 H=1557:00 +CPU= 27.641
  12:38:14 STEP 1558 H=1558:00 +CPU= 17.306
  12:38:31 STEP 1559 H=1559:00 +CPU= 17.512
  12:39:03 STEP 1560 H=1560:00 +CPU= 31.420
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001560
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001560
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001560
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001536
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001536
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001536
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001548
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001548
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001548
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_64.zip
Uploading the intermediate file: upload_file_64.zip
  12:39:20 STEP 1561 H=1561:00 +CPU= 16.904
  12:39:37 STEP 1562 H=1562:00 +CPU= 17.653
  12:40:05 STEP 1563 H=1563:00 +CPU= 27.501
  12:40:22 STEP 1564 H=1564:00 +CPU= 17.449
  12:40:39 STEP 1565 H=1565:00 +CPU= 16.862
  12:41:08 STEP 1566 H=1566:00 +CPU= 28.159
  12:41:24 STEP 1567 H=1567:00 +CPU= 16.787
  12:41:42 STEP 1568 H=1568:00 +CPU= 17.257
  12:42:09 STEP 1569 H=1569:00 +CPU= 27.153
  12:42:27 STEP 1570 H=1570:00 +CPU= 17.521
  12:42:43 STEP 1571 H=1571:00 +CPU= 16.722
  12:43:13 STEP 1572 H=1572:00 +CPU= 30.148
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001572
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001572
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001572
  12:43:30 STEP 1573 H=1573:00 +CPU= 16.612
  12:43:47 STEP 1574 H=1574:00 +CPU= 17.160
  12:44:15 STEP 1575 H=1575:00 +CPU= 27.414
  12:44:32 STEP 1576 H=1576:00 +CPU= 17.272
  12:44:49 STEP 1577 H=1577:00 +CPU= 17.086
  12:45:17 STEP 1578 H=1578:00 +CPU= 28.000
  12:45:34 STEP 1579 H=1579:00 +CPU= 16.765
  12:45:52 STEP 1580 H=1580:00 +CPU= 17.323
  12:46:19 STEP 1581 H=1581:00 +CPU= 27.256
  12:46:36 STEP 1582 H=1582:00 +CPU= 17.415
  12:46:54 STEP 1583 H=1583:00 +CPU= 17.572
  12:47:25 STEP 1584 H=1584:00 +CPU= 31.253
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001584
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001584
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001584
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001560
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001560
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001560
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001572
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001572
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001572
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_65.zip
Uploading the intermediate file: upload_file_65.zip
  12:47:42 STEP 1585 H=1585:00 +CPU= 16.658
  12:47:59 STEP 1586 H=1586:00 +CPU= 17.318
  12:48:27 STEP 1587 H=1587:00 +CPU= 27.198
  12:48:44 STEP 1588 H=1588:00 +CPU= 17.405
  12:49:01 STEP 1589 H=1589:00 +CPU= 16.697
  12:49:29 STEP 1590 H=1590:00 +CPU= 27.841
  12:49:46 STEP 1591 H=1591:00 +CPU= 16.912
  12:50:03 STEP 1592 H=1592:00 +CPU= 17.415
  12:50:31 STEP 1593 H=1593:00 +CPU= 27.314
  12:50:48 STEP 1594 H=1594:00 +CPU= 17.697
  12:51:05 STEP 1595 H=1595:00 +CPU= 16.723
  12:51:36 STEP 1596 H=1596:00 +CPU= 30.337
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001596
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001596
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001596
  12:51:52 STEP 1597 H=1597:00 +CPU= 16.793
  12:52:10 STEP 1598 H=1598:00 +CPU= 17.495
  12:52:37 STEP 1599 H=1599:00 +CPU= 27.058
  12:52:54 STEP 1600 H=1600:00 +CPU= 17.371
  12:53:11 STEP 1601 H=1601:00 +CPU= 16.805
  12:53:39 STEP 1602 H=1602:00 +CPU= 27.727
  12:53:56 STEP 1603 H=1603:00 +CPU= 16.589
  12:54:13 STEP 1604 H=1604:00 +CPU= 17.300
  12:54:40 STEP 1605 H=1605:00 +CPU= 27.015
  12:54:57 STEP 1606 H=1606:00 +CPU= 17.144
  12:55:14 STEP 1607 H=1607:00 +CPU= 17.015
  12:55:47 STEP 1608 H=1608:00 +CPU= 31.906
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001608
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001608
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001608
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001584
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001584
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001584
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001596
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001596
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001596
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_66.zip
Uploading the intermediate file: upload_file_66.zip
  12:56:04 STEP 1609 H=1609:00 +CPU= 17.206
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:06:26 STEP 1610 H=1610:00 +CPU= 17.593
  13:06:53 STEP 1611 H=1611:00 +CPU= 27.126
  13:07:11 STEP 1612 H=1612:00 +CPU= 17.488
  13:07:28 STEP 1613 H=1613:00 +CPU= 16.940
  13:07:56 STEP 1614 H=1614:00 +CPU= 28.200
  13:08:13 STEP 1615 H=1615:00 +CPU= 16.601
  13:08:30 STEP 1616 H=1616:00 +CPU= 17.145
  13:08:57 STEP 1617 H=1617:00 +CPU= 27.107
  13:09:14 STEP 1618 H=1618:00 +CPU= 17.308
  13:09:31 STEP 1619 H=1619:00 +CPU= 16.537
  13:10:01 STEP 1620 H=1620:00 +CPU= 29.975
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001620
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001620
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001620
  13:10:18 STEP 1621 H=1621:00 +CPU= 16.640
  13:10:35 STEP 1622 H=1622:00 +CPU= 17.382
  13:11:02 STEP 1623 H=1623:00 +CPU= 27.240
  13:11:20 STEP 1624 H=1624:00 +CPU= 17.304
  13:11:36 STEP 1625 H=1625:00 +CPU= 16.580
  13:12:04 STEP 1626 H=1626:00 +CPU= 27.679
  13:12:21 STEP 1627 H=1627:00 +CPU= 16.670
  13:12:38 STEP 1628 H=1628:00 +CPU= 17.228
  13:13:05 STEP 1629 H=1629:00 +CPU= 27.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:02:32 STEP 1630 H=1630:00 +CPU= 18.124
  14:02:50 STEP 1631 H=1631:00 +CPU= 17.840
  14:03:23 STEP 1632 H=1632:00 +CPU= 32.905
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001632
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001632
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001632
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001608
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001608
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001608
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001620
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001620
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001620
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_67.zip
Uploading the intermediate file: upload_file_67.zip
  14:03:41 STEP 1633 H=1633:00 +CPU= 17.947
Uploading trickle at timestep: 5875200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:20 STEP 1634 H=1634:00 +CPU= 18.008
  14:04:47 STEP 1635 H=1635:00 +CPU= 27.419
  14:05:05 STEP 1636 H=1636:00 +CPU= 17.456
  14:05:22 STEP 1637 H=1637:00 +CPU= 17.263
  14:05:51 STEP 1638 H=1638:00 +CPU= 29.085
  14:06:08 STEP 1639 H=1639:00 +CPU= 16.885
  14:06:27 STEP 1640 H=1640:00 +CPU= 18.061
  14:06:55 STEP 1641 H=1641:00 +CPU= 28.462
  14:07:13 STEP 1642 H=1642:00 +CPU= 17.802
  14:07:30 STEP 1643 H=1643:00 +CPU= 17.096
  14:08:01 STEP 1644 H=1644:00 +CPU= 30.844
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001644
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001644
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001644
  14:08:19 STEP 1645 H=1645:00 +CPU= 17.467
  14:08:37 STEP 1646 H=1646:00 +CPU= 18.142
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:09:28 STEP 1647 H=1647:00 +CPU= 28.993
  14:09:48 STEP 1648 H=1648:00 +CPU= 19.517
  14:10:07 STEP 1649 H=1649:00 +CPU= 19.020
  14:10:36 STEP 1650 H=1650:00 +CPU= 29.219
  14:10:55 STEP 1651 H=1651:00 +CPU= 18.889
  14:11:14 STEP 1652 H=1652:00 +CPU= 18.984
  14:11:44 STEP 1653 H=1653:00 +CPU= 30.011
  14:12:02 STEP 1654 H=1654:00 +CPU= 18.113
  14:12:21 STEP 1655 H=1655:00 +CPU= 18.101
  14:12:55 STEP 1656 H=1656:00 +CPU= 33.789
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001656
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001656
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001656
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001632
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001632
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001632
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001644
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001644
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001644
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_68.zip
Uploading the intermediate file: upload_file_68.zip
  14:13:12 STEP 1657 H=1657:00 +CPU= 17.416
  14:13:31 STEP 1658 H=1658:00 +CPU= 18.473
  14:14:00 STEP 1659 H=1659:00 +CPU= 29.503
  14:14:18 STEP 1660 H=1660:00 +CPU= 18.162
  14:14:35 STEP 1661 H=1661:00 +CPU= 16.763
  14:15:06 STEP 1662 H=1662:00 +CPU= 30.763
  14:15:26 STEP 1663 H=1663:00 +CPU= 19.632
  14:15:46 STEP 1664 H=1664:00 +CPU= 19.621
  14:16:15 STEP 1665 H=1665:00 +CPU= 29.095
  14:16:33 STEP 1666 H=1666:00 +CPU= 18.086
  14:16:51 STEP 1667 H=1667:00 +CPU= 18.365
  14:17:25 STEP 1668 H=1668:00 +CPU= 33.387
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001668
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001668
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001668
  14:17:43 STEP 1669 H=1669:00 +CPU= 17.945
  14:18:03 STEP 1670 H=1670:00 +CPU= 20.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:18:56 STEP 1671 H=1671:00 +CPU= 31.656
  14:19:14 STEP 1672 H=1672:00 +CPU= 18.012
  14:19:32 STEP 1673 H=1673:00 +CPU= 17.756
  14:20:02 STEP 1674 H=1674:00 +CPU= 30.068
  14:20:20 STEP 1675 H=1675:00 +CPU= 17.997
  14:20:39 STEP 1676 H=1676:00 +CPU= 19.104
  14:21:09 STEP 1677 H=1677:00 +CPU= 29.617
  14:21:29 STEP 1678 H=1678:00 +CPU= 19.233
  14:21:47 STEP 1679 H=1679:00 +CPU= 18.199
  14:22:19 STEP 1680 H=1680:00 +CPU= 31.558
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001680
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001680
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001680
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001656
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001656
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001656
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001668
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001668
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001668
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_69.zip
Uploading the intermediate file: upload_file_69.zip
  14:22:38 STEP 1681 H=1681:00 +CPU= 18.864
  14:22:58 STEP 1682 H=1682:00 +CPU= 19.949
  14:23:27 STEP 1683 H=1683:00 +CPU= 28.709
  14:23:46 STEP 1684 H=1684:00 +CPU= 18.911
  14:24:04 STEP 1685 H=1685:00 +CPU= 18.202
  14:24:35 STEP 1686 H=1686:00 +CPU= 31.115
  14:24:54 STEP 1687 H=1687:00 +CPU= 18.995
  14:25:15 STEP 1688 H=1688:00 +CPU= 20.772
  14:25:44 STEP 1689 H=1689:00 +CPU= 28.755
  14:26:05 STEP 1690 H=1690:00 +CPU= 20.707
  14:26:23 STEP 1691 H=1691:00 +CPU= 17.856
  14:26:55 STEP 1692 H=1692:00 +CPU= 32.095
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001692
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001692
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001692
  14:27:14 STEP 1693 H=1693:00 +CPU= 19.152
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:27:53 STEP 1694 H=1694:00 +CPU= 17.658
  14:28:23 STEP 1695 H=1695:00 +CPU= 30.395
  14:28:42 STEP 1696 H=1696:00 +CPU= 18.397
  14:29:00 STEP 1697 H=1697:00 +CPU= 18.057
  14:29:31 STEP 1698 H=1698:00 +CPU= 31.355
  14:29:50 STEP 1699 H=1699:00 +CPU= 18.224
  14:30:07 STEP 1700 H=1700:00 +CPU= 17.283
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:40:39 STEP 1701 H=1701:00 +CPU= 29.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:04:15 STEP 1702 H=1702:00 +CPU= 18.264
  15:04:36 STEP 1703 H=1703:00 +CPU= 20.328
  15:05:13 STEP 1704 H=1704:00 +CPU= 36.812
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001704
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001704
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001704
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001680
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001680
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001680
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001692
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001692
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001692
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_70.zip
Uploading the intermediate file: upload_file_70.zip
  15:05:33 STEP 1705 H=1705:00 +CPU= 19.854
  15:05:52 STEP 1706 H=1706:00 +CPU= 19.497
  15:06:24 STEP 1707 H=1707:00 +CPU= 31.422
  15:06:45 STEP 1708 H=1708:00 +CPU= 20.329
  15:07:06 STEP 1709 H=1709:00 +CPU= 20.974
Quit request received from BOINC client, ending the child process
15:07:16 (20883): BOINC client no longer exists - exiting
15:07:16 (20883): timer handler: client dead, exiting
15:07:21 (20883): BOINC client no longer exists - exiting
15:07:21 (20883): timer handler: client dead, exiting
15:07:26 (20883): BOINC client no longer exists - exiting
15:07:26 (20883): timer handler: client dead, exiting
15:07:31 (20883): BOINC client no longer exists - exiting
15:07:31 (20883): timer handler: client dead, exiting
15:07:36 (20883): BOINC client no longer exists - exiting
15:07:36 (20883): timer handler: client dead, exiting
15:07:41 (20883): BOINC client no longer exists - exiting
15:07:41 (20883): timer handler: client dead, exiting
15:07:46 (20883): BOINC client no longer exists - exiting
15:07:46 (20883): timer handler: client dead, exiting
15:07:51 (20883): BOINC client no longer exists - exiting
15:07:51 (20883): timer handler: client dead, exiting
15:07:56 (20883): BOINC client no longer exists - exiting
15:07:56 (20883): timer handler: client dead, exiting
15:08:01 (20883): BOINC client no longer exists - exiting
15:08:01 (20883): timer handler: client dead, exiting
15:08:06 (20883): BOINC client no longer exists - exiting
15:08:06 (20883): timer handler: client dead, exiting
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0890
(argv4) batchid: 977
(argv5) wuid: 12194534
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /mnt/sdc/home/seti/BOINC/slots/0
Project directory is: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534
..mkdir for temp folder for results failed
Copying: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0f549d298ad8ad7ef3459f157df8268d to: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_0890_2008050100_123_977_12194534.zip
Unzipping the namelist zip file: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_0890_2008050100_123_977_12194534.zip
ic_ancil_file: ic_ancil_12194534
ifsdata_file: ifsdata_12194534
climate_data_file: clim_data_12194534
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_7755f98133394211772ad41e723989d9 to: /mnt/sdc/home/seti/BOINC/slots/0/ic_ancil_12194534.zip
Unzipping the IC ancils zip file: /mnt/sdc/home/seti/BOINC/slots/0/ic_ancil_12194534.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a923a529b4be20437774f5b9554a4fa3 to: /mnt/sdc/home/seti/BOINC/slots/0/ifsdata/ifsdata_12194534.zip
Unzipping the ifsdata_zip file: /mnt/sdc/home/seti/BOINC/slots/0/ifsdata/ifsdata_12194534.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/sdc/home/seti/BOINC/slots/0/159l_2/clim_data_12194534.zip
Unzipping the climate data zip file: /mnt/sdc/home/seti/BOINC/slots/0/159l_2/clim_data_12194534.zip
Checking for progress XML file: /mnt/sdc/home/seti/BOINC/slots/0/progress_file_12194534.xml
Opened progress file ok : /mnt/sdc/home/seti/BOINC/slots/0/progress_file_12194534.xml
-- Model is restarting --
Adjusting last_iter, 1709, to previous model restart step.
Creating progress file: /mnt/sdc/home/seti/BOINC/slots/0/progress_file_12194534.xml
last_cpu_time: 39017
upload_file_number: 71
last_iter: 1703
last_upload: 6134400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616
The child process has been launched with process id: 29788
Executing the command: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:29788:29788] [20230102:102946:1672651786:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:30:38 STEP 1704 H=1704:00 +CPU= 32.864
  10:30:56 STEP 1705 H=1705:00 +CPU= 17.664
  10:31:15 STEP 1706 H=1706:00 +CPU= 18.429
  10:31:44 STEP 1707 H=1707:00 +CPU= 29.227
  10:32:04 STEP 1708 H=1708:00 +CPU= 19.107
  10:32:22 STEP 1709 H=1709:00 +CPU= 18.335
  10:32:52 STEP 1710 H=1710:00 +CPU= 29.733
  10:33:10 STEP 1711 H=1711:00 +CPU= 18.292
  10:33:29 STEP 1712 H=1712:00 +CPU= 18.795
  10:34:00 STEP 1713 H=1713:00 +CPU= 30.063
  10:34:18 STEP 1714 H=1714:00 +CPU= 18.373
  10:34:37 STEP 1715 H=1715:00 +CPU= 19.134
  10:35:10 STEP 1716 H=1716:00 +CPU= 32.255
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001716
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001716
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001716
  10:35:30 STEP 1717 H=1717:00 +CPU= 19.828
  10:35:49 STEP 1718 H=1718:00 +CPU= 18.841
  10:36:19 STEP 1719 H=1719:00 +CPU= 29.676
  10:36:37 STEP 1720 H=1720:00 +CPU= 17.954
  10:36:55 STEP 1721 H=1721:00 +CPU= 18.380
  10:37:25 STEP 1722 H=1722:00 +CPU= 29.769
  10:37:44 STEP 1723 H=1723:00 +CPU= 18.506
  10:38:02 STEP 1724 H=1724:00 +CPU= 18.256
  10:38:32 STEP 1725 H=1725:00 +CPU= 29.360
  10:38:51 STEP 1726 H=1726:00 +CPU= 19.192
  10:39:10 STEP 1727 H=1727:00 +CPU= 18.690
  10:39:44 STEP 1728 H=1728:00 +CPU= 33.535
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001728
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001728
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001728
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001704
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001704
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001704
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001716
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001716
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001716
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_71.zip
Uploading the intermediate file: upload_file_71.zip
  10:40:03 STEP 1729 H=1729:00 +CPU= 18.511
  10:40:22 STEP 1730 H=1730:00 +CPU= 19.404
  10:40:52 STEP 1731 H=1731:00 +CPU= 29.750
  10:41:10 STEP 1732 H=1732:00 +CPU= 18.017
  10:41:27 STEP 1733 H=1733:00 +CPU= 17.052
  10:41:57 STEP 1734 H=1734:00 +CPU= 29.013
  10:42:15 STEP 1735 H=1735:00 +CPU= 18.392
  10:42:33 STEP 1736 H=1736:00 +CPU= 18.072
  10:43:04 STEP 1737 H=1737:00 +CPU= 30.363
  10:43:23 STEP 1738 H=1738:00 +CPU= 19.183
  10:43:40 STEP 1739 H=1739:00 +CPU= 17.042
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:44:36 STEP 1740 H=1740:00 +CPU= 33.201
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001740
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001740
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001740
  10:44:54 STEP 1741 H=1741:00 +CPU= 17.882
  10:45:12 STEP 1742 H=1742:00 +CPU= 18.781
  10:45:42 STEP 1743 H=1743:00 +CPU= 29.051
  10:46:01 STEP 1744 H=1744:00 +CPU= 18.823
  10:46:20 STEP 1745 H=1745:00 +CPU= 19.243
  10:46:50 STEP 1746 H=1746:00 +CPU= 29.607
  10:47:09 STEP 1747 H=1747:00 +CPU= 18.969
  10:47:29 STEP 1748 H=1748:00 +CPU= 19.707
  10:47:58 STEP 1749 H=1749:00 +CPU= 29.147
  10:48:17 STEP 1750 H=1750:00 +CPU= 18.545
  10:48:35 STEP 1751 H=1751:00 +CPU= 18.510
  10:49:09 STEP 1752 H=1752:00 +CPU= 32.904
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001752
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001752
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001752
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001728
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001728
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001728
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001740
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001740
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001740
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_72.zip
Uploading the intermediate file: upload_file_72.zip
  10:49:27 STEP 1753 H=1753:00 +CPU= 17.895
  10:49:45 STEP 1754 H=1754:00 +CPU= 18.727
  10:50:15 STEP 1755 H=1755:00 +CPU= 29.284
  10:50:36 STEP 1756 H=1756:00 +CPU= 20.702
  10:50:54 STEP 1757 H=1757:00 +CPU= 18.478
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0890
(argv4) batchid: 977
(argv5) wuid: 12194534
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /mnt/sdc/home/seti/BOINC/slots/0
Project directory is: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534
..mkdir for temp folder for results failed
Copying: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0f549d298ad8ad7ef3459f157df8268d to: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_0890_2008050100_123_977_12194534.zip
Unzipping the namelist zip file: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_ps_0890_2008050100_123_977_12194534.zip
ic_ancil_file: ic_ancil_12194534
ifsdata_file: ifsdata_12194534
climate_data_file: clim_data_12194534
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_7755f98133394211772ad41e723989d9 to: /mnt/sdc/home/seti/BOINC/slots/0/ic_ancil_12194534.zip
Unzipping the IC ancils zip file: /mnt/sdc/home/seti/BOINC/slots/0/ic_ancil_12194534.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a923a529b4be20437774f5b9554a4fa3 to: /mnt/sdc/home/seti/BOINC/slots/0/ifsdata/ifsdata_12194534.zip
Unzipping the ifsdata_zip file: /mnt/sdc/home/seti/BOINC/slots/0/ifsdata/ifsdata_12194534.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/sdc/home/seti/BOINC/slots/0/159l_2/clim_data_12194534.zip
Unzipping the climate data zip file: /mnt/sdc/home/seti/BOINC/slots/0/159l_2/clim_data_12194534.zip
Checking for progress XML file: /mnt/sdc/home/seti/BOINC/slots/0/progress_file_12194534.xml
Opened progress file ok : /mnt/sdc/home/seti/BOINC/slots/0/progress_file_12194534.xml
-- Model is restarting --
Adjusting last_iter, 1758, to previous model restart step.
Creating progress file: /mnt/sdc/home/seti/BOINC/slots/0/progress_file_12194534.xml
last_cpu_time: 40246
upload_file_number: 73
last_iter: 1751
last_upload: 6307200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616
The child process has been launched with process id: 22461
Executing the command: /mnt/sdc/home/seti/BOINC/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:merkur:1:1:22461:22461] [20230105:091445:1672906485:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:16:29 STEP 1752 H=1752:00 +CPU= 30.905
  09:16:46 STEP 1753 H=1753:00 +CPU= 16.828
  09:17:03 STEP 1754 H=1754:00 +CPU= 17.291
  09:17:30 STEP 1755 H=1755:00 +CPU= 27.133
  09:17:48 STEP 1756 H=1756:00 +CPU= 17.448
  09:18:05 STEP 1757 H=1757:00 +CPU= 16.869
  09:18:33 STEP 1758 H=1758:00 +CPU= 28.163
  09:18:50 STEP 1759 H=1759:00 +CPU= 16.983
  09:19:08 STEP 1760 H=1760:00 +CPU= 17.543
  09:19:35 STEP 1761 H=1761:00 +CPU= 27.253
  09:19:53 STEP 1762 H=1762:00 +CPU= 17.967
  09:20:10 STEP 1763 H=1763:00 +CPU= 16.846
  09:20:40 STEP 1764 H=1764:00 +CPU= 29.943
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001764
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001764
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001764
  09:20:57 STEP 1765 H=1765:00 +CPU= 16.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:21:35 STEP 1766 H=1766:00 +CPU= 17.480
  09:22:02 STEP 1767 H=1767:00 +CPU= 27.127
  09:22:20 STEP 1768 H=1768:00 +CPU= 17.411
  09:22:37 STEP 1769 H=1769:00 +CPU= 16.791
  09:23:05 STEP 1770 H=1770:00 +CPU= 27.795
  09:23:21 STEP 1771 H=1771:00 +CPU= 16.641
  09:23:39 STEP 1772 H=1772:00 +CPU= 17.322
  09:24:07 STEP 1773 H=1773:00 +CPU= 27.748
  09:24:24 STEP 1774 H=1774:00 +CPU= 17.398
  09:24:41 STEP 1775 H=1775:00 +CPU= 17.333
  09:25:13 STEP 1776 H=1776:00 +CPU= 31.343
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001776
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001776
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001776
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001752
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001752
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001752
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001764
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001764
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001764
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_73.zip
Uploading the intermediate file: upload_file_73.zip
  09:25:30 STEP 1777 H=1777:00 +CPU= 16.746
  09:25:47 STEP 1778 H=1778:00 +CPU= 17.269
  09:26:15 STEP 1779 H=1779:00 +CPU= 27.715
  09:26:32 STEP 1780 H=1780:00 +CPU= 17.291
  09:26:49 STEP 1781 H=1781:00 +CPU= 17.037
  09:27:17 STEP 1782 H=1782:00 +CPU= 27.725
  09:27:34 STEP 1783 H=1783:00 +CPU= 16.885
  09:27:52 STEP 1784 H=1784:00 +CPU= 17.965
  09:28:19 STEP 1785 H=1785:00 +CPU= 27.226
  09:28:37 STEP 1786 H=1786:00 +CPU= 17.611
  09:28:54 STEP 1787 H=1787:00 +CPU= 16.961
  09:29:25 STEP 1788 H=1788:00 +CPU= 30.669
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001788
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001788
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001788
  09:29:42 STEP 1789 H=1789:00 +CPU= 16.735
  09:29:59 STEP 1790 H=1790:00 +CPU= 17.317
  09:30:27 STEP 1791 H=1791:00 +CPU= 27.386
  09:30:44 STEP 1792 H=1792:00 +CPU= 17.528
  09:31:01 STEP 1793 H=1793:00 +CPU= 17.038
  09:31:29 STEP 1794 H=1794:00 +CPU= 27.726
  09:31:46 STEP 1795 H=1795:00 +CPU= 16.737
  09:32:03 STEP 1796 H=1796:00 +CPU= 17.496
  09:32:31 STEP 1797 H=1797:00 +CPU= 27.554
  09:32:49 STEP 1798 H=1798:00 +CPU= 17.407
  09:33:06 STEP 1799 H=1799:00 +CPU= 17.365
  09:33:38 STEP 1800 H=1800:00 +CPU= 31.636
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001800
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001800
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001800
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001776
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001776
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001776
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001788
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001788
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001788
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_74.zip
Uploading the intermediate file: upload_file_74.zip
  09:33:55 STEP 1801 H=1801:00 +CPU= 16.940
  09:34:12 STEP 1802 H=1802:00 +CPU= 17.516
  09:34:39 STEP 1803 H=1803:00 +CPU= 27.045
  09:34:57 STEP 1804 H=1804:00 +CPU= 17.151
  09:35:14 STEP 1805 H=1805:00 +CPU= 16.913
  09:35:42 STEP 1806 H=1806:00 +CPU= 28.406
  09:36:00 STEP 1807 H=1807:00 +CPU= 17.358
  09:36:18 STEP 1808 H=1808:00 +CPU= 17.826
  09:36:47 STEP 1809 H=1809:00 +CPU= 28.530
  09:37:06 STEP 1810 H=1810:00 +CPU= 18.497
  09:37:24 STEP 1811 H=1811:00 +CPU= 17.689
  09:37:55 STEP 1812 H=1812:00 +CPU= 30.649
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001812
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001812
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001812
  09:38:12 STEP 1813 H=1813:00 +CPU= 17.372
  09:38:31 STEP 1814 H=1814:00 +CPU= 18.014
  09:38:59 STEP 1815 H=1815:00 +CPU= 28.260
  09:39:18 STEP 1816 H=1816:00 +CPU= 18.139
  09:39:35 STEP 1817 H=1817:00 +CPU= 17.019
  09:40:03 STEP 1818 H=1818:00 +CPU= 27.905
  09:40:20 STEP 1819 H=1819:00 +CPU= 17.358
  09:40:38 STEP 1820 H=1820:00 +CPU= 17.470
  09:41:05 STEP 1821 H=1821:00 +CPU= 27.234
  09:41:23 STEP 1822 H=1822:00 +CPU= 17.807
  09:41:41 STEP 1823 H=1823:00 +CPU= 17.788
  09:42:14 STEP 1824 H=1824:00 +CPU= 32.635
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001824
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001824
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001824
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001800
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001800
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001800
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001812
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001812
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001812
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_75.zip
Uploading the intermediate file: upload_file_75.zip
  09:42:32 STEP 1825 H=1825:00 +CPU= 17.294
  09:42:50 STEP 1826 H=1826:00 +CPU= 18.281
  09:43:19 STEP 1827 H=1827:00 +CPU= 28.505
  09:43:38 STEP 1828 H=1828:00 +CPU= 18.584
  09:43:56 STEP 1829 H=1829:00 +CPU= 17.125
  09:44:24 STEP 1830 H=1830:00 +CPU= 28.112
  09:44:41 STEP 1831 H=1831:00 +CPU= 17.066
  09:44:58 STEP 1832 H=1832:00 +CPU= 17.331
  09:45:26 STEP 1833 H=1833:00 +CPU= 27.521
  09:45:43 STEP 1834 H=1834:00 +CPU= 17.381
  09:46:01 STEP 1835 H=1835:00 +CPU= 17.403
  09:46:31 STEP 1836 H=1836:00 +CPU= 30.148
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001836
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001836
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001836
  09:46:48 STEP 1837 H=1837:00 +CPU= 17.010
  09:47:06 STEP 1838 H=1838:00 +CPU= 17.514
  09:47:34 STEP 1839 H=1839:00 +CPU= 27.713
  09:47:51 STEP 1840 H=1840:00 +CPU= 17.422
  09:48:08 STEP 1841 H=1841:00 +CPU= 16.847
  09:48:36 STEP 1842 H=1842:00 +CPU= 27.924
  09:48:53 STEP 1843 H=1843:00 +CPU= 16.914
  09:49:11 STEP 1844 H=1844:00 +CPU= 17.674
  09:49:38 STEP 1845 H=1845:00 +CPU= 27.373
  09:49:56 STEP 1846 H=1846:00 +CPU= 17.302
  09:50:13 STEP 1847 H=1847:00 +CPU= 17.193
  09:50:44 STEP 1848 H=1848:00 +CPU= 31.218
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001848
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001848
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001848
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001824
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001824
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001824
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001836
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001836
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001836
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_76.zip
Uploading the intermediate file: upload_file_76.zip
  09:51:02 STEP 1849 H=1849:00 +CPU= 17.078
  09:51:19 STEP 1850 H=1850:00 +CPU= 17.333
  09:51:47 STEP 1851 H=1851:00 +CPU= 27.825
  09:52:04 STEP 1852 H=1852:00 +CPU= 17.163
  09:52:21 STEP 1853 H=1853:00 +CPU= 16.887
  09:52:49 STEP 1854 H=1854:00 +CPU= 27.896
  09:53:06 STEP 1855 H=1855:00 +CPU= 16.877
  09:53:24 STEP 1856 H=1856:00 +CPU= 17.335
  09:53:52 STEP 1857 H=1857:00 +CPU= 27.829
  09:54:09 STEP 1858 H=1858:00 +CPU= 17.259
  09:54:26 STEP 1859 H=1859:00 +CPU= 17.077
  09:54:56 STEP 1860 H=1860:00 +CPU= 30.146
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001860
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001860
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001860
  09:55:13 STEP 1861 H=1861:00 +CPU= 16.849
  09:55:31 STEP 1862 H=1862:00 +CPU= 17.402
  09:55:58 STEP 1863 H=1863:00 +CPU= 27.109
  09:56:15 STEP 1864 H=1864:00 +CPU= 17.310
  09:56:32 STEP 1865 H=1865:00 +CPU= 16.753
  09:57:00 STEP 1866 H=1866:00 +CPU= 27.927
  09:57:17 STEP 1867 H=1867:00 +CPU= 16.761
  09:57:35 STEP 1868 H=1868:00 +CPU= 17.823
  09:58:02 STEP 1869 H=1869:00 +CPU= 26.962
  09:58:20 STEP 1870 H=1870:00 +CPU= 17.816
  09:58:37 STEP 1871 H=1871:00 +CPU= 17.231
  09:59:09 STEP 1872 H=1872:00 +CPU= 31.806
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001872
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001872
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001872
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001848
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001848
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001848
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001860
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001860
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001860
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_77.zip
Uploading the intermediate file: upload_file_77.zip
  09:59:26 STEP 1873 H=1873:00 +CPU= 16.719
  09:59:43 STEP 1874 H=1874:00 +CPU= 17.368
  10:00:11 STEP 1875 H=1875:00 +CPU= 27.183
  10:00:28 STEP 1876 H=1876:00 +CPU= 17.364
  10:00:45 STEP 1877 H=1877:00 +CPU= 17.258
  10:01:13 STEP 1878 H=1878:00 +CPU= 27.802
  10:01:30 STEP 1879 H=1879:00 +CPU= 16.690
  10:01:47 STEP 1880 H=1880:00 +CPU= 17.246
  10:02:15 STEP 1881 H=1881:00 +CPU= 27.350
  10:02:32 STEP 1882 H=1882:00 +CPU= 17.257
  10:02:49 STEP 1883 H=1883:00 +CPU= 16.691
  10:03:19 STEP 1884 H=1884:00 +CPU= 30.439
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001884
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001884
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001884
  10:03:36 STEP 1885 H=1885:00 +CPU= 16.756
  10:03:53 STEP 1886 H=1886:00 +CPU= 17.104
  10:04:21 STEP 1887 H=1887:00 +CPU= 27.263
  10:04:38 STEP 1888 H=1888:00 +CPU= 17.462
  10:04:55 STEP 1889 H=1889:00 +CPU= 16.698
  10:05:23 STEP 1890 H=1890:00 +CPU= 27.732
  10:05:40 STEP 1891 H=1891:00 +CPU= 17.057
  10:05:58 STEP 1892 H=1892:00 +CPU= 17.694
  10:06:25 STEP 1893 H=1893:00 +CPU= 27.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:07:04 STEP 1894 H=1894:00 +CPU= 17.338
  10:07:21 STEP 1895 H=1895:00 +CPU= 17.444
  10:07:53 STEP 1896 H=1896:00 +CPU= 31.505
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMGGhq0f+001896
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMSHhq0f+001896
Moving to projects directory: /mnt/sdc/home/seti/BOINC/slots/0/ICMUAhq0f+001896
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001872
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001872
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001872
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMGGhq0f+001884
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMSHhq0f+001884
Adding to the zip: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_12194534/ICMUAhq0f+001884
Zipping up the intermediate file: /mnt/sdc/home/seti/BOINC/projects/climateprediction.net/oifs_43r3_ps_0890_2008050100_123_977_12194534_0_r1275881616_78.zip
Uploading the intermediate file: upload_file_78.zip
  10:08:10 STEP 1897 H=1897:00 +CPU= 16.861
  10:08:27 STEP 1898 H=1898:00 +CPU= 17.697
  10:08:55 STEP 1899 H=1899:00 +CPU= 27.777
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:05:07 1220995 22280390 oifs_43r3_ps_0890_2008050100_123_977_12194534_0 5,875,200 37,299 0.0063
09 May 2023 16:04:05 1220995 22280390 oifs_43r3_ps_0890_2008050100_123_977_12194534_0 4,665,600 29,121 0.0062
09 May 2023 16:02:56 1220995 22280390 oifs_43r3_ps_0890_2008050100_123_977_12194534_0 3,801,600 23,999 0.0063
09 May 2023 16:02:48 1220995 22280390 oifs_43r3_ps_0890_2008050100_123_977_12194534_0 2,937,600 19,024 0.0065
09 May 2023 16:02:48 1220995 22280390 oifs_43r3_ps_0890_2008050100_123_977_12194534_0 2,937,600 19,024 0.0065
09 May 2023 15:58:52 1220995 22280390 oifs_43r3_ps_0890_2008050100_123_977_12194534_0 864,000 5,996 0.0069


©2024 climateprediction.net