climateprediction.net home page
Task 22442841

Task 22442841

Name oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0
Workunit 12289256
Created 12 Jun 2024, 21:18:55 UTC
Sent 14 Jun 2024, 4:09:20 UTC
Report deadline 13 Aug 2024, 4:09:20 UTC
Received 15 Jun 2024, 13:07:46 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1318882
Run time 14 hours 15 min 21 sec
CPU time 14 hours 15 min 21 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 6.46 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,515.07 MB
Peak swap size 5,981.08 MB
Peak disk usage 1,351.06 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
t/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0_r840727218_14.zip
Uploading the intermediate file: upload_file_14.zip
  10:17:38 STEP 1441 H= 360:15 +CPU= 25.352
  10:18:04 STEP 1442 H= 360:30 +CPU= 23.918
  10:18:29 STEP 1443 H= 360:45 +CPU= 23.273
  10:18:57 STEP 1444 H= 361:00 +CPU= 24.740
  10:19:21 STEP 1445 H= 361:15 +CPU= 22.785
  10:19:46 STEP 1446 H= 361:30 +CPU= 22.616
  10:20:15 STEP 1447 H= 361:45 +CPU= 25.877
  10:20:41 STEP 1448 H= 362:00 +CPU= 23.699
  10:21:10 STEP 1449 H= 362:15 +CPU= 25.505
  10:21:36 STEP 1450 H= 362:30 +CPU= 24.236
  10:22:02 STEP 1451 H= 362:45 +CPU= 23.805
  10:22:37 STEP 1452 H= 363:00 +CPU= 31.715
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001452
  10:23:03 STEP 1453 H= 363:15 +CPU= 23.915
  10:23:32 STEP 1454 H= 363:30 +CPU= 25.957
  10:24:00 STEP 1455 H= 363:45 +CPU= 25.168
  10:24:29 STEP 1456 H= 364:00 +CPU= 25.655
  10:24:54 STEP 1457 H= 364:15 +CPU= 23.281
  10:25:21 STEP 1458 H= 364:30 +CPU= 24.411
  10:25:50 STEP 1459 H= 364:45 +CPU= 25.058
  10:26:18 STEP 1460 H= 365:00 +CPU= 25.334
  10:26:44 STEP 1461 H= 365:15 +CPU= 23.692
  10:27:11 STEP 1462 H= 365:30 +CPU= 24.005
  10:27:37 STEP 1463 H= 365:45 +CPU= 23.630
  10:28:10 STEP 1464 H= 366:00 +CPU= 29.818
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001464
  10:28:35 STEP 1465 H= 366:15 +CPU= 22.881
  10:28:59 STEP 1466 H= 366:30 +CPU= 21.836
  10:29:25 STEP 1467 H= 366:45 +CPU= 24.113
  10:29:54 STEP 1468 H= 367:00 +CPU= 25.835
  10:30:22 STEP 1469 H= 367:15 +CPU= 25.380
  10:30:50 STEP 1470 H= 367:30 +CPU= 24.781
  10:31:19 STEP 1471 H= 367:45 +CPU= 25.390
  10:31:46 STEP 1472 H= 368:00 +CPU= 24.679
  10:32:12 STEP 1473 H= 368:15 +CPU= 23.530
  10:32:39 STEP 1474 H= 368:30 +CPU= 24.528
  10:33:06 STEP 1475 H= 368:45 +CPU= 24.332
  10:33:39 STEP 1476 H= 369:00 +CPU= 29.929
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001476
  10:34:06 STEP 1477 H= 369:15 +CPU= 23.769
  10:34:31 STEP 1478 H= 369:30 +CPU= 23.088
  10:34:58 STEP 1479 H= 369:45 +CPU= 24.724
  10:35:26 STEP 1480 H= 370:00 +CPU= 25.027
  10:35:52 STEP 1481 H= 370:15 +CPU= 23.796
  10:36:18 STEP 1482 H= 370:30 +CPU= 23.418
  10:36:43 STEP 1483 H= 370:45 +CPU= 23.042
  10:37:07 STEP 1484 H= 371:00 +CPU= 22.956
  10:37:31 STEP 1485 H= 371:15 +CPU= 22.540
  10:37:56 STEP 1486 H= 371:30 +CPU= 22.774
  10:38:20 STEP 1487 H= 371:45 +CPU= 22.521
  10:38:54 STEP 1488 H= 372:00 +CPU= 30.913
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001488
  10:39:18 STEP 1489 H= 372:15 +CPU= 22.362
  10:39:42 STEP 1490 H= 372:30 +CPU= 22.758
  10:40:06 STEP 1491 H= 372:45 +CPU= 22.836
  10:40:33 STEP 1492 H= 373:00 +CPU= 24.363
  10:40:59 STEP 1493 H= 373:15 +CPU= 23.893
  10:41:23 STEP 1494 H= 373:30 +CPU= 23.082
  10:41:48 STEP 1495 H= 373:45 +CPU= 23.188
  10:42:13 STEP 1496 H= 374:00 +CPU= 23.467
  10:42:38 STEP 1497 H= 374:15 +CPU= 22.863
  10:43:02 STEP 1498 H= 374:30 +CPU= 22.933
  10:43:29 STEP 1499 H= 374:45 +CPU= 24.653
  10:44:03 STEP 1500 H= 375:00 +CPU= 31.227
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001500
  10:44:28 STEP 1501 H= 375:15 +CPU= 22.868
  10:44:52 STEP 1502 H= 375:30 +CPU= 22.476
  10:45:16 STEP 1503 H= 375:45 +CPU= 22.595
  10:45:41 STEP 1504 H= 376:00 +CPU= 23.939
  10:46:07 STEP 1505 H= 376:15 +CPU= 23.502
  10:46:31 STEP 1506 H= 376:30 +CPU= 22.648
  10:46:55 STEP 1507 H= 376:45 +CPU= 22.567
  10:47:20 STEP 1508 H= 377:00 +CPU= 23.192
  10:47:44 STEP 1509 H= 377:15 +CPU= 22.836
  10:48:08 STEP 1510 H= 377:30 +CPU= 22.718
  10:48:33 STEP 1511 H= 377:45 +CPU= 23.455
  10:49:06 STEP 1512 H= 378:00 +CPU= 30.192
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001512
  10:49:32 STEP 1513 H= 378:15 +CPU= 24.105
  10:49:56 STEP 1514 H= 378:30 +CPU= 22.946
  10:50:24 STEP 1515 H= 378:45 +CPU= 25.170
  10:50:51 STEP 1516 H= 379:00 +CPU= 24.755
  10:51:17 STEP 1517 H= 379:15 +CPU= 23.582
  10:51:42 STEP 1518 H= 379:30 +CPU= 23.724
  10:52:07 STEP 1519 H= 379:45 +CPU= 22.650
  10:52:32 STEP 1520 H= 380:00 +CPU= 23.328
  10:52:59 STEP 1521 H= 380:15 +CPU= 24.870
  10:53:24 STEP 1522 H= 380:30 +CPU= 23.721
  10:53:50 STEP 1523 H= 380:45 +CPU= 24.057
  10:54:18 STEP 1524 H= 381:00 +CPU= 26.396
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001524
  10:54:43 STEP 1525 H= 381:15 +CPU= 23.125
  10:55:08 STEP 1526 H= 381:30 +CPU= 22.976
  10:55:33 STEP 1527 H= 381:45 +CPU= 23.214
  10:55:59 STEP 1528 H= 382:00 +CPU= 24.253
  10:56:23 STEP 1529 H= 382:15 +CPU= 22.722
  10:56:47 STEP 1530 H= 382:30 +CPU= 22.771
  10:57:13 STEP 1531 H= 382:45 +CPU= 23.645
  10:57:39 STEP 1532 H= 383:00 +CPU= 24.505
  10:58:03 STEP 1533 H= 383:15 +CPU= 22.523
  10:58:27 STEP 1534 H= 383:30 +CPU= 22.604
  10:58:51 STEP 1535 H= 383:45 +CPU= 22.933
  10:59:25 STEP 1536 H= 384:00 +CPU= 30.793
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0_r840727218_15.zip
Uploading the intermediate file: upload_file_15.zip
  10:59:51 STEP 1537 H= 384:15 +CPU= 23.854
  11:00:18 STEP 1538 H= 384:30 +CPU= 24.697
  11:00:49 STEP 1539 H= 384:45 +CPU= 26.591
  11:01:18 STEP 1540 H= 385:00 +CPU= 25.705
  11:01:44 STEP 1541 H= 385:15 +CPU= 24.014
  11:02:10 STEP 1542 H= 385:30 +CPU= 24.242
  11:02:37 STEP 1543 H= 385:45 +CPU= 24.720
  11:03:04 STEP 1544 H= 386:00 +CPU= 24.597
  11:03:29 STEP 1545 H= 386:15 +CPU= 23.223
  11:03:55 STEP 1546 H= 386:30 +CPU= 23.954
  11:04:24 STEP 1547 H= 386:45 +CPU= 25.942
  11:05:02 STEP 1548 H= 387:00 +CPU= 33.405
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001548
  11:05:30 STEP 1549 H= 387:15 +CPU= 26.004
  11:05:58 STEP 1550 H= 387:30 +CPU= 24.667
  11:06:24 STEP 1551 H= 387:45 +CPU= 24.288
  11:06:50 STEP 1552 H= 388:00 +CPU= 24.079
  11:07:15 STEP 1553 H= 388:15 +CPU= 23.273
  11:07:40 STEP 1554 H= 388:30 +CPU= 23.154
  11:08:05 STEP 1555 H= 388:45 +CPU= 23.505
  11:08:32 STEP 1556 H= 389:00 +CPU= 24.609
  11:08:56 STEP 1557 H= 389:15 +CPU= 22.852
  11:09:23 STEP 1558 H= 389:30 +CPU= 24.430
  11:09:49 STEP 1559 H= 389:45 +CPU= 23.847
  11:10:21 STEP 1560 H= 390:00 +CPU= 29.647
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001560
  11:10:47 STEP 1561 H= 390:15 +CPU= 23.948
  11:11:14 STEP 1562 H= 390:30 +CPU= 24.092
  11:11:38 STEP 1563 H= 390:45 +CPU= 22.443
  11:12:05 STEP 1564 H= 391:00 +CPU= 24.460
  11:12:36 STEP 1565 H= 391:15 +CPU= 27.142
  11:13:05 STEP 1566 H= 391:30 +CPU= 26.099
  11:13:32 STEP 1567 H= 391:45 +CPU= 24.154
  11:13:57 STEP 1568 H= 392:00 +CPU= 23.173
  11:14:23 STEP 1569 H= 392:15 +CPU= 23.674
  11:14:49 STEP 1570 H= 392:30 +CPU= 24.035
  11:15:15 STEP 1571 H= 392:45 +CPU= 23.962
  11:15:48 STEP 1572 H= 393:00 +CPU= 29.994
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001572
  11:16:18 STEP 1573 H= 393:15 +CPU= 26.363
  11:16:46 STEP 1574 H= 393:30 +CPU= 24.319
  11:17:12 STEP 1575 H= 393:45 +CPU= 23.692
  11:17:41 STEP 1576 H= 394:00 +CPU= 25.941
  11:18:11 STEP 1577 H= 394:15 +CPU= 26.702
  11:18:39 STEP 1578 H= 394:30 +CPU= 25.059
  11:19:05 STEP 1579 H= 394:45 +CPU= 23.980
  11:19:34 STEP 1580 H= 395:00 +CPU= 26.125
  11:20:03 STEP 1581 H= 395:15 +CPU= 25.827
  11:20:29 STEP 1582 H= 395:30 +CPU= 23.836
  11:20:55 STEP 1583 H= 395:45 +CPU= 23.880
  11:21:31 STEP 1584 H= 396:00 +CPU= 32.517
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001584
  11:21:55 STEP 1585 H= 396:15 +CPU= 23.186
  11:22:21 STEP 1586 H= 396:30 +CPU= 23.685
  11:22:46 STEP 1587 H= 396:45 +CPU= 23.367
  11:23:12 STEP 1588 H= 397:00 +CPU= 24.465
  11:23:37 STEP 1589 H= 397:15 +CPU= 23.220
  11:24:02 STEP 1590 H= 397:30 +CPU= 22.980
  11:24:26 STEP 1591 H= 397:45 +CPU= 23.016
  11:24:52 STEP 1592 H= 398:00 +CPU= 24.299
  11:25:19 STEP 1593 H= 398:15 +CPU= 24.144
  11:25:43 STEP 1594 H= 398:30 +CPU= 22.924
  11:26:09 STEP 1595 H= 398:45 +CPU= 23.824
  11:26:39 STEP 1596 H= 399:00 +CPU= 28.386
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001596
  11:27:03 STEP 1597 H= 399:15 +CPU= 22.705
  11:27:28 STEP 1598 H= 399:30 +CPU= 22.660
  11:27:53 STEP 1599 H= 399:45 +CPU= 22.876
  11:28:18 STEP 1600 H= 400:00 +CPU= 23.186
  11:28:43 STEP 1601 H= 400:15 +CPU= 22.613
  11:29:06 STEP 1602 H= 400:30 +CPU= 22.264
  11:29:31 STEP 1603 H= 400:45 +CPU= 22.377
  11:29:56 STEP 1604 H= 401:00 +CPU= 23.497
  11:30:22 STEP 1605 H= 401:15 +CPU= 23.798
  11:30:46 STEP 1606 H= 401:30 +CPU= 22.722
  11:31:13 STEP 1607 H= 401:45 +CPU= 23.857
  11:31:46 STEP 1608 H= 402:00 +CPU= 30.830
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001608
  11:32:13 STEP 1609 H= 402:15 +CPU= 24.466
  11:32:40 STEP 1610 H= 402:30 +CPU= 24.517
  11:33:06 STEP 1611 H= 402:45 +CPU= 24.390
  11:33:38 STEP 1612 H= 403:00 +CPU= 27.460
  11:34:09 STEP 1613 H= 403:15 +CPU= 26.867
  11:34:38 STEP 1614 H= 403:30 +CPU= 25.990
  11:35:09 STEP 1615 H= 403:45 +CPU= 26.891
  11:35:41 STEP 1616 H= 404:00 +CPU= 28.017
  11:36:10 STEP 1617 H= 404:15 +CPU= 26.304
  11:36:41 STEP 1618 H= 404:30 +CPU= 27.051
  11:37:09 STEP 1619 H= 404:45 +CPU= 25.732
  11:37:43 STEP 1620 H= 405:00 +CPU= 30.854
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001620
  11:38:10 STEP 1621 H= 405:15 +CPU= 24.275
  11:38:36 STEP 1622 H= 405:30 +CPU= 24.663
  11:39:02 STEP 1623 H= 405:45 +CPU= 23.995
  11:39:29 STEP 1624 H= 406:00 +CPU= 24.885
  11:39:56 STEP 1625 H= 406:15 +CPU= 23.961
  11:40:23 STEP 1626 H= 406:30 +CPU= 24.775
  11:40:48 STEP 1627 H= 406:45 +CPU= 23.568
  11:41:16 STEP 1628 H= 407:00 +CPU= 25.548
  11:41:46 STEP 1629 H= 407:15 +CPU= 26.402
  11:42:12 STEP 1630 H= 407:30 +CPU= 24.033
  11:42:37 STEP 1631 H= 407:45 +CPU= 23.441
  11:43:10 STEP 1632 H= 408:00 +CPU= 30.376
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0_r840727218_16.zip
Uploading the intermediate file: upload_file_16.zip
  11:43:35 STEP 1633 H= 408:15 +CPU= 23.132
  11:44:01 STEP 1634 H= 408:30 +CPU= 24.314
  11:44:26 STEP 1635 H= 408:45 +CPU= 23.267
  11:44:52 STEP 1636 H= 409:00 +CPU= 23.994
  11:45:17 STEP 1637 H= 409:15 +CPU= 22.924
  11:45:42 STEP 1638 H= 409:30 +CPU= 23.000
  11:46:08 STEP 1639 H= 409:45 +CPU= 23.667
  11:46:36 STEP 1640 H= 410:00 +CPU= 25.350
  11:47:01 STEP 1641 H= 410:15 +CPU= 23.617
  11:47:29 STEP 1642 H= 410:30 +CPU= 25.057
  11:47:56 STEP 1643 H= 410:45 +CPU= 24.805
  11:48:28 STEP 1644 H= 411:00 +CPU= 29.378
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001644
  11:48:53 STEP 1645 H= 411:15 +CPU= 23.617
  11:49:19 STEP 1646 H= 411:30 +CPU= 23.604
  11:49:43 STEP 1647 H= 411:45 +CPU= 23.022
  11:50:11 STEP 1648 H= 412:00 +CPU= 25.295
  11:50:38 STEP 1649 H= 412:15 +CPU= 24.511
  11:51:04 STEP 1650 H= 412:30 +CPU= 24.095
  11:51:31 STEP 1651 H= 412:45 +CPU= 24.886
  11:52:00 STEP 1652 H= 413:00 +CPU= 25.564
  11:52:27 STEP 1653 H= 413:15 +CPU= 25.166
  11:52:55 STEP 1654 H= 413:30 +CPU= 25.108
  11:53:23 STEP 1655 H= 413:45 +CPU= 25.503
  11:53:57 STEP 1656 H= 414:00 +CPU= 31.146
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001656
  11:54:25 STEP 1657 H= 414:15 +CPU= 25.620
  11:54:55 STEP 1658 H= 414:30 +CPU= 26.242
  11:55:24 STEP 1659 H= 414:45 +CPU= 25.631
  11:55:51 STEP 1660 H= 415:00 +CPU= 25.376
  11:56:18 STEP 1661 H= 415:15 +CPU= 24.874
  11:56:46 STEP 1662 H= 415:30 +CPU= 25.028
  11:57:12 STEP 1663 H= 415:45 +CPU= 24.464
  11:57:39 STEP 1664 H= 416:00 +CPU= 24.857
  11:58:05 STEP 1665 H= 416:15 +CPU= 24.114
  11:58:32 STEP 1666 H= 416:30 +CPU= 24.830
  11:59:02 STEP 1667 H= 416:45 +CPU= 26.489
  11:59:37 STEP 1668 H= 417:00 +CPU= 31.812
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001668
  12:00:05 STEP 1669 H= 417:15 +CPU= 25.588
  12:00:33 STEP 1670 H= 417:30 +CPU= 25.265
  12:01:00 STEP 1671 H= 417:45 +CPU= 24.747
  12:01:27 STEP 1672 H= 418:00 +CPU= 24.562
  12:01:56 STEP 1673 H= 418:15 +CPU= 26.220
  12:02:18 STEP 1674 H= 418:30 +CPU= 21.180
  12:02:41 STEP 1675 H= 418:45 +CPU= 21.297
  12:03:02 STEP 1676 H= 419:00 +CPU= 20.820
  12:03:23 STEP 1677 H= 419:15 +CPU= 20.236
  12:03:45 STEP 1678 H= 419:30 +CPU= 20.575
  12:04:07 STEP 1679 H= 419:45 +CPU= 21.185
  12:04:37 STEP 1680 H= 420:00 +CPU= 28.377
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001680
  12:04:58 STEP 1681 H= 420:15 +CPU= 20.245
  12:05:20 STEP 1682 H= 420:30 +CPU= 20.326
  12:05:42 STEP 1683 H= 420:45 +CPU= 21.057
  12:06:07 STEP 1684 H= 421:00 +CPU= 23.134
  12:06:32 STEP 1685 H= 421:15 +CPU= 22.899
  12:06:56 STEP 1686 H= 421:30 +CPU= 22.692
  12:07:20 STEP 1687 H= 421:45 +CPU= 22.318
  12:07:43 STEP 1688 H= 422:00 +CPU= 21.640
  12:08:06 STEP 1689 H= 422:15 +CPU= 21.381
  12:08:30 STEP 1690 H= 422:30 +CPU= 22.902
  12:08:58 STEP 1691 H= 422:45 +CPU= 24.939
  12:09:28 STEP 1692 H= 423:00 +CPU= 27.552
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001692
  12:09:50 STEP 1693 H= 423:15 +CPU= 21.084
  12:10:15 STEP 1694 H= 423:30 +CPU= 22.670
  12:10:36 STEP 1695 H= 423:45 +CPU= 20.699
  12:11:04 STEP 1696 H= 424:00 +CPU= 25.403
  12:11:32 STEP 1697 H= 424:15 +CPU= 25.193
  12:12:02 STEP 1698 H= 424:30 +CPU= 25.972
  12:12:27 STEP 1699 H= 424:45 +CPU= 23.174
  12:12:50 STEP 1700 H= 425:00 +CPU= 22.111
  12:13:14 STEP 1701 H= 425:15 +CPU= 22.239
  12:13:38 STEP 1702 H= 425:30 +CPU= 22.669
  12:14:01 STEP 1703 H= 425:45 +CPU= 21.331
  12:14:30 STEP 1704 H= 426:00 +CPU= 27.162
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001704
  12:14:53 STEP 1705 H= 426:15 +CPU= 21.939
  12:15:17 STEP 1706 H= 426:30 +CPU= 22.581
  12:15:42 STEP 1707 H= 426:45 +CPU= 22.822
  12:16:05 STEP 1708 H= 427:00 +CPU= 22.109
  12:16:29 STEP 1709 H= 427:15 +CPU= 22.085
  12:16:54 STEP 1710 H= 427:30 +CPU= 23.003
  12:17:17 STEP 1711 H= 427:45 +CPU= 21.179
  12:17:39 STEP 1712 H= 428:00 +CPU= 21.120
  12:18:02 STEP 1713 H= 428:15 +CPU= 21.482
  12:18:24 STEP 1714 H= 428:30 +CPU= 20.814
  12:18:46 STEP 1715 H= 428:45 +CPU= 20.684
  12:19:14 STEP 1716 H= 429:00 +CPU= 26.324
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001716
  12:19:37 STEP 1717 H= 429:15 +CPU= 21.458
  12:19:58 STEP 1718 H= 429:30 +CPU= 20.099
  12:20:19 STEP 1719 H= 429:45 +CPU= 19.916
  12:20:43 STEP 1720 H= 430:00 +CPU= 22.136
  12:21:04 STEP 1721 H= 430:15 +CPU= 20.702
  12:21:27 STEP 1722 H= 430:30 +CPU= 21.543
  12:21:52 STEP 1723 H= 430:45 +CPU= 22.777
  12:22:16 STEP 1724 H= 431:00 +CPU= 22.119
  12:22:40 STEP 1725 H= 431:15 +CPU= 22.408
  12:23:04 STEP 1726 H= 431:30 +CPU= 21.849
  12:23:28 STEP 1727 H= 431:45 +CPU= 22.558
  12:23:59 STEP 1728 H= 432:00 +CPU= 29.480
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0_r840727218_17.zip
Uploading the intermediate file: upload_file_17.zip
  12:24:25 STEP 1729 H= 432:15 +CPU= 23.698
  12:24:52 STEP 1730 H= 432:30 +CPU= 23.774
  12:25:17 STEP 1731 H= 432:45 +CPU= 23.158
  12:25:41 STEP 1732 H= 433:00 +CPU= 22.734
  12:26:03 STEP 1733 H= 433:15 +CPU= 20.814
  12:26:26 STEP 1734 H= 433:30 +CPU= 21.111
  12:26:48 STEP 1735 H= 433:45 +CPU= 20.640
  12:27:10 STEP 1736 H= 434:00 +CPU= 21.375
  12:27:34 STEP 1737 H= 434:15 +CPU= 21.914
  12:27:57 STEP 1738 H= 434:30 +CPU= 21.693
  12:28:19 STEP 1739 H= 434:45 +CPU= 20.731
  12:28:52 STEP 1740 H= 435:00 +CPU= 29.925
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001740
  12:29:22 STEP 1741 H= 435:15 +CPU= 26.118
  12:29:49 STEP 1742 H= 435:30 +CPU= 24.402
  12:30:14 STEP 1743 H= 435:45 +CPU= 23.559
  12:30:36 STEP 1744 H= 436:00 +CPU= 20.760
  12:30:58 STEP 1745 H= 436:15 +CPU= 20.796
  12:31:19 STEP 1746 H= 436:30 +CPU= 20.288
  12:31:40 STEP 1747 H= 436:45 +CPU= 20.181
  12:32:01 STEP 1748 H= 437:00 +CPU= 20.729
  12:32:24 STEP 1749 H= 437:15 +CPU= 21.462
  12:32:51 STEP 1750 H= 437:30 +CPU= 24.042
  12:33:15 STEP 1751 H= 437:45 +CPU= 22.310
  12:33:49 STEP 1752 H= 438:00 +CPU= 30.662
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001752
  12:34:14 STEP 1753 H= 438:15 +CPU= 23.319
  12:34:38 STEP 1754 H= 438:30 +CPU= 22.645
  12:35:01 STEP 1755 H= 438:45 +CPU= 21.688
  12:35:26 STEP 1756 H= 439:00 +CPU= 23.274
  12:35:51 STEP 1757 H= 439:15 +CPU= 22.822
  12:36:14 STEP 1758 H= 439:30 +CPU= 21.295
  12:36:37 STEP 1759 H= 439:45 +CPU= 22.095
  12:37:03 STEP 1760 H= 440:00 +CPU= 23.747
  12:37:25 STEP 1761 H= 440:15 +CPU= 20.874
  12:37:47 STEP 1762 H= 440:30 +CPU= 20.648
  12:38:08 STEP 1763 H= 440:45 +CPU= 20.199
  12:38:34 STEP 1764 H= 441:00 +CPU= 24.878
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001764
  12:38:55 STEP 1765 H= 441:15 +CPU= 20.558
  12:39:17 STEP 1766 H= 441:30 +CPU= 20.404
  12:39:38 STEP 1767 H= 441:45 +CPU= 20.372
  12:40:00 STEP 1768 H= 442:00 +CPU= 20.999
  12:40:21 STEP 1769 H= 442:15 +CPU= 20.474
  12:40:43 STEP 1770 H= 442:30 +CPU= 20.569
  12:41:04 STEP 1771 H= 442:45 +CPU= 20.472
  12:41:26 STEP 1772 H= 443:00 +CPU= 20.589
  12:41:48 STEP 1773 H= 443:15 +CPU= 21.100
  12:42:09 STEP 1774 H= 443:30 +CPU= 20.503
  12:42:31 STEP 1775 H= 443:45 +CPU= 20.391
  12:43:02 STEP 1776 H= 444:00 +CPU= 29.204
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001776
  12:43:30 STEP 1777 H= 444:15 +CPU= 25.033
  12:43:53 STEP 1778 H= 444:30 +CPU= 22.275
  12:44:15 STEP 1779 H= 444:45 +CPU= 20.983
  12:44:38 STEP 1780 H= 445:00 +CPU= 22.048
  12:45:01 STEP 1781 H= 445:15 +CPU= 21.944
  12:45:25 STEP 1782 H= 445:30 +CPU= 22.503
  12:45:53 STEP 1783 H= 445:45 +CPU= 24.720
  12:46:20 STEP 1784 H= 446:00 +CPU= 24.770
  12:46:50 STEP 1785 H= 446:15 +CPU= 26.916
  12:47:20 STEP 1786 H= 446:30 +CPU= 26.549
  12:47:48 STEP 1787 H= 446:45 +CPU= 24.890
  12:48:21 STEP 1788 H= 447:00 +CPU= 30.032
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001788
  12:48:48 STEP 1789 H= 447:15 +CPU= 24.113
  12:49:14 STEP 1790 H= 447:30 +CPU= 23.991
  12:49:41 STEP 1791 H= 447:45 +CPU= 24.355
  12:50:08 STEP 1792 H= 448:00 +CPU= 24.995
  12:50:36 STEP 1793 H= 448:15 +CPU= 25.299
  12:51:03 STEP 1794 H= 448:30 +CPU= 24.158
  12:51:30 STEP 1795 H= 448:45 +CPU= 24.854
  12:51:58 STEP 1796 H= 449:00 +CPU= 25.356
  12:52:26 STEP 1797 H= 449:15 +CPU= 24.987
  12:52:53 STEP 1798 H= 449:30 +CPU= 24.812
  12:53:21 STEP 1799 H= 449:45 +CPU= 25.149
  12:53:55 STEP 1800 H= 450:00 +CPU= 30.365
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001800
  12:54:22 STEP 1801 H= 450:15 +CPU= 24.811
  12:54:48 STEP 1802 H= 450:30 +CPU= 24.296
  12:55:12 STEP 1803 H= 450:45 +CPU= 22.201
  12:55:37 STEP 1804 H= 451:00 +CPU= 23.315
  12:56:05 STEP 1805 H= 451:15 +CPU= 25.331
  12:56:33 STEP 1806 H= 451:30 +CPU= 25.338
  12:57:02 STEP 1807 H= 451:45 +CPU= 25.408
  12:57:34 STEP 1808 H= 452:00 +CPU= 28.306
  12:58:03 STEP 1809 H= 452:15 +CPU= 26.050
  12:58:32 STEP 1810 H= 452:30 +CPU= 25.371
  12:59:00 STEP 1811 H= 452:45 +CPU= 25.220
  12:59:34 STEP 1812 H= 453:00 +CPU= 31.397
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001812
  13:00:00 STEP 1813 H= 453:15 +CPU= 23.704
  13:00:28 STEP 1814 H= 453:30 +CPU= 25.523
  13:00:55 STEP 1815 H= 453:45 +CPU= 24.971
  13:01:24 STEP 1816 H= 454:00 +CPU= 26.305
  13:01:51 STEP 1817 H= 454:15 +CPU= 24.316
  13:02:17 STEP 1818 H= 454:30 +CPU= 23.795
  13:02:45 STEP 1819 H= 454:45 +CPU= 24.856
  13:03:13 STEP 1820 H= 455:00 +CPU= 25.635
  13:03:39 STEP 1821 H= 455:15 +CPU= 23.862
  13:04:06 STEP 1822 H= 455:30 +CPU= 24.946
  13:04:33 STEP 1823 H= 455:45 +CPU= 24.540
  13:05:06 STEP 1824 H= 456:00 +CPU= 30.388
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0_r840727218_18.zip
Uploading the intermediate file: upload_file_18.zip
  13:05:29 STEP 1825 H= 456:15 +CPU= 21.778
  13:05:52 STEP 1826 H= 456:30 +CPU= 21.381
  13:06:15 STEP 1827 H= 456:45 +CPU= 21.679
  13:06:37 STEP 1828 H= 457:00 +CPU= 21.238
  13:07:00 STEP 1829 H= 457:15 +CPU= 21.255
  13:07:22 STEP 1830 H= 457:30 +CPU= 21.605
  13:07:45 STEP 1831 H= 457:45 +CPU= 21.375
  13:08:09 STEP 1832 H= 458:00 +CPU= 22.109
  13:08:31 STEP 1833 H= 458:15 +CPU= 20.705
  13:08:53 STEP 1834 H= 458:30 +CPU= 20.889
  13:09:16 STEP 1835 H= 458:45 +CPU= 21.832
  13:09:46 STEP 1836 H= 459:00 +CPU= 27.482
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001836
  13:10:08 STEP 1837 H= 459:15 +CPU= 21.463
  13:10:31 STEP 1838 H= 459:30 +CPU= 21.112
  13:10:52 STEP 1839 H= 459:45 +CPU= 20.220
  13:11:14 STEP 1840 H= 460:00 +CPU= 20.531
  13:11:35 STEP 1841 H= 460:15 +CPU= 20.270
  13:11:56 STEP 1842 H= 460:30 +CPU= 20.439
  13:12:21 STEP 1843 H= 460:45 +CPU= 23.287
  13:12:46 STEP 1844 H= 461:00 +CPU= 23.485
  13:13:09 STEP 1845 H= 461:15 +CPU= 22.159
  13:13:30 STEP 1846 H= 461:30 +CPU= 20.406
  13:13:51 STEP 1847 H= 461:45 +CPU= 20.152
  13:14:17 STEP 1848 H= 462:00 +CPU= 24.708
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001848
  13:14:39 STEP 1849 H= 462:15 +CPU= 21.032
  13:15:00 STEP 1850 H= 462:30 +CPU= 20.054
  13:15:22 STEP 1851 H= 462:45 +CPU= 21.012
  13:15:43 STEP 1852 H= 463:00 +CPU= 20.344
  13:16:04 STEP 1853 H= 463:15 +CPU= 19.889
  13:16:24 STEP 1854 H= 463:30 +CPU= 19.901
  13:16:45 STEP 1855 H= 463:45 +CPU= 20.163
  13:17:07 STEP 1856 H= 464:00 +CPU= 20.953
  13:17:31 STEP 1857 H= 464:15 +CPU= 22.473
  13:17:54 STEP 1858 H= 464:30 +CPU= 21.289
  13:18:16 STEP 1859 H= 464:45 +CPU= 20.865
  13:18:44 STEP 1860 H= 465:00 +CPU= 26.120
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001860
  13:19:06 STEP 1861 H= 465:15 +CPU= 20.880
  13:19:31 STEP 1862 H= 465:30 +CPU= 23.136
  13:19:56 STEP 1863 H= 465:45 +CPU= 23.144
  13:20:18 STEP 1864 H= 466:00 +CPU= 20.936
  13:20:38 STEP 1865 H= 466:15 +CPU= 19.908
  13:21:00 STEP 1866 H= 466:30 +CPU= 20.953
  13:21:22 STEP 1867 H= 466:45 +CPU= 20.522
  13:21:45 STEP 1868 H= 467:00 +CPU= 21.481
  13:22:05 STEP 1869 H= 467:15 +CPU= 20.089
  13:22:28 STEP 1870 H= 467:30 +CPU= 21.084
  13:22:49 STEP 1871 H= 467:45 +CPU= 20.761
  13:23:18 STEP 1872 H= 468:00 +CPU= 26.613
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001872
  13:23:38 STEP 1873 H= 468:15 +CPU= 19.855
  13:23:59 STEP 1874 H= 468:30 +CPU= 20.207
  13:24:18 STEP 1875 H= 468:45 +CPU= 18.261
  13:24:36 STEP 1876 H= 469:00 +CPU= 17.867
  13:24:58 STEP 1877 H= 469:15 +CPU= 20.996
  13:25:23 STEP 1878 H= 469:30 +CPU= 22.670
  13:25:45 STEP 1879 H= 469:45 +CPU= 21.005
  13:26:07 STEP 1880 H= 470:00 +CPU= 21.390
  13:26:30 STEP 1881 H= 470:15 +CPU= 21.302
  13:26:51 STEP 1882 H= 470:30 +CPU= 20.404
  13:27:13 STEP 1883 H= 470:45 +CPU= 21.043
  13:27:42 STEP 1884 H= 471:00 +CPU= 26.913
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001884
  13:28:04 STEP 1885 H= 471:15 +CPU= 21.409
  13:28:27 STEP 1886 H= 471:30 +CPU= 21.356
  13:28:50 STEP 1887 H= 471:45 +CPU= 21.946
  13:29:13 STEP 1888 H= 472:00 +CPU= 22.099
  13:29:36 STEP 1889 H= 472:15 +CPU= 21.147
  13:29:59 STEP 1890 H= 472:30 +CPU= 21.996
  13:30:22 STEP 1891 H= 472:45 +CPU= 21.610
  13:30:45 STEP 1892 H= 473:00 +CPU= 21.711
  13:31:06 STEP 1893 H= 473:15 +CPU= 20.315
  13:31:28 STEP 1894 H= 473:30 +CPU= 21.276
  13:31:51 STEP 1895 H= 473:45 +CPU= 21.520
  13:32:19 STEP 1896 H= 474:00 +CPU= 26.454
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001896
  13:32:43 STEP 1897 H= 474:15 +CPU= 22.429
  13:33:06 STEP 1898 H= 474:30 +CPU= 22.010
  13:33:30 STEP 1899 H= 474:45 +CPU= 22.713
  13:33:55 STEP 1900 H= 475:00 +CPU= 23.346
  13:34:18 STEP 1901 H= 475:15 +CPU= 21.800
  13:34:40 STEP 1902 H= 475:30 +CPU= 20.933
  13:35:01 STEP 1903 H= 475:45 +CPU= 20.196
  13:35:24 STEP 1904 H= 476:00 +CPU= 21.634
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0pt
(argv4) batchid: 1018
(argv5) wuid: 12289256
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_b31a22419e3392943505b86ef7842402 to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256.zip
ic_ancil_file: ic_ancil_12289256
ifsdata_file: ifsdata_12289256
climate_data_file: clim_data_12289256
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/1/ic_ancil_12289256.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12289256.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_37fd7826551b33fc95978d20c878daed to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12289256.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12289256.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12289256.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12289256.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12289256.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12289256.xml
-- Model is restarting --
Adjusting last_iter, 1905, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12289256.xml
last_cpu_time: 50223
upload_file_number: 19
last_iter: 1873
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0_r840727218
The child process has been launched with process id: 488464
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:johansson:1:1:488464:488464] [20240615:134406:1718451846:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  13:44:31 STEP 1872 H= 468:00 +CPU= 20.830
  13:44:53 STEP 1873 H= 468:15 +CPU= 21.049
  13:45:17 STEP 1874 H= 468:30 +CPU= 22.545
  13:45:39 STEP 1875 H= 468:45 +CPU= 21.045
  13:46:03 STEP 1876 H= 469:00 +CPU= 22.844
  13:46:25 STEP 1877 H= 469:15 +CPU= 20.809
  13:46:47 STEP 1878 H= 469:30 +CPU= 21.033
  13:47:08 STEP 1879 H= 469:45 +CPU= 19.949
  13:47:28 STEP 1880 H= 470:00 +CPU= 19.684
  13:47:46 STEP 1881 H= 470:15 +CPU= 17.668
  13:48:06 STEP 1882 H= 470:30 +CPU= 19.293
  13:48:27 STEP 1883 H= 470:45 +CPU= 20.621
  13:48:54 STEP 1884 H= 471:00 +CPU= 25.471
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001884
  13:49:15 STEP 1885 H= 471:15 +CPU= 20.317
  13:49:37 STEP 1886 H= 471:30 +CPU= 20.434
  13:49:59 STEP 1887 H= 471:45 +CPU= 21.236
  13:50:21 STEP 1888 H= 472:00 +CPU= 20.968
  13:50:43 STEP 1889 H= 472:15 +CPU= 20.622
  13:51:05 STEP 1890 H= 472:30 +CPU= 21.329
  13:51:27 STEP 1891 H= 472:45 +CPU= 20.313
  13:51:49 STEP 1892 H= 473:00 +CPU= 21.086
  13:52:12 STEP 1893 H= 473:15 +CPU= 21.777
  13:52:34 STEP 1894 H= 473:30 +CPU= 20.582
  13:52:56 STEP 1895 H= 473:45 +CPU= 20.783
  13:53:22 STEP 1896 H= 474:00 +CPU= 25.221
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001896
  13:53:44 STEP 1897 H= 474:15 +CPU= 20.452
  13:54:06 STEP 1898 H= 474:30 +CPU= 21.319
  13:54:31 STEP 1899 H= 474:45 +CPU= 22.824
  13:54:55 STEP 1900 H= 475:00 +CPU= 22.687
  13:55:18 STEP 1901 H= 475:15 +CPU= 21.933
  13:55:39 STEP 1902 H= 475:30 +CPU= 20.435
  13:56:00 STEP 1903 H= 475:45 +CPU= 19.519
  13:56:21 STEP 1904 H= 476:00 +CPU= 20.092
  13:56:42 STEP 1905 H= 476:15 +CPU= 20.293
  13:57:06 STEP 1906 H= 476:30 +CPU= 22.590
  13:57:37 STEP 1907 H= 476:45 +CPU= 26.701
  13:58:10 STEP 1908 H= 477:00 +CPU= 31.108
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001908
  13:58:37 STEP 1909 H= 477:15 +CPU= 25.015
  13:59:04 STEP 1910 H= 477:30 +CPU= 24.895
  13:59:32 STEP 1911 H= 477:45 +CPU= 25.649
  13:59:59 STEP 1912 H= 478:00 +CPU= 25.197
  14:00:25 STEP 1913 H= 478:15 +CPU= 25.073
  14:00:52 STEP 1914 H= 478:30 +CPU= 24.841
  14:01:21 STEP 1915 H= 478:45 +CPU= 25.765
  14:01:49 STEP 1916 H= 479:00 +CPU= 25.097
  14:02:13 STEP 1917 H= 479:15 +CPU= 22.563
  14:02:39 STEP 1918 H= 479:30 +CPU= 23.760
  14:03:03 STEP 1919 H= 479:45 +CPU= 22.804
  14:03:33 STEP 1920 H= 480:00 +CPU= 27.784
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1632763207E-20      I       0.4286636413E-06      I
ICloud fractionI      0.0000000000E+00      I       0.2813956640E-01      I
I  Relat. Hum. I      0.1170184154E+00      I       0.2092543138E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2083103399E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    295170388 BYTES IN      7 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0076( 2.6%)
 14:03:33 STEP 1920 H= 480:00 +CPU= 27.784
 END CNT3
 NSTEP =  1920 CNT0   000000000  21.115  21.115  22.420   0.942               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         1430/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     28    0       46      1044068     1430 ************     3.6     2.2     1.3     2.7     4.4
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 1167059.3 1167059.3    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 1163514.9 1163514.9     99.70      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                   53    3244.7    3244.7     14.74      0.00
   9 SPCM     - SPECTRAL COMP.                      48     763.1     763.1      3.14      0.00
  10 SCAN2M - PHYSICS                               49   12139.2   12139.2     50.97      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  4     664.6     664.6      0.23      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  3      31.8      31.8      0.01      0.00
  17 GRIDFPOS IN CNT4                                4      21.3      21.3      0.01      0.00
  27 RESTART FILES - WRITING                         1    2652.9    2652.9      0.23      0.00
  28 RESTART FILES - READING                         1    3268.9    3268.9      0.28      0.00
  29 SU4FPOS IN CNT4                                 4       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                 4    5030.7    5030.7      1.72      0.00
  31 POSDDH IN STEPO                                 1       4.3       4.3      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                     49    4617.7    4617.7     19.39      0.00
  39 SU0YOMB                                         1     179.3     179.3      0.02      0.00
  51 SCAN2M   - SL COMM. PART 1                     49      18.1      18.1      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               48     390.8     390.8      1.61      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               48      45.3      45.3      0.19      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  3       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      108     331.7     331.7      3.07      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       100     202.7     202.7      1.74      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        100      67.3      67.3      0.58      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       108     128.2     128.2      1.19      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      28.9      28.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            108      28.3      28.3      0.26      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            100      18.7      18.7      0.16      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            108      73.3      73.3      0.68      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            100      23.8      23.8      0.20      0.00
 400 GSTATS                                       6667       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                  6149       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     1167.1 CPU TIME    1098.3 VECTOR TIME     1098.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY     1485.2
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/1/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/1/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289256/ICMUAh7zg+001920
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0_r840727218_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
14:07:03 (488459): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 12:05:48 1318882 22442841 oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0 1,727,100 51,321 0.0297
15 Jun 2024 04:46:48 1318882 22442841 oifs_43r3_bl_a0pt_2016092300_20_1018_12289256_0 864,000 27,489 0.0318


©2024 climateprediction.net