climateprediction.net home page
Task 22283013

Task 22283013

Name oifs_43r3_ps_0353_2011050100_123_980_12196997_0
Workunit 12196997
Created 24 Dec 2022, 2:08:13 UTC
Sent 16 Jan 2023, 9:42:44 UTC
Report deadline 15 Feb 2023, 9:42:44 UTC
Received 17 Jan 2023, 9:11:11 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1498981
Run time 12 hours 36 min 44 sec
CPU time 12 hours 24 min 42 sec
Validate state Invalid
Credit 8,951.04
Device peak FLOPS 5.18 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.32 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,674.36 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
dding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_56.zip
Uploading the intermediate file: upload_file_56.zip
  06:18:35 STEP 1369 H=1369:00 +CPU= 16.356
  06:18:53 STEP 1370 H=1370:00 +CPU= 17.019
  06:19:18 STEP 1371 H=1371:00 +CPU= 25.663
  06:19:35 STEP 1372 H=1372:00 +CPU= 16.916
  06:19:52 STEP 1373 H=1373:00 +CPU= 16.600
  06:20:19 STEP 1374 H=1374:00 +CPU= 26.342
  06:20:35 STEP 1375 H=1375:00 +CPU= 16.634
  06:20:53 STEP 1376 H=1376:00 +CPU= 17.096
  06:21:19 STEP 1377 H=1377:00 +CPU= 25.795
  06:21:36 STEP 1378 H=1378:00 +CPU= 17.067
  06:21:52 STEP 1379 H=1379:00 +CPU= 16.596
  06:22:21 STEP 1380 H=1380:00 +CPU= 28.417
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001380
  06:22:38 STEP 1381 H=1381:00 +CPU= 16.630
  06:22:55 STEP 1382 H=1382:00 +CPU= 16.883
  06:23:21 STEP 1383 H=1383:00 +CPU= 25.940
  06:23:38 STEP 1384 H=1384:00 +CPU= 17.119
  06:23:55 STEP 1385 H=1385:00 +CPU= 16.432
  06:24:21 STEP 1386 H=1386:00 +CPU= 26.421
  06:24:38 STEP 1387 H=1387:00 +CPU= 16.479
  06:24:55 STEP 1388 H=1388:00 +CPU= 16.951
  06:25:21 STEP 1389 H=1389:00 +CPU= 25.869
  06:25:38 STEP 1390 H=1390:00 +CPU= 16.879
  06:25:55 STEP 1391 H=1391:00 +CPU= 17.121
  06:26:26 STEP 1392 H=1392:00 +CPU= 31.007
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_57.zip
Uploading the intermediate file: upload_file_57.zip
  06:26:43 STEP 1393 H=1393:00 +CPU= 16.555
  06:27:01 STEP 1394 H=1394:00 +CPU= 17.045
  06:27:26 STEP 1395 H=1395:00 +CPU= 25.708
  06:27:44 STEP 1396 H=1396:00 +CPU= 17.047
  06:28:00 STEP 1397 H=1397:00 +CPU= 16.588
  06:28:27 STEP 1398 H=1398:00 +CPU= 26.334
  06:28:43 STEP 1399 H=1399:00 +CPU= 16.624
  06:29:01 STEP 1400 H=1400:00 +CPU= 17.107
  06:29:27 STEP 1401 H=1401:00 +CPU= 25.799
  06:29:44 STEP 1402 H=1402:00 +CPU= 17.391
  06:30:01 STEP 1403 H=1403:00 +CPU= 16.927
  06:30:32 STEP 1404 H=1404:00 +CPU= 30.670
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001404
  06:30:50 STEP 1405 H=1405:00 +CPU= 18.219
  06:31:09 STEP 1406 H=1406:00 +CPU= 18.445
  06:31:37 STEP 1407 H=1407:00 +CPU= 27.692
  06:31:56 STEP 1408 H=1408:00 +CPU= 18.732
  06:32:14 STEP 1409 H=1409:00 +CPU= 18.071
  06:32:43 STEP 1410 H=1410:00 +CPU= 28.073
  06:33:01 STEP 1411 H=1411:00 +CPU= 18.241
  06:33:20 STEP 1412 H=1412:00 +CPU= 18.767
  06:33:48 STEP 1413 H=1413:00 +CPU= 27.678
  06:34:07 STEP 1414 H=1414:00 +CPU= 18.730
  06:34:25 STEP 1415 H=1415:00 +CPU= 18.719
  06:34:59 STEP 1416 H=1416:00 +CPU= 33.293
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_58.zip
Uploading the intermediate file: upload_file_58.zip
  06:35:18 STEP 1417 H=1417:00 +CPU= 18.107
  06:35:36 STEP 1418 H=1418:00 +CPU= 18.567
  06:36:04 STEP 1419 H=1419:00 +CPU= 27.721
  06:36:23 STEP 1420 H=1420:00 +CPU= 18.722
  06:36:41 STEP 1421 H=1421:00 +CPU= 18.193
  06:37:10 STEP 1422 H=1422:00 +CPU= 28.363
  06:37:28 STEP 1423 H=1423:00 +CPU= 18.244
  06:37:47 STEP 1424 H=1424:00 +CPU= 18.582
  06:38:15 STEP 1425 H=1425:00 +CPU= 27.579
  06:38:33 STEP 1426 H=1426:00 +CPU= 18.751
  06:38:52 STEP 1427 H=1427:00 +CPU= 18.256
  06:39:23 STEP 1428 H=1428:00 +CPU= 30.651
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001428
  06:39:41 STEP 1429 H=1429:00 +CPU= 18.153
  06:40:00 STEP 1430 H=1430:00 +CPU= 18.720
  06:40:27 STEP 1431 H=1431:00 +CPU= 27.589
  06:40:46 STEP 1432 H=1432:00 +CPU= 18.481
  06:41:04 STEP 1433 H=1433:00 +CPU= 18.215
  06:41:33 STEP 1434 H=1434:00 +CPU= 28.018
  06:41:51 STEP 1435 H=1435:00 +CPU= 18.223
  06:42:10 STEP 1436 H=1436:00 +CPU= 18.707
  06:42:37 STEP 1437 H=1437:00 +CPU= 27.479
  06:42:56 STEP 1438 H=1438:00 +CPU= 18.515
  06:43:15 STEP 1439 H=1439:00 +CPU= 18.580
  06:43:48 STEP 1440 H=1440:00 +CPU= 33.133
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_59.zip
Uploading the intermediate file: upload_file_59.zip
  06:44:07 STEP 1441 H=1441:00 +CPU= 18.022
  06:44:26 STEP 1442 H=1442:00 +CPU= 18.614
  06:44:53 STEP 1443 H=1443:00 +CPU= 27.560
  06:45:12 STEP 1444 H=1444:00 +CPU= 18.627
  06:45:30 STEP 1445 H=1445:00 +CPU= 18.179
  06:45:59 STEP 1446 H=1446:00 +CPU= 28.082
  06:46:17 STEP 1447 H=1447:00 +CPU= 18.209
  06:46:36 STEP 1448 H=1448:00 +CPU= 18.663
  06:47:03 STEP 1449 H=1449:00 +CPU= 27.631
  06:47:22 STEP 1450 H=1450:00 +CPU= 18.503
  06:47:40 STEP 1451 H=1451:00 +CPU= 18.162
  06:48:11 STEP 1452 H=1452:00 +CPU= 30.495
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001452
  06:48:29 STEP 1453 H=1453:00 +CPU= 18.026
  06:48:48 STEP 1454 H=1454:00 +CPU= 18.694
  06:49:16 STEP 1455 H=1455:00 +CPU= 27.727
  06:49:35 STEP 1456 H=1456:00 +CPU= 18.289
  06:49:53 STEP 1457 H=1457:00 +CPU= 18.244
  06:50:22 STEP 1458 H=1458:00 +CPU= 28.636
  06:50:40 STEP 1459 H=1459:00 +CPU= 17.624
  06:50:58 STEP 1460 H=1460:00 +CPU= 18.664
  06:51:27 STEP 1461 H=1461:00 +CPU= 28.138
  06:51:45 STEP 1462 H=1462:00 +CPU= 18.091
  06:52:04 STEP 1463 H=1463:00 +CPU= 18.673
  06:52:37 STEP 1464 H=1464:00 +CPU= 33.627
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_60.zip
Uploading the intermediate file: upload_file_60.zip
  06:52:55 STEP 1465 H=1465:00 +CPU= 17.550
  06:53:14 STEP 1466 H=1466:00 +CPU= 18.442
  06:53:42 STEP 1467 H=1467:00 +CPU= 28.158
  06:54:00 STEP 1468 H=1468:00 +CPU= 18.030
  06:54:19 STEP 1469 H=1469:00 +CPU= 18.145
  06:54:47 STEP 1470 H=1470:00 +CPU= 28.466
  06:55:05 STEP 1471 H=1471:00 +CPU= 17.500
  06:55:24 STEP 1472 H=1472:00 +CPU= 18.655
  06:55:52 STEP 1473 H=1473:00 +CPU= 28.154
  06:56:10 STEP 1474 H=1474:00 +CPU= 18.145
  06:56:28 STEP 1475 H=1475:00 +CPU= 18.207
  06:57:00 STEP 1476 H=1476:00 +CPU= 31.167
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001476
  06:57:17 STEP 1477 H=1477:00 +CPU= 17.523
  06:57:36 STEP 1478 H=1478:00 +CPU= 18.708
  06:58:05 STEP 1479 H=1479:00 +CPU= 28.146
  06:58:23 STEP 1480 H=1480:00 +CPU= 18.088
  06:58:41 STEP 1481 H=1481:00 +CPU= 18.104
  06:59:10 STEP 1482 H=1482:00 +CPU= 28.612
  06:59:28 STEP 1483 H=1483:00 +CPU= 17.912
  06:59:46 STEP 1484 H=1484:00 +CPU= 18.263
  07:00:15 STEP 1485 H=1485:00 +CPU= 27.969
  07:00:33 STEP 1486 H=1486:00 +CPU= 18.497
  07:00:52 STEP 1487 H=1487:00 +CPU= 18.272
  07:01:26 STEP 1488 H=1488:00 +CPU= 33.882
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_61.zip
Uploading the intermediate file: upload_file_61.zip
  07:01:44 STEP 1489 H=1489:00 +CPU= 17.871
  07:02:02 STEP 1490 H=1490:00 +CPU= 18.254
  07:02:31 STEP 1491 H=1491:00 +CPU= 28.121
  07:02:49 STEP 1492 H=1492:00 +CPU= 18.693
  07:03:07 STEP 1493 H=1493:00 +CPU= 17.728
  07:03:36 STEP 1494 H=1494:00 +CPU= 28.676
  07:03:54 STEP 1495 H=1495:00 +CPU= 18.243
  07:04:13 STEP 1496 H=1496:00 +CPU= 18.080
  07:04:41 STEP 1497 H=1497:00 +CPU= 28.137
  07:05:00 STEP 1498 H=1498:00 +CPU= 18.724
  07:05:17 STEP 1499 H=1499:00 +CPU= 17.581
  07:05:49 STEP 1500 H=1500:00 +CPU= 31.159
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001500
  07:06:07 STEP 1501 H=1501:00 +CPU= 18.223
  07:06:25 STEP 1502 H=1502:00 +CPU= 18.085
  07:06:53 STEP 1503 H=1503:00 +CPU= 27.834
  07:07:12 STEP 1504 H=1504:00 +CPU= 18.691
  07:07:30 STEP 1505 H=1505:00 +CPU= 17.549
  07:07:59 STEP 1506 H=1506:00 +CPU= 28.601
  07:08:17 STEP 1507 H=1507:00 +CPU= 18.191
  07:08:35 STEP 1508 H=1508:00 +CPU= 18.116
  07:09:03 STEP 1509 H=1509:00 +CPU= 27.952
  07:09:22 STEP 1510 H=1510:00 +CPU= 18.723
  07:09:41 STEP 1511 H=1511:00 +CPU= 18.540
  07:10:14 STEP 1512 H=1512:00 +CPU= 33.106
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_62.zip
Uploading the intermediate file: upload_file_62.zip
  07:10:33 STEP 1513 H=1513:00 +CPU= 18.174
  07:10:51 STEP 1514 H=1514:00 +CPU= 18.068
  07:11:19 STEP 1515 H=1515:00 +CPU= 27.907
  07:11:38 STEP 1516 H=1516:00 +CPU= 18.748
  07:11:56 STEP 1517 H=1517:00 +CPU= 17.691
  07:12:24 STEP 1518 H=1518:00 +CPU= 28.322
  07:12:43 STEP 1519 H=1519:00 +CPU= 18.215
  07:13:01 STEP 1520 H=1520:00 +CPU= 18.520
  07:13:29 STEP 1521 H=1521:00 +CPU= 27.643
  07:13:48 STEP 1522 H=1522:00 +CPU= 18.636
  07:14:07 STEP 1523 H=1523:00 +CPU= 18.138
  07:14:38 STEP 1524 H=1524:00 +CPU= 30.520
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001524
  07:14:56 STEP 1525 H=1525:00 +CPU= 18.241
  07:15:15 STEP 1526 H=1526:00 +CPU= 18.603
  07:15:42 STEP 1527 H=1527:00 +CPU= 27.652
  07:16:01 STEP 1528 H=1528:00 +CPU= 18.627
  07:16:20 STEP 1529 H=1529:00 +CPU= 18.238
  07:16:48 STEP 1530 H=1530:00 +CPU= 28.138
  07:17:06 STEP 1531 H=1531:00 +CPU= 18.243
  07:17:25 STEP 1532 H=1532:00 +CPU= 18.626
  07:17:53 STEP 1533 H=1533:00 +CPU= 27.509
  07:18:12 STEP 1534 H=1534:00 +CPU= 18.660
  07:18:30 STEP 1535 H=1535:00 +CPU= 18.614
  07:19:04 STEP 1536 H=1536:00 +CPU= 33.310
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_63.zip
Uploading the intermediate file: upload_file_63.zip
  07:19:22 STEP 1537 H=1537:00 +CPU= 18.008
Uploading trickle at timestep: 5529600
  07:19:41 STEP 1538 H=1538:00 +CPU= 18.652
  07:20:09 STEP 1539 H=1539:00 +CPU= 27.627
  07:20:27 STEP 1540 H=1540:00 +CPU= 18.639
  07:20:46 STEP 1541 H=1541:00 +CPU= 18.123
  07:21:14 STEP 1542 H=1542:00 +CPU= 28.250
  07:21:33 STEP 1543 H=1543:00 +CPU= 18.416
  07:21:51 STEP 1544 H=1544:00 +CPU= 18.826
  07:22:19 STEP 1545 H=1545:00 +CPU= 27.408
  07:22:38 STEP 1546 H=1546:00 +CPU= 18.676
  07:22:56 STEP 1547 H=1547:00 +CPU= 18.192
  07:23:27 STEP 1548 H=1548:00 +CPU= 30.369
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001548
  07:23:45 STEP 1549 H=1549:00 +CPU= 18.177
  07:24:04 STEP 1550 H=1550:00 +CPU= 18.730
  07:24:32 STEP 1551 H=1551:00 +CPU= 27.573
  07:24:50 STEP 1552 H=1552:00 +CPU= 18.556
  07:25:09 STEP 1553 H=1553:00 +CPU= 18.202
  07:25:37 STEP 1554 H=1554:00 +CPU= 28.059
  07:25:55 STEP 1555 H=1555:00 +CPU= 17.966
  07:26:14 STEP 1556 H=1556:00 +CPU= 18.626
  07:26:42 STEP 1557 H=1557:00 +CPU= 27.802
  07:27:00 STEP 1558 H=1558:00 +CPU= 18.257
  07:27:19 STEP 1559 H=1559:00 +CPU= 18.720
  07:27:52 STEP 1560 H=1560:00 +CPU= 33.023
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_64.zip
Uploading the intermediate file: upload_file_64.zip
  07:28:10 STEP 1561 H=1561:00 +CPU= 17.940
  07:28:29 STEP 1562 H=1562:00 +CPU= 18.670
  07:28:57 STEP 1563 H=1563:00 +CPU= 27.719
  07:29:16 STEP 1564 H=1564:00 +CPU= 18.418
  07:29:34 STEP 1565 H=1565:00 +CPU= 18.195
  07:30:02 STEP 1566 H=1566:00 +CPU= 28.376
  07:30:20 STEP 1567 H=1567:00 +CPU= 17.737
  07:30:39 STEP 1568 H=1568:00 +CPU= 18.493
  07:31:07 STEP 1569 H=1569:00 +CPU= 27.860
  07:31:25 STEP 1570 H=1570:00 +CPU= 18.131
  07:31:43 STEP 1571 H=1571:00 +CPU= 18.055
  07:32:14 STEP 1572 H=1572:00 +CPU= 30.850
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001572
  07:32:32 STEP 1573 H=1573:00 +CPU= 17.800
  07:32:51 STEP 1574 H=1574:00 +CPU= 18.733
  07:33:20 STEP 1575 H=1575:00 +CPU= 28.099
  07:33:38 STEP 1576 H=1576:00 +CPU= 18.122
  07:33:56 STEP 1577 H=1577:00 +CPU= 18.094
  07:34:25 STEP 1578 H=1578:00 +CPU= 28.637
  07:34:43 STEP 1579 H=1579:00 +CPU= 17.570
  07:35:01 STEP 1580 H=1580:00 +CPU= 18.658
  07:35:30 STEP 1581 H=1581:00 +CPU= 28.120
  07:35:48 STEP 1582 H=1582:00 +CPU= 18.049
  07:36:07 STEP 1583 H=1583:00 +CPU= 18.682
  07:36:41 STEP 1584 H=1584:00 +CPU= 33.824
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_65.zip
Uploading the intermediate file: upload_file_65.zip
  07:36:59 STEP 1585 H=1585:00 +CPU= 17.457
  07:37:18 STEP 1586 H=1586:00 +CPU= 18.596
  07:37:46 STEP 1587 H=1587:00 +CPU= 28.011
  07:38:04 STEP 1588 H=1588:00 +CPU= 18.004
  07:38:22 STEP 1589 H=1589:00 +CPU= 18.085
  07:38:52 STEP 1590 H=1590:00 +CPU= 28.586
  07:39:09 STEP 1591 H=1591:00 +CPU= 17.476
  07:39:28 STEP 1592 H=1592:00 +CPU= 18.679
  07:39:56 STEP 1593 H=1593:00 +CPU= 28.065
  07:40:15 STEP 1594 H=1594:00 +CPU= 18.012
  07:40:33 STEP 1595 H=1595:00 +CPU= 18.179
  07:41:04 STEP 1596 H=1596:00 +CPU= 31.037
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001596
  07:41:22 STEP 1597 H=1597:00 +CPU= 17.548
  07:41:40 STEP 1598 H=1598:00 +CPU= 18.529
  07:42:09 STEP 1599 H=1599:00 +CPU= 28.079
  07:42:27 STEP 1600 H=1600:00 +CPU= 18.234
  07:42:45 STEP 1601 H=1601:00 +CPU= 17.943
  07:43:14 STEP 1602 H=1602:00 +CPU= 28.587
  07:43:32 STEP 1603 H=1603:00 +CPU= 17.820
  07:43:51 STEP 1604 H=1604:00 +CPU= 18.215
  07:44:19 STEP 1605 H=1605:00 +CPU= 27.908
  07:44:37 STEP 1606 H=1606:00 +CPU= 18.104
  07:44:55 STEP 1607 H=1607:00 +CPU= 18.306
  07:45:29 STEP 1608 H=1608:00 +CPU= 33.631
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_66.zip
Uploading the intermediate file: upload_file_66.zip
  07:45:47 STEP 1609 H=1609:00 +CPU= 17.569
  07:46:06 STEP 1610 H=1610:00 +CPU= 18.428
  07:46:34 STEP 1611 H=1611:00 +CPU= 27.920
  07:46:52 STEP 1612 H=1612:00 +CPU= 18.208
  07:47:10 STEP 1613 H=1613:00 +CPU= 17.849
  07:47:39 STEP 1614 H=1614:00 +CPU= 28.630
  07:47:57 STEP 1615 H=1615:00 +CPU= 17.636
  07:48:15 STEP 1616 H=1616:00 +CPU= 18.218
  07:48:43 STEP 1617 H=1617:00 +CPU= 28.020
  07:49:01 STEP 1618 H=1618:00 +CPU= 18.211
  07:49:19 STEP 1619 H=1619:00 +CPU= 17.808
  07:49:50 STEP 1620 H=1620:00 +CPU= 30.935
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001620
  07:50:08 STEP 1621 H=1621:00 +CPU= 17.814
  07:50:27 STEP 1622 H=1622:00 +CPU= 18.353
  07:50:55 STEP 1623 H=1623:00 +CPU= 28.016
  07:51:13 STEP 1624 H=1624:00 +CPU= 18.297
  07:51:31 STEP 1625 H=1625:00 +CPU= 17.643
  07:52:00 STEP 1626 H=1626:00 +CPU= 28.534
  07:52:18 STEP 1627 H=1627:00 +CPU= 18.135
  07:52:36 STEP 1628 H=1628:00 +CPU= 17.959
  07:53:04 STEP 1629 H=1629:00 +CPU= 28.025
  07:53:23 STEP 1630 H=1630:00 +CPU= 18.544
  07:53:41 STEP 1631 H=1631:00 +CPU= 17.929
  07:54:15 STEP 1632 H=1632:00 +CPU= 33.504
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_67.zip
Uploading the intermediate file: upload_file_67.zip
  07:54:33 STEP 1633 H=1633:00 +CPU= 17.948
  07:54:51 STEP 1634 H=1634:00 +CPU= 18.050
  07:55:19 STEP 1635 H=1635:00 +CPU= 27.939
  07:55:38 STEP 1636 H=1636:00 +CPU= 18.650
  07:55:55 STEP 1637 H=1637:00 +CPU= 17.438
  07:56:24 STEP 1638 H=1638:00 +CPU= 28.508
  07:56:42 STEP 1639 H=1639:00 +CPU= 18.132
  07:57:01 STEP 1640 H=1640:00 +CPU= 18.055
  07:57:28 STEP 1641 H=1641:00 +CPU= 27.809
  07:57:47 STEP 1642 H=1642:00 +CPU= 18.542
  07:58:05 STEP 1643 H=1643:00 +CPU= 17.793
  07:58:36 STEP 1644 H=1644:00 +CPU= 30.663
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001644
  07:58:54 STEP 1645 H=1645:00 +CPU= 18.070
  07:59:13 STEP 1646 H=1646:00 +CPU= 18.443
  07:59:41 STEP 1647 H=1647:00 +CPU= 28.093
  08:00:00 STEP 1648 H=1648:00 +CPU= 18.537
  08:00:18 STEP 1649 H=1649:00 +CPU= 17.878
  08:00:47 STEP 1650 H=1650:00 +CPU= 28.436
  08:01:07 STEP 1651 H=1651:00 +CPU= 18.759
  08:01:27 STEP 1652 H=1652:00 +CPU= 18.785
  08:01:56 STEP 1653 H=1653:00 +CPU= 28.254
  08:02:16 STEP 1654 H=1654:00 +CPU= 19.059
  08:02:35 STEP 1655 H=1655:00 +CPU= 18.664
  08:03:10 STEP 1656 H=1656:00 +CPU= 34.192
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_68.zip
Uploading the intermediate file: upload_file_68.zip
  08:03:29 STEP 1657 H=1657:00 +CPU= 18.399
  08:03:49 STEP 1658 H=1658:00 +CPU= 18.718
  08:04:18 STEP 1659 H=1659:00 +CPU= 28.569
  08:04:37 STEP 1660 H=1660:00 +CPU= 18.760
  08:04:55 STEP 1661 H=1661:00 +CPU= 17.950
  08:05:25 STEP 1662 H=1662:00 +CPU= 28.994
  08:05:43 STEP 1663 H=1663:00 +CPU= 18.325
  08:06:02 STEP 1664 H=1664:00 +CPU= 18.471
  08:06:31 STEP 1665 H=1665:00 +CPU= 28.592
  08:06:50 STEP 1666 H=1666:00 +CPU= 18.793
  08:07:08 STEP 1667 H=1667:00 +CPU= 17.896
  08:07:40 STEP 1668 H=1668:00 +CPU= 31.600
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001668
  08:07:58 STEP 1669 H=1669:00 +CPU= 18.432
  08:08:17 STEP 1670 H=1670:00 +CPU= 18.266
  08:08:46 STEP 1671 H=1671:00 +CPU= 28.473
  08:09:05 STEP 1672 H=1672:00 +CPU= 18.995
  08:09:23 STEP 1673 H=1673:00 +CPU= 17.833
  08:09:52 STEP 1674 H=1674:00 +CPU= 28.802
  08:10:11 STEP 1675 H=1675:00 +CPU= 18.403
  08:10:30 STEP 1676 H=1676:00 +CPU= 18.362
  08:10:59 STEP 1677 H=1677:00 +CPU= 28.591
  08:11:18 STEP 1678 H=1678:00 +CPU= 18.911
  08:11:37 STEP 1679 H=1679:00 +CPU= 18.490
  08:12:11 STEP 1680 H=1680:00 +CPU= 34.096
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001668
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_69.zip
Uploading the intermediate file: upload_file_69.zip
  08:12:30 STEP 1681 H=1681:00 +CPU= 18.245
  08:12:48 STEP 1682 H=1682:00 +CPU= 18.384
  08:13:17 STEP 1683 H=1683:00 +CPU= 28.422
  08:13:36 STEP 1684 H=1684:00 +CPU= 18.975
  08:13:54 STEP 1685 H=1685:00 +CPU= 17.870
  08:14:23 STEP 1686 H=1686:00 +CPU= 28.945
  08:14:42 STEP 1687 H=1687:00 +CPU= 18.474
  08:15:01 STEP 1688 H=1688:00 +CPU= 18.464
  08:15:29 STEP 1689 H=1689:00 +CPU= 28.105
  08:15:49 STEP 1690 H=1690:00 +CPU= 18.939
  08:16:07 STEP 1691 H=1691:00 +CPU= 18.176
  08:16:39 STEP 1692 H=1692:00 +CPU= 31.334
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001692
  08:16:58 STEP 1693 H=1693:00 +CPU= 18.575
  08:17:16 STEP 1694 H=1694:00 +CPU= 18.480
  08:17:45 STEP 1695 H=1695:00 +CPU= 28.176
  08:18:04 STEP 1696 H=1696:00 +CPU= 18.826
  08:18:22 STEP 1697 H=1697:00 +CPU= 18.082
  08:18:51 STEP 1698 H=1698:00 +CPU= 28.827
  08:19:10 STEP 1699 H=1699:00 +CPU= 18.498
  08:19:29 STEP 1700 H=1700:00 +CPU= 18.593
  08:19:57 STEP 1701 H=1701:00 +CPU= 28.148
  08:20:17 STEP 1702 H=1702:00 +CPU= 18.829
  08:20:36 STEP 1703 H=1703:00 +CPU= 18.580
  08:21:10 STEP 1704 H=1704:00 +CPU= 33.790
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001692
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_70.zip
Uploading the intermediate file: upload_file_70.zip
  08:21:28 STEP 1705 H=1705:00 +CPU= 18.067
  08:21:47 STEP 1706 H=1706:00 +CPU= 18.413
  08:22:16 STEP 1707 H=1707:00 +CPU= 28.338
  08:22:35 STEP 1708 H=1708:00 +CPU= 18.834
  08:22:53 STEP 1709 H=1709:00 +CPU= 17.958
  08:23:22 STEP 1710 H=1710:00 +CPU= 28.766
  08:23:40 STEP 1711 H=1711:00 +CPU= 18.155
  08:23:59 STEP 1712 H=1712:00 +CPU= 18.605
  08:24:28 STEP 1713 H=1713:00 +CPU= 28.397
  08:24:47 STEP 1714 H=1714:00 +CPU= 18.611
  08:25:05 STEP 1715 H=1715:00 +CPU= 18.004
  08:25:37 STEP 1716 H=1716:00 +CPU= 31.355
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001716
  08:25:55 STEP 1717 H=1717:00 +CPU= 18.018
  08:26:14 STEP 1718 H=1718:00 +CPU= 18.650
  08:26:43 STEP 1719 H=1719:00 +CPU= 28.369
  08:27:01 STEP 1720 H=1720:00 +CPU= 18.643
  08:27:20 STEP 1721 H=1721:00 +CPU= 18.045
  08:27:49 STEP 1722 H=1722:00 +CPU= 28.641
  08:28:07 STEP 1723 H=1723:00 +CPU= 18.052
  08:28:26 STEP 1724 H=1724:00 +CPU= 18.663
  08:28:55 STEP 1725 H=1725:00 +CPU= 28.270
  08:29:14 STEP 1726 H=1726:00 +CPU= 18.718
  08:29:33 STEP 1727 H=1727:00 +CPU= 18.839
  08:30:07 STEP 1728 H=1728:00 +CPU= 34.030
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_71.zip
Uploading the intermediate file: upload_file_71.zip
  08:30:26 STEP 1729 H=1729:00 +CPU= 17.978
  08:30:44 STEP 1730 H=1730:00 +CPU= 18.492
  08:31:13 STEP 1731 H=1731:00 +CPU= 28.435
  08:31:32 STEP 1732 H=1732:00 +CPU= 18.644
  08:31:51 STEP 1733 H=1733:00 +CPU= 18.052
  08:32:20 STEP 1734 H=1734:00 +CPU= 28.940
  08:32:38 STEP 1735 H=1735:00 +CPU= 18.124
  08:32:57 STEP 1736 H=1736:00 +CPU= 18.607
  08:33:26 STEP 1737 H=1737:00 +CPU= 28.322
  08:33:44 STEP 1738 H=1738:00 +CPU= 18.731
  08:34:02 STEP 1739 H=1739:00 +CPU= 18.042
  08:34:34 STEP 1740 H=1740:00 +CPU= 31.417
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001740
  08:34:53 STEP 1741 H=1741:00 +CPU= 18.153
  08:35:12 STEP 1742 H=1742:00 +CPU= 18.734
  08:35:41 STEP 1743 H=1743:00 +CPU= 28.454
  08:35:59 STEP 1744 H=1744:00 +CPU= 18.632
  08:36:18 STEP 1745 H=1745:00 +CPU= 18.117
  08:36:47 STEP 1746 H=1746:00 +CPU= 28.994
  08:37:06 STEP 1747 H=1747:00 +CPU= 18.204
  08:37:24 STEP 1748 H=1748:00 +CPU= 18.513
  08:37:53 STEP 1749 H=1749:00 +CPU= 28.461
  08:38:12 STEP 1750 H=1750:00 +CPU= 18.579
  08:38:31 STEP 1751 H=1751:00 +CPU= 18.677
  08:39:06 STEP 1752 H=1752:00 +CPU= 34.104
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001740
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_72.zip
Uploading the intermediate file: upload_file_72.zip
  08:39:25 STEP 1753 H=1753:00 +CPU= 18.123
  08:39:44 STEP 1754 H=1754:00 +CPU= 18.549
  08:40:12 STEP 1755 H=1755:00 +CPU= 28.279
  08:40:30 STEP 1756 H=1756:00 +CPU= 18.492
  08:40:49 STEP 1757 H=1757:00 +CPU= 18.082
  08:41:18 STEP 1758 H=1758:00 +CPU= 28.900
  08:41:36 STEP 1759 H=1759:00 +CPU= 18.114
  08:41:55 STEP 1760 H=1760:00 +CPU= 18.636
  08:42:23 STEP 1761 H=1761:00 +CPU= 28.451
  08:42:42 STEP 1762 H=1762:00 +CPU= 18.640
  08:43:01 STEP 1763 H=1763:00 +CPU= 18.227
  08:43:32 STEP 1764 H=1764:00 +CPU= 31.310
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001764
  08:43:50 STEP 1765 H=1765:00 +CPU= 18.086
  08:44:09 STEP 1766 H=1766:00 +CPU= 18.648
  08:44:38 STEP 1767 H=1767:00 +CPU= 28.258
  08:44:57 STEP 1768 H=1768:00 +CPU= 18.643
  08:45:15 STEP 1769 H=1769:00 +CPU= 18.129
  08:45:45 STEP 1770 H=1770:00 +CPU= 28.805
  08:46:03 STEP 1771 H=1771:00 +CPU= 18.194
  08:46:22 STEP 1772 H=1772:00 +CPU= 18.623
  08:46:50 STEP 1773 H=1773:00 +CPU= 28.314
  08:47:09 STEP 1774 H=1774:00 +CPU= 18.661
  08:47:28 STEP 1775 H=1775:00 +CPU= 18.691
  08:48:02 STEP 1776 H=1776:00 +CPU= 33.825
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001764
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_73.zip
Uploading the intermediate file: upload_file_73.zip
  08:48:21 STEP 1777 H=1777:00 +CPU= 18.291
Uploading trickle at timestep: 6393600
  08:48:39 STEP 1778 H=1778:00 +CPU= 18.554
  08:49:08 STEP 1779 H=1779:00 +CPU= 28.169
  08:49:27 STEP 1780 H=1780:00 +CPU= 18.952
  08:49:45 STEP 1781 H=1781:00 +CPU= 18.203
  08:50:14 STEP 1782 H=1782:00 +CPU= 28.714
  08:50:33 STEP 1783 H=1783:00 +CPU= 18.389
  08:50:52 STEP 1784 H=1784:00 +CPU= 18.669
  08:51:20 STEP 1785 H=1785:00 +CPU= 28.146
  08:51:39 STEP 1786 H=1786:00 +CPU= 18.690
  08:51:57 STEP 1787 H=1787:00 +CPU= 18.200
  08:52:29 STEP 1788 H=1788:00 +CPU= 31.240
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001788
  08:52:47 STEP 1789 H=1789:00 +CPU= 18.290
  08:53:06 STEP 1790 H=1790:00 +CPU= 18.621
  08:53:34 STEP 1791 H=1791:00 +CPU= 28.176
  08:53:54 STEP 1792 H=1792:00 +CPU= 18.784
  08:54:12 STEP 1793 H=1793:00 +CPU= 18.170
  08:54:41 STEP 1794 H=1794:00 +CPU= 28.478
  08:54:59 STEP 1795 H=1795:00 +CPU= 18.443
  08:55:19 STEP 1796 H=1796:00 +CPU= 18.607
  08:55:47 STEP 1797 H=1797:00 +CPU= 28.360
  08:56:06 STEP 1798 H=1798:00 +CPU= 18.706
  08:56:25 STEP 1799 H=1799:00 +CPU= 18.740
  08:57:00 STEP 1800 H=1800:00 +CPU= 34.131
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001788
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_74.zip
Uploading the intermediate file: upload_file_74.zip
  08:57:19 STEP 1801 H=1801:00 +CPU= 18.405
  08:57:38 STEP 1802 H=1802:00 +CPU= 18.661
  08:58:07 STEP 1803 H=1803:00 +CPU= 28.431
  08:58:25 STEP 1804 H=1804:00 +CPU= 18.436
  08:58:43 STEP 1805 H=1805:00 +CPU= 17.831
  08:59:11 STEP 1806 H=1806:00 +CPU= 28.022
  08:59:29 STEP 1807 H=1807:00 +CPU= 17.955
  08:59:48 STEP 1808 H=1808:00 +CPU= 18.181
  09:00:15 STEP 1809 H=1809:00 +CPU= 27.642
  09:00:34 STEP 1810 H=1810:00 +CPU= 18.648
  09:00:52 STEP 1811 H=1811:00 +CPU= 17.838
  09:01:23 STEP 1812 H=1812:00 +CPU= 30.608
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001812
  09:01:41 STEP 1813 H=1813:00 +CPU= 17.910
  09:01:59 STEP 1814 H=1814:00 +CPU= 18.215
  09:02:27 STEP 1815 H=1815:00 +CPU= 27.617
  09:02:45 STEP 1816 H=1816:00 +CPU= 18.586
  09:03:03 STEP 1817 H=1817:00 +CPU= 17.738
  09:03:31 STEP 1818 H=1818:00 +CPU= 28.154
  09:03:50 STEP 1819 H=1819:00 +CPU= 18.168
  09:04:08 STEP 1820 H=1820:00 +CPU= 18.311
  09:04:36 STEP 1821 H=1821:00 +CPU= 27.674
  09:04:55 STEP 1822 H=1822:00 +CPU= 18.547
  09:05:13 STEP 1823 H=1823:00 +CPU= 18.311
  09:05:47 STEP 1824 H=1824:00 +CPU= 33.383
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_75.zip
Uploading the intermediate file: upload_file_75.zip
  09:06:05 STEP 1825 H=1825:00 +CPU= 17.998
  09:06:25 STEP 1826 H=1826:00 +CPU= 18.631
  09:06:54 STEP 1827 H=1827:00 +CPU= 28.123
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.066] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3547MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.066] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.066] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.066] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.066] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3547 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            CPG_DRV 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-CPG                  (1025) 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090656:1673942816:11605.067] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              CPG 
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090657:1673942817:11606.067] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:pc213-04:1:1:172834:172834] [20230117:090657:1673942817:11606.067] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DB4580  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000C2E82A  cpg_                      668  cpg.F90
oifs_43r3_model.e  0000000000C2C638  cpg_drv_                  351  cpg_drv.F90
oifs_43r3_model.e  0000000000BF17C8  gp_model_                 419  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2011050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0353
(argv4) batchid: 980
(argv5) wuid: 12196997
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5671b15ad3504dd2c9874ada285ddde0 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0353_2011050100_123_980_12196997.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0353_2011050100_123_980_12196997.zip
ic_ancil_file: ic_ancil_12196997
ifsdata_file: ifsdata_12196997
climate_data_file: clim_data_12196997
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0fd5deba2a5dcb482fe63981f57eef03 to: /var/lib/boinc-client/slots/4/ic_ancil_12196997.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12196997.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c57407c0614e32421700f0c7b69d327d to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12196997.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12196997.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12196997.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12196997.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12196997.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12196997.xml
-- Model is restarting --
Adjusting last_iter, 1827, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12196997.xml
last_cpu_time: 44106
upload_file_number: 76
last_iter: 1823
last_upload: 6566400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519
The child process has been launched with process id: 2137
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:pc213-04:1:1:2137:2137] [20230117:090925:1673942965:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:10:08 STEP 1824 H=1824:00 +CPU= 29.736
  09:10:26 STEP 1825 H=1825:00 +CPU= 18.108
  09:10:44 STEP 1826 H=1826:00 +CPU= 18.529
  09:11:12 STEP 1827 H=1827:00 +CPU= 27.356
  09:11:30 STEP 1828 H=1828:00 +CPU= 18.621
  09:11:49 STEP 1829 H=1829:00 +CPU= 18.206
  09:12:17 STEP 1830 H=1830:00 +CPU= 28.088
  09:12:35 STEP 1831 H=1831:00 +CPU= 17.870
  09:12:53 STEP 1832 H=1832:00 +CPU= 18.615
  09:13:21 STEP 1833 H=1833:00 +CPU= 27.874
  09:13:39 STEP 1834 H=1834:00 +CPU= 18.139
  09:13:58 STEP 1835 H=1835:00 +CPU= 18.107
  09:14:29 STEP 1836 H=1836:00 +CPU= 30.713
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001836
  09:14:46 STEP 1837 H=1837:00 +CPU= 17.481
  09:15:05 STEP 1838 H=1838:00 +CPU= 18.454
  09:15:32 STEP 1839 H=1839:00 +CPU= 27.659
  09:15:50 STEP 1840 H=1840:00 +CPU= 17.839
  09:16:08 STEP 1841 H=1841:00 +CPU= 17.944
  09:16:36 STEP 1842 H=1842:00 +CPU= 28.259
  09:16:54 STEP 1843 H=1843:00 +CPU= 17.336
  09:17:12 STEP 1844 H=1844:00 +CPU= 18.207
  09:17:40 STEP 1845 H=1845:00 +CPU= 27.656
  09:18:00 STEP 1846 H=1846:00 +CPU= 18.248
  09:18:21 STEP 1847 H=1847:00 +CPU= 18.577
  09:18:55 STEP 1848 H=1848:00 +CPU= 33.676
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12196997/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0353_2011050100_123_980_12196997_0_r613871519_76.zip
Uploading the intermediate file: upload_file_76.zip
  09:19:15 STEP 1849 H=1849:00 +CPU= 17.940
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:23:48 1498981 22283013 oifs_43r3_ps_0353_2011050100_123_980_12196997_0 6,393,600 43,001 0.0067
09 May 2023 16:23:40 1498981 22283013 oifs_43r3_ps_0353_2011050100_123_980_12196997_0 5,529,600 37,720 0.0068
09 May 2023 16:23:10 1498981 22283013 oifs_43r3_ps_0353_2011050100_123_980_12196997_0 2,764,800 19,905 0.0072


©2024 cpdn.org