climateprediction.net home page
Task 22446701

Task 22446701

Name oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0
Workunit 12293089
Created 13 Jun 2024, 2:54:54 UTC
Sent 13 Jun 2024, 20:08:00 UTC
Report deadline 12 Aug 2024, 20:08:00 UTC
Received 16 Jun 2024, 6:56:19 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1532599
Run time 15 hours 34 min 20 sec
CPU time 15 hours 25 min 12 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 3.11 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,018.58 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,387.11 MB
Stderr
<core_client_version>8.1.0</core_client_version>
<![CDATA[
<stderr_txt>
/oifs_43r3_bl_12293089/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0_r1385854505_14.zip
Uploading the intermediate file: upload_file_14.zip
  18:32:13 STEP 1441 H= 360:15 +CPU= 29.669
  18:33:06 STEP 1442 H= 360:30 +CPU= 29.803
  18:33:56 STEP 1443 H= 360:45 +CPU= 29.493
  18:34:49 STEP 1444 H= 361:00 +CPU= 30.294
  18:35:42 STEP 1445 H= 361:15 +CPU= 30.138
  18:36:32 STEP 1446 H= 361:30 +CPU= 30.121
  18:37:21 STEP 1447 H= 361:45 +CPU= 29.904
  18:38:14 STEP 1448 H= 362:00 +CPU= 30.898
  18:39:07 STEP 1449 H= 362:15 +CPU= 30.475
  18:39:59 STEP 1450 H= 362:30 +CPU= 30.328
  18:40:49 STEP 1451 H= 362:45 +CPU= 30.149
  18:41:53 STEP 1452 H= 363:00 +CPU= 37.451
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001452
  18:42:44 STEP 1453 H= 363:15 +CPU= 30.237
  18:43:35 STEP 1454 H= 363:30 +CPU= 29.791
  18:44:23 STEP 1455 H= 363:45 +CPU= 29.803
  18:45:14 STEP 1456 H= 364:00 +CPU= 30.688
  18:46:01 STEP 1457 H= 364:15 +CPU= 29.949
  18:46:50 STEP 1458 H= 364:30 +CPU= 30.958
  18:47:34 STEP 1459 H= 364:45 +CPU= 30.690
  18:48:20 STEP 1460 H= 365:00 +CPU= 31.530
  18:49:06 STEP 1461 H= 365:15 +CPU= 30.998
  18:49:52 STEP 1462 H= 365:30 +CPU= 30.434
  18:50:38 STEP 1463 H= 365:45 +CPU= 30.937
  18:51:35 STEP 1464 H= 366:00 +CPU= 38.052
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001464
  18:52:22 STEP 1465 H= 366:15 +CPU= 30.760
  18:53:10 STEP 1466 H= 366:30 +CPU= 31.179
  18:53:54 STEP 1467 H= 366:45 +CPU= 30.702
  18:54:41 STEP 1468 H= 367:00 +CPU= 31.859
  18:55:27 STEP 1469 H= 367:15 +CPU= 30.853
  18:56:13 STEP 1470 H= 367:30 +CPU= 30.323
  18:57:00 STEP 1471 H= 367:45 +CPU= 30.344
  18:57:49 STEP 1472 H= 368:00 +CPU= 31.508
  18:58:36 STEP 1473 H= 368:15 +CPU= 30.490
  18:59:22 STEP 1474 H= 368:30 +CPU= 30.796
  19:00:09 STEP 1475 H= 368:45 +CPU= 30.228
  19:01:05 STEP 1476 H= 369:00 +CPU= 37.137
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001476
  19:01:55 STEP 1477 H= 369:15 +CPU= 30.030
  19:02:42 STEP 1478 H= 369:30 +CPU= 29.664
  19:03:30 STEP 1479 H= 369:45 +CPU= 29.266
  19:04:27 STEP 1480 H= 370:00 +CPU= 29.262
  19:05:20 STEP 1481 H= 370:15 +CPU= 28.980
  19:06:07 STEP 1482 H= 370:30 +CPU= 29.023
  19:06:54 STEP 1483 H= 370:45 +CPU= 29.064
  19:07:43 STEP 1484 H= 371:00 +CPU= 29.237
  19:08:38 STEP 1485 H= 371:15 +CPU= 28.870
  19:09:34 STEP 1486 H= 371:30 +CPU= 28.577
  19:10:20 STEP 1487 H= 371:45 +CPU= 28.797
  19:11:19 STEP 1488 H= 372:00 +CPU= 42.848
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001488
  19:11:57 STEP 1489 H= 372:15 +CPU= 32.775
  19:12:24 STEP 1490 H= 372:30 +CPU= 27.036
  19:12:55 STEP 1491 H= 372:45 +CPU= 30.044
  19:13:24 STEP 1492 H= 373:00 +CPU= 29.502
  19:13:53 STEP 1493 H= 373:15 +CPU= 29.192
  19:14:23 STEP 1494 H= 373:30 +CPU= 29.287
  19:14:54 STEP 1495 H= 373:45 +CPU= 30.684
  19:15:23 STEP 1496 H= 374:00 +CPU= 29.017
  19:15:53 STEP 1497 H= 374:15 +CPU= 30.005
  19:16:22 STEP 1498 H= 374:30 +CPU= 29.337
  19:16:53 STEP 1499 H= 374:45 +CPU= 30.334
  19:17:29 STEP 1500 H= 375:00 +CPU= 35.956
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001500
  19:18:00 STEP 1501 H= 375:15 +CPU= 30.304
  19:18:28 STEP 1502 H= 375:30 +CPU= 28.596
  19:18:58 STEP 1503 H= 375:45 +CPU= 29.112
  19:19:29 STEP 1504 H= 376:00 +CPU= 30.883
  19:19:59 STEP 1505 H= 376:15 +CPU= 30.100
  19:20:29 STEP 1506 H= 376:30 +CPU= 30.142
  19:21:00 STEP 1507 H= 376:45 +CPU= 30.337
  19:21:31 STEP 1508 H= 377:00 +CPU= 30.714
  19:22:01 STEP 1509 H= 377:15 +CPU= 30.628
  19:22:32 STEP 1510 H= 377:30 +CPU= 30.409
  19:23:04 STEP 1511 H= 377:45 +CPU= 31.635
  19:23:42 STEP 1512 H= 378:00 +CPU= 37.515
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001512
  19:24:13 STEP 1513 H= 378:15 +CPU= 30.789
  19:24:42 STEP 1514 H= 378:30 +CPU= 29.757
  19:25:14 STEP 1515 H= 378:45 +CPU= 31.252
  19:25:44 STEP 1516 H= 379:00 +CPU= 30.305
  19:26:15 STEP 1517 H= 379:15 +CPU= 30.648
  19:26:45 STEP 1518 H= 379:30 +CPU= 29.852
  19:27:17 STEP 1519 H= 379:45 +CPU= 31.640
  19:27:49 STEP 1520 H= 380:00 +CPU= 31.592
  19:28:18 STEP 1521 H= 380:15 +CPU= 28.713
  19:28:48 STEP 1522 H= 380:30 +CPU= 30.412
  19:29:19 STEP 1523 H= 380:45 +CPU= 31.135
  19:29:58 STEP 1524 H= 381:00 +CPU= 38.142
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001524
  19:30:27 STEP 1525 H= 381:15 +CPU= 29.127
  19:30:56 STEP 1526 H= 381:30 +CPU= 28.880
  19:31:27 STEP 1527 H= 381:45 +CPU= 30.324
  19:31:58 STEP 1528 H= 382:00 +CPU= 31.058
  19:32:28 STEP 1529 H= 382:15 +CPU= 30.316
  19:32:58 STEP 1530 H= 382:30 +CPU= 30.025
  19:33:29 STEP 1531 H= 382:45 +CPU= 30.090
  19:34:00 STEP 1532 H= 383:00 +CPU= 30.835
  19:34:30 STEP 1533 H= 383:15 +CPU= 30.357
  19:35:01 STEP 1534 H= 383:30 +CPU= 30.144
  19:35:29 STEP 1535 H= 383:45 +CPU= 28.653
  19:36:09 STEP 1536 H= 384:00 +CPU= 39.801
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0_r1385854505_15.zip
Uploading the intermediate file: upload_file_15.zip
  19:36:41 STEP 1537 H= 384:15 +CPU= 31.637
  19:37:13 STEP 1538 H= 384:30 +CPU= 31.585
  19:37:44 STEP 1539 H= 384:45 +CPU= 31.498
  19:38:17 STEP 1540 H= 385:00 +CPU= 32.457
  19:38:48 STEP 1541 H= 385:15 +CPU= 30.239
  19:39:18 STEP 1542 H= 385:30 +CPU= 30.093
  19:39:48 STEP 1543 H= 385:45 +CPU= 29.987
  19:40:18 STEP 1544 H= 386:00 +CPU= 30.152
  19:40:48 STEP 1545 H= 386:15 +CPU= 29.734
  19:41:18 STEP 1546 H= 386:30 +CPU= 29.834
  19:41:49 STEP 1547 H= 386:45 +CPU= 31.130
  19:42:27 STEP 1548 H= 387:00 +CPU= 37.240
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001548
  19:42:58 STEP 1549 H= 387:15 +CPU= 30.508
  19:43:27 STEP 1550 H= 387:30 +CPU= 29.558
  19:43:57 STEP 1551 H= 387:45 +CPU= 29.407
  19:44:28 STEP 1552 H= 388:00 +CPU= 30.901
  19:44:59 STEP 1553 H= 388:15 +CPU= 30.587
  19:45:29 STEP 1554 H= 388:30 +CPU= 30.064
  19:45:59 STEP 1555 H= 388:45 +CPU= 30.052
  19:46:29 STEP 1556 H= 389:00 +CPU= 30.171
  19:47:00 STEP 1557 H= 389:15 +CPU= 30.017
  19:47:31 STEP 1558 H= 389:30 +CPU= 31.205
  19:48:02 STEP 1559 H= 389:45 +CPU= 30.824
  19:48:38 STEP 1560 H= 390:00 +CPU= 35.459
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001560
  19:49:07 STEP 1561 H= 390:15 +CPU= 29.444
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a3oa
(argv4) batchid: 1018
(argv5) wuid: 12293089
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/11
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/11/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/11/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_d9a9888201b9a8b39d64ce07d0fc1847 to: /var/lib/boinc-client/slots/11/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/11/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089.zip
ic_ancil_file: ic_ancil_12293089
ifsdata_file: ifsdata_12293089
climate_data_file: clim_data_12293089
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/11/ic_ancil_12293089.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/11/ic_ancil_12293089.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_bd81978297b044ce3394313b6de486d9 to: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12293089.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/11/ifsdata/ifsdata_12293089.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/11/159l_2/clim_data_12293089.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/11/159l_2/clim_data_12293089.zip
Checking for progress XML file: /var/lib/boinc-client/slots/11/progress_file_12293089.xml
Opened progress file ok : /var/lib/boinc-client/slots/11/progress_file_12293089.xml
-- Model is restarting --
Adjusting last_iter, 1562, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/11/progress_file_12293089.xml
last_cpu_time: 48651
upload_file_number: 16
last_iter: 1537
last_upload: 1382400
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0_r1385854505
The child process has been launched with process id: 84353
Executing the command: /var/lib/boinc-client/slots/11/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:84353:84353] [20240615:235714:1718510234:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  23:57:42 STEP 1536 H= 384:00 +CPU= 24.317
  23:58:05 STEP 1537 H= 384:15 +CPU= 22.887
  23:58:28 STEP 1538 H= 384:30 +CPU= 23.133
  23:58:52 STEP 1539 H= 384:45 +CPU= 23.501
  23:59:17 STEP 1540 H= 385:00 +CPU= 25.088
  23:59:41 STEP 1541 H= 385:15 +CPU= 23.566
  00:00:04 STEP 1542 H= 385:30 +CPU= 22.965
  00:00:26 STEP 1543 H= 385:45 +CPU= 22.226
  00:00:51 STEP 1544 H= 386:00 +CPU= 24.294
  00:01:14 STEP 1545 H= 386:15 +CPU= 23.754
  00:01:38 STEP 1546 H= 386:30 +CPU= 23.045
  00:02:00 STEP 1547 H= 386:45 +CPU= 22.768
  00:02:30 STEP 1548 H= 387:00 +CPU= 29.055
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001548
  00:02:54 STEP 1549 H= 387:15 +CPU= 23.948
  00:03:17 STEP 1550 H= 387:30 +CPU= 23.108
  00:03:41 STEP 1551 H= 387:45 +CPU= 24.009
  00:04:06 STEP 1552 H= 388:00 +CPU= 25.197
  00:04:30 STEP 1553 H= 388:15 +CPU= 23.615
  00:04:53 STEP 1554 H= 388:30 +CPU= 23.002
  00:05:17 STEP 1555 H= 388:45 +CPU= 24.125
  00:05:42 STEP 1556 H= 389:00 +CPU= 24.250
  00:06:04 STEP 1557 H= 389:15 +CPU= 22.458
  00:06:27 STEP 1558 H= 389:30 +CPU= 22.829
  00:06:52 STEP 1559 H= 389:45 +CPU= 24.247
  00:07:22 STEP 1560 H= 390:00 +CPU= 29.848
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001560
  00:07:46 STEP 1561 H= 390:15 +CPU= 24.142
  00:08:10 STEP 1562 H= 390:30 +CPU= 23.847
  00:08:32 STEP 1563 H= 390:45 +CPU= 22.463
  00:08:57 STEP 1564 H= 391:00 +CPU= 24.184
  00:09:21 STEP 1565 H= 391:15 +CPU= 23.921
  00:09:46 STEP 1566 H= 391:30 +CPU= 24.766
  00:10:10 STEP 1567 H= 391:45 +CPU= 24.634
  00:10:34 STEP 1568 H= 392:00 +CPU= 23.727
  00:10:57 STEP 1569 H= 392:15 +CPU= 23.038
  00:11:20 STEP 1570 H= 392:30 +CPU= 22.673
  00:11:42 STEP 1571 H= 392:45 +CPU= 22.331
  00:12:12 STEP 1572 H= 393:00 +CPU= 29.711
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001572
  00:12:36 STEP 1573 H= 393:15 +CPU= 23.514
  00:12:59 STEP 1574 H= 393:30 +CPU= 23.526
  00:13:24 STEP 1575 H= 393:45 +CPU= 24.829
  00:13:48 STEP 1576 H= 394:00 +CPU= 23.861
  00:14:13 STEP 1577 H= 394:15 +CPU= 24.279
  00:14:36 STEP 1578 H= 394:30 +CPU= 23.294
  00:15:01 STEP 1579 H= 394:45 +CPU= 24.687
  00:15:26 STEP 1580 H= 395:00 +CPU= 24.529
  00:15:49 STEP 1581 H= 395:15 +CPU= 23.301
  00:16:12 STEP 1582 H= 395:30 +CPU= 22.950
  00:16:35 STEP 1583 H= 395:45 +CPU= 22.926
  00:17:07 STEP 1584 H= 396:00 +CPU= 31.523
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001584
  00:17:30 STEP 1585 H= 396:15 +CPU= 22.956
  00:17:53 STEP 1586 H= 396:30 +CPU= 23.126
  00:18:17 STEP 1587 H= 396:45 +CPU= 24.106
  00:18:42 STEP 1588 H= 397:00 +CPU= 24.561
  00:19:05 STEP 1589 H= 397:15 +CPU= 22.722
  00:19:28 STEP 1590 H= 397:30 +CPU= 22.936
  00:19:51 STEP 1591 H= 397:45 +CPU= 23.525
  00:20:14 STEP 1592 H= 398:00 +CPU= 23.146
  00:20:37 STEP 1593 H= 398:15 +CPU= 22.696
  00:21:00 STEP 1594 H= 398:30 +CPU= 22.511
  00:21:23 STEP 1595 H= 398:45 +CPU= 23.153
  00:21:52 STEP 1596 H= 399:00 +CPU= 28.432
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001596
  00:22:15 STEP 1597 H= 399:15 +CPU= 23.447
  00:22:37 STEP 1598 H= 399:30 +CPU= 21.582
  00:22:59 STEP 1599 H= 399:45 +CPU= 22.349
  00:23:24 STEP 1600 H= 400:00 +CPU= 24.260
  00:23:47 STEP 1601 H= 400:15 +CPU= 23.309
  00:24:09 STEP 1602 H= 400:30 +CPU= 22.374
  00:24:32 STEP 1603 H= 400:45 +CPU= 22.488
  00:24:56 STEP 1604 H= 401:00 +CPU= 23.927
  00:25:20 STEP 1605 H= 401:15 +CPU= 23.724
  00:25:42 STEP 1606 H= 401:30 +CPU= 22.539
  00:26:07 STEP 1607 H= 401:45 +CPU= 24.153
  00:26:36 STEP 1608 H= 402:00 +CPU= 29.312
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001608
  00:27:00 STEP 1609 H= 402:15 +CPU= 24.196
  00:27:23 STEP 1610 H= 402:30 +CPU= 22.565
  00:27:46 STEP 1611 H= 402:45 +CPU= 22.435
  00:28:10 STEP 1612 H= 403:00 +CPU= 24.450
  00:28:33 STEP 1613 H= 403:15 +CPU= 23.348
  00:28:56 STEP 1614 H= 403:30 +CPU= 22.501
  00:29:19 STEP 1615 H= 403:45 +CPU= 22.734
  00:29:42 STEP 1616 H= 404:00 +CPU= 23.490
  00:30:06 STEP 1617 H= 404:15 +CPU= 23.819
  00:30:29 STEP 1618 H= 404:30 +CPU= 22.997
  00:30:53 STEP 1619 H= 404:45 +CPU= 23.252
  00:31:23 STEP 1620 H= 405:00 +CPU= 29.649
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001620
  00:31:46 STEP 1621 H= 405:15 +CPU= 23.353
  00:32:09 STEP 1622 H= 405:30 +CPU= 22.679
  00:32:32 STEP 1623 H= 405:45 +CPU= 23.353
  00:32:57 STEP 1624 H= 406:00 +CPU= 25.012
  00:33:20 STEP 1625 H= 406:15 +CPU= 22.548
  00:33:42 STEP 1626 H= 406:30 +CPU= 22.491
  00:34:06 STEP 1627 H= 406:45 +CPU= 23.067
  00:34:30 STEP 1628 H= 407:00 +CPU= 23.912
  00:34:53 STEP 1629 H= 407:15 +CPU= 22.844
  00:35:16 STEP 1630 H= 407:30 +CPU= 23.180
  00:35:39 STEP 1631 H= 407:45 +CPU= 22.970
  00:36:12 STEP 1632 H= 408:00 +CPU= 33.094
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0_r1385854505_16.zip
Uploading the intermediate file: upload_file_16.zip
  00:36:35 STEP 1633 H= 408:15 +CPU= 22.641
  00:36:58 STEP 1634 H= 408:30 +CPU= 23.389
  00:37:21 STEP 1635 H= 408:45 +CPU= 22.517
  00:37:44 STEP 1636 H= 409:00 +CPU= 23.408
  00:38:07 STEP 1637 H= 409:15 +CPU= 22.433
  00:38:30 STEP 1638 H= 409:30 +CPU= 22.904
  00:38:53 STEP 1639 H= 409:45 +CPU= 22.994
  00:39:16 STEP 1640 H= 410:00 +CPU= 23.021
  00:39:41 STEP 1641 H= 410:15 +CPU= 24.617
  00:40:04 STEP 1642 H= 410:30 +CPU= 23.148
  00:40:28 STEP 1643 H= 410:45 +CPU= 24.406
  00:40:56 STEP 1644 H= 411:00 +CPU= 27.888
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001644
  00:41:21 STEP 1645 H= 411:15 +CPU= 24.283
  00:41:45 STEP 1646 H= 411:30 +CPU= 23.977
  00:42:08 STEP 1647 H= 411:45 +CPU= 23.196
  00:42:31 STEP 1648 H= 412:00 +CPU= 23.214
  00:42:54 STEP 1649 H= 412:15 +CPU= 22.110
  00:43:16 STEP 1650 H= 412:30 +CPU= 22.262
  00:43:39 STEP 1651 H= 412:45 +CPU= 23.166
  00:44:02 STEP 1652 H= 413:00 +CPU= 22.743
  00:44:26 STEP 1653 H= 413:15 +CPU= 23.435
  00:44:49 STEP 1654 H= 413:30 +CPU= 23.056
  00:45:12 STEP 1655 H= 413:45 +CPU= 23.155
  00:45:41 STEP 1656 H= 414:00 +CPU= 28.925
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001656
  00:46:04 STEP 1657 H= 414:15 +CPU= 23.233
  00:46:27 STEP 1658 H= 414:30 +CPU= 22.617
  00:46:50 STEP 1659 H= 414:45 +CPU= 22.320
  00:47:13 STEP 1660 H= 415:00 +CPU= 23.144
  00:47:37 STEP 1661 H= 415:15 +CPU= 24.287
  00:47:59 STEP 1662 H= 415:30 +CPU= 22.142
  00:48:22 STEP 1663 H= 415:45 +CPU= 22.113
  00:48:45 STEP 1664 H= 416:00 +CPU= 23.368
  00:49:07 STEP 1665 H= 416:15 +CPU= 22.243
  00:49:30 STEP 1666 H= 416:30 +CPU= 22.763
  00:49:53 STEP 1667 H= 416:45 +CPU= 22.783
  00:50:22 STEP 1668 H= 417:00 +CPU= 28.939
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001668
  00:50:46 STEP 1669 H= 417:15 +CPU= 23.435
  00:51:09 STEP 1670 H= 417:30 +CPU= 23.037
  00:51:32 STEP 1671 H= 417:45 +CPU= 23.010
  00:51:56 STEP 1672 H= 418:00 +CPU= 23.880
  00:52:19 STEP 1673 H= 418:15 +CPU= 23.107
  00:52:41 STEP 1674 H= 418:30 +CPU= 22.239
  00:53:05 STEP 1675 H= 418:45 +CPU= 23.315
  00:53:30 STEP 1676 H= 419:00 +CPU= 25.055
  00:53:53 STEP 1677 H= 419:15 +CPU= 22.826
  00:54:17 STEP 1678 H= 419:30 +CPU= 23.730
  00:54:40 STEP 1679 H= 419:45 +CPU= 22.712
  00:55:12 STEP 1680 H= 420:00 +CPU= 32.180
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001680
  00:55:36 STEP 1681 H= 420:15 +CPU= 23.658
  00:55:59 STEP 1682 H= 420:30 +CPU= 23.140
  00:56:22 STEP 1683 H= 420:45 +CPU= 22.631
  00:56:46 STEP 1684 H= 421:00 +CPU= 24.169
  00:57:09 STEP 1685 H= 421:15 +CPU= 23.020
  00:57:32 STEP 1686 H= 421:30 +CPU= 23.126
  00:57:55 STEP 1687 H= 421:45 +CPU= 22.955
  00:58:17 STEP 1688 H= 422:00 +CPU= 22.130
  00:58:38 STEP 1689 H= 422:15 +CPU= 20.261
  00:58:58 STEP 1690 H= 422:30 +CPU= 20.097
  00:59:16 STEP 1691 H= 422:45 +CPU= 17.648
  00:59:37 STEP 1692 H= 423:00 +CPU= 20.979
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001692
  00:59:53 STEP 1693 H= 423:15 +CPU= 16.173
  01:00:08 STEP 1694 H= 423:30 +CPU= 14.735
  01:00:22 STEP 1695 H= 423:45 +CPU= 14.678
  01:00:38 STEP 1696 H= 424:00 +CPU= 15.028
  01:00:52 STEP 1697 H= 424:15 +CPU= 14.389
  01:01:07 STEP 1698 H= 424:30 +CPU= 14.739
  01:01:21 STEP 1699 H= 424:45 +CPU= 14.553
  01:01:37 STEP 1700 H= 425:00 +CPU= 15.135
  01:01:51 STEP 1701 H= 425:15 +CPU= 14.536
  01:02:06 STEP 1702 H= 425:30 +CPU= 14.539
  01:02:20 STEP 1703 H= 425:45 +CPU= 14.803
  01:02:39 STEP 1704 H= 426:00 +CPU= 18.616
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001704
  01:02:54 STEP 1705 H= 426:15 +CPU= 14.740
  01:03:09 STEP 1706 H= 426:30 +CPU= 14.785
  01:03:24 STEP 1707 H= 426:45 +CPU= 14.806
  01:03:38 STEP 1708 H= 427:00 +CPU= 14.853
  01:03:53 STEP 1709 H= 427:15 +CPU= 14.677
  01:04:08 STEP 1710 H= 427:30 +CPU= 14.600
  01:04:23 STEP 1711 H= 427:45 +CPU= 14.985
  01:04:38 STEP 1712 H= 428:00 +CPU= 15.186
  01:04:53 STEP 1713 H= 428:15 +CPU= 14.586
  01:05:08 STEP 1714 H= 428:30 +CPU= 14.894
  01:05:22 STEP 1715 H= 428:45 +CPU= 14.459
  01:05:41 STEP 1716 H= 429:00 +CPU= 18.373
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001716
  01:05:55 STEP 1717 H= 429:15 +CPU= 14.633
  01:06:10 STEP 1718 H= 429:30 +CPU= 14.455
  01:06:24 STEP 1719 H= 429:45 +CPU= 14.614
  01:06:39 STEP 1720 H= 430:00 +CPU= 15.108
  01:06:54 STEP 1721 H= 430:15 +CPU= 14.699
  01:07:09 STEP 1722 H= 430:30 +CPU= 14.606
  01:07:24 STEP 1723 H= 430:45 +CPU= 14.745
  01:07:38 STEP 1724 H= 431:00 +CPU= 14.819
  01:07:53 STEP 1725 H= 431:15 +CPU= 14.722
  01:08:08 STEP 1726 H= 431:30 +CPU= 14.491
  01:08:22 STEP 1727 H= 431:45 +CPU= 14.406
  01:08:43 STEP 1728 H= 432:00 +CPU= 20.627
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0_r1385854505_17.zip
Uploading the intermediate file: upload_file_17.zip
  01:08:58 STEP 1729 H= 432:15 +CPU= 15.431
  01:09:13 STEP 1730 H= 432:30 +CPU= 14.844
  01:09:28 STEP 1731 H= 432:45 +CPU= 14.705
  01:09:43 STEP 1732 H= 433:00 +CPU= 15.003
  01:09:58 STEP 1733 H= 433:15 +CPU= 14.880
  01:10:13 STEP 1734 H= 433:30 +CPU= 14.983
  01:10:27 STEP 1735 H= 433:45 +CPU= 14.422
  01:10:42 STEP 1736 H= 434:00 +CPU= 14.834
  01:10:57 STEP 1737 H= 434:15 +CPU= 14.726
  01:11:12 STEP 1738 H= 434:30 +CPU= 14.731
  01:11:26 STEP 1739 H= 434:45 +CPU= 14.529
  01:11:45 STEP 1740 H= 435:00 +CPU= 18.438
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001740
  01:11:59 STEP 1741 H= 435:15 +CPU= 14.475
  01:12:14 STEP 1742 H= 435:30 +CPU= 14.781
  01:12:29 STEP 1743 H= 435:45 +CPU= 14.776
  01:12:44 STEP 1744 H= 436:00 +CPU= 15.249
  01:12:59 STEP 1745 H= 436:15 +CPU= 14.977
  01:13:14 STEP 1746 H= 436:30 +CPU= 14.818
  01:13:29 STEP 1747 H= 436:45 +CPU= 15.124
  01:13:45 STEP 1748 H= 437:00 +CPU= 15.292
  01:13:59 STEP 1749 H= 437:15 +CPU= 14.611
  01:14:14 STEP 1750 H= 437:30 +CPU= 14.323
  01:14:28 STEP 1751 H= 437:45 +CPU= 14.566
  01:14:47 STEP 1752 H= 438:00 +CPU= 18.736
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001752
  01:15:01 STEP 1753 H= 438:15 +CPU= 14.405
  01:15:16 STEP 1754 H= 438:30 +CPU= 14.572
  01:15:31 STEP 1755 H= 438:45 +CPU= 14.804
  01:15:46 STEP 1756 H= 439:00 +CPU= 15.153
  01:16:01 STEP 1757 H= 439:15 +CPU= 14.635
  01:16:16 STEP 1758 H= 439:30 +CPU= 14.821
  01:16:30 STEP 1759 H= 439:45 +CPU= 14.753
  01:16:44 STEP 1760 H= 440:00 +CPU= 14.158
  01:16:58 STEP 1761 H= 440:15 +CPU= 13.499
  01:17:11 STEP 1762 H= 440:30 +CPU= 13.471
  01:17:25 STEP 1763 H= 440:45 +CPU= 13.445
  01:17:42 STEP 1764 H= 441:00 +CPU= 17.335
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001764
  01:17:56 STEP 1765 H= 441:15 +CPU= 13.426
  01:18:09 STEP 1766 H= 441:30 +CPU= 13.569
  01:18:23 STEP 1767 H= 441:45 +CPU= 13.529
  01:18:37 STEP 1768 H= 442:00 +CPU= 13.893
  01:18:50 STEP 1769 H= 442:15 +CPU= 13.557
  01:19:04 STEP 1770 H= 442:30 +CPU= 13.529
  01:19:17 STEP 1771 H= 442:45 +CPU= 13.536
  01:19:31 STEP 1772 H= 443:00 +CPU= 13.918
  01:19:45 STEP 1773 H= 443:15 +CPU= 13.601
  01:19:59 STEP 1774 H= 443:30 +CPU= 13.519
  01:20:12 STEP 1775 H= 443:45 +CPU= 13.568
  01:20:32 STEP 1776 H= 444:00 +CPU= 19.445
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001776
  01:20:45 STEP 1777 H= 444:15 +CPU= 13.384
  01:20:59 STEP 1778 H= 444:30 +CPU= 13.555
  01:21:12 STEP 1779 H= 444:45 +CPU= 13.463
  01:21:26 STEP 1780 H= 445:00 +CPU= 13.799
  01:21:39 STEP 1781 H= 445:15 +CPU= 13.551
  01:21:53 STEP 1782 H= 445:30 +CPU= 13.691
  01:22:07 STEP 1783 H= 445:45 +CPU= 13.455
  01:22:20 STEP 1784 H= 446:00 +CPU= 13.762
  01:22:34 STEP 1785 H= 446:15 +CPU= 13.273
  01:22:47 STEP 1786 H= 446:30 +CPU= 13.468
  01:23:01 STEP 1787 H= 446:45 +CPU= 13.576
  01:23:18 STEP 1788 H= 447:00 +CPU= 17.010
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001788
  01:23:31 STEP 1789 H= 447:15 +CPU= 12.886
  01:23:44 STEP 1790 H= 447:30 +CPU= 12.654
  01:23:56 STEP 1791 H= 447:45 +CPU= 12.644
  01:24:09 STEP 1792 H= 448:00 +CPU= 12.934
  01:24:22 STEP 1793 H= 448:15 +CPU= 12.803
  01:24:35 STEP 1794 H= 448:30 +CPU= 12.801
  01:24:48 STEP 1795 H= 448:45 +CPU= 12.751
  01:25:01 STEP 1796 H= 449:00 +CPU= 13.074
  01:25:13 STEP 1797 H= 449:15 +CPU= 12.576
  01:25:26 STEP 1798 H= 449:30 +CPU= 12.532
  01:25:38 STEP 1799 H= 449:45 +CPU= 12.624
  01:25:55 STEP 1800 H= 450:00 +CPU= 16.137
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001800
  01:26:07 STEP 1801 H= 450:15 +CPU= 12.496
  01:26:20 STEP 1802 H= 450:30 +CPU= 12.500
  01:26:32 STEP 1803 H= 450:45 +CPU= 12.483
  01:26:45 STEP 1804 H= 451:00 +CPU= 13.030
  01:26:58 STEP 1805 H= 451:15 +CPU= 12.671
  01:27:10 STEP 1806 H= 451:30 +CPU= 12.642
  01:27:23 STEP 1807 H= 451:45 +CPU= 12.524
  01:27:36 STEP 1808 H= 452:00 +CPU= 13.114
  01:27:49 STEP 1809 H= 452:15 +CPU= 12.632
  01:28:01 STEP 1810 H= 452:30 +CPU= 12.513
  01:28:14 STEP 1811 H= 452:45 +CPU= 12.472
  01:28:30 STEP 1812 H= 453:00 +CPU= 15.935
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001812
  01:28:42 STEP 1813 H= 453:15 +CPU= 12.586
  01:28:55 STEP 1814 H= 453:30 +CPU= 12.620
  01:29:07 STEP 1815 H= 453:45 +CPU= 12.530
  01:29:20 STEP 1816 H= 454:00 +CPU= 12.978
  01:29:33 STEP 1817 H= 454:15 +CPU= 12.788
  01:29:46 STEP 1818 H= 454:30 +CPU= 12.436
  01:29:58 STEP 1819 H= 454:45 +CPU= 12.561
  01:30:11 STEP 1820 H= 455:00 +CPU= 13.097
  01:30:24 STEP 1821 H= 455:15 +CPU= 12.630
  01:30:37 STEP 1822 H= 455:30 +CPU= 12.659
  01:30:49 STEP 1823 H= 455:45 +CPU= 12.535
  01:31:08 STEP 1824 H= 456:00 +CPU= 18.387
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0_r1385854505_18.zip
Uploading the intermediate file: upload_file_18.zip
  01:31:20 STEP 1825 H= 456:15 +CPU= 12.661
  01:31:33 STEP 1826 H= 456:30 +CPU= 12.547
  01:31:46 STEP 1827 H= 456:45 +CPU= 12.711
  01:31:59 STEP 1828 H= 457:00 +CPU= 13.139
  01:32:12 STEP 1829 H= 457:15 +CPU= 12.720
  01:32:24 STEP 1830 H= 457:30 +CPU= 12.542
  01:32:37 STEP 1831 H= 457:45 +CPU= 12.581
  01:32:50 STEP 1832 H= 458:00 +CPU= 12.826
  01:33:02 STEP 1833 H= 458:15 +CPU= 12.547
  01:33:15 STEP 1834 H= 458:30 +CPU= 12.597
  01:33:27 STEP 1835 H= 458:45 +CPU= 12.632
  01:33:44 STEP 1836 H= 459:00 +CPU= 16.157
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001836
  01:33:56 STEP 1837 H= 459:15 +CPU= 12.762
  01:34:09 STEP 1838 H= 459:30 +CPU= 12.700
  01:34:22 STEP 1839 H= 459:45 +CPU= 12.571
  01:34:34 STEP 1840 H= 460:00 +CPU= 12.860
  01:34:47 STEP 1841 H= 460:15 +CPU= 12.682
  01:35:00 STEP 1842 H= 460:30 +CPU= 12.654
  01:35:12 STEP 1843 H= 460:45 +CPU= 12.519
  01:35:25 STEP 1844 H= 461:00 +CPU= 13.082
  01:35:38 STEP 1845 H= 461:15 +CPU= 12.552
  01:35:51 STEP 1846 H= 461:30 +CPU= 12.600
  01:36:03 STEP 1847 H= 461:45 +CPU= 12.573
  01:36:19 STEP 1848 H= 462:00 +CPU= 16.187
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001848
  01:36:32 STEP 1849 H= 462:15 +CPU= 12.543
  01:36:45 STEP 1850 H= 462:30 +CPU= 12.555
  01:36:57 STEP 1851 H= 462:45 +CPU= 12.621
  01:37:10 STEP 1852 H= 463:00 +CPU= 12.881
  01:37:23 STEP 1853 H= 463:15 +CPU= 12.786
  01:37:36 STEP 1854 H= 463:30 +CPU= 12.609
  01:37:48 STEP 1855 H= 463:45 +CPU= 12.736
  01:38:01 STEP 1856 H= 464:00 +CPU= 13.048
  01:38:14 STEP 1857 H= 464:15 +CPU= 12.621
  01:38:27 STEP 1858 H= 464:30 +CPU= 12.680
  01:38:39 STEP 1859 H= 464:45 +CPU= 12.540
  01:38:55 STEP 1860 H= 465:00 +CPU= 15.875
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001860
  01:39:08 STEP 1861 H= 465:15 +CPU= 12.583
  01:39:20 STEP 1862 H= 465:30 +CPU= 12.638
  01:39:33 STEP 1863 H= 465:45 +CPU= 12.445
  01:39:46 STEP 1864 H= 466:00 +CPU= 12.889
  01:39:58 STEP 1865 H= 466:15 +CPU= 12.537
  01:40:11 STEP 1866 H= 466:30 +CPU= 12.680
  01:40:24 STEP 1867 H= 466:45 +CPU= 12.597
  01:40:36 STEP 1868 H= 467:00 +CPU= 12.825
  01:40:49 STEP 1869 H= 467:15 +CPU= 12.543
  01:41:02 STEP 1870 H= 467:30 +CPU= 12.640
  01:41:14 STEP 1871 H= 467:45 +CPU= 12.570
  01:41:33 STEP 1872 H= 468:00 +CPU= 18.462
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001872
  01:41:45 STEP 1873 H= 468:15 +CPU= 12.589
  01:41:58 STEP 1874 H= 468:30 +CPU= 12.585
  01:42:10 STEP 1875 H= 468:45 +CPU= 12.595
  01:42:23 STEP 1876 H= 469:00 +CPU= 12.973
  01:42:36 STEP 1877 H= 469:15 +CPU= 12.636
  01:42:49 STEP 1878 H= 469:30 +CPU= 12.527
  01:43:01 STEP 1879 H= 469:45 +CPU= 12.429
  01:43:14 STEP 1880 H= 470:00 +CPU= 12.953
  01:43:27 STEP 1881 H= 470:15 +CPU= 12.568
  01:43:39 STEP 1882 H= 470:30 +CPU= 12.620
  01:43:52 STEP 1883 H= 470:45 +CPU= 12.651
  01:44:08 STEP 1884 H= 471:00 +CPU= 16.037
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001884
  01:44:21 STEP 1885 H= 471:15 +CPU= 12.739
  01:44:34 STEP 1886 H= 471:30 +CPU= 12.768
  01:44:46 STEP 1887 H= 471:45 +CPU= 12.574
  01:44:59 STEP 1888 H= 472:00 +CPU= 13.003
  01:45:12 STEP 1889 H= 472:15 +CPU= 12.592
  01:45:25 STEP 1890 H= 472:30 +CPU= 12.674
  01:45:37 STEP 1891 H= 472:45 +CPU= 12.633
  01:45:50 STEP 1892 H= 473:00 +CPU= 13.058
  01:46:03 STEP 1893 H= 473:15 +CPU= 12.496
  01:46:15 STEP 1894 H= 473:30 +CPU= 12.682
  01:46:28 STEP 1895 H= 473:45 +CPU= 12.623
  01:46:44 STEP 1896 H= 474:00 +CPU= 16.142
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001896
  01:46:57 STEP 1897 H= 474:15 +CPU= 12.785
  01:47:10 STEP 1898 H= 474:30 +CPU= 12.598
  01:47:22 STEP 1899 H= 474:45 +CPU= 12.518
  01:47:35 STEP 1900 H= 475:00 +CPU= 12.959
  01:47:48 STEP 1901 H= 475:15 +CPU= 12.657
  01:48:00 STEP 1902 H= 475:30 +CPU= 12.579
  01:48:13 STEP 1903 H= 475:45 +CPU= 12.586
  01:48:26 STEP 1904 H= 476:00 +CPU= 12.765
  01:48:38 STEP 1905 H= 476:15 +CPU= 12.383
  01:48:51 STEP 1906 H= 476:30 +CPU= 12.512
  01:49:03 STEP 1907 H= 476:45 +CPU= 12.477
  01:49:19 STEP 1908 H= 477:00 +CPU= 15.981
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001908
  01:49:32 STEP 1909 H= 477:15 +CPU= 12.561
  01:49:44 STEP 1910 H= 477:30 +CPU= 12.650
  01:49:57 STEP 1911 H= 477:45 +CPU= 12.445
  01:50:10 STEP 1912 H= 478:00 +CPU= 13.001
  01:50:23 STEP 1913 H= 478:15 +CPU= 12.623
  01:50:35 STEP 1914 H= 478:30 +CPU= 12.449
  01:50:48 STEP 1915 H= 478:45 +CPU= 12.574
  01:51:00 STEP 1916 H= 479:00 +CPU= 12.717
  01:51:13 STEP 1917 H= 479:15 +CPU= 12.584
  01:51:25 STEP 1918 H= 479:30 +CPU= 12.428
  01:51:38 STEP 1919 H= 479:45 +CPU= 12.350
  01:51:55 STEP 1920 H= 480:00 +CPU= 17.408
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1234580036E-20      I      -0.7966890281E-07      I
ICloud fractionI      0.0000000000E+00      I       0.4852677235E-01      I
I  Relat. Hum. I      0.3356399846E+00      I       0.4240829925E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6760988731E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    314000708 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS785001.8 Mbytes/s, TOTAL TIME=    0.0055( 7.3%)
 01:51:55 STEP 1920 H= 480:00 +CPU= 17.408
 END CNT3
 NSTEP =  1920 CNT0   000000000  12.076  12.076  12.087   0.999               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        11440/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    203    0       46      1044068    11440 ************    15.4     1.3    14.1     2.1    27.4
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 6880971.5 6880971.5    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 6877499.0 6877499.0     99.95      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  417    3115.8    3115.8     18.88      0.00
   9 SPCM     - SPECTRAL COMP.                     384     495.6     495.6      2.77      0.00
  10 SCAN2M - PHYSICS                              385    8176.4    8176.4     45.75      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 32     655.9     655.9      0.31      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 17      34.5      34.5      0.01      0.00
  17 GRIDFPOS IN CNT4                               32      18.7      18.7      0.01      0.00
  27 RESTART FILES - WRITING                         8    2623.5    2623.5      0.31      0.00
  28 RESTART FILES - READING                         1    3002.5    3002.5      0.04      0.00
  29 SU4FPOS IN CNT4                                32       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                32    3728.5    3728.5      1.73      0.00
  31 POSDDH IN STEPO                                 2       0.4       0.4      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    385    3796.3    3796.3     21.24      0.00
  39 SU0YOMB                                         1     305.3     305.3      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    385      14.5      14.5      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              384     236.8     236.8      1.32      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              384      42.7      42.7      0.24      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 17       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      836     234.8     234.8      2.85      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       800     135.0     135.0      1.57      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        800      53.7      53.7      0.62      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       836      68.0      68.0      0.83      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      27.6      27.6      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            836      53.2      53.2      0.65      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            800      28.0      28.0      0.33      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            836      46.5      46.5      0.57      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            800      25.6      25.6      0.30      0.00
 400 GSTATS                                      45508       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 42288       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     6881.0 CPU TIME    6861.3 VECTOR TIME     6861.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      251.3
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/11/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/11/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/11/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293089/ICMSHh7zg+001884
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0_r1385854505_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
01:55:23 (84346): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
16 Jun 2024 05:54:18 1532599 22446701 oifs_43r3_bl_a3oa_2016092300_20_1018_12293089_0 1,727,100 55,512 0.0321


©2024 cpdn.org