Name | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 |
Workunit | 12184948 |
Created | 22 Dec 2022, 18:28:27 UTC |
Sent | 15 Jan 2023, 20:23:13 UTC |
Report deadline | 14 Feb 2023, 20:23:13 UTC |
Received | 18 Jan 2023, 18:29:12 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 9 (0x00000009) Unknown error code |
Computer ID | 1478188 |
Run time | 19 hours 9 min 10 sec |
CPU time | 18 hours 32 min 2 sec |
Validate state | Invalid |
Credit | 9,072.00 |
Device peak FLOPS | 4.20 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,402.76 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 3,646.07 MB |
Stderr | <core_client_version>7.20.2</core_client_version> <![CDATA[ <message> process exited with code 9 (0x9, -247)</message> <stderr_txt> ctory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001488 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001464 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001464 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001464 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001476 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001476 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001476 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_61.zip Uploading the intermediate file: upload_file_61.zip 20:58:37 STEP 1489 H=1489:00 +CPU= 28.647 Uploading trickle at timestep: 5356800 20:59:09 STEP 1490 H=1490:00 +CPU= 31.543 20:59:58 STEP 1491 H=1491:00 +CPU= 47.868 21:00:30 STEP 1492 H=1492:00 +CPU= 32.222 21:01:01 STEP 1493 H=1493:00 +CPU= 30.735 21:01:52 STEP 1494 H=1494:00 +CPU= 50.288 21:02:23 STEP 1495 H=1495:00 +CPU= 30.839 21:02:54 STEP 1496 H=1496:00 +CPU= 30.747 21:03:42 STEP 1497 H=1497:00 +CPU= 47.468 21:04:13 STEP 1498 H=1498:00 +CPU= 31.235 21:04:44 STEP 1499 H=1499:00 +CPU= 29.847 21:05:37 STEP 1500 H=1500:00 +CPU= 53.040 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001500 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001500 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001500 21:06:09 STEP 1501 H=1501:00 +CPU= 31.263 21:06:39 STEP 1502 H=1502:00 +CPU= 29.522 21:07:27 STEP 1503 H=1503:00 +CPU= 47.465 21:07:58 STEP 1504 H=1504:00 +CPU= 31.207 21:08:28 STEP 1505 H=1505:00 +CPU= 29.800 21:09:19 STEP 1506 H=1506:00 +CPU= 50.043 21:09:49 STEP 1507 H=1507:00 +CPU= 29.649 21:10:21 STEP 1508 H=1508:00 +CPU= 31.633 21:11:10 STEP 1509 H=1509:00 +CPU= 49.007 21:11:42 STEP 1510 H=1510:00 +CPU= 31.308 21:12:13 STEP 1511 H=1511:00 +CPU= 31.046 21:13:09 STEP 1512 H=1512:00 +CPU= 55.970 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001512 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001512 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001512 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001488 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001488 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001488 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001500 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001500 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001500 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_62.zip Uploading the intermediate file: upload_file_62.zip 21:13:40 STEP 1513 H=1513:00 +CPU= 30.145 21:14:11 STEP 1514 H=1514:00 +CPU= 30.630 21:15:00 STEP 1515 H=1515:00 +CPU= 48.975 21:15:31 STEP 1516 H=1516:00 +CPU= 30.241 21:16:01 STEP 1517 H=1517:00 +CPU= 30.091 21:16:50 STEP 1518 H=1518:00 +CPU= 48.640 21:17:20 STEP 1519 H=1519:00 +CPU= 29.094 21:17:51 STEP 1520 H=1520:00 +CPU= 31.327 21:18:40 STEP 1521 H=1521:00 +CPU= 48.399 21:19:11 STEP 1522 H=1522:00 +CPU= 31.037 21:19:42 STEP 1523 H=1523:00 +CPU= 29.962 21:20:36 STEP 1524 H=1524:00 +CPU= 53.634 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001524 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001524 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001524 21:21:06 STEP 1525 H=1525:00 +CPU= 30.311 21:21:37 STEP 1526 H=1526:00 +CPU= 30.476 21:22:26 STEP 1527 H=1527:00 +CPU= 48.350 21:22:57 STEP 1528 H=1528:00 +CPU= 30.746 21:23:26 STEP 1529 H=1529:00 +CPU= 28.937 21:24:16 STEP 1530 H=1530:00 +CPU= 48.997 21:24:46 STEP 1531 H=1531:00 +CPU= 29.862 21:25:17 STEP 1532 H=1532:00 +CPU= 31.073 21:26:05 STEP 1533 H=1533:00 +CPU= 47.720 21:26:36 STEP 1534 H=1534:00 +CPU= 30.192 21:27:07 STEP 1535 H=1535:00 +CPU= 30.762 21:28:04 STEP 1536 H=1536:00 +CPU= 56.258 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001536 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001536 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001536 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001512 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001512 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001512 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001524 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001524 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001524 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_63.zip Uploading the intermediate file: upload_file_63.zip 21:28:35 STEP 1537 H=1537:00 +CPU= 30.719 21:29:06 STEP 1538 H=1538:00 +CPU= 30.929 21:29:55 STEP 1539 H=1539:00 +CPU= 48.532 21:30:27 STEP 1540 H=1540:00 +CPU= 31.648 21:30:57 STEP 1541 H=1541:00 +CPU= 29.896 21:31:48 STEP 1542 H=1542:00 +CPU= 50.403 21:32:19 STEP 1543 H=1543:00 +CPU= 30.500 21:32:51 STEP 1544 H=1544:00 +CPU= 31.381 21:33:39 STEP 1545 H=1545:00 +CPU= 47.895 21:34:11 STEP 1546 H=1546:00 +CPU= 31.135 21:34:41 STEP 1547 H=1547:00 +CPU= 29.881 21:35:35 STEP 1548 H=1548:00 +CPU= 53.327 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001548 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001548 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001548 21:36:05 STEP 1549 H=1549:00 +CPU= 30.348 21:36:37 STEP 1550 H=1550:00 +CPU= 31.729 21:37:25 STEP 1551 H=1551:00 +CPU= 47.162 21:37:57 STEP 1552 H=1552:00 +CPU= 31.726 21:38:27 STEP 1553 H=1553:00 +CPU= 29.681 21:39:17 STEP 1554 H=1554:00 +CPU= 50.040 21:39:47 STEP 1555 H=1555:00 +CPU= 29.636 21:40:19 STEP 1556 H=1556:00 +CPU= 31.521 21:41:08 STEP 1557 H=1557:00 +CPU= 48.750 21:41:39 STEP 1558 H=1558:00 +CPU= 31.022 21:42:11 STEP 1559 H=1559:00 +CPU= 31.319 21:43:09 STEP 1560 H=1560:00 +CPU= 57.096 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001560 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001560 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001560 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001536 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001536 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001536 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001548 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001548 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001548 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_64.zip Uploading the intermediate file: upload_file_64.zip 21:43:39 STEP 1561 H=1561:00 +CPU= 29.665 21:44:10 STEP 1562 H=1562:00 +CPU= 31.097 21:44:59 STEP 1563 H=1563:00 +CPU= 48.190 21:45:30 STEP 1564 H=1564:00 +CPU= 30.444 21:45:59 STEP 1565 H=1565:00 +CPU= 29.466 21:46:49 STEP 1566 H=1566:00 +CPU= 49.152 21:47:19 STEP 1567 H=1567:00 +CPU= 29.784 Suspend request received from the BOINC client, suspending the child process Resuming the child process [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.587] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4412MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.620] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.620] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.620] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.620] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.620] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4412 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.620] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223917:1673995157:41304.621] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDSC [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223918:1673995158:41305.628] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:50498:50498] [20230117:223918:1673995158:41305.628] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001021F36 cloudsc_ 1068 cloudsc.F90 oifs_43r3_model.e 0000000001014D57 cloud_layer_ 131 cloud_layer.F90 oifs_43r3_model.e 0000000000F10C1B callpar_ 1011 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1999050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0304 (argv4) batchid: 968 (argv5) wuid: 12184948 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /data/boinc/boinc-client/slots/23 Project directory is: /data/boinc/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948 ..mkdir for temp folder for results failed Copying: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_363f329e3f9995b0ad4e29c296b93b38 to: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_0304_1999050100_123_968_12184948.zip Unzipping the namelist zip file: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_0304_1999050100_123_968_12184948.zip ic_ancil_file: ic_ancil_12184948 ifsdata_file: ifsdata_12184948 climate_data_file: clim_data_12184948 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_20504c40489c6109352abe3407c4fae8 to: /data/boinc/boinc-client/slots/23/ic_ancil_12184948.zip Unzipping the IC ancils zip file: /data/boinc/boinc-client/slots/23/ic_ancil_12184948.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7fa72421af36ba59b198767f22f7816e to: /data/boinc/boinc-client/slots/23/ifsdata/ifsdata_12184948.zip Unzipping the ifsdata_zip file: /data/boinc/boinc-client/slots/23/ifsdata/ifsdata_12184948.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /data/boinc/boinc-client/slots/23/159l_2/clim_data_12184948.zip Unzipping the climate data zip file: /data/boinc/boinc-client/slots/23/159l_2/clim_data_12184948.zip Checking for progress XML file: /data/boinc/boinc-client/slots/23/progress_file_12184948.xml Opened progress file ok : /data/boinc/boinc-client/slots/23/progress_file_12184948.xml -- Model is restarting -- Adjusting last_iter, 1568, to previous model restart step. Creating progress file: /data/boinc/boinc-client/slots/23/progress_file_12184948.xml last_cpu_time: 53394 upload_file_number: 65 last_iter: 1559 last_upload: 5616000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995 The child process has been launched with process id: 13633 Executing the command: /data/boinc/boinc-client/slots/23/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:13633:13633] [20230118:085734:1674032254:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:08:29 STEP 1560 H=1560:00 +CPU= 57.456 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:36:23 STEP 1561 H=1561:00 +CPU= 30.703 09:36:54 STEP 1562 H=1562:00 +CPU= 30.577 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:39:44 STEP 1563 H=1563:00 +CPU= 48.438 09:40:15 STEP 1564 H=1564:00 +CPU= 30.400 09:40:46 STEP 1565 H=1565:00 +CPU= 30.466 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:19:10 STEP 1566 H=1566:00 +CPU= 48.505 10:19:39 STEP 1567 H=1567:00 +CPU= 29.173 10:20:11 STEP 1568 H=1568:00 +CPU= 31.666 10:21:00 STEP 1569 H=1569:00 +CPU= 47.977 10:21:31 STEP 1570 H=1570:00 +CPU= 30.534 10:22:02 STEP 1571 H=1571:00 +CPU= 30.606 10:22:56 STEP 1572 H=1572:00 +CPU= 53.349 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001572 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001572 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001572 10:23:26 STEP 1573 H=1573:00 +CPU= 30.535 10:23:57 STEP 1574 H=1574:00 +CPU= 30.551 10:24:45 STEP 1575 H=1575:00 +CPU= 47.131 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:38:08 STEP 1576 H=1576:00 +CPU= 30.666 10:38:39 STEP 1577 H=1577:00 +CPU= 30.713 10:39:29 STEP 1578 H=1578:00 +CPU= 49.324 10:39:59 STEP 1579 H=1579:00 +CPU= 29.608 10:40:30 STEP 1580 H=1580:00 +CPU= 30.517 10:41:19 STEP 1581 H=1581:00 +CPU= 48.528 10:41:50 STEP 1582 H=1582:00 +CPU= 30.053 10:42:21 STEP 1583 H=1583:00 +CPU= 30.969 10:43:20 STEP 1584 H=1584:00 +CPU= 57.353 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001584 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001584 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001584 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001560 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001560 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001560 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001572 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001572 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001572 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_65.zip Uploading the intermediate file: upload_file_65.zip 10:43:43 STEP 1585 H=1585:00 +CPU= 22.620 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:04:51 STEP 1586 H=1586:00 +CPU= 30.931 12:05:40 STEP 1587 H=1587:00 +CPU= 48.143 12:06:11 STEP 1588 H=1588:00 +CPU= 30.821 12:06:43 STEP 1589 H=1589:00 +CPU= 31.023 12:07:32 STEP 1590 H=1590:00 +CPU= 48.344 12:08:02 STEP 1591 H=1591:00 +CPU= 29.968 12:08:33 STEP 1592 H=1592:00 +CPU= 31.215 12:09:22 STEP 1593 H=1593:00 +CPU= 48.128 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:22:42 STEP 1594 H=1594:00 +CPU= 30.564 12:23:13 STEP 1595 H=1595:00 +CPU= 31.066 12:24:08 STEP 1596 H=1596:00 +CPU= 53.195 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001596 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001596 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001596 12:24:38 STEP 1597 H=1597:00 +CPU= 30.400 12:25:10 STEP 1598 H=1598:00 +CPU= 31.161 12:25:59 STEP 1599 H=1599:00 +CPU= 48.014 12:26:31 STEP 1600 H=1600:00 +CPU= 31.929 12:27:03 STEP 1601 H=1601:00 +CPU= 30.536 12:27:53 STEP 1602 H=1602:00 +CPU= 49.885 12:28:25 STEP 1603 H=1603:00 +CPU= 30.856 12:28:57 STEP 1604 H=1604:00 +CPU= 32.127 12:29:47 STEP 1605 H=1605:00 +CPU= 48.634 12:30:20 STEP 1606 H=1606:00 +CPU= 32.259 12:30:50 STEP 1607 H=1607:00 +CPU= 29.812 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:43:32 STEP 1608 H=1608:00 +CPU= 56.319 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001608 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001608 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001608 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001584 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001584 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001584 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001596 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001596 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001596 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_66.zip Uploading the intermediate file: upload_file_66.zip 12:44:03 STEP 1609 H=1609:00 +CPU= 30.678 12:44:36 STEP 1610 H=1610:00 +CPU= 31.762 12:45:24 STEP 1611 H=1611:00 +CPU= 47.860 12:45:56 STEP 1612 H=1612:00 +CPU= 31.100 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:39:34 STEP 1613 H=1613:00 +CPU= 29.469 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:58:06 STEP 1614 H=1614:00 +CPU= 49.902 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:02:40 STEP 1615 H=1615:00 +CPU= 31.524 14:03:13 STEP 1616 H=1616:00 +CPU= 31.932 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:17:29 STEP 1617 H=1617:00 +CPU= 47.144 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:20:00 STEP 1618 H=1618:00 +CPU= 30.762 14:20:31 STEP 1619 H=1619:00 +CPU= 30.221 14:21:25 STEP 1620 H=1620:00 +CPU= 52.729 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001620 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001620 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001620 14:21:54 STEP 1621 H=1621:00 +CPU= 29.537 14:22:26 STEP 1622 H=1622:00 +CPU= 31.022 14:23:14 STEP 1623 H=1623:00 +CPU= 47.300 14:23:46 STEP 1624 H=1624:00 +CPU= 31.241 14:24:16 STEP 1625 H=1625:00 +CPU= 29.863 14:25:06 STEP 1626 H=1626:00 +CPU= 48.934 14:25:36 STEP 1627 H=1627:00 +CPU= 29.674 14:26:07 STEP 1628 H=1628:00 +CPU= 31.395 14:26:56 STEP 1629 H=1629:00 +CPU= 47.636 14:27:26 STEP 1630 H=1630:00 +CPU= 30.259 14:27:58 STEP 1631 H=1631:00 +CPU= 31.047 14:28:57 STEP 1632 H=1632:00 +CPU= 57.927 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001632 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001632 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001632 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001608 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001608 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001608 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001620 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001620 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001620 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_67.zip Uploading the intermediate file: upload_file_67.zip 14:29:27 STEP 1633 H=1633:00 +CPU= 29.619 14:29:58 STEP 1634 H=1634:00 +CPU= 30.398 14:30:45 STEP 1635 H=1635:00 +CPU= 47.043 14:31:17 STEP 1636 H=1636:00 +CPU= 31.044 14:31:47 STEP 1637 H=1637:00 +CPU= 29.732 14:32:37 STEP 1638 H=1638:00 +CPU= 48.815 14:33:06 STEP 1639 H=1639:00 +CPU= 29.432 14:33:38 STEP 1640 H=1640:00 +CPU= 31.000 14:34:26 STEP 1641 H=1641:00 +CPU= 47.372 14:34:57 STEP 1642 H=1642:00 +CPU= 30.969 14:35:28 STEP 1643 H=1643:00 +CPU= 29.888 14:36:21 STEP 1644 H=1644:00 +CPU= 52.546 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001644 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001644 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001644 14:36:51 STEP 1645 H=1645:00 +CPU= 29.868 14:37:23 STEP 1646 H=1646:00 +CPU= 31.158 14:38:10 STEP 1647 H=1647:00 +CPU= 47.207 14:38:42 STEP 1648 H=1648:00 +CPU= 30.819 14:39:11 STEP 1649 H=1649:00 +CPU= 29.462 14:40:00 STEP 1650 H=1650:00 +CPU= 48.151 14:40:31 STEP 1651 H=1651:00 +CPU= 30.400 14:41:02 STEP 1652 H=1652:00 +CPU= 30.897 14:41:51 STEP 1653 H=1653:00 +CPU= 47.709 14:42:21 STEP 1654 H=1654:00 +CPU= 30.463 14:42:52 STEP 1655 H=1655:00 +CPU= 30.700 14:43:50 STEP 1656 H=1656:00 +CPU= 56.393 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001656 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001656 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001656 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001632 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001632 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001632 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001644 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001644 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001644 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_68.zip Uploading the intermediate file: upload_file_68.zip 14:44:20 STEP 1657 H=1657:00 +CPU= 29.802 14:44:50 STEP 1658 H=1658:00 +CPU= 30.062 14:45:39 STEP 1659 H=1659:00 +CPU= 47.479 14:46:09 STEP 1660 H=1660:00 +CPU= 30.442 14:46:39 STEP 1661 H=1661:00 +CPU= 29.468 14:47:28 STEP 1662 H=1662:00 +CPU= 48.605 14:47:58 STEP 1663 H=1663:00 +CPU= 29.687 14:48:30 STEP 1664 H=1664:00 +CPU= 30.967 14:49:17 STEP 1665 H=1665:00 +CPU= 47.165 14:49:48 STEP 1666 H=1666:00 +CPU= 30.095 14:50:18 STEP 1667 H=1667:00 +CPU= 29.399 14:51:10 STEP 1668 H=1668:00 +CPU= 52.078 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001668 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001668 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001668 14:51:41 STEP 1669 H=1669:00 +CPU= 30.157 14:52:12 STEP 1670 H=1670:00 +CPU= 30.637 14:53:00 STEP 1671 H=1671:00 +CPU= 48.032 14:53:32 STEP 1672 H=1672:00 +CPU= 31.064 14:54:02 STEP 1673 H=1673:00 +CPU= 29.408 14:54:50 STEP 1674 H=1674:00 +CPU= 47.934 14:55:19 STEP 1675 H=1675:00 +CPU= 28.774 14:55:51 STEP 1676 H=1676:00 +CPU= 30.855 14:56:38 STEP 1677 H=1677:00 +CPU= 46.694 14:57:08 STEP 1678 H=1678:00 +CPU= 30.243 14:57:39 STEP 1679 H=1679:00 +CPU= 30.652 14:58:36 STEP 1680 H=1680:00 +CPU= 55.965 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001680 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001680 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001680 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001656 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001656 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001656 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001668 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001668 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001668 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_69.zip Uploading the intermediate file: upload_file_69.zip 14:59:06 STEP 1681 H=1681:00 +CPU= 29.836 14:59:37 STEP 1682 H=1682:00 +CPU= 30.572 15:00:25 STEP 1683 H=1683:00 +CPU= 46.608 15:00:56 STEP 1684 H=1684:00 +CPU= 30.942 15:01:25 STEP 1685 H=1685:00 +CPU= 29.111 15:02:15 STEP 1686 H=1686:00 +CPU= 48.922 15:02:45 STEP 1687 H=1687:00 +CPU= 30.233 15:03:17 STEP 1688 H=1688:00 +CPU= 30.832 15:04:04 STEP 1689 H=1689:00 +CPU= 46.580 15:04:35 STEP 1690 H=1690:00 +CPU= 30.658 15:05:05 STEP 1691 H=1691:00 +CPU= 29.958 15:05:58 STEP 1692 H=1692:00 +CPU= 52.467 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001692 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001692 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001692 15:06:27 STEP 1693 H=1693:00 +CPU= 29.116 15:06:58 STEP 1694 H=1694:00 +CPU= 30.587 15:07:46 STEP 1695 H=1695:00 +CPU= 47.400 15:08:16 STEP 1696 H=1696:00 +CPU= 29.706 15:08:46 STEP 1697 H=1697:00 +CPU= 29.335 15:09:35 STEP 1698 H=1698:00 +CPU= 48.620 15:10:05 STEP 1699 H=1699:00 +CPU= 28.918 15:10:36 STEP 1700 H=1700:00 +CPU= 31.117 15:11:22 STEP 1701 H=1701:00 +CPU= 45.915 15:11:53 STEP 1702 H=1702:00 +CPU= 30.020 15:12:23 STEP 1703 H=1703:00 +CPU= 30.234 15:13:20 STEP 1704 H=1704:00 +CPU= 55.869 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001704 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001704 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001704 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001680 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001680 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001680 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001692 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001692 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001692 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_70.zip Uploading the intermediate file: upload_file_70.zip 15:13:50 STEP 1705 H=1705:00 +CPU= 29.294 15:14:21 STEP 1706 H=1706:00 +CPU= 30.495 15:15:08 STEP 1707 H=1707:00 +CPU= 46.854 15:15:39 STEP 1708 H=1708:00 +CPU= 30.936 15:16:09 STEP 1709 H=1709:00 +CPU= 29.721 15:16:58 STEP 1710 H=1710:00 +CPU= 48.211 15:17:28 STEP 1711 H=1711:00 +CPU= 29.550 15:17:58 STEP 1712 H=1712:00 +CPU= 30.250 15:18:46 STEP 1713 H=1713:00 +CPU= 47.270 15:19:17 STEP 1714 H=1714:00 +CPU= 30.554 15:19:47 STEP 1715 H=1715:00 +CPU= 29.690 15:20:40 STEP 1716 H=1716:00 +CPU= 51.949 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001716 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001716 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001716 15:21:09 STEP 1717 H=1717:00 +CPU= 29.329 15:21:40 STEP 1718 H=1718:00 +CPU= 30.139 15:22:28 STEP 1719 H=1719:00 +CPU= 47.237 15:22:59 STEP 1720 H=1720:00 +CPU= 30.668 15:23:28 STEP 1721 H=1721:00 +CPU= 29.366 15:24:17 STEP 1722 H=1722:00 +CPU= 48.112 15:24:47 STEP 1723 H=1723:00 +CPU= 29.111 15:25:17 STEP 1724 H=1724:00 +CPU= 29.820 15:26:03 STEP 1725 H=1725:00 +CPU= 45.456 15:26:34 STEP 1726 H=1726:00 +CPU= 30.640 15:27:04 STEP 1727 H=1727:00 +CPU= 30.512 15:28:02 STEP 1728 H=1728:00 +CPU= 56.703 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001728 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001728 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001728 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001704 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001704 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001704 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001716 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001716 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001716 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_71.zip Uploading the intermediate file: upload_file_71.zip 15:28:31 STEP 1729 H=1729:00 +CPU= 29.244 15:29:03 STEP 1730 H=1730:00 +CPU= 30.988 15:29:51 STEP 1731 H=1731:00 +CPU= 47.727 15:30:21 STEP 1732 H=1732:00 +CPU= 30.376 15:30:52 STEP 1733 H=1733:00 +CPU= 29.903 15:31:41 STEP 1734 H=1734:00 +CPU= 48.964 15:32:11 STEP 1735 H=1735:00 +CPU= 29.332 15:32:43 STEP 1736 H=1736:00 +CPU= 31.468 15:33:31 STEP 1737 H=1737:00 +CPU= 47.773 15:34:02 STEP 1738 H=1738:00 +CPU= 30.655 15:34:32 STEP 1739 H=1739:00 +CPU= 29.701 15:35:25 STEP 1740 H=1740:00 +CPU= 52.807 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001740 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001740 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001740 15:35:55 STEP 1741 H=1741:00 +CPU= 29.392 15:36:26 STEP 1742 H=1742:00 +CPU= 30.622 15:37:14 STEP 1743 H=1743:00 +CPU= 47.106 15:37:46 STEP 1744 H=1744:00 +CPU= 31.346 15:38:15 STEP 1745 H=1745:00 +CPU= 28.799 15:39:04 STEP 1746 H=1746:00 +CPU= 48.242 15:39:34 STEP 1747 H=1747:00 +CPU= 29.554 15:40:05 STEP 1748 H=1748:00 +CPU= 30.598 15:40:51 STEP 1749 H=1749:00 +CPU= 46.128 15:41:22 STEP 1750 H=1750:00 +CPU= 30.574 15:41:53 STEP 1751 H=1751:00 +CPU= 30.188 15:42:49 STEP 1752 H=1752:00 +CPU= 56.008 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001752 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001752 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001752 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001728 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001728 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001728 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001740 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001740 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001740 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_72.zip Uploading the intermediate file: upload_file_72.zip 15:43:19 STEP 1753 H=1753:00 +CPU= 29.070 15:43:50 STEP 1754 H=1754:00 +CPU= 30.715 15:44:37 STEP 1755 H=1755:00 +CPU= 46.742 15:45:08 STEP 1756 H=1756:00 +CPU= 30.823 15:45:39 STEP 1757 H=1757:00 +CPU= 30.025 15:46:28 STEP 1758 H=1758:00 +CPU= 48.356 15:46:57 STEP 1759 H=1759:00 +CPU= 29.571 15:47:28 STEP 1760 H=1760:00 +CPU= 30.587 15:48:16 STEP 1761 H=1761:00 +CPU= 46.727 15:48:46 STEP 1762 H=1762:00 +CPU= 30.250 15:49:16 STEP 1763 H=1763:00 +CPU= 29.324 15:50:09 STEP 1764 H=1764:00 +CPU= 52.064 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001764 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001764 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001764 15:50:38 STEP 1765 H=1765:00 +CPU= 29.117 15:51:09 STEP 1766 H=1766:00 +CPU= 30.755 15:51:57 STEP 1767 H=1767:00 +CPU= 47.128 15:52:28 STEP 1768 H=1768:00 +CPU= 30.814 15:52:58 STEP 1769 H=1769:00 +CPU= 29.571 15:53:47 STEP 1770 H=1770:00 +CPU= 48.674 15:54:16 STEP 1771 H=1771:00 +CPU= 28.820 15:54:47 STEP 1772 H=1772:00 +CPU= 29.964 15:55:34 STEP 1773 H=1773:00 +CPU= 47.169 15:56:06 STEP 1774 H=1774:00 +CPU= 31.130 15:56:37 STEP 1775 H=1775:00 +CPU= 30.835 15:57:34 STEP 1776 H=1776:00 +CPU= 56.220 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001776 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001776 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001776 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001752 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001752 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001752 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001764 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001764 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001764 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_73.zip Uploading the intermediate file: upload_file_73.zip 15:58:04 STEP 1777 H=1777:00 +CPU= 29.548 15:58:35 STEP 1778 H=1778:00 +CPU= 30.494 15:59:22 STEP 1779 H=1779:00 +CPU= 46.686 15:59:52 STEP 1780 H=1780:00 +CPU= 29.725 16:00:22 STEP 1781 H=1781:00 +CPU= 29.712 16:01:11 STEP 1782 H=1782:00 +CPU= 48.335 16:01:42 STEP 1783 H=1783:00 +CPU= 29.967 16:02:13 STEP 1784 H=1784:00 +CPU= 30.904 16:03:01 STEP 1785 H=1785:00 +CPU= 47.305 16:03:32 STEP 1786 H=1786:00 +CPU= 30.561 16:04:01 STEP 1787 H=1787:00 +CPU= 29.204 16:04:55 STEP 1788 H=1788:00 +CPU= 52.353 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001788 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001788 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001788 16:05:25 STEP 1789 H=1789:00 +CPU= 30.165 16:05:56 STEP 1790 H=1790:00 +CPU= 30.827 16:06:43 STEP 1791 H=1791:00 +CPU= 46.338 16:07:14 STEP 1792 H=1792:00 +CPU= 30.688 16:07:44 STEP 1793 H=1793:00 +CPU= 29.227 16:08:32 STEP 1794 H=1794:00 +CPU= 48.069 16:09:01 STEP 1795 H=1795:00 +CPU= 28.853 16:09:32 STEP 1796 H=1796:00 +CPU= 30.038 16:10:20 STEP 1797 H=1797:00 +CPU= 47.539 16:10:51 STEP 1798 H=1798:00 +CPU= 30.927 16:11:21 STEP 1799 H=1799:00 +CPU= 29.923 16:12:17 STEP 1800 H=1800:00 +CPU= 54.991 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001800 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001800 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001800 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001776 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001776 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001776 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001788 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001788 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001788 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_74.zip Uploading the intermediate file: upload_file_74.zip Uploading trickle at timestep: 6480000 16:12:48 STEP 1801 H=1801:00 +CPU= 29.828 16:13:18 STEP 1802 H=1802:00 +CPU= 30.331 16:14:05 STEP 1803 H=1803:00 +CPU= 46.055 16:14:37 STEP 1804 H=1804:00 +CPU= 31.077 16:15:06 STEP 1805 H=1805:00 +CPU= 28.580 16:15:54 STEP 1806 H=1806:00 +CPU= 47.566 16:16:24 STEP 1807 H=1807:00 +CPU= 29.940 16:16:55 STEP 1808 H=1808:00 +CPU= 30.486 16:17:42 STEP 1809 H=1809:00 +CPU= 47.155 16:18:14 STEP 1810 H=1810:00 +CPU= 31.112 16:18:44 STEP 1811 H=1811:00 +CPU= 29.708 16:19:37 STEP 1812 H=1812:00 +CPU= 52.315 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001812 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001812 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001812 16:20:06 STEP 1813 H=1813:00 +CPU= 29.363 16:20:37 STEP 1814 H=1814:00 +CPU= 30.570 16:21:25 STEP 1815 H=1815:00 +CPU= 47.428 16:21:57 STEP 1816 H=1816:00 +CPU= 30.908 16:22:27 STEP 1817 H=1817:00 +CPU= 29.561 16:23:16 STEP 1818 H=1818:00 +CPU= 48.563 16:23:46 STEP 1819 H=1819:00 +CPU= 29.652 16:24:16 STEP 1820 H=1820:00 +CPU= 30.022 16:25:04 STEP 1821 H=1821:00 +CPU= 47.701 16:25:35 STEP 1822 H=1822:00 +CPU= 30.425 16:26:06 STEP 1823 H=1823:00 +CPU= 30.211 16:27:03 STEP 1824 H=1824:00 +CPU= 56.599 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001824 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001824 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001824 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001800 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001800 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001800 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001812 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001812 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001812 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_75.zip Uploading the intermediate file: upload_file_75.zip 16:27:33 STEP 1825 H=1825:00 +CPU= 29.351 16:28:04 STEP 1826 H=1826:00 +CPU= 30.724 16:28:52 STEP 1827 H=1827:00 +CPU= 47.346 16:29:24 STEP 1828 H=1828:00 +CPU= 31.611 16:29:55 STEP 1829 H=1829:00 +CPU= 30.729 16:30:46 STEP 1830 H=1830:00 +CPU= 49.820 16:31:16 STEP 1831 H=1831:00 +CPU= 30.305 16:31:47 STEP 1832 H=1832:00 +CPU= 30.418 16:32:35 STEP 1833 H=1833:00 +CPU= 47.475 16:33:06 STEP 1834 H=1834:00 +CPU= 30.891 16:33:36 STEP 1835 H=1835:00 +CPU= 29.496 16:34:30 STEP 1836 H=1836:00 +CPU= 53.265 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001836 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001836 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001836 16:34:59 STEP 1837 H=1837:00 +CPU= 29.134 16:35:31 STEP 1838 H=1838:00 +CPU= 30.986 16:36:18 STEP 1839 H=1839:00 +CPU= 47.068 16:36:49 STEP 1840 H=1840:00 +CPU= 30.290 16:37:19 STEP 1841 H=1841:00 +CPU= 29.530 16:38:08 STEP 1842 H=1842:00 +CPU= 48.638 16:38:38 STEP 1843 H=1843:00 +CPU= 29.312 16:39:08 STEP 1844 H=1844:00 +CPU= 30.256 16:39:57 STEP 1845 H=1845:00 +CPU= 47.724 16:40:28 STEP 1846 H=1846:00 +CPU= 30.682 16:40:59 STEP 1847 H=1847:00 +CPU= 30.878 16:41:56 STEP 1848 H=1848:00 +CPU= 56.339 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001848 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001848 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001848 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001824 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001824 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001824 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001836 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001836 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001836 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_76.zip Uploading the intermediate file: upload_file_76.zip 16:42:26 STEP 1849 H=1849:00 +CPU= 29.758 16:42:58 STEP 1850 H=1850:00 +CPU= 31.003 16:43:45 STEP 1851 H=1851:00 +CPU= 46.788 16:44:16 STEP 1852 H=1852:00 +CPU= 30.758 16:44:46 STEP 1853 H=1853:00 +CPU= 29.597 16:45:35 STEP 1854 H=1854:00 +CPU= 48.626 16:46:06 STEP 1855 H=1855:00 +CPU= 30.156 16:46:37 STEP 1856 H=1856:00 +CPU= 30.911 16:47:25 STEP 1857 H=1857:00 +CPU= 47.158 16:47:56 STEP 1858 H=1858:00 +CPU= 31.014 16:48:26 STEP 1859 H=1859:00 +CPU= 29.980 16:49:21 STEP 1860 H=1860:00 +CPU= 53.757 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001860 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001860 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001860 16:49:52 STEP 1861 H=1861:00 +CPU= 30.702 16:50:24 STEP 1862 H=1862:00 +CPU= 31.671 16:51:12 STEP 1863 H=1863:00 +CPU= 47.800 16:51:44 STEP 1864 H=1864:00 +CPU= 31.060 16:52:14 STEP 1865 H=1865:00 +CPU= 29.540 16:53:03 STEP 1866 H=1866:00 +CPU= 48.891 16:53:34 STEP 1867 H=1867:00 +CPU= 30.018 16:54:04 STEP 1868 H=1868:00 +CPU= 29.899 16:54:51 STEP 1869 H=1869:00 +CPU= 46.877 16:55:22 STEP 1870 H=1870:00 +CPU= 30.381 16:55:53 STEP 1871 H=1871:00 +CPU= 31.109 16:56:53 STEP 1872 H=1872:00 +CPU= 58.362 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001872 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001872 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001872 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001848 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001848 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001848 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001860 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001860 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001860 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_77.zip Uploading the intermediate file: upload_file_77.zip 16:57:23 STEP 1873 H=1873:00 +CPU= 29.612 16:57:54 STEP 1874 H=1874:00 +CPU= 30.708 16:58:42 STEP 1875 H=1875:00 +CPU= 48.152 16:59:13 STEP 1876 H=1876:00 +CPU= 30.518 16:59:43 STEP 1877 H=1877:00 +CPU= 29.994 17:00:32 STEP 1878 H=1878:00 +CPU= 48.239 17:01:02 STEP 1879 H=1879:00 +CPU= 29.653 17:01:34 STEP 1880 H=1880:00 +CPU= 31.332 17:02:22 STEP 1881 H=1881:00 +CPU= 47.150 17:02:53 STEP 1882 H=1882:00 +CPU= 30.585 17:03:23 STEP 1883 H=1883:00 +CPU= 29.984 17:04:17 STEP 1884 H=1884:00 +CPU= 53.133 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001884 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001884 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001884 17:04:47 STEP 1885 H=1885:00 +CPU= 30.346 17:05:19 STEP 1886 H=1886:00 +CPU= 31.552 17:06:07 STEP 1887 H=1887:00 +CPU= 47.288 17:06:39 STEP 1888 H=1888:00 +CPU= 31.135 17:07:09 STEP 1889 H=1889:00 +CPU= 29.625 17:07:58 STEP 1890 H=1890:00 +CPU= 49.006 17:08:28 STEP 1891 H=1891:00 +CPU= 29.870 17:08:59 STEP 1892 H=1892:00 +CPU= 30.624 17:09:48 STEP 1893 H=1893:00 +CPU= 48.102 17:10:20 STEP 1894 H=1894:00 +CPU= 31.683 17:10:52 STEP 1895 H=1895:00 +CPU= 31.481 17:11:50 STEP 1896 H=1896:00 +CPU= 57.158 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001896 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001896 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001896 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001872 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001872 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001872 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001884 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001884 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001884 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_78.zip Uploading the intermediate file: upload_file_78.zip 17:12:20 STEP 1897 H=1897:00 +CPU= 29.633 17:12:51 STEP 1898 H=1898:00 +CPU= 30.793 17:13:40 STEP 1899 H=1899:00 +CPU= 48.449 17:14:12 STEP 1900 H=1900:00 +CPU= 31.433 17:14:43 STEP 1901 H=1901:00 +CPU= 31.104 17:15:34 STEP 1902 H=1902:00 +CPU= 49.908 17:16:04 STEP 1903 H=1903:00 +CPU= 29.896 17:16:36 STEP 1904 H=1904:00 +CPU= 31.339 17:17:24 STEP 1905 H=1905:00 +CPU= 47.867 17:17:56 STEP 1906 H=1906:00 +CPU= 31.070 17:18:26 STEP 1907 H=1907:00 +CPU= 30.173 17:19:20 STEP 1908 H=1908:00 +CPU= 53.222 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001908 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001908 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001908 17:19:51 STEP 1909 H=1909:00 +CPU= 30.789 17:20:21 STEP 1910 H=1910:00 +CPU= 29.884 17:21:11 STEP 1911 H=1911:00 +CPU= 48.443 17:21:42 STEP 1912 H=1912:00 +CPU= 31.279 17:22:12 STEP 1913 H=1913:00 +CPU= 29.621 17:23:02 STEP 1914 H=1914:00 +CPU= 49.181 17:23:32 STEP 1915 H=1915:00 +CPU= 30.331 17:24:04 STEP 1916 H=1916:00 +CPU= 31.251 17:24:52 STEP 1917 H=1917:00 +CPU= 47.635 17:25:25 STEP 1918 H=1918:00 +CPU= 32.388 17:25:56 STEP 1919 H=1919:00 +CPU= 30.985 17:26:54 STEP 1920 H=1920:00 +CPU= 57.046 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+001920 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+001920 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+001920 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001896 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001896 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001896 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMGGhq0f+001908 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMSHhq0f+001908 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184948/ICMUAhq0f+001908 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0304_1999050100_123_968_12184948_0_r212459995_79.zip Uploading the intermediate file: upload_file_79.zip 17:27:25 STEP 1921 H=1921:00 +CPU= 30.285 double free or corruption (out) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:26:46 | 1478188 | 22270226 | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 | 6,480,000 | 62,253 | 0.0096 |
09 May 2023 16:24:58 | 1478188 | 22270226 | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 | 5,356,800 | 50,471 | 0.0094 |
09 May 2023 16:24:45 | 1478188 | 22270226 | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 | 4,492,800 | 42,012 | 0.0094 |
09 May 2023 16:24:35 | 1478188 | 22270226 | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 | 3,628,800 | 34,066 | 0.0094 |
09 May 2023 16:23:51 | 1478188 | 22270226 | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 | 2,678,400 | 24,433 | 0.0091 |
09 May 2023 16:22:42 | 1478188 | 22270226 | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 | 1,814,400 | 16,425 | 0.0091 |
09 May 2023 16:22:31 | 1478188 | 22270226 | oifs_43r3_ps_0304_1999050100_123_968_12184948_0 | 950,400 | 8,373 | 0.0088 |
©2024 cpdn.org