climateprediction.net home page
Task 16150788

Task 16150788

Name hadcm3n_o16m_1980_40_008386045_4
Workunit 8536904
Created 20 Dec 2013, 9:31:26 UTC
Sent 20 Dec 2013, 9:31:36 UTC
Report deadline 21 Mar 2014, 16:58:47 UTC
Received 24 Jan 2014, 21:53:22 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1235271
Run time 10 days 21 hours 49 min 49 sec
CPU time 9 days 12 hours 50 min 13 sec
Validate state Invalid
Credit 10,886.40
Device peak FLOPS 3.12 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>7.2.33</core_client_version>
<![CDATA[
<message>
The device does not recognize the command.
 (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
00:20:55 (1140): No heartbeat from core client for 30 sec - exiting
00:20:56 (1140): No heartbeat from core client for 30 sec - exiting
00:20:57 (1140): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:20:58 (1140): No heartbeat from core client for 30 sec - exiting
00:20:59 (1140): No heartbeat from core client for 30 sec - exiting
00:21:00 (1140): No heartbeat from core client for 30 sec - exiting
00:21:01 (1140): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3924, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3436, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
15:59:49 (3240): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:08:13 (2132): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:08:14 (2132): No heartbeat from core client for 30 sec - exiting
22:08:15 (2132): No heartbeat from core client for 30 sec - exiting
22:08:16 (2132): No heartbeat from core client for 30 sec - exiting
22:08:17 (2132): No heartbeat from core client for 30 sec - exiting
22:08:19 (2132): No heartbeat from core client for 30 sec - exiting
22:08:20 (2132): No heartbeat from core client for 30 sec - exiting
22:08:21 (2132): No heartbeat from core client for 30 sec - exiting
22:08:22 (2132): No heartbeat from core client for 30 sec - exiting
22:08:23 (2132): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:27:23 (1156): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
15:27:24 (1156): No heartbeat from core client for 30 sec - exiting
15:27:25 (1156): No heartbeat from core client for 30 sec - exiting
15:27:26 (1156): No heartbeat from core client for 30 sec - exiting
15:27:28 (1156): No heartbeat from core client for 30 sec - exiting
15:27:29 (1156): No heartbeat from core client for 30 sec - exiting
15:27:30 (1156): No heartbeat from core client for 30 sec - exiting
15:27:31 (1156): No heartbeat from core client for 30 sec - exiting
15:27:32 (1156): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3904, iMonCtr=1
Model crash detected, will try to restart...
17:23:21 (4792): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:38:42 (3424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:38:43 (3424): No heartbeat from core client for 30 sec - exiting
14:38:44 (3424): No heartbeat from core client for 30 sec - exiting
14:38:45 (3424): No heartbeat from core client for 30 sec - exiting
14:38:46 (3424): No heartbeat from core client for 30 sec - exiting
14:38:48 (3424): No heartbeat from core client for 30 sec - exiting
14:38:49 (3424): No heartbeat from core client for 30 sec - exiting
14:38:50 (3424): No heartbeat from core client for 30 sec - exiting
14:38:51 (3424): No heartbeat from core client for 30 sec - exiting
14:38:52 (3424): No heartbeat from core client for 30 sec - exiting
14:38:53 (3424): No heartbeat from core client for 30 sec - exiting
14:38:54 (3424): No heartbeat from core client for 30 sec - exiting
14:38:55 (3424): No heartbeat from core client for 30 sec - exiting
14:38:56 (3424): No heartbeat from core client for 30 sec - exiting
14:38:57 (3424): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3376, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3224, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1088, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4612, iMonCtr=1
Model crash detected, will try to restart...
13:32:12 (3152): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:32:13 (3152): No heartbeat from core client for 30 sec - exiting
13:32:14 (3152): No heartbeat from core client for 30 sec - exiting
13:32:15 (3152): No heartbeat from core client for 30 sec - exiting
13:32:16 (3152): No heartbeat from core client for 30 sec - exiting
13:32:17 (3152): No heartbeat from core client for 30 sec - exiting
13:32:18 (3152): No heartbeat from core client for 30 sec - exiting
13:32:19 (3152): No heartbeat from core client for 30 sec - exiting
13:32:21 (3152): No heartbeat from core client for 30 sec - exiting
13:32:22 (3152): No heartbeat from core client for 30 sec - exiting
13:32:23 (3152): No heartbeat from core client for 30 sec - exiting
13:32:24 (3152): No heartbeat from core client for 30 sec - exiting
13:32:25 (3152): No heartbeat from core client for 30 sec - exiting
13:32:26 (3152): No heartbeat from core client for 30 sec - exiting
13:32:27 (3152): No heartbeat from core client for 30 sec - exiting
13:32:28 (3152): No heartbeat from core client for 30 sec - exiting
13:32:29 (3152): No heartbeat from core client for 30 sec - exiting
13:32:30 (3152): No heartbeat from core client for 30 sec - exiting
13:32:32 (3152): No heartbeat from core client for 30 sec - exiting
13:32:33 (3152): No heartbeat from core client for 30 sec - exiting
13:32:35 (3152): No heartbeat from core client for 30 sec - exiting
13:32:36 (3152): No heartbeat from core client for 30 sec - exiting
13:32:37 (3152): No heartbeat from core client for 30 sec - exiting
13:32:38 (3152): No heartbeat from core client for 30 sec - exiting
13:32:40 (3152): No heartbeat from core client for 30 sec - exiting
13:32:41 (3152): No heartbeat from core client for 30 sec - exiting
13:32:42 (3152): No heartbeat from core client for 30 sec - exiting
13:32:43 (3152): No heartbeat from core client for 30 sec - exiting
13:32:44 (3152): No heartbeat from core client for 30 sec - exiting
13:32:45 (3152): No heartbeat from core client for 30 sec - exiting
13:32:46 (3152): No heartbeat from core client for 30 sec - exiting
13:32:47 (3152): No heartbeat from core client for 30 sec - exiting
13:32:49 (3152): No heartbeat from core client for 30 sec - exiting
13:32:50 (3152): No heartbeat from core client for 30 sec - exiting
13:32:51 (3152): No heartbeat from core client for 30 sec - exiting
13:32:52 (3152): No heartbeat from core client for 30 sec - exiting
13:32:53 (3152): No heartbeat from core client for 30 sec - exiting
13:32:54 (3152): No heartbeat from core client for 30 sec - exiting
13:32:56 (3152): No heartbeat from core client for 30 sec - exiting
13:32:57 (3152): No heartbeat from core client for 30 sec - exiting
13:32:58 (3152): No heartbeat from core client for 30 sec - exiting
13:32:59 (3152): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
13:56:16 (3288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:56:17 (3288): No heartbeat from core client for 30 sec - exiting
13:56:18 (3288): No heartbeat from core client for 30 sec - exiting
13:56:19 (3288): No heartbeat from core client for 30 sec - exiting
13:56:20 (3288): No heartbeat from core client for 30 sec - exiting
13:56:21 (3288): No heartbeat from core client for 30 sec - exiting
13:56:23 (3288): No heartbeat from core client for 30 sec - exiting
13:56:24 (3288): No heartbeat from core client for 30 sec - exiting
13:56:25 (3288): No heartbeat from core client for 30 sec - exiting
13:56:26 (3288): No heartbeat from core client for 30 sec - exiting
13:56:27 (3288): No heartbeat from core client for 30 sec - exiting
13:56:28 (3288): No heartbeat from core client for 30 sec - exiting
13:56:29 (3288): No heartbeat from core client for 30 sec - exiting
13:56:30 (3288): No heartbeat from core client for 30 sec - exiting
13:56:31 (3288): No heartbeat from core client for 30 sec - exiting
13:56:32 (3288): No heartbeat from core client for 30 sec - exiting
13:56:33 (3288): No heartbeat from core client for 30 sec - exiting
13:56:35 (3288): No heartbeat from core client for 30 sec - exiting
13:56:36 (3288): No heartbeat from core client for 30 sec - exiting
13:56:37 (3288): No heartbeat from core client for 30 sec - exiting
13:56:38 (3288): No heartbeat from core client for 30 sec - exiting
13:56:39 (3288): No heartbeat from core client for 30 sec - exiting
13:56:40 (3288): No heartbeat from core client for 30 sec - exiting
13:56:41 (3288): No heartbeat from core client for 30 sec - exiting
13:56:42 (3288): No heartbeat from core client for 30 sec - exiting
13:56:43 (3288): No heartbeat from core client for 30 sec - exiting
13:56:44 (3288): No heartbeat from core client for 30 sec - exiting
13:56:45 (3288): No heartbeat from core client for 30 sec - exiting
13:56:47 (3288): No heartbeat from core client for 30 sec - exiting
13:56:48 (3288): No heartbeat from core client for 30 sec - exiting
13:56:49 (3288): No heartbeat from core client for 30 sec - exiting
13:56:50 (3288): No heartbeat from core client for 30 sec - exiting
13:56:51 (3288): No heartbeat from core client for 30 sec - exiting
13:56:52 (3288): No heartbeat from core client for 30 sec - exiting
13:56:53 (3288): No heartbeat from core client for 30 sec - exiting
13:56:54 (3288): No heartbeat from core client for 30 sec - exiting
13:56:55 (3288): No heartbeat from core client for 30 sec - exiting
13:56:57 (3288): No heartbeat from core client for 30 sec - exiting
13:56:58 (3288): No heartbeat from core client for 30 sec - exiting
13:56:59 (3288): No heartbeat from core client for 30 sec - exiting
13:57:00 (3288): No heartbeat from core client for 30 sec - exiting
13:57:01 (3288): No heartbeat from core client for 30 sec - exiting
13:57:02 (3288): No heartbeat from core client for 30 sec - exiting
13:57:03 (3288): No heartbeat from core client for 30 sec - exiting
13:57:04 (3288): No heartbeat from core client for 30 sec - exiting
13:57:05 (3288): No heartbeat from core client for 30 sec - exiting
13:57:06 (3288): No heartbeat from core client for 30 sec - exiting
13:57:07 (3288): No heartbeat from core client for 30 sec - exiting
13:57:09 (3288): No heartbeat from core client for 30 sec - exiting
13:57:10 (3288): No heartbeat from core client for 30 sec - exiting
13:57:11 (3288): No heartbeat from core client for 30 sec - exiting
13:57:12 (3288): No heartbeat from core client for 30 sec - exiting
13:57:13 (3288): No heartbeat from core client for 30 sec - exiting
13:57:14 (3288): No heartbeat from core client for 30 sec - exiting
13:57:15 (3288): No heartbeat from core client for 30 sec - exiting
13:57:16 (3288): No heartbeat from core client for 30 sec - exiting
13:57:17 (3288): No heartbeat from core client for 30 sec - exiting
13:57:19 (3288): No heartbeat from core client for 30 sec - exiting
13:57:20 (3288): No heartbeat from core client for 30 sec - exiting
13:57:21 (3288): No heartbeat from core client for 30 sec - exiting
13:57:22 (3288): No heartbeat from core client for 30 sec - exiting
13:57:23 (3288): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5780, iMonCtr=1
Model crash detected, will try to restart...
13:42:38 (1020): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:42:39 (1020): No heartbeat from core client for 30 sec - exiting
13:42:40 (1020): No heartbeat from core client for 30 sec - exiting
13:42:41 (1020): No heartbeat from core client for 30 sec - exiting
13:42:42 (1020): No heartbeat from core client for 30 sec - exiting
13:42:44 (1020): No heartbeat from core client for 30 sec - exiting
13:42:45 (1020): No heartbeat from core client for 30 sec - exiting
13:42:48 (1020): No heartbeat from core client for 30 sec - exiting
13:42:49 (1020): No heartbeat from core client for 30 sec - exiting
13:42:50 (1020): No heartbeat from core client for 30 sec - exiting
13:42:51 (1020): No heartbeat from core client for 30 sec - exiting
13:42:52 (1020): No heartbeat from core client for 30 sec - exiting
13:42:53 (1020): No heartbeat from core client for 30 sec - exiting
13:42:54 (1020): No heartbeat from core client for 30 sec - exiting
13:42:55 (1020): No heartbeat from core client for 30 sec - exiting
13:42:56 (1020): No heartbeat from core client for 30 sec - exiting
13:42:57 (1020): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4352, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:37:17 (4452): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4584, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4200, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3536, iMonCtr=1
Model crash detected, will try to restart...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4564, selfPID=4564, iMonCtr=1
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3324, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=120, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=120, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=120, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=120, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=120, iMonCtr=1
Model crash detected, will try to restart...
19:14:09 (4028): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5256, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
24 Jan 2014 10:40:11 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 907,200 804,312 0.8866
24 Jan 2014 10:40:11 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 881,280 781,760 0.8871
22 Jan 2014 10:58:51 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 855,360 758,418 0.8867
20 Jan 2014 18:36:41 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 829,440 735,107 0.8863
20 Jan 2014 11:30:53 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 803,520 711,643 0.8857
15 Jan 2014 22:41:01 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 777,600 688,239 0.8851
14 Jan 2014 15:12:52 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 751,680 666,245 0.8863
13 Jan 2014 00:16:00 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 725,760 645,566 0.8895
12 Jan 2014 16:07:41 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 699,840 620,817 0.8871
12 Jan 2014 07:13:06 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 673,920 596,372 0.8849
11 Jan 2014 23:16:10 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 648,000 571,989 0.8827
11 Jan 2014 15:39:37 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 622,080 547,867 0.8807
11 Jan 2014 08:02:27 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 596,160 523,324 0.8778
11 Jan 2014 00:34:20 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 570,240 499,080 0.8752
10 Jan 2014 17:10:48 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 544,320 474,934 0.8725
10 Jan 2014 01:10:06 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 518,400 453,434 0.8747
09 Jan 2014 17:31:50 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 492,480 430,469 0.8741
08 Jan 2014 20:23:26 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 466,560 405,950 0.8701
08 Jan 2014 13:37:26 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 440,640 383,074 0.8694
05 Jan 2014 21:05:19 1235271 16150788 hadcm3n_o16m_1980_40_008386045_4 414,720 359,899 0.8678


©2024 cpdn.org