climateprediction.net home page
Task 15904558

Task 15904558

Name hadcm3n_o1x0_1940_40_008404533_0
Workunit 8555389
Created 23 Jul 2013, 19:06:11 UTC
Sent 24 Jul 2013, 12:51:55 UTC
Report deadline 23 Oct 2013, 20:19:06 UTC
Received 1 Oct 2013, 14:49:16 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1145695
Run time 17 days 23 hours 31 min 40 sec
CPU time 16 days 10 hours 23 min 36 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.32 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>7.0.64</core_client_version>
<![CDATA[
<message>
The device does not recognize the command.
 (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6616, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4888, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5852, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9328, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3328, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5224, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5488, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6124, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4656, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6052, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5924, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5028, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3984, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4740, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1376, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4864, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4412, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3760, iMonCtr=1
Model crash detected, will try to restart...
CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5948, iMonCtr=1
Model crash detected, will try to restart...
06:29:10 (3520): No heartbeat from core client for 30 sec - exiting
06:29:11 (3520): No heartbeat from core client for 30 sec - exiting
06:29:12 (3520): No heartbeat from core client for 30 sec - exiting
06:29:13 (3520): No heartbeat from core client for 30 sec - exiting
06:29:15 (3520): No heartbeat from core client for 30 sec - exiting
06:29:16 (3520): No heartbeat from core client for 30 sec - exiting
06:29:17 (3520): No heartbeat from core client for 30 sec - exiting
06:29:18 (3520): No heartbeat from core client for 30 sec - exiting
06:29:19 (3520): No heartbeat from core client for 30 sec - exiting
06:29:20 (3520): No heartbeat from core client for 30 sec - exiting
06:29:21 (3520): No heartbeat from core client for 30 sec - exiting
06:29:22 (3520): No heartbeat from core client for 30 sec - exiting
06:29:23 (3520): No heartbeat from core client for 30 sec - exiting
06:29:24 (3520): No heartbeat from core client for 30 sec - exiting
06:29:26 (3520): No heartbeat from core client for 30 sec - exiting
06:29:27 (3520): No heartbeat from core client for 30 sec - exiting
06:29:28 (3520): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:51:49 (7236): No heartbeat from core client for 30 sec - exiting
06:51:50 (7236): No heartbeat from core client for 30 sec - exiting
06:51:51 (7236): No heartbeat from core client for 30 sec - exiting
06:51:52 (7236): No heartbeat from core client for 30 sec - exiting
06:51:54 (7236): No heartbeat from core client for 30 sec - exiting
06:51:55 (7236): No heartbeat from core client for 30 sec - exiting
06:51:56 (7236): No heartbeat from core client for 30 sec - exiting
06:51:57 (7236): No heartbeat from core client for 30 sec - exiting
06:51:58 (7236): No heartbeat from core client for 30 sec - exiting
06:51:59 (7236): No heartbeat from core client for 30 sec - exiting
06:52:00 (7236): No heartbeat from core client for 30 sec - exiting
06:52:01 (7236): No heartbeat from core client for 30 sec - exiting
06:52:02 (7236): No heartbeat from core client for 30 sec - exiting
06:52:03 (7236): No heartbeat from core client for 30 sec - exiting
06:52:04 (7236): No heartbeat from core client for 30 sec - exiting
06:52:06 (7236): No heartbeat from core client for 30 sec - exiting
06:52:07 (7236): No heartbeat from core client for 30 sec - exiting
06:52:08 (7236): No heartbeat from core client for 30 sec - exiting
06:52:09 (7236): No heartbeat from core client for 30 sec - exiting
06:52:10 (7236): No heartbeat from core client for 30 sec - exiting
06:52:11 (7236): No heartbeat from core client for 30 sec - exiting
06:52:12 (7236): No heartbeat from core client for 30 sec - exiting
06:52:13 (7236): No heartbeat from core client for 30 sec - exiting
06:52:14 (7236): No heartbeat from core client for 30 sec - exiting
06:52:15 (7236): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:46:58 (5284): No heartbeat from core client for 30 sec - exiting
07:46:59 (5284): No heartbeat from core client for 30 sec - exiting
07:47:00 (5284): No heartbeat from core client for 30 sec - exiting
07:47:01 (5284): No heartbeat from core client for 30 sec - exiting
07:47:02 (5284): No heartbeat from core client for 30 sec - exiting
07:47:03 (5284): No heartbeat from core client for 30 sec - exiting
07:47:04 (5284): No heartbeat from core client for 30 sec - exiting
07:47:06 (5284): No heartbeat from core client for 30 sec - exiting
07:47:07 (5284): No heartbeat from core client for 30 sec - exiting
07:47:08 (5284): No heartbeat from core client for 30 sec - exiting
07:47:09 (5284): No heartbeat from core client for 30 sec - exiting
07:47:10 (5284): No heartbeat from core client for 30 sec - exiting
07:47:11 (5284): No heartbeat from core client for 30 sec - exiting
07:47:12 (5284): No heartbeat from core client for 30 sec - exiting
07:47:13 (5284): No heartbeat from core client for 30 sec - exiting
07:47:14 (5284): No heartbeat from core client for 30 sec - exiting
07:47:15 (5284): No heartbeat from core client for 30 sec - exiting
07:47:16 (5284): No heartbeat from core client for 30 sec - exiting
07:47:17 (5284): No heartbeat from core client for 30 sec - exiting
07:47:19 (5284): No heartbeat from core client for 30 sec - exiting
07:47:20 (5284): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:00:05 (6072): No heartbeat from core client for 30 sec - exiting
08:00:06 (6072): No heartbeat from core client for 30 sec - exiting
08:00:07 (6072): No heartbeat from core client for 30 sec - exiting
08:00:08 (6072): No heartbeat from core client for 30 sec - exiting
08:00:09 (6072): No heartbeat from core client for 30 sec - exiting
08:00:10 (6072): No heartbeat from core client for 30 sec - exiting
08:00:11 (6072): No heartbeat from core client for 30 sec - exiting
08:00:13 (6072): No heartbeat from core client for 30 sec - exiting
08:00:14 (6072): No heartbeat from core client for 30 sec - exiting
08:00:15 (6072): No heartbeat from core client for 30 sec - exiting
08:00:16 (6072): No heartbeat from core client for 30 sec - exiting
08:00:17 (6072): No heartbeat from core client for 30 sec - exiting
08:00:18 (6072): No heartbeat from core client for 30 sec - exiting
08:00:19 (6072): No heartbeat from core client for 30 sec - exiting
08:00:20 (6072): No heartbeat from core client for 30 sec - exiting
08:00:21 (6072): No heartbeat from core client for 30 sec - exiting
08:00:22 (6072): No heartbeat from core client for 30 sec - exiting
08:00:23 (6072): No heartbeat from core client for 30 sec - exiting
08:00:24 (6072): No heartbeat from core client for 30 sec - exiting
08:00:26 (6072): No heartbeat from core client for 30 sec - exiting
08:00:27 (6072): No heartbeat from core client for 30 sec - exiting
08:00:28 (6072): No heartbeat from core client for 30 sec - exiting
08:00:29 (6072): No heartbeat from core client for 30 sec - exiting
08:00:30 (6072): No heartbeat from core client for 30 sec - exiting
08:00:31 (6072): No heartbeat from core client for 30 sec - exiting
08:00:32 (6072): No heartbeat from core client for 30 sec - exiting
08:00:33 (6072): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
11:44:28 (4976): No heartbeat from core client for 30 sec - exiting
11:44:29 (4976): No heartbeat from core client for 30 sec - exiting
11:44:30 (4976): No heartbeat from core client for 30 sec - exiting
11:44:31 (4976): No heartbeat from core client for 30 sec - exiting
11:44:32 (4976): No heartbeat from core client for 30 sec - exiting
11:44:33 (4976): No heartbeat from core client for 30 sec - exiting
11:44:35 (4976): No heartbeat from core client for 30 sec - exiting
11:44:36 (4976): No heartbeat from core client for 30 sec - exiting
11:44:37 (4976): No heartbeat from core client for 30 sec - exiting
11:44:38 (4976): No heartbeat from core client for 30 sec - exiting
11:44:39 (4976): No heartbeat from core client for 30 sec - exiting
11:44:40 (4976): No heartbeat from core client for 30 sec - exiting
11:44:41 (4976): No heartbeat from core client for 30 sec - exiting
11:44:42 (4976): No heartbeat from core client for 30 sec - exiting
11:44:43 (4976): No heartbeat from core client for 30 sec - exiting
11:44:44 (4976): No heartbeat from core client for 30 sec - exiting
11:44:46 (4976): No heartbeat from core client for 30 sec - exiting
11:44:47 (4976): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:53:08 (3592): No heartbeat from core client for 30 sec - exiting
12:53:10 (3592): No heartbeat from core client for 30 sec - exiting
12:53:11 (3592): No heartbeat from core client for 30 sec - exiting
12:53:12 (3592): No heartbeat from core client for 30 sec - exiting
12:53:13 (3592): No heartbeat from core client for 30 sec - exiting
12:53:14 (3592): No heartbeat from core client for 30 sec - exiting
12:53:15 (3592): No heartbeat from core client for 30 sec - exiting
12:53:16 (3592): No heartbeat from core client for 30 sec - exiting
12:53:17 (3592): No heartbeat from core client for 30 sec - exiting
12:53:18 (3592): No heartbeat from core client for 30 sec - exiting
12:53:19 (3592): No heartbeat from core client for 30 sec - exiting
12:53:20 (3592): No heartbeat from core client for 30 sec - exiting
12:53:22 (3592): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:56:50 (5128): No heartbeat from core client for 30 sec - exiting
13:56:51 (5128): No heartbeat from core client for 30 sec - exiting
13:56:52 (5128): No heartbeat from core client for 30 sec - exiting
13:56:53 (5128): No heartbeat from core client for 30 sec - exiting
13:56:55 (5128): No heartbeat from core client for 30 sec - exiting
13:56:56 (5128): No heartbeat from core client for 30 sec - exiting
13:56:57 (5128): No heartbeat from core client for 30 sec - exiting
13:56:58 (5128): No heartbeat from core client for 30 sec - exiting
13:56:59 (5128): No heartbeat from core client for 30 sec - exiting
13:57:00 (5128): No heartbeat from core client for 30 sec - exiting
13:57:01 (5128): No heartbeat from core client for 30 sec - exiting
13:57:02 (5128): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:19:21 (5028): No heartbeat from core client for 30 sec - exiting
08:19:22 (5028): No heartbeat from core client for 30 sec - exiting
08:19:23 (5028): No heartbeat from core client for 30 sec - exiting
08:19:24 (5028): No heartbeat from core client for 30 sec - exiting
08:19:25 (5028): No heartbeat from core client for 30 sec - exiting
08:19:26 (5028): No heartbeat from core client for 30 sec - exiting
08:19:27 (5028): No heartbeat from core client for 30 sec - exiting
08:19:29 (5028): No heartbeat from core client for 30 sec - exiting
08:19:30 (5028): No heartbeat from core client for 30 sec - exiting
08:19:31 (5028): No heartbeat from core client for 30 sec - exiting
08:19:32 (5028): No heartbeat from core client for 30 sec - exiting
08:19:33 (5028): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:55:01 (2236): No heartbeat from core client for 30 sec - exiting
11:55:02 (2236): No heartbeat from core client for 30 sec - exiting
11:55:03 (2236): No heartbeat from core client for 30 sec - exiting
11:55:05 (2236): No heartbeat from core client for 30 sec - exiting
11:55:06 (2236): No heartbeat from core client for 30 sec - exiting
11:55:07 (2236): No heartbeat from core client for 30 sec - exiting
11:55:08 (2236): No heartbeat from core client for 30 sec - exiting
11:55:09 (2236): No heartbeat from core client for 30 sec - exiting
11:55:10 (2236): No heartbeat from core client for 30 sec - exiting
11:55:11 (2236): No heartbeat from core client for 30 sec - exiting
11:55:12 (2236): No heartbeat from core client for 30 sec - exiting
11:55:13 (2236): No heartbeat from core client for 30 sec - exiting
11:55:14 (2236): No heartbeat from core client for 30 sec - exiting
11:55:16 (2236): No heartbeat from core client for 30 sec - exiting
11:55:17 (2236): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9192, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9192, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9192, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9192, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9192, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9192, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 Sep 2013 17:16:15 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 518,400 1,377,069 2.6564
13 Sep 2013 07:35:47 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 492,480 1,310,095 2.6602
11 Sep 2013 13:02:10 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 466,560 1,241,264 2.6605
09 Sep 2013 16:07:35 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 440,640 1,173,090 2.6622
07 Sep 2013 17:41:44 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 414,720 1,104,638 2.6636
05 Sep 2013 18:26:51 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 388,800 1,035,073 2.6622
02 Sep 2013 21:40:51 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 362,880 964,422 2.6577
01 Sep 2013 14:36:34 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 336,960 895,974 2.6590
29 Aug 2013 15:53:14 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 311,040 827,955 2.6619
26 Aug 2013 16:00:57 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 285,120 758,863 2.6616
24 Aug 2013 16:44:58 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 259,200 691,505 2.6678
23 Aug 2013 12:47:02 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 233,280 622,050 2.6665
20 Aug 2013 18:11:01 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 207,360 552,788 2.6658
14 Aug 2013 15:29:27 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 181,440 484,169 2.6685
14 Aug 2013 15:29:27 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 155,520 416,071 2.6754
14 Aug 2013 15:29:27 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 129,600 347,223 2.6792
14 Aug 2013 15:29:27 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 103,680 279,706 2.6978
14 Aug 2013 15:29:27 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 77,760 211,134 2.7152
14 Aug 2013 15:29:27 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 51,840 142,280 2.7446
26 Jul 2013 13:32:34 1145695 15904558 hadcm3n_o1x0_1940_40_008404533_0 25,920 71,817 2.7707


©2024 cpdn.org