Task 13805285

Name	hadcm3n_t6k6_1940_40_007616679_2
Workunit	7794809
Created	22 Dec 2011, 0:48:32 UTC
Sent	22 Dec 2011, 1:06:08 UTC
Report deadline	22 Mar 2012, 8:33:19 UTC
Received	30 Dec 2011, 19:36:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1176428
Run time	5 days 8 hours 57 min 40 sec
CPU time	4 days 8 hours 59 min 33 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	2.27 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:02:55 (3608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:02:56 (3608): No heartbeat from core client for 30 sec - exiting 16:04:57 (1008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:29:47 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:36:35 (3872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:37:59 (6172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:38 (8816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:13:57 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:36:26 (4204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:19:42 (3804): No heartbeat from core client for 30 sec - exiting 20:19:43 (3804): No heartbeat from core client for 30 sec - exiting 20:19:44 (3804): No heartbeat from core client for 30 sec - exiting 20:19:45 (3804): No heartbeat from core client for 30 sec - exiting 20:19:46 (3804): No heartbeat from core client for 30 sec - exiting 20:19:47 (3804): No heartbeat from core client for 30 sec - exiting 20:19:48 (3804): No heartbeat from core client for 30 sec - exiting 20:19:50 (3804): No heartbeat from core client for 30 sec - exiting 20:19:51 (3804): No heartbeat from core client for 30 sec - exiting 20:19:52 (3804): No heartbeat from core client for 30 sec - exiting 20:19:53 (3804): No heartbeat from core client for 30 sec - exiting 20:19:54 (3804): No heartbeat from core client for 30 sec - exiting 20:19:55 (3804): No heartbeat from core client for 30 sec - exiting 20:19:56 (3804): No heartbeat from core client for 30 sec - exiting 20:19:57 (3804): No heartbeat from core client for 30 sec - exiting 20:19:58 (3804): No heartbeat from core client for 30 sec - exiting 20:19:59 (3804): No heartbeat from core client for 30 sec - exiting 20:20:00 (3804): No heartbeat from core client for 30 sec - exiting 20:20:02 (3804): No heartbeat from core client for 30 sec - exiting 20:20:03 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:20:04 (3804): No heartbeat from core client for 30 sec - exiting 20:36:00 (4072): No heartbeat from core client for 30 sec - exiting 20:36:01 (4072): No heartbeat from core client for 30 sec - exiting 20:36:02 (4072): No heartbeat from core client for 30 sec - exiting 20:36:03 (4072): No heartbeat from core client for 30 sec - exiting 20:36:04 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:36:05 (4072): No heartbeat from core client for 30 sec - exiting 20:36:45 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:26 (4908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:43:45 (1764): No heartbeat from core client for 30 sec - exiting 21:43:46 (1764): No heartbeat from core client for 30 sec - exiting 21:43:47 (1764): No heartbeat from core client for 30 sec - exiting 21:43:48 (1764): No heartbeat from core client for 30 sec - exiting 21:43:49 (1764): No heartbeat from core client for 30 sec - exiting 21:43:50 (1764): No heartbeat from core client for 30 sec - exiting 21:43:51 (1764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Suspended CPDN Monitor - Suspend request from BOINC... 16:55:30 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 19:55:56 (2592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 19:57:27 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 20:00:15 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:16 (4452): No heartbeat from core client for 30 sec - exiting 20:00:17 (4452): No heartbeat from core client for 30 sec - exiting 20:00:18 (4452): No heartbeat from core client for 30 sec - exiting 20:00:19 (4452): No heartbeat from core client for 30 sec - exiting 20:00:20 (4452): No heartbeat from core client for 30 sec - exiting 20:00:21 (4452): No heartbeat from core client for 30 sec - exiting 20:00:22 (4452): No heartbeat from core client for 30 sec - exiting 20:00:23 (4452): No heartbeat from core client for 30 sec - exiting 20:00:24 (4452): No heartbeat from core client for 30 sec - exiting 20:00:25 (4452): No heartbeat from core client for 30 sec - exiting BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Dec 2011 08:43:53	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	207,360	347,700	1.6768
28 Dec 2011 07:24:26	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	181,440	303,797	1.6744
27 Dec 2011 07:17:16	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	155,520	259,681	1.6698
26 Dec 2011 11:50:46	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	129,600	216,579	1.6711
25 Dec 2011 15:25:14	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	103,680	173,773	1.6761
24 Dec 2011 14:46:04	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	77,760	130,391	1.6768
23 Dec 2011 14:32:30	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	51,840	87,843	1.6945
22 Dec 2011 18:19:25	1176428	13805285	hadcm3n_t6k6_1940_40_007616679_2	25,920	43,475	1.6773