Task 15289004

Name	hadcm3n_zhk1_1880_40_008199509_2
Workunit	8354633
Created	16 Sep 2012, 18:40:12 UTC
Sent	16 Sep 2012, 18:40:42 UTC
Report deadline	17 Dec 2012, 2:07:53 UTC
Received	17 Sep 2012, 20:43:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1220576
Run time	23 hours 55 min 55 sec
CPU time	4 hours 45 min 20 sec
Validate state	Invalid
Credit	1,244.16
Device peak FLOPS	3.39 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:48:07 (7328): No heartbeat from core client for 30 sec - exiting 09:48:08 (7328): No heartbeat from core client for 30 sec - exiting 09:48:09 (7328): No heartbeat from core client for 30 sec - exiting 09:48:10 (7328): No heartbeat from core client for 30 sec - exiting 09:48:11 (7328): No heartbeat from core client for 30 sec - exiting 09:48:12 (7328): No heartbeat from core client for 30 sec - exiting 09:48:13 (7328): No heartbeat from core client for 30 sec - exiting 09:48:14 (7328): No heartbeat from core client for 30 sec - exiting 09:48:15 (7328): No heartbeat from core client for 30 sec - exiting 09:48:16 (7328): No heartbeat from core client for 30 sec - exiting 09:48:17 (7328): No heartbeat from core client for 30 sec - exiting 09:48:18 (7328): No heartbeat from core client for 30 sec - exiting 09:48:19 (7328): No heartbeat from core client for 30 sec - exiting 09:48:20 (7328): No heartbeat from core client for 30 sec - exiting 09:48:21 (7328): No heartbeat from core client for 30 sec - exiting 09:48:22 (7328): No heartbeat from core client for 30 sec - exiting 09:48:23 (7328): No heartbeat from core client for 30 sec - exiting 09:48:24 (7328): No heartbeat from core client for 30 sec - exiting 09:48:25 (7328): No heartbeat from core client for 30 sec - exiting 09:48:26 (7328): No heartbeat from core client for 30 sec - exiting 09:48:27 (7328): No heartbeat from core client for 30 sec - exiting 09:48:28 (7328): No heartbeat from core client for 30 sec - exiting 09:48:29 (7328): No heartbeat from core client for 30 sec - exiting 09:48:30 (7328): No heartbeat from core client for 30 sec - exiting 09:48:31 (7328): No heartbeat from core client for 30 sec - exiting 09:48:32 (7328): No heartbeat from core client for 30 sec - exiting 09:48:33 (7328): No heartbeat from core client for 30 sec - exiting 09:48:34 (7328): No heartbeat from core client for 30 sec - exiting 09:48:35 (7328): No heartbeat from core client for 30 sec - exiting 09:48:36 (7328): No heartbeat from core client for 30 sec - exiting 09:48:37 (7328): No heartbeat from core client for 30 sec - exiting 09:48:38 (7328): No heartbeat from core client for 30 sec - exiting 09:48:39 (7328): No heartbeat from core client for 30 sec - exiting 09:48:40 (7328): No heartbeat from core client for 30 sec - exiting 09:48:41 (7328): No heartbeat from core client for 30 sec - exiting 09:48:42 (7328): No heartbeat from core client for 30 sec - exiting 09:48:43 (7328): No heartbeat from core client for 30 sec - exiting 09:48:44 (7328): No heartbeat from core client for 30 sec - exiting 09:48:45 (7328): No heartbeat from core client for 30 sec - exiting 09:48:46 (7328): No heartbeat from core client for 30 sec - exiting 09:48:47 (7328): No heartbeat from core client for 30 sec - exiting 09:48:48 (7328): No heartbeat from core client for 30 sec - exiting 09:48:49 (7328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Sep 2012 18:37:41	1220576	15289004	hadcm3n_zhk1_1880_40_008199509_2	103,680	80,777	0.7791
17 Sep 2012 12:26:04	1220576	15289004	hadcm3n_zhk1_1880_40_008199509_2	77,760	60,490	0.7779
17 Sep 2012 06:44:44	1220576	15289004	hadcm3n_zhk1_1880_40_008199509_2	51,840	40,746	0.7860
17 Sep 2012 00:53:22	1220576	15289004	hadcm3n_zhk1_1880_40_008199509_2	25,920	19,952	0.7698