Task 12739064

Name	hadcm3n_o2s7_1900_40_007198938_0
Workunit	7397218
Created	28 Mar 2011, 14:04:41 UTC
Sent	31 Mar 2011, 19:46:34 UTC
Report deadline	1 Jul 2011, 3:13:45 UTC
Received	7 Apr 2011, 14:03:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	964619
Run time	6 days 0 hours 5 min 40 sec
CPU time	5 days 7 hours 58 min 13 sec
Validate state	Invalid
Credit	4,043.52
Device peak FLOPS	2.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17616, iMonCtr=1 Model crash detected, will try to restart... 20:12:37 (6764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4724, iMonCtr=1 Model crash detected, will try to restart... 12:02:25 (3964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 06:02:53 (5072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:02:55 (5072): No heartbeat from core client for 30 sec - exiting 06:02:56 (5072): No heartbeat from core client for 30 sec - exiting 06:02:57 (5072): No heartbeat from core client for 30 sec - exiting 06:02:58 (5072): No heartbeat from core client for 30 sec - exiting 06:02:59 (5072): No heartbeat from core client for 30 sec - exiting 06:03:00 (5072): No heartbeat from core client for 30 sec - exiting 06:03:01 (5072): No heartbeat from core client for 30 sec - exiting 06:03:02 (5072): No heartbeat from core client for 30 sec - exiting 06:03:03 (5072): No heartbeat from core client for 30 sec - exiting 06:03:06 (5072): No heartbeat from core client for 30 sec - exiting 06:03:07 (5072): No heartbeat from core client for 30 sec - exiting 06:03:08 (5072): No heartbeat from core client for 30 sec - exiting 06:03:09 (5072): No heartbeat from core client for 30 sec - exiting 06:14:34 (97224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:15:24 (97224): No heartbeat from core client for 30 sec - exiting 06:15:25 (97224): No heartbeat from core client for 30 sec - exiting 06:15:26 (97224): No heartbeat from core client for 30 sec - exiting 06:15:27 (97224): No heartbeat from core client for 30 sec - exiting 06:15:28 (97224): No heartbeat from core client for 30 sec - exiting 06:15:29 (97224): No heartbeat from core client for 30 sec - exiting 06:15:30 (97224): No heartbeat from core client for 30 sec - exiting 06:15:31 (97224): No heartbeat from core client for 30 sec - exiting 06:15:32 (97224): No heartbeat from core client for 30 sec - exiting 06:15:33 (97224): No heartbeat from core client for 30 sec - exiting Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Apr 2011 09:02:22	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	336,960	501,053	1.4870
12 Apr 2011 09:02:06	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	311,040	461,953	1.4852
12 Apr 2011 09:01:54	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	285,120	423,362	1.4849
05 Apr 2011 22:42:56	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	259,200	384,487	1.4834
05 Apr 2011 11:50:35	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	233,280	346,103	1.4836
05 Apr 2011 00:56:38	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	207,360	307,460	1.4827
04 Apr 2011 14:01:59	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	181,440	269,176	1.4836
03 Apr 2011 15:41:08	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	155,520	230,673	1.4832
03 Apr 2011 04:57:35	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	129,600	192,526	1.4855
02 Apr 2011 18:40:29	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	103,680	154,610	1.4912
02 Apr 2011 07:05:54	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	77,760	116,149	1.4937
01 Apr 2011 19:46:43	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	51,840	76,846	1.4824
01 Apr 2011 08:49:57	964619	12739064	hadcm3n_o2s7_1900_40_007198938_0	25,920	38,331	1.4788