Task 12355562

Name	famous_vchr_1799_200_006701957_5
Workunit	6905210
Created	1 Dec 2010, 22:31:29 UTC
Sent	1 Dec 2010, 22:40:38 UTC
Report deadline	3 Mar 2011, 6:07:49 UTC
Received	15 Dec 2010, 0:46:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1119481
Run time	20 hours 38 min 53 sec
CPU time	19 hours 25 min 37 sec
Validate state	Invalid
Credit	586.84
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:49:36 (840): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:37:35 (3372): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO tmp/pipe_dummy Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:24:06 (4164): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vchr_1799_200_006701957\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vchr_1799_200_006701957\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vchr_1799_200_006701957\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vchr_1799_200_006701957\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vchr_1799_200_006701957\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vchr_1799_200_006701957\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( 19:45:24 (3612): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Dec 2010 00:25:49	1119481	12355562	famous_vchr_1799_200_006701957_5	177,866	69,471	0.3906
14 Dec 2010 23:22:32	1119481	12355562	famous_vchr_1799_200_006701957_5	168,506	65,785	0.3904
14 Dec 2010 22:21:15	1119481	12355562	famous_vchr_1799_200_006701957_5	159,146	62,107	0.3903
14 Dec 2010 21:13:14	1119481	12355562	famous_vchr_1799_200_006701957_5	149,786	58,423	0.3900
14 Dec 2010 20:13:05	1119481	12355562	famous_vchr_1799_200_006701957_5	140,426	54,739	0.3898
14 Dec 2010 14:11:59	1119481	12355562	famous_vchr_1799_200_006701957_5	131,066	51,091	0.3898
14 Dec 2010 14:11:59	1119481	12355562	famous_vchr_1799_200_006701957_5	121,706	47,461	0.3900
14 Dec 2010 00:56:40	1119481	12355562	famous_vchr_1799_200_006701957_5	112,346	43,804	0.3899
13 Dec 2010 21:23:24	1119481	12355562	famous_vchr_1799_200_006701957_5	102,986	40,108	0.3895
13 Dec 2010 20:18:48	1119481	12355562	famous_vchr_1799_200_006701957_5	93,626	36,426	0.3891
13 Dec 2010 09:06:47	1119481	12355562	famous_vchr_1799_200_006701957_5	84,266	32,738	0.3885
12 Dec 2010 19:18:04	1119481	12355562	famous_vchr_1799_200_006701957_5	74,906	29,052	0.3878
12 Dec 2010 14:16:08	1119481	12355562	famous_vchr_1799_200_006701957_5	65,546	25,408	0.3876
12 Dec 2010 09:27:32	1119481	12355562	famous_vchr_1799_200_006701957_5	56,186	21,798	0.3880
11 Dec 2010 23:49:45	1119481	12355562	famous_vchr_1799_200_006701957_5	46,826	18,166	0.3879
11 Dec 2010 17:34:08	1119481	12355562	famous_vchr_1799_200_006701957_5	37,466	14,536	0.3880
11 Dec 2010 10:00:06	1119481	12355562	famous_vchr_1799_200_006701957_5	28,106	10,920	0.3885
11 Dec 2010 10:00:06	1119481	12355562	famous_vchr_1799_200_006701957_5	18,746	7,268	0.3877
10 Dec 2010 13:54:48	1119481	12355562	famous_vchr_1799_200_006701957_5	9,386	3,630	0.3867