Task 11776747

Name	famous_vdlp_799_200_006703395_3
Workunit	6906648
Created	26 Aug 2010, 16:47:53 UTC
Sent	29 Nov 2010, 0:37:15 UTC
Report deadline	28 Feb 2011, 8:04:26 UTC
Received	9 Dec 2010, 19:19:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1119481
Run time	6 days 0 hours 29 min 50 sec
CPU time	5 days 15 hours 53 min 40 sec
Validate state	Invalid
Credit	4,138.23
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 19:20:15 (3252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:36:49 (2152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:36:51 (2152): No heartbeat from core client for 30 sec - exiting 00:36:52 (2152): No heartbeat from core client for 30 sec - exiting 00:36:53 (2152): No heartbeat from core client for 30 sec - exiting 00:36:54 (2152): No heartbeat from core client for 30 sec - exiting 00:36:55 (2152): No heartbeat from core client for 30 sec - exiting 00:36:56 (2152): No heartbeat from core client for 30 sec - exiting 00:36:57 (2152): No heartbeat from core client for 30 sec - exiting 00:36:58 (2152): No heartbeat from core client for 30 sec - exiting 00:34:49 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:37:50 (1324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:39:55 (1320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO tmp/pipe_dummy Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 MainError: 05:15:41 PM No files match the supplied pattern. MainError: 05:15:41 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:39:17 (3732): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO tmp/pipe_dummy Ocean Restart file copy failed on vdlplo#da00000093071+ Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:02:07 (808): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 09:14:11 (3360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:46:03 (4948): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vdlp_799_200_006703395\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2908, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vdlp_799_200_006703395\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2908, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vdlp_799_200_006703395\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2908, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vdlp_799_200_006703395\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2908, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vdlp_799_200_006703395\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2908, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vdlp_799_200_006703395\tmp\cp.namelists, line 1, position 0 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2908, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( 12:49:13 (2908): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Dec 2010 16:38:50	1119481	11776747	famous_vdlp_799_200_006703395_3	1,254,266	487,379	0.3886
09 Dec 2010 15:36:00	1119481	11776747	famous_vdlp_799_200_006703395_3	1,244,906	483,673	0.3885
09 Dec 2010 09:23:40	1119481	11776747	famous_vdlp_799_200_006703395_3	1,235,546	479,948	0.3885
09 Dec 2010 09:23:40	1119481	11776747	famous_vdlp_799_200_006703395_3	1,226,186	476,255	0.3884
09 Dec 2010 00:48:28	1119481	11776747	famous_vdlp_799_200_006703395_3	1,216,826	472,530	0.3883
08 Dec 2010 23:43:53	1119481	11776747	famous_vdlp_799_200_006703395_3	1,207,466	468,830	0.3883
08 Dec 2010 22:25:01	1119481	11776747	famous_vdlp_799_200_006703395_3	1,198,106	465,084	0.3882
08 Dec 2010 21:18:07	1119481	11776747	famous_vdlp_799_200_006703395_3	1,188,746	461,378	0.3881
08 Dec 2010 20:14:14	1119481	11776747	famous_vdlp_799_200_006703395_3	1,179,386	457,696	0.3881
08 Dec 2010 19:08:18	1119481	11776747	famous_vdlp_799_200_006703395_3	1,170,026	454,007	0.3880
08 Dec 2010 18:05:32	1119481	11776747	famous_vdlp_799_200_006703395_3	1,160,666	450,315	0.3880
08 Dec 2010 14:24:26	1119481	11776747	famous_vdlp_799_200_006703395_3	1,151,306	446,624	0.3879
08 Dec 2010 13:40:22	1119481	11776747	famous_vdlp_799_200_006703395_3	1,141,946	442,937	0.3879
08 Dec 2010 12:25:42	1119481	11776747	famous_vdlp_799_200_006703395_3	1,132,586	439,256	0.3878
08 Dec 2010 12:13:46	1119481	11776747	famous_vdlp_799_200_006703395_3	1,123,226	435,573	0.3878
08 Dec 2010 09:53:45	1119481	11776747	famous_vdlp_799_200_006703395_3	1,113,866	431,885	0.3877
08 Dec 2010 08:53:13	1119481	11776747	famous_vdlp_799_200_006703395_3	1,104,506	428,194	0.3877
08 Dec 2010 08:53:13	1119481	11776747	famous_vdlp_799_200_006703395_3	1,095,146	424,517	0.3876
08 Dec 2010 08:53:13	1119481	11776747	famous_vdlp_799_200_006703395_3	1,085,786	420,811	0.3876
08 Dec 2010 08:53:13	1119481	11776747	famous_vdlp_799_200_006703395_3	1,076,426	417,121	0.3875