Task 11421741

Name	famous_u11d_1599_200_006634188_2
Workunit	6837560
Created	10 Jun 2010, 11:12:15 UTC
Sent	15 Jul 2010, 4:07:43 UTC
Report deadline	14 Oct 2010, 11:34:54 UTC
Received	17 Jul 2010, 17:33:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1077167
Run time	1 days 9 hours 55 min 16 sec
CPU time	1 days 8 hours 11 min 27 sec
Validate state	Invalid
Credit	833.89
Device peak FLOPS	2.83 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 11:26:17 (3452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:21:29 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:21:30 (4488): No heartbeat from core client for 30 sec - exiting 14:21:31 (4488): No heartbeat from core client for 30 sec - exiting 14:21:32 (4488): No heartbeat from core client for 30 sec - exiting 14:26:58 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:27:05 (4908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:51 (1784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:57 (1124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:17:25 (3732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:41:42 (3280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:16:22 (5332): No heartbeat from core client for 30 sec - exiting 00:16:23 (5332): No heartbeat from core client for 30 sec - exiting 00:16:24 (5332): No heartbeat from core client for 30 sec - exiting 00:16:26 (5332): No heartbeat from core client for 30 sec - exiting 00:16:27 (5332): No heartbeat from core client for 30 sec - exiting 00:16:28 (5332): No heartbeat from core client for 30 sec - exiting 00:16:29 (5332): No heartbeat from core client for 30 sec - exiting 00:16:30 (5332): No heartbeat from core client for 30 sec - exiting 00:16:31 (5332): No heartbeat from core client for 30 sec - exiting 00:16:32 (5332): No heartbeat from core client for 30 sec - exiting 00:16:33 (5332): No heartbeat from core client for 30 sec - exiting 00:16:34 (5332): No heartbeat from core client for 30 sec - exiting 00:16:35 (5332): No heartbeat from core client for 30 sec - exiting 00:16:36 (5332): No heartbeat from core client for 30 sec - exiting 00:16:38 (5332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... 00:38:16 (1212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:00:58 (2484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:54:48 (2056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:56:01 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:59:55 (5568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:55:39 (4388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:55 (5492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:12:51 (4340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:52:56 (3700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:10:40 (4332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:11:56 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:12:36 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:44:34 (2456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:07 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:08 (5112): No heartbeat from core client for 30 sec - exiting 19:45:09 (5112): No heartbeat from core client for 30 sec - exiting 19:45:10 (5112): No heartbeat from core client for 30 sec - exiting 19:45:11 (5112): No heartbeat from core client for 30 sec - exiting 19:45:12 (5112): No heartbeat from core client for 30 sec - exiting 19:45:13 (5112): No heartbeat from core client for 30 sec - exiting 19:45:14 (5112): No heartbeat from core client for 30 sec - exiting 19:45:15 (5112): No heartbeat from core client for 30 sec - exiting 19:45:16 (5112): No heartbeat from core client for 30 sec - exiting 19:45:17 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:01:22 (3208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:02:48 (676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:44 (2772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:33 (3156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:59 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:18 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:39:04 (4188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:40:29 (1696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:08:57 (3480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:55:17 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:01:20 (2684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:52 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:52 (5304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:27:07 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:27:47 (5548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:37 (5420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:53:08 (6100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:16:34 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:17:37 (4752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:23:30 (5116): No heartbeat from core client for 30 sec - exiting 03:23:31 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... 03:40:35 (5156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:41:29 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:51:47 (2684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:53:38 (676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:58:11 (6004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:59:09 (5340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:19:03 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:48:05 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:46 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:02 (1716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:47:38 (6508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_u11d_1599_200_006634188/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_u11d_1599_200_006634188/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_u11d_1599_200_006634188/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_u11d_1599_200_006634188/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_u11d_1599_200_006634188/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_u11d_1599_200_006634188/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy Sorry, too many model crashes! :-( 13:31:32 (3988): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Jul 2010 16:55:04	1077167	11421741	famous_u11d_1599_200_006634188_2	252,746	115,167	0.4557
17 Jul 2010 13:05:58	1077167	11421741	famous_u11d_1599_200_006634188_2	243,386	111,223	0.4570
17 Jul 2010 06:04:28	1077167	11421741	famous_u11d_1599_200_006634188_2	234,026	107,215	0.4581
17 Jul 2010 04:39:33	1077167	11421741	famous_u11d_1599_200_006634188_2	224,666	102,678	0.4570
17 Jul 2010 03:34:03	1077167	11421741	famous_u11d_1599_200_006634188_2	215,306	98,392	0.4570
17 Jul 2010 01:16:53	1077167	11421741	famous_u11d_1599_200_006634188_2	205,946	94,353	0.4581
17 Jul 2010 00:49:33	1077167	11421741	famous_u11d_1599_200_006634188_2	196,586	90,000	0.4578
16 Jul 2010 20:49:11	1077167	11421741	famous_u11d_1599_200_006634188_2	187,226	86,099	0.4599
16 Jul 2010 20:23:03	1077167	11421741	famous_u11d_1599_200_006634188_2	177,866	81,493	0.4582
16 Jul 2010 17:48:49	1077167	11421741	famous_u11d_1599_200_006634188_2	168,506	77,254	0.4585
16 Jul 2010 17:38:26	1077167	11421741	famous_u11d_1599_200_006634188_2	159,146	72,843	0.4577
16 Jul 2010 08:17:38	1077167	11421741	famous_u11d_1599_200_006634188_2	149,786	69,056	0.4610
16 Jul 2010 06:51:04	1077167	11421741	famous_u11d_1599_200_006634188_2	140,426	64,397	0.4586
16 Jul 2010 06:39:51	1077167	11421741	famous_u11d_1599_200_006634188_2	131,066	60,402	0.4609
16 Jul 2010 06:39:51	1077167	11421741	famous_u11d_1599_200_006634188_2	121,706	56,081	0.4608
16 Jul 2010 03:00:52	1077167	11421741	famous_u11d_1599_200_006634188_2	112,346	51,666	0.4599
16 Jul 2010 02:35:22	1077167	11421741	famous_u11d_1599_200_006634188_2	102,986	47,670	0.4629
16 Jul 2010 00:02:28	1077167	11421741	famous_u11d_1599_200_006634188_2	93,626	42,936	0.4586
15 Jul 2010 23:55:33	1077167	11421741	famous_u11d_1599_200_006634188_2	84,266	38,774	0.4601
15 Jul 2010 21:31:32	1077167	11421741	famous_u11d_1599_200_006634188_2	74,906	34,452	0.4599