Task 15520195

Name	hadcm3n_104w_1980_40_008280770_0
Workunit	8431905
Created	1 Jan 2013, 22:39:30 UTC
Sent	1 Jan 2013, 22:39:37 UTC
Report deadline	3 Apr 2013, 6:06:48 UTC
Received	24 Jan 2013, 2:46:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	811338
Run time	7 days 21 hours 51 min 17 sec
CPU time	7 days 21 hours 51 min 17 sec
Validate state	Invalid
Credit	2,177.28
Device peak FLOPS	1.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.2.18</core_client_version> <![CDATA[ <message> The device does not recognise the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:16:41 (2908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:47:48 (2284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:29:02 (1988): No heartbeat from core client for 30 sec - exiting 13:29:03 (1988): No heartbeat from core client for 30 sec - exiting 13:29:04 (1988): No heartbeat from core client for 30 sec - exiting 13:29:05 (1988): No heartbeat from core client for 30 sec - exiting 13:29:06 (1988): No heartbeat from core client for 30 sec - exiting 13:29:07 (1988): No heartbeat from core client for 30 sec - exiting 13:29:08 (1988): No heartbeat from core client for 30 sec - exiting 13:29:09 (1988): No heartbeat from core client for 30 sec - exiting 13:29:10 (1988): No heartbeat from core client for 30 sec - exiting 13:29:11 (1988): No heartbeat from core client for 30 sec - exiting 13:29:12 (1988): No heartbeat from core client for 30 sec - exiting 13:29:14 (1988): No heartbeat from core client for 30 sec - exiting 13:29:15 (1988): No heartbeat from core client for 30 sec - exiting 13:29:16 (1988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:05:02 (6272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:36:49 (4940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:59:53 (332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:51:04 (4628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:10:23 (180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:27:42 (1972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:27:43 (1972): No heartbeat from core client for 30 sec - exiting 15:43:12 (4964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:55 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:55:48 (8064): No heartbeat from core client for 30 sec - exiting 13:55:50 (8064): No heartbeat from core client for 30 sec - exiting 13:55:51 (8064): No heartbeat from core client for 30 sec - exiting 13:55:52 (8064): No heartbeat from core client for 30 sec - exiting 13:55:53 (8064): No heartbeat from core client for 30 sec - exiting 13:55:54 (8064): No heartbeat from core client for 30 sec - exiting 13:55:55 (8064): No heartbeat from core client for 30 sec - exiting 13:55:56 (8064): No heartbeat from core client for 30 sec - exiting 13:55:57 (8064): No heartbeat from core client for 30 sec - exiting 13:55:58 (8064): No heartbeat from core client for 30 sec - exiting 13:55:59 (8064): No heartbeat from core client for 30 sec - exiting 13:56:00 (8064): No heartbeat from core client for 30 sec - exiting 13:56:01 (8064): No heartbeat from core client for 30 sec - exiting 13:56:02 (8064): No heartbeat from core client for 30 sec - exiting 13:56:03 (8064): No heartbeat from core client for 30 sec - exiting 13:56:04 (8064): No heartbeat from core client for 30 sec - exiting 13:56:05 (8064): No heartbeat from core client for 30 sec - exiting 13:56:06 (8064): No heartbeat from core client for 30 sec - exiting 13:56:07 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:56:08 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 01:28:35 (11912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:23:14 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/104wko.pji7c10 Error converting file to netcdf: dataout/104wko.pii7c10 Error converting file to netcdf: dataout/104wko.pfi7c10 Error converting file to netcdf: dataout/104wka.phi7c10 Error converting file to netcdf: dataout/104wka.pgi7c10 Error converting file to netcdf: dataout/104wka.pei7c10 Error converting file to netcdf: dataout/104wka.pdi7c10 03:24:55 (11232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:24:56 (11232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:34:37 (2192): No heartbeat from core client for 30 sec - exiting 14:34:38 (2192): No heartbeat from core client for 30 sec - exiting 14:34:39 (2192): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Jan 2013 13:25:20	811338	15520195	hadcm3n_104w_1980_40_008280770_0	181,440	639,158	3.5227
15 Jan 2013 21:59:43	811338	15520195	hadcm3n_104w_1980_40_008280770_0	155,520	543,163	3.4926
14 Jan 2013 17:11:28	811338	15520195	hadcm3n_104w_1980_40_008280770_0	129,600	449,854	3.4711
13 Jan 2013 11:19:49	811338	15520195	hadcm3n_104w_1980_40_008280770_0	103,680	358,478	3.4575
05 Jan 2013 17:11:15	811338	15520195	hadcm3n_104w_1980_40_008280770_0	77,760	273,362	3.5155
04 Jan 2013 14:27:39	811338	15520195	hadcm3n_104w_1980_40_008280770_0	51,840	184,471	3.5585
03 Jan 2013 07:50:44	811338	15520195	hadcm3n_104w_1980_40_008280770_0	25,920	95,315	3.6773