Task 15453945

Name	hadcm3n_ziwe_1880_40_008245801_4
Workunit	8400925
Created	22 Nov 2012, 15:03:18 UTC
Sent	22 Nov 2012, 15:03:57 UTC
Report deadline	21 Feb 2013, 22:31:08 UTC
Received	6 Jan 2013, 16:15:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1225442
Run time	10 days 16 hours 2 min 19 sec
CPU time	9 days 19 hours 13 min 42 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	1.93 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4432, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4628, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1784, iMonCtr=1 Model crash detected, will try to restart... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2496, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:17:31 (5060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:07:45 (4240): No heartbeat from core client for 30 sec - exiting 20:07:47 (4240): No heartbeat from core client for 30 sec - exiting 20:07:48 (4240): No heartbeat from core client for 30 sec - exiting 20:07:49 (4240): No heartbeat from core client for 30 sec - exiting 20:07:50 (4240): No heartbeat from core client for 30 sec - exiting 20:07:51 (4240): No heartbeat from core client for 30 sec - exiting 20:07:52 (4240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 21:14:59 (4636): No heartbeat from core client for 30 sec - exiting 21:15:00 (4636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:38:28 (2632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:41:35 (4884): No heartbeat from core client for 30 sec - exiting 03:41:36 (4884): No heartbeat from core client for 30 sec - exiting 03:41:37 (4884): No heartbeat from core client for 30 sec - exiting 03:41:38 (4884): No heartbeat from core client for 30 sec - exiting 03:41:39 (4884): No heartbeat from core client for 30 sec - exiting 03:41:41 (4884): No heartbeat from core client for 30 sec - exiting 03:41:42 (4884): No heartbeat from core client for 30 sec - exiting 03:41:43 (4884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:04:44 (4316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1320, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jan 2013 19:02:09	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	414,720	803,267	1.9369
31 Dec 2012 21:18:10	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	388,800	754,139	1.9397
27 Dec 2012 20:14:20	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	362,880	703,629	1.9390
24 Dec 2012 12:30:14	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	336,960	656,263	1.9476
20 Dec 2012 21:57:21	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	311,040	605,553	1.9469
17 Dec 2012 19:57:53	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	285,120	554,333	1.9442
15 Dec 2012 16:51:03	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	259,200	503,065	1.9408
14 Dec 2012 15:44:08	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	233,280	453,449	1.9438
14 Dec 2012 15:44:08	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	207,360	403,784	1.9473
07 Dec 2012 12:52:40	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	181,440	353,384	1.9477
04 Dec 2012 17:49:28	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	155,520	304,245	1.9563
02 Dec 2012 15:49:56	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	129,600	254,004	1.9599
29 Nov 2012 10:17:52	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	103,680	203,042	1.9584
27 Nov 2012 17:30:41	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	77,760	152,150	1.9567
26 Nov 2012 11:25:51	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	51,840	101,293	1.9540
23 Nov 2012 23:31:17	1225442	15453945	hadcm3n_ziwe_1880_40_008245801_4	25,920	50,665	1.9547