Task 15801090

Name	hadcm3n_n0tw_1880_40_008372741_0
Workunit	8523600
Created	29 May 2013, 17:28:30 UTC
Sent	29 May 2013, 17:32:33 UTC
Report deadline	29 Aug 2013, 0:59:44 UTC
Received	18 Aug 2013, 19:29:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1003068
Run time	7 days 7 hours 16 min 1 sec
CPU time	7 days 3 hours 6 min 8 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	2.83 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 01:00:06 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:11 (2556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 05:57:22 (2264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:32:51 (4036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 16:38:29 (3640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:58:54 (3176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:59:58 (4240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:00:04 (4052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:00:07 (3132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:00:08 (3832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:25:24 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1612, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1612, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1612, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1612, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1612, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1832, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Jun 2013 18:46:45	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	414,720	602,019	1.4516
05 Jun 2013 07:49:55	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	388,800	563,603	1.4496
04 Jun 2013 21:04:14	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	362,880	526,273	1.4503
04 Jun 2013 10:35:43	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	336,960	489,013	1.4512
03 Jun 2013 23:59:31	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	311,040	451,398	1.4513
03 Jun 2013 13:17:25	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	285,120	413,397	1.4499
03 Jun 2013 02:48:50	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	259,200	376,219	1.4515
02 Jun 2013 16:21:44	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	233,280	338,729	1.4520
02 Jun 2013 05:47:39	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	207,360	300,792	1.4506
01 Jun 2013 19:17:21	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	181,440	263,132	1.4502
01 Jun 2013 08:54:16	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	155,520	225,918	1.4527
31 May 2013 22:29:54	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	129,600	188,426	1.4539
31 May 2013 12:02:32	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	103,680	150,834	1.4548
31 May 2013 01:38:28	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	77,760	113,636	1.4614
30 May 2013 15:13:10	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	51,840	76,302	1.4719
30 May 2013 04:47:40	1003068	15801090	hadcm3n_n0tw_1880_40_008372741_0	25,920	38,701	1.4931