Task 13096549

Name	hadcm3n_yaej_1900_40_007346341_1
Workunit	7543771
Created	6 Jul 2011, 13:37:06 UTC
Sent	19 Jul 2011, 19:48:16 UTC
Report deadline	19 Oct 2011, 3:15:27 UTC
Received	26 Aug 2011, 20:13:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	986477
Run time	31 days 12 hours 7 min 52 sec
CPU time	26 days 7 hours 9 min
Validate state	Invalid
Credit	6,842.88
Device peak FLOPS	2.19 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:15:39 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:53:29 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:27:52 (7384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7016, iMonCtr=1 Model crash detected, will try to restart... 10:12:44 (4704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:34:27 (4168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:04 (3392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:09:33 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:41:48 (4572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:41:49 (4572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:10:35 (3240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2544, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:38:34 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:27:40 (4580): Can't acquire lockfile (32) - waiting 35s 14:27:51 (2160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:01:28 (4580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:10 (5656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:57:55 (2484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:58:25 (5500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3368, selfPID=3368, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:39:41 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5560, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5560, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5560, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5560, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5560, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5560, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Aug 2011 02:37:26	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	570,240	2,221,156	3.8951
23 Aug 2011 13:01:06	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	544,320	2,119,715	3.8942
22 Aug 2011 00:27:58	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	518,400	2,020,700	3.8980
20 Aug 2011 10:05:09	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	492,480	1,908,802	3.8759
18 Aug 2011 20:44:41	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	466,560	1,805,497	3.8698
17 Aug 2011 07:47:02	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	440,640	1,701,041	3.8604
16 Aug 2011 05:06:12	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	414,720	1,616,741	3.8984
14 Aug 2011 21:47:48	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	388,800	1,521,088	3.9123
13 Aug 2011 10:10:26	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	362,880	1,414,195	3.8971
12 Aug 2011 06:42:07	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	336,960	1,328,208	3.9417
10 Aug 2011 04:37:30	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	311,040	1,235,857	3.9733
08 Aug 2011 16:22:43	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	285,120	1,128,108	3.9566
07 Aug 2011 06:29:29	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	259,200	1,025,999	3.9583
05 Aug 2011 16:00:42	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	233,280	925,327	3.9666
04 Aug 2011 04:11:36	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	207,360	817,565	3.9427
02 Aug 2011 18:45:30	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	181,440	716,335	3.9481
01 Aug 2011 10:58:45	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	155,520	618,574	3.9775
31 Jul 2011 01:34:39	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	129,600	517,028	3.9894
29 Jul 2011 17:41:26	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	103,680	417,028	4.0223
28 Jul 2011 04:36:31	986477	13096549	hadcm3n_yaej_1900_40_007346341_1	77,760	310,160	3.9887