Task 15431566

Name	hadcm3n_zeq1_1880_40_008198308_3
Workunit	8353432
Created	8 Nov 2012, 8:35:37 UTC
Sent	8 Nov 2012, 8:35:45 UTC
Report deadline	7 Feb 2013, 16:02:56 UTC
Received	11 Jan 2013, 16:35:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	25 (0x00000019) Unknown error code
Computer ID	1241225
Run time	48 days 17 hours 47 min 50 sec
CPU time	35 days 8 hours 7 min 55 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	1.19 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The drive cannot locate a specific area or track on the disk. (0x19) - exit code 25 (0x19) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:30:11 (3768): No heartbeat from core client for 30 sec - exiting 16:30:12 (3768): No heartbeat from core client for 30 sec - exiting 16:30:13 (3768): No heartbeat from core client for 30 sec - exiting 16:30:15 (3768): No heartbeat from core client for 30 sec - exiting 16:30:16 (3768): No heartbeat from core client for 30 sec - exiting 16:30:17 (3768): No heartbeat from core client for 30 sec - exiting 16:30:18 (3768): No heartbeat from core client for 30 sec - exiting 16:30:19 (3768): No heartbeat from core client for 30 sec - exiting 16:30:20 (3768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:21 (3768): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:14:11 (1688): No heartbeat from core client for 30 sec - exiting 16:14:13 (1688): No heartbeat from core client for 30 sec - exiting 16:14:17 (1688): No heartbeat from core client for 30 sec - exiting 16:14:18 (1688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:15:07 (6036): Can't acquire lockfile (32) - waiting 35s Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6036, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 14:46:17 (4480): No heartbeat from core client for 30 sec - exiting 14:46:18 (4480): No heartbeat from core client for 30 sec - exiting 14:46:19 (4480): No heartbeat from core client for 30 sec - exiting 14:46:20 (4480): No heartbeat from core client for 30 sec - exiting 14:46:22 (4480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:14:38 (4524): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:24:37 (3740): No heartbeat from core client for 30 sec - exiting 13:24:38 (3740): No heartbeat from core client for 30 sec - exiting 13:24:39 (3740): No heartbeat from core client for 30 sec - exiting 13:24:40 (3740): No heartbeat from core client for 30 sec - exiting 13:24:41 (3740): No heartbeat from core client for 30 sec - exiting 13:24:42 (3740): No heartbeat from core client for 30 sec - exiting 13:24:44 (3740): No heartbeat from core client for 30 sec - exiting 13:24:45 (3740): No heartbeat from core client for 30 sec - exiting 13:24:46 (3740): No heartbeat from core client for 30 sec - exiting 13:24:47 (3740): No heartbeat from core client for 30 sec - exiting 13:24:49 (3740): No heartbeat from core client for 30 sec - exiting 13:24:51 (3740): No heartbeat from core client for 30 sec - exiting 13:24:52 (3740): No heartbeat from core client for 30 sec - exiting 13:24:53 (3740): No heartbeat from core client for 30 sec - exiting 13:24:54 (3740): No heartbeat from core client for 30 sec - exiting 13:24:55 (3740): No heartbeat from core client for 30 sec - exiting 13:24:57 (3740): No heartbeat from core client for 30 sec - exiting 13:24:59 (3740): No heartbeat from core client for 30 sec - exiting 13:25:00 (3740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:25:01 (3740): No heartbeat from core client for 30 sec - exiting 16:20:34 (4108): No heartbeat from core client for 30 sec - exiting 16:20:35 (4108): No heartbeat from core client for 30 sec - exiting 16:20:36 (4108): No heartbeat from core client for 30 sec - exiting 16:20:38 (4108): No heartbeat from core client for 30 sec - exiting 16:20:39 (4108): No heartbeat from core client for 30 sec - exiting 16:20:40 (4108): No heartbeat from core client for 30 sec - exiting 16:20:41 (4108): No heartbeat from core client for 30 sec - exiting 16:20:42 (4108): No heartbeat from core client for 30 sec - exiting 16:20:43 (4108): No heartbeat from core client for 30 sec - exiting 16:20:44 (4108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5244, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jan 2013 23:12:58	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	544,320	2,968,925	5.4544
05 Jan 2013 14:40:41	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	518,400	2,813,251	5.4268
02 Jan 2013 08:45:05	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	492,480	2,667,253	5.4160
30 Dec 2012 03:43:11	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	466,560	2,521,896	5.4053
24 Dec 2012 03:31:24	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	440,640	2,362,022	5.3604
20 Dec 2012 00:54:31	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	414,720	2,203,778	5.3139
16 Dec 2012 14:00:36	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	388,800	2,068,430	5.3200
14 Dec 2012 15:54:56	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	362,880	1,924,888	5.3045
14 Dec 2012 15:54:56	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	336,960	1,789,527	5.3108
14 Dec 2012 15:54:56	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	311,040	1,638,611	5.2682
04 Dec 2012 18:08:10	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	285,120	1,503,770	5.2742
02 Dec 2012 05:58:06	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	259,200	1,365,215	5.2670
29 Nov 2012 11:43:08	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	233,280	1,217,969	5.2211
26 Nov 2012 12:46:33	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	207,360	1,077,800	5.1977
24 Nov 2012 07:03:07	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	181,440	941,317	5.1880
21 Nov 2012 15:04:22	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	155,520	807,695	5.1935
18 Nov 2012 22:01:59	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	129,600	678,378	5.2344
17 Nov 2012 08:10:21	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	103,680	545,873	5.2650
15 Nov 2012 12:19:46	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	77,760	411,531	5.2923
12 Nov 2012 21:16:15	1241225	15431566	hadcm3n_zeq1_1880_40_008198308_3	51,840	283,186	5.4627