Task 15512508

Name	hadcm3n_zjmp_1880_40_008250866_3
Workunit	8405990
Created	26 Dec 2012, 15:23:22 UTC
Sent	27 Dec 2012, 18:17:56 UTC
Report deadline	29 Mar 2013, 1:45:07 UTC
Received	17 Feb 2013, 17:54:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	25 (0x00000019) Unknown error code
Computer ID	1223127
Run time	20 days 6 hours 3 min 43 sec
CPU time	14 days 0 hours 44 min 21 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	0.94 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The drive cannot locate a specific area or track on the disk. (0x19) - exit code 25 (0x19) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 17:55:20 (10520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:55:22 (10520): No heartbeat from core client for 30 sec - exiting 17:55:23 (10520): No heartbeat from core client for 30 sec - exiting 17:55:24 (10520): No heartbeat from core client for 30 sec - exiting 17:55:25 (10520): No heartbeat from core client for 30 sec - exiting 17:55:27 (10520): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:55:05 (5900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:55:07 (5900): No heartbeat from core client for 30 sec - exiting 17:55:08 (5900): No heartbeat from core client for 30 sec - exiting 17:55:09 (5900): No heartbeat from core client for 30 sec - exiting 17:55:10 (5900): No heartbeat from core client for 30 sec - exiting 17:55:11 (5900): No heartbeat from core client for 30 sec - exiting 17:55:12 (5900): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 19:05:59 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:59 (6320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:09:01 (6320): No heartbeat from core client for 30 sec - exiting 19:30:40 (3536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:02 (2240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:04 (2240): No heartbeat from core client for 30 sec - exiting 19:55:15 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:55:16 (6300): No heartbeat from core client for 30 sec - exiting 19:55:17 (6300): No heartbeat from core client for 30 sec - exiting 19:55:18 (6300): No heartbeat from core client for 30 sec - exiting 19:55:19 (6300): No heartbeat from core client for 30 sec - exiting 13:36:28 (5572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:14:11 (6808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zjmpko.pj82c10 Error converting file to netcdf: dataout/zjmpko.pi82c10 Error converting file to netcdf: dataout/zjmpko.pf82c10 Error converting file to netcdf: dataout/zjmpka.ph82c10 Error converting file to netcdf: dataout/zjmpka.pg82c10 Error converting file to netcdf: dataout/zjmpka.pe82c10 Error converting file to netcdf: dataout/zjmpka.pd82c10 00:26:22 (4464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:08:59 (8408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:09:00 (8408): No heartbeat from core client for 30 sec - exiting 01:09:01 (8408): No heartbeat from core client for 30 sec - exiting 01:09:02 (8408): No heartbeat from core client for 30 sec - exiting 02:12:31 (8644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:12:32 (8644): No heartbeat from core client for 30 sec - exiting 02:12:33 (8644): No heartbeat from core client for 30 sec - exiting 02:12:34 (8644): No heartbeat from core client for 30 sec - exiting 02:12:35 (8644): No heartbeat from core client for 30 sec - exiting 02:12:36 (8644): No heartbeat from core client for 30 sec - exiting 02:57:54 (5656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:57:55 (5656): No heartbeat from core client for 30 sec - exiting 02:57:56 (5656): No heartbeat from core client for 30 sec - exiting 02:57:57 (5656): No heartbeat from core client for 30 sec - exiting 03:49:23 (8500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:49:24 (8500): No heartbeat from core client for 30 sec - exiting 03:49:25 (8500): No heartbeat from core client for 30 sec - exiting 03:49:26 (8500): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:19:18 (6268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:44:22 (8332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:15:02 (1432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:15:03 (1432): No heartbeat from core client for 30 sec - exiting 23:36:57 (7516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:08 (2728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:06 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:53:28 (10460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:27:06 (8856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:27:07 (8856): No heartbeat from core client for 30 sec - exiting 02:27:08 (8856): No heartbeat from core client for 30 sec - exiting 03:24:59 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8032, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 12:48:45 (5088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:48:47 (5088): No heartbeat from core client for 30 sec - exiting 12:48:48 (5088): No heartbeat from core client for 30 sec - exiting 12:48:49 (5088): No heartbeat from core client for 30 sec - exiting 12:48:50 (5088): No heartbeat from core client for 30 sec - exiting 12:48:51 (5088): No heartbeat from core client for 30 sec - exiting 12:48:52 (5088): No heartbeat from core client for 30 sec - exiting 12:48:53 (5088): No heartbeat from core client for 30 sec - exiting 12:48:54 (5088): No heartbeat from core client for 30 sec - exiting 12:48:55 (5088): No heartbeat from core client for 30 sec - exiting 12:48:56 (5088): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:02:42 (948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:53 (6844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:06 (1888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:58:11 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:52 (8944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:20 (2604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:27:44 (408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:31:07 (4836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:31:08 (4836): No heartbeat from core client for 30 sec - exiting 11:31:09 (4836): No heartbeat from core client for 30 sec - exiting 11:31:10 (4836): No heartbeat from core client for 30 sec - exiting 12:34:39 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:13 (7076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:38:41 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:50:47 (1308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:53:52 (1056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:53:53 (1056): No heartbeat from core client for 30 sec - exiting 13:53:54 (1056): No heartbeat from core client for 30 sec - exiting 13:53:55 (1056): No heartbeat from core client for 30 sec - exiting 13:53:56 (1056): No heartbeat from core client for 30 sec - exiting 14:12:05 (6652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:18:10 (332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:24:14 (8688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:04 (8524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:12:26 (332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:37 (4884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:38 (4884): No heartbeat from core client for 30 sec - exiting 22:24:42 (864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:24:43 (864): No heartbeat from core client for 30 sec - exiting 22:24:44 (864): No heartbeat from core client for 30 sec - exiting 22:39:53 (6648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:39:54 (6648): No heartbeat from core client for 30 sec - exiting 22:42:58 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:49:17 (6908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:34:53 (2900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:10 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:31 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:50:57 (7028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:06:04 (5180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:24:24 (7600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:36:35 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:45:40 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:43 (8468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:54:52 (7408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:59 (4644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:29:14 (5676): No heartbeat from core client for 30 sec - exiting 05:29:15 (5676): No heartbeat from core client for 30 sec - exiting 05:29:16 (5676): No heartbeat from core client for 30 sec - exiting 05:29:17 (5676): No heartbeat from core client for 30 sec - exiting 05:29:18 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:41:38 (8360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:24:07 (7424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:24:08 (7424): No heartbeat from core client for 30 sec - exiting 06:51:23 (1644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:51:24 (1644): No heartbeat from core client for 30 sec - exiting 07:00:33 (8612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:13 (1520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:58 (6280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:27 (2628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:25:06 (6156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:28:10 (7528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:55:56 (6156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:14:14 (8900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:36 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:01:07 (9280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:14 (7184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:20 (7184): No heartbeat from core client for 30 sec - exiting 16:07:21 (7184): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 10:33:38 (9304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:31:04 (8148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:08 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:09 (5728): No heartbeat from core client for 30 sec - exiting 12:37:45 (9796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:45:00 (9460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:06 (8516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:07 (8516): No heartbeat from core client for 30 sec - exiting 16:46:08 (8516): No heartbeat from core client for 30 sec - exiting 16:46:09 (8516): No heartbeat from core client for 30 sec - exiting 16:46:10 (8516): No heartbeat from core client for 30 sec - exiting 16:46:11 (8516): No heartbeat from core client for 30 sec - exiting 16:46:12 (8516): No heartbeat from core client for 30 sec - exiting 16:46:13 (8516): No heartbeat from core client for 30 sec - exiting 16:58:21 (4080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:56:07 (7688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:26 (6560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:30:32 (9616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:08:04 (9324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:06 (3208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:27:45 (5176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:13:00 (7436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:04 (2412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:41 (9408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:38:08 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:18 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:54:55 (740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10156, iMonCtr=1 Model crash detected, will try to restart... 13:17:21 (5976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:07:29 (8136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:38 (3616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:11:34 (6616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:23:42 (4176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:54:59 (8048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:02:59 (208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:40:45 (7320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:18:30 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:18:31 (4932): No heartbeat from core client for 30 sec - exiting 06:18:32 (4932): No heartbeat from core client for 30 sec - exiting 06:18:34 (4932): No heartbeat from core client for 30 sec - exiting 06:18:35 (4932): No heartbeat from core client for 30 sec - exiting 06:18:36 (4932): No heartbeat from core client for 30 sec - exiting 06:18:37 (4932): No heartbeat from core client for 30 sec - exiting 06:18:38 (4932): No heartbeat from core client for 30 sec - exiting 06:18:39 (4932): No heartbeat from core client for 30 sec - exiting 06:18:40 (4932): No heartbeat from core client for 30 sec - exiting 06:18:41 (4932): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1216, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=684, iMonCtr=1 Model crash detected, will try to restart... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Feb 2013 14:10:22	1223127	15512508	hadcm3n_zjmp_1880_40_008250866_3	155,520	1,113,558	7.1602
06 Feb 2013 11:28:39	1223127	15512508	hadcm3n_zjmp_1880_40_008250866_3	129,600	927,012	7.1529
29 Jan 2013 10:34:47	1223127	15512508	hadcm3n_zjmp_1880_40_008250866_3	103,680	739,781	7.1352
18 Jan 2013 21:52:29	1223127	15512508	hadcm3n_zjmp_1880_40_008250866_3	77,760	558,153	7.1779
11 Jan 2013 05:23:43	1223127	15512508	hadcm3n_zjmp_1880_40_008250866_3	51,840	362,128	6.9855
04 Jan 2013 22:09:54	1223127	15512508	hadcm3n_zjmp_1880_40_008250866_3	25,920	187,311	7.2265