Task 13641318

Name	hadcm3n_t1vf_1940_40_007536459_4
Workunit	7733691
Created	17 Nov 2011, 6:32:17 UTC
Sent	17 Nov 2011, 6:36:34 UTC
Report deadline	16 Feb 2012, 14:03:45 UTC
Received	14 Dec 2011, 2:57:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1140685
Run time	8 days 18 hours 16 min 39 sec
CPU time	5 days 14 hours 17 min 49 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.20</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 00:48:03 (3640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:27:43 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:42 (3684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:25:41 (2256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:24:35 (1900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:42:58 (2360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on t1vfko.dae61t0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:52:39 (3360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2248, selfPID=2248, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t1vfko.pje6c10 03:51:33 (5468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:33:14 (3076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:30:57 (4476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:29:57 (200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:28:56 (5664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:27:54 (3252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:22:27 (5972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:26:14 (5388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:17:54 (5940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7C910CCE read attempt to address 0x3FEFB467 Engaging BOINC Windows Runtime Debugger... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4296, selfPID=4296, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7C911295 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 02:24:04 (3340): No heartbeat from core client for 30 sec - exiting 02:24:05 (3340): No heartbeat from core client for 30 sec - exiting 02:24:06 (3340): No heartbeat from core client for 30 sec - exiting 02:24:07 (3340): No heartbeat from core client for 30 sec - exiting 02:24:08 (3340): No heartbeat from core client for 30 sec - exiting 02:24:10 (3340): No heartbeat from core client for 30 sec - exiting 02:24:11 (3340): No heartbeat from core client for 30 sec - exiting 02:24:12 (3340): No heartbeat from core client for 30 sec - exiting 02:24:13 (3340): No heartbeat from core client for 30 sec - exiting 02:24:14 (3340): No heartbeat from core client for 30 sec - exiting 02:24:15 (3340): No heartbeat from core client for 30 sec - exiting 02:24:16 (3340): No heartbeat from core client for 30 sec - exiting 02:24:17 (3340): No heartbeat from core client for 30 sec - exiting 02:24:18 (3340): No heartbeat from core client for 30 sec - exiting 02:24:19 (3340): No heartbeat from core client for 30 sec - exiting 02:24:20 (3340): No heartbeat from core client for 30 sec - exiting 02:24:22 (3340): No heartbeat from core client for 30 sec - exiting 02:24:23 (3340): No heartbeat from core client for 30 sec - exiting 02:24:24 (3340): No heartbeat from core client for 30 sec - exiting 02:24:25 (3340): No heartbeat from core client for 30 sec - exiting 02:24:26 (3340): No heartbeat from core client for 30 sec - exiting 02:24:27 (3340): No heartbeat from core client for 30 sec - exiting 02:24:28 (3340): No heartbeat from core client for 30 sec - exiting 02:24:29 (3340): No heartbeat from core client for 30 sec - exiting 02:24:30 (3340): No heartbeat from core client for 30 sec - exiting 02:24:31 (3340): No heartbeat from core client for 30 sec - exiting 12:23:03 (3340): No heartbeat from core client for 30 sec - exiting 12:23:04 (3340): No heartbeat from core client for 30 sec - exiting 12:23:05 (3340): No heartbeat from core client for 30 sec - exiting 12:23:06 (3340): No heartbeat from core client for 30 sec - exiting 12:23:07 (3340): No heartbeat from core client for 30 sec - exiting 12:23:09 (3340): No heartbeat from core client for 30 sec - exiting 12:23:10 (3340): No heartbeat from core client for 30 sec - exiting 12:23:11 (3340): No heartbeat from core client for 30 sec - exiting 12:23:12 (3340): No heartbeat from core client for 30 sec - exiting 12:23:13 (3340): No heartbeat from core client for 30 sec - exiting 12:23:14 (3340): No heartbeat from core client for 30 sec - exiting 12:23:15 (3340): No heartbeat from core client for 30 sec - exiting 12:23:16 (3340): No heartbeat from core client for 30 sec - exiting 12:23:17 (3340): No heartbeat from core client for 30 sec - exiting 12:23:18 (3340): No heartbeat from core client for 30 sec - exiting 12:23:19 (3340): No heartbeat from core client for 30 sec - exiting 12:23:21 (3340): No heartbeat from core client for 30 sec - exiting 12:23:22 (3340): No heartbeat from core client for 30 sec - exiting 12:23:23 (3340): No heartbeat from core client for 30 sec - exiting 12:23:24 (3340): No heartbeat from core client for 30 sec - exiting 12:23:25 (3340): No heartbeat from core client for 30 sec - exiting 12:23:26 (3340): No heartbeat from core client for 30 sec - exiting 12:23:27 (3340): No heartbeat from core client for 30 sec - exiting 12:23:28 (3340): No heartbeat from core client for 30 sec - exiting 12:23:29 (3340): No heartbeat from core client for 30 sec - exiting 12:23:30 (3340): No heartbeat from core client for 30 sec - exiting 12:23:32 (3340): No heartbeat from core client for 30 sec - exiting 00:30:17 (3340): No heartbeat from core client for 30 sec - exiting 00:30:18 (3340): No heartbeat from core client for 30 sec - exiting 00:30:19 (3340): No heartbeat from core client for 30 sec - exiting 00:30:20 (3340): No heartbeat from core client for 30 sec - exiting 00:30:21 (3340): No heartbeat from core client for 30 sec - exiting 00:30:22 (3340): No heartbeat from core client for 30 sec - exiting 00:30:23 (3340): No heartbeat from core client for 30 sec - exiting 00:30:24 (3340): No heartbeat from core client for 30 sec - exiting 00:30:25 (3340): No heartbeat from core client for 30 sec - exiting 00:30:26 (3340): No heartbeat from core client for 30 sec - exiting 00:30:27 (3340): No heartbeat from core client for 30 sec - exiting 00:30:28 (3340): No heartbeat from core client for 30 sec - exiting 00:30:29 (3340): No heartbeat from core client for 30 sec - exiting 00:30:30 (3340): No heartbeat from core client for 30 sec - exiting 00:30:31 (3340): No heartbeat from core client for 30 sec - exiting 00:30:32 (3340): No heartbeat from core client for 30 sec - exiting 00:30:33 (3340): No heartbeat from core client for 30 sec - exiting 00:30:34 (3340): No heartbeat from core client for 30 sec - exiting 00:30:35 (3340): No heartbeat from core client for 30 sec - exiting 00:30:36 (3340): No heartbeat from core client for 30 sec - exiting 00:30:37 (3340): No heartbeat from core client for 30 sec - exiting 00:30:38 (3340): No heartbeat from core client for 30 sec - exiting 00:30:39 (3340): No heartbeat from core client for 30 sec - exiting 00:30:40 (3340): No heartbeat from core client for 30 sec - exiting 00:30:41 (3340): No heartbeat from core client for 30 sec - exiting 00:30:42 (3340): No heartbeat from core client for 30 sec - exiting 00:30:43 (3340): No heartbeat from core client for 30 sec - exiting 00:30:44 (3340): No heartbeat from core client for 30 sec - exiting 00:30:45 (3340): No heartbeat from core client for 30 sec - exiting 00:30:46 (3340): No heartbeat from core client for 30 sec - exiting 00:30:47 (3340): No heartbeat from core client for 30 sec - exiting 00:30:48 (3340): No heartbeat from core client for 30 sec - exiting 00:30:49 (3340): No heartbeat from core client for 30 sec - exiting 00:30:50 (3340): No heartbeat from core client for 30 sec - exiting 00:30:51 (3340): No heartbeat from core client for 30 sec - exiting 00:30:52 (3340): No heartbeat from core client for 30 sec - exiting 00:30:53 (3340): No heartbeat from core client for 30 sec - exiting 00:30:54 (3340): No heartbeat from core client for 30 sec - exiting 00:30:55 (3340): No heartbeat from core client for 30 sec - exiting 00:30:57 (3340): No heartbeat from core client for 30 sec - exiting 00:31:01 (3340): No heartbeat from core client for 30 sec - exiting 00:31:03 (3340): No heartbeat from core client for 30 sec - exiting 00:31:06 (3340): No heartbeat from core client for 30 sec - exiting 00:31:08 (3340): No heartbeat from core client for 30 sec - exiting 00:31:09 (3340): No heartbeat from core client for 30 sec - exiting 00:31:10 (3340): No heartbeat from core client for 30 sec - exiting 00:31:11 (3340): No heartbeat from core client for 30 sec - exiting 00:31:12 (3340): No heartbeat from core client for 30 sec - exiting 00:31:13 (3340): No heartbeat from core client for 30 sec - exiting 00:31:14 (3340): No heartbeat from core client for 30 sec - exiting 00:31:15 (3340): No heartbeat from core client for 30 sec - exiting 00:31:16 (3340): No heartbeat from core client for 30 sec - exiting 00:31:17 (3340): No heartbeat from core client for 30 sec - exiting 00:31:18 (3340): No heartbeat from core client for 30 sec - exiting 00:31:19 (3340): No heartbeat from core client for 30 sec - exiting 00:31:20 (3340): No heartbeat from core client for 30 sec - exiting 00:31:21 (3340): No heartbeat from core client for 30 sec - exiting 00:31:22 (3340): No heartbeat from core client for 30 sec - exiting 00:31:23 (3340): No heartbeat from core client for 30 sec - exiting 00:31:24 (3340): No heartbeat from core client for 30 sec - exiting 00:31:25 (3340): No heartbeat from core client for 30 sec - exiting 00:31:26 (3340): No heartbeat from core client for 30 sec - exiting 00:31:27 (3340): No heartbeat from core client for 30 sec - exiting 00:31:28 (3340): No heartbeat from core client for 30 sec - exiting 00:31:30 (3340): No heartbeat from core client for 30 sec - exiting 00:31:31 (3340): No heartbeat from core client for 30 sec - exiting 00:31:32 (3340): No heartbeat from core client for 30 sec - exiting 00:31:33 (3340): No heartbeat from core client for 30 sec - exiting 00:31:34 (3340): No heartbeat from core client for 30 sec - exiting 00:31:35 (3340): No heartbeat from core client for 30 sec - exiting 00:31:36 (3340): No heartbeat from core client for 30 sec - exiting 00:31:37 (3340): No heartbeat from core client for 30 sec - exiting 00:31:38 (3340): No heartbeat from core client for 30 sec - exiting 00:31:39 (3340): No heartbeat from core client for 30 sec - exiting 00:31:40 (3340): No heartbeat from core client for 30 sec - exiting 00:31:41 (3340): No heartbeat from core client for 30 sec - exiting 00:31:42 (3340): No heartbeat from core client for 30 sec - exiting 00:31:43 (3340): No heartbeat from core client for 30 sec - exiting 00:31:44 (3340): No heartbeat from core client for 30 sec - exiting 00:31:45 (3340): No heartbeat from core client for 30 sec - exiting 00:31:46 (3340): No heartbeat from core client for 30 sec - exiting 00:31:47 (3340): No heartbeat from core client for 30 sec - exiting 00:31:49 (3340): No heartbeat from core client for 30 sec - exiting 00:31:50 (3340): No heartbeat from core client for 30 sec - exiting 00:31:51 (3340): No heartbeat from core client for 30 sec - exiting 00:31:52 (3340): No heartbeat from core client for 30 sec - exiting 00:31:53 (3340): No heartbeat from core client for 30 sec - exiting 00:31:54 (3340): No heartbeat from core client for 30 sec - exiting 00:31:55 (3340): No heartbeat from core client for 30 sec - exiting 00:31:56 (3340): No heartbeat from core client for 30 sec - exiting 00:31:57 (3340): No heartbeat from core client for 30 sec - exiting 00:31:58 (3340): No heartbeat from core client for 30 sec - exiting 00:31:59 (3340): No heartbeat from core client for 30 sec - exiting 00:32:00 (3340): No heartbeat from core client for 30 sec - exiting 00:32:02 (3340): No heartbeat from core client for 30 sec - exiting 00:32:03 (3340): No heartbeat from core client for 30 sec - exiting 00:32:05 (3340): No heartbeat from core client for 30 sec - exiting 00:32:06 (3340): No heartbeat from core client for 30 sec - exiting 00:32:07 (3340): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Dec 2011 18:51:05	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	259,200	482,865	1.8629
08 Dec 2011 07:52:56	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	233,280	436,094	1.8694
07 Dec 2011 02:13:35	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	207,360	390,118	1.8814
05 Dec 2011 11:51:45	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	181,440	342,040	1.8851
04 Dec 2011 09:04:59	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	155,520	295,003	1.8969
02 Dec 2011 05:36:00	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	129,600	248,448	1.9170
25 Nov 2011 21:26:18	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	103,680	200,297	1.9319
21 Nov 2011 15:05:09	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	77,760	151,209	1.9446
20 Nov 2011 14:34:07	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	51,840	101,287	1.9538
19 Nov 2011 22:30:22	1140685	13641318	hadcm3n_t1vf_1940_40_007536459_4	25,920	50,870	1.9626