Task 13679850

Name	hadcm3n_t5id_1940_40_007432602_2
Workunit	7630105
Created	1 Dec 2011, 3:25:47 UTC
Sent	1 Dec 2011, 3:29:48 UTC
Report deadline	1 Mar 2012, 10:56:59 UTC
Received	16 Jan 2012, 0:13:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1065767
Run time	11 days 6 hours 7 min 40 sec
CPU time	7 days 23 hours 17 min 20 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:47:00 (4304): No heartbeat from core client for 30 sec - exiting 13:47:01 (4304): No heartbeat from core client for 30 sec - exiting 13:47:02 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:48:37 (5272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:38:11 (1104): No heartbeat from core client for 30 sec - exiting 15:38:12 (1104): No heartbeat from core client for 30 sec - exiting 15:38:13 (1104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t5idko.pje4c10 Error converting file to netcdf: dataout/t5idko.pie4c10 Error converting file to netcdf: dataout/t5idko.pfe4c10 Error converting file to netcdf: dataout/t5idka.phe4c10 Error converting file to netcdf: dataout/t5idka.pge4c10 Error converting file to netcdf: dataout/t5idka.pee4c10 Error converting file to netcdf: dataout/t5idka.pde4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7468, selfPID=7468, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CBUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1976, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 07:26:59 (5384): No heartbeat from core client for 30 sec - exiting 07:27:00 (5384): No heartbeat from core client for 30 sec - exiting 07:27:01 (5384): No heartbeat from core client for 30 sec - exiting 07:27:02 (5384): No heartbeat from core client for 30 sec - exiting 07:27:03 (5384): No heartbeat from core client for 30 sec - exiting 07:27:04 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:27:05 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CBUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t5idko.pjf6c10 Error converting file to netcdf: dataout/t5idko.pif6c10 Error converting file to netcdf: dataout/t5idko.pff6c10 Error converting file to netcdf: dataout/t5idka.phf6c10 Error converting file to netcdf: dataout/t5idka.pgf6c10 Error converting file to netcdf: dataout/t5idka.pef6c10 Error converting file to netcdf: dataout/t5idka.pdf6c10 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7356, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5528, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5752, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5752, iMonCtr=1 Model crash detected, will try to restart... 13:18:53 (5588): No heartbeat from core client for 30 sec - exiting 13:18:54 (5588): No heartbeat from core client for 30 sec - exiting 13:18:55 (5588): No heartbeat from core client for 30 sec - exiting 13:18:56 (5588): No heartbeat from core client for 30 sec - exiting 13:18:57 (5588): No heartbeat from core client for 30 sec - exiting 13:18:58 (5588): No heartbeat from core client for 30 sec - exiting 13:18:59 (5588): No heartbeat from core client for 30 sec - exiting 13:19:01 (5588): No heartbeat from core client for 30 sec - exiting 13:19:02 (5588): No heartbeat from core client for 30 sec - exiting 13:19:03 (5588): No heartbeat from core client for 30 sec - exiting 13:19:04 (5588): No heartbeat from core client for 30 sec - exiting 13:19:05 (5588): No heartbeat from core client for 30 sec - exiting 13:19:06 (5588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2728, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77E871A8 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773A3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Jan 2012 17:54:23	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	518,400	682,993	1.3175
11 Jan 2012 17:57:54	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	492,480	648,883	1.3176
09 Jan 2012 01:21:32	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	466,560	614,649	1.3174
06 Jan 2012 23:15:08	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	440,640	580,342	1.3170
04 Jan 2012 13:02:12	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	414,720	546,937	1.3188
30 Dec 2011 20:42:32	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	388,800	513,871	1.3217
29 Dec 2011 19:54:56	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	362,880	480,369	1.3238
27 Dec 2011 01:05:38	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	336,960	446,522	1.3251
26 Dec 2011 02:59:04	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	311,040	412,034	1.3247
24 Dec 2011 22:26:09	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	285,120	377,926	1.3255
23 Dec 2011 20:32:11	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	259,200	343,671	1.3259
21 Dec 2011 01:05:54	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	233,280	309,812	1.3281
16 Dec 2011 21:25:21	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	207,360	274,910	1.3258
13 Dec 2011 20:24:09	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	181,440	241,336	1.3301
12 Dec 2011 19:59:33	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	155,520	206,680	1.3290
11 Dec 2011 19:13:28	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	129,600	172,307	1.3295
09 Dec 2011 14:54:07	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	103,680	137,881	1.3299
08 Dec 2011 02:41:16	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	77,760	103,551	1.3317
06 Dec 2011 23:37:36	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	51,840	68,985	1.3307
04 Dec 2011 01:58:21	1065767	13679850	hadcm3n_t5id_1940_40_007432602_2	25,920	34,275	1.3223