Task 13668739

Name	hadcm3n_o3gl_1940_40_007545922_3
Workunit	7743154
Created	29 Nov 2011, 2:38:42 UTC
Sent	1 Dec 2011, 17:03:26 UTC
Report deadline	2 Mar 2012, 0:30:37 UTC
Received	12 Jan 2012, 18:47:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1152893
Run time	9 days 16 hours 54 min 51 sec
CPU time	8 days 1 hours 24 min 36 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3988, iMonCtr=1 Model crash detected, will try to restart... 14:36:47 (4024): No heartbeat from core client for 30 sec - exiting 14:36:48 (4024): No heartbeat from core client for 30 sec - exiting 14:36:49 (4024): No heartbeat from core client for 30 sec - exiting 14:36:50 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:01:49 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:42:21 (3836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:49:46 (2072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3756, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3756, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3756, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3756, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3756, iMonCtr=1 Model crash detected, will try to restart... 12:31:41 (3764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:45:58 (3096): No heartbeat from core client for 30 sec - exiting 12:45:59 (3096): No heartbeat from core client for 30 sec - exiting 12:46:00 (3096): No heartbeat from core client for 30 sec - exiting 12:46:01 (3096): No heartbeat from core client for 30 sec - exiting 12:46:03 (3096): No heartbeat from core client for 30 sec - exiting 12:46:04 (3096): No heartbeat from core client for 30 sec - exiting 12:46:05 (3096): No heartbeat from core client for 30 sec - exiting 12:46:06 (3096): No heartbeat from core client for 30 sec - exiting 12:46:07 (3096): No heartbeat from core client for 30 sec - exiting 12:46:08 (3096): No heartbeat from core client for 30 sec - exiting 12:46:09 (3096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:10 (3096): No heartbeat from core client for 30 sec - exiting 01:02:50 (3176): No heartbeat from core client for 30 sec - exiting 01:02:51 (3176): No heartbeat from core client for 30 sec - exiting 01:02:52 (3176): No heartbeat from core client for 30 sec - exiting 01:02:53 (3176): No heartbeat from core client for 30 sec - exiting 01:02:54 (3176): No heartbeat from core client for 30 sec - exiting 01:02:55 (3176): No heartbeat from core client for 30 sec - exiting 01:02:56 (3176): No heartbeat from core client for 30 sec - exiting 01:02:57 (3176): No heartbeat from core client for 30 sec - exiting 01:02:58 (3176): No heartbeat from core client for 30 sec - exiting 01:02:59 (3176): No heartbeat from core client for 30 sec - exiting 01:03:00 (3176): No heartbeat from core client for 30 sec - exiting 01:03:01 (3176): No heartbeat from core client for 30 sec - exiting 01:03:02 (3176): No heartbeat from core client for 30 sec - exiting 01:03:03 (3176): No heartbeat from core client for 30 sec - exiting 01:03:04 (3176): No heartbeat from core client for 30 sec - exiting 01:03:05 (3176): No heartbeat from core client for 30 sec - exiting 01:03:06 (3176): No heartbeat from core client for 30 sec - exiting 01:03:07 (3176): No heartbeat from core client for 30 sec - exiting 01:03:09 (3176): No heartbeat from core client for 30 sec - exiting 01:03:10 (3176): No heartbeat from core client for 30 sec - exiting 01:03:11 (3176): No heartbeat from core client for 30 sec - exiting 01:03:12 (3176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:19 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:05:56 (4088): No heartbeat from core client for 30 sec - exiting 20:05:58 (4088): No heartbeat from core client for 30 sec - exiting 20:05:59 (4088): No heartbeat from core client for 30 sec - exiting 20:06:00 (4088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4132, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 18:27:22 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4752, iMonCtr=1 Model crash detected, will try to restart... 18:22:47 (1780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:02:35 (3428): No heartbeat from core client for 30 sec - exiting 12:02:36 (3428): No heartbeat from core client for 30 sec - exiting 12:02:37 (3428): No heartbeat from core client for 30 sec - exiting 12:02:38 (3428): No heartbeat from core client for 30 sec - exiting 12:02:40 (3428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5048, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:56:16 (4008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... C10:51:26 (3948): No heartbeat from core client for 30 sec - exiting 10:51:27 (3948): No heartbeat from core client for 30 sec - exiting 10:51:28 (3948): No heartbeat from core client for 30 sec - exiting 10:51:29 (3948): No heartbeat from core client for 30 sec - exiting 10:51:31 (3948): No heartbeat from core client for 30 sec - exiting 10:51:32 (3948): No heartbeat from core client for 30 sec - exiting 10:51:33 (3948): No heartbeat from core client for 30 sec - exiting 10:51:34 (3948): No heartbeat from core client for 30 sec - exiting 10:51:35 (3948): No heartbeat from core client for 30 sec - exiting 10:51:36 (3948): No heartbeat from core client for 30 sec - exiting 10:51:37 (3948): No heartbeat from core client for 30 sec - exiting 10:51:38 (3948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:14:58 (3568): No heartbeat from core client for 30 sec - exiting 00:15:00 (3568): No heartbeat from core client for 30 sec - exiting 00:15:01 (3568): No heartbeat from core client for 30 sec - exiting 00:15:02 (3568): No heartbeat from core client for 30 sec - exiting 00:15:03 (3568): No heartbeat from core client for 30 sec - exiting 00:15:04 (3568): No heartbeat from core client for 30 sec - exiting 00:15:05 (3568): No heartbeat from core client for 30 sec - exiting 00:15:06 (3568): No heartbeat from core client for 30 sec - exiting 00:15:07 (3568): No heartbeat from core client for 30 sec - exiting 00:15:08 (3568): No heartbeat from core client for 30 sec - exiting 00:15:09 (3568): No heartbeat from core client for 30 sec - exiting 00:15:10 (3568): No heartbeat from core client for 30 sec - exiting 00:15:11 (3568): No heartbeat from core client for 30 sec - exiting 00:15:12 (3568): No heartbeat from core client for 30 sec - exiting 00:15:13 (3568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:00:36 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:58:38 (3652): No heartbeat from core client for 30 sec - exiting 16:58:39 (3652): No heartbeat from core client for 30 sec - exiting 16:58:40 (3652): No heartbeat from core client for 30 sec - exiting 16:58:41 (3652): No heartbeat from core client for 30 sec - exiting 16:58:42 (3652): No heartbeat from core client for 30 sec - exiting 16:58:43 (3652): No heartbeat from core client for 30 sec - exiting 16:58:44 (3652): No heartbeat from core client for 30 sec - exiting 16:58:45 (3652): No heartbeat from core client for 30 sec - exiting 16:58:46 (3652): No heartbeat from core client for 30 sec - exiting 16:58:47 (3652): No heartbeat from core client for 30 sec - exiting 16:58:48 (3652): No heartbeat from core client for 30 sec - exiting 16:58:49 (3652): No heartbeat from core client for 30 sec - exiting 16:58:50 (3652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:02:20 (4352): No heartbeat from core client for 30 sec - exiting 12:02:22 (4352): No heartbeat from core client for 30 sec - exiting 12:02:23 (4352): No heartbeat from core client for 30 sec - exiting 12:02:24 (4352): No heartbeat from core client for 30 sec - exiting 12:02:25 (4352): No heartbeat from core client for 30 sec - exiting 12:02:26 (4352): No heartbeat from core client for 30 sec - exiting 12:02:27 (4352): No heartbeat from core client for 30 sec - exiting 12:02:28 (4352): No heartbeat from core client for 30 sec - exiting 12:02:29 (4352): No heartbeat from core client for 30 sec - exiting 12:02:30 (4352): No heartbeat from core client for 30 sec - exiting 12:02:31 (4352): No heartbeat from core client for 30 sec - exiting 12:02:32 (4352): No heartbeat from core client for 30 sec - exiting 12:02:33 (4352): No heartbeat from core client for 30 sec - exiting 12:02:34 (4352): No heartbeat from core client for 30 sec - exiting 12:02:35 (4352): No heartbeat from core client for 30 sec - exiting 12:02:36 (4352): No heartbeat from core client for 30 sec - exiting 12:02:37 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:56:12 (3564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3992, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77DFB84B write attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 18:30:57 (3872): No heartbeat from core client for 30 sec - exiting 18:30:58 (3872): No heartbeat from core client for 30 sec - exiting 18:30:59 (3872): No heartbeat from core client for 30 sec - exiting 18:31:00 (3872): No heartbeat from core client for 30 sec - exiting 18:31:01 (3872): No heartbeat from core client for 30 sec - exiting 18:31:02 (3872): No heartbeat from core client for 30 sec - exiting 18:31:03 (3872): No heartbeat from core client for 30 sec - exiting 18:31:04 (3872): No heartbeat from core client for 30 sec - exiting 18:31:05 (3872): No heartbeat from core client for 30 sec - exiting 18:31:06 (3872): No heartbeat from core client for 30 sec - exiting 18:31:07 (3872): No heartbeat from core client for 30 sec - exiting 18:31:08 (3872): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77003A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Jan 2012 19:40:17	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	518,400	684,421	1.3203
08 Jan 2012 22:15:37	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	492,480	650,693	1.3213
06 Jan 2012 22:39:59	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	466,560	617,228	1.3229
04 Jan 2012 20:50:55	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	440,640	583,508	1.3242
28 Dec 2011 18:06:04	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	414,720	550,173	1.3266
23 Dec 2011 17:59:19	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	388,800	516,636	1.3288
22 Dec 2011 19:03:33	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	362,880	482,662	1.3301
20 Dec 2011 00:30:47	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	336,960	447,940	1.3294
17 Dec 2011 19:17:50	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	311,040	413,004	1.3278
17 Dec 2011 08:13:48	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	285,120	378,585	1.3278
16 Dec 2011 17:29:16	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	259,200	344,315	1.3284
15 Dec 2011 18:25:23	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	233,280	309,720	1.3277
13 Dec 2011 17:23:49	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	207,360	275,072	1.3265
11 Dec 2011 18:41:55	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	181,440	240,580	1.3259
10 Dec 2011 16:27:48	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	155,520	206,524	1.3280
09 Dec 2011 01:10:50	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	129,600	172,165	1.3284
06 Dec 2011 18:49:16	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	103,680	137,591	1.3271
04 Dec 2011 18:22:36	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	77,760	103,167	1.3267
03 Dec 2011 19:41:03	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	51,840	68,740	1.3260
02 Dec 2011 16:39:04	1152893	13668739	hadcm3n_o3gl_1940_40_007545922_3	25,920	34,203	1.3196