Task 13018500

Name	hadcm3n_t5r9_1940_40_007313363_0
Workunit	7510793
Created	28 Jun 2011, 7:10:00 UTC
Sent	28 Jun 2011, 7:15:26 UTC
Report deadline	27 Sep 2011, 14:42:37 UTC
Received	1 Aug 2011, 13:03:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1156294
Run time	13 days 14 hours 48 min 43 sec
CPU time	12 days 13 hours 35 min 39 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.38 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5208, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2840, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2840, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1244, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4288, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:19:01 (4588): No heartbeat from core client for 30 sec - exiting 07:19:02 (4588): No heartbeat from core client for 30 sec - exiting 07:19:03 (4588): No heartbeat from core client for 30 sec - exiting 07:19:04 (4588): No heartbeat from core client for 30 sec - exiting 07:19:05 (4588): No heartbeat from core client for 30 sec - exiting 07:19:07 (4588): No heartbeat from core client for 30 sec - exiting 07:19:08 (4588): No heartbeat from core client for 30 sec - exiting 07:19:09 (4588): No heartbeat from core client for 30 sec - exiting 07:19:10 (4588): No heartbeat from core client for 30 sec - exiting 07:19:11 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6684, iMonCtr=1 Model crash detected, will try to restart... 08:43:56 (1152): No heartbeat from core client for 30 sec - exiting 08:43:57 (1152): No heartbeat from core client for 30 sec - exiting 08:43:58 (1152): No heartbeat from core client for 30 sec - exiting 08:43:59 (1152): No heartbeat from core client for 30 sec - exiting 08:44:00 (1152): No heartbeat from core client for 30 sec - exiting 08:44:01 (1152): No heartbeat from core client for 30 sec - exiting 08:44:02 (1152): No heartbeat from core client for 30 sec - exiting 08:44:03 (1152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2832, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6516, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4724, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4736, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77713A93 read attempt to address 0x40BB01D3 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x777171F5 read attempt to address 0x40BB01D3 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_t5r9_1940_40_007313363/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Aug 2011 07:38:37	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	518,400	1,068,449	2.0611
29 Jul 2011 11:45:10	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	492,480	1,010,814	2.0525
27 Jul 2011 14:22:29	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	466,560	952,753	2.0421
26 Jul 2011 09:38:25	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	440,640	894,750	2.0306
25 Jul 2011 21:55:05	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	414,720	837,067	2.0184
25 Jul 2011 19:08:20	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	388,800	779,333	2.0045
25 Jul 2011 18:51:51	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	362,880	722,219	1.9902
25 Jul 2011 17:23:58	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	336,960	664,501	1.9720
25 Jul 2011 15:34:53	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	311,040	606,548	1.9501
25 Jul 2011 12:36:16	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	285,120	549,465	1.9271
25 Jul 2011 12:36:16	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	259,200	492,526	1.9002
10 Jul 2011 15:29:25	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	233,280	435,419	1.8665
08 Jul 2011 15:06:23	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	207,360	377,388	1.8200
07 Jul 2011 15:36:39	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	181,440	320,376	1.7657
05 Jul 2011 09:33:13	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	155,520	267,260	1.7185
04 Jul 2011 06:16:44	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	129,600	220,250	1.6995
02 Jul 2011 11:47:42	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	103,680	173,152	1.6701
01 Jul 2011 07:08:04	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	77,760	127,902	1.6448
30 Jun 2011 06:53:44	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	51,840	84,034	1.6210
29 Jun 2011 06:26:38	1156294	13018500	hadcm3n_t5r9_1940_40_007313363_0	25,920	41,127	1.5867