Task 12736601

Name	hadcm3n_o1u1_1900_40_007197708_0
Workunit	7395988
Created	28 Mar 2011, 14:01:40 UTC
Sent	1 Apr 2011, 11:57:55 UTC
Report deadline	1 Jul 2011, 19:25:06 UTC
Received	1 Jul 2011, 13:20:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	956479
Run time	38 days 14 hours 15 min 47 sec
CPU time	23 days 2 hours 42 min 38 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2240, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 16:54:05 (3240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:54:57 (3240): No heartbeat from core client for 30 sec - exiting 16:56:22 (3240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:14:51 (3932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:40:56 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:55:15 (8032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:58:54 (8240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:03:45 (8160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:40:33 (2844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:40:34 (2844): No heartbeat from core client for 30 sec - exiting 15:40:35 (2844): No heartbeat from core client for 30 sec - exiting 15:40:36 (2844): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4304, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=1 Model crash detected, will try to restart... 17:08:00 (4904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:31:48 (6068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6448, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:15:16 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:17 (4852): No heartbeat from core client for 30 sec - exiting 11:15:18 (4852): No heartbeat from core client for 30 sec - exiting 11:15:19 (4852): No heartbeat from core client for 30 sec - exiting 11:15:20 (4852): No heartbeat from core client for 30 sec - exiting 11:15:21 (4852): No heartbeat from core client for 30 sec - exiting 11:15:22 (4852): No heartbeat from core client for 30 sec - exiting 11:15:23 (4852): No heartbeat from core client for 30 sec - exiting 11:15:24 (4852): No heartbeat from core client for 30 sec - exiting 11:15:26 (4852): No heartbeat from core client for 30 sec - exiting 11:15:27 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77062AB1 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77062AB1 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1u1_1900_40_007197708/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Jul 2011 08:38:46	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	1,036,800	1,996,264	1.9254
27 Jun 2011 07:48:31	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	1,010,880	1,947,448	1.9265
25 Jun 2011 15:41:09	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	984,960	1,897,509	1.9265
23 Jun 2011 18:36:13	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	959,040	1,849,138	1.9281
21 Jun 2011 20:55:56	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	933,120	1,801,654	1.9308
20 Jun 2011 20:01:37	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	907,200	1,753,771	1.9332
20 Jun 2011 05:18:45	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	881,280	1,705,525	1.9353
19 Jun 2011 22:49:29	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	855,360	1,658,231	1.9386
16 Jun 2011 17:36:35	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	829,440	1,608,853	1.9397
15 Jun 2011 10:32:23	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	803,520	1,560,384	1.9419
13 Jun 2011 14:16:26	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	777,600	1,510,354	1.9423
11 Jun 2011 07:50:13	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	751,680	1,459,688	1.9419
10 Jun 2011 05:39:12	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	725,760	1,410,320	1.9432
09 Jun 2011 12:21:47	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	699,840	1,361,161	1.9450
08 Jun 2011 10:26:53	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	673,920	1,312,404	1.9474
06 Jun 2011 09:05:21	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	648,000	1,262,637	1.9485
02 Jun 2011 09:03:08	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	622,080	1,210,997	1.9467
30 May 2011 20:14:08	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	596,160	1,162,881	1.9506
29 May 2011 06:31:50	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	570,240	1,113,622	1.9529
27 May 2011 17:14:43	956479	12736601	hadcm3n_o1u1_1900_40_007197708_0	544,320	1,064,899	1.9564