Task 15670937

Name	hadcm3n_zlsv_1920_40_008256429_1
Workunit	8411553
Created	18 Mar 2013, 4:48:20 UTC
Sent	18 Mar 2013, 4:48:33 UTC
Report deadline	17 Jun 2013, 12:15:44 UTC
Received	28 Apr 2013, 14:04:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1265248
Run time	11 days 13 hours 20 min 19 sec
CPU time	10 days 6 hours 0 min 4 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.40 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5576, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:48:54 (5324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:58:24 (4876): No heartbeat from core client for 30 sec - exiting 10:58:25 (4876): No heartbeat from core client for 30 sec - exiting 10:58:26 (4876): No heartbeat from core client for 30 sec - exiting 10:58:27 (4876): No heartbeat from core client for 30 sec - exiting 10:58:28 (4876): No heartbeat from core client for 30 sec - exiting 10:58:29 (4876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:39:22 (5188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:13 (5392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:32 (2584): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:16:45 (3888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1744, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... C09:12:16 (5624): No heartbeat from core client for 30 sec - exiting 09:12:17 (5624): No heartbeat from core client for 30 sec - exiting 09:12:19 (5624): No heartbeat from core client for 30 sec - exiting 09:12:20 (5624): No heartbeat from core client for 30 sec - exiting 09:12:21 (5624): No heartbeat from core client for 30 sec - exiting 09:12:22 (5624): No heartbeat from core client for 30 sec - exiting 09:12:23 (5624): No heartbeat from core client for 30 sec - exiting 09:12:24 (5624): No heartbeat from core client for 30 sec - exiting 09:12:25 (5624): No heartbeat from core client for 30 sec - exiting 09:12:26 (5624): No heartbeat from core client for 30 sec - exiting 09:12:27 (5624): No heartbeat from core client for 30 sec - exiting 09:12:28 (5624): No heartbeat from core client for 30 sec - exiting 09:12:29 (5624): No heartbeat from core client for 30 sec - exiting 09:12:30 (5624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2556, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5224, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4432, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4440, iMonCtr=1 Model crash detected, will try to restart... 09:34:06 (4868): No heartbeat from core client for 30 sec - exiting 09:34:08 (4868): No heartbeat from core client for 30 sec - exiting 09:34:09 (4868): No heartbeat from core client for 30 sec - exiting 09:34:10 (4868): No heartbeat from core client for 30 sec - exiting 09:34:11 (4868): No heartbeat from core client for 30 sec - exiting 09:34:12 (4868): No heartbeat from core client for 30 sec - exiting 09:34:13 (4868): No heartbeat from core client for 30 sec - exiting 09:34:14 (4868): No heartbeat from core client for 30 sec - exiting 09:34:15 (4868): No heartbeat from core client for 30 sec - exiting 09:34:16 (4868): No heartbeat from core client for 30 sec - exiting 09:34:17 (4868): No heartbeat from core client for 30 sec - exiting 09:34:18 (4868): No heartbeat from core client for 30 sec - exiting 09:34:19 (4868): No heartbeat from core client for 30 sec - exiting 09:34:20 (4868): No heartbeat from core client for 30 sec - exiting 09:34:21 (4868): No heartbeat from core client for 30 sec - exiting 09:34:22 (4868): No heartbeat from core client for 30 sec - exiting 09:34:23 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:06:12 (5048): No heartbeat from core client for 30 sec - exiting 18:06:13 (5048): No heartbeat from core client for 30 sec - exiting 18:06:14 (5048): No heartbeat from core client for 30 sec - exiting 18:06:15 (5048): No heartbeat from core client for 30 sec - exiting 18:06:16 (5048): No heartbeat from core client for 30 sec - exiting 18:06:17 (5048): No heartbeat from core client for 30 sec - exiting 18:06:18 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... 08:59:01 (5960): No heartbeat from core client for 30 sec - exiting 08:59:03 (5960): No heartbeat from core client for 30 sec - exiting 08:59:04 (5960): No heartbeat from core client for 30 sec - exiting 08:59:05 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:59:06 (5960): No heartbeat from core client for 30 sec - exiting 09:00:07 (3660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2100, iMonCtr=1 Model crash detected, will try to restart... 16:36:04 (4648): No heartbeat from core client for 30 sec - exiting 16:36:05 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:25:24 (5832): No heartbeat from core client for 30 sec - exiting 22:25:25 (5832): No heartbeat from core client for 30 sec - exiting 22:25:26 (5832): No heartbeat from core client for 30 sec - exiting 22:25:27 (5832): No heartbeat from core client for 30 sec - exiting 22:25:28 (5832): No heartbeat from core client for 30 sec - exiting 22:25:29 (5832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:37:42 (5280): No heartbeat from core client for 30 sec - exiting 18:37:43 (5280): No heartbeat from core client for 30 sec - exiting 18:37:44 (5280): No heartbeat from core client for 30 sec - exiting 18:37:45 (5280): No heartbeat from core client for 30 sec - exiting 18:37:46 (5280): No heartbeat from core client for 30 sec - exiting 18:37:47 (5280): No heartbeat from core client for 30 sec - exiting 18:37:48 (5280): No heartbeat from core client for 30 sec - exiting 18:37:49 (5280): No heartbeat from core client for 30 sec - exiting 18:37:50 (5280): No heartbeat from core client for 30 sec - exiting 18:37:51 (5280): No heartbeat from core client for 30 sec - exiting 18:37:52 (5280): No heartbeat from core client for 30 sec - exiting 18:37:53 (5280): No heartbeat from core client for 30 sec - exiting 18:37:54 (5280): No heartbeat from core client for 30 sec - exiting 18:37:55 (5280): No heartbeat from core client for 30 sec - exiting 18:37:56 (5280): No heartbeat from core client for 30 sec - exiting 18:37:57 (5280): No heartbeat from core client for 30 sec - exiting 18:37:58 (5280): No heartbeat from core client for 30 sec - exiting 18:37:59 (5280): No heartbeat from core client for 30 sec - exiting 18:38:00 (5280): No heartbeat from core client for 30 sec - exiting 18:38:01 (5280): No heartbeat from core client for 30 sec - exiting 18:38:02 (5280): No heartbeat from core client for 30 sec - exiting 18:38:03 (5280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:38:04 (5280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6092, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4072, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1936, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1936, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1936, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:04:52 (5360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:00:11 (2540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4612, iMonCtr=1 Model crash detected, will try to restart... 09:13:10 (5896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:02 (6040): No heartbeat from core client for 30 sec - exiting 12:56:03 (6040): No heartbeat from core client for 30 sec - exiting 12:56:04 (6040): No heartbeat from core client for 30 sec - exiting 12:56:05 (6040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:27:39 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:15:40 (6600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:41 (6600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:43:31 (5792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5740, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77727373 read attempt to address 0x40C11215 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77723AB3 read attempt to address 0x40C1121D Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zlsv_1920_40_008256429/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Apr 2013 23:29:52	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	259,200	841,449	3.2463
22 Apr 2013 11:39:50	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	233,280	740,688	3.1751
17 Apr 2013 22:00:20	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	207,360	653,725	3.1526
15 Apr 2013 02:23:20	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	181,440	556,148	3.0652
10 Apr 2013 02:20:49	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	155,520	463,960	2.9833
04 Apr 2013 22:20:37	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	129,600	378,232	2.9185
31 Mar 2013 03:17:43	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	103,680	295,249	2.8477
25 Mar 2013 05:11:21	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	77,760	204,385	2.6284
21 Mar 2013 17:31:04	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	51,840	129,771	2.5033
20 Mar 2013 02:00:23	1265248	15670937	hadcm3n_zlsv_1920_40_008256429_1	25,920	74,629	2.8792