Task 15493821

Name	hadcm3n_3frc_1940_40_008264788_0
Workunit	8419912
Created	21 Dec 2012, 10:09:47 UTC
Sent	21 Dec 2012, 10:19:33 UTC
Report deadline	22 Mar 2013, 17:46:44 UTC
Received	2 Feb 2013, 0:14:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1284994
Run time	19 days 4 hours 5 min 19 sec
CPU time	16 days 13 hours 40 min
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.42</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3524, iMonCtr=1 Model crash detected, will try to restart... 21:07:11 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3836, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5288, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4040, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4412, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3frcko.pjh5c10 Error converting file to netcdf: dataout/3frcko.pih5c10 Error converting file to netcdf: dataout/3frcko.pfh5c10 Error converting file to netcdf: dataout/3frcka.phh5c10 Error converting file to netcdf: dataout/3frcka.pgh5c10 Error converting file to netcdf: dataout/3frcka.peh5c10 Error converting file to netcdf: dataout/3frcka.pdh5c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773D3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3frc_1940_40_008264788/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Feb 2013 00:18:38	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	1,036,800	1,431,593	1.3808
30 Jan 2013 12:10:00	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	1,010,880	1,400,364	1.3853
27 Jan 2013 21:47:09	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	984,960	1,370,802	1.3917
27 Jan 2013 04:55:21	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	959,040	1,341,006	1.3983
26 Jan 2013 08:24:51	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	933,120	1,311,389	1.4054
25 Jan 2013 23:33:19	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	907,200	1,281,977	1.4131
23 Jan 2013 12:25:04	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	881,280	1,251,791	1.4204
21 Jan 2013 09:46:31	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	855,360	1,219,482	1.4257
19 Jan 2013 23:42:49	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	829,440	1,183,965	1.4274
19 Jan 2013 11:25:40	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	803,520	1,148,516	1.4294
18 Jan 2013 13:05:16	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	777,600	1,113,017	1.4313
15 Jan 2013 12:50:11	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	751,680	1,076,713	1.4324
14 Jan 2013 03:57:33	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	725,760	1,039,830	1.4327
13 Jan 2013 01:43:21	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	699,840	1,002,773	1.4329
12 Jan 2013 02:48:12	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	673,920	965,952	1.4333
11 Jan 2013 02:28:14	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	648,000	929,015	1.4337
08 Jan 2013 12:25:08	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	622,080	890,301	1.4312
06 Jan 2013 05:40:23	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	596,160	852,207	1.4295
05 Jan 2013 06:00:13	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	570,240	814,794	1.4289
03 Jan 2013 10:21:16	1130983	15493821	hadcm3n_3frc_1940_40_008264788_0	544,320	776,536	1.4266