Task 15527892

Name	hadcm3n_zdwg_1880_40_008250949_4
Workunit	8406073
Created	10 Jan 2013, 7:24:00 UTC
Sent	10 Jan 2013, 7:24:03 UTC
Report deadline	11 Apr 2013, 14:51:14 UTC
Received	17 Feb 2013, 23:55:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1254207
Run time	5 days 11 hours 39 min 21 sec
CPU time	4 days 19 hours 6 min 40 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.15 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:23:09 (2172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:28:46 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:22:31 (7196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:07:08 (6616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:33:31 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:00:33 (5756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:35:24 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:53:58 (5404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:49:03 (9280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:16:19 (10260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:29:44 (9556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10764, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 12:58:59 (10620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:27:24 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10756, iMonCtr=1 Model crash detected, will try to restart... 17:15:55 (3728): No heartbeat from core client for 30 sec - exiting 17:15:56 (3728): No heartbeat from core client for 30 sec - exiting 17:15:57 (3728): No heartbeat from core client for 30 sec - exiting 17:15:58 (3728): No heartbeat from core client for 30 sec - exiting 17:15:59 (3728): No heartbeat from core client for 30 sec - exiting 17:16:00 (3728): No heartbeat from core client for 30 sec - exiting 17:16:02 (3728): No heartbeat from core client for 30 sec - exiting 17:16:03 (3728): No heartbeat from core client for 30 sec - exiting 17:16:04 (3728): No heartbeat from core client for 30 sec - exiting 17:16:05 (3728): No heartbeat from core client for 30 sec - exiting 17:16:06 (3728): No heartbeat from core client for 30 sec - exiting 17:16:08 (3728): No heartbeat from core client for 30 sec - exiting 17:16:09 (3728): No heartbeat from core client for 30 sec - exiting 17:16:10 (3728): No heartbeat from core client for 30 sec - exiting 17:16:11 (3728): No heartbeat from core client for 30 sec - exiting 17:16:13 (3728): No heartbeat from core client for 30 sec - exiting 17:16:14 (3728): No heartbeat from core client for 30 sec - exiting 17:16:15 (3728): No heartbeat from core client for 30 sec - exiting 17:16:16 (3728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:47:56 (3920): No heartbeat from core client for 30 sec - exiting 14:47:57 (3920): No heartbeat from core client for 30 sec - exiting 14:47:59 (3920): No heartbeat from core client for 30 sec - exiting 14:48:00 (3920): No heartbeat from core client for 30 sec - exiting 14:48:01 (3920): No heartbeat from core client for 30 sec - exiting 14:48:02 (3920): No heartbeat from core client for 30 sec - exiting 14:48:03 (3920): No heartbeat from core client for 30 sec - exiting 14:48:04 (3920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:49:10 (3304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zdwgko.pj88c10 Error converting file to netcdf: dataout/zdwgko.pi88c10 Error converting file to netcdf: dataout/zdwgko.pf88c10 Error converting file to netcdf: dataout/zdwgka.ph88c10 Error converting file to netcdf: dataout/zdwgka.pg88c10 Error converting file to netcdf: dataout/zdwgka.pe88c10 Error converting file to netcdf: dataout/zdwgka.pd88c10 07:27:31 (6364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zdwgko.pj88c10 Error converting file to netcdf: dataout/zdwgko.pi88c10 Error converting file to netcdf: dataout/zdwgko.pf88c10 Error converting file to netcdf: dataout/zdwgka.ph88c10 Error converting file to netcdf: dataout/zdwgka.pg88c10 Error converting file to netcdf: dataout/zdwgka.pe88c10 Error converting file to netcdf: dataout/zdwgka.pd88c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:41:41 (7124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7980, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:18:07 (6884): No heartbeat from core client for 30 sec - exiting 17:18:08 (6884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:18:09 (6884): No heartbeat from core client for 30 sec - exiting 17:18:10 (6884): No heartbeat from core client for 30 sec - exiting 17:18:11 (6884): No heartbeat from core client for 30 sec - exiting 17:18:12 (6884): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77CD3242 read attempt to address 0x40BF824A Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77CD3242 read attempt to address 0x40BF824A Engaging BOINC Windows Runtime Debugger... Cannot serialize file G:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdwg_1880_40_008250949/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Feb 2013 07:07:19	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	259,200	413,434	1.5950
04 Feb 2013 08:37:47	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	233,280	369,894	1.5856
29 Jan 2013 23:31:58	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	207,360	329,996	1.5914
28 Jan 2013 02:28:30	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	181,440	286,036	1.5765
22 Jan 2013 07:50:19	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	155,520	243,263	1.5642
18 Jan 2013 03:48:42	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	129,600	202,394	1.5617
16 Jan 2013 01:25:56	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	103,680	163,949	1.5813
14 Jan 2013 20:49:38	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	77,760	124,615	1.6026
14 Jan 2013 08:48:47	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	51,840	83,633	1.6133
11 Jan 2013 03:43:27	1254207	15527892	hadcm3n_zdwg_1880_40_008250949_4	25,920	42,087	1.6237