Task 12931850

Name	hadcm3n_o612_1940_40_007266440_2
Workunit	7464680
Created	4 Jun 2011, 4:53:00 UTC
Sent	4 Jun 2011, 4:53:10 UTC
Report deadline	3 Sep 2011, 12:20:21 UTC
Received	30 Jun 2011, 9:38:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1112974
Run time	6 days 1 hours 52 min 38 sec
CPU time	5 days 21 hours 39 min 14 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3228, selfPID=3228, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:42:03 (3564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D6210E8 read attempt to address 0x4142586F Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3536, selfPID=3536, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... 00:47:01 (3984): Can't acquire lockfile (32) - waiting 35s Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... 09:02:51 (3524): Can't acquire lockfile (32) - waiting 35s Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... 05:20:26 (3432): No heartbeat from core client for 30 sec - exiting 05:20:27 (3432): No heartbeat from core client for 30 sec - exiting 05:20:28 (3432): No heartbeat from core client for 30 sec - exiting 05:20:29 (3432): No heartbeat from core client for 30 sec - exiting 05:20:30 (3432): No heartbeat from core client for 30 sec - exiting 12:20:35 (3432): No heartbeat from core client for 30 sec - exiting 12:20:36 (3432): No heartbeat from core client for 30 sec - exiting 12:20:37 (3432): No heartbeat from core client for 30 sec - exiting 12:20:38 (3432): No heartbeat from core client for 30 sec - exiting 12:20:39 (3432): No heartbeat from core client for 30 sec - exiting 12:20:40 (3432): No heartbeat from core client for 30 sec - exiting 19:45:56 (3548): Can't acquire lockfile (32) - waiting 35s Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7D621413 read attempt to address 0x40651F42 Engaging BOINC Windows Runtime Debugger... cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o612_1940_40_007266440/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o612_1940_40_007266440/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o612_1940_40_007266440/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o612_1940_40_007266440/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o612_1940_40_007266440/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o612_1940_40_007266440/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Jun 2011 19:13:47	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	259,200	509,861	1.9671
12 Jun 2011 07:07:38	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	233,280	463,586	1.9873
11 Jun 2011 17:59:39	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	207,360	417,693	2.0143
11 Jun 2011 02:52:23	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	181,440	364,833	2.0108
10 Jun 2011 12:50:41	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	155,520	312,514	2.0095
09 Jun 2011 19:46:17	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	129,600	260,629	2.0110
09 Jun 2011 04:59:21	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	103,680	207,694	2.0032
08 Jun 2011 13:54:26	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	77,760	154,143	1.9823
07 Jun 2011 21:21:25	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	51,840	105,380	2.0328
07 Jun 2011 05:52:55	1112974	12931850	hadcm3n_o612_1940_40_007266440_2	25,920	53,426	2.0612