Task 15615281

Name	hadcm3n_zdj6_1880_40_008247512_2
Workunit	8402636
Created	20 Feb 2013, 15:33:46 UTC
Sent	20 Feb 2013, 15:33:50 UTC
Report deadline	22 May 2013, 23:01:01 UTC
Received	22 Jun 2013, 21:01:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1260838
Run time	14 days 12 hours 0 min 49 sec
CPU time	13 days 10 hours 42 min 27 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.63 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:34:33 (5940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on zdj6ko.da885q0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4636, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7707FF2B write attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on zdj6ko.da91270 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... CBUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zdj6ko.pj91c10 Error converting file to netcdf: dataout/zdj6ko.pi91c10 Error converting file to netcdf: dataout/zdj6ko.pf91c10 Error converting file to netcdf: dataout/zdj6ka.ph91c10 Error converting file to netcdf: dataout/zdj6ka.pg91c10 Error converting file to netcdf: dataout/zdj6ka.pe91c10 Error converting file to netcdf: dataout/zdj6ka.pd91c10 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zdj6ko.pj91c10 Error converting file to netcdf: dataout/zdj6ko.pi91c10 Error converting file to netcdf: dataout/zdj6ko.pf91c10 Error converting file to netcdf: dataout/zdj6ka.ph91c10 Error converting file to netcdf: dataout/zdj6ka.pg91c10 Error converting file to netcdf: dataout/zdj6ka.pe91c10 Error converting file to netcdf: dataout/zdj6ka.pd91c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on zdj6ko.da92340 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4572, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on zdj6ko.da934b0 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=924, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4584, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4256, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Jun 2013 21:06:13	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	777,600	1,161,743	1.4940
10 Jun 2013 09:58:20	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	751,680	1,123,399	1.4945
06 Jun 2013 16:55:04	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	725,760	1,086,005	1.4964
04 Jun 2013 18:19:43	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	699,840	1,047,443	1.4967
03 Jun 2013 10:14:45	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	673,920	1,010,172	1.4989
02 Jun 2013 10:14:14	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	648,000	972,180	1.5003
01 Jun 2013 06:06:58	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	622,080	933,868	1.5012
31 May 2013 17:36:47	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	596,160	894,878	1.5011
30 May 2013 20:49:45	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	570,240	856,954	1.5028
26 May 2013 22:50:30	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	544,320	818,611	1.5039
26 May 2013 10:50:47	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	518,400	779,898	1.5044
25 May 2013 23:02:06	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	492,480	741,644	1.5059
25 May 2013 10:47:45	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	466,560	701,954	1.5045
24 May 2013 22:48:30	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	440,640	663,462	1.5057
24 May 2013 10:28:26	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	414,720	624,630	1.5061
23 May 2013 22:29:18	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	388,800	586,488	1.5085
23 May 2013 10:43:44	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	362,880	547,843	1.5097
22 May 2013 16:26:29	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	336,960	509,256	1.5113
02 May 2013 04:29:37	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	311,040	470,707	1.5133
25 Apr 2013 07:31:32	1260838	15615281	hadcm3n_zdj6_1880_40_008247512_2	285,120	430,353	1.5094