Task 15900027

Name	hadcm3n_n5fa_1960_40_008394224_1
Workunit	8545083
Created	22 Jul 2013, 13:38:33 UTC
Sent	22 Jul 2013, 13:38:37 UTC
Report deadline	21 Oct 2013, 21:05:48 UTC
Received	21 Aug 2013, 22:19:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1181788
Run time	15 days 2 hours 34 min 56 sec
CPU time	14 days 23 hours 50 min 31 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.49 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:07:58 (1744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n5fako.pjh0c10 Error converting file to netcdf: dataout/n5fako.pih0c10 Error converting file to netcdf: dataout/n5fako.pfh0c10 Error converting file to netcdf: dataout/n5fako.pch0c10 Error converting file to netcdf: dataout/n5fako.pbh0c10 Error converting file to netcdf: dataout/n5fako.pah0c10 Error converting file to netcdf: dataout/n5faka.phh0c10 Error converting file to netcdf: dataout/n5faka.pgh0c10 Error converting file to netcdf: dataout/n5faka.peh0c10 Error converting file to netcdf: dataout/n5faka.pdh0c10 17:27:52 (6420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:27:53 (6420): No heartbeat from core client for 30 sec - exiting 17:27:54 (6420): No heartbeat from core client for 30 sec - exiting 17:27:55 (6420): No heartbeat from core client for 30 sec - exiting 17:27:56 (6420): No heartbeat from core client for 30 sec - exiting 17:27:57 (6420): No heartbeat from core client for 30 sec - exiting 17:27:58 (6420): No heartbeat from core client for 30 sec - exiting 17:27:59 (6420): No heartbeat from core client for 30 sec - exiting 17:28:00 (6420): No heartbeat from core client for 30 sec - exiting 17:28:01 (6420): No heartbeat from core client for 30 sec - exiting 17:28:02 (6420): No heartbeat from core client for 30 sec - exiting 17:28:40 (6908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:28:41 (6908): No heartbeat from core client for 30 sec - exiting 17:28:42 (6908): No heartbeat from core client for 30 sec - exiting 17:28:43 (6908): No heartbeat from core client for 30 sec - exiting 17:28:44 (6908): No heartbeat from core client for 30 sec - exiting 17:28:45 (6908): No heartbeat from core client for 30 sec - exiting 17:28:46 (6908): No heartbeat from core client for 30 sec - exiting 17:28:47 (6908): No heartbeat from core client for 30 sec - exiting 17:28:48 (6908): No heartbeat from core client for 30 sec - exiting 17:28:49 (6908): No heartbeat from core client for 30 sec - exiting 17:28:50 (6908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5216, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5216, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5216, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5216, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5216, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5216, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	777,600	1,283,644	1.6508
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	751,680	1,237,979	1.6469
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	725,760	1,192,491	1.6431
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	699,840	1,147,624	1.6398
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	673,920	1,103,696	1.6377
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	648,000	1,059,027	1.6343
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	622,080	1,014,959	1.6316
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	596,160	970,566	1.6280
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	570,240	926,521	1.6248
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	544,320	882,767	1.6218
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	518,400	840,284	1.6209
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	492,480	798,039	1.6204
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	466,560	755,312	1.6189
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	440,640	712,992	1.6181
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	414,720	670,811	1.6175
21 Aug 2013 22:20:18	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	388,800	628,785	1.6172
30 Jul 2013 09:46:31	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	362,880	586,408	1.6160
30 Jul 2013 09:46:31	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	336,960	545,066	1.6176
29 Jul 2013 14:02:45	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	311,040	502,794	1.6165
29 Jul 2013 14:02:44	1181788	15900027	hadcm3n_n5fa_1960_40_008394224_1	285,120	460,512	1.6152