Task 13617406

Name	hadcm3n_ybe8_1980_40_007536873_2
Workunit	7734105
Created	8 Nov 2011, 3:55:09 UTC
Sent	8 Nov 2011, 3:59:45 UTC
Report deadline	7 Feb 2012, 11:26:56 UTC
Received	30 Dec 2011, 10:29:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1117241
Run time	9 days 20 hours 19 min 44 sec
CPU time	8 days 11 hours 19 min 30 sec
Validate state	Invalid
Credit	5,909.76
Device peak FLOPS	2.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1028, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3536, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3456, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3452, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3536, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3604, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3428, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3740, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3412, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3536, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3588, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3528, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3500, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model cBUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashedCalled boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Dec 2011 21:31:12	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	492,480	788,757	1.6016
29 Dec 2011 11:24:42	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	466,560	752,371	1.6126
24 Dec 2011 05:19:19	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	440,640	715,030	1.6227
21 Dec 2011 04:31:56	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	414,720	676,919	1.6322
19 Dec 2011 02:16:13	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	388,800	637,740	1.6403
16 Dec 2011 08:06:21	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	362,880	596,664	1.6442
13 Dec 2011 05:39:46	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	336,960	555,908	1.6498
10 Dec 2011 05:00:18	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	311,040	514,853	1.6553
07 Dec 2011 06:15:08	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	285,120	474,256	1.6634
03 Dec 2011 05:45:35	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	259,200	433,995	1.6744
30 Nov 2011 07:16:11	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	233,280	395,308	1.6946
28 Nov 2011 01:38:27	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	207,360	357,556	1.7243
25 Nov 2011 09:01:02	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	181,440	316,037	1.7418
23 Nov 2011 05:06:47	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	155,520	270,693	1.7406
19 Nov 2011 09:43:52	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	129,600	224,569	1.7328
17 Nov 2011 04:42:18	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	103,680	178,904	1.7255
16 Nov 2011 04:22:56	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	77,760	132,855	1.7085
16 Nov 2011 04:22:56	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	51,840	89,652	1.7294
10 Nov 2011 02:25:22	1117241	13617406	hadcm3n_ybe8_1980_40_007536873_2	25,920	44,944	1.7340