Task 15475963

Name	hadcm3n_zdr8_1880_40_008245352_2
Workunit	8400476
Created	14 Dec 2012, 1:45:33 UTC
Sent	14 Dec 2012, 1:45:38 UTC
Report deadline	15 Mar 2013, 9:12:49 UTC
Received	4 Jan 2013, 20:56:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1257938
Run time	12 days 21 hours 46 min 25 sec
CPU time	11 days 4 hours 31 min 58 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.67 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3908, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4756, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4996, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4976, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2524, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zdr8_1880_40_008245352/dataout/ocean_restart.day after 11 attempts Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2524, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jan 2013 17:01:16	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	518,400	966,778	1.8649
01 Jan 2013 20:14:40	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	492,480	919,163	1.8664
31 Dec 2012 19:02:00	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	466,560	870,715	1.8662
30 Dec 2012 21:45:42	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	440,640	824,693	1.8716
29 Dec 2012 22:46:52	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	414,720	775,720	1.8705
29 Dec 2012 03:24:11	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	388,800	722,517	1.8583
28 Dec 2012 11:45:08	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	362,880	671,192	1.8496
27 Dec 2012 10:44:47	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	336,960	624,053	1.8520
26 Dec 2012 14:50:16	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	311,040	575,629	1.8507
25 Dec 2012 18:01:48	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	285,120	526,106	1.8452
24 Dec 2012 23:30:20	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	259,200	477,116	1.8407
23 Dec 2012 19:38:17	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	233,280	428,772	1.8380
22 Dec 2012 11:14:56	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	207,360	380,816	1.8365
21 Dec 2012 10:52:12	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	181,440	334,600	1.8441
19 Dec 2012 12:47:42	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	155,520	289,219	1.8597
18 Dec 2012 18:31:29	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	129,600	238,171	1.8377
18 Dec 2012 03:05:42	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	103,680	188,236	1.8155
17 Dec 2012 12:15:06	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	77,760	138,413	1.7800
16 Dec 2012 08:24:31	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	51,840	89,937	1.7349
15 Dec 2012 00:26:48	1257938	15475963	hadcm3n_zdr8_1880_40_008245352_2	25,920	45,668	1.7619