Task 15600564

Name	hadcm3n_zh85_1880_40_008253697_4
Workunit	8408821
Created	8 Feb 2013, 8:59:32 UTC
Sent	8 Feb 2013, 9:06:57 UTC
Report deadline	10 May 2013, 16:34:08 UTC
Received	15 Mar 2013, 6:05:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1146914
Run time	11 days 15 hours 29 min 44 sec
CPU time	11 days 4 hours 42 min 8 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.08 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3516, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1172, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3520, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3520, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3520, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4560, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4744, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5316, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5240, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5240, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4856, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1620, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1620, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1620, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2956, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5908, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4852, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6048, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5852, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2872, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5528, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4924, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5352, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4284, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4860, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5140, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2932, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2932, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2932, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2932, iMonCtr=1 Model crash detected, will try to restart... CBUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zh85ko.pja0c10 Error converting file to netcdf: dataout/zh85ko.pia0c10 Error converting file to netcdf: dataout/zh85ko.pfa0c10 Error converting file to netcdf: dataout/zh85ko.pca0c10 Error converting file to netcdf: dataout/zh85ko.pba0c10 Error converting file to netcdf: dataout/zh85ko.paa0c10 Error converting file to netcdf: dataout/zh85ka.pha0c10 Error converting file to netcdf: dataout/zh85ka.pga0c10 Error converting file to netcdf: dataout/zh85ka.pea0c10 Error converting file to netcdf: dataout/zh85ka.pda0c10 Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Mar 2013 13:45:11	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	518,400	967,326	1.8660
12 Mar 2013 07:56:17	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	492,480	921,211	1.8706
11 Mar 2013 06:12:09	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	466,560	868,269	1.8610
09 Mar 2013 05:26:29	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	440,640	810,555	1.8395
08 Mar 2013 04:10:00	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	414,720	760,116	1.8328
05 Mar 2013 09:31:05	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	388,800	709,340	1.8244
03 Mar 2013 17:16:13	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	362,880	663,504	1.8284
02 Mar 2013 16:20:53	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	336,960	617,533	1.8327
01 Mar 2013 14:50:45	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	311,040	570,903	1.8355
28 Feb 2013 16:30:41	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	285,120	521,908	1.8305
27 Feb 2013 14:20:23	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	259,200	472,268	1.8220
26 Feb 2013 06:14:30	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	233,280	426,091	1.8265
24 Feb 2013 17:06:59	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	207,360	379,756	1.8314
22 Feb 2013 19:41:46	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	181,440	333,578	1.8385
21 Feb 2013 14:35:33	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	155,520	288,178	1.8530
20 Feb 2013 04:37:13	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	129,600	240,476	1.8555
16 Feb 2013 07:23:39	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	103,680	190,833	1.8406
10 Feb 2013 07:11:34	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	77,760	139,557	1.7947
09 Feb 2013 17:06:08	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	51,840	93,194	1.7977
09 Feb 2013 00:09:37	1146914	15600564	hadcm3n_zh85_1880_40_008253697_4	25,920	47,981	1.8511