Task 13419015

Name	hadcm3n_yeh2_1940_40_007461960_1
Workunit	7659463
Created	25 Sep 2011, 0:32:34 UTC
Sent	25 Sep 2011, 0:39:55 UTC
Report deadline	25 Dec 2011, 8:07:06 UTC
Received	25 Oct 2011, 16:00:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1120966
Run time	13 days 14 hours 19 min 34 sec
CPU time	13 days 5 hours 55 min 45 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	2.59 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(194,0xa09f1540) malloc: * error for object 0x801004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(194,0xa09f1540) malloc: * error for object 0x801000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=194, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=194, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=194, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=194, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=194, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=194, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2011 13:58:33	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	648,000	1,114,110	1.7193
31 Oct 2011 13:58:32	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	622,080	1,064,870	1.7118
31 Oct 2011 13:58:32	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	596,160	1,015,941	1.7041
31 Oct 2011 13:58:32	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	570,240	966,836	1.6955
31 Oct 2011 13:58:32	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	544,320	917,951	1.6864
14 Oct 2011 20:45:23	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	518,400	868,235	1.6748
14 Oct 2011 06:32:20	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	492,480	818,166	1.6613
06 Oct 2011 00:15:01	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	466,560	772,748	1.6563
05 Oct 2011 13:30:06	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	440,640	733,642	1.6649
05 Oct 2011 02:08:18	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	414,720	694,270	1.6741
04 Oct 2011 14:47:23	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	388,800	654,903	1.6844
04 Oct 2011 03:26:37	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	362,880	615,380	1.6958
03 Oct 2011 16:07:12	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	336,960	576,144	1.7098
03 Oct 2011 04:32:59	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	311,040	536,661	1.7254
02 Oct 2011 17:08:44	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	285,120	496,675	1.7420
02 Oct 2011 05:53:26	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	259,200	457,401	1.7647
01 Oct 2011 18:48:12	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	233,280	417,882	1.7913
01 Oct 2011 07:42:40	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	207,360	378,724	1.8264
30 Sep 2011 20:31:07	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	181,440	339,762	1.8726
30 Sep 2011 08:15:27	1120966	13419015	hadcm3n_yeh2_1940_40_007461960_1	155,520	297,805	1.9149