Task 12818821

Name	hadcm3n_p1fu_1900_40_007219450_0
Workunit	7417690
Created	26 Apr 2011, 15:17:39 UTC
Sent	3 May 2011, 23:14:25 UTC
Report deadline	3 Aug 2011, 6:41:36 UTC
Received	26 May 2011, 22:12:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1137196
Run time	16 days 20 hours 44 min 2 sec
CPU time	16 days 3 hours 44 min 53 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	2.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.18</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:35:56 (6260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:01:38 (5764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:28:50 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:38:16 (14769): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 May 2011 07:46:11	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	881,280	1,380,958	1.5670
25 May 2011 10:09:30	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	855,360	1,340,422	1.5671
24 May 2011 21:57:47	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	829,440	1,299,361	1.5666
24 May 2011 08:38:36	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	803,520	1,258,957	1.5668
23 May 2011 20:48:43	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	777,600	1,218,817	1.5674
23 May 2011 08:42:11	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	751,680	1,178,765	1.5682
22 May 2011 14:57:44	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	725,760	1,138,745	1.5690
21 May 2011 09:32:11	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	699,840	1,097,632	1.5684
20 May 2011 17:47:20	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	673,920	1,057,849	1.5697
20 May 2011 06:24:23	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	648,000	1,017,675	1.5705
19 May 2011 18:39:43	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	622,080	977,576	1.5715
18 May 2011 04:54:55	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	596,160	937,465	1.5725
17 May 2011 17:13:03	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	570,240	896,865	1.5728
17 May 2011 04:17:23	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	544,320	855,908	1.5724
16 May 2011 17:13:23	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	518,400	814,827	1.5718
16 May 2011 03:19:21	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	492,480	774,121	1.5719
14 May 2011 17:22:19	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	466,560	733,824	1.5728
13 May 2011 22:46:48	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	440,640	692,390	1.5713
13 May 2011 11:21:33	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	414,720	651,952	1.5720
12 May 2011 23:51:46	1137196	12818821	hadcm3n_p1fu_1900_40_007219450_0	388,800	611,426	1.5726