Task 12822268

Name	hadcm3n_p2rp_1900_40_007221173_0
Workunit	7419413
Created	26 Apr 2011, 15:22:14 UTC
Sent	1 May 2011, 23:46:34 UTC
Report deadline	1 Aug 2011, 7:13:45 UTC
Received	16 Jun 2011, 21:40:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1021733
Run time	21 days 16 hours 49 min 8 sec
CPU time	21 days 8 hours 22 min 0 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.76 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:51:41 (3144): No heartbeat from core client for 30 sec - exiting 08:51:42 (3144): No heartbeat from core client for 30 sec - exiting 08:51:43 (3144): No heartbeat from core client for 30 sec - exiting 08:51:44 (3144): No heartbeat from core client for 30 sec - exiting 08:51:45 (3144): No heartbeat from core client for 30 sec - exiting 08:51:46 (3144): No heartbeat from core client for 30 sec - exiting 08:51:47 (3144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:02:35 (3416): No heartbeat from core client for 30 sec - exiting 18:02:37 (3416): No heartbeat from core client for 30 sec - exiting 18:02:38 (3416): No heartbeat from core client for 30 sec - exiting 18:02:39 (3416): No heartbeat from core client for 30 sec - exiting 18:02:40 (3416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2844, iMonCtr=1 Model crash detected, will try to restart... 00:20:20 (2844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:54:23 (2176): No heartbeat from core client for 30 sec - exiting 00:54:24 (2176): No heartbeat from core client for 30 sec - exiting 00:54:25 (2176): No heartbeat from core client for 30 sec - exiting 00:54:26 (2176): No heartbeat from core client for 30 sec - exiting 00:54:27 (2176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:46:04 (3440): No heartbeat from core client for 30 sec - exiting 01:46:05 (3440): No heartbeat from core client for 30 sec - exiting 01:46:06 (3440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=520, selfPID=520, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/p2rpko.pjc4c10 Error converting file to netcdf: dataout/p2rpko.pic4c10 Error converting file to netcdf: dataout/p2rpko.pfc4c10 Error converting file to netcdf: dataout/p2rpka.phc4c10 Error converting file to netcdf: dataout/p2rpka.pgc4c10 Error converting file to netcdf: dataout/p2rpka.pec4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Jun 2011 20:44:04	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	777,600	1,844,514	2.3721
16 Jun 2011 05:29:49	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	751,680	1,793,337	2.3858
15 Jun 2011 13:23:48	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	725,760	1,742,765	2.4013
14 Jun 2011 18:56:29	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	699,840	1,690,256	2.4152
13 Jun 2011 23:13:18	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	673,920	1,639,955	2.4335
13 Jun 2011 03:52:29	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	648,000	1,588,812	2.4519
12 Jun 2011 08:38:13	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	622,080	1,548,514	2.4893
11 Jun 2011 18:05:20	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	596,160	1,505,838	2.5259
09 Jun 2011 13:52:16	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	570,240	1,451,217	2.5449
07 Jun 2011 14:27:50	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	544,320	1,392,501	2.5582
05 Jun 2011 07:20:16	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	518,400	1,329,747	2.5651
30 May 2011 12:40:13	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	492,480	1,267,981	2.5747
29 May 2011 19:03:42	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	466,560	1,205,034	2.5828
29 May 2011 03:46:20	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	440,640	1,150,265	2.6104
27 May 2011 16:57:48	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	414,720	1,093,478	2.6367
26 May 2011 17:56:45	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	388,800	1,011,553	2.6017
25 May 2011 21:17:57	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	362,880	941,339	2.5941
24 May 2011 07:58:20	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	336,960	872,043	2.5880
23 May 2011 03:45:18	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	311,040	808,859	2.6005
22 May 2011 07:11:27	1021733	12822268	hadcm3n_p2rp_1900_40_007221173_0	285,120	737,829	2.5878