Task 13027953

Name	hadcm3n_t799_1940_40_007317030_0
Workunit	7514460
Created	29 Jun 2011, 6:00:39 UTC
Sent	29 Jun 2011, 12:09:42 UTC
Report deadline	28 Sep 2011, 19:36:53 UTC
Received	13 Sep 2011, 14:31:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1129125
Run time	16 days 21 hours 21 min 9 sec
CPU time	8 days 23 hours 6 min 16 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.37 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4972, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3964, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2924, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 08:25:40 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3012, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4464, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:38:09 (5608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:08:57 (4148): No heartbeat from core client for 30 sec - exiting 07:08:58 (4148): No heartbeat from core client for 30 sec - exiting 07:08:59 (4148): No heartbeat from core client for 30 sec - exiting 07:09:00 (4148): No heartbeat from core client for 30 sec - exiting 07:09:01 (4148): No heartbeat from core client for 30 sec - exiting 07:09:02 (4148): No heartbeat from core client for 30 sec - exiting 07:09:03 (4148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:25:11 (5712): No heartbeat from core client for 30 sec - exiting 14:25:12 (5712): No heartbeat from core client for 30 sec - exiting 14:25:13 (5712): No heartbeat from core client for 30 sec - exiting 14:25:14 (5712): No heartbeat from core client for 30 sec - exiting 14:25:15 (5712): No heartbeat from core client for 30 sec - exiting 14:25:16 (5712): No heartbeat from core client for 30 sec - exiting 14:25:17 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:51:00 (4472): No heartbeat from core client for 30 sec - exiting 23:51:01 (4472): No heartbeat from core client for 30 sec - exiting 23:51:02 (4472): No heartbeat from core client for 30 sec - exiting 23:51:03 (4472): No heartbeat from core client for 30 sec - exiting 23:51:04 (4472): No heartbeat from core client for 30 sec - exiting 23:51:05 (4472): No heartbeat from core client for 30 sec - exiting 23:51:06 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5496, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:31:50 (4788): No heartbeat from core client for 30 sec - exiting 07:31:51 (4788): No heartbeat from core client for 30 sec - exiting 07:31:53 (4788): No heartbeat from core client for 30 sec - exiting 07:31:54 (4788): No heartbeat from core client for 30 sec - exiting 07:31:55 (4788): No heartbeat from core client for 30 sec - exiting 07:31:56 (4788): No heartbeat from core client for 30 sec - exiting 07:31:57 (4788): No heartbeat from core client for 30 sec - exiting 07:31:58 (4788): No heartbeat from core client for 30 sec - exiting 07:31:59 (4788): No heartbeat from core client for 30 sec - exiting 07:32:00 (4788): No heartbeat from core client for 30 sec - exiting 07:32:02 (4788): No heartbeat from core client for 30 sec - exiting 07:32:03 (4788): No heartbeat from core client for 30 sec - exiting 07:32:04 (4788): No heartbeat from core client for 30 sec - exiting 07:32:05 (4788): No heartbeat from core client for 30 sec - exiting 07:32:06 (4788): No heartbeat from core client for 30 sec - exiting 07:32:07 (4788): No heartbeat from core client for 30 sec - exiting 07:32:08 (4788): No heartbeat from core client for 30 sec - exiting 07:32:09 (4788): No heartbeat from core client for 30 sec - exiting 07:32:10 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2000, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2572, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 09:31:13 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3160, iMonCtr=1 Model crash detected, will try to restart... 16:39:46 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:58:22 (5644): No heartbeat from core client for 30 sec - exiting 04:58:23 (5644): No heartbeat from core client for 30 sec - exiting 04:58:24 (5644): No heartbeat from core client for 30 sec - exiting 04:58:25 (5644): No heartbeat from core client for 30 sec - exiting 04:58:26 (5644): No heartbeat from core client for 30 sec - exiting 04:58:27 (5644): No heartbeat from core client for 30 sec - exiting 04:58:28 (5644): No heartbeat from core client for 30 sec - exiting 04:58:29 (5644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... 12:56:10 (2268): No heartbeat from core client for 30 sec - exiting 12:56:11 (2268): No heartbeat from core client for 30 sec - exiting 12:56:12 (2268): No heartbeat from core client for 30 sec - exiting 12:56:13 (2268): No heartbeat from core client for 30 sec - exiting 12:56:14 (2268): No heartbeat from core client for 30 sec - exiting 12:56:15 (2268): No heartbeat from core client for 30 sec - exiting 12:56:16 (2268): No heartbeat from core client for 30 sec - exiting 12:56:17 (2268): No heartbeat from core client for 30 sec - exiting 12:56:18 (2268): No heartbeat from core client for 30 sec - exiting 12:56:19 (2268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:20 (2268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3492, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5508, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2764, iMonCtr=1 Model crash detected, will try to restart... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Sep 2011 14:31:09	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	259,200	774,371	2.9875
07 Sep 2011 16:27:50	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	233,280	694,833	2.9785
16 Aug 2011 12:15:21	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	207,360	611,593	2.9494
03 Aug 2011 18:45:32	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	181,440	522,687	2.8808
27 Jul 2011 11:36:27	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	155,520	443,177	2.8496
25 Jul 2011 20:29:13	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	129,600	359,316	2.7725
25 Jul 2011 15:03:27	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	103,680	285,567	2.7543
08 Jul 2011 14:40:08	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	77,760	211,799	2.7238
05 Jul 2011 07:28:21	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	51,840	138,911	2.6796
03 Jul 2011 09:14:50	1129125	13027953	hadcm3n_t799_1940_40_007317030_0	25,920	66,333	2.5591