Task 12243755

Name	hadam3p_saf_1ikp_1984_1_006961505_0
Workunit	7164821
Created	23 Nov 2010, 10:31:09 UTC
Sent	8 Mar 2011, 11:40:02 UTC
Report deadline	18 Feb 2012, 17:00:02 UTC
Received	12 Mar 2011, 19:51:24 UTC
Server state	Over
Outcome	No reply
Client state	Compute error
Exit status	194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID	570789
Run time	3 days 20 hours 48 min 15 sec
CPU time	3 days 13 hours 11 min 21 sec
Validate state	Invalid
Credit	1,496.58
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.08 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> Got ack for job that's till active </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:15:11 (1504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:16:06 (7924): No heartbeat from core client for 30 sec - exiting 08:16:07 (7924): No heartbeat from core client for 30 sec - exiting 08:16:08 (7924): No heartbeat from core client for 30 sec - exiting 08:16:09 (7924): No heartbeat from core client for 30 sec - exiting 08:16:10 (7924): No heartbeat from core client for 30 sec - exiting 08:16:44 (7924): No heartbeat from core client for 30 sec - exiting 08:16:46 (7924): No heartbeat from core client for 30 sec - exiting 08:16:47 (7924): No heartbeat from core client for 30 sec - exiting 08:16:48 (7924): No heartbeat from core client for 30 sec - exiting 08:16:49 (7924): No heartbeat from core client for 30 sec - exiting 08:16:50 (7924): No heartbeat from core client for 30 sec - exiting 08:16:51 (7924): No heartbeat from core client for 30 sec - exiting 08:16:52 (7924): No heartbeat from core client for 30 sec - exiting 08:16:53 (7924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:57:49 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=2 Model crash detected, will try to restart... 22:10:54 (3560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:11:51 (3036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:11:52 (3036): No heartbeat from core client for 30 sec - exiting 22:11:53 (3036): No heartbeat from core client for 30 sec - exiting 22:11:54 (3036): No heartbeat from core client for 30 sec - exiting 22:14:41 (3380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:16:45 (3060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4536, selfPID=4536, iMonCtr=2 22:17:41 (3248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:29:34 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:31:23 (7756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:19:13 (2348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:18:07 (6704): No heartbeat from core client for 30 sec - exiting 18:18:08 (6704): No heartbeat from core client for 30 sec - exiting 18:18:09 (6704): No heartbeat from core client for 30 sec - exiting 18:18:10 (6704): No heartbeat from core client for 30 sec - exiting 18:18:11 (6704): No heartbeat from core client for 30 sec - exiting 18:18:23 (7644): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - No 'heartbeat' from BOINC... 18:18:29 (7280): start_timer_thread(): CreateThread() failed, errno 0 18:27:03 (7180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:27:04 (7180): No heartbeat from core client for 30 sec - exiting 18:27:42 (5668): start_timer_thread(): CreateThread() failed, errno 0 18:28:33 (7332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:26 (7760): start_timer_thread(): CreateThread() failed, errno 0 18:31:11 (4208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:32:19 (7112): start_timer_thread(): CreateThread() failed, errno 0 18:33:48 (7648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:34:07 (7480): start_timer_thread(): CreateThread() failed, errno 0 18:35:30 (2852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:35:49 (2764): start_timer_thread(): CreateThread() failed, errno 0 18:37:19 (1760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:37:43 (3880): start_timer_thread(): CreateThread() failed, errno 0 18:38:26 (4700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:39:05 (2184): start_timer_thread(): CreateThread() failed, errno 0 18:39:35 (5348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:39:36 (5348): No heartbeat from core client for 30 sec - exiting 18:39:37 (5348): No heartbeat from core client for 30 sec - exiting 18:39:38 (5348): No heartbeat from core client for 30 sec - exiting 18:39:50 (4004): start_timer_thread(): CreateThread() failed, errno 0 18:40:33 (2764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:42:32 (5048): start_timer_thread(): CreateThread() failed, errno 0 18:43:23 (7392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:44:48 (5628): start_timer_thread(): CreateThread() failed, errno 0 18:45:32 (5500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:46:36 (3068): start_timer_thread(): CreateThread() failed, errno 0 18:47:14 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:32 (5228): start_timer_thread(): CreateThread() failed, errno 0 18:50:15 (6708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:50:17 (6708): No heartbeat from core client for 30 sec - exiting 18:52:33 (3760): No heartbeat from core client for 30 sec - exiting 18:52:34 (3760): No heartbeat from core client for 30 sec - exiting 18:52:35 (3760): No heartbeat from core client for 30 sec - exiting 18:52:36 (3760): No heartbeat from core client for 30 sec - exiting 18:52:37 (3760): No heartbeat from core client for 30 sec - exiting 18:52:38 (3760): No heartbeat from core client for 30 sec - exiting 18:52:39 (3760): No heartbeat from core client for 30 sec - exiting 18:52:40 (3760): No heartbeat from core client for 30 sec - exiting 18:52:41 (3760): No heartbeat from core client for 30 sec - exiting 18:52:42 (3760): No heartbeat from core client for 30 sec - exiting 18:52:43 (3760): No heartbeat from core client for 30 sec - exiting 18:52:44 (3760): No heartbeat from core client for 30 sec - exiting 18:52:45 (3760): No heartbeat from core client for 30 sec - exiting 18:52:46 (3760): No heartbeat from core client for 30 sec - exiting 18:52:47 (3760): No heartbeat from core client for 30 sec - exiting 18:52:48 (5876): start_timer_thread(): CreateThread() failed, errno 0 18:52:48 (3760): No heartbeat from core client for 30 sec - exiting 18:52:49 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:10 (7080): start_timer_thread(): CreateThread() failed, errno 0 18:53:34 (8056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:55:25 (1452): start_timer_thread(): CreateThread() failed, errno 0 18:56:18 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:43 (7968): start_timer_thread(): CreateThread() failed, errno 0 18:59:13 (6344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:00:10 (5840): start_timer_thread(): CreateThread() failed, errno 0 19:01:17 (3880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:01:54 (2224): start_timer_thread(): CreateThread() failed, errno 0 19:03:12 (5224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:59 (7172): start_timer_thread(): CreateThread() failed, errno 0 19:04:29 (5228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:04:43 (1524): start_timer_thread(): CreateThread() failed, errno 0 19:05:25 (2592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:05:26 (2592): No heartbeat from core client for 30 sec - exiting 19:06:37 (7444): start_timer_thread(): CreateThread() failed, errno 0 19:07:21 (5256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:09:29 (5796): start_timer_thread(): CreateThread() failed, errno 0 19:11:02 (6632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:12:18 (3444): start_timer_thread(): CreateThread() failed, errno 0 19:12:47 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:14:44 (5020): start_timer_thread(): CreateThread() failed, errno 0 19:15:27 (2736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:17:00 (5808): start_timer_thread(): CreateThread() failed, errno 0 19:17:44 (6788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:17:45 (6788): No heartbeat from core client for 30 sec - exiting 19:18:57 (4680): start_timer_thread(): CreateThread() failed, errno 0 19:19:26 (7172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:27 (7172): No heartbeat from core client for 30 sec - exiting 19:19:28 (7172): No heartbeat from core client for 30 sec - exiting 19:19:29 (7172): No heartbeat from core client for 30 sec - exiting 19:19:30 (7172): No heartbeat from core client for 30 sec - exiting 19:21:03 (6696): start_timer_thread(): CreateThread() failed, errno 0 19:21:50 (6700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:15 (5156): No heartbeat from core client for 30 sec - exiting 19:24:16 (5156): No heartbeat from core client for 30 sec - exiting 19:24:17 (5156): No heartbeat from core client for 30 sec - exiting 19:24:18 (5156): No heartbeat from core client for 30 sec - exiting 19:24:19 (5156): No heartbeat from core client for 30 sec - exiting 19:24:20 (5156): No heartbeat from core client for 30 sec - exiting 19:24:21 (5156): No heartbeat from core client for 30 sec - exiting 19:24:22 (5156): No heartbeat from core client for 30 sec - exiting 19:24:23 (5156): No heartbeat from core client for 30 sec - exiting 19:24:24 (5156): No heartbeat from core client for 30 sec - exiting 19:24:25 (5156): No heartbeat from core client for 30 sec - exiting 19:24:26 (5156): No heartbeat from core client for 30 sec - exiting 19:24:26 (1088): start_timer_thread(): CreateThread() failed, errno 0 19:24:27 (5156): No heartbeat from core client for 30 sec - exiting 19:24:28 (5156): No heartbeat from core client for 30 sec - exiting 19:24:29 (5156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:30 (5156): No heartbeat from core client for 30 sec - exiting 19:24:45 (4036): start_timer_thread(): CreateThread() failed, errno 0 19:25:25 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:27:10 (5792): start_timer_thread(): CreateThread() failed, errno 0 19:27:39 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7296, selfPID=7296, iMonCtr=2 19:27:55 (6284): start_timer_thread(): CreateThread() failed, errno 0 19:29:36 (7048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:30:44 (1528): start_timer_thread(): CreateThread() failed, errno 0 19:32:14 (4056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:33:34 (2608): start_timer_thread(): CreateThread() failed, errno 0 19:34:03 (6304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2940, selfPID=19:34:04 (6304): No heartbeat from core client for 30 sec - exiting 19:34:24 (3524): start_timer_thread(): CreateThread() failed, errno 0 19:36:12 (6988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:43 (6276): start_timer_thread(): CreateThread() failed, errno 0 19:39:10 (8140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:46 (4948): start_timer_thread(): CreateThread() failed, errno 0 19:40:16 (7808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:40:33 (4688): start_timer_thread(): CreateThread() failed, errno 0 19:41:36 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:43:35 (7404): start_timer_thread(): CreateThread() failed, errno 0 19:44:14 (5912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process 9:44:15 (5912): No heartbeat from core client for 30 sec - exiting 19:46:13 (7532): start_timer_thread(): CreateThread() failed, errno 0 19:50:29 (3380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:39 (1804): start_timer_thread(): CreateThread() failed, errno 0 19:52:19 (6232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, check9:52:20 (6232): No heartbeat from core client for 30 sec - exiting 19:54:16 (3520): start_timer_thread(): CreateThread() failed, errno 0 19:54:45 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:55:04 (5856): start_timer_thread(): CreateThread() failed, errno 0 19:55:34 (5792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:58:02 (5752): start_timer_thread(): CreateThread() failed, errno 0 19:58:58 (7820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:17 (3976): start_timer_thread(): CreateThread() failed, errno 0 20:01:00 (3276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:02 (5764): start_timer_thread(): CreateThread() failed, errno 0 20:03:31 (7300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:42 (3496): start_timer_thread(): CreateThread() failed, errno 0 20:04:11 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:04:12 (5116): No heartbeat from core client for 30 sec - exiting 20:04:13 (5116): No heartbeat from core client for 30 sec - exiting 20:04:14 (5116): No heartbeat from core client for 30 sec - exiting 20:04:15 (5116): No heartbeat from core client for 30 sec - exiting 20:04:16 (5116): No heartbeat from core client for 30 sec - exiting 20:04:17 (5116): No heartbeat from core client for 30 sec - exiting 20:06:57 (7828): start_timer_thread(): CreateThread() failed, errno 0 20:07:27 (1460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:07:28 (1460): No heartbeat from core client for 30 sec - exiting 20:07:29 (1460): No heartbeat from core client for 30 sec - exiting 20:07:30 (1460): No heartbeat from core client for 30 sec - exiting 20:07:31 (1460): No heartbeat from core client for 30 sec - exiting 20:08:32 (7088): start_timer_thread(): CreateThread() failed, errno 0 20:09:19 (7924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3400, selfPID=3400, iMonCtr=2 20:10:35 (7908): start_timer_thread(): CreateThread() failed, errno 0 20:11:05 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:59 (7236): start_timer_thread(): CreateThread() failed, errno 0 20:12:29 (6708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7988, selfPID=7988, iMonCtr=2 20:16:03 (1872): start_timer_thread(): CreateThread() failed, errno 0 20:16:32 (2116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7224, selfPID=7224, iMonCtr=2 20:18:48 (3196): start_timer_thread(): CreateThread() failed, errno 0 20:19:27 (5092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:21:25 (1948): start_timer_thread(): CreateThread() failed, errno 0 20:21:53 (7352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:21:54 (7352): No heartbeat from core client for 30 sec - exiting 20:21:55 (7352): No heartbeat from core client for 30 sec - exiting 20:23:24 (4868): start_timer_thread(): CreateThread() failed, errno 0 20:24:13 (7260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:12 (2768): start_timer_thread(): CreateThread() failed, errno 0 20:26:41 (7988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:52 (6348): start_timer_thread(): CreateThread() failed, errno 0 20:27:42 (5156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:29:57 (5112): start_timer_thread(): CreateThread() failed, errno 0 20:30:26 (7372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:39 (5796): start_timer_thread(): CreateThread() failed, errno 0 20:34:06 (1064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:35:15 (7972): start_timer_thread(): CreateThread() failed, errno 0 20:35:48 (6912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:36:04 (5764): start_timer_thread(): CreateThread() failed, errno 0 20:36:46 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:52 (7344): start_timer_thread(): CreateThread() failed, errno 0 20:38:18 (6308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:40:53 (1540): start_timer_thread(): CreateThread() failed, errno 0 BUFFOUT: C I/O Error - Return code = 32 Model crashed: STWORK : Error in PP_FILE tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4868, selfPID=4868, iMonCtr=2 Leaving CPDN_Main::Monitor... zip error: Output file write failure (write error on zip file) 20:49:35 (7956): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Mar 2011 10:36:15	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	92,256	285,108	3.0904
11 Mar 2011 22:51:17	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	80,736	249,136	3.0858
11 Mar 2011 09:59:05	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	69,216	213,845	3.0895
10 Mar 2011 23:15:58	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	57,696	178,962	3.1018
10 Mar 2011 12:59:55	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	46,176	143,061	3.0982
10 Mar 2011 01:36:16	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	34,656	107,143	3.0916
09 Mar 2011 14:17:48	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	23,136	70,913	3.0651
09 Mar 2011 01:14:42	570789	12243755	hadam3p_saf_1ikp_1984_1_006961505_0	11,616	35,486	3.0549