Task 16449341

Name	hadam3p_anz_a96x_2012_1_008618249_0
Workunit	8764761
Created	2 Apr 2014, 15:51:33 UTC
Sent	25 Apr 2014, 4:32:04 UTC
Report deadline	7 Apr 2015, 9:52:04 UTC
Received	7 May 2014, 17:13:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1164992
Run time	7 days 10 hours 28 min 0 sec
CPU time	5 days 11 hours 9 min 16 sec
Validate state	Invalid
Credit	2,497.00
Device peak FLOPS	1.91 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.31</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6196, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4320, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:46:41 (3220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:44 (5124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:39:08 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:39:10 (5604): No heartbeat from core client for 30 sec - exiting 09:08:13 (2200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:15 (2200): No heartbeat from core client for 30 sec - exiting 10:06:06 (6724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:06:07 (6724): No heartbeat from core client for 30 sec - exiting 11:07:16 (6760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:07:18 (6760): No heartbeat from core client for 30 sec - exiting 13:18:14 (6176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:15:30 (1216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:15:32 (1216): No heartbeat from core client for 30 sec - exiting 14:44:04 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:05 (6180): No heartbeat from core client for 30 sec - exiting 15:25:18 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:53:59 (6900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:54:00 (6900): No heartbeat from core client for 30 sec - exiting 17:39:39 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:39:40 (5516): No heartbeat from core client for 30 sec - exiting 18:08:23 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:08:24 (4676): No heartbeat from core client for 30 sec - exiting 18:16:14 (3092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:48:48 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:48:49 (6180): No heartbeat from core client for 30 sec - exiting 18:48:50 (6180): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3628, selfPID=3628, iMonCtr=2 19:38:04 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:38:05 (5216): No heartbeat from core client for 30 sec - exiting 19:54:28 (6408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6804, selfPID=6804, iMonCtr=2 20:15:03 (3144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:15:04 (3144): No heartbeat from core client for 30 sec - exiting 21:03:54 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:55 (7088): No heartbeat from core client for 30 sec - exiting 21:24:15 (6196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:16 (6196): No heartbeat from core client for 30 sec - exiting 22:05:04 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:05 (6356): No heartbeat from core client for 30 sec - exiting 22:41:09 (6688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:41:10 (6688): No heartbeat from core client for 30 sec - exiting 23:05:53 (6480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:01 (7136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:10:59 (5600): No heartbeat from core client for 30 sec - exiting 00:11:00 (5600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:59 (6804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:00 (6804): No heartbeat from core client for 30 sec - exiting 01:24:01 (6804): No heartbeat from core client for 30 sec - exiting 01:24:03 (6804): No heartbeat from core client for 30 sec - exiting 01:24:04 (6804): No heartbeat from core client for 30 sec - exiting 01:52:42 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:49:22 (6936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:49:24 (6936): No heartbeat from core client for 30 sec - exiting 02:49:25 (6936): No heartbeat from core client for 30 sec - exiting 03:30:11 (8112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:36:33 (7460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:05:21 (4000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:10:23 (7596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:10:24 (7596): No heartbeat from core client for 30 sec - exiting 07:54:56 (3400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:54:57 (3400): No heartbeat from core client for 30 sec - exiting 07:54:58 (3400): No heartbeat from core client for 30 sec - exiting 08:03:22 (7344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:03:23 (7344): No heartbeat from core client for 30 sec - exiting 08:03:24 (7344): No heartbeat from core client for 30 sec - exiting 08:31:58 (6568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:32:00 (6568): No heartbeat from core client for 30 sec - exiting 09:13:34 (7712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:36 (7712): No heartbeat from core client for 30 sec - exiting 10:18:50 (7220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:18:51 (7220): No heartbeat from core client for 30 sec - exiting 10:39:01 (8588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:39:02 (8588): No heartbeat from core client for 30 sec - exiting 12:05:30 (7472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:05:31 (7472): No heartbeat from core client for 30 sec - exiting 12:37:45 (8240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:37:46 (8240): No heartbeat from core client for 30 sec - exiting 12:37:47 (8240): No heartbeat from core client for 30 sec - exiting 12:37:48 (8240): No heartbeat from core client for 30 sec - exiting 12:37:49 (8240): No heartbeat from core client for 30 sec - exiting 13:38:59 (8904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:39:00 (8904): No heartbeat from core client for 30 sec - exiting 14:20:10 (7432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:12 (7432): No heartbeat from core client for 30 sec - exiting 16:44:11 (8512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:44:12 (8512): No heartbeat from core client for 30 sec - exiting 18:15:03 (7520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:15:05 (7520): No heartbeat from core client for 30 sec - exiting 19:24:27 (7584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:13 (3940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:14 (3940): No heartbeat from core client for 30 sec - exiting 20:21:51 (7712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:21:52 (7712): No heartbeat from core client for 30 sec - exiting 20:21:54 (7712): No heartbeat from core client for 30 sec - exiting 20:21:55 (7712): No heartbeat from core client for 30 sec - exiting 21:19:11 (9088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:19:12 (9088): No heartbeat from core client for 30 sec - exiting 21:19:13 (9088): No heartbeat from core client for 30 sec - exiting 21:48:05 (7280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:48:06 (7280): No heartbeat from core client for 30 sec - exiting 22:04:18 (9088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:16:24 (8440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:19 (9112): No heartbeat from core client for 30 sec - exiting 22:20:20 (9112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:22 (7588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:23 (7588): No heartbeat from core client for 30 sec - exiting 00:18:01 (7880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:18:02 (7880): No heartbeat from core client for 30 sec - exiting 00:38:18 (6236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:19 (6236): No heartbeat from core client for 30 sec - exiting 02:07:26 (5436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:27 (5436): No heartbeat from core client for 30 sec - exiting 02:07:28 (5436): No heartbeat from core client for 30 sec - exiting 03:36:56 (7080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:36:58 (7080): No heartbeat from core client for 30 sec - exiting 04:21:54 (5028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:55 (5028): No heartbeat from core client for 30 sec - exiting 05:02:46 (6820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:59:56 (9980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:40 (10124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:41 (10124): No heartbeat from core client for 30 sec - exiting 06:29:06 (7684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:39 (10164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:02:44 (10092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:02:46 (10092): No heartbeat from core client for 30 sec - exiting 14:03:38 (10420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:15:14 (9972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:28 (10672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:10:21 (10372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:46:01 (11128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:46:02 (11128): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10460, selfPID=10460, iMonCtr=2 23:19:26 (7808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:19:27 (7808): No heartbeat from core client for 30 sec - exiting 00:23:06 (10088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:23:07 (10088): No heartbeat from core client for 30 sec - exiting 01:21:28 (11040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:21:29 (11040): No heartbeat from core client for 30 sec - exiting 01:21:31 (11040): No heartbeat from core client for 30 sec - exiting 01:21:32 (11040): No heartbeat from core client for 30 sec - exiting 01:34:26 (9988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:49:03 (10148): No heartbeat from core client for 30 sec - exiting 03:49:05 (10148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:09:31 (10788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:09:32 (10788): No heartbeat from core client for 30 sec - exiting 04:09:33 (10788): No heartbeat from core client for 30 sec - exiting 04:09:35 (10788): No heartbeat from core client for 30 sec - exiting 04:09:36 (10788): No heartbeat from core client for 30 sec - exiting 04:22:27 (10380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:13:47 (10424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:13:41 (11448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:55:31 (10468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10120, selfPID=10120, iMonCtr=2 11:21:01 (11276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:32 (11156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:47:25 (12244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:47:27 (12244): No heartbeat from core client for 30 sec - exiting 13:51:30 (11068): No heartbeat from core client for 30 sec - exiting 13:51:31 (11068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:07:16 (11796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:20:44 (12092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:20:45 (12092): No heartbeat from core client for 30 sec - exiting 23:30:09 (11928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:30:10 (11928): No heartbeat from core client for 30 sec - exiting 23:30:11 (11928): No heartbeat from core client for 30 sec - exiting 23:46:35 (12692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:46:36 (12692): No heartbeat from core client for 30 sec - exiting 23:46:37 (12692): No heartbeat from core client for 30 sec - exiting 23:46:39 (12692): No heartbeat from core client for 30 sec - exiting 02:41:41 (13128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:41:42 (13128): No heartbeat from core client for 30 sec - exiting 03:22:39 (12692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:22:40 (12692): No heartbeat from core client for 30 sec - exiting 03:22:42 (12692): No heartbeat from core client for 30 sec - exiting 03:22:43 (12692): No heartbeat from core client for 30 sec - exiting 03:51:52 (12644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:51:53 (12644): No heartbeat from core client for 30 sec - exiting 04:45:43 (12544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:45:45 (12544): No heartbeat from core client for 30 sec - exiting 06:44:09 (11404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:44:11 (11404): No heartbeat from core client for 30 sec - exiting 07:00:24 (13712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:08:36 (13988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:29:30 (14288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:29:31 (14288): No heartbeat from core client for 30 sec - exiting 08:02:12 (13828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:13 (13828): No heartbeat from core client for 30 sec - exiting 09:08:28 (12588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:29 (12588): No heartbeat from core client for 30 sec - exiting 09:08:30 (12588): No heartbeat from core client for 30 sec - exiting 09:08:31 (12588): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13564, selfPID=13564, iMonCtr=2 09:16:56 (14252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:20:52 (14072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:20:53 (14072): No heartbeat from core client for 30 sec - exiting 09:20:54 (14072): No heartbeat from core client for 30 sec - exiting 09:20:55 (14072): No heartbeat from core client for 30 sec - exiting 09:20:56 (14072): No heartbeat from core client for 30 sec - exiting 09:20:57 (14072): No heartbeat from core client for 30 sec - exiting 09:58:25 (14092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 May 2014 15:20:03	1164992	16449341	hadam3p_anz_a96x_2012_1_008618249_0	57,899	471,407	8.1419
29 Apr 2014 22:30:46	1164992	16449341	hadam3p_anz_a96x_2012_1_008618249_0	46,379	382,387	8.2448
28 Apr 2014 18:47:46	1164992	16449341	hadam3p_anz_a96x_2012_1_008618249_0	34,859	287,578	8.2497
27 Apr 2014 15:02:37	1164992	16449341	hadam3p_anz_a96x_2012_1_008618249_0	23,339	192,511	8.2485
26 Apr 2014 11:08:57	1164992	16449341	hadam3p_anz_a96x_2012_1_008618249_0	11,819	97,214	8.2252