Task 13684552

Name	hadam3p_saf_79sp_2002_1_007569634_0
Workunit	7747764
Created	2 Dec 2011, 15:10:50 UTC
Sent	19 Dec 2011, 17:35:44 UTC
Report deadline	30 Nov 2012, 22:55:44 UTC
Received	10 Feb 2012, 16:03:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	951616
Run time	7 days 12 hours 10 min 53 sec
CPU time	3 days 19 hours 14 min 1 sec
Validate state	Invalid
Credit	1,496.58
Device peak FLOPS	2.38 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1292, selfPID=1292, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4792, selfPID=4792, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5768, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3836, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3448, selfPID=3448, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=640, selfPID=640, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3652, selfPID=3652, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3420, selfPID=3420, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3640, selfPID=3640, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5580, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3516, selfPID=3516, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2696, selfPID=2696, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4920, selfPID=4920, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2924, selfPID=2924, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3204, selfPID=3204, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... GCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4384, selfPID=4384, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6044, selfPID=6044, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5664, selfPID=5332, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3220, selfPID=368, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5820, selfPID=5820, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3100, selfPID=3100, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3364, selfPID=3364, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=264, selfPID=264, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5868, selfPID=5868, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1772, selfPID=1772, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:28:44 (5072): No heartbeat from core client for 30 sec - exiting 17:28:45 (5072): No heartbeat from core client for 30 sec - exiting 17:28:46 (5072): No heartbeat from core client for 30 sec - exiting 17:28:47 (5072): No heartbeat from core client for 30 sec - exiting 17:28:48 (5072): No heartbeat from core client for 30 sec - exiting 17:28:49 (5072): No heartbeat from core client for 30 sec - exiting 17:28:50 (5072): No heartbeat from core client for 30 sec - exiting 17:28:51 (5072): No heartbeat from core client for 30 sec - exiting 17:28:52 (5072): No heartbeat from core client for 30 sec - exiting 17:28:53 (5072): No heartbeat from core client for 30 sec - exiting 17:28:54 (5072): No heartbeat from core client for 30 sec - exiting 17:28:55 (5072): No heartbeat from core client for 30 sec - exiting 17:28:56 (5072): No heartbeat from core client for 30 sec - exiting 17:28:57 (5072): No heartbeat from core client for 30 sec - exiting 17:28:58 (5072): No heartbeat from core client for 30 sec - exiting 17:28:59 (5072): No heartbeat from core client for 30 sec - exiting 17:29:00 (5072): No heartbeat from core client for 30 sec - exiting 17:29:01 (5072): No heartbeat from core client for 30 sec - exiting 17:29:02 (5072): No heartbeat from core client for 30 sec - exiting 17:29:03 (5072): No heartbeat from core client for 30 sec - exiting 17:29:04 (5072): No heartbeat from core client for 30 sec - exiting 17:29:05 (5072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=156, iMonCtr=2 Model crash detected, will try to restart... 17:31:23 (4212): No heartbeat from core client for 30 sec - exiting 17:31:24 (4212): No heartbeat from core client for 30 sec - exiting 17:31:25 (4212): No heartbeat from core client for 30 sec - exiting 17:31:26 (4212): No heartbeat from core client for 30 sec - exiting 17:31:27 (4212): No heartbeat from core client for 30 sec - exiting 17:31:28 (4212): No heartbeat from core client for 30 sec - exiting 17:31:29 (4212): No heartbeat from core client for 30 sec - exiting 17:31:39 (4212): No heartbeat from core client for 30 sec - exiting 17:31:40 (4212): No heartbeat from core client for 30 sec - exiting 17:31:41 (4212): No heartbeat from core client for 30 sec - exiting 17:31:42 (4212): No heartbeat from core client for 30 sec - exiting 17:31:43 (4212): No heartbeat from core client for 30 sec - exiting 17:31:44 (4212): No heartbeat from core client for 30 sec - exiting 17:31:45 (4212): No heartbeat from core client for 30 sec - exiting 17:31:46 (4212): No heartbeat from core client for 30 sec - exiting 17:31:47 (4212): No heartbeat from core client for 30 sec - exiting 17:31:48 (4212): No heartbeat from core client for 30 sec - exiting 17:31:49 (4212): No heartbeat from core client for 30 sec - exiting 17:31:50 (4212): No heartbeat from core client for 30 sec - exiting 17:31:51 (4212): No heartbeat from core client for 30 sec - exiting 17:31:52 (4212): No heartbeat from core client for 30 sec - exiting 17:31:53 (4212): No heartbeat from core client for 30 sec - exiting 17:31:54 (4212): No heartbeat from core client for 30 sec - exiting 17:31:55 (4212): No heartbeat from core client for 30 sec - exiting 17:31:56 (4212): No heartbeat from core client for 30 sec - exiting 17:31:57 (4212): No heartbeat from core client for 30 sec - exiting 17:31:58 (4212): No heartbeat from core client for 30 sec - exiting 17:32:01 (4212): No heartbeat from core client for 30 sec - exiting 17:32:02 (4212): No heartbeat from core client for 30 sec - exiting 17:32:03 (4212): No heartbeat from core client for 30 sec - exiting 17:32:04 (4212): No heartbeat from core client for 30 sec - exiting 17:32:05 (4212): No heartbeat from core client for 30 sec - exiting 17:32:06 (4212): No heartbeat from core client for 30 sec - exiting 17:32:07 (4212): No heartbeat from core client for 30 sec - exiting 17:32:08 (4212): No heartbeat from core client for 30 sec - exiting 17:32:09 (4212): No heartbeat from core client for 30 sec - exiting 17:32:10 (4212): No heartbeat from core client for 30 sec - exiting 17:32:11 (4212): No heartbeat from core client for 30 sec - exiting 17:32:12 (4212): No heartbeat from core client for 30 sec - exiting 17:32:13 (4212): No heartbeat from core client for 30 sec - exiting 17:32:14 (4212): No heartbeat from core client for 30 sec - exiting 17:32:15 (4212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:19:25 (5596): No heartbeat from core client for 30 sec - exiting 17:19:26 (5596): No heartbeat from core client for 30 sec - exiting 17:19:27 (5596): No heartbeat from core client for 30 sec - exiting 17:19:28 (5596): No heartbeat from core client for 30 sec - exiting 17:19:29 (5596): No heartbeat from core client for 30 sec - exiting 17:19:30 (5596): No heartbeat from core client for 30 sec - exiting 17:19:31 (5596): No heartbeat from core client for 30 sec - exiting 17:19:32 (5596): No heartbeat from core client for 30 sec - exiting 17:19:33 (5596): No heartbeat from core client for 30 sec - exiting 17:19:34 (5596): No heartbeat from core client for 30 sec - exiting 17:19:35 (5596): No heartbeat from core client for 30 sec - exiting 17:19:36 (5596): No heartbeat from core client for 30 sec - exiting 17:19:37 (5596): No heartbeat from core client for 30 sec - exiting 17:19:38 (5596): No heartbeat from core client for 30 sec - exiting 17:19:39 (5596): No heartbeat from core client for 30 sec - exiting 17:19:40 (5596): No heartbeat from core client for 30 sec - exiting 17:19:41 (5596): No heartbeat from core client for 30 sec - exiting 17:19:42 (5596): No heartbeat from core client for 30 sec - exiting 17:19:43 (5596): No heartbeat from core client for 30 sec - exiting 17:19:44 (5596): No heartbeat from core client for 30 sec - exiting 17:19:45 (5596): No heartbeat from core client for 30 sec - exiting 17:19:46 (5596): No heartbeat from core client for 30 sec - exiting 17:19:47 (5596): No heartbeat from core client for 30 sec - exiting 17:19:48 (5596): No heartbeat from core client for 30 sec - exiting 17:19:49 (5596): No heartbeat from core client for 30 sec - exiting 17:19:50 (5596): No heartbeat from core client for 30 sec - exiting 17:19:51 (5596): No heartbeat from core client for 30 sec - exiting 17:19:52 (5596): No heartbeat from core client for 30 sec - exiting 17:19:53 (5596): No heartbeat from core client for 30 sec - exiting 17:19:54 (5596): No heartbeat from core client for 30 sec - exiting 17:19:55 (5596): No heartbeat from core client for 30 sec - exiting 17:19:56 (5596): No heartbeat from core client for 30 sec - exiting 17:19:57 (5596): No heartbeat from core client for 30 sec - exiting 17:19:58 (5596): No heartbeat from core client for 30 sec - exiting 17:19:59 (5596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1856, selfPID=1856, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4372, selfPID=4952, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:13:29 (3324): No heartbeat from core client for 30 sec - exiting 20:13:30 (3324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=452, selfPID=452, iMonCtr=2 17:27:33 (4540): No heartbeat from core client for 30 sec - exiting 17:27:34 (4540): No heartbeat from core client for 30 sec - exiting 17:27:35 (4540): No heartbeat from core client for 30 sec - exiting 17:27:36 (4540): No heartbeat from core client for 30 sec - exiting 17:27:37 (4540): No heartbeat from core client for 30 sec - exiting 17:27:38 (4540): No heartbeat from core client for 30 sec - exiting 17:27:39 (4540): No heartbeat from core client for 30 sec - exiting 17:27:40 (4540): No heartbeat from core client for 30 sec - exiting 17:27:41 (4540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4212, selfPID=4212, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3784, selfPID=5664, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5132, selfPID=5132, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4224, selfPID=4224, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2664, selfPID=2664, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5924, selfPID=4596, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5924, selfPID=5924, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4184, selfPID=4184, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5408, selfPID=5408, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4972, selfPID=216, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4532, selfPID=4532, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3848, selfPID=3848, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4304, selfPID=4304, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=5184, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3092, selfPID=3092, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Feb 2012 18:44:53	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	92,256	301,635	3.2695
30 Jan 2012 19:00:42	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	80,736	260,379	3.2251
27 Jan 2012 03:12:13	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	69,216	219,054	3.1648
24 Jan 2012 01:06:03	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	57,696	177,777	3.0813
18 Jan 2012 06:13:37	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	46,176	136,324	2.9523
12 Jan 2012 23:45:43	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	34,656	94,607	2.7299
05 Jan 2012 01:32:41	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	23,136	51,555	2.2283
29 Dec 2011 09:24:05	951616	13684552	hadam3p_saf_79sp_2002_1_007569634_0	11,616	9,018	0.7763