Task 13439817

Name	hadam3p_eu_6d32_2006_1_007478971_0
Workunit	7676474
Created	26 Sep 2011, 20:46:06 UTC
Sent	27 Sep 2011, 21:02:56 UTC
Report deadline	9 Sep 2012, 2:22:56 UTC
Received	11 Dec 2011, 12:27:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1069880
Run time	4 days 4 hours 43 min 42 sec
CPU time	11 min 7 sec
Validate state	Invalid
Credit	2,187.67
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5248, selfPID=5248, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4892, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5768, selfPID=5768, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5708, selfPID=5708, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3516, iMonCtr=2 Model crash detected, will try to restart... 18:58:51 (5028): No heartbeat from core client for 30 sec - exiting 18:58:53 (5028): No heartbeat from core client for 30 sec - exiting 18:58:54 (5028): No heartbeat from core client for 30 sec - exiting 18:58:55 (5028): No heartbeat from core client for 30 sec - exiting 18:58:56 (5028): No heartbeat from core client for 30 sec - exiting 18:58:57 (5028): No heartbeat from core client for 30 sec - exiting 18:58:58 (5028): No heartbeat from core client for 30 sec - exiting 18:58:59 (5028): No heartbeat from core client for 30 sec - exiting 18:59:00 (5028): No heartbeat from core client for 30 sec - exiting 18:59:01 (5028): No heartbeat from core client for 30 sec - exiting 18:59:02 (5028): No heartbeat from core client for 30 sec - exiting 18:59:03 (5028): No heartbeat from core client for 30 sec - exiting 18:59:05 (5028): No heartbeat from core client for 30 sec - exiting 18:59:06 (5028): No heartbeat from core client for 30 sec - exiting 18:59:07 (5028): No heartbeat from core client for 30 sec - exiting 18:59:08 (5028): No heartbeat from core client for 30 sec - exiting 18:59:09 (5028): No heartbeat from core client for 30 sec - exiting 18:59:10 (5028): No heartbeat from core client for 30 sec - exiting 18:59:11 (5028): No heartbeat from core client for 30 sec - exiting 18:59:12 (5028): No heartbeat from core client for 30 sec - exiting 18:59:13 (5028): No heartbeat from core client for 30 sec - exiting 18:59:14 (5028): No heartbeat from core client for 30 sec - exiting 18:59:15 (5028): No heartbeat from core client for 30 sec - exiting 18:59:17 (5028): No heartbeat from core client for 30 sec - exiting 18:59:18 (5028): No heartbeat from core client for 30 sec - exiting 18:59:19 (5028): No heartbeat from core client for 30 sec - exiting 18:59:20 (5028): No heartbeat from core client for 30 sec - exiting 18:59:21 (5028): No heartbeat from core client for 30 sec - exiting 18:59:22 (5028): No heartbeat from core client for 30 sec - exiting 18:59:23 (5028): No heartbeat from core client for 30 sec - exiting 18:59:24 (5028): No heartbeat from core client for 30 sec - exiting 18:59:25 (5028): No heartbeat from core client for 30 sec - exiting 18:59:26 (5028): No heartbeat from core client for 30 sec - exiting 18:59:27 (5028): No heartbeat from core client for 30 sec - exiting 18:59:29 (5028): No heartbeat from core client for 30 sec - exiting 18:59:30 (5028): No heartbeat from core client for 30 sec - exiting 18:59:31 (5028): No heartbeat from core client for 30 sec - exiting 18:59:32 (5028): No heartbeat from core client for 30 sec - exiting 18:59:33 (5028): No heartbeat from core client for 30 sec - exiting 18:59:34 (5028): No heartbeat from core client for 30 sec - exiting 18:59:35 (5028): No heartbeat from core client for 30 sec - exiting 18:59:36 (5028): No heartbeat from core client for 30 sec - exiting 18:59:37 (5028): No heartbeat from core client for 30 sec - exiting 18:59:38 (5028): No heartbeat from core client for 30 sec - exiting 18:59:39 (5028): No heartbeat from core client for 30 sec - exiting 18:59:41 (5028): No heartbeat from core client for 30 sec - exiting 18:59:42 (5028): No heartbeat from core client for 30 sec - exiting 18:59:43 (5028): No heartbeat from core client for 30 sec - exiting 18:59:44 (5028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:35:36 (5692): No heartbeat from core client for 30 sec - exiting 15:35:38 (5692): No heartbeat from core client for 30 sec - exiting 15:35:39 (5692): No heartbeat from core client for 30 sec - exiting 15:35:40 (5692): No heartbeat from core client for 30 sec - exiting 15:35:41 (5692): No heartbeat from core client for 30 sec - exiting 15:35:42 (5692): No heartbeat from core client for 30 sec - exiting 15:35:43 (5692): No heartbeat from core client for 30 sec - exiting 15:35:44 (5692): No heartbeat from core client for 30 sec - exiting 15:35:45 (5692): No heartbeat from core client for 30 sec - exiting 15:35:46 (5692): No heartbeat from core client for 30 sec - exiting 15:35:47 (5692): No heartbeat from core client for 30 sec - exiting 15:35:48 (5692): No heartbeat from core client for 30 sec - exiting 15:35:50 (5692): No heartbeat from core client for 30 sec - exiting 15:35:51 (5692): No heartbeat from core client for 30 sec - exiting 15:35:52 (5692): No heartbeat from core client for 30 sec - exiting 15:35:53 (5692): No heartbeat from core client for 30 sec - exiting 15:35:54 (5692): No heartbeat from core client for 30 sec - exiting 15:35:55 (5692): No heartbeat from core client for 30 sec - exiting 15:35:56 (5692): No heartbeat from core client for 30 sec - exiting 15:35:57 (5692): No heartbeat from core client for 30 sec - exiting 15:35:58 (5692): No heartbeat from core client for 30 sec - exiting 15:35:59 (5692): No heartbeat from core client for 30 sec - exiting 15:36:00 (5692): No heartbeat from core client for 30 sec - exiting 15:36:02 (5692): No heartbeat from core client for 30 sec - exiting 15:36:03 (5692): No heartbeat from core client for 30 sec - exiting 15:36:04 (5692): No heartbeat from core client for 30 sec - exiting 15:36:05 (5692): No heartbeat from core client for 30 sec - exiting 15:36:06 (5692): No heartbeat from core client for 30 sec - exiting 15:36:07 (5692): No heartbeat from core client for 30 sec - exiting 15:36:08 (5692): No heartbeat from core client for 30 sec - exiting 15:36:09 (5692): No heartbeat from core client for 30 sec - exiting 15:36:10 (5692): No heartbeat from core client for 30 sec - exiting 15:36:11 (5692): No heartbeat from core client for 30 sec - exiting 15:36:12 (5692): No heartbeat from core client for 30 sec - exiting 15:36:14 (5692): No heartbeat from core client for 30 sec - exiting 15:36:15 (5692): No heartbeat from core client for 30 sec - exiting 15:36:16 (5692): No heartbeat from core client for 30 sec - exiting 15:36:17 (5692): No heartbeat from core client for 30 sec - exiting 15:36:18 (5692): No heartbeat from core client for 30 sec - exiting 15:36:19 (5692): No heartbeat from core client for 30 sec - exiting 15:36:20 (5692): No heartbeat from core client for 30 sec - exiting 15:36:21 (5692): No heartbeat from core client for 30 sec - exiting 15:36:22 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5444, selfPID=5444, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5924, selfPID=5924, iMonCtr=2 18:30:37 (4884): No heartbeat from core client for 30 sec - exiting 18:30:38 (4884): No heartbeat from core client for 30 sec - exiting 18:30:39 (4884): No heartbeat from core client for 30 sec - exiting 18:30:40 (4884): No heartbeat from core client for 30 sec - exiting 18:30:41 (4884): No heartbeat from core client for 30 sec - exiting 18:30:43 (4884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:40:12 (4240): No heartbeat from core client for 30 sec - exiting 18:40:14 (4240): No heartbeat from core client for 30 sec - exiting 18:40:15 (4240): No heartbeat from core client for 30 sec - exiting 18:40:16 (4240): No heartbeat from core client for 30 sec - exiting 18:40:17 (4240): No heartbeat from core client for 30 sec - exiting 18:40:18 (4240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4952, selfPID=4952, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4512, iMonCtr=2 11:07:13 (4920): No heartbeat from core client for 30 sec - exiting 11:07:14 (4920): No heartbeat from core client for 30 sec - exiting 11:07:15 (4920): No heartbeat from core client for 30 sec - exiting 11:07:16 (4920): No heartbeat from core client for 30 sec - exiting 11:07:17 (4920): No heartbeat from core client for 30 sec - exiting 11:07:18 (4920): No heartbeat from core client for 30 sec - exiting 11:07:19 (4920): No heartbeat from core client for 30 sec - exiting 11:07:20 (4920): No heartbeat from core client for 30 sec - exiting 11:07:21 (4920): No heartbeat from core client for 30 sec - exiting 11:07:22 (4920): No heartbeat from core client for 30 sec - exiting 11:07:24 (4920): No heartbeat from core client for 30 sec - exiting 11:07:25 (4920): No heartbeat from core client for 30 sec - exiting 11:07:26 (4920): No heartbeat from core client for 30 sec - exiting 11:07:27 (4920): No heartbeat from core client for 30 sec - exiting 11:07:28 (4920): No heartbeat from core client for 30 sec - exiting 11:07:29 (4920): No heartbeat from core client for 30 sec - exiting 11:07:30 (4920): No heartbeat from core client for 30 sec - exiting 11:07:31 (4920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:36:54 (4732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1328, selfPID=1328, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2752, selfPID=2752, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:31:39 (1328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:55:15 (5844): No heartbeat from core client for 30 sec - exiting 21:55:16 (5844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:58:33 (2540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6076, selfPID=6076, iMonCtr=2 21:59:38 (5076): No heartbeat from core client for 30 sec - exiting 21:59:39 (5076): No heartbeat from core client for 30 sec - exiting 21:59:40 (5076): No heartbeat from core client for 30 sec - exiting 21:59:41 (5076): No heartbeat from core client for 30 sec - exiting 21:59:42 (5076): No heartbeat from core client for 30 sec - exiting 21:59:43 (5076): No heartbeat from core client for 30 sec - exiting 21:59:44 (5076): No heartbeat from core client for 30 sec - exiting 21:59:45 (5076): No heartbeat from core client for 30 sec - exiting 21:59:46 (5076): No heartbeat from core client for 30 sec - exiting 21:59:47 (5076): No heartbeat from core client for 30 sec - exiting 21:59:50 (5076): No heartbeat from core client for 30 sec - exiting 21:59:51 (5076): No heartbeat from core client for 30 sec - exiting 21:59:52 (5076): No heartbeat from core client for 30 sec - exiting 21:59:53 (5076): No heartbeat from core client for 30 sec - exiting 21:59:54 (5076): No heartbeat from core client for 30 sec - exiting 21:59:55 (5076): No heartbeat from core client for 30 sec - exiting 21:59:56 (5076): No heartbeat from core client for 30 sec - exiting 21:59:57 (5076): No heartbeat from core client for 30 sec - exiting 21:59:58 (5076): No heartbeat from core client for 30 sec - exiting 21:59:59 (5076): No heartbeat from core client for 30 sec - exiting 22:00:00 (5076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:37:54 (4848): No heartbeat from core client for 30 sec - exiting 13:37:56 (4848): No heartbeat from core client for 30 sec - exiting 13:37:57 (4848): No heartbeat from core client for 30 sec - exiting 13:37:58 (4848): No heartbeat from core client for 30 sec - exiting 13:37:59 (4848): No heartbeat from core client for 30 sec - exiting 13:38:00 (4848): No heartbeat from core client for 30 sec - exiting 13:38:01 (4848): No heartbeat from core client for 30 sec - exiting 13:38:02 (4848): No heartbeat from core client for 30 sec - exiting 13:38:03 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=2 Model crash detected, will try to restart... 11:49:41 (5048): No heartbeat from core client for 30 sec - exiting 11:49:43 (5048): No heartbeat from core client for 30 sec - exiting 11:49:44 (5048): No heartbeat from core client for 30 sec - exiting 11:49:45 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5072, selfPID=5072, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4312, selfPID=4312, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CGntrollelobrl :: ker:: CPDN proCPDN procet runn ningr, exiting, bRetVabRetV, checkPIDec, selfPID=48fPID=2412t,=2 MonCtr=2 Model crash detected, will try to restart... 12:41:20 (2072): No heartbeat from core client for 30 sec - exiting 12:41:21 (2072): No heartbeat from core client for 30 sec - exiting 12:41:22 (2072): No heartbeat from core client for 30 sec - exiting 12:41:23 (2072): No heartbeat from core client for 30 sec - exiting 12:41:24 (2072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:16:00 (4792): No heartbeat from core client for 30 sec - exiting 10:16:01 (4792): No heartbeat from core client for 30 sec - exiting 10:16:02 (4792): No heartbeat from core client for 30 sec - exiting 10:16:03 (4792): No heartbeat from core client for 30 sec - exiting 10:16:05 (4792): No heartbeat from core client for 30 sec - exiting 10:16:06 (4792): No heartbeat from core client for 30 sec - exiting 10:16:07 (4792): No heartbeat from core client for 30 sec - exiting 10:16:08 (4792): No heartbeat from core client for 30 sec - exiting 10:16:09 (4792): No heartbeat from core client for 30 sec - exiting 10:16:10 (4792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:11:06 (4728): No heartbeat from core client for 30 sec - exiting 12:11:07 (4728): No heartbeat from core client for 30 sec - exiting 12:11:08 (4728): No heartbeat from core client for 30 sec - exiting 12:11:09 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=584, selfPID=584, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2632, selfPID=2632, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1968, selfPID=1968, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5408, selfPID=5408, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:02:01 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:02:55 (4888): No heartbeat from core client for 30 sec - exiting 13:02:56 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2060, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4776, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Work15:27:34 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6012, iMonCtr=2 18:07:02 (4884): No heartbeat from core client for 30 sec - exiting 18:07:03 (4884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:59 (424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:13:24 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:16:59 (3172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:18:10 (3164): No heartbeat from core client for 30 sec - exiting 18:18:11 (3164): No heartbeat from core client for 30 sec - exiting 18:18:12 (3164): No heartbeat from core client for 30 sec - exiting 18:18:13 (3164): No heartbeat from core client for 30 sec - exiting 18:18:14 (3164): No heartbeat from core client for 30 sec - exiting 18:18:15 (3164): No heartbeat from core client for 30 sec - exiting 18:18:16 (3164): No heartbeat from core client for 30 sec - exiting 18:18:17 (3164): No heartbeat from core client for 30 sec - exiting 18:18:18 (3164): No heartbeat from core client for 30 sec - exiting 18:18:19 (3164): No heartbeat from core client for 30 sec - exiting 18:18:20 (3164): No heartbeat from core client for 30 sec - exiting 18:18:21 (3164): No heartbeat from core client for 30 sec - exiting 18:18:22 (3164): No heartbeat from core client for 30 sec - exiting 18:18:23 (3164): No heartbeat from core client for 30 sec - exiting 18:18:24 (3164): No heartbeat from core client for 30 sec - exiting 18:18:25 (3164): No heartbeat from core client for 30 sec - exiting 18:18:26 (3164): No heartbeat from core client for 30 sec - exiting 18:18:27 (3164): No heartbeat from core client for 30 sec - exiting 18:18:28 (3164): No heartbeat from core client for 30 sec - exiting 18:18:29 (3164): No heartbeat from core client for 30 sec - exiting 18:18:30 (3164): No heartbeat from core client for 30 sec - exiting 18:19:01 (3164): No heartbeat from core client for 30 sec - exiting 18:19:02 (3164): No heartbeat from core client for 30 sec - exiting 18:19:03 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_eu_6d32_2006_1_007478971_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Nov 2011 17:39:53	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	126,816	338,244	2.6672
19 Nov 2011 17:41:20	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	115,296	310,020	2.6889
19 Nov 2011 10:18:59	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	103,776	282,230	2.7196
19 Nov 2011 09:13:46	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	92,260	252,687	2.7389
19 Nov 2011 09:13:46	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	92,256	252,338	2.7352
06 Nov 2011 16:28:11	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	80,736	221,958	2.7492
05 Nov 2011 19:28:14	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	69,216	191,709	2.7697
04 Nov 2011 22:07:43	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	57,703	160,383	2.7795
04 Nov 2011 21:19:21	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	57,698	160,000	2.7731
31 Oct 2011 18:42:41	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	57,696	159,646	2.7670
31 Oct 2011 18:09:04	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	46,182	128,788	2.7887
31 Oct 2011 18:09:04	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	46,176	128,388	2.7804
14 Oct 2011 12:17:21	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	34,656	96,875	2.7953
11 Oct 2011 12:49:14	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	23,136	64,983	2.8087
06 Oct 2011 15:31:29	1069880	13439817	hadam3p_eu_6d32_2006_1_007478971_0	11,616	32,980	2.8392