Task 17968213

Name	hadam3pm2_k2q3_1959_10_009464758_2
Workunit	9546992
Created	20 Feb 2015, 3:30:03 UTC
Sent	20 Feb 2015, 13:10:36 UTC
Report deadline	2 Feb 2016, 18:30:36 UTC
Received	12 Mar 2015, 2:55:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1100148
Run time	13 days 21 hours 22 min 49 sec
CPU time	11 days 18 hours 36 min
Validate state	Invalid
Credit	10,064.32
Device peak FLOPS	2.72 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.27</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|9dec.nc oa.pc\|0jan.nc oa.pc\|0feb.nc oa.pc\|0mar.nc oa.pc\|0apr.nc oa.pc\|0may.nc oa.pc\|0jun.nc oa.pc\|0jul.nc oa.pc\|0aug.nc oa.pc\|0sep.nc oa.pc\|0oct.nc oa.pc\|0nov.nc oa.pe\|9dec.nc oa.pe\|0jan.nc oa.pe\|0feb.nc oa.pe\|0mar.nc oa.pe\|0apr.nc oa.pe\|0may.nc oa.pe\|0jun.nc oa.pe\|0jul.nc oa.pe\|0aug.nc oa.pe\|0sep.nc oa.pe\|0oct.nc oa.pe\|0nov.nc Suspended CPDN Monitor - Suspend request from BOINC... 12:00:35 (14628): No heartbeat from client for 30 sec - exiting 12:00:35 (14628): timer handler: client dead, exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:23:16 (8100): No heartbeat from client for 30 sec - exiting 03:23:16 (8100): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:24:30 (7867): No heartbeat from client for 30 sec - exiting 03:24:30 (7867): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:26:30 (8062): No heartbeat from client for 30 sec - exiting 03:26:30 (8062): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:31 (8176): No heartbeat from client for 30 sec - exiting 03:27:31 (8176): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:30:11 (8286): No heartbeat from client for 30 sec - exiting 03:30:11 (8286): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:31:53 (8529): No heartbeat from client for 30 sec - exiting 03:31:53 (8529): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:33:47 (8700): No heartbeat from client for 30 sec - exiting 03:33:47 (8700): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:35:41 (8810): No heartbeat from client for 30 sec - exiting 03:35:41 (8810): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:36:28 (8993): No heartbeat from client for 30 sec - exiting 03:36:28 (8993): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:37:40 (9102): No heartbeat from client for 30 sec - exiting 03:37:40 (9102): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:38:56 (9273): No heartbeat from client for 30 sec - exiting 03:38:56 (9273): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:40:19 (9389): No heartbeat from client for 30 sec - exiting 03:40:19 (9389): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:42:26 (9568): No heartbeat from client for 30 sec - exiting 03:42:26 (9568): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:45:03 (9739): No heartbeat from client for 30 sec - exiting 03:45:03 (9739): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:47:04 (9923): No heartbeat from client for 30 sec - exiting 03:47:04 (9923): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:16:57 (4239): No heartbeat from client for 30 sec - exiting 04:16:57 (4239): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:08 (3141): No heartbeat from client for 30 sec - exiting 04:18:08 (3141): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:59 (3260): No heartbeat from client for 30 sec - exiting 04:18:59 (3260): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:25 (3461): No heartbeat from client for 30 sec - exiting 04:21:25 (3461): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:23:06 (3664): No heartbeat from client for 30 sec - exiting 04:23:06 (3664): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:26:05 (3934): No heartbeat from client for 30 sec - exiting 04:26:05 (3934): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:27:26 (4058): No heartbeat from client for 30 sec - exiting 04:27:26 (4058): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:28:20 (4179): No heartbeat from client for 30 sec - exiting 04:28:20 (4179): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:29:19 (4294): No heartbeat from client for 30 sec - exiting 04:29:19 (4294): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:02 (4410): No heartbeat from client for 30 sec - exiting 04:31:02 (4410): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:33:08 (4659): No heartbeat from client for 30 sec - exiting 04:33:08 (4659): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:35:39 (4832): No heartbeat from client for 30 sec - exiting 04:35:39 (4832): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:39:17 (5086): No heartbeat from client for 30 sec - exiting 04:39:17 (5086): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:40:16 (5265): No heartbeat from client for 30 sec - exiting 04:40:16 (5265): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|2dec.nc oa.pc\|3jan.nc oa.pc\|3feb.nc oa.pc\|3mar.nc oa.pc\|3apr.nc oa.pc\|3may.nc oa.pc\|3jun.nc oa.pc\|3jul.nc oa.pc\|3aug.nc oa.pc\|3sep.nc oa.pc\|3oct.nc oa.pc\|3nov.nc oa.pe\|2dec.nc oa.pe\|3jan.nc oa.pe\|3feb.nc oa.pe\|3mar.nc oa.pe\|3apr.nc oa.pe\|3may.nc oa.pe\|3jun.nc oa.pe\|3jul.nc oa.pe\|3aug.nc oa.pe\|3sep.nc oa.pe\|3oct.nc oa.pe\|3nov.nc 03:08:55 (5381): No heartbeat from client for 30 sec - exiting 03:08:55 (5381): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:11:37 (11433): No heartbeat from client for 30 sec - exiting 03:11:37 (11433): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:18 (11625): No heartbeat from client for 30 sec - exiting 03:12:18 (11625): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:15:59 (11946): No heartbeat from client for 30 sec - exiting 03:15:59 (11946): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:22:57 (12455): No heartbeat from client for 30 sec - exiting 03:22:57 (12455): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:29:27 (12902): No heartbeat from client for 30 sec - exiting 03:29:27 (12902): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:32:40 (13151): No heartbeat from client for 30 sec - exiting 03:32:40 (13151): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:35:22 (13389): No heartbeat from client for 30 sec - exiting 03:35:22 (13389): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:39:02 (13718): No heartbeat from client for 30 sec - exiting 03:39:02 (13718): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:00:43 (13908): No heartbeat from client for 30 sec - exiting 04:00:43 (13908): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:02:23 (7809): No heartbeat from client for 30 sec - exiting 04:02:23 (7809): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:07:08 (8259): No heartbeat from client for 30 sec - exiting 04:07:08 (8259): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|5dec.nc oa.pc\|6jan.nc oa.pc\|6feb.nc oa.pc\|6mar.nc oa.pc\|6apr.nc oa.pc\|6may.nc oa.pc\|6jun.nc oa.pc\|6jul.nc oa.pc\|6aug.nc oa.pc\|6sep.nc oa.pc\|6oct.nc oa.pc\|6nov.nc oa.pe\|5dec.nc oa.pe\|6jan.nc oa.pe\|6feb.nc oa.pe\|6mar.nc oa.pe\|6apr.nc oa.pe\|6may.nc oa.pe\|6jun.nc oa.pe\|6jul.nc oa.pe\|6aug.nc oa.pe\|6sep.nc oa.pe\|6oct.nc oa.pe\|6nov.nc oa.pc\|6dec.nc oa.pc\|7jan.nc oa.pc\|7feb.nc oa.pc\|7mar.nc oa.pc\|7apr.nc oa.pc\|7may.nc oa.pc\|7jun.nc oa.pc\|7jul.nc oa.pc\|7aug.nc oa.pc\|7sep.nc oa.pc\|7oct.nc oa.pc\|7nov.nc oa.pe\|6dec.nc oa.pe\|7jan.nc oa.pe\|7feb.nc oa.pe\|7mar.nc oa.pe\|7apr.nc oa.pe\|7may.nc oa.pe\|7jun.nc oa.pe\|7jul.nc oa.pe\|7aug.nc oa.pe\|7sep.nc oa.pe\|7oct.nc oa.pe\|7nov.nc 09:18:37 (8620): No heartbeat from client for 30 sec - exiting 09:18:37 (8620): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:22:40 (26402): No heartbeat from client for 30 sec - exiting 09:22:40 (26402): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:51 (26666): No heartbeat from client for 30 sec - exiting 09:54:51 (26666): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:05 (28700): No heartbeat from client for 30 sec - exiting 09:57:05 (28700): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:59:36 (28889): No heartbeat from client for 30 sec - exiting 09:59:36 (28889): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:00:27 (29076): No heartbeat from client for 30 sec - exiting 10:00:27 (29076): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:02:14 (29270): No heartbeat from client for 30 sec - exiting 10:02:14 (29270): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:05:43 (29520): No heartbeat from client for 30 sec - exiting 10:05:43 (29520): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:06:32 (29658): No heartbeat from client for 30 sec - exiting 10:06:32 (29658): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:08:04 (29965): No heartbeat from client for 30 sec - exiting 10:08:04 (29965): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:12:42 (30296): No heartbeat from client for 30 sec - exiting 10:12:42 (30296): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:13:24 (30420): No heartbeat from client for 30 sec - exiting 10:13:24 (30420): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:15:02 (30604): No heartbeat from client for 30 sec - exiting 10:15:02 (30604): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:18:05 (30866): No heartbeat from client for 30 sec - exiting 10:18:05 (30866): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:20:59 (31054): No heartbeat from client for 30 sec - exiting 10:20:59 (31054): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:17 (31252): No heartbeat from client for 30 sec - exiting 10:23:17 (31252): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:55 (31434): No heartbeat from client for 30 sec - exiting 10:24:55 (31434): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:27:23 (31636): No heartbeat from client for 30 sec - exiting 10:27:23 (31636): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:05 (31821): No heartbeat from client for 30 sec - exiting 10:28:05 (31821): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:20 (32009): No heartbeat from client for 30 sec - exiting 10:30:20 (32009): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:59 (32196): No heartbeat from client for 30 sec - exiting 10:31:59 (32196): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:34:54 (32446): No heartbeat from client for 30 sec - exiting 10:34:54 (32446): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:37:07 (32640): No heartbeat from client for 30 sec - exiting 10:37:07 (32640): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:46 (348): No heartbeat from client for 30 sec - exiting 10:38:46 (348): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:41:36 (554): No heartbeat from client for 30 sec - exiting 10:41:36 (554): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:42:25 (672): No heartbeat from client for 30 sec - exiting 10:42:25 (672): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:43:53 (856): No heartbeat from client for 30 sec - exiting 10:43:53 (856): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:46:37 (1049): No heartbeat from client for 30 sec - exiting 10:46:37 (1049): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:55 (1688): No heartbeat from client for 30 sec - exiting 10:54:55 (1688): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:10 (3525): No heartbeat from client for 30 sec - exiting 11:18:10 (3525): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:42:03 (5173): No heartbeat from client for 30 sec - exiting 11:42:03 (5173): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:05:18 (6723): No heartbeat from client for 30 sec - exiting 12:05:18 (6723): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:29:54 (8535): No heartbeat from client for 30 sec - exiting 12:29:54 (8535): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:53:48 (10178): No heartbeat from client for 30 sec - exiting 12:53:48 (10178): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:21 (11817): No heartbeat from client for 30 sec - exiting 13:16:21 (11817): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:38:55 (13371): No heartbeat from client for 30 sec - exiting 13:38:55 (13371): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:49 (15020): No heartbeat from client for 30 sec - exiting 14:02:49 (15020): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:23 (16638): No heartbeat from client for 30 sec - exiting 14:25:23 (16638): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:47:56 (18204): No heartbeat from client for 30 sec - exiting 14:47:56 (18204): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:13:11 (20019): No heartbeat from client for 30 sec - exiting 15:13:11 (20019): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:35:45 (21581): No heartbeat from client for 30 sec - exiting 15:35:45 (21581): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:54 (23270): No heartbeat from client for 30 sec - exiting 16:00:54 (23270): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:33:16 (25539): No heartbeat from client for 30 sec - exiting 16:33:16 (25539): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:03:37 (27615): No heartbeat from client for 30 sec - exiting 17:03:37 (27615): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:32:38 (29689): No heartbeat from client for 30 sec - exiting 17:32:38 (29689): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:39 (31904): No heartbeat from client for 30 sec - exiting 18:05:39 (31904): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:34:39 (1528): No heartbeat from client for 30 sec - exiting 18:34:39 (1528): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:59 (3853): No heartbeat from client for 30 sec - exiting 19:06:59 (3853): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:19 (5982): No heartbeat from client for 30 sec - exiting 19:37:19 (5982): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:19 (7937): No heartbeat from client for 30 sec - exiting 20:06:19 (7937): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:39:20 (10290): No heartbeat from client for 30 sec - exiting 20:39:20 (10290): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:09:00 (12421): No heartbeat from client for 30 sec - exiting 21:09:00 (12421): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|8dec.nc oa.pc\|9jan.nc oa.pc\|9feb.nc oa.pc\|9mar.nc oa.pc\|9apr.nc oa.pc\|9may.nc oa.pc\|9jun.nc oa.pc\|9jul.nc oa.pc\|9aug.nc oa.pc\|9sep.nc oa.pc\|9oct.nc oa.pc\|9nov.nc oa.pe\|8dec.nc oa.pe\|9jan.nc oa.pe\|9feb.nc oa.pe\|9mar.nc oa.pe\|9apr.nc oa.pe\|9may.nc oa.pe\|9jun.nc oa.pe\|9jul.nc oa.pe\|9aug.nc oa.pe\|9sep.nc oa.pe\|9oct.nc oa.pe\|9nov.nc Suspended CPDN Monitor - Suspend request from BOINC... 02:56:27 (13094): called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3pm2_k2q3_1959_10_009464758_2_2.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3pm2_k2q3_1959_10_009464758_2_3.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3pm2_k2q3_1959_10_009464758_2_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3pm2_k2q3_1959_10_009464758_2_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3pm2_k2q3_1959_10_009464758_2_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Mar 2015 01:58:03	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	348,548	1,193,143	3.4232
11 Mar 2015 21:00:56	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	342,788	1,172,772	3.4213
11 Mar 2015 18:30:00	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	339,908	1,163,531	3.4231
11 Mar 2015 16:04:14	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	337,028	1,154,514	3.4256
11 Mar 2015 13:23:12	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	334,148	1,145,711	3.4288
11 Mar 2015 10:52:12	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	331,268	1,136,359	3.4303
11 Mar 2015 07:41:00	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	328,388	1,127,313	3.4329
11 Mar 2015 05:34:03	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	325,508	1,118,190	3.4352
11 Mar 2015 02:31:23	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	322,628	1,109,551	3.4391
11 Mar 2015 00:02:57	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	319,748	1,100,609	3.4421
10 Mar 2015 21:17:21	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	316,868	1,090,708	3.4422
10 Mar 2015 18:36:41	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	313,988	1,081,153	3.4433
10 Mar 2015 16:31:40	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	293,828	1,015,087	3.4547
10 Mar 2015 16:31:29	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	290,948	1,004,672	3.4531
10 Mar 2015 16:31:12	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	288,068	994,187	3.4512
10 Mar 2015 16:30:54	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	285,188	983,738	3.4494
10 Mar 2015 16:30:32	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	282,308	971,838	3.4425
10 Mar 2015 16:30:03	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	279,428	957,149	3.4254
10 Mar 2015 16:29:15	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	273,668	928,752	3.3937
10 Mar 2015 16:28:55	1100148	17968213	hadam3pm2_k2q3_1959_10_009464758_2	270,788	918,440	3.3917