Task 15442816

Name	hadcm3n_zlic_1880_40_008246625_1
Workunit	8401749
Created	21 Nov 2012, 2:31:22 UTC
Sent	21 Nov 2012, 2:31:31 UTC
Report deadline	20 Feb 2013, 9:58:42 UTC
Received	22 Dec 2012, 21:06:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1084861
Run time	16 days 13 hours 21 min 40 sec
CPU time	16 days 3 hours 25 min 44 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	3.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:05:52 (2904): No heartbeat from core client for 30 sec - exiting 17:05:53 (2904): No heartbeat from core client for 30 sec - exiting 17:05:54 (2904): No heartbeat from core client for 30 sec - exiting 17:05:55 (2904): No heartbeat from core client for 30 sec - exiting 17:05:56 (2904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:05:57 (2904): No heartbeat from core client for 30 sec - exiting 17:06:42 (2524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:07:42 (3872): No heartbeat from core client for 30 sec - exiting 17:07:43 (3872): No heartbeat from core client for 30 sec - exiting 17:07:44 (3872): No heartbeat from core client for 30 sec - exiting 17:07:46 (3872): No heartbeat from core client for 30 sec - exiting 17:07:47 (3872): No heartbeat from core client for 30 sec - exiting 17:07:48 (3872): No heartbeat from core client for 30 sec - exiting 17:07:49 (3872): No heartbeat from core client for 30 sec - exiting 17:07:50 (3872): No heartbeat from core client for 30 sec - exiting 17:07:51 (3872): No heartbeat from core client for 30 sec - exiting 17:07:52 (3872): No heartbeat from core client for 30 sec - exiting 17:07:53 (3872): No heartbeat from core client for 30 sec - exiting 17:07:54 (3872): No heartbeat from core client for 30 sec - exiting 17:07:55 (3872): No heartbeat from core client for 30 sec - exiting 17:07:56 (3872): No heartbeat from core client for 30 sec - exiting 17:07:58 (3872): No heartbeat from core client for 30 sec - exiting 17:07:59 (3872): No heartbeat from core client for 30 sec - exiting 17:08:00 (3872): No heartbeat from core client for 30 sec - exiting 17:08:01 (3872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:21:47 (5444): No heartbeat from core client for 30 sec - exiting 04:21:48 (5444): No heartbeat from core client for 30 sec - exiting 04:21:49 (5444): No heartbeat from core client for 30 sec - exiting 04:21:50 (5444): No heartbeat from core client for 30 sec - exiting 04:21:51 (5444): No heartbeat from core client for 30 sec - exiting 04:21:53 (5444): No heartbeat from core client for 30 sec - exiting 04:21:54 (5444): No heartbeat from core client for 30 sec - exiting 04:21:55 (5444): No heartbeat from core client for 30 sec - exiting 04:21:56 (5444): No heartbeat from core client for 30 sec - exiting 04:21:57 (5444): No heartbeat from core client for 30 sec - exiting 04:21:58 (5444): No heartbeat from core client for 30 sec - exiting 04:21:59 (5444): No heartbeat from core client for 30 sec - exiting 04:22:00 (5444): No heartbeat from core client for 30 sec - exiting 04:22:01 (5444): No heartbeat from core client for 30 sec - exiting 04:22:02 (5444): No heartbeat from core client for 30 sec - exiting 04:22:04 (5444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:22:05 (5444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 05:24:13 (4988): No heartbeat from core client for 30 sec - exiting 05:24:14 (4988): No heartbeat from core client for 30 sec - exiting 05:24:15 (4988): No heartbeat from core client for 30 sec - exiting 05:24:16 (4988): No heartbeat from core client for 30 sec - exiting 05:24:17 (4988): No heartbeat from core client for 30 sec - exiting 05:24:19 (4988): No heartbeat from core client for 30 sec - exiting 05:24:20 (4988): No heartbeat from core client for 30 sec - exiting 05:24:21 (4988): No heartbeat from core client for 30 sec - exiting 05:24:22 (4988): No heartbeat from core client for 30 sec - exiting 05:24:23 (4988): No heartbeat from core client for 30 sec - exiting 05:24:24 (4988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:27:33 (7240): No heartbeat from core client for 30 sec - exiting 06:27:34 (7240): No heartbeat from core client for 30 sec - exiting 06:27:35 (7240): No heartbeat from core client for 30 sec - exiting 06:27:37 (7240): No heartbeat from core client for 30 sec - exiting 06:27:38 (7240): No heartbeat from core client for 30 sec - exiting 06:27:39 (7240): No heartbeat from core client for 30 sec - exiting 06:27:40 (7240): No heartbeat from core client for 30 sec - exiting 06:27:41 (7240): No heartbeat from core client for 30 sec - exiting 06:27:42 (7240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:27:43 (7240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:21:22 (5768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zlicko.pj83c10 Error converting file to netcdf: dataout/zlicko.pi83c10 Error converting file to netcdf: dataout/zlicko.pf83c10 Error converting file to netcdf: dataout/zlicka.ph83c10 Error converting file to netcdf: dataout/zlicka.pg83c10 Error converting file to netcdf: dataout/zlicka.pe83c10 Error converting file to netcdf: dataout/zlicka.pd83c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:10:41 (5560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:07:39 (7896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:28:26 (6540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:29:07 (3372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:26:46 (7636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:26 (7960): No heartbeat from core client for 30 sec - exiting 03:27:27 (7960): No heartbeat from core client for 30 sec - exiting 03:27:28 (7960): No heartbeat from core client for 30 sec - exiting 03:27:29 (7960): No heartbeat from core client for 30 sec - exiting 03:27:30 (7960): No heartbeat from core client for 30 sec - exiting 03:27:31 (7960): No heartbeat from core client for 30 sec - exiting 03:27:32 (7960): No heartbeat from core client for 30 sec - exiting 03:27:33 (7960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:20:57 (7188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:45 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:35:15 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:48:03 (6184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:18:26 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:07:09 (6988): No heartbeat from core client for 30 sec - exiting 18:07:10 (6988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:34:11 (4592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:35:52 (7496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:16:48 (7928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:17:25 (5300): No heartbeat from core client for 30 sec - exiting 18:17:26 (5300): No heartbeat from core client for 30 sec - exiting 18:17:27 (5300): No heartbeat from core client for 30 sec - exiting 18:17:28 (5300): No heartbeat from core client for 30 sec - exiting 18:17:29 (5300): No heartbeat from core client for 30 sec - exiting 18:17:31 (5300): No heartbeat from core client for 30 sec - exiting 18:17:32 (5300): No heartbeat from core client for 30 sec - exiting 18:17:33 (5300): No heartbeat from core client for 30 sec - exiting 18:17:34 (5300): No heartbeat from core client for 30 sec - exiting 18:17:35 (5300): No heartbeat from core client for 30 sec - exiting 18:17:36 (5300): No heartbeat from core client for 30 sec - exiting 18:17:37 (5300): No heartbeat from core client for 30 sec - exiting 18:17:38 (5300): No heartbeat from core client for 30 sec - exiting 18:17:39 (5300): No heartbeat from core client for 30 sec - exiting 18:17:40 (5300): No heartbeat from core client for 30 sec - exiting 18:17:41 (5300): No heartbeat from core client for 30 sec - exiting 18:17:43 (5300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:16:44 (7280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:17:43 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:05:05 (7964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:14:52 (3348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:02:58 (6364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:33:14 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:21:05 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:03:36 (4344): No heartbeat from core client for 30 sec - exiting 12:03:38 (4344): No heartbeat from core client for 30 sec - exiting 12:03:39 (4344): No heartbeat from core client for 30 sec - exiting 12:03:40 (4344): No heartbeat from core client for 30 sec - exiting 12:03:41 (4344): No heartbeat from core client for 30 sec - exiting 12:03:43 (4344): No heartbeat from core client for 30 sec - exiting 12:03:44 (4344): No heartbeat from core client for 30 sec - exiting 12:03:45 (4344): No heartbeat from core client for 30 sec - exiting 12:03:46 (4344): No heartbeat from core client for 30 sec - exiting 12:03:47 (4344): No heartbeat from core client for 30 sec - exiting 12:03:48 (4344): No heartbeat from core client for 30 sec - exiting 12:03:49 (4344): No heartbeat from core client for 30 sec - exiting 12:03:50 (4344): No heartbeat from core client for 30 sec - exiting 12:03:51 (4344): No heartbeat from core client for 30 sec - exiting 12:03:52 (4344): No heartbeat from core client for 30 sec - exiting 12:03:53 (4344): No heartbeat from core client for 30 sec - exiting 12:03:55 (4344): No heartbeat from core client for 30 sec - exiting 12:03:56 (4344): No heartbeat from core client for 30 sec - exiting 12:03:57 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:04:41 (10188): No heartbeat from core client for 30 sec - exiting 12:04:42 (10188): No heartbeat from core client for 30 sec - exiting 12:04:43 (10188): No heartbeat from core client for 30 sec - exiting 12:04:44 (10188): No heartbeat from core client for 30 sec - exiting 12:04:45 (10188): No heartbeat from core client for 30 sec - exiting 12:04:46 (10188): No heartbeat from core client for 30 sec - exiting 12:04:47 (10188): No heartbeat from core client for 30 sec - exiting 12:04:48 (10188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:43:15 (7972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:45:15 (8632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:12:14 (2780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:19:51 (8764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:12:06 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:25:32 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:26:30 (10004): No heartbeat from core client for 30 sec - exiting 11:26:31 (10004): No heartbeat from core client for 30 sec - exiting 11:26:33 (10004): No heartbeat from core client for 30 sec - exiting 11:26:34 (10004): No heartbeat from core client for 30 sec - exiting 11:26:35 (10004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:12:03 (10080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:55:47 (8240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:47:10 (6668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:57:18 (10180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:23:10 (7008): No heartbeat from core client for 30 sec - exiting 13:23:11 (7008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:18:08 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:19:00 (5804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:06:03 (7972): No heartbeat from core client for 30 sec - exiting 18:06:04 (7972): No heartbeat from core client for 30 sec - exiting 18:06:05 (7972): No heartbeat from core client for 30 sec - exiting 18:06:06 (7972): No heartbeat from core client for 30 sec - exiting 18:06:07 (7972): No heartbeat from core client for 30 sec - exiting 18:06:08 (7972): No heartbeat from core client for 30 sec - exiting 18:06:09 (7972): No heartbeat from core client for 30 sec - exiting 18:06:10 (7972): No heartbeat from core client for 30 sec - exiting 18:06:11 (7972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:02:00 (5308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:24:48 (6736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:45:18 (7180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:50:15 (7228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:07:16 (9716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:19:10 (7732): No heartbeat from core client for 30 sec - exiting 19:19:11 (7732): No heartbeat from core client for 30 sec - exiting 19:19:12 (7732): No heartbeat from core client for 30 sec - exiting 19:19:13 (7732): No heartbeat from core client for 30 sec - exiting 19:19:14 (7732): No heartbeat from core client for 30 sec - exiting 19:19:15 (7732): No heartbeat from core client for 30 sec - exiting 19:19:16 (7732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:17 (7732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:26:20 (9972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:05:42 (8928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:08:34 (9920): No heartbeat from core client for 30 sec - exiting 10:08:35 (9920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:33:19 (1880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:40:58 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:43:25 (6724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:51:23 (8504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:06:02 (4852): No heartbeat from core client for 30 sec - exiting 18:06:03 (4852): No heartbeat from core client for 30 sec - exiting 18:06:04 (4852): No heartbeat from core client for 30 sec - exiting 18:06:05 (4852): No heartbeat from core client for 30 sec - exiting 18:06:06 (4852): No heartbeat from core client for 30 sec - exiting 18:06:07 (4852): No heartbeat from core client for 30 sec - exiting 18:06:08 (4852): No heartbeat from core client for 30 sec - exiting 18:06:09 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:11 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:09:39 (3204): No heartbeat from core client for 30 sec - exiting 19:09:40 (3204): No heartbeat from core client for 30 sec - exiting 19:09:41 (3204): No heartbeat from core client for 30 sec - exiting 19:09:42 (3204): No heartbeat from core client for 30 sec - exiting 19:09:43 (3204): No heartbeat from core client for 30 sec - exiting 19:09:44 (3204): No heartbeat from core client for 30 sec - exiting 19:09:45 (3204): No heartbeat from core client for 30 sec - exiting 19:09:46 (3204): No heartbeat from core client for 30 sec - exiting 19:09:47 (3204): No heartbeat from core client for 30 sec - exiting 19:09:49 (3204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:27:33 (9832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:08:00 (9628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:57:29 (9164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:44:12 (11020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:11 (5200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:31:01 (4480): No heartbeat from core client for 30 sec - exiting 21:31:05 (4480): No heartbeat from core client for 30 sec - exiting 21:31:06 (4480): No heartbeat from core client for 30 sec - exiting 21:31:08 (4480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:31:09 (4480): No heartbeat from core client for 30 sec - exiting 21:31:10 (4480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:46:46 (9720): No heartbeat from core client for 30 sec - exiting 22:46:48 (9720): No heartbeat from core client for 30 sec - exiting 22:46:49 (9720): No heartbeat from core client for 30 sec - exiting 22:46:50 (9720): No heartbeat from core client for 30 sec - exiting 22:46:51 (9720): No heartbeat from core client for 30 sec - exiting 22:46:52 (9720): No heartbeat from core client for 30 sec - exiting 22:46:54 (9720): No heartbeat from core client for 30 sec - exiting 22:46:55 (9720): No heartbeat from core client for 30 sec - exiting 22:46:56 (9720): No heartbeat from core client for 30 sec - exiting 22:46:57 (9720): No heartbeat from core client for 30 sec - exiting 22:46:58 (9720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:46:59 (9720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:01:40 (256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:53:25 (2660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:19:58 (10536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:22:01 (5328): No heartbeat from core client for 30 sec - exiting 16:22:02 (5328): No heartbeat from core client for 30 sec - exiting 16:22:03 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:04 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:04:58 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:18:49 (10304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:28:47 (9264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:26:47 (11296): No heartbeat from core client for 30 sec - exiting 19:26:48 (11296): No heartbeat from core client for 30 sec - exiting 19:26:49 (11296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:30:43 (7264): No heartbeat from core client for 30 sec - exiting 20:30:44 (7264): No heartbeat from core client for 30 sec - exiting 20:30:46 (7264): No heartbeat from core client for 30 sec - exiting 20:30:47 (7264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:50:44 (11228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:54:44 (10080): No heartbeat from core client for 30 sec - exiting 17:54:45 (10080): No heartbeat from core client for 30 sec - exiting 17:54:46 (10080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:13:05 (1544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:13:50 (8724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:30 (12140): No heartbeat from core client for 30 sec - exiting 16:14:31 (12140): No heartbeat from core client for 30 sec - exiting 16:14:32 (12140): No heartbeat from core client for 30 sec - exiting 16:14:33 (12140): No heartbeat from core client for 30 sec - exiting 16:14:34 (12140): No heartbeat from core client for 30 sec - exiting 16:14:35 (12140): No heartbeat from core client for 30 sec - exiting 16:14:36 (12140): No heartbeat from core client for 30 sec - exiting 16:14:37 (12140): No heartbeat from core client for 30 sec - exiting 16:14:38 (12140): No heartbeat from core client for 30 sec - exiting 16:14:39 (12140): No heartbeat from core client for 30 sec - exiting 16:14:41 (12140): No heartbeat from core client for 30 sec - exiting 16:14:42 (12140): No heartbeat from core client for 30 sec - exiting 16:14:43 (12140): No heartbeat from core client for 30 sec - exiting 16:14:44 (12140): No heartbeat from core client for 30 sec - exiting 16:14:45 (12140): No heartbeat from core client for 30 sec - exiting 16:14:46 (12140): No heartbeat from core client for 30 sec - exiting 16:14:47 (12140): No heartbeat from core client for 30 sec - exiting 16:14:48 (12140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:33:20 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:45:32 (2932): No heartbeat from core client for 30 sec - exiting 03:45:33 (2932): No heartbeat from core client for 30 sec - exiting 03:45:34 (2932): No heartbeat from core client for 30 sec - exiting 03:45:36 (2932): No heartbeat from core client for 30 sec - exiting 03:45:37 (2932): No heartbeat from core client for 30 sec - exiting 03:45:38 (2932): No heartbeat from core client for 30 sec - exiting 03:45:39 (2932): No heartbeat from core client for 30 sec - exiting 03:45:40 (2932): No heartbeat from core client for 30 sec - exiting 03:45:41 (2932): No heartbeat from core client for 30 sec - exiting 03:45:42 (2932): No heartbeat from core client for 30 sec - exiting 03:45:43 (2932): No heartbeat from core client for 30 sec - exiting 03:45:44 (2932): No heartbeat from core client for 30 sec - exiting 03:45:45 (2932): No heartbeat from core client for 30 sec - exiting 03:45:46 (2932): No heartbeat from core client for 30 sec - exiting 03:45:48 (2932): No heartbeat from core client for 30 sec - exiting 03:45:49 (2932): No heartbeat from core client for 30 sec - exiting 03:45:50 (2932): No heartbeat from core client for 30 sec - exiting 03:45:51 (2932): No heartbeat from core client for 30 sec - exiting 03:45:52 (2932): No heartbeat from core client for 30 sec - exiting 03:45:53 (2932): No heartbeat from core client for 30 sec - exiting 03:45:54 (2932): No heartbeat from core client for 30 sec - exiting 03:45:55 (2932): No heartbeat from core client for 30 sec - exiting 03:45:56 (2932): No heartbeat from core client for 30 sec - exiting 03:45:57 (2932): No heartbeat from core client for 30 sec - exiting 03:45:58 (2932): No heartbeat from core client for 30 sec - exiting 03:46:00 (2932): No heartbeat from core client for 30 sec - exiting 03:46:01 (2932): No heartbeat from core client for 30 sec - exiting 03:46:02 (2932): No heartbeat from core client for 30 sec - exiting 03:46:03 (2932): No heartbeat from core client for 30 sec - exiting 03:46:04 (2932): No heartbeat from core client for 30 sec - exiting 03:46:05 (2932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:20:47 (5668): No heartbeat from core client for 30 sec - exiting 12:20:48 (5668): No heartbeat from core client for 30 sec - exiting 12:20:49 (5668): No heartbeat from core client for 30 sec - exiting 12:20:50 (5668): No heartbeat from core client for 30 sec - exiting 12:20:51 (5668): No heartbeat from core client for 30 sec - exiting 12:20:52 (5668): No heartbeat from core client for 30 sec - exiting 12:20:53 (5668): No heartbeat from core client for 30 sec - exiting 12:20:54 (5668): No heartbeat from core client for 30 sec - exiting 12:20:56 (5668): No heartbeat from core client for 30 sec - exiting 12:20:57 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:52:57 (7048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... 15:27:46 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:14:14 (4316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:21:34 (5628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:07:57 (5492): No heartbeat from core client for 30 sec - exiting 11:07:58 (5492): No heartbeat from core client for 30 sec - exiting 11:08:00 (5492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:31:15 (3972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Dec 2012 00:18:10	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	440,640	1,325,090	3.0072
19 Dec 2012 03:46:06	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	414,720	1,255,962	3.0285
16 Dec 2012 18:09:06	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	388,800	1,177,035	3.0274
14 Dec 2012 19:58:59	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	362,880	1,100,478	3.0326
14 Dec 2012 10:56:52	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	336,960	1,021,643	3.0319
14 Dec 2012 10:56:52	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	311,040	946,126	3.0418
14 Dec 2012 10:56:52	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	285,120	872,388	3.0597
06 Dec 2012 01:57:06	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	259,200	801,072	3.0906
04 Dec 2012 10:17:33	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	233,280	724,135	3.1041
02 Dec 2012 21:46:58	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	207,360	636,571	3.0699
01 Dec 2012 12:06:55	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	181,440	551,546	3.0398
30 Nov 2012 02:00:23	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	155,520	464,441	2.9864
27 Nov 2012 15:45:09	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	129,600	388,983	3.0014
26 Nov 2012 12:31:27	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	103,680	323,248	3.1177
25 Nov 2012 02:59:12	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	77,760	245,920	3.1626
23 Nov 2012 14:29:04	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	51,840	158,240	3.0525
22 Nov 2012 12:48:20	1084861	15442816	hadcm3n_zlic_1880_40_008246625_1	25,920	81,089	3.1284