Task 16045745

Name	hadcm3n_ofd4_1900_40_008475035_0
Workunit	8625874
Created	27 Sep 2013, 10:34:04 UTC
Sent	27 Sep 2013, 19:10:24 UTC
Report deadline	28 Dec 2013, 2:37:35 UTC
Received	16 Oct 2013, 12:40:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1229309
Run time	3 days 17 hours 52 min 57 sec
CPU time	3 days 15 hours 10 min 30 sec
Validate state	Invalid
Credit	2,177.28
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> lient for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:00:40 (6204): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:40:32 (6924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:49:11 (4088): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/15/13 05:39:19 Install Directory : Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_ofd4_1900_40_008475035;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 7, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 52, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 106256, QuotaPeakPagedPoolUsage: 106256 QuotaNonPagedPoolUsage: 5280, QuotaPeakNonPagedPoolUsage: 5280 - Virtual Memory Usage - VirtualSize: 897261568, PeakVirtualSize: 897261568 - Pagefile Usage - PagefileUsage: 33357824, PeakPagefileUsage: 106422272 - Working Set Size - WorkingSetSize: 4050944, PeakWorkingSetSize: 4050944, PageFaultCount: 1001 * Dump of thread ID 6448 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 37269072.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 - Registers - eax=1132fa04 ebx=00000001 ecx=00000000 edx=00000000 esi=1132fa8c edi=fffffffe eip=7d4e22d6 esp=1132fa00 ebp=1132fa54 cs=120023 ss=12002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa54 007bfa90 e06d7363 00000001 00000003 1132fa80 !RaiseException+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e22d6' 1132fa8c 007bbbcb 1132fa9c 00860978 0082f1b4 0085d354 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bfa90' 1132faa8 00406e8c 04600000 01000000 00500000 004083ee !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bbbcb' 1132fb10 00408130 1132fb24 001224ef 1132fb24 00122451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00406e8c' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 e06d7363 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 3900 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 37269068.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=337cff74 eip=7d61ccc6 esp=337cff38 ebp=337cff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 337cff9c 7d4d14ef 00000064 00000000 337cffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 337cffac 00749924 00000064 337cffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 337cffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 337cffec 00000000 00749910 00000000 00000000 33ce0000 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/15/13 05:39:38 Install Directory : Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_ofd4_1900_40_008475035;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 7, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 52, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 106256, QuotaPeakPagedPoolUsage: 106256 QuotaNonPagedPoolUsage: 5200, QuotaPeakNonPagedPoolUsage: 5200 - Virtual Memory Usage - VirtualSize: 880480256, PeakVirtualSize: 880480256 - Pagefile Usage - PagefileUsage: 16543744, PeakPagefileUsage: 32874496 - Working Set Size - WorkingSetSize: 4042752, PeakWorkingSetSize: 4042752, PageFaultCount: 999 * Dump of thread ID 7016 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 37270288.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 - Registers - eax=1132fa08 ebx=00000001 ecx=00000000 edx=00000000 esi=1132fa90 edi=fffffffe eip=7d4e22d6 esp=1132fa04 ebp=1132fa58 cs=7d610023 ss=2002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa58 007bfa90 e06d7363 00000001 00000003 1132fa84 !RaiseException+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e22d6' 1132fa90 007bbbcb 1132faa0 00860978 0082f1b4 0085d354 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bfa90' 1132faac 00406e7d 01000000 00500000 004083ee 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bbbcb' 1132fb10 00408130 1132fb24 001224ef 1132fb24 00122451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00406e7d' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 e06d7363 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 4008 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 37270288.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=337cff74 eip=7d61ccc6 esp=337cff38 ebp=337cff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 337cff9c 7d4d14ef 00000064 00000000 337cffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 337cffac 00749924 00000064 337cffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 337cffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 337cffec 00000000 00749910 00000000 00000000 00340050 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/15/13 05:39:43 Install Directory : Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_ofd4_1900_40_008475035;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 7, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 52, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 106256, QuotaPeakPagedPoolUsage: 106256 QuotaNonPagedPoolUsage: 5120, QuotaPeakNonPagedPoolUsage: 5120 - Virtual Memory Usage - VirtualSize: 875233280, PeakVirtualSize: 875233280 - Pagefile Usage - PagefileUsage: 11284480, PeakPagefileUsage: 16060416 - Working Set Size - WorkingSetSize: 4042752, PeakWorkingSetSize: 4042752, PageFaultCount: 999 * Dump of thread ID 5656 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 312500.000000, User Time: 156250.000000, Wait Time: 37270632.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 - Registers - eax=1132fa0c ebx=00000001 ecx=00000000 edx=00000000 esi=1132fa94 edi=fffffffe eip=7d4e22d6 esp=1132fa08 ebp=1132fa5c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa5c 007bfa90 e06d7363 00000001 00000003 1132fa88 !RaiseException+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e22d6' 1132fa94 007bbbcb 1132faa4 00860978 0082f1b4 0085d354 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bfa90' 1132fab0 00406e6e 00500000 004083ee 00000000 7d61f768 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bbbcb' 1132fb10 00408130 1132fb24 001224ef 1132fb24 00122451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00406e6e' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 e06d7363 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 3452 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 37270632.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=337cff74 eip=7d61ccc6 esp=337cff38 ebp=337cff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 337cff9c 7d4d14ef 00000064 00000000 337cffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 337cffac 00749924 00000064 337cffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 337cffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 337cffec 00000000 00749910 00000000 00000000 000000c8 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/15/13 05:39:46 Install Directory : Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_ofd4_1900_40_008475035;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 7, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 52, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 106256, QuotaPeakPagedPoolUsage: 106256 QuotaNonPagedPoolUsage: 5120, QuotaPeakNonPagedPoolUsage: 5120 - Virtual Memory Usage - VirtualSize: 875233280, PeakVirtualSize: 875233280 - Pagefile Usage - PagefileUsage: 11284480, PeakPagefileUsage: 16060416 - Working Set Size - WorkingSetSize: 4046848, PeakWorkingSetSize: 4046848, PageFaultCount: 1000 * Dump of thread ID 3708 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 37270832.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 - Registers - eax=1132fa0c ebx=00000001 ecx=00000000 edx=00000000 esi=1132fa94 edi=fffffffe eip=7d4e22d6 esp=1132fa08 ebp=1132fa5c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa5c 007bfa90 e06d7363 00000001 00000003 1132fa88 !RaiseException+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e22d6' 1132fa94 007bbbcb 1132faa4 00860978 0082f1b4 0085d354 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bfa90' 1132fab0 00406e6e 00500000 004083ee 00000000 7d61f768 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bbbcb' 1132fb10 00408130 1132fb24 001224ef 1132fb24 00122451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00406e6e' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 e06d7363 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 1660 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 37270832.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=337cff74 eip=7d61ccc6 esp=337cff38 ebp=337cff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 337cff9c 7d4d14ef 00000064 00000000 337cffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 337cffac 00749924 00000064 337cffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 337cffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 337cffec 00000000 00749910 00000000 00000000 000000c8 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/15/13 05:39:49 Install Directory : Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_ofd4_1900_40_008475035;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 7, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 52, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 106256, QuotaPeakPagedPoolUsage: 106256 QuotaNonPagedPoolUsage: 5120, QuotaPeakNonPagedPoolUsage: 5120 - Virtual Memory Usage - VirtualSize: 875233280, PeakVirtualSize: 875233280 - Pagefile Usage - PagefileUsage: 11284480, PeakPagefileUsage: 16060416 - Working Set Size - WorkingSetSize: 4038656, PeakWorkingSetSize: 4038656, PageFaultCount: 998 * Dump of thread ID 3456 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 37271004.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 - Registers - eax=1132fa0c ebx=00000001 ecx=00000000 edx=00000000 esi=1132fa94 edi=fffffffe eip=7d4e22d6 esp=1132fa08 ebp=1132fa5c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa5c 007bfa90 e06d7363 00000001 00000003 1132fa88 !RaiseException+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e22d6' 1132fa94 007bbbcb 1132faa4 00860978 0082f1b4 0085d354 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bfa90' 1132fab0 00406e6e 00500000 004083ee 00000000 7d61f768 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bbbcb' 1132fb10 00408130 1132fb24 001224ef 1132fb24 00122451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00406e6e' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 e06d7363 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 3932 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 37271004.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=337cff74 eip=7d61ccc6 esp=337cff38 ebp=337cff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 337cff9c 7d4d14ef 00000064 00000000 337cffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 337cffac 00749924 00000064 337cffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 337cffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 337cffec 00000000 00749910 00000000 00000000 000000c8 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/15/13 05:39:52 Install Directory : Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_ofd4_1900_40_008475035;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 7, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 52, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 106256, QuotaPeakPagedPoolUsage: 106256 QuotaNonPagedPoolUsage: 5280, QuotaPeakNonPagedPoolUsage: 5280 - Virtual Memory Usage - VirtualSize: 897261568, PeakVirtualSize: 897261568 - Pagefile Usage - PagefileUsage: 33357824, PeakPagefileUsage: 106422272 - Working Set Size - WorkingSetSize: 4079616, PeakWorkingSetSize: 4079616, PageFaultCount: 1008 * Dump of thread ID 6092 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 37271208.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E22D6 - Registers - eax=1132fa04 ebx=00000001 ecx=00000000 edx=00000000 esi=1132fa8c edi=fffffffe eip=7d4e22d6 esp=1132fa00 ebp=1132fa54 cs=120023 ss=12002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa54 007bfa90 e06d7363 00000001 00000003 1132fa80 !RaiseException+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e22d6' 1132fa8c 007bbbcb 1132fa9c 00860978 0082f1b4 0085d354 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bfa90' 1132faa8 00406e8c 04600000 01000000 00500000 004083ee !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007bbbcb' 1132fb10 00408130 1132fb24 001224ef 1132fb24 00122451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00406e8c' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 e06d7363 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 3912 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 37271208.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=337cff74 eip=7d61ccc6 esp=337cff38 ebp=337cff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 337cff9c 7d4d14ef 00000064 00000000 337cffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 337cffac 00749924 00000064 337cffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 337cffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 337cffec 00000000 00749910 00000000 00000000 33ce0000 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Oct 2013 16:35:56	1229309	16045745	hadcm3n_ofd4_1900_40_008475035_0	181,440	287,109	1.5824
12 Oct 2013 10:20:34	1229309	16045745	hadcm3n_ofd4_1900_40_008475035_0	155,520	246,054	1.5821
11 Oct 2013 01:56:36	1229309	16045745	hadcm3n_ofd4_1900_40_008475035_0	129,600	203,680	1.5716
07 Oct 2013 12:12:54	1229309	16045745	hadcm3n_ofd4_1900_40_008475035_0	103,680	162,386	1.5662
04 Oct 2013 18:47:23	1229309	16045745	hadcm3n_ofd4_1900_40_008475035_0	77,760	121,906	1.5677
02 Oct 2013 22:36:13	1229309	16045745	hadcm3n_ofd4_1900_40_008475035_0	51,840	81,942	1.5807
28 Sep 2013 15:40:56	1229309	16045745	hadcm3n_ofd4_1900_40_008475035_0	25,920	41,980	1.6196