Task 16053644

Name	hadcm3n_og0l_1900_40_008475880_1
Workunit	8626719
Created	2 Oct 2013, 11:50:51 UTC
Sent	2 Oct 2013, 12:35:49 UTC
Report deadline	1 Jan 2014, 20:03:00 UTC
Received	12 Oct 2013, 2:20:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1285160
Run time	4 days 11 hours 11 min 57 sec
CPU time	4 days 2 hours 8 min 44 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.16 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:29:12 (5828): No heartbeat from core client for 30 sec - exiting 11:29:13 (5828): No heartbeat from core client for 30 sec - exiting 11:29:14 (5828): No heartbeat from core client for 30 sec - exiting 11:29:15 (5828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:16:31 (1448): No heartbeat from core client for 30 sec - exiting 12:16:32 (1448): No heartbeat from core client for 30 sec - exiting 12:16:33 (1448): No heartbeat from core client for 30 sec - exiting 12:16:34 (1448): No heartbeat from core client for 30 sec - exiting 12:16:35 (1448): No heartbeat from core client for 30 sec - exiting 12:16:36 (1448): No heartbeat from core client for 30 sec - exiting 12:16:37 (1448): No heartbeat from core client for 30 sec - exiting 12:16:38 (1448): No heartbeat from core client for 30 sec - exiting 12:16:39 (1448): No heartbeat from core client for 30 sec - exiting 12:16:40 (1448): No heartbeat from core client for 30 sec - exiting 12:16:41 (1448): No heartbeat from core client for 30 sec - exiting 12:16:42 (1448): No heartbeat from core client for 30 sec - exiting 12:16:43 (1448): No heartbeat from core client for 30 sec - exiting 12:16:44 (1448): No heartbeat from core client for 30 sec - exiting 12:16:45 (1448): No heartbeat from core client for 30 sec - exiting 12:16:46 (1448): No heartbeat from core client for 30 sec - exiting 12:16:47 (1448): No heartbeat from core client for 30 sec - exiting 12:16:48 (1448): No heartbeat from core client for 30 sec - exiting 12:16:49 (1448): No heartbeat from core client for 30 sec - exiting 12:16:50 (1448): No heartbeat from core client for 30 sec - exiting 12:16:51 (1448): No heartbeat from core client for 30 sec - exiting 12:16:52 (1448): No heartbeat from core client for 30 sec - exiting 12:16:53 (1448): No heartbeat from core client for 30 sec - exiting 12:16:54 (1448): No heartbeat from core client for 30 sec - exiting 12:16:55 (1448): No heartbeat from core client for 30 sec - exiting 12:16:56 (1448): No heartbeat from core client for 30 sec - exiting 12:16:57 (1448): No heartbeat from core client for 30 sec - exiting 12:16:58 (1448): No heartbeat from core client for 30 sec - exiting 12:16:59 (1448): No heartbeat from core client for 30 sec - exiting 12:17:00 (1448): No heartbeat from core client for 30 sec - exiting 12:17:01 (1448): No heartbeat from core client for 30 sec - exiting 12:17:02 (1448): No heartbeat from core client for 30 sec - exiting 12:17:03 (1448): No heartbeat from core client for 30 sec - exiting 12:17:04 (1448): No heartbeat from core client for 30 sec - exiting 12:17:05 (1448): No heartbeat from core client for 30 sec - exiting 12:17:06 (1448): No heartbeat from core client for 30 sec - exiting 12:17:07 (1448): No heartbeat from core client for 30 sec - exiting 12:17:08 (1448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:10:52 (5356): No heartbeat from core client for 30 sec - exiting 17:10:53 (5356): No heartbeat from core client for 30 sec - exiting 17:10:54 (5356): No heartbeat from core client for 30 sec - exiting 17:10:55 (5356): No heartbeat from core client for 30 sec - exiting 17:10:56 (5356): No heartbeat from core client for 30 sec - exiting 17:10:57 (5356): No heartbeat from core client for 30 sec - exiting 17:10:58 (5356): No heartbeat from core client for 30 sec - exiting 17:10:59 (5356): No heartbeat from core client for 30 sec - exiting 17:11:00 (5356): No heartbeat from core client for 30 sec - exiting 17:11:01 (5356): No heartbeat from core client for 30 sec - exiting 17:11:02 (5356): No heartbeat from core client for 30 sec - exiting 17:11:03 (5356): No heartbeat from core client for 30 sec - exiting 17:11:04 (5356): No heartbeat from core client for 30 sec - exiting 17:11:05 (5356): No heartbeat from core client for 30 sec - exiting 17:11:06 (5356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:14:00 (1896): No heartbeat from core client for 30 sec - exiting 00:14:02 (1896): No heartbeat from core client for 30 sec - exiting 00:14:03 (1896): No heartbeat from core client for 30 sec - exiting 00:14:04 (1896): No heartbeat from core client for 30 sec - exiting 00:14:05 (1896): No heartbeat from core client for 30 sec - exiting 00:14:06 (1896): No heartbeat from core client for 30 sec - exiting 00:14:07 (1896): No heartbeat from core client for 30 sec - exiting 00:14:08 (1896): No heartbeat from core client for 30 sec - exiting 00:14:09 (1896): No heartbeat from core client for 30 sec - exiting 00:14:10 (1896): No heartbeat from core client for 30 sec - exiting 00:14:11 (1896): No heartbeat from core client for 30 sec - exiting 00:14:12 (1896): No heartbeat from core client for 30 sec - exiting 00:14:13 (1896): No heartbeat from core client for 30 sec - exiting 00:14:14 (1896): No heartbeat from core client for 30 sec - exiting 00:14:15 (1896): No heartbeat from core client for 30 sec - exiting 00:14:16 (1896): No heartbeat from core client for 30 sec - exiting 00:14:17 (1896): No heartbeat from core client for 30 sec - exiting 00:14:18 (1896): No heartbeat from core client for 30 sec - exiting 00:14:19 (1896): No heartbeat from core client for 30 sec - exiting 00:14:20 (1896): No heartbeat from core client for 30 sec - exiting 00:14:21 (1896): No heartbeat from core client for 30 sec - exiting 00:14:22 (1896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:14:23 (1896): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:14:42 (1256): No heartbeat from core client for 30 sec - exiting 20:14:43 (1256): No heartbeat from core client for 30 sec - exiting 20:14:44 (1256): No heartbeat from core client for 30 sec - exiting 20:14:45 (1256): No heartbeat from core client for 30 sec - exiting 20:14:46 (1256): No heartbeat from core client for 30 sec - exiting 20:14:47 (1256): No heartbeat from core client for 30 sec - exiting 20:14:48 (1256): No heartbeat from core client for 30 sec - exiting 20:14:49 (1256): No heartbeat from core client for 30 sec - exiting 20:14:50 (1256): No heartbeat from core client for 30 sec - exiting 20:14:51 (1256): No heartbeat from core client for 30 sec - exiting 20:14:52 (1256): No heartbeat from core client for 30 sec - exiting 20:14:53 (1256): No heartbeat from core client for 30 sec - exiting 20:14:54 (1256): No heartbeat from core client for 30 sec - exiting 20:14:55 (1256): No heartbeat from core client for 30 sec - exiting 20:14:56 (1256): No heartbeat from core client for 30 sec - exiting 20:14:57 (1256): No heartbeat from core client for 30 sec - exiting 20:14:58 (1256): No heartbeat from core client for 30 sec - exiting 20:14:59 (1256): No heartbeat from core client for 30 sec - exiting 20:15:00 (1256): No heartbeat from core client for 30 sec - exiting 20:15:01 (1256): No heartbeat from core client for 30 sec - exiting 20:15:02 (1256): No heartbeat from core client for 30 sec - exiting 20:15:03 (1256): No heartbeat from core client for 30 sec - exiting 20:15:04 (1256): No heartbeat from core client for 30 sec - exiting 20:15:05 (1256): No heartbeat from core client for 30 sec - exiting 20:15:06 (1256): No heartbeat from core client for 30 sec - exiting 20:15:07 (1256): No heartbeat from core client for 30 sec - exiting 20:15:08 (1256): No heartbeat from core client for 30 sec - exiting 20:15:09 (1256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:59:25 (7472): No heartbeat from core client for 30 sec - exiting 23:59:26 (7472): No heartbeat from core client for 30 sec - exiting 23:59:27 (7472): No heartbeat from core client for 30 sec - exiting 23:59:28 (7472): No heartbeat from core client for 30 sec - exiting 23:59:29 (7472): No heartbeat from core client for 30 sec - exiting 23:59:30 (7472): No heartbeat from core client for 30 sec - exiting 23:59:31 (7472): No heartbeat from core client for 30 sec - exiting 23:59:32 (7472): No heartbeat from core client for 30 sec - exiting 23:59:33 (7472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/og0lko.pja9c10 Error converting file to netcdf: dataout/og0lko.pia9c10 Error converting file to netcdf: dataout/og0lko.pfa9c10 Error converting file to netcdf: dataout/og0lka.pha9c10 Error converting file to netcdf: dataout/og0lka.pga9c10 Error converting file to netcdf: dataout/og0lka.pea9c10 Error converting file to netcdf: dataout/og0lka.pda9c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:26:00 (9656): No heartbeat from core client for 30 sec - exiting 00:26:01 (9656): No heartbeat from core client for 30 sec - exiting 00:26:02 (9656): No heartbeat from core client for 30 sec - exiting 00:26:03 (9656): No heartbeat from core client for 30 sec - exiting 00:26:04 (9656): No heartbeat from core client for 30 sec - exiting 00:26:05 (9656): No heartbeat from core client for 30 sec - exiting 00:26:06 (9656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x772F7383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file D:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_og0l_1900_40_008475880/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Oct 2013 05:18:02	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	259,200	353,310	1.3631
10 Oct 2013 11:32:45	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	233,280	317,961	1.3630
09 Oct 2013 16:09:53	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	207,360	282,774	1.3637
09 Oct 2013 02:09:53	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	181,440	248,350	1.3688
08 Oct 2013 04:46:39	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	155,520	213,338	1.3718
07 Oct 2013 09:45:42	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	129,600	177,808	1.3720
06 Oct 2013 07:19:04	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	103,680	140,921	1.3592
05 Oct 2013 11:19:48	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	77,760	104,019	1.3377
04 Oct 2013 16:44:42	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	51,840	68,117	1.3140
03 Oct 2013 23:01:15	1285160	16053644	hadcm3n_og0l_1900_40_008475880_1	25,920	34,048	1.3136