Task 15678984

Name	hadcm3n_n3ur_1960_40_008335565_0
Workunit	8486426
Created	23 Mar 2013, 2:16:27 UTC
Sent	23 Mar 2013, 2:21:00 UTC
Report deadline	22 Jun 2013, 9:48:11 UTC
Received	30 Mar 2013, 18:46:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1267713
Run time	4 days 18 hours 1 min 9 sec
CPU time	4 days 13 hours 31 min 23 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 00:18:57 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:18:58 (3900): No heartbeat from core client for 30 sec - exiting 00:18:59 (3900): No heartbeat from core client for 30 sec - exiting 00:19:00 (3900): No heartbeat from core client for 30 sec - exiting 00:20:10 (3096): No heartbeat from core client for 30 sec - exiting 00:20:11 (3096): No heartbeat from core client for 30 sec - exiting 00:20:12 (3096): No heartbeat from core client for 30 sec - exiting 00:20:13 (3096): No heartbeat from core client for 30 sec - exiting 00:20:14 (3096): No heartbeat from core client for 30 sec - exiting 00:20:15 (3096): No heartbeat from core client for 30 sec - exiting 00:20:16 (3096): No heartbeat from core client for 30 sec - exiting 00:20:17 (3096): No heartbeat from core client for 30 sec - exiting 00:20:18 (3096): No heartbeat from core client for 30 sec - exiting 00:20:19 (3096): No heartbeat from core client for 30 sec - exiting 00:20:20 (3096): No heartbeat from core client for 30 sec - exiting 00:20:21 (3096): No heartbeat from core client for 30 sec - exiting 00:20:22 (3096): No heartbeat from core client for 30 sec - exiting 00:20:23 (3096): No heartbeat from core client for 30 sec - exiting 00:20:24 (3096): No heartbeat from core client for 30 sec - exiting 00:20:25 (3096): No heartbeat from core client for 30 sec - exiting 00:20:26 (3096): No heartbeat from core client for 30 sec - exiting 00:20:27 (3096): No heartbeat from core client for 30 sec - exiting 00:20:28 (3096): No heartbeat from core client for 30 sec - exiting 00:20:29 (3096): No heartbeat from core client for 30 sec - exiting 00:20:30 (3096): No heartbeat from core client for 30 sec - exiting 00:20:31 (3096): No heartbeat from core client for 30 sec - exiting 00:20:32 (3096): No heartbeat from core client for 30 sec - exiting 00:20:33 (3096): No heartbeat from core client for 30 sec - exiting 00:20:34 (3096): No heartbeat from core client for 30 sec - exiting 00:20:35 (3096): No heartbeat from core client for 30 sec - exiting 00:20:36 (3096): No heartbeat from core client for 30 sec - exiting 00:20:37 (3096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:20:38 (3096): No heartbeat from core client for 30 sec - exiting 14:32:57 (6224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:32:59 (6224): No heartbeat from core client for 30 sec - exiting 14:33:00 (6224): No heartbeat from core client for 30 sec - exiting 16:32:05 (12528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:32:07 (12528): No heartbeat from core client for 30 sec - exiting 18:05:19 (18032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:21 (18032): No heartbeat from core client for 30 sec - exiting 18:05:22 (18032): No heartbeat from core client for 30 sec - exiting 18:05:23 (18032): No heartbeat from core client for 30 sec - exiting 18:06:43 (23204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:22:07 (4712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:12:35 (7376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:12:36 (7376): No heartbeat from core client for 30 sec - exiting 03:16:37 (1696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:16:39 (1696): No heartbeat from core client for 30 sec - exiting 03:16:40 (1696): No heartbeat from core client for 30 sec - exiting 03:16:41 (1696): No heartbeat from core client for 30 sec - exiting 03:16:42 (1696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:00:34 (1036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:31 (3452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:32 (3452): No heartbeat from core client for 30 sec - exiting 12:10:35 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:10:36 (5748): No heartbeat from core client for 30 sec - exiting 12:11:14 (5480): No heartbeat from core client for 30 sec - exiting 12:11:15 (5480): No heartbeat from core client for 30 sec - exiting 12:11:16 (5480): No heartbeat from core client for 30 sec - exiting 12:11:17 (5480): No heartbeat from core client for 30 sec - exiting 12:11:18 (5480): No heartbeat from core client for 30 sec - exiting 12:11:19 (5480): No heartbeat from core client for 30 sec - exiting 12:11:20 (5480): No heartbeat from core client for 30 sec - exiting 12:11:21 (5480): No heartbeat from core client for 30 sec - exiting 12:11:22 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:11:23 (5480): No heartbeat from core client for 30 sec - exiting 12:14:14 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:22 (4536): No heartbeat from core client for 30 sec - exiting 12:15:23 (4536): No heartbeat from core client for 30 sec - exiting 12:15:24 (4536): No heartbeat from core client for 30 sec - exiting 12:15:25 (4536): No heartbeat from core client for 30 sec - exiting 12:15:26 (4536): No heartbeat from core client for 30 sec - exiting 12:15:27 (4536): No heartbeat from core client for 30 sec - exiting 12:15:28 (4536): No heartbeat from core client for 30 sec - exiting 12:15:29 (4536): No heartbeat from core client for 30 sec - exiting 12:15:30 (4536): No heartbeat from core client for 30 sec - exiting 12:15:31 (4536): No heartbeat from core client for 30 sec - exiting 12:15:32 (4536): No heartbeat from core client for 30 sec - exiting 12:15:33 (4536): No heartbeat from core client for 30 sec - exiting 12:15:34 (4536): No heartbeat from core client for 30 sec - exiting 12:15:35 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:36 (4536): No heartbeat from core client for 30 sec - exiting 12:15:37 (4536): No heartbeat from core client for 30 sec - exiting 12:15:38 (4536): No heartbeat from core client for 30 sec - exiting 12:15:39 (4536): No heartbeat from core client for 30 sec - exiting 14:11:56 (7660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:11:57 (7660): No heartbeat from core client for 30 sec - exiting 14:11:58 (7660): No heartbeat from core client for 30 sec - exiting 14:11:59 (7660): No heartbeat from core client for 30 sec - exiting 14:12:45 (8000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:15:58 (7240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:16:03 (7240): No heartbeat from core client for 30 sec - exiting 14:16:04 (7240): No heartbeat from core client for 30 sec - exiting 14:16:05 (7240): No heartbeat from core client for 30 sec - exiting 14:16:06 (7240): No heartbeat from core client for 30 sec - exiting 14:16:07 (7240): No heartbeat from core client for 30 sec - exiting 14:16:08 (7240): No heartbeat from core client for 30 sec - exiting 14:16:09 (7240): No heartbeat from core client for 30 sec - exiting 14:17:28 (6576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:32 (6576): No heartbeat from core client for 30 sec - exiting 14:17:33 (6576): No heartbeat from core client for 30 sec - exiting 14:17:34 (6576): No heartbeat from core client for 30 sec - exiting 14:17:35 (6576): No heartbeat from core client for 30 sec - exiting 14:17:36 (6576): No heartbeat from core client for 30 sec - exiting 14:19:32 (7020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 14:24:16 (7364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:24:55 (3504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:03 (8408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:04 (8408): No heartbeat from core client for 30 sec - exiting 14:28:05 (8408): No heartbeat from core client for 30 sec - exiting 14:28:06 (8408): No heartbeat from core client for 30 sec - exiting 14:28:07 (8408): No heartbeat from core client for 30 sec - exiting 14:28:08 (8408): No heartbeat from core client for 30 sec - exiting 14:28:09 (8408): No heartbeat from core client for 30 sec - exiting 14:28:10 (8408): No heartbeat from core client for 30 sec - exiting 14:28:11 (8408): No heartbeat from core client for 30 sec - exiting 14:28:12 (8408): No heartbeat from core client for 30 sec - exiting 14:28:13 (8408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:27:26 (7412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:27 (7412): No heartbeat from core client for 30 sec - exiting 20:27:28 (7412): No heartbeat from core client for 30 sec - exiting 20:27:29 (7412): No heartbeat from core client for 30 sec - exiting 20:27:30 (7412): No heartbeat from core client for 30 sec - exiting 20:28:21 (9800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:27 (3512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:40 (6176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:41 (6176): No heartbeat from core client for 30 sec - exiting 21:57:42 (6176): No heartbeat from core client for 30 sec - exiting 21:58:26 (1508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:06:15 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:01 (9680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:18 (10548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:38:27 (10052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:38:28 (10052): No heartbeat from core client for 30 sec - exiting 22:38:29 (10052): No heartbeat from core client for 30 sec - exiting 22:38:30 (10052): No heartbeat from core client for 30 sec - exiting 22:38:31 (10052): No heartbeat from core client for 30 sec - exiting 22:38:32 (10052): No heartbeat from core client for 30 sec - exiting 22:38:33 (10052): No heartbeat from core client for 30 sec - exiting 22:38:34 (10052): No heartbeat from core client for 30 sec - exiting 22:38:35 (10052): No heartbeat from core client for 30 sec - exiting 22:38:36 (10052): No heartbeat from core client for 30 sec - exiting 22:38:37 (10052): No heartbeat from core client for 30 sec - exiting 22:40:26 (11236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:42:39 (8836): No heartbeat from core client for 30 sec - exiting 22:42:40 (8836): No heartbeat from core client for 30 sec - exiting 22:42:41 (8836): No heartbeat from core client for 30 sec - exiting 22:42:42 (8836): No heartbeat from core client for 30 sec - exiting 22:42:43 (8836): No heartbeat from core client for 30 sec - exiting 22:42:44 (8836): No heartbeat from core client for 30 sec - exiting 22:42:45 (8836): No heartbeat from core client for 30 sec - exiting 22:42:46 (8836): No heartbeat from core client for 30 sec - exiting 22:42:47 (8836): No heartbeat from core client for 30 sec - exiting 22:42:48 (8836): No heartbeat from core client for 30 sec - exiting 22:42:49 (8836): No heartbeat from core client for 30 sec - exiting 22:42:50 (8836): No heartbeat from core client for 30 sec - exiting 22:42:51 (8836): No heartbeat from core client for 30 sec - exiting 22:42:52 (8836): No heartbeat from core client for 30 sec - exiting 22:42:53 (8836): No heartbeat from core client for 30 sec - exiting 22:42:54 (8836): No heartbeat from core client for 30 sec - exiting 22:42:55 (8836): No heartbeat from core client for 30 sec - exiting 22:42:56 (8836): No heartbeat from core client for 30 sec - exiting 22:42:57 (8836): No heartbeat from core client for 30 sec - exiting 22:42:58 (8836): No heartbeat from core client for 30 sec - exiting 22:42:59 (8836): No heartbeat from core client for 30 sec - exiting 22:43:00 (8836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:13:05 (10120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:13:06 (10120): No heartbeat from core client for 30 sec - exiting 13:13:07 (10120): No heartbeat from core client for 30 sec - exiting 13:13:08 (10120): No heartbeat from core client for 30 sec - exiting 21:52:04 (10572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:52:06 (10572): No heartbeat from core client for 30 sec - exiting 21:52:07 (10572): No heartbeat from core client for 30 sec - exiting 21:52:08 (10572): No heartbeat from core client for 30 sec - exiting 21:52:09 (10572): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11268, iMonCtr=1 Model crash detected, will try to restart... 02:32:29 (304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:34:38 (2680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B03AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_n3ur_1960_40_008335565/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Mar 2013 18:51:39	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	259,200	394,275	1.5211
30 Mar 2013 07:03:52	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	233,280	353,845	1.5168
29 Mar 2013 20:53:56	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	207,360	318,057	1.5338
29 Mar 2013 10:46:38	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	181,440	282,151	1.5551
29 Mar 2013 00:06:17	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	155,520	246,895	1.5875
28 Mar 2013 12:10:24	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	129,600	208,971	1.6124
28 Mar 2013 00:40:46	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	103,680	168,041	1.6208
27 Mar 2013 15:10:34	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	77,760	134,073	1.7242
27 Mar 2013 00:43:56	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	51,840	91,257	1.7604
26 Mar 2013 11:17:05	1267713	15678984	hadcm3n_n3ur_1960_40_008335565_0	25,920	45,197	1.7437