Task 15823142

Name	hadcm3n_o3z0_1980_40_008387335_0
Workunit	8538194
Created	3 Jun 2013, 9:36:28 UTC
Sent	3 Jun 2013, 9:47:28 UTC
Report deadline	2 Sep 2013, 17:14:39 UTC
Received	14 Jun 2013, 22:27:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1276055
Run time	3 days 6 hours 28 min 50 sec
CPU time	3 days 5 hours 29 min 31 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.00 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:57:27 (6912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:28 (6912): No heartbeat from core client for 30 sec - exiting 18:57:29 (6912): No heartbeat from core client for 30 sec - exiting 18:57:30 (6912): No heartbeat from core client for 30 sec - exiting 18:57:31 (6912): No heartbeat from core client for 30 sec - exiting 18:57:32 (6912): No heartbeat from core client for 30 sec - exiting 18:57:33 (6912): No heartbeat from core client for 30 sec - exiting 18:57:34 (6912): No heartbeat from core client for 30 sec - exiting 18:57:35 (6912): No heartbeat from core client for 30 sec - exiting 18:57:36 (6912): No heartbeat from core client for 30 sec - exiting 18:57:37 (6912): No heartbeat from core client for 30 sec - exiting 18:59:10 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:59:11 (3936): No heartbeat from core client for 30 sec - exiting 18:59:12 (3936): No heartbeat from core client for 30 sec - exiting 18:59:13 (3936): No heartbeat from core client for 30 sec - exiting 18:59:14 (3936): No heartbeat from core client for 30 sec - exiting 18:59:15 (3936): No heartbeat from core client for 30 sec - exiting 18:59:16 (3936): No heartbeat from core client for 30 sec - exiting 18:59:17 (3936): No heartbeat from core client for 30 sec - exiting 18:59:18 (3936): No heartbeat from core client for 30 sec - exiting 18:59:19 (3936): No heartbeat from core client for 30 sec - exiting 18:59:20 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:56:43 (5784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:22:44 (1560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:45 (1560): No heartbeat from core client for 30 sec - exiting 19:22:46 (1560): No heartbeat from core client for 30 sec - exiting 19:22:47 (1560): No heartbeat from core client for 30 sec - exiting 19:22:48 (1560): No heartbeat from core client for 30 sec - exiting 19:22:49 (1560): No heartbeat from core client for 30 sec - exiting 19:22:50 (1560): No heartbeat from core client for 30 sec - exiting 19:22:51 (1560): No heartbeat from core client for 30 sec - exiting 19:22:52 (1560): No heartbeat from core client for 30 sec - exiting 19:22:53 (1560): No heartbeat from core client for 30 sec - exiting 19:22:54 (1560): No heartbeat from core client for 30 sec - exiting 19:26:15 (6892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:49 (5572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:38 (5196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:49:57 (7192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:08 (8112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:43:18 (1832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:46:07 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:13 (3912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:14 (3912): No heartbeat from core client for 30 sec - exiting 18:47:15 (3912): No heartbeat from core client for 30 sec - exiting 18:47:16 (3912): No heartbeat from core client for 30 sec - exiting 18:47:17 (3912): No heartbeat from core client for 30 sec - exiting 18:47:18 (3912): No heartbeat from core client for 30 sec - exiting 18:47:19 (3912): No heartbeat from core client for 30 sec - exiting 18:47:20 (3912): No heartbeat from core client for 30 sec - exiting 18:47:21 (3912): No heartbeat from core client for 30 sec - exiting 18:47:22 (3912): No heartbeat from core client for 30 sec - exiting 18:47:23 (3912): No heartbeat from core client for 30 sec - exiting 18:50:58 (7268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:17 (7268): No heartbeat from core client for 30 sec - exiting 18:53:18 (7268): No heartbeat from core client for 30 sec - exiting 18:53:19 (7268): No heartbeat from core client for 30 sec - exiting 18:53:20 (7268): No heartbeat from core client for 30 sec - exiting 18:53:21 (7268): No heartbeat from core client for 30 sec - exiting 18:53:22 (7268): No heartbeat from core client for 30 sec - exiting 18:53:23 (7268): No heartbeat from core client for 30 sec - exiting 18:53:24 (7268): No heartbeat from core client for 30 sec - exiting 18:53:25 (7268): No heartbeat from core client for 30 sec - exiting 18:53:26 (7268): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o3z0ko.pji7c10 Error converting file to netcdf: dataout/o3z0ko.pii7c10 Error converting file to netcdf: dataout/o3z0ko.pfi7c10 Error converting file to netcdf: dataout/o3z0ka.phi7c10 Error converting file to netcdf: dataout/o3z0ka.pgi7c10 Error converting file to netcdf: dataout/o3z0ka.pei7c10 Error converting file to netcdf: dataout/o3z0ka.pdi7c10 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:47:23 (7308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:50:10 (7236): No heartbeat from core client for 30 sec - exiting 09:50:11 (7236): No heartbeat from core client for 30 sec - exiting 09:50:12 (7236): No heartbeat from core client for 30 sec - exiting 09:50:13 (7236): No heartbeat from core client for 30 sec - exiting 09:50:14 (7236): No heartbeat from core client for 30 sec - exiting 09:50:15 (7236): No heartbeat from core client for 30 sec - exiting 09:50:16 (7236): No heartbeat from core client for 30 sec - exiting 09:50:17 (7236): No heartbeat from core client for 30 sec - exiting 09:50:18 (7236): No heartbeat from core client for 30 sec - exiting 09:50:19 (7236): No heartbeat from core client for 30 sec - exiting 09:50:20 (7236): No heartbeat from core client for 30 sec - exiting 09:50:21 (7236): No heartbeat from core client for 30 sec - exiting 09:50:22 (7236): No heartbeat from core client for 30 sec - exiting 09:50:23 (7236): No heartbeat from core client for 30 sec - exiting 09:50:24 (7236): No heartbeat from core client for 30 sec - exiting 09:50:25 (7236): No heartbeat from core client for 30 sec - exiting 09:50:26 (7236): No heartbeat from core client for 30 sec - exiting 09:50:27 (7236): No heartbeat from core client for 30 sec - exiting 09:50:28 (7236): No heartbeat from core client for 30 sec - exiting 09:50:29 (7236): No heartbeat from core client for 30 sec - exiting 09:50:30 (7236): No heartbeat from core client for 30 sec - exiting 09:50:31 (7236): No heartbeat from core client for 30 sec - exiting 09:50:32 (7236): No heartbeat from core client for 30 sec - exiting 09:50:33 (7236): No heartbeat from core client for 30 sec - exiting 09:50:34 (7236): No heartbeat from core client for 30 sec - exiting 09:50:35 (7236): No heartbeat from core client for 30 sec - exiting 09:50:36 (7236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 09:54:47 (7672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:48 (7672): No heartbeat from core client for 30 sec - exiting 09:54:49 (7672): No heartbeat from core client for 30 sec - exiting 09:54:50 (7672): No heartbeat from core client for 30 sec - exiting 09:54:51 (7672): No heartbeat from core client for 30 sec - exiting 09:54:52 (7672): No heartbeat from core client for 30 sec - exiting 09:54:53 (7672): No heartbeat from core client for 30 sec - exiting 09:54:54 (7672): No heartbeat from core client for 30 sec - exiting 09:54:55 (7672): No heartbeat from core client for 30 sec - exiting 09:54:56 (7672): No heartbeat from core client for 30 sec - exiting 09:54:57 (7672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Cforrtl: Access is denied. Ocean Restart file copy failed on o3z0ko.daj0430 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6596, selfPID=6596, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Jun 2013 22:29:30	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	259,200	278,958	1.0762
14 Jun 2013 11:44:03	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	233,280	249,717	1.0705
14 Jun 2013 11:44:03	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	207,360	221,509	1.0682
12 Jun 2013 12:30:05	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	181,440	193,217	1.0649
12 Jun 2013 12:30:05	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	155,520	165,014	1.0610
12 Jun 2013 12:30:05	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	129,600	136,503	1.0533
12 Jun 2013 12:30:05	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	103,680	108,102	1.0427
12 Jun 2013 12:30:05	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	77,760	79,291	1.0197
12 Jun 2013 12:30:05	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	51,840	49,789	0.9604
05 Jun 2013 03:07:54	1276055	15823142	hadcm3n_o3z0_1980_40_008387335_0	25,920	23,593	0.9102