Task 12847999

Name	hadcm3n_p49s_1900_40_007223120_2
Workunit	7421360
Created	1 May 2011, 15:53:10 UTC
Sent	1 May 2011, 16:03:51 UTC
Report deadline	31 Jul 2011, 23:31:02 UTC
Received	1 Aug 2011, 10:11:49 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1146254
Run time	25 days 10 hours 48 min 9 sec
CPU time	20 days 21 hours 46 min 59 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <stderr_txt> CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2184, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3960, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=456, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4100, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3312, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3180, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2052, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1600, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/p49sko.pjd4c10 Error converting file to netcdf: dataout/p49sko.pid4c10 Error converting file to netcdf: dataout/p49sko.pfd4c10 Error converting file to netcdf: dataout/p49ska.phd4c10 Error converting file to netcdf: dataout/p49ska.pgd4c10 Error converting file to netcdf: dataout/p49ska.ped4c10 Error converting file to netcdf: dataout/p49ska.pdd4c10 20:05:13 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:26 (3736): No heartbeat from core client for 30 sec - exiting 08:52:28 (3736): No heartbeat from core client for 30 sec - exiting 08:52:29 (3736): No heartbeat from core client for 30 sec - exiting 08:52:30 (3736): No heartbeat from core client for 30 sec - exiting 08:52:31 (3736): No heartbeat from core client for 30 sec - exiting 08:52:32 (3736): No heartbeat from core client for 30 sec - exiting 08:52:33 (3736): No heartbeat from core client for 30 sec - exiting 08:52:34 (3736): No heartbeat from core client for 30 sec - exiting 08:52:35 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:41:08 (4428): No heartbeat from core client for 30 sec - exiting 15:41:09 (4428): No heartbeat from core client for 30 sec - exiting 15:41:11 (4428): No heartbeat from core client for 30 sec - exiting 15:41:12 (4428): No heartbeat from core client for 30 sec - exiting 15:41:13 (4428): No heartbeat from core client for 30 sec - exiting 15:41:14 (4428): No heartbeat from core client for 30 sec - exiting 15:41:15 (4428): No heartbeat from core client for 30 sec - exiting 15:41:16 (4428): No heartbeat from core client for 30 sec - exiting 15:41:17 (4428): No heartbeat from core client for 30 sec - exiting 15:41:18 (4428): No heartbeat from core client for 30 sec - exiting 15:41:19 (4428): No heartbeat from core client for 30 sec - exiting 15:41:20 (4428): No heartbeat from core client for 30 sec - exiting 15:41:22 (4428): No heartbeat from core client for 30 sec - exiting 15:41:23 (4428): No heartbeat from core client for 30 sec - exiting 15:41:24 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:02:25 (3492): No heartbeat from core client for 30 sec - exiting 12:02:26 (3492): No heartbeat from core client for 30 sec - exiting 12:02:28 (3492): No heartbeat from core client for 30 sec - exiting 12:02:29 (3492): No heartbeat from core client for 30 sec - exiting 12:02:30 (3492): No heartbeat from core client for 30 sec - exiting 12:02:32 (3492): No heartbeat from core client for 30 sec - exiting 12:02:33 (3492): No heartbeat from core client for 30 sec - exiting 12:02:34 (3492): No heartbeat from core client for 30 sec - exiting 12:02:35 (3492): No heartbeat from core client for 30 sec - exiting 12:02:36 (3492): No heartbeat from core client for 30 sec - exiting 12:02:37 (3492): No heartbeat from core client for 30 sec - exiting 12:02:38 (3492): No heartbeat from core client for 30 sec - exiting 12:02:39 (3492): No heartbeat from core client for 30 sec - exiting 12:02:40 (3492): No heartbeat from core client for 30 sec - exiting 12:02:42 (3492): No heartbeat from core client for 30 sec - exiting 12:02:43 (3492): No heartbeat from core client for 30 sec - exiting 12:02:44 (3492): No heartbeat from core client for 30 sec - exiting 12:02:45 (3492): No heartbeat from core client for 30 sec - exiting 12:02:46 (3492): No heartbeat from core client for 30 sec - exiting 12:02:47 (3492): No heartbeat from core client for 30 sec - exiting 12:02:48 (3492): No heartbeat from core client for 30 sec - exiting 12:02:49 (3492): No heartbeat from core client for 30 sec - exiting 12:02:50 (3492): No heartbeat from core client for 30 sec - exiting 12:02:51 (3492): No heartbeat from core client for 30 sec - exiting 12:02:53 (3492): No heartbeat from core client for 30 sec - exiting 12:02:54 (3492): No heartbeat from core client for 30 sec - exiting 12:02:55 (3492): No heartbeat from core client for 30 sec - exiting 12:02:56 (3492): No heartbeat from core client for 30 sec - exiting 12:02:57 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:29 (2164): No heartbeat from core client for 30 sec - exiting 14:01:30 (2164): No heartbeat from core client for 30 sec - exiting 14:01:31 (2164): No heartbeat from core client for 30 sec - exiting 14:01:32 (2164): No heartbeat from core client for 30 sec - exiting 14:01:33 (2164): No heartbeat from core client for 30 sec - exiting 14:01:34 (2164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:16 (4328): No heartbeat from core client for 30 sec - exiting 17:23:17 (4328): No heartbeat from core client for 30 sec - exiting 17:23:19 (4328): No heartbeat from core client for 30 sec - exiting 17:23:20 (4328): No heartbeat from core client for 30 sec - exiting 17:23:21 (4328): No heartbeat from core client for 30 sec - exiting 17:23:22 (4328): No heartbeat from core client for 30 sec - exiting 17:23:23 (4328): No heartbeat from core client for 30 sec - exiting 17:23:24 (4328): No heartbeat from core client for 30 sec - exiting 17:23:25 (4328): No heartbeat from core client for 30 sec - exiting 17:23:26 (4328): No heartbeat from core client for 30 sec - exiting 17:23:27 (4328): No heartbeat from core client for 30 sec - exiting 17:23:28 (4328): No heartbeat from core client for 30 sec - exiting 17:23:29 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:10 (4268): No heartbeat from core client for 30 sec - exiting 08:45:11 (4268): No heartbeat from core client for 30 sec - exiting 08:45:12 (4268): No heartbeat from core client for 30 sec - exiting 08:45:13 (4268): No heartbeat from core client for 30 sec - exiting 08:45:14 (4268): No heartbeat from core client for 30 sec - exiting 08:45:15 (4268): No heartbeat from core client for 30 sec - exiting 08:45:16 (4268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Ocean Restart file copy failed on p49sko.dad7320 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5036, iMonCtr=1 Model crash detected, will try to restart... 11:21:42 (3916): No heartbeat from core client for 30 sec - exiting 11:21:43 (3916): No heartbeat from core client for 30 sec - exiting 11:21:45 (3916): No heartbeat from core client for 30 sec - exiting 11:21:46 (3916): No heartbeat from core client for 30 sec - exiting 11:21:47 (3916): No heartbeat from core client for 30 sec - exiting 11:21:48 (3916): No heartbeat from core client for 30 sec - exiting 11:21:49 (3916): No heartbeat from core client for 30 sec - exiting 11:21:50 (3916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:21:51 (3916): No heartbeat from core client for 30 sec - exiting Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Aug 2011 10:14:51	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	1,036,800	1,805,798	1.7417
31 Jul 2011 20:16:40	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	1,010,880	1,759,426	1.7405
30 Jul 2011 06:42:56	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	984,960	1,713,317	1.7395
29 Jul 2011 17:42:26	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	959,040	1,671,272	1.7427
27 Jul 2011 16:25:56	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	933,120	1,627,857	1.7445
25 Jul 2011 22:50:55	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	907,200	1,584,273	1.7463
25 Jul 2011 20:41:46	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	881,280	1,541,187	1.7488
25 Jul 2011 16:20:34	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	855,360	1,495,970	1.7489
25 Jul 2011 16:20:34	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	829,440	1,450,715	1.7490
07 Jul 2011 19:48:36	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	803,520	1,405,172	1.7488
07 Jul 2011 15:36:54	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	777,600	1,359,598	1.7485
01 Jul 2011 10:19:22	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	751,680	1,314,177	1.7483
29 Jun 2011 04:12:11	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	725,760	1,269,359	1.7490
28 Jun 2011 08:44:01	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	699,840	1,225,385	1.7510
24 Jun 2011 18:04:50	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	673,920	1,179,030	1.7495
20 Jun 2011 13:37:28	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	648,000	1,132,891	1.7483
13 Jun 2011 03:42:25	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	622,080	1,088,579	1.7499
11 Jun 2011 10:00:56	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	596,160	1,042,729	1.7491
06 Jun 2011 06:54:19	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	570,240	998,255	1.7506
04 Jun 2011 12:41:10	1146254	12847999	hadcm3n_p49s_1900_40_007223120_2	544,320	952,620	1.7501