Task 15814771

Name	hadcm3n_o3v0_1940_40_008383120_0
Workunit	8533979
Created	1 Jun 2013, 7:13:27 UTC
Sent	2 Jun 2013, 3:44:31 UTC
Report deadline	1 Sep 2013, 11:11:42 UTC
Received	18 Jul 2013, 11:20:55 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1270442
Run time	12 days 0 hours 12 min 29 sec
CPU time	9 days 3 hours 45 min 59 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.83 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11568, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: ´ÅÅÌ¿Õ¼ä²»×ã¡£ Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11220, iMonCtr=1 Model crash detected, will try to restart... forrtl: ´ÅÅÌ¿Õ¼ä²»×ã¡£ Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11220, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:29:37 (5568): No heartbeat from core client for 30 sec - exiting 16:29:38 (5568): No heartbeat from core client for 30 sec - exiting 16:29:39 (5568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:01 (5240): No heartbeat from core client for 30 sec - exiting 16:43:02 (5240): No heartbeat from core client for 30 sec - exiting 16:43:03 (5240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:41 (4428): No heartbeat from core client for 30 sec - exiting 16:45:42 (4428): No heartbeat from core client for 30 sec - exiting 16:45:43 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:48:22 (6412): No heartbeat from core client for 30 sec - exiting 16:48:23 (6412): No heartbeat from core client for 30 sec - exiting 16:48:24 (6412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:02 (4228): No heartbeat from core client for 30 sec - exiting 16:51:03 (4228): No heartbeat from core client for 30 sec - exiting 16:51:04 (4228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:53:42 (5276): No heartbeat from core client for 30 sec - exiting 16:53:43 (5276): No heartbeat from core client for 30 sec - exiting 16:53:44 (5276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:56:22 (5156): No heartbeat from core client for 30 sec - exiting 16:56:23 (5156): No heartbeat from core client for 30 sec - exiting 16:56:24 (5156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:03 (4740): No heartbeat from core client for 30 sec - exiting 16:59:04 (4740): No heartbeat from core client for 30 sec - exiting 16:59:05 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:01:43 (6748): No heartbeat from core client for 30 sec - exiting 17:01:44 (6748): No heartbeat from core client for 30 sec - exiting 17:01:45 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:04:23 (3688): No heartbeat from core client for 30 sec - exiting 17:04:24 (3688): No heartbeat from core client for 30 sec - exiting 17:04:25 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:07:03 (6316): No heartbeat from core client for 30 sec - exiting 17:07:04 (6316): No heartbeat from core client for 30 sec - exiting 17:07:05 (6316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:09:44 (3080): No heartbeat from core client for 30 sec - exiting 17:09:45 (3080): No heartbeat from core client for 30 sec - exiting 17:09:46 (3080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:32:32 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:11 (4940): No heartbeat from core client for 30 sec - exiting 17:35:12 (4940): No heartbeat from core client for 30 sec - exiting 17:35:13 (4940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:37:52 (4040): No heartbeat from core client for 30 sec - exiting 17:37:53 (4040): No heartbeat from core client for 30 sec - exiting 17:37:54 (4040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:49:08 (10432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2013 14:50:58	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	1,036,800	790,805	0.7627
23 Jul 2013 14:50:58	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	1,010,880	768,153	0.7599
23 Jul 2013 14:50:58	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	984,960	745,354	0.7567
11 Jul 2013 21:55:56	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	959,040	722,508	0.7534
11 Jul 2013 15:33:03	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	933,120	699,584	0.7497
11 Jul 2013 09:11:16	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	907,200	676,623	0.7458
11 Jul 2013 02:33:22	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	881,280	653,808	0.7419
10 Jul 2013 20:10:04	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	855,360	630,903	0.7376
10 Jul 2013 13:52:00	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	829,440	608,019	0.7330
10 Jul 2013 07:30:35	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	803,520	585,276	0.7284
10 Jul 2013 01:23:49	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	777,600	563,236	0.7243
09 Jul 2013 19:30:36	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	751,680	541,027	0.7198
09 Jul 2013 07:06:10	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	725,760	519,035	0.7152
09 Jul 2013 00:53:58	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	699,840	496,787	0.7099
08 Jul 2013 18:55:46	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	673,920	475,354	0.7054
08 Jul 2013 07:47:07	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	648,000	453,259	0.6995
07 Jul 2013 22:39:49	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	622,080	430,762	0.6925
07 Jul 2013 16:17:46	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	596,160	407,862	0.6841
22 Jun 2013 13:38:23	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	570,240	386,105	0.6771
21 Jun 2013 22:59:04	1270442	15814771	hadcm3n_o3v0_1940_40_008383120_0	544,320	366,318	0.6730