Task 15279494

Name	hadcm3n_zj89_1880_40_008200432_0
Workunit	8355556
Created	13 Sep 2012, 7:05:13 UTC
Sent	13 Sep 2012, 7:20:47 UTC
Report deadline	13 Dec 2012, 14:47:58 UTC
Received	27 Sep 2012, 13:48:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1240784
Run time	13 days 9 hours 8 min 39 sec
CPU time	3 days 15 hours 0 min 11 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	2.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.27</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> 02:56:32 (7384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:57:09 (31749): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:59:01 (31789): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:04:25 (31856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:10:25 (31939): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:46:16 (32061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:54:56 (32665): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:13:41 (345): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:40:07 (679): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:35 (1164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:51 (2029): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:12 (2159): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_zj89_1880_40_008200432/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 B75324D3 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2741, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_zj89_1880_40_008200432/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3nCalled boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Sep 2012 13:44:55	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	855,360	1,109,103	1.2967
26 Sep 2012 05:57:02	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	829,440	1,081,484	1.3039
25 Sep 2012 22:14:52	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	803,520	1,053,980	1.3117
25 Sep 2012 14:29:11	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	777,600	1,026,423	1.3200
25 Sep 2012 06:37:30	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	751,680	998,792	1.3287
24 Sep 2012 22:45:37	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	725,760	971,240	1.3382
24 Sep 2012 15:01:22	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	699,840	943,697	1.3484
24 Sep 2012 07:14:33	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	673,920	916,159	1.3594
23 Sep 2012 23:27:46	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	648,000	888,605	1.3713
23 Sep 2012 15:42:44	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	622,080	861,079	1.3842
23 Sep 2012 07:56:00	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	596,160	833,497	1.3981
23 Sep 2012 00:09:03	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	570,240	806,001	1.4134
22 Sep 2012 16:24:45	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	544,320	778,465	1.4302
22 Sep 2012 08:42:56	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	518,400	750,898	1.4485
22 Sep 2012 00:56:13	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	492,480	723,422	1.4689
21 Sep 2012 16:40:10	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	466,560	693,898	1.4873
21 Sep 2012 07:52:39	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	440,640	659,248	1.4961
20 Sep 2012 21:09:08	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	414,720	624,718	1.5064
20 Sep 2012 11:27:21	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	388,800	590,140	1.5178
20 Sep 2012 00:59:34	1193690	15279494	hadcm3n_zj89_1880_40_008200432_0	362,880	553,191	1.5244