Task 15631962

Name	hadcm3n_3jsg_1980_40_008318030_0
Workunit	8469165
Created	24 Feb 2013, 3:44:15 UTC
Sent	24 Feb 2013, 3:44:22 UTC
Report deadline	26 May 2013, 11:11:33 UTC
Received	9 Apr 2013, 16:01:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	1 (0x00000001) Unknown error code
Computer ID	1257063
Run time	24 days 16 hours 48 min 44 sec
CPU time	19 days 18 hours 36 min 37 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	0.99 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> Incorrect function. (0x1) - exit code 1 (0x1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1236, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 14:54:49 (3080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:17:41 (2184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1120, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:11:44 (2300): No heartbeat from core client for 30 sec - exiting 06:11:46 (2300): No heartbeat from core client for 30 sec - exiting 06:11:47 (2300): No heartbeat from core client for 30 sec - exiting 06:11:48 (2300): No heartbeat from core client for 30 sec - exiting 06:11:49 (2300): No heartbeat from core client for 30 sec - exiting 06:11:50 (2300): No heartbeat from core client for 30 sec - exiting 06:11:51 (2300): No heartbeat from core client for 30 sec - exiting 06:11:52 (2300): No heartbeat from core client for 30 sec - exiting 06:11:53 (2300): No heartbeat from core client for 30 sec - exiting 06:11:54 (2300): No heartbeat from core client for 30 sec - exiting 06:11:55 (2300): No heartbeat from core client for 30 sec - exiting 06:11:57 (2300): No heartbeat from core client for 30 sec - exiting 06:11:58 (2300): No heartbeat from core client for 30 sec - exiting 06:11:59 (2300): No heartbeat from core client for 30 sec - exiting 06:12:00 (2300): No heartbeat from core client for 30 sec - exiting 06:12:01 (2300): No heartbeat from core client for 30 sec - exiting 06:12:02 (2300): No heartbeat from core client for 30 sec - exiting 06:12:03 (2300): No heartbeat from core client for 30 sec - exiting 06:12:04 (2300): No heartbeat from core client for 30 sec - exiting 06:12:05 (2300): No heartbeat from core client for 30 sec - exiting 06:12:06 (2300): No heartbeat from core client for 30 sec - exiting 06:12:07 (2300): No heartbeat from core client for 30 sec - exiting 06:12:09 (2300): No heartbeat from core client for 30 sec - exiting 06:12:10 (2300): No heartbeat from core client for 30 sec - exiting 06:12:11 (2300): No heartbeat from core client for 30 sec - exiting 06:12:12 (2300): No heartbeat from core client for 30 sec - exiting 06:12:13 (2300): No heartbeat from core client for 30 sec - exiting 06:12:14 (2300): No heartbeat from core client for 30 sec - exiting 06:12:15 (2300): No heartbeat from core client for 30 sec - exiting 06:12:16 (2300): No heartbeat from core client for 30 sec - exiting 06:12:17 (2300): No heartbeat from core client for 30 sec - exiting 06:12:18 (2300): No heartbeat from core client for 30 sec - exiting 06:12:19 (2300): No heartbeat from core client for 30 sec - exiting 06:12:21 (2300): No heartbeat from core client for 30 sec - exiting 06:12:22 (2300): No heartbeat from core client for 30 sec - exiting 06:12:23 (2300): No heartbeat from core client for 30 sec - exiting 06:12:24 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:29 (1636): Can't acquire lockfile (32) - waiting 35s BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3jsgko.pjk8c10 Error converting file to netcdf: dataout/3jsgko.pik8c10 Error converting file to netcdf: dataout/3jsgko.pfk8c10 Error converting file to netcdf: dataout/3jsgka.phk8c10 Error converting file to netcdf: dataout/3jsgka.pgk8c10 Error converting file to netcdf: dataout/3jsgka.pek8c10 Error converting file to netcdf: dataout/3jsgka.pdk8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:35:40 (3544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:21:52 (2624): No heartbeat from core client for 30 sec - exiting 06:21:53 (2624): No heartbeat from core client for 30 sec - exiting 06:21:54 (2624): No heartbeat from core client for 30 sec - exiting 06:21:55 (2624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. CPDN Monitor - Quit request from BOINC... no start tag in app init data 06:44:40 (3108): Can't parse init data file - running in standalone mode Could not change to project directory Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Apr 2013 13:18:46	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	855,360	1,716,557	2.0068
07 Apr 2013 14:40:26	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	829,440	1,657,321	1.9981
06 Apr 2013 17:01:30	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	803,520	1,613,731	2.0083
05 Apr 2013 18:43:22	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	777,600	1,560,217	2.0065
04 Apr 2013 21:04:43	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	751,680	1,502,166	1.9984
02 Apr 2013 22:51:31	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	725,760	1,443,000	1.9883
01 Apr 2013 15:00:44	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	699,840	1,382,435	1.9754
31 Mar 2013 16:51:14	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	673,920	1,341,488	1.9906
30 Mar 2013 19:27:11	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	648,000	1,293,176	1.9956
30 Mar 2013 15:30:37	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	622,080	1,245,006	2.0014
26 Mar 2013 04:09:21	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	596,160	1,181,142	1.9813
25 Mar 2013 03:20:54	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	570,240	1,113,040	1.9519
23 Mar 2013 11:34:21	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	544,320	1,058,291	1.9442
22 Mar 2013 15:00:24	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	518,400	1,003,915	1.9366
21 Mar 2013 18:26:32	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	492,480	950,001	1.9290
19 Mar 2013 00:11:56	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	466,560	946,994	2.0297
18 Mar 2013 00:26:29	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	440,640	887,851	2.0149
17 Mar 2013 02:22:35	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	414,720	828,149	1.9969
16 Mar 2013 04:06:40	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	388,800	767,593	1.9743
13 Mar 2013 03:23:33	1257063	15631962	hadcm3n_3jsg_1980_40_008318030_0	362,880	706,578	1.9471