Task 13289371

Name	hadcm3n_p4ea_1940_40_007420704_1
Workunit	7618339
Created	24 Aug 2011, 23:48:23 UTC
Sent	24 Aug 2011, 23:49:06 UTC
Report deadline	24 Nov 2011, 7:16:17 UTC
Received	12 Sep 2011, 17:31:51 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1142344
Run time	6 days 21 hours 30 min 26 sec
CPU time	5 days 16 hours 43 min 37 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.59</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:42:38 (15089): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:43:40 (15089): No heartbeat from core client for 30 sec - exiting 13:43:41 (15089): No heartbeat from core client for 30 sec - exiting 13:43:42 (15089): No heartbeat from core client for 30 sec - exiting 13:43:43 (15089): No heartbeat from core client for 30 sec - exiting 13:43:44 (15089): No heartbeat from core client for 30 sec - exiting 13:45:27 (15089): No heartbeat from core client for 30 sec - exiting 13:45:40 (15089): No heartbeat from core client for 30 sec - exiting 13:45:41 (15089): No heartbeat from core client for 30 sec - exiting 13:45:42 (15089): No heartbeat from core client for 30 sec - exiting 13:45:43 (15089): No heartbeat from core client for 30 sec - exiting 13:45:44 (15089): No heartbeat from core client for 30 sec - exiting 13:52:40 (23856): No heartbeat from core client for 30 sec - exiting 13:52:41 (23856): No heartbeat from core client for 30 sec - exiting 13:52:42 (23856): No heartbeat from core client for 30 sec - exiting 13:52:43 (23856): No heartbeat from core client for 30 sec - exiting 13:52:44 (23856): No heartbeat from core client for 30 sec - exiting 13:52:45 (23856): No heartbeat from core client for 30 sec - exiting 13:52:46 (23856): No heartbeat from core client for 30 sec - exiting 13:52:47 (23856): No heartbeat from core client for 30 sec - exiting 13:52:48 (23856): No heartbeat from core client for 30 sec - exiting 13:52:49 (23856): No heartbeat from core client for 30 sec - exiting 13:52:50 (23856): No heartbeat from core client for 30 sec - exiting 13:52:52 (23856): No heartbeat from core client for 30 sec - exiting 13:52:53 (23856): No heartbeat from core client for 30 sec - exiting 13:52:54 (23856): No heartbeat from core client for 30 sec - exiting 13:52:55 (23856): No heartbeat from core client for 30 sec - exiting 13:52:56 (23856): No heartbeat from core client for 30 sec - exiting 13:52:57 (23856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_p4ea_1940_40_007420704/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 084554BF Unknown Unknown Unknown hadcm3n_um_6.07_i 083721F7 Unknown Unknown Unknown hadcm3n_um_6.07_i 0837FE35 Unknown Unknown Unknown hadcm3n_um_6.07_i 0822B1DE Unknown Unknown Unknown hadcm3n_um_6.07_i 0818A767 Unknown Unknown Unknown hadcm3n_um_6.07_i 0818D749 Unknown Unknown Unknown hadcm3n_um_6.07_i 08391957 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F8B7 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 001CAE37 Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20320, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_p4ea_1940_40_007420704/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 00618E37 Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPICalled boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Sep 2011 12:58:36	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	233,280	478,726	2.0522
11 Sep 2011 20:33:56	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	207,360	424,914	2.0492
10 Sep 2011 21:12:36	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	181,440	370,744	2.0433
10 Sep 2011 00:54:42	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	155,520	317,068	2.0388
08 Sep 2011 08:52:21	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	129,600	263,483	2.0330
02 Sep 2011 10:31:51	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	103,680	209,592	2.0215
31 Aug 2011 20:59:27	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	77,760	156,839	2.0170
29 Aug 2011 19:39:21	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	51,840	104,483	2.0155
28 Aug 2011 17:22:02	1142344	13289371	hadcm3n_p4ea_1940_40_007420704_1	25,920	52,374	2.0206