Task 13143372

Name	hadcm3n_yd5d_1900_40_007349899_2
Workunit	7547329
Created	17 Jul 2011, 9:37:37 UTC
Sent	17 Jul 2011, 9:52:28 UTC
Report deadline	16 Oct 2011, 17:19:39 UTC
Received	23 Aug 2011, 14:54:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	826910
Run time	11 days 23 hours 11 min 23 sec
CPU time	11 days 23 hours 7 min 7 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.26 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 07:43:38 (8141): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2193, selfPID=2193, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2186, selfPID=2186, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8668, selfPID=8668, iMonCtr=1 cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadcm3n_yd5d_1900_40_007349899/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Aug 2011 05:39:45	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	518,400	1,033,643	1.9939
21 Aug 2011 16:49:10	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	492,480	987,928	2.0060
20 Aug 2011 15:58:57	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	466,560	941,774	2.0185
19 Aug 2011 20:43:40	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	440,640	895,586	2.0325
17 Aug 2011 20:33:12	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	414,720	842,243	2.0309
15 Aug 2011 15:55:44	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	388,800	788,938	2.0292
14 Aug 2011 08:47:58	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	362,880	735,776	2.0276
13 Aug 2011 10:20:29	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	336,960	682,420	2.0252
12 Aug 2011 19:26:34	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	311,040	629,151	2.0227
10 Aug 2011 04:07:04	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	285,120	575,192	2.0174
07 Aug 2011 16:34:10	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	259,200	520,404	2.0077
06 Aug 2011 17:56:34	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	233,280	465,682	1.9962
05 Aug 2011 16:12:40	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	207,360	412,130	1.9875
02 Aug 2011 18:12:05	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	181,440	358,611	1.9765
31 Jul 2011 09:30:33	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	155,520	321,252	2.0657
30 Jul 2011 07:26:36	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	129,600	267,367	2.0630
28 Jul 2011 03:25:09	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	103,680	213,881	2.0629
25 Jul 2011 22:09:47	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	77,760	160,443	2.0633
25 Jul 2011 20:44:30	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	51,840	106,864	2.0614
25 Jul 2011 19:35:56	826910	13143372	hadcm3n_yd5d_1900_40_007349899_2	25,920	53,582	2.0672