Task 15681610

Name	hadcm3n_3lp9_1940_40_008265917_4
Workunit	8421041
Created	24 Mar 2013, 17:06:22 UTC
Sent	24 Mar 2013, 17:06:28 UTC
Report deadline	24 Jun 2013, 0:33:39 UTC
Received	17 Apr 2013, 12:08:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1331187
Run time	5 days 20 hours 11 min 7 sec
CPU time	5 days 13 hours 34 min 8 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	1.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.27</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:45:53 (1368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... OPEN: File Creation Failed: No such file or directory OPEN: Unable to Open File dataout/3lp9ko.dae7ba0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_se_6.07_i686-pc-linux-gnu.so after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/ocean_restart.day after 11 attempts execl(/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu, 135045) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1490, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_se_6.07_i686-pc-linux-gnu.so after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/ocean_restart.day after 11 attempts execl(/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu, 135045) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1490, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_se_6.07_i686-pc-linux-gnu.so after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/ocean_restart.day after 11 attempts execl(/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu, 135045) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1490, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_se_6.07_i686-pc-linux-gnu.so after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/ocean_restart.day after 11 attempts execl(/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu, 135045) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1490, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_se_6.07_i686-pc-linux-gnu.so after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/jobs/xabnk.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_3lp9_1940_40_008265917/dataout/ocean_restart.day after 11 attempts execl(/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu, 135045) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1490, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Apr 2013 19:03:48	1275376	15681610	hadcm3n_3lp9_1940_40_008265917_4	155,520	417,099	2.6820
13 Apr 2013 10:49:10	1275376	15681610	hadcm3n_3lp9_1940_40_008265917_4	129,600	346,077	2.6703
10 Apr 2013 13:50:07	1275376	15681610	hadcm3n_3lp9_1940_40_008265917_4	103,680	276,944	2.6711
07 Apr 2013 10:18:24	1275376	15681610	hadcm3n_3lp9_1940_40_008265917_4	77,760	206,412	2.6545
03 Apr 2013 18:11:11	1275376	15681610	hadcm3n_3lp9_1940_40_008265917_4	51,840	136,555	2.6342
01 Apr 2013 12:15:45	1275376	15681610	hadcm3n_3lp9_1940_40_008265917_4	25,920	68,911	2.6586