Task 15458731

Name	hadcm3n_zhhv_1880_40_008253096_1
Workunit	8408220
Created	23 Nov 2012, 17:56:33 UTC
Sent	23 Nov 2012, 18:01:06 UTC
Report deadline	23 Feb 2013, 1:28:17 UTC
Received	6 Jan 2013, 16:00:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1238098
Run time	20 days 14 hours 42 min 15 sec
CPU time	19 days 21 hours 0 min 6 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	3.13 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file /opt/BOINC/projects/climateprediction.net/hadcm3n_zhhv_1880_40_008253096/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08452BFF Unknown Unknown Unknown hadcm3n_um_6.07_i 08340198 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F395 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F756E003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file /opt/BOINC/projects/climateprediction.net/hadcm3n_zhhv_1880_40_008253096/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08452BFF Unknown Unknown Unknown hadcm3n_um_6.07_i 08340198 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F395 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75EE003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file /opt/BOINC/projects/climateprediction.net/hadcm3n_zhhv_1880_40_008253096/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08452BFF Unknown Unknown Unknown hadcm3n_um_6.07_i 08340198 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F395 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F7602003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file /opt/BOINC/projects/climateprediction.net/hadcm3n_zhhv_1880_40_008253096/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08452BFF Unknown Unknown Unknown hadcm3n_um_6.07_i 08340198 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F395 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75C1003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file /opt/BOINC/projects/climateprediction.net/hadcm3n_zhhv_1880_40_008253096/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08452BFF Unknown Unknown Unknown hadcm3n_um_6.07_i 08340198 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F395 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75E3003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file /opt/BOINC/projects/climateprediction.net/hadcm3n_zhhv_1880_40_008253096/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08452BFF Unknown Unknown Unknown hadcm3n_um_6.07_i 08340198 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F395 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75D9003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Jan 2013 16:00:56	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	855,360	1,685,622	1.9707
05 Jan 2013 16:00:56	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	829,440	1,637,694	1.9745
04 Jan 2013 16:02:32	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	803,520	1,589,258	1.9779
03 Jan 2013 16:01:00	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	777,600	1,540,455	1.9810
02 Jan 2013 16:01:50	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	751,680	1,492,516	1.9856
01 Jan 2013 16:01:03	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	725,760	1,444,625	1.9905
31 Dec 2012 16:01:01	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	699,840	1,395,791	1.9944
30 Dec 2012 16:01:05	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	673,920	1,347,220	1.9991
29 Dec 2012 16:02:24	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	648,000	1,298,451	2.0038
28 Dec 2012 16:01:06	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	622,080	1,249,669	2.0089
27 Dec 2012 16:01:02	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	596,160	1,197,744	2.0091
26 Dec 2012 16:00:35	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	570,240	1,144,760	2.0075
26 Dec 2012 16:00:35	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	544,320	1,092,111	2.0064
25 Dec 2012 16:01:04	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	518,400	1,040,845	2.0078
24 Dec 2012 16:01:07	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	492,480	987,302	2.0048
24 Dec 2012 16:01:07	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	466,560	937,343	2.0091
23 Dec 2012 16:04:42	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	440,640	885,952	2.0106
22 Dec 2012 16:00:51	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	414,720	831,296	2.0045
19 Dec 2012 16:03:13	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	388,800	777,349	1.9994
16 Dec 2012 16:01:05	1238098	15458731	hadcm3n_zhhv_1880_40_008253096_1	362,880	726,675	2.0025