Task 11580428

Name	famous_upim_1399_200_006665913_0
Workunit	6869285
Created	10 Jun 2010, 15:49:56 UTC
Sent	11 Jun 2010, 14:53:51 UTC
Report deadline	10 Sep 2010, 22:21:02 UTC
Received	18 Jun 2010, 9:25:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1081230
Run time	23 hours 30 min 48 sec
CPU time	20 hours 11 min 53 sec
Validate state	Invalid
Credit	339.78
Device peak FLOPS	2.14 GFLOPS
Application version	UK Met Office FAMOUS v6.10 windows_intelx86
Stderr	<core_client_version>6.10.57</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2564, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:33:04 (576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001400c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001400c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001400c1+ Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001401c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001401c1+ No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3432, selfPID=3432, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001401c1+ Signal 4 received, exiting... 02:41:42 (2352): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2724, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 03:10:25 (524): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2724, iMonCtr=1 Model crash detected, will try to restart... Signal 4 received, exiting... 03:25:50 (3532): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2724, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001402c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001402c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001402c1+ CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 4 received, exiting... 08:43:48 (1036): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4020, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Signal 4 received, exiting... 09:08:55 (2352): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4020, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 11 received, exiting... 09:16:20 (3220): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4020, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001403c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001403c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001403c1+ CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 4 received, exiting... 18:42:48 (2488): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2328, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001404c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001404c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001404c1+ CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 14:57:46 (256): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4028, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001405c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001405c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001405c1+ Signal 11 received, exiting... 22:52:37 (1752): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2904, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001406c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001406c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001406c1+ 01:53:20 (2660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 07:53:47 (2704): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2360, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001407c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001407c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001407c1+ CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 16:06:49 (2524): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2108, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001408c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001408c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001408c1+ CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001409c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001409c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001409c1+ CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 06:53:30 (2340): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1808, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 11:08:12 (3988): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=868, iMonCtr=1 Model crash detected, will try to restart... Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pa000001410c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimla#pb000001410c1+ Error converting file to netcdf: C:\Documents and Settings\All Users.WINDOWS.0\Application Data\BOINC/projects/climateprediction.net/famous_upim_1399_200_006665913/dataout\upimlo#pj000001410c1+ CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( 12:24:51 (1516): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Jun 2010 08:16:02	1081230	11580428	famous_upim_1399_200_006665913_0	102,986	70,599	0.6855
18 Jun 2010 02:08:21	1081230	11580428	famous_upim_1399_200_006665913_0	93,626	63,702	0.6804
17 Jun 2010 18:31:37	1081230	11580428	famous_upim_1399_200_006665913_0	84,266	56,876	0.6750
17 Jun 2010 12:40:05	1081230	11580428	famous_upim_1399_200_006665913_0	74,906	50,267	0.6711
16 Jun 2010 22:47:07	1081230	11580428	famous_upim_1399_200_006665913_0	65,546	44,193	0.6742
16 Jun 2010 18:59:20	1081230	11580428	famous_upim_1399_200_006665913_0	56,186	38,128	0.6786
16 Jun 2010 09:41:11	1081230	11580428	famous_upim_1399_200_006665913_0	46,826	31,943	0.6822
15 Jun 2010 07:53:33	1081230	11580428	famous_upim_1399_200_006665913_0	37,466	25,670	0.6852
15 Jun 2010 02:34:11	1081230	11580428	famous_upim_1399_200_006665913_0	28,106	18,644	0.6633
14 Jun 2010 23:53:04	1081230	11580428	famous_upim_1399_200_006665913_0	18,746	11,868	0.6331
14 Jun 2010 12:47:39	1081230	11580428	famous_upim_1399_200_006665913_0	9,386	5,527	0.5889