Task 11737657

Name	famous_v7kj_1799_200_006695577_3
Workunit	6898830
Created	26 Aug 2010, 16:24:38 UTC
Sent	11 Dec 2010, 16:14:20 UTC
Report deadline	12 Mar 2011, 23:41:31 UTC
Received	21 Dec 2010, 16:14:19 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1046641
Run time	16 hours 9 min 14 sec
CPU time	14 hours 15 min 38 sec
Validate state	Invalid
Credit	370.67
Device peak FLOPS	2.79 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4644, selfPID=4644, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2520, selfPID=2520, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3748, selfPID=3748, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2812, selfPID=2812, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4600, selfPID=4600, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4780, selfPID=4780, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4652, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4652, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4732, selfPID=4732, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4352, selfPID=4352, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2144, selfPID=2144, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1316, selfPID=1316, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=648, selfPID=648, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1316, selfPID=1316, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4872, selfPID=4872, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2856, selfPID=2856, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4040, selfPID=4040, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4884, selfPID=4884, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4640, selfPID=4640, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2536, selfPID=2536, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4968, selfPID=4968, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4204, selfPID=4204, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4000, selfPID=4000, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4664, selfPID=4664, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4192, selfPID=4192, iMonCtr=1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4384, selfPID=4384, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=728, selfPID=728, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=4048, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 03:00:11 (3388): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4620, selfPID=4620, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4476, selfPID=4476, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4584, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4584, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1684, selfPID=1684, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4744, selfPID=4744, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1860, selfPID=1860, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=300, selfPID=300, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2864, selfPID=2864, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1352, selfPID=1352, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4316, selfPID=4316, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4288, selfPID=4288, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2600, selfPID=2600, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=376, selfPID=376, iMonCtr=1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4068, selfPID=4068, iMonCtr=1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=132, selfPID=132, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=300, selfPID=300, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2492, selfPID=2492, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4344, selfPID=4344, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2280, selfPID=2280, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3992, selfPID=3992, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2544, selfPID=2544, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=216, selfPID=216, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 20:51:56 (4564): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4824, selfPID=4824, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1356, selfPID=1356, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4584, selfPID=4584, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4056, selfPID=4056, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4496, selfPID=4496, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3496, selfPID=3496, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1896, selfPID=1896, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3880, selfPID=3880, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1620, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5024, selfPID=5024, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1208, selfPID=1208, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3032, selfPID=3032, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2784, selfPID=2784, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1712, selfPID=1712, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3352, selfPID=3352, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3804, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3804, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3804, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3804, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3804, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_v7kj_1799_200_006695577\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3804, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5348, selfPID=5348, iMonCtr=1 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_v7kj_1799_200_006695577/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Sorry, too many model crashes! :-( 11:11:31 (2540): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Dec 2010 15:32:11	1046641	11737657	famous_v7kj_1799_200_006695577_3	112,346	51,161	0.4554
20 Dec 2010 13:12:37	1046641	11737657	famous_v7kj_1799_200_006695577_3	102,986	46,796	0.4544
19 Dec 2010 22:33:40	1046641	11737657	famous_v7kj_1799_200_006695577_3	93,626	42,488	0.4538
19 Dec 2010 16:30:38	1046641	11737657	famous_v7kj_1799_200_006695577_3	84,266	38,071	0.4518
17 Dec 2010 15:03:02	1046641	11737657	famous_v7kj_1799_200_006695577_3	74,906	33,703	0.4499
16 Dec 2010 22:02:19	1046641	11737657	famous_v7kj_1799_200_006695577_3	65,546	29,399	0.4485
15 Dec 2010 13:05:00	1046641	11737657	famous_v7kj_1799_200_006695577_3	56,186	25,313	0.4505
14 Dec 2010 21:34:51	1046641	11737657	famous_v7kj_1799_200_006695577_3	46,826	21,586	0.4610
13 Dec 2010 23:25:54	1046641	11737657	famous_v7kj_1799_200_006695577_3	37,466	17,276	0.4611
12 Dec 2010 21:43:44	1046641	11737657	famous_v7kj_1799_200_006695577_3	28,106	12,968	0.4614
12 Dec 2010 14:55:09	1046641	11737657	famous_v7kj_1799_200_006695577_3	18,746	8,631	0.4604
11 Dec 2010 18:44:24	1046641	11737657	famous_v7kj_1799_200_006695577_3	9,386	4,317	0.4599