Task 11884340

Name	famous_vpas_1999_200_006718554_5
Workunit	6921807
Created	13 Sep 2010, 21:25:33 UTC
Sent	13 Sep 2010, 21:31:05 UTC
Report deadline	14 Dec 2010, 4:58:16 UTC
Received	20 Sep 2010, 12:33:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1100788
Run time	16 hours 32 min 44 sec
CPU time	15 hours 46 min 30 sec
Validate state	Invalid
Credit	185.38
Device peak FLOPS	0.91 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002000c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002000c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002000c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002000c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002000c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002000c1+ CPDN Monitor - Quit request from BOINC... file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002001c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002001c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002001c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002001c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002001c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002001c1+ CPDN Monitor - Quit request from BOINC... file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002002c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002002c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002002c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002002c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002002c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002002c1+ CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002003c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002003c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002003c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002003c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002003c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002003c1+ CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... (12578): No heartbeat from core client for 30 sec - exiting (12578): No heartbeat from core client for 30 sec - exiting (12578): No heartbeat from core client for 30 sec - exiting (12578): No heartbeat from core client for 30 sec - exiting (12578): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002004c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002004c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002004c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002004c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002004c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002004c1+ CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002005c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpasla#pb000002005c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002005c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pi000002005c1+ file : /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002005c1+ does not exist Error converting file to netcdf: /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/vpaslo#pj000002005c1+ CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/stdout_um.txt Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083DF6B0 Unknown Unknown Unknown famous_um_6.11_i6 080B4684 Unknown Unknown Unknown famous_um_6.11_i6 0807C91A Unknown Unknown Unknown famous_um_6.11_i6 08314811 Unknown Unknown Unknown famous_um_6.11_i6 0832DC4D Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 003D3CE7 Unknown Unknown Unknown famous_um_6.11_i6 0804C041 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2774, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/stdout_um.txt Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083E043D Unknown Unknown Unknown famous_um_6.11_i6 08267F53 Unknown Unknown Unknown famous_um_6.11_i6 082EFC2C Unknown Unknown Unknown famous_um_6.11_i6 08328714 Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 001F5CE7 Unknown Unknown Unknown famous_um_6.11_i6 0804C041 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2774, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/stdout_um.txt Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083E03A1 Unknown Unknown Unknown famous_um_6.11_i6 082DBF15 Unknown Unknown Unknown famous_um_6.11_i6 082B24E4 Unknown Unknown Unknown famous_um_6.11_i6 0832AFFB Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 00A33CE7 Unknown Unknown Unknown famous_um_6.11_i6 0804C041 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2774, iMonCtr=1 Model crash detected, will try to restart... forrtl: No spaceController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2774, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/stdout_um.txt Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083E03A1 Unknown Unknown Unknown famous_um_6.11_i6 08162C80 Unknown Unknown Unknown famous_um_6.11_i6 08163229 Unknown Unknown Unknown famous_um_6.11_i6 081777BF Unknown Unknown Unknown famous_um_6.11_i6 08325C74 Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 0049CCE7 Unknown Unknown Unknown famous_um_6.11_i6 0804C041 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2774, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/famous_vpas_1999_200_006718554/dataout/stdout_um.txt Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083E043D Unknown Unknown Unknown famous_um_6.11_i6 080EA698 Unknown Unknown Unknown famous_um_6.11_i6 08202CBF Unknown Unknown Unknown famous_um_6.11_i6 08173370 Unknown Unknown Unknown famous_um_6.11_i6 0817613B Unknown Unknown Unknown famous_um_6.11_i6 08325C74 Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 00200CE7 Unknown Unknown Unknown famous_um_6.11_i6 0804C041 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2774, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( (2774): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Sep 2010 17:31:50	1100788	11884340	famous_vpas_1999_200_006718554_5	56,186	48,931	0.8709
14 Sep 2010 19:39:20	1100788	11884340	famous_vpas_1999_200_006718554_5	46,826	40,664	0.8684
14 Sep 2010 12:18:17	1100788	11884340	famous_vpas_1999_200_006718554_5	37,466	32,577	0.8695
14 Sep 2010 07:54:30	1100788	11884340	famous_vpas_1999_200_006718554_5	28,106	24,480	0.8710
14 Sep 2010 04:52:34	1100788	11884340	famous_vpas_1999_200_006718554_5	18,746	16,375	0.8735
14 Sep 2010 01:33:21	1100788	11884340	famous_vpas_1999_200_006718554_5	9,386	8,300	0.8843