Name | hadcm3n_p3py_1900_40_007222406_2 |
Workunit | 7420646 |
Created | 26 May 2011, 18:21:18 UTC |
Sent | 26 May 2011, 18:21:23 UTC |
Report deadline | 26 Aug 2011, 1:48:34 UTC |
Received | 27 Jun 2011, 22:22:50 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1130793 |
Run time | 6 days 15 hours 35 min 55 sec |
CPU time | 5 days 23 hours 23 min 40 sec |
Validate state | Invalid |
Credit | 3,732.48 |
Device peak FLOPS | 2.81 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin |
Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 19:16:20 (87640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x101ca04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1038004: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1038000: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1038004: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1038000: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x866004: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x101ca04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1038004: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1038000: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1076a04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1076a00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x828404: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x828400: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x828404: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x828400: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1075c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1076a04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x1076a00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22393,0xa0364540) malloc: *** error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=22398, selfPID=22398, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(30220,0xa0364540) malloc: *** error for object 0x84ec04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30220,0xa0364540) malloc: *** error for object 0x1028604: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30220,0xa0364540) malloc: *** error for object 0x84ec04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30220,0xa0364540) malloc: *** error for object 0x1021e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30220,0xa0364540) malloc: *** error for object 0x1021e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30220,0xa0364540) malloc: *** error for object 0x1021e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x83de04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x83de00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x83de04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x83de00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31361,0xa0364540) malloc: *** error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error: Output file: dataout/p3pyko.pib2c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyko.pib2c10 Error: Output file: dataout/p3pyko.pfb2c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyko.pfb2c10 Error: Output file: dataout/p3pyka.phb2c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyka.phb2c10 Error: Output file: dataout/p3pyka.pgb2 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error: Output file: dataout/p3pyko.pib3c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyko.pib3c10 Error: Output file: dataout/p3pyko.pfb3c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyko.pfb3c10 Error: Output file: dataout/p3pyka.phb3c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyka.phb3c10 Error: Output file: dataout/p3pyka.pgb3c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyka.pgb3c10 Error: Output file: dataout/p3pyka.peb3c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyka.peb3c10 Error: Output file: dataout/p3pyka.pdb3c10.nc already exists or could not be opened. Error converting file to netcdf: dataout/p3pyka.pdb3c10 Skipping gmts_generator due to netcdf error 28 - No space left on device Skipping gmts_generator due to netcdf error 28 - No space left on device OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/p3pyka.dab3c20 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_p3py_1900_40_007222406/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003E1E32 Unknown Unknown Unknown hadcm3n_um_6.07_i 003DF4CC Unknown Unknown Unknown hadcm3n_um_6.07_i 003651C5 Unknown Unknown Unknown hadcm3n_um_6.07_i 002DE95F Unknown Unknown Unknown hadcm3n_um_6.07_i 00389F90 Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=461, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_p3py_1900_40_007222406/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003E1E32 Unknown Unknown Unknown hadcm3n_um_6.07_i 003DF4CC Unknown Unknown Unknown hadcm3n_um_6.07_i 003651C5 Unknown Unknown Unknown hadcm3n_um_6.07_i 002DE95F Unknown Unknown Unknown hadcm3n_um_6.07_i 00389F90 Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
27 Jun 2011 09:54:19 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 311,040 | 478,136 | 1.5372 |
20 Jun 2011 19:00:06 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 285,120 | 438,553 | 1.5381 |
13 Jun 2011 12:35:43 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 259,200 | 398,513 | 1.5375 |
11 Jun 2011 19:37:08 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 233,280 | 358,497 | 1.5368 |
10 Jun 2011 21:13:30 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 207,360 | 319,078 | 1.5388 |
09 Jun 2011 20:11:32 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 181,440 | 279,229 | 1.5390 |
08 Jun 2011 14:50:08 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 155,520 | 239,570 | 1.5404 |
07 Jun 2011 10:20:23 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 129,600 | 199,722 | 1.5411 |
06 Jun 2011 08:40:11 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 103,680 | 160,090 | 1.5441 |
04 Jun 2011 22:41:08 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 77,760 | 121,065 | 1.5569 |
03 Jun 2011 17:50:48 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 51,840 | 81,907 | 1.5800 |
02 Jun 2011 20:57:10 | 1130793 | 12910340 | hadcm3n_p3py_1900_40_007222406_2 | 25,920 | 42,393 | 1.6355 |
©2024 cpdn.org