Task 13218669

Name	hadcm3n_ymp3_1900_40_007362273_2
Workunit	7559703
Created	9 Aug 2011, 19:14:55 UTC
Sent	9 Aug 2011, 19:15:22 UTC
Report deadline	9 Nov 2011, 2:42:33 UTC
Received	28 Aug 2011, 23:42:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	958046
Run time	11 days 13 hours 48 min 43 sec
CPU time	9 days 17 hours 28 min
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.23 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:35:37 (11958): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:04:46 (67898): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:04:48 (67898): No heartbeat from core client for 30 sec - exiting 05:04:49 (67898): No heartbeat from core client for 30 sec - exiting 05:04:50 (67898): No heartbeat from core client for 30 sec - exiting 05:04:51 (67898): No heartbeat from core client for 30 sec - exiting 05:04:52 (67898): No heartbeat from core client for 30 sec - exiting 05:04:53 (67898): No heartbeat from core client for 30 sec - exiting 05:04:54 (67898): No heartbeat from core client for 30 sec - exiting 05:04:55 (67898): No heartbeat from core client for 30 sec - exiting 05:04:56 (67898): No heartbeat from core client for 30 sec - exiting 05:04:57 (67898): No heartbeat from core client for 30 sec - exiting 05:04:58 (67898): No heartbeat from core client for 30 sec - exiting 05:04:59 (67898): No heartbeat from core client for 30 sec - exiting 05:05:00 (67898): No heartbeat from core client for 30 sec - exiting 05:05:01 (67898): No heartbeat from core client for 30 sec - exiting 05:05:02 (67898): No heartbeat from core client for 30 sec - exiting 05:05:03 (67898): No heartbeat from core client for 30 sec - exiting 05:05:04 (67898): No heartbeat from core client for 30 sec - exiting 05:05:05 (67898): No heartbeat from core client for 30 sec - exiting 05:05:06 (67898): No heartbeat from core client for 30 sec - exiting 05:05:07 (67898): No heartbeat from core client for 30 sec - exiting 05:05:08 (67898): No heartbeat from core client for 30 sec - exiting 05:05:09 (67898): No heartbeat from core client for 30 sec - exiting 05:05:10 (67898): No heartbeat from core client for 30 sec - exiting 05:05:11 (67898): No heartbeat from core client for 30 sec - exiting 05:05:12 (67898): No heartbeat from core client for 30 sec - exiting 05:05:13 (67898): No heartbeat from core client for 30 sec - exiting 05:05:14 (67898): No heartbeat from core client for 30 sec - exiting 05:05:15 (67898): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:31:06 (29560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:27:59 (46056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:28:00 (46056): No heartbeat from core client for 30 sec - exiting 17:28:01 (46056): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:49:25 (46113): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:49:26 (46113): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 00:07:58 (53454): No heartbeat from core client for 30 sec - exiting 00:07:59 (53454): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:04:19 (63182): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:10:26 (63521): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:03:11 (51476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:57:54 (67935): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1042404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1042400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1837604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1852c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1852c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x821004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x83c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x83c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1042404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1042400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1042404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x1042400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21826,0xa075b540) malloc: * error for object 0x821004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=21841, selfPID=21841, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x281b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x849004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x182ac04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x381b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x401d600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x1800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x1800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(57208,0xa075b540) malloc: * error for object 0x381c404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:13:15 (5942): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:56:16 (35980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:12:58 (37010): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:13:00 (37010): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:07:13 (70613): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:16:32 (80616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:18:19 (80635): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:31:24 (80655): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:10:34 (82027): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:58:38 (11401): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:54 (11415): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:56 (11415): No heartbeat from core client for 30 sec - exiting 17:31:57 (11415): No heartbeat from core client for 30 sec - exiting 17:31:58 (11415): No heartbeat from core client for 30 sec - exiting 17:31:59 (11415): No heartbeat from core client for 30 sec - exiting 17:32:00 (11415): No heartbeat from core client for 30 sec - exiting 17:32:01 (11415): No heartbeat from core client for 30 sec - exiting 17:32:02 (11415): No heartbeat from core client for 30 sec - exiting 17:32:03 (11415): No heartbeat from core client for 30 sec - exiting 17:32:04 (11415): No heartbeat from core client for 30 sec - exiting 17:32:05 (11415): No heartbeat from core client for 30 sec - exiting 17:32:06 (11415): No heartbeat from core client for 30 sec - exiting 17:32:07 (11415): No heartbeat from core client for 30 sec - exiting 17:32:08 (11415): No heartbeat from core client for 30 sec - exiting 17:32:09 (11415): No heartbeat from core client for 30 sec - exiting 17:32:10 (11415): No heartbeat from core client for 30 sec - exiting 17:32:11 (11415): No heartbeat from core client for 30 sec - exiting 17:32:12 (11415): No heartbeat from core client for 30 sec - exiting 17:32:13 (11415): No heartbeat from core client for 30 sec - exiting 17:32:14 (11415): No heartbeat from core client for 30 sec - exiting 17:32:15 (11415): No heartbeat from core client for 30 sec - exiting 17:32:16 (11415): No heartbeat from core client for 30 sec - exiting 17:32:17 (11415): No heartbeat from core client for 30 sec - exiting 17:32:18 (11415): No heartbeat from core client for 30 sec - exiting 17:32:19 (11415): No heartbeat from core client for 30 sec - exiting 17:32:20 (11415): No heartbeat from core client for 30 sec - exiting 17:32:21 (11415): No heartbeat from core client for 30 sec - exiting 17:32:22 (11415): No heartbeat from core client for 30 sec - exiting 17:32:23 (11415): No heartbeat from core client for 30 sec - exiting 17:32:24 (11415): No heartbeat from core client for 30 sec - exiting 17:33:48 (12934): No heartbeat from core client for 30 sec - exiting 17:33:49 (12934): No heartbeat from core client for 30 sec - exiting 17:33:50 (12934): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:11:58 (13518): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:15:53 (34981): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:15:54 (34981): No heartbeat from core client for 30 sec - exiting 18:15:55 (34981): No heartbeat from core client for 30 sec - exiting 18:15:56 (34981): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:38:03 (36159): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=36313, selfPID=36313, iMonCtr=1 03:31:54 (1555): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:24:57 (58143): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:24:58 (58143): No heartbeat from core client for 30 sec - exiting 18:24:59 (58143): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 136190) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59438, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 136190) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59438, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 136190) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59438, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 136190) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59438, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 136190) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59438, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 136190) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59438, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Aug 2011 19:06:28	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	518,400	834,166	1.6091
26 Aug 2011 15:14:27	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	492,480	790,962	1.6061
25 Aug 2011 18:58:44	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	466,560	746,341	1.5997
23 Aug 2011 14:43:49	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	440,640	702,760	1.5949
22 Aug 2011 07:53:41	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	414,720	658,351	1.5875
21 Aug 2011 17:49:24	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	388,800	618,096	1.5898
21 Aug 2011 02:45:51	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	362,880	576,339	1.5882
20 Aug 2011 12:28:37	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	336,960	534,404	1.5860
19 Aug 2011 19:35:13	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	311,040	493,695	1.5872
19 Aug 2011 02:13:50	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	285,120	452,342	1.5865
18 Aug 2011 07:15:32	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	259,200	409,433	1.5796
17 Aug 2011 14:38:47	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	233,280	368,628	1.5802
16 Aug 2011 19:50:28	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	207,360	329,227	1.5877
15 Aug 2011 14:08:39	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	181,440	288,302	1.5890
14 Aug 2011 19:47:32	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	155,520	247,044	1.5885
13 Aug 2011 15:24:50	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	129,600	206,381	1.5924
13 Aug 2011 00:42:09	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	103,680	166,994	1.6107
12 Aug 2011 08:46:07	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	77,760	123,089	1.5829
11 Aug 2011 15:20:39	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	51,840	80,938	1.5613
11 Aug 2011 00:29:41	958046	13218669	hadcm3n_ymp3_1900_40_007362273_2	25,920	40,412	1.5591