climateprediction.net home page
Task 13024186

Task 13024186

Name hadcm3n_o5v8_1900_40_007202935_2
Workunit 7401215
Created 28 Jun 2011, 22:16:32 UTC
Sent 28 Jun 2011, 22:31:56 UTC
Report deadline 28 Sep 2011, 5:59:07 UTC
Received 19 Jul 2011, 1:45:49 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 942621
Run time 12 days 5 hours 18 min 55 sec
CPU time 8 days 3 hours 20 min 27 sec
Validate state Invalid
Credit 5,287.68
Device peak FLOPS 3.44 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>6.6.36</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:19:24 (13855): No heartbeat from core client for 30 sec - exiting
21:19:26 (13855): No heartbeat from core client for 30 sec - exiting
21:19:27 (13855): No heartbeat from core client for 30 sec - exiting
21:19:28 (13855): No heartbeat from core client for 30 sec - exiting
21:19:29 (13855): No heartbeat from core client for 30 sec - exiting
21:19:30 (13855): No heartbeat from core client for 30 sec - exiting
21:19:31 (13855): No heartbeat from core client for 30 sec - exiting
21:19:32 (13855): No heartbeat from core client for 30 sec - exiting
21:19:33 (13855): No heartbeat from core client for 30 sec - exiting
21:19:34 (13855): No heartbeat from core client for 30 sec - exiting
21:19:35 (13855): No heartbeat from core client for 30 sec - exiting
21:19:36 (13855): No heartbeat from core client for 30 sec - exiting
21:19:37 (13855): No heartbeat from core client for 30 sec - exiting
21:19:38 (13855): No heartbeat from core client for 30 sec - exiting
21:19:39 (13855): No heartbeat from core client for 30 sec - exiting
21:19:40 (13855): No heartbeat from core client for 30 sec - exiting
21:19:41 (13855): No heartbeat from core client for 30 sec - exiting
21:19:42 (13855): No heartbeat from core client for 30 sec - exiting
21:19:43 (13855): No heartbeat from core client for 30 sec - exiting
21:19:44 (13855): No heartbeat from core client for 30 sec - exiting
21:19:45 (13855): No heartbeat from core client for 30 sec - exiting
21:19:46 (13855): No heartbeat from core client for 30 sec - exiting
21:19:47 (13855): No heartbeat from core client for 30 sec - exiting
21:19:48 (13855): No heartbeat from core client for 30 sec - exiting
21:19:49 (13855): No heartbeat from core client for 30 sec - exiting
21:19:50 (13855): No heartbeat from core client for 30 sec - exiting
21:19:51 (13855): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:28:56 (23586): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
04:32:45 (35764): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:32:47 (35764): No heartbeat from core client for 30 sec - exiting
04:32:48 (35764): No heartbeat from core client for 30 sec - exiting
04:32:49 (35764): No heartbeat from core client for 30 sec - exiting
04:32:50 (35764): No heartbeat from core client for 30 sec - exiting
04:32:51 (35764): No heartbeat from core client for 30 sec - exiting
04:32:52 (35764): No heartbeat from core client for 30 sec - exiting
04:32:53 (35764): No heartbeat from core client for 30 sec - exiting
04:32:54 (35764): No heartbeat from core client for 30 sec - exiting
04:32:55 (35764): No heartbeat from core client for 30 sec - exiting
04:32:56 (35764): No heartbeat from core client for 30 sec - exiting
04:32:57 (35764): No heartbeat from core client for 30 sec - exiting
04:32:58 (35764): No heartbeat from core client for 30 sec - exiting
04:32:59 (35764): No heartbeat from core client for 30 sec - exiting
04:33:00 (35764): No heartbeat from core client for 30 sec - exiting
05:00:31 (39742): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:11:34 (40302): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:11:35 (40302): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:42:42 (47067): No heartbeat from core client for 30 sec - exiting
10:42:44 (47067): No heartbeat from core client for 30 sec - exiting
10:42:45 (47067): No heartbeat from core client for 30 sec - exiting
10:42:46 (47067): No heartbeat from core client for 30 sec - exiting
10:42:47 (47067): No heartbeat from core client for 30 sec - exiting
10:42:48 (47067): No heartbeat from core client for 30 sec - exiting
10:42:49 (47067): No heartbeat from core client for 30 sec - exiting
10:42:51 (47067): No heartbeat from core client for 30 sec - exiting
10:42:52 (47067): No heartbeat from core client for 30 sec - exiting
10:42:53 (47067): No heartbeat from core client for 30 sec - exiting
10:42:54 (47067): No heartbeat from core client for 30 sec - exiting
10:42:55 (47067): No heartbeat from core client for 30 sec - exiting
10:42:56 (47067): No heartbeat from core client for 30 sec - exiting
10:42:57 (47067): No heartbeat from core client for 30 sec - exiting
10:42:58 (47067): No heartbeat from core client for 30 sec - exiting
10:42:59 (47067): No heartbeat from core client for 30 sec - exiting
10:43:00 (47067): No heartbeat from core client for 30 sec - exiting
10:43:01 (47067): No heartbeat from core client for 30 sec - exiting
10:43:02 (47067): No heartbeat from core client for 30 sec - exiting
10:43:03 (47067): No heartbeat from core client for 30 sec - exiting
10:43:04 (47067): No heartbeat from core client for 30 sec - exiting
10:43:05 (47067): No heartbeat from core client for 30 sec - exiting
10:43:06 (47067): No heartbeat from core client for 30 sec - exiting
10:43:07 (47067): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
12:45:36 (49261): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
16:57:48 (50933): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x4035404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x4035400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x2809e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(54474,0xa0aa3540) malloc: *** error for object 0x2809e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
20:23:41 (54474): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o5v8_1900_40_007202935/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003E1BF7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000D93BE  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0015826A  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00163AB7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0038CE7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0038A1ED  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000048E1  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000027FB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00002729  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58815, iMonCtr=1
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o5v8_1900_40_007202935/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003E1BF7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003DF4CC  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00236007  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0025C80A  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0025BCF5  Unknown               Unknown  UnknownCPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:44:11 (73579): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:05:35 (88921): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
00:19:29 (91538): No heartbeat from core client for 30 sec - exiting
00:19:31 (91538): No heartbeat from core client for 30 sec - exiting
00:19:32 (91538): No heartbeat from core client for 30 sec - exiting
00:19:33 (91538): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:21:24 (91559): No heartbeat from core client for 30 sec - exiting
00:21:27 (91559): No heartbeat from core client for 30 sec - exiting
00:21:28 (91559): No heartbeat from core client for 30 sec - exiting
00:21:29 (91559): No heartbeat from core client for 30 sec - exiting
00:21:30 (91559): No heartbeat from core client for 30 sec - exiting
00:21:31 (91559): No heartbeat from core client for 30 sec - exiting
00:21:32 (91559): No heartbeat from core client for 30 sec - exiting
00:21:33 (91559): No heartbeat from core client for 30 sec - exiting
00:21:34 (91559): No heartbeat from core client for 30 sec - exiting
00:21:35 (91559): No heartbeat from core client for 30 sec - exiting
00:21:36 (91559): No heartbeat from core client for 30 sec - exiting
00:21:37 (91559): No heartbeat from core client for 30 sec - exiting
00:21:38 (91559): No heartbeat from core client for 30 sec - exiting
00:21:39 (91559): No heartbeat from core client for 30 sec - exiting
00:21:40 (91559): No heartbeat from core client for 30 sec - exiting
00:21:41 (91559): No heartbeat from core client for 30 sec - exiting
00:21:42 (91559): No heartbeat from core client for 30 sec - exiting
00:21:43 (91559): No heartbeat from core client for 30 sec - exiting
00:21:44 (91559): No heartbeat from core client for 30 sec - exiting
00:21:45 (91559): No heartbeat from core client for 30 sec - exiting
00:21:46 (91559): No heartbeat from core client for 30 sec - exiting
00:21:47 (91559): No heartbeat from core client for 30 sec - exiting
00:21:48 (91559): No heartbeat from core client for 30 sec - exiting
00:21:49 (91559): No heartbeat from core client for 30 sec - exiting
00:21:50 (91559): No heartbeat from core client for 30 sec - exiting
00:21:51 (91559): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:24:36 (91585): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:26:16 (91635): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:27:41 (91669): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:28:45 (91704): No heartbeat from core client for 30 sec - exiting
00:28:46 (91704): No heartbeat from core client for 30 sec - exiting
00:28:47 (91704): No heartbeat from core client for 30 sec - exiting
00:28:48 (91704): No heartbeat from core client for 30 sec - exiting
00:28:49 (91704): No heartbeat from core client for 30 sec - exiting
00:28:50 (91704): No heartbeat from core client for 30 sec - exiting
00:28:51 (91704): No heartbeat from core client for 30 sec - exiting
00:28:52 (91704): No heartbeat from core client for 30 sec - exiting
00:28:53 (91704): No heartbeat from core client for 30 sec - exiting
00:28:54 (91704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:30:51 (91723): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:32:51 (91763): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:34:36 (733): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
10:40:51 (871): No heartbeat from core client for 30 sec - exiting
10:40:53 (871): No heartbeat from core client for 30 sec - exiting
10:40:54 (871): No heartbeat from core client for 30 sec - exiting
10:40:55 (871): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
19:01:57 (11000): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
00:42:50 (17799): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
09:17:51 (28106): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
09:41:16 (28770): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:32:18 (37350): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:58:21 (37929): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:40:40 (37950): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:17:47 (44413): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
05:00:26 (46386): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
07:03:07 (52440): No heartbeat from core client for 30 sec - exiting
07:03:09 (52440): No heartbeat from core client for 30 sec - exiting
07:03:10 (52440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:10:09 (52463): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:10:10 (52463): No heartbeat from core client for 30 sec - exiting
07:10:11 (52463): No heartbeat from core client for 30 sec - exiting
07:31:41 (52565): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:06:39 (53337): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:28:42 (61922): No heartbeat from core client for 30 sec - exiting
18:28:44 (61922): No heartbeat from core client for 30 sec - exiting
18:28:45 (61922): No heartbeat from core client for 30 sec - exiting
18:28:46 (61922): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:34:45 (61940): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:34:47 (61940): No heartbeat from core client for 30 sec - exiting
22:36:02 (64454): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:39:11 (65290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:13:05 (31864): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
06:38:46 (35155): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:38:47 (35155): No heartbeat from core client for 30 sec - exiting
06:38:48 (35155): No heartbeat from core client for 30 sec - exiting
06:38:49 (35155): No heartbeat from core client for 30 sec - exiting
06:38:50 (35155): No heartbeat from core client for 30 sec - exiting
06:38:51 (35155): No heartbeat from core client for 30 sec - exiting
06:38:52 (35155): No heartbeat from core client for 30 sec - exiting
06:38:53 (35155): No heartbeat from core client for 30 sec - exiting
06:38:54 (35155): No heartbeat from core client for 30 sec - exiting
06:38:55 (35155): No heartbeat from core client for 30 sec - exiting
06:38:56 (35155): No heartbeat from core client for 30 sec - exiting
06:38:57 (35155): No heartbeat from core client for 30 sec - exiting
06:38:58 (35155): No heartbeat from core client for 30 sec - exiting
06:38:59 (35155): No heartbeat from core client for 30 sec - exiting
06:39:00 (35155): No heartbeat from core client for 30 sec - exiting
06:39:01 (35155): No heartbeat from core client for 30 sec - exiting
06:39:02 (35155): No heartbeat from core client for 30 sec - exiting
06:39:03 (35155): No heartbeat from core client for 30 sec - exiting
06:39:04 (35155): No heartbeat from core client for 30 sec - exiting
06:39:05 (35155): No heartbeat from core client for 30 sec - exiting
06:39:06 (35155): No heartbeat from core client for 30 sec - exiting
06:39:07 (35155): No heartbeat from core client for 30 sec - exiting
06:39:08 (35155): No heartbeat from core client for 30 sec - exiting
06:39:09 (35155): No heartbeat from core client for 30 sec - exiting
06:39:10 (35155): No heartbeat from core client for 30 sec - exiting
06:39:11 (35155): No heartbeat from core client for 30 sec - exiting
06:39:12 (35155): No heartbeat from core client for 30 sec - exiting
06:39:13 (35155): No heartbeat from core client for 30 sec - exiting
06:39:14 (35155): No heartbeat from core client for 30 sec - exiting
06:39:15 (35155): No heartbeat from core client for 30 sec - exiting
06:39:16 (35155): No heartbeat from core client for 30 sec - exiting
06:39:17 (35155): No heartbeat from core client for 30 sec - exiting
06:39:18 (35155): No heartbeat from core client for 30 sec - exiting
06:39:19 (35155): No heartbeat from core client for 30 sec - exiting
06:39:20 (35155): No heartbeat from core client for 30 sec - exiting
06:39:21 (35155): No heartbeat from core client for 30 sec - exiting
06:39:22 (35155): No heartbeat from core client for 30 sec - exiting
06:39:23 (35155): No heartbeat from core client for 30 sec - exiting
06:39:24 (35155): No heartbeat from core client for 30 sec - exiting
06:39:25 (35155): No heartbeat from core client for 30 sec - exiting
06:39:26 (35155): No heartbeat from core client for 30 sec - exiting
06:39:27 (35155): No heartbeat from core client for 30 sec - exiting
06:39:28 (35155): No heartbeat from core client for 30 sec - exiting
06:39:29 (35155): No heartbeat from core client for 30 sec - exiting
06:39:30 (35155): No heartbeat from core client for 30 sec - exiting
06:39:31 (35155): No heartbeat from core client for 30 sec - exiting
06:39:33 (35155): No heartbeat from core client for 30 sec - exiting
06:39:34 (35155): No heartbeat from core client for 30 sec - exiting
06:39:35 (35155): No heartbeat from core client for 30 sec - exiting
06:39:36 (35155): No heartbeat from core client for 30 sec - exiting
06:39:37 (35155): No heartbeat from core client for 30 sec - exiting
06:39:38 (35155): No heartbeat from core client for 30 sec - exiting
06:39:39 (35155): No heartbeat from core client for 30 sec - exiting
06:39:40 (35155): No heartbeat from core client for 30 sec - exiting
06:39:41 (35155): No heartbeat from core client for 30 sec - exiting
06:39:42 (35155): No heartbeat from core client for 30 sec - exiting
06:39:43 (35155): No heartbeat from core client for 30 sec - exiting
06:39:44 (35155): No heartbeat from core client for 30 sec - exiting
06:39:45 (35155): No heartbeat from core client for 30 sec - exiting
06:39:46 (35155): No heartbeat from core client for 30 sec - exiting
06:39:47 (35155): No heartbeat from core client for 30 sec - exiting
06:39:48 (35155): No heartbeat from core client for 30 sec - exiting
06:39:49 (35155): No heartbeat from core client for 30 sec - exiting
06:39:50 (35155): No heartbeat from core client for 30 sec - exiting
06:39:51 (35155): No heartbeat from core client for 30 sec - exiting
06:39:52 (35155): No heartbeat from core client for 30 sec - exiting
06:39:53 (35155): No heartbeat from core client for 30 sec - exiting
06:39:54 (35155): No heartbeat from core client for 30 sec - exiting
06:39:55 (35155): No heartbeat from core client for 30 sec - exiting
06:39:56 (35155): No heartbeat from core client for 30 sec - exiting
06:39:57 (35155): No heartbeat from core client for 30 sec - exiting
06:39:58 (35155): No heartbeat from core client for 30 sec - exiting
06:39:59 (35155): No heartbeat from core client for 30 sec - exiting
06:40:00 (35155): No heartbeat from core client for 30 sec - exiting
06:40:01 (35155): No heartbeat from core client for 30 sec - exiting
06:40:02 (35155): No heartbeat from core client for 30 sec - exiting
06:40:03 (35155): No heartbeat from core client for 30 sec - exiting
06:40:04 (35155): No heartbeat from core client for 30 sec - exiting
06:40:05 (35155): No heartbeat from core client for 30 sec - exiting
06:40:06 (35155): No heartbeat from core client for 30 sec - exiting
06:40:07 (35155): No heartbeat from core client for 30 sec - exiting
06:40:08 (35155): No heartbeat from core client for 30 sec - exiting
06:40:09 (35155): No heartbeat from core client for 30 sec - exiting
06:40:10 (35155): No heartbeat from core client for 30 sec - exiting
06:40:11 (35155): No heartbeat from core client for 30 sec - exiting
06:40:12 (35155): No heartbeat from core client for 30 sec - exiting
06:40:13 (35155): No heartbeat from core client for 30 sec - exiting
06:40:14 (35155): No heartbeat from core client for 30 sec - exiting
06:40:15 (35155): No heartbeat from core client for 30 sec - exiting
06:40:16 (35155): No heartbeat from core client for 30 sec - exiting
06:40:17 (35155): No heartbeat from core client for 30 sec - exiting
06:40:18 (35155): No heartbeat from core client for 30 sec - exiting
06:40:19 (35155): No heartbeat from core client for 30 sec - exiting
06:40:20 (35155): No heartbeat from core client for 30 sec - exiting
06:40:21 (35155): No heartbeat from core client for 30 sec - exiting
06:40:22 (35155): No heartbeat from core client for 30 sec - exiting
06:40:23 (35155): No heartbeat from core client for 30 sec - exiting
06:40:24 (35155): No heartbeat from core client for 30 sec - exiting
06:40:25 (35155): No heartbeat from core client for 30 sec - exiting
06:40:26 (35155): No heartbeat from core client for 30 sec - exiting
06:40:27 (35155): No heartbeat from core client for 30 sec - exiting
06:40:28 (35155): No heartbeat from core client for 30 sec - exiting
06:40:29 (35155): No heartbeat from core client for 30 sec - exiting
06:40:30 (35155): No heartbeat from core client for 30 sec - exiting
06:40:31 (35155): No heartbeat from core client for 30 sec - exiting
06:40:32 (35155): No heartbeat from core client for 30 sec - exiting
06:40:33 (35155): No heartbeat from core client for 30 sec - exiting
06:40:34 (35155): No heartbeat from core client for 30 sec - exiting
06:40:36 (35155): No heartbeat from core client for 30 sec - exiting
06:40:37 (35155): No heartbeat from core client for 30 sec - exiting
06:40:38 (35155): No heartbeat from core client for 30 sec - exiting
06:40:39 (35155): No heartbeat from core client for 30 sec - exiting
06:40:40 (35155): No heartbeat from core client for 30 sec - exiting
06:40:41 (35155): No heartbeat from core client for 30 sec - exiting
06:40:42 (35155): No heartbeat from core client for 30 sec - exiting
06:40:43 (35155): No heartbeat from core client for 30 sec - exiting
06:40:44 (35155): No heartbeat from core client for 30 sec - exiting
06:40:45 (35155): No heartbeat from core client for 30 sec - exiting
06:40:46 (35155): No heartbeat from core client for 30 sec - exiting
06:40:47 (35155): No heartbeat from core client for 30 sec - exiting
06:40:48 (35155): No heartbeat from core client for 30 sec - exiting
06:40:49 (35155): No heartbeat from core client for 30 sec - exiting
06:40:50 (35155): No heartbeat from core client for 30 sec - exiting
06:40:51 (35155): No heartbeat from core client for 30 sec - exiting
06:40:52 (35155): No heartbeat from core client for 30 sec - exiting
06:40:53 (35155): No heartbeat from core client for 30 sec - exiting
06:40:54 (35155): No heartbeat from core client for 30 sec - exiting
06:40:55 (35155): No heartbeat from core client for 30 sec - exiting
06:40:56 (35155): No heartbeat from core client for 30 sec - exiting
06:40:57 (35155): No heartbeat from core client for 30 sec - exiting
06:40:58 (35155): No heartbeat from core client for 30 sec - exiting
06:40:59 (35155): No heartbeat from core client for 30 sec - exiting
06:41:00 (35155): No heartbeat from core client for 30 sec - exiting
06:41:01 (35155): No heartbeat from core client for 30 sec - exiting
06:41:02 (35155): No heartbeat from core client for 30 sec - exiting
06:41:03 (35155): No heartbeat from core client for 30 sec - exiting
06:41:04 (35155): No heartbeat from core client for 30 sec - exiting
06:41:05 (35155): No heartbeat from core client for 30 sec - exiting
06:41:06 (35155): No heartbeat from core client for 30 sec - exiting
06:41:07 (35155): No heartbeat from core client for 30 sec - exiting
06:41:08 (35155): No heartbeat from core client for 30 sec - exiting
06:41:09 (35155): No heartbeat from core client for 30 sec - exiting
06:41:10 (35155): No heartbeat from core client for 30 sec - exiting
06:41:11 (35155): No heartbeat from core client for 30 sec - exiting
06:41:12 (35155): No heartbeat from core client for 30 sec - exiting
06:41:13 (35155): No heartbeat from core client for 30 sec - exiting
06:41:14 (35155): No heartbeat from core client for 30 sec - exiting
06:41:15 (35155): No heartbeat from core client for 30 sec - exiting
06:41:16 (35155): No heartbeat from core client for 30 sec - exiting
06:41:17 (35155): No heartbeat from core client for 30 sec - exiting
06:41:18 (35155): No heartbeat from core client for 30 sec - exiting
06:41:19 (35155): No heartbeat from core client for 30 sec - exiting
06:41:20 (35155): No heartbeat from core client for 30 sec - exiting
06:41:21 (35155): No heartbeat from core client for 30 sec - exiting
06:41:22 (35155): No heartbeat from core client for 30 sec - exiting
06:41:23 (35155): No heartbeat from core client for 30 sec - exiting
06:41:24 (35155): No heartbeat from core client for 30 sec - exiting
06:41:25 (35155): No heartbeat from core client for 30 sec - exiting
06:41:26 (35155): No heartbeat from core client for 30 sec - exiting
06:41:27 (35155): No heartbeat from core client for 30 sec - exiting
06:41:28 (35155): No heartbeat from core client for 30 sec - exiting
06:41:29 (35155): No heartbeat from core client for 30 sec - exiting
06:41:30 (35155): No heartbeat from core client for 30 sec - exiting
06:41:31 (35155): No heartbeat from core client for 30 sec - exiting
06:41:32 (35155): No heartbeat from core client for 30 sec - exiting
06:41:33 (35155): No heartbeat from core client for 30 sec - exiting
06:41:34 (35155): No heartbeat from core client for 30 sec - exiting
06:41:35 (35155): No heartbeat from core client for 30 sec - exiting
06:41:36 (35155): No heartbeat from core client for 30 sec - exiting
06:41:37 (35155): No heartbeat from core client for 30 sec - exiting
06:41:38 (35155): No heartbeat from core client for 30 sec - exiting
06:41:39 (35155): No heartbeat from core client for 30 sec - exiting
06:41:40 (35155): No heartbeat from core client for 30 sec - exiting
06:41:42 (35155): No heartbeat from core client for 30 sec - exiting
06:41:43 (35155): No heartbeat from core client for 30 sec - exiting
06:41:44 (35155): No heartbeat from core client for 30 sec - exiting
06:41:45 (35155): No heartbeat from core client for 30 sec - exiting
06:41:46 (35155): No heartbeat from core client for 30 sec - exiting
06:41:47 (35155): No heartbeat from core client for 30 sec - exiting
06:41:48 (35155): No heartbeat from core client for 30 sec - exiting
06:41:49 (35155): No heartbeat from core client for 30 sec - exiting
06:41:50 (35155): No heartbeat from core client for 30 sec - exiting
06:41:51 (35155): No heartbeat from core client for 30 sec - exiting
06:41:52 (35155): No heartbeat from core client for 30 sec - exiting
06:41:53 (35155): No heartbeat from core client for 30 sec - exiting
06:41:54 (35155): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:37:21 (53802): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:39:55 (54356): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
02:39:41 (55342): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:39:42 (55342): No heartbeat from core client for 30 sec - exiting
02:39:43 (55342): No heartbeat from core client for 30 sec - exiting
02:39:44 (55342): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 02:43:00 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58511, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 02:43:04 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58511, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 02:43:07 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58511, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 02:43:09 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58511, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 02:43:15 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58511, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 02:43:19 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58511, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Jul 2011 17:26:31 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 440,640 687,244 1.5596
25 Jul 2011 16:07:11 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 414,720 643,968 1.5528
25 Jul 2011 15:31:56 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 388,800 603,191 1.5514
25 Jul 2011 13:28:54 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 362,880 566,341 1.5607
25 Jul 2011 13:28:54 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 336,960 523,572 1.5538
25 Jul 2011 13:28:54 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 311,040 482,535 1.5514
25 Jul 2011 13:28:54 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 285,120 442,490 1.5519
25 Jul 2011 13:28:54 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 259,200 404,035 1.5588
10 Jul 2011 18:52:28 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 233,280 366,701 1.5719
09 Jul 2011 17:54:31 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 207,360 325,572 1.5701
08 Jul 2011 17:41:30 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 181,440 283,414 1.5620
07 Jul 2011 15:40:16 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 155,520 242,662 1.5603
07 Jul 2011 15:40:16 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 129,600 201,069 1.5515
05 Jul 2011 17:26:23 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 103,680 160,432 1.5474
04 Jul 2011 22:38:11 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 77,760 120,956 1.5555
30 Jun 2011 23:34:29 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 51,840 80,943 1.5614
29 Jun 2011 21:50:20 942621 13024186 hadcm3n_o5v8_1900_40_007202935_2 25,920 40,372 1.5576


©2024 climateprediction.net