climateprediction.net home page
Task 13018941

Task 13018941

Name hadcm3n_o6jk_1900_40_007203811_2
Workunit 7402091
Created 28 Jun 2011, 9:22:35 UTC
Sent 28 Jun 2011, 9:22:54 UTC
Report deadline 27 Sep 2011, 16:50:05 UTC
Received 19 Jul 2011, 3:47:30 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 942621
Run time 14 days 11 hours 38 min 18 sec
CPU time 9 days 20 hours 19 min 43 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 3.44 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>6.6.36</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:19:24 (13854): No heartbeat from core client for 30 sec - exiting
21:19:28 (13854): No heartbeat from core client for 30 sec - exiting
21:19:29 (13854): No heartbeat from core client for 30 sec - exiting
21:19:30 (13854): No heartbeat from core client for 30 sec - exiting
21:19:31 (13854):CPDN Monitor - No 'heartbeat' from BOINC...
 No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:28:56 (23585): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
04:32:45 (35763): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:32:47 (35763): No heartbeat from core client for 30 sec - exiting
04:32:48 (35763): No heartbeat from core client for 30 sec - exiting
04:32:49 (35763): No heartbeat from core client for 30 sec - exiting
OPEN:  File Creation Failed: No space left on device
OPEN:  Unable to Open File dataout/o6jkka.dab3c50 for Read/Write

Model crashed: DUMPCTL : Fail to open output dump - may already exist                                                                                                                                                                                                          tmp/pipe_dummy                                                                  2048    
05:00:31 (39744): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:11:34 (40299): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:42:43 (47066): No heartbeat from core client for 30 sec - exiting
10:42:45 (47066): No heartbeat from core client for 30 sec - exiting
10:42:46 (47066): No heartbeat from core client for 30 sec - exiting
10:42:47 (47066): No heartbeat from core client for 30 sec - exiting
10:42:49 (47066): No heartbeat from core client for 30 sec - exiting
10:42:50 (47066): No heartbeat from core client for 30 sec - exiting
10:42:51 (47066): No heartbeat from core client for 30 sec - exiting
10:42:52 (47066): No heartbeat from core client for 30 sec - exiting
10:42:53 (47066): No heartbeat from core client for 30 sec - exiting
10:42:54 (47066): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:42:55 (47066): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
12:45:36 (49260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:45:37 (49260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
16:57:49 (50931): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:23:41 (54479): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o6jk_1900_40_007203811/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003E6199  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0036B2BB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00378344  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00209003  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00160E7A  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00164453  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0038CE7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0038A1ED  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000048E1  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000027FB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00002729  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=58814, iMonCtr=1
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o6jk_1900_40_007203811/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003E1BF7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003DF4CC  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0020BEDF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00160E7A  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00164453  Unknown               UnknoCPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:44:11 (73578): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:05:34 (88919): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
00:19:29 (91537): No heartbeat from core client for 30 sec - exiting
00:19:31 (91537): No heartbeat from core client for 30 sec - exiting
00:19:32 (91537): No heartbeat from core client for 30 sec - exiting
00:19:33 (91537): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:21:26 (91563): No heartbeat from core client for 30 sec - exiting
00:21:28 (91563): No heartbeat from core client for 30 sec - exiting
00:21:29 (91563): No heartbeat from core client for 30 sec - exiting
00:21:30 (91563): No heartbeat from core client for 30 sec - exiting
00:21:31 (91563): No heartbeat from core client for 30 sec - exiting
00:21:32 (91563): No heartbeat from core client for 30 sec - exiting
00:21:33 (91563): No heartbeat from core client for 30 sec - exiting
00:21:34 (91563): No heartbeat from core client for 30 sec - exiting
00:21:35 (91563): No heartbeat from core client for 30 sec - exiting
00:21:36 (91563): No heartbeat from core client for 30 sec - exiting
00:21:37 (91563): No heartbeat from core client for 30 sec - exiting
00:21:38 (91563): No heartbeat from core client for 30 sec - exiting
00:21:39 (91563): No heartbeat from core client for 30 sec - exiting
00:21:40 (91563): No heartbeat from core client for 30 sec - exiting
00:21:41 (91563): No heartbeat from core client for 30 sec - exiting
00:21:42 (91563): No heartbeat from core client for 30 sec - exiting
00:21:43 (91563): No heartbeat from core client for 30 sec - exiting
00:21:44 (91563): No heartbeat from core client for 30 sec - exiting
00:21:45 (91563): No heartbeat from core client for 30 sec - exiting
00:21:46 (91563): No heartbeat from core client for 30 sec - exiting
00:21:47 (91563): No heartbeat from core client for 30 sec - exiting
00:21:48 (91563): No heartbeat from core client for 30 sec - exiting
00:21:49 (91563): No heartbeat from core client for 30 sec - exiting
00:21:50 (91563): No heartbeat from core client for 30 sec - exiting
00:21:51 (91563): No heartbeat from core client for 30 sec - exiting
00:21:52 (91563): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:24:35 (91584): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:26:16 (91636): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:26:18 (91636): No heartbeat from core client for 30 sec - exiting
00:26:19 (91636): No heartbeat from core client for 30 sec - exiting
00:27:41 (91667): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:28:44 (91705): No heartbeat from core client for 30 sec - exiting
00:28:46 (91705): No heartbeat from core client for 30 sec - exiting
00:28:47 (91705): No heartbeat from core client for 30 sec - exiting
00:28:48 (91705): No heartbeat from core client for 30 sec - exiting
00:28:49 (91705): No heartbeat from core client for 30 sec - exiting
00:28:50 (91705): No heartbeat from core client for 30 sec - exiting
00:28:51 (91705): No heartbeat from core client for 30 sec - exiting
00:28:52 (91705): No heartbeat from core client for 30 sec - exiting
00:28:53 (91705): No heartbeat from core client for 30 sec - exiting
00:28:54 (91705): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:30:51 (91728): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:30:53 (91728): No heartbeat from core client for 30 sec - exiting
00:32:51 (91762): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:32:52 (91762): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:34:36 (732): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
10:40:52 (870): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
19:01:57 (10999): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
00:42:50 (17797): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
09:17:51 (28105): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
09:41:16 (28768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:30:57 (29975): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:32:18 (37349): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:58:21 (37927): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:40:40 (37951): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:17:47 (44412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
05:00:26 (46385): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
07:03:07 (52439): No heartbeat from core client for 30 sec - exiting
07:03:09 (52439): No heartbeat from core client for 30 sec - exiting
07:03:10 (52439): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
07:10:08 (52466): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:31:40 (52563): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:06:39 (53335): No heartbeat from core client for 30 sec - exiting
08:06:42 (53335): No heartbeat from core client for 30 sec - exiting
08:06:43 (53335): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:28:42 (61921): No heartbeat from core client for 30 sec - exiting
18:28:44 (61921): No heartbeat from core client for 30 sec - exiting
18:28:45 (61921): No heartbeat from core client for 30 sec - exiting
18:28:46 (61921): No heartbeat from core client for 30 sec - exiting
18:28:47 (61921): No heartbeat from core client for 30 sec - exiting
18:28:48 (61921): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:34:45 (61942): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:34:46 (61942): No heartbeat from core client for 30 sec - exiting
22:36:02 (64453): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:36:03 (64453): No heartbeat from core client for 30 sec - exiting
22:36:04 (64453): No heartbeat from core client for 30 sec - exiting
22:36:05 (64453): No heartbeat from core client for 30 sec - exiting
22:36:06 (64453): No heartbeat from core client for 30 sec - exiting
22:36:07 (64453): No heartbeat from core client for 30 sec - exiting
22:36:08 (64453): No heartbeat from core client for 30 sec - exiting
22:36:09 (64453): No heartbeat from core client for 30 sec - exiting
22:36:10 (64453): No heartbeat from core client for 30 sec - exiting
22:39:11 (65291): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
hadcm3n_6.07_i686-apple-darwin(32057,0xa0aa3540) malloc: *** error for object 0x80ac04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=32067, selfPID=32067, iMonCtr=1
hadcm3n_6.07_i686-apple-darwin(33695,0xa0aa3540) malloc: *** error for object 0x3005c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x3008e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(35154,0xa0aa3540) malloc: *** error for object 0x3008e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
06:38:46 (35154): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:38:47 (35154): No heartbeat from core client for 30 sec - exiting
06:38:48 (35154): No heartbeat from core client for 30 sec - exiting
06:38:49 (35154): No heartbeat from core client for 30 sec - exiting
06:38:50 (35154): No heartbeat from core client for 30 sec - exiting
06:38:51 (35154): No heartbeat from core client for 30 sec - exiting
06:38:52 (35154): No heartbeat from core client for 30 sec - exiting
06:38:53 (35154): No heartbeat from core client for 30 sec - exiting
06:38:54 (35154): No heartbeat from core client for 30 sec - exiting
06:38:55 (35154): No heartbeat from core client for 30 sec - exiting
06:38:56 (35154): No heartbeat from core client for 30 sec - exiting
06:38:57 (35154): No heartbeat from core client for 30 sec - exiting
06:38:58 (35154): No heartbeat from core client for 30 sec - exiting
06:38:59 (35154): No heartbeat from core client for 30 sec - exiting
06:39:00 (35154): No heartbeat from core client for 30 sec - exiting
06:39:01 (35154): No heartbeat from core client for 30 sec - exiting
06:39:02 (35154): No heartbeat from core client for 30 sec - exiting
06:39:03 (35154): No heartbeat from core client for 30 sec - exiting
06:39:04 (35154): No heartbeat from core client for 30 sec - exiting
06:39:05 (35154): No heartbeat from core client for 30 sec - exiting
06:39:06 (35154): No heartbeat from core client for 30 sec - exiting
06:39:07 (35154): No heartbeat from core client for 30 sec - exiting
06:39:08 (35154): No heartbeat from core client for 30 sec - exiting
06:39:09 (35154): No heartbeat from core client for 30 sec - exiting
06:39:10 (35154): No heartbeat from core client for 30 sec - exiting
06:39:11 (35154): No heartbeat from core client for 30 sec - exiting
06:39:12 (35154): No heartbeat from core client for 30 sec - exiting
06:39:13 (35154): No heartbeat from core client for 30 sec - exiting
06:39:14 (35154): No heartbeat from core client for 30 sec - exiting
06:39:15 (35154): No heartbeat from core client for 30 sec - exiting
06:39:16 (35154): No heartbeat from core client for 30 sec - exiting
06:39:17 (35154): No heartbeat from core client for 30 sec - exiting
06:39:18 (35154): No heartbeat from core client for 30 sec - exiting
06:39:19 (35154): No heartbeat from core client for 30 sec - exiting
06:39:20 (35154): No heartbeat from core client for 30 sec - exiting
06:39:21 (35154): No heartbeat from core client for 30 sec - exiting
06:39:22 (35154): No heartbeat from core client for 30 sec - exiting
06:39:23 (35154): No heartbeat from core client for 30 sec - exiting
06:39:24 (35154): No heartbeat from core client for 30 sec - exiting
06:39:25 (35154): No heartbeat from core client for 30 sec - exiting
06:39:26 (35154): No heartbeat from core client for 30 sec - exiting
06:39:27 (35154): No heartbeat from core client for 30 sec - exiting
06:39:28 (35154): No heartbeat from core client for 30 sec - exiting
06:39:29 (35154): No heartbeat from core client for 30 sec - exiting
06:39:30 (35154): No heartbeat from core client for 30 sec - exiting
06:39:31 (35154): No heartbeat from core client for 30 sec - exiting
06:39:32 (35154): No heartbeat from core client for 30 sec - exiting
06:39:33 (35154): No heartbeat from core client for 30 sec - exiting
06:39:34 (35154): No heartbeat from core client for 30 sec - exiting
06:39:36 (35154): No heartbeat from core client for 30 sec - exiting
06:39:37 (35154): No heartbeat from core client for 30 sec - exiting
06:39:38 (35154): No heartbeat from core client for 30 sec - exiting
06:39:39 (35154): No heartbeat from core client for 30 sec - exiting
06:39:40 (35154): No heartbeat from core client for 30 sec - exiting
06:39:41 (35154): No heartbeat from core client for 30 sec - exiting
06:39:42 (35154): No heartbeat from core client for 30 sec - exiting
06:39:43 (35154): No heartbeat from core client for 30 sec - exiting
06:39:44 (35154): No heartbeat from core client for 30 sec - exiting
06:39:45 (35154): No heartbeat from core client for 30 sec - exiting
06:39:46 (35154): No heartbeat from core client for 30 sec - exiting
06:39:47 (35154): No heartbeat from core client for 30 sec - exiting
06:39:48 (35154): No heartbeat from core client for 30 sec - exiting
06:39:49 (35154): No heartbeat from core client for 30 sec - exiting
06:39:50 (35154): No heartbeat from core client for 30 sec - exiting
06:39:51 (35154): No heartbeat from core client for 30 sec - exiting
06:39:52 (35154): No heartbeat from core client for 30 sec - exiting
06:39:53 (35154): No heartbeat from core client for 30 sec - exiting
06:39:54 (35154): No heartbeat from core client for 30 sec - exiting
06:39:55 (35154): No heartbeat from core client for 30 sec - exiting
06:39:56 (35154): No heartbeat from core client for 30 sec - exiting
06:39:57 (35154): No heartbeat from core client for 30 sec - exiting
06:39:58 (35154): No heartbeat from core client for 30 sec - exiting
06:39:59 (35154): No heartbeat from core client for 30 sec - exiting
06:40:00 (35154): No heartbeat from core client for 30 sec - exiting
06:40:01 (35154): No heartbeat from core client for 30 sec - exiting
06:40:02 (35154): No heartbeat from core client for 30 sec - exiting
06:40:03 (35154): No heartbeat from core client for 30 sec - exiting
06:40:04 (35154): No heartbeat from core client for 30 sec - exiting
06:40:05 (35154): No heartbeat from core client for 30 sec - exiting
06:40:06 (35154): No heartbeat from core client for 30 sec - exiting
06:40:07 (35154): No heartbeat from core client for 30 sec - exiting
06:40:08 (35154): No heartbeat from core client for 30 sec - exiting
06:40:09 (35154): No heartbeat from core client for 30 sec - exiting
06:40:10 (35154): No heartbeat from core client for 30 sec - exiting
06:40:11 (35154): No heartbeat from core client for 30 sec - exiting
06:40:12 (35154): No heartbeat from core client for 30 sec - exiting
06:40:13 (35154): No heartbeat from core client for 30 sec - exiting
06:40:14 (35154): No heartbeat from core client for 30 sec - exiting
06:40:15 (35154): No heartbeat from core client for 30 sec - exiting
06:40:16 (35154): No heartbeat from core client for 30 sec - exiting
06:40:17 (35154): No heartbeat from core client for 30 sec - exiting
06:40:18 (35154): No heartbeat from core client for 30 sec - exiting
06:40:19 (35154): No heartbeat from core client for 30 sec - exiting
06:40:20 (35154): No heartbeat from core client for 30 sec - exiting
06:40:21 (35154): No heartbeat from core client for 30 sec - exiting
06:40:22 (35154): No heartbeat from core client for 30 sec - exiting
06:40:23 (35154): No heartbeat from core client for 30 sec - exiting
06:40:24 (35154): No heartbeat from core client for 30 sec - exiting
06:40:25 (35154): No heartbeat from core client for 30 sec - exiting
06:40:26 (35154): No heartbeat from core client for 30 sec - exiting
06:40:27 (35154): No heartbeat from core client for 30 sec - exiting
06:40:28 (35154): No heartbeat from core client for 30 sec - exiting
06:40:29 (35154): No heartbeat from core client for 30 sec - exiting
06:40:30 (35154): No heartbeat from core client for 30 sec - exiting
06:40:31 (35154): No heartbeat from core client for 30 sec - exiting
06:40:32 (35154): No heartbeat from core client for 30 sec - exiting
06:40:33 (35154): No heartbeat from core client for 30 sec - exiting
06:40:34 (35154): No heartbeat from core client for 30 sec - exiting
06:40:35 (35154): No heartbeat from core client for 30 sec - exiting
06:40:36 (35154): No heartbeat from core client for 30 sec - exiting
06:40:37 (35154): No heartbeat from core client for 30 sec - exiting
06:40:38 (35154): No heartbeat from core client for 30 sec - exiting
06:40:39 (35154): No heartbeat from core client for 30 sec - exiting
06:40:40 (35154): No heartbeat from core client for 30 sec - exiting
06:40:41 (35154): No heartbeat from core client for 30 sec - exiting
06:40:42 (35154): No heartbeat from core client for 30 sec - exiting
06:40:43 (35154): No heartbeat from core client for 30 sec - exiting
06:40:44 (35154): No heartbeat from core client for 30 sec - exiting
06:40:45 (35154): No heartbeat from core client for 30 sec - exiting
06:40:46 (35154): No heartbeat from core client for 30 sec - exiting
06:40:47 (35154): No heartbeat from core client for 30 sec - exiting
06:40:48 (35154): No heartbeat from core client for 30 sec - exiting
06:40:49 (35154): No heartbeat from core client for 30 sec - exiting
06:40:50 (35154): No heartbeat from core client for 30 sec - exiting
06:40:51 (35154): No heartbeat from core client for 30 sec - exiting
06:40:53 (35154): No heartbeat from core client for 30 sec - exiting
06:40:54 (35154): No heartbeat from core client for 30 sec - exiting
06:40:55 (35154): No heartbeat from core client for 30 sec - exiting
06:40:56 (35154): No heartbeat from core client for 30 sec - exiting
06:40:57 (35154): No heartbeat from core client for 30 sec - exiting
06:40:58 (35154): No heartbeat from core client for 30 sec - exiting
06:40:59 (35154): No heartbeat from core client for 30 sec - exiting
06:41:00 (35154): No heartbeat from core client for 30 sec - exiting
06:41:01 (35154): No heartbeat from core client for 30 sec - exiting
06:41:02 (35154): No heartbeat from core client for 30 sec - exiting
06:41:03 (35154): No heartbeat from core client for 30 sec - exiting
06:41:04 (35154): No heartbeat from core client for 30 sec - exiting
06:41:05 (35154): No heartbeat from core client for 30 sec - exiting
06:41:06 (35154): No heartbeat from core client for 30 sec - exiting
06:41:07 (35154): No heartbeat from core client for 30 sec - exiting
06:41:08 (35154): No heartbeat from core client for 30 sec - exiting
06:41:09 (35154): No heartbeat from core client for 30 sec - exiting
06:41:10 (35154): No heartbeat from core client for 30 sec - exiting
06:41:11 (35154): No heartbeat from core client for 30 sec - exiting
06:41:12 (35154): No heartbeat from core client for 30 sec - exiting
06:41:13 (35154): No heartbeat from core client for 30 sec - exiting
06:41:14 (35154): No heartbeat from core client for 30 sec - exiting
06:41:15 (35154): No heartbeat from core client for 30 sec - exiting
06:41:16 (35154): No heartbeat from core client for 30 sec - exiting
06:41:17 (35154): No heartbeat from core client for 30 sec - exiting
06:41:18 (35154): No heartbeat from core client for 30 sec - exiting
06:41:19 (35154): No heartbeat from core client for 30 sec - exiting
06:41:20 (35154): No heartbeat from core client for 30 sec - exiting
06:41:21 (35154): No heartbeat from core client for 30 sec - exiting
06:41:22 (35154): No heartbeat from core client for 30 sec - exiting
06:41:23 (35154): No heartbeat from core client for 30 sec - exiting
06:41:24 (35154): No heartbeat from core client for 30 sec - exiting
06:41:25 (35154): No heartbeat from core client for 30 sec - exiting
06:41:26 (35154): No heartbeat from core client for 30 sec - exiting
06:41:27 (35154): No heartbeat from core client for 30 sec - exiting
06:41:28 (35154): No heartbeat from core client for 30 sec - exiting
06:41:29 (35154): No heartbeat from core client for 30 sec - exiting
06:41:30 (35154): No heartbeat from core client for 30 sec - exiting
06:41:31 (35154): No heartbeat from core client for 30 sec - exiting
06:41:32 (35154): No heartbeat from core client for 30 sec - exiting
06:41:33 (35154): No heartbeat from core client for 30 sec - exiting
06:41:34 (35154): No heartbeat from core client for 30 sec - exiting
06:41:35 (35154): No heartbeat from core client for 30 sec - exiting
06:41:36 (35154): No heartbeat from core client for 30 sec - exiting
06:41:37 (35154): No heartbeat from core client for 30 sec - exiting
06:41:38 (35154): No heartbeat from core client for 30 sec - exiting
06:41:39 (35154): No heartbeat from core client for 30 sec - exiting
06:41:40 (35154): No heartbeat from core client for 30 sec - exiting
06:41:41 (35154): No heartbeat from core client for 30 sec - exiting
06:41:42 (35154): No heartbeat from core client for 30 sec - exiting
06:41:43 (35154): No heartbeat from core client for 30 sec - exiting
06:41:44 (35154): No heartbeat from core client for 30 sec - exiting
06:41:45 (35154): No heartbeat from core client for 30 sec - exiting
06:41:46 (35154): No heartbeat from core client for 30 sec - exiting
06:41:47 (35154): No heartbeat from core client for 30 sec - exiting
06:41:48 (35154): No heartbeat from core client for 30 sec - exiting
06:41:49 (35154): No heartbeat from core client for 30 sec - exiting
06:41:50 (35154): No heartbeat from core client for 30 sec - exiting
06:41:51 (35154): No heartbeat from core client for 30 sec - exiting
06:41:52 (35154): No heartbeat from core client for 30 sec - exiting
06:41:53 (35154): No heartbeat from core client for 30 sec - exiting
06:41:54 (35154): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:37:21 (53801): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:39:56 (54357): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
02:39:41 (55341): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:39:42 (55341): No heartbeat from core client for 30 sec - exiting
02:39:43 (55341): No heartbeat from core client for 30 sec - exiting
02:39:44 (55341): No heartbeat from core client for 30 sec - exiting
02:39:45 (55341): No heartbeat from core client for 30 sec - exiting
02:39:46 (55341): No heartbeat from core client for 30 sec - exiting
02:39:47 (55341): No heartbeat from core client for 30 sec - exiting
02:39:48 (55341): No heartbeat from core client for 30 sec - exiting
02:39:49 (55341): No heartbeat from core client for 30 sec - exiting
02:39:50 (55341): No heartbeat from core client for 30 sec - exiting
02:39:51 (55341): No heartbeat from core client for 30 sec - exiting
02:39:52 (55341): No heartbeat from core client for 30 sec - exiting
02:39:53 (55341): No heartbeat from core client for 30 sec - exiting
02:39:54 (55341): No heartbeat from core client for 30 sec - exiting
02:39:55 (55341): No heartbeat from core client for 30 sec - exiting
02:39:56 (55341): No heartbeat from core client for 30 sec - exiting
02:39:57 (55341): No heartbeat from core client for 30 sec - exiting
02:39:58 (55341): No heartbeat from core client for 30 sec - exiting
02:39:59 (55341): No heartbeat from core client for 30 sec - exiting
02:40:00 (55341): No heartbeat from core client for 30 sec - exiting
02:40:01 (55341): No heartbeat from core client for 30 sec - exiting
02:40:02 (55341): No heartbeat from core client for 30 sec - exiting
02:40:03 (55341): No heartbeat from core client for 30 sec - exiting
02:40:04 (55341): No heartbeat from core client for 30 sec - exiting
02:40:05 (55341): No heartbeat from core client for 30 sec - exiting
02:40:06 (55341): No heartbeat from core client for 30 sec - exiting
02:40:07 (55341): No heartbeat from core client for 30 sec - exiting
02:40:08 (55341): No heartbeat from core client for 30 sec - exiting
02:40:09 (55341): No heartbeat from core client for 30 sec - exiting
02:40:10 (55341): No heartbeat from core client for 30 sec - exiting
02:40:11 (55341): No heartbeat from core client for 30 sec - exiting
02:40:12 (55341): No heartbeat from core client for 30 sec - exiting
02:40:13 (55341): No heartbeat from core client for 30 sec - exiting
02:40:14 (55341): No heartbeat from core client for 30 sec - exiting
02:40:15 (55341): No heartbeat from core client for 30 sec - exiting
02:40:16 (55341): No heartbeat from core client for 30 sec - exiting
02:40:17 (55341): No heartbeat from core client for 30 sec - exiting
02:40:18 (55341): No heartbeat from core client for 30 sec - exiting
02:40:19 (55341): No heartbeat from core client for 30 sec - exiting
02:40:20 (55341): No heartbeat from core client for 30 sec - exiting
02:40:21 (55341): No heartbeat from core client for 30 sec - exiting
02:40:22 (55341): No heartbeat from core client for 30 sec - exiting
02:40:23 (55341): No heartbeat from core client for 30 sec - exiting
02:40:24 (55341): No heartbeat from core client for 30 sec - exiting
02:40:25 (55341): No heartbeat from core client for 30 sec - exiting
02:40:26 (55341): No heartbeat from core client for 30 sec - exiting
02:40:27 (55341): No heartbeat from core client for 30 sec - exiting
02:40:28 (55341): No heartbeat from core client for 30 sec - exiting
02:40:29 (55341): No heartbeat from core client for 30 sec - exiting
02:40:30 (55341): No heartbeat from core client for 30 sec - exiting
02:40:31 (55341): No heartbeat from core client for 30 sec - exiting
02:40:32 (55341): No heartbeat from core client for 30 sec - exiting
02:40:33 (55341): No heartbeat from core client for 30 sec - exiting
02:40:34 (55341): No heartbeat from core client for 30 sec - exiting
02:40:35 (55341): No heartbeat from core client for 30 sec - exiting
02:40:36 (55341): No heartbeat from core client for 30 sec - exiting
02:40:37 (55341): No heartbeat from core client for 30 sec - exiting
02:45:28 (58513): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:57:00 (58595): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:07:45 (58788): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:16:00 (58947): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:24:57 (59103): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:39:07 (59271): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:56:04 (59486): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 03:56:47 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59767, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 03:56:50 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59767, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 03:56:52 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59767, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 03:56:55 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59767, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 03:56:58 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59767, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 19 03:57:01 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x981afc5a ___kill + 10
1   libSystem.B.dylib                   0x982425a5 _raise + 26
2   libSystem.B.dylib                   0x982586e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x9814ec0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x981d6429 _nanosleep + 345
2   libSystem.B.dylib                   0x981d62ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x9817c259 __pthread_start + 345
6   libSystem.B.dylib                   0x9817c0de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc9c8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x944f8000 - 0x94506fff /usr/lib/libz.1.dylib
0x95749000 - 0x9574cfff /usr/lib/system/libmathCommon.A.dylib
0x9814e000 - 0x982f5fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59767, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Jul 2011 16:34:30 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 518,400 807,455 1.5576
25 Jul 2011 15:50:41 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 492,480 765,790 1.5550
25 Jul 2011 13:28:55 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 466,560 727,311 1.5589
25 Jul 2011 13:28:54 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 440,640 686,448 1.5578
25 Jul 2011 13:28:54 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 414,720 645,253 1.5559
25 Jul 2011 13:28:54 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 388,800 604,419 1.5546
25 Jul 2011 13:28:54 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 362,880 563,739 1.5535
25 Jul 2011 13:28:54 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 336,960 529,339 1.5709
09 Jul 2011 17:45:06 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 311,040 487,917 1.5687
07 Jul 2011 22:45:12 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 285,120 445,785 1.5635
07 Jul 2011 15:40:16 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 259,200 403,924 1.5583
07 Jul 2011 15:40:16 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 233,280 362,575 1.5542
05 Jul 2011 10:43:56 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 207,360 322,834 1.5569
04 Jul 2011 14:12:21 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 181,440 282,910 1.5592
03 Jul 2011 19:39:06 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 155,520 244,940 1.5750
03 Jul 2011 02:43:13 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 129,600 205,076 1.5824
02 Jul 2011 02:21:53 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 103,680 160,923 1.5521
01 Jul 2011 07:48:18 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 77,760 120,744 1.5528
30 Jun 2011 03:22:33 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 51,840 80,397 1.5509
29 Jun 2011 03:36:52 942621 13018941 hadcm3n_o6jk_1900_40_007203811_2 25,920 40,186 1.5504


©2024 climateprediction.net