Task 16026736

Name	hadcm3n_85qg_1980_40_008465388_0
Workunit	8616227
Created	19 Sep 2013, 15:00:12 UTC
Sent	19 Sep 2013, 18:01:14 UTC
Report deadline	20 Dec 2013, 1:28:25 UTC
Received	25 Sep 2013, 14:28:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1266912
Run time	1 days 18 hours 43 min 25 sec
CPU time	1 days 15 hours 48 min 44 sec
Validate state	Invalid
Credit	933.12
Device peak FLOPS	2.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.43</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> 22:54:11 (15836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:54:19 (15836): No heartbeat from core client for 30 sec - exiting 22:57:59 (16061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:58:00 (16061): No heartbeat from core client for 30 sec - exiting 22:58:01 (16061): No heartbeat from core client for 30 sec - exiting 22:58:02 (16061): No heartbeat from core client for 30 sec - exiting 22:58:03 (16061): No heartbeat from core client for 30 sec - exiting 23:01:43 (16229): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:26 (16377): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:27 (16377): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 23:09:30 (16544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:09:31 (16544): No heartbeat from core client for 30 sec - exiting 23:09:32 (16544): No heartbeat from core client for 30 sec - exiting 23:09:33 (16544): No heartbeat from core client for 30 sec - exiting 23:09:34 (16544): No heartbeat from core client for 30 sec - exiting 23:09:35 (16544): No heartbeat from core client for 30 sec - exiting 23:09:36 (16544): No heartbeat from core client for 30 sec - exiting 23:09:37 (16544): No heartbeat from core client for 30 sec - exiting 23:09:38 (16544): No heartbeat from core client for 30 sec - exiting 23:09:39 (16544): No heartbeat from core client for 30 sec - exiting 23:09:40 (16544): No heartbeat from core client for 30 sec - exiting 23:09:41 (16544): No heartbeat from core client for 30 sec - exiting 23:09:42 (16544): No heartbeat from core client for 30 sec - exiting 23:09:43 (16544): No heartbeat from core client for 30 sec - exiting 23:09:44 (16544): No heartbeat from core client for 30 sec - exiting 23:09:45 (16544): No heartbeat from core client for 30 sec - exiting 23:09:46 (16544): No heartbeat from core client for 30 sec - exiting 23:18:24 (16669): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:18:25 (16669): No heartbeat from core client for 30 sec - exiting 23:18:26 (16669): No heartbeat from core client for 30 sec - exiting 23:21:45 (17046): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:56 (17166): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:03 (17282): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:21 (17686): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:41 (18968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:42 (18968): No heartbeat from core client for 30 sec - exiting 00:07:43 (18968): No heartbeat from core client for 30 sec - exiting 00:07:44 (18968): No heartbeat from core client for 30 sec - exiting 00:07:45 (18968): No heartbeat from core client for 30 sec - exiting 00:07:46 (18968): No heartbeat from core client for 30 sec - exiting 00:07:47 (18968): No heartbeat from core client for 30 sec - exiting 00:07:48 (18968): No heartbeat from core client for 30 sec - exiting 00:07:49 (18968): No heartbeat from core client for 30 sec - exiting 00:10:05 (19651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:13:07 (19717): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:17:08 (19853): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:20:15 (20040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:35 (20139): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:44 (20139): No heartbeat from core client for 30 sec - exiting 00:22:45 (20139): No heartbeat from core client for 30 sec - exiting 00:22:46 (20139): No heartbeat from core client for 30 sec - exiting 00:24:57 (20182): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:25:05 (20182): No heartbeat from core client for 30 sec - exiting 00:25:06 (20182): No heartbeat from core client for 30 sec - exiting 00:25:07 (20182): No heartbeat from core client for 30 sec - exiting 00:36:52 (20248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:39:58 (21125): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:42:50 (21179): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:42:51 (21179): No heartbeat from core client for 30 sec - exiting 00:42:52 (21179): No heartbeat from core client for 30 sec - exiting 00:42:53 (21179): No heartbeat from core client for 30 sec - exiting 00:45:09 (21332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:05:12 (21451): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:08:27 (22212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:10:48 (22371): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:15:01 (22450): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:17:48 (22629): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:46 (22686): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:27:36 (22916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:27:37 (22916): No heartbeat from core client for 30 sec - exiting 01:27:38 (22916): No heartbeat from core client for 30 sec - exiting 01:27:39 (22916): No heartbeat from core client for 30 sec - exiting 01:27:40 (22916): No heartbeat from core client for 30 sec - exiting 01:27:41 (22916): No heartbeat from core client for 30 sec - exiting 01:27:42 (22916): No heartbeat from core client for 30 sec - exiting 01:27:43 (22916): No heartbeat from core client for 30 sec - exiting 01:27:44 (22916): No heartbeat from core client for 30 sec - exiting 01:27:45 (22916): No heartbeat from core client for 30 sec - exiting 01:27:46 (22916): No heartbeat from core client for 30 sec - exiting 01:27:47 (22916): No heartbeat from core client for 30 sec - exiting 01:27:48 (22916): No heartbeat from core client for 30 sec - exiting 01:27:49 (22916): No heartbeat from core client for 30 sec - exiting 01:27:50 (22916): No heartbeat from core client for 30 sec - exiting 01:27:51 (22916): No heartbeat from core client for 30 sec - exiting 01:27:52 (22916): No heartbeat from core client for 30 sec - exiting 01:27:53 (22916): No heartbeat from core client for 30 sec - exiting 01:27:54 (22916): No heartbeat from core client for 30 sec - exiting 01:27:55 (22916): No heartbeat from core client for 30 sec - exiting 01:27:56 (22916): No heartbeat from core client for 30 sec - exiting 01:30:02 (22969): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:30:18 (22969): No heartbeat from core client for 30 sec - exiting 01:30:19 (22969): No heartbeat from core client for 30 sec - exiting 01:32:25 (23083): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:32:44 (23083): No heartbeat from core client for 30 sec - exiting 01:32:45 (23083): No heartbeat from core client for 30 sec - exiting 01:34:41 (23122): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:44:20 (6843): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:16:44 (7751): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:46:11 (17147): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:18 (17147): No heartbeat from core client for 30 sec - exiting 12:46:19 (17147): No heartbeat from core client for 30 sec - exiting 13:44:11 (17693): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:29 (19911): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 14:39:03 (21977): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:08:27 (8699): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:16:29 (11138): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:10 (11358): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:11 (11358): No heartbeat from core client for 30 sec - exiting 00:38:12 (11358): No heartbeat from core client for 30 sec - exiting 00:38:13 (11358): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:12:30 (19873): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:11:13 (20238): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:53:41 (31220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:53:42 (31220): No heartbeat from core client for 30 sec - exiting 12:53:43 (31220): No heartbeat from core client for 30 sec - exiting 12:53:44 (31220): No heartbeat from core client for 30 sec - exiting 12:53:45 (31220): No heartbeat from core client for 30 sec - exiting 12:53:46 (31220): No heartbeat from core client for 30 sec - exiting 12:53:47 (31220): No heartbeat from core client for 30 sec - exiting 12:53:48 (31220): No heartbeat from core client for 30 sec - exiting 12:53:49 (31220): No heartbeat from core client for 30 sec - exiting 12:53:50 (31220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:59:06 (14431): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:59:14 (14810): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:59:15 (14810): No heartbeat from core client for 30 sec - exiting 21:04:37 (17300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:04:38 (17300): No heartbeat from core client for 30 sec - exiting 21:49:04 (17556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:10:56 (30872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:12 (30872): No heartbeat from core client for 30 sec - exiting 02:11:13 (30872): No heartbeat from core client for 30 sec - exiting 02:46:58 (31028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:47:00 (31028): No heartbeat from core client for 30 sec - exiting 02:47:01 (31028): No heartbeat from core client for 30 sec - exiting 02:49:58 (32446): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:52:47 (32551): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:52:48 (32551): No heartbeat from core client for 30 sec - exiting 02:52:49 (32551): No heartbeat from core client for 30 sec - exiting 02:52:50 (32551): No heartbeat from core client for 30 sec - exiting 02:52:51 (32551): No heartbeat from core client for 30 sec - exiting 02:52:52 (32551): No heartbeat from core client for 30 sec - exiting 02:52:53 (32551): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 06:11:06 (7289): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:13:20 (8357): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:01:52 (8400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:01:56 (8400): No heartbeat from core client for 30 sec - exiting 07:01:57 (8400): No heartbeat from core client for 30 sec - exiting 07:01:58 (8400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:55:06 (19554): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 11:14:32 (19677): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:24:40 (28042): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:14 (28209): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:15 (28209): No heartbeat from core client for 30 sec - exiting 15:25:08 (28644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:25:14 (5519): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:29 (7971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:42:12 (17800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:59:17 (18197): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:09:50 (18839): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:15:08 (20171): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:30 (20407): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:14:45 (20916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:33:33 (23263): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:33:47 (23263): No heartbeat from core client for 30 sec - exiting 01:33:49 (23263): No heartbeat from core client for 30 sec - exiting 01:58:29 (23999): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:58:46 (23999): No heartbeat from core client for 30 sec - exiting 01:58:47 (23999): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 06:41:25 (3230): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:41:26 (3230): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:11:03 (19727): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:38 (19918): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:39 (19918): No heartbeat from core client for 30 sec - exiting 13:16:40 (19918): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:03:11 (1557): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:19 (1557): No heartbeat from core client for 30 sec - exiting 19:03:20 (1557): No heartbeat from core client for 30 sec - exiting 19:41:57 (1823): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:42:00 (1823): No heartbeat from core client for 30 sec - exiting 19:42:01 (1823): No heartbeat from core client for 30 sec - exiting 20:40:54 (3353): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:40:55 (3353): No heartbeat from core client for 30 sec - exiting 20:40:56 (3353): No heartbeat from core client for 30 sec - exiting 20:40:57 (3353): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 00:42:00 (15112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:53:58 (16342): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:29 (16818): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:30:47 (24695): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:15:46 (26569): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:48 (27237): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:35:12 (8309): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:35:28 (8309): No heartbeat from core client for 30 sec - exiting 11:35:29 (8309): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:31:24 (21673): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:25 (21673): No heartbeat from core client for 30 sec - exiting 16:31:26 (21673): No heartbeat from core client for 30 sec - exiting 16:31:27 (21673): No heartbeat from core client for 30 sec - exiting 16:31:28 (21673): No heartbeat from core client for 30 sec - exiting 16:31:29 (21673): No heartbeat from core client for 30 sec - exiting 16:31:30 (21673): No heartbeat from core client for 30 sec - exiting 16:31:31 (21673): No heartbeat from core client for 30 sec - exiting 16:31:32 (21673): No heartbeat from core client for 30 sec - exiting 16:31:33 (21673): No heartbeat from core client for 30 sec - exiting 16:31:34 (21673): No heartbeat from core client for 30 sec - exiting 16:31:35 (21673): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7785400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7785430] /lib/libc.so.6(gsignal+0x4f)[0xf75bf31f] /lib/libc.so.6(abort+0x143)[0xf75c0c03] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75aa3d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21850, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf771b400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf771b430] /lib/libc.so.6(gsignal+0x4f)[0xf755531f] /lib/libc.so.6(abort+0x143)[0xf7556c03] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75403d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21850, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7766400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7766430] /lib/libc.so.6(gsignal+0x4f)[0xf75a031f] /lib/libc.so.6(abort+0x143)[0xf75a1c03] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf758b3d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21850, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7766400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7766430] /lib/libc.so.6(gsignal+0x4f)[0xf75a031f] /lib/libc.so.6(abort+0x143)[0xf75a1c03] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf758b3d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21850, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77a1400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf77a1430] /lib/libc.so.6(gsignal+0x4f)[0xf75db31f] /lib/libc.so.6(abort+0x143)[0xf75dcc03] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75c63d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21850, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7766400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7766430] /lib/libc.so.6(gsignal+0x4f)[0xf75a031f] /lib/libc.so.6(abort+0x143)[0xf75a1c03] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf758b3d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21850, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Sep 2013 09:49:40	1266912	16026736	hadcm3n_85qg_1980_40_008465388_0	77,760	137,647	1.7702
23 Sep 2013 11:22:03	1266912	16026736	hadcm3n_85qg_1980_40_008465388_0	51,840	90,677	1.7492
21 Sep 2013 15:47:23	1266912	16026736	hadcm3n_85qg_1980_40_008465388_0	25,920	45,688	1.7627