Task 14669450

Name	hadcm3n_o0es_1980_40_007959451_1
Workunit	8114563
Created	15 May 2012, 11:40:05 UTC
Sent	15 May 2012, 11:40:31 UTC
Report deadline	14 Aug 2012, 19:07:42 UTC
Received	21 Jun 2012, 12:48:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1170024
Run time	21 days 14 hours 0 min 47 sec
CPU time	16 days 22 hours 15 min 33 sec
Validate state	Invalid
Credit	7,153.92
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:12:53 (1585): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:23:21 (1611): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:31:55 (3738): No heartbeat from core client for 30 sec - exiting 23:31:56 (3738): No heartbeat from core client for 30 sec - exiting 23:31:57 (3738): No heartbeat from core client for 30 sec - exiting 23:31:58 (3738): No heartbeat from core client for 30 sec - exiting 23:31:59 (3738): No heartbeat from core client for 30 sec - exiting 23:32:00 (3738): No heartbeat from core client for 30 sec - exiting 23:32:01 (3738): No heartbeat from core client for 30 sec - exiting 23:32:02 (3738): No heartbeat from core client for 30 sec - exiting 23:32:03 (3738): No heartbeat from core client for 30 sec - exiting 23:32:04 (3738): No heartbeat from core client for 30 sec - exiting 23:32:05 (3738): No heartbeat from core client for 30 sec - exiting 23:32:06 (3738): No heartbeat from core client for 30 sec - exiting 23:32:07 (3738): No heartbeat from core client for 30 sec - exiting 23:32:08 (3738): No heartbeat from core client for 30 sec - exiting 23:32:09 (3738): No heartbeat from core client for 30 sec - exiting 23:32:10 (3738): No heartbeat from core client for 30 sec - exiting 23:32:11 (3738): No heartbeat from core client for 30 sec - exiting 23:32:12 (3738): No heartbeat from core client for 30 sec - exiting 23:32:13 (3738): No heartbeat from core client for 30 sec - exiting 23:32:14 (3738): No heartbeat from core client for 30 sec - exiting 23:32:15 (3738): No heartbeat from core client for 30 sec - exiting 23:32:16 (3738): No heartbeat from core client for 30 sec - exiting 23:32:17 (3738): No heartbeat from core client for 30 sec - exiting 23:32:18 (3738): No heartbeat from core client for 30 sec - exiting 23:32:19 (3738): No heartbeat from core client for 30 sec - exiting 23:32:20 (3738): No heartbeat from core client for 30 sec - exiting 23:32:21 (3738): No heartbeat from core client for 30 sec - exiting 23:32:22 (3738): No heartbeat from core client for 30 sec - exiting 23:32:23 (3738): No heartbeat from core client for 30 sec - exiting 23:32:24 (3738): No heartbeat from core client for 30 sec - exiting 23:32:25 (3738): No heartbeat from core client for 30 sec - exiting 23:32:26 (3738): No heartbeat from core client for 30 sec - exiting 23:32:27 (3738): No heartbeat from core client for 30 sec - exiting 23:32:28 (3738): No heartbeat from core client for 30 sec - exiting 23:32:29 (3738): No heartbeat from core client for 30 sec - exiting 23:32:30 (3738): No heartbeat from core client for 30 sec - exiting 23:32:31 (3738): No heartbeat from core client for 30 sec - exiting 23:32:32 (3738): No heartbeat from core client for 30 sec - exiting 23:32:33 (3738): No heartbeat from core client for 30 sec - exiting 23:32:34 (3738): No heartbeat from core client for 30 sec - exiting 23:32:35 (3738): No heartbeat from core client for 30 sec - exiting 23:32:36 (3738): No heartbeat from core client for 30 sec - exiting 23:32:37 (3738): No heartbeat from core client for 30 sec - exiting 23:32:38 (3738): No heartbeat from core client for 30 sec - exiting 23:32:39 (3738): No heartbeat from core client for 30 sec - exiting 23:32:40 (3738): No heartbeat from core client for 30 sec - exiting 23:32:41 (3738): No heartbeat from core client for 30 sec - exiting 23:32:42 (3738): No heartbeat from core client for 30 sec - exiting 23:32:43 (3738): No heartbeat from core client for 30 sec - exiting 23:32:44 (3738): No heartbeat from core client for 30 sec - exiting 23:32:45 (3738): No heartbeat from core client for 30 sec - exiting 23:32:46 (3738): No heartbeat from core client for 30 sec - exiting 23:32:47 (3738): No heartbeat from core client for 30 sec - exiting 23:32:48 (3738): No heartbeat from core client for 30 sec - exiting 23:32:49 (3738): No heartbeat from core client for 30 sec - exiting 23:32:50 (3738): No heartbeat from core client for 30 sec - exiting 23:32:51 (3738): No heartbeat from core client for 30 sec - exiting 23:32:52 (3738): No heartbeat from core client for 30 sec - exiting 23:32:53 (3738): No heartbeat from core client for 30 sec - exiting 23:32:54 (3738): No heartbeat from core client for 30 sec - exiting 23:32:55 (3738): No heartbeat from core client for 30 sec - exiting 23:32:56 (3738): No heartbeat from core client for 30 sec - exiting 23:32:57 (3738): No heartbeat from core client for 30 sec - exiting 23:32:58 (3738): No heartbeat from core client for 30 sec - exiting 23:32:59 (3738): No heartbeat from core client for 30 sec - exiting 23:33:00 (3738): No heartbeat from core client for 30 sec - exiting 23:33:01 (3738): No heartbeat from core client for 30 sec - exiting 23:33:02 (3738): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:06 (3747): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:36:49 (3759): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:42 (3777): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:44:36 (3791): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:52:42 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:52:43 (3800): No heartbeat from core client for 30 sec - exiting 23:52:44 (3800): No heartbeat from core client for 30 sec - exiting 23:52:45 (3800): No heartbeat from core client for 30 sec - exiting 23:52:46 (3800): No heartbeat from core client for 30 sec - exiting 23:52:47 (3800): No heartbeat from core client for 30 sec - exiting 23:52:48 (3800): No heartbeat from core client for 30 sec - exiting 00:37:20 (3825): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:36:15 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:36:16 (3848): No heartbeat from core client for 30 sec - exiting 01:36:17 (3848): No heartbeat from core client for 30 sec - exiting 01:36:18 (3848): No heartbeat from core client for 30 sec - exiting 01:36:19 (3848): No heartbeat from core client for 30 sec - exiting 01:36:20 (3848): No heartbeat from core client for 30 sec - exiting 01:36:21 (3848): No heartbeat from core client for 30 sec - exiting 01:36:22 (3848): No heartbeat from core client for 30 sec - exiting 01:36:23 (3848): No heartbeat from core client for 30 sec - exiting 01:36:24 (3848): No heartbeat from core client for 30 sec - exiting 01:36:25 (3848): No heartbeat from core client for 30 sec - exiting 01:36:26 (3848): No heartbeat from core client for 30 sec - exiting 01:36:27 (3848): No heartbeat from core client for 30 sec - exiting 01:36:28 (3848): No heartbeat from core client for 30 sec - exiting 01:36:29 (3848): No heartbeat from core client for 30 sec - exiting 01:36:30 (3848): No heartbeat from core client for 30 sec - exiting 01:36:31 (3848): No heartbeat from core client for 30 sec - exiting 01:36:32 (3848): No heartbeat from core client for 30 sec - exiting 01:36:33 (3848): No heartbeat from core client for 30 sec - exiting 01:36:34 (3848): No heartbeat from core client for 30 sec - exiting 01:36:35 (3848): No heartbeat from core client for 30 sec - exiting 01:36:36 (3848): No heartbeat from core client for 30 sec - exiting 01:36:37 (3848): No heartbeat from core client for 30 sec - exiting 01:36:38 (3848): No heartbeat from core client for 30 sec - exiting 01:36:39 (3848): No heartbeat from core client for 30 sec - exiting 01:36:40 (3848): No heartbeat from core client for 30 sec - exiting 01:36:41 (3848): No heartbeat from core client for 30 sec - exiting 01:36:42 (3848): No heartbeat from core client for 30 sec - exiting 01:36:43 (3848): No heartbeat from core client for 30 sec - exiting 01:36:44 (3848): No heartbeat from core client for 30 sec - exiting 01:36:45 (3848): No heartbeat from core client for 30 sec - exiting 01:36:46 (3848): No heartbeat from core client for 30 sec - exiting 01:36:47 (3848): No heartbeat from core client for 30 sec - exiting 01:36:48 (3848): No heartbeat from core client for 30 sec - exiting 01:36:49 (3848): No heartbeat from core client for 30 sec - exiting 01:36:50 (3848): No heartbeat from core client for 30 sec - exiting 01:36:51 (3848): No heartbeat from core client for 30 sec - exiting 01:36:52 (3848): No heartbeat from core client for 30 sec - exiting 01:36:53 (3848): No heartbeat from core client for 30 sec - exiting 01:36:54 (3848): No heartbeat from core client for 30 sec - exiting 01:36:55 (3848): No heartbeat from core client for 30 sec - exiting 01:36:56 (3848): No heartbeat from core client for 30 sec - exiting 01:36:57 (3848): No heartbeat from core client for 30 sec - exiting 01:36:58 (3848): No heartbeat from core client for 30 sec - exiting 01:36:59 (3848): No heartbeat from core client for 30 sec - exiting 01:37:00 (3848): No heartbeat from core client for 30 sec - exiting 01:37:01 (3848): No heartbeat from core client for 30 sec - exiting 01:37:02 (3848): No heartbeat from core client for 30 sec - exiting 01:37:03 (3848): No heartbeat from core client for 30 sec - exiting 01:37:04 (3848): No heartbeat from core client for 30 sec - exiting 01:37:05 (3848): No heartbeat from core client for 30 sec - exiting 01:37:06 (3848): No heartbeat from core client for 30 sec - exiting 01:37:07 (3848): No heartbeat from core client for 30 sec - exiting 01:37:08 (3848): No heartbeat from core client for 30 sec - exiting 01:37:09 (3848): No heartbeat from core client for 30 sec - exiting 01:37:10 (3848): No heartbeat from core client for 30 sec - exiting 01:37:11 (3848): No heartbeat from core client for 30 sec - exiting 01:37:12 (3848): No heartbeat from core client for 30 sec - exiting 01:37:13 (3848): No heartbeat from core client for 30 sec - exiting 01:37:14 (3848): No heartbeat from core client for 30 sec - exiting 01:37:15 (3848): No heartbeat from core client for 30 sec - exiting 01:37:16 (3848): No heartbeat from core client for 30 sec - exiting 01:37:17 (3848): No heartbeat from core client for 30 sec - exiting 01:37:18 (3848): No heartbeat from core client for 30 sec - exiting 01:37:19 (3848): No heartbeat from core client for 30 sec - exiting 01:37:20 (3848): No heartbeat from core client for 30 sec - exiting 01:37:21 (3848): No heartbeat from core client for 30 sec - exiting 01:37:22 (3848): No heartbeat from core client for 30 sec - exiting 01:37:23 (3848): No heartbeat from core client for 30 sec - exiting 01:37:24 (3848): No heartbeat from core client for 30 sec - exiting 01:37:25 (3848): No heartbeat from core client for 30 sec - exiting 01:37:26 (3848): No heartbeat from core client for 30 sec - exiting 01:37:27 (3848): No heartbeat from core client for 30 sec - exiting 01:37:28 (3848): No heartbeat from core client for 30 sec - exiting forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o0es_1980_40_007959451/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75C4CA6 Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3901, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o0es_1980_40_007959451/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75F5CA6 Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3901, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o0es_1980_40_007959451/dataout/stdout_um.txt Image PC Routine Line Source Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf76e5400] [0xf76e5425] /lib32/libc.so.6(gsignal+0x51)[0xf7564a71] /lib32/libc.so.6(abort+0x182)[0xf7567e92] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7550ca6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1628, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7764400] [0xf7764425] /lib32/libc.so.6(gsignal+0x51)[0xf75e3a71] /lib32/libc.so.6(abort+0x182)[0xf75e6e92] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75cfca6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1628, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf77c1400] [0xf77c1425] /lib32/libc.so.6(gsignal+0x51)[0xf7640a71] /lib32/libc.so.6(abort+0x182)[0xf7643e92] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf762cca6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1628, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf77d8400] [0xf77d8425] /lib32/libc.so.6(gsignal+0x51)[0xf7657a71] /lib32/libc.so.6(abort+0x182)[0xf765ae92] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7643ca6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1628, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7764400] [0xf7764425] /lib32/libc.so.6(gsignal+0x51)[0xf75e3a71] /lib32/libc.so.6(abort+0x182)[0xf75e6e92] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75cfca6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1628, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf77c4400] [0xf77c4425] /lib32/libc.so.6(gsignal+0x51)[0xf7643a71] /lib32/libc.so.6(abort+0x182)[0xf7646e92] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf762fca6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1628, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Jun 2012 18:15:55	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	596,160	902,980	1.5147
06 Jun 2012 14:49:12	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	570,240	864,132	1.5154
05 Jun 2012 11:25:06	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	544,320	824,961	1.5156
03 Jun 2012 22:40:17	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	518,400	786,204	1.5166
02 Jun 2012 12:07:09	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	492,480	747,271	1.5174
01 Jun 2012 10:19:37	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	466,560	708,293	1.5181
30 May 2012 22:35:26	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	440,640	669,139	1.5186
30 May 2012 10:05:48	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	414,720	629,900	1.5189
28 May 2012 20:28:22	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	388,800	591,016	1.5201
28 May 2012 02:31:43	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	362,880	552,006	1.5212
26 May 2012 10:48:20	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	336,960	513,184	1.5230
25 May 2012 19:52:40	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	311,040	474,437	1.5253
24 May 2012 23:30:37	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	285,120	435,696	1.5281
24 May 2012 07:33:28	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	259,200	396,727	1.5306
22 May 2012 07:27:02	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	233,280	357,503	1.5325
21 May 2012 16:20:39	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	207,360	318,787	1.5374
20 May 2012 01:47:05	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	181,440	280,051	1.5435
19 May 2012 03:58:54	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	155,520	240,117	1.5440
18 May 2012 16:59:45	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	129,600	200,059	1.5437
18 May 2012 05:37:34	1170024	14669450	hadcm3n_o0es_1980_40_007959451_1	103,680	160,035	1.5435