climateprediction.net (CPDN) home page
Task 12847007

Task 12847007

Name hadcm3n_o0tt_1900_40_007196404_2
Workunit 7394684
Created 30 Apr 2011, 23:13:40 UTC
Sent 30 Apr 2011, 23:20:10 UTC
Report deadline 31 Jul 2011, 6:47:21 UTC
Received 23 May 2011, 1:20:39 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1129554
Run time 15 days 21 hours 8 min 12 sec
CPU time 8 days 10 hours 32 min 58 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 1.53 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.59</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
04:41:30 (4310): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
20:47:11 (2245): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 22 to Word Address 169984 Failed with Error Code -1

Model crashed: SETPOS: Unit 22 to Word Address 169984 Failed with Error Code -1

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 22 to Word Address 952320 Failed with Error Code -1

Model crashed: SETPOS: Unit 22 to Word Address 952320 Failed with Error Code -1

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
Suspended CPDN Monitor - Suspend request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
05:57:04 (3253): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code 07:55:14 (6687): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0938a670 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6b511)[0xf74dc511]
/lib32/libc.so.6(+0x6ce1b)[0xf74dde1b]
/lib32/libc.so.6(cfree+0x6d)[0xf74e0f8d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76ba841]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76ba89d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf7487e37]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 fc:01 4600117                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 fc:01 4600117                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09334000-093a6000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6ff3000-f746e000 rw-s 00000000 fc:01 4607991                            /var/lib/boinc-client/slots/28/136910
f746e000-f7471000 rw-p 00000000 00:00 0 
f7471000-f75c7000 r-xp 00000000 fc:01 3538996                            /lib32/libc-2.13.so
f75c7000-f75c8000 ---p 00156000 fc:01 3538996                            /lib32/libc-2.13.so
f75c8000-f75ca000 r--p 00156000 fc:01 3538996                            /lib32/libc-2.13.so
f75ca000-f75cb000 rw-p 00158000 fc:01 3538996                            /lib32/libc-2.13.so
f75cb000-f75ce000 rw-p 00000000 00:00 0 
f75ce000-f75e8000 r-xp 00000000 fc:01 4982074                            /usr/lib32/libgcc_s.so.1
f75e8000-f75e9000 r--p 00019000 fc:01 4982074                            /usr/lib32/libgcc_s.so.1
f75e9000-f75ea000 rw-p 0001a000 fc:01 4982074                            /usr/lib32/libgcc_s.so.1
f75ea000-f760e000 r-xp 00000000 fc:01 3539000                            /lib32/libm-2.13.so
f760e000-f760f000 r--p 00023000 fc:01 3539000                            /lib32/libm-2.13.so
f760f000-f7610000 rw-p 00024000 fc:01 3539000                            /lib32/libm-2.13.so
f7610000-f76ef000 r-xp 00000000 fc:01 4982065                            /usr/lib32/libstdc++.so.6.0.14
f76ef000-f76f3000 r--p 000de000 fc:01 4982065                            /usr/lib32/libstdc++.so.6.0.14
f76f3000-f76f4000 rw-p 000e2000 fc:01 4982065                            /usr/lib32/libstdc++.so.6.0.14
f76f4000-f76fb000 rw-p 00000000 00:00 0 
f76fb000-f76fd000 r-xp 00000000 fc:01 3538999                            /lib32/libdl-2.13.so
f76fd000-f76fe000 r--p 00001000 fc:01 3538999                            /lib32/libdl-2.13.so
f76fe000-f76ff000 rw-p 00002000 fc:01 3538999                            /lib32/libdl-2.13.so
f76ff000-f7714000 r-xp 00000000 fc:01 3539010                            /lib32/libpthread-2.13.so
f7714000-f7715000 r--p 00014000 fc:01 3539010                            /lib32/libpthread-2.13.so
f7715000-f7716000 rw-p 00015000 fc:01 3539010                            /lib32/libpthread-2.13.so
f7716000-f7719000 rw-p 00000000 00:00 0 
f7736000-f7737000 rw-p 00000000 00:00 0 
f7737000-f7738000 ---p 00000000 00:00 0 
f7738000-f773b000 rw-p 00000000 00:00 0 
f773b000-f773d000 rw-s 00000000 fc:01 4607911                            /var/lib/boinc-client/slots/28/boinc_mmap_file
f773d000-f773e000 rw-p 00000000 00:00 0 
f773e000-f773f000 r-xp 00000000 00:00 0                                  [vdso]
f773f000-f775b000 r-xp 00000000 fc:01 3538993                            /lib32/ld-2.13.so
f775b000-f775c000 r--p 0001b000 fc:01 3538993                            /lib32/ld-2.13.so
f775c000-f775d000 rw-p 0001c000 fc:01 3538993                            /lib32/ld-2.13.so
ffea6000-fff15000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf773e400]
[0xf773e425]
/lib32/libc.so.6(gsignal+0x51)[0xf749bea1]
/lib32/libc.so.6(abort+0x17e)[0xf749f2ce]
/lib32/libc.so.6(+0x61277)[0xf74d2277]
/lib32/libc.so.6(+0x6b511)[0xf74dc511]
/lib32/libc.so.6(+0x6ce1b)[0xf74dde1b]
/lib32/libc.so.6(cfree+0x6d)[0xf74e0f8d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76ba841]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76ba89d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf7487e37]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 May 2011 01:24:22 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 518,400 729,176 1.4066
22 May 2011 07:42:06 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 492,480 665,677 1.3517
21 May 2011 14:06:32 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 466,560 602,171 1.2907
20 May 2011 20:26:14 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 440,640 538,738 1.2226
20 May 2011 02:53:47 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 414,720 475,447 1.1464
19 May 2011 08:08:19 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 388,800 408,062 1.0495
18 May 2011 12:51:46 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 362,880 338,824 0.9337
17 May 2011 17:13:54 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 336,960 577,962 1.7152
16 May 2011 21:29:14 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 311,040 507,489 1.6316
16 May 2011 01:53:46 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 285,120 437,013 1.5327
15 May 2011 06:29:27 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 259,200 366,550 1.4142
13 May 2011 20:40:05 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 233,280 294,951 1.2644
07 May 2011 13:15:38 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 207,360 225,823 1.0890
07 May 2011 08:17:18 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 181,440 161,437 0.8898
06 May 2011 00:12:10 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 155,520 92,623 0.5956
05 May 2011 05:13:47 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 129,600 85,126 0.6568
04 May 2011 10:23:37 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 103,680 202,894 1.9569
03 May 2011 14:51:26 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 77,760 132,791 1.7077
02 May 2011 19:05:15 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 51,840 77,210 1.4894
01 May 2011 23:09:28 1129554 12847007 hadcm3n_o0tt_1900_40_007196404_2 25,920 69,727 2.6901


©2024 cpdn.org