climateprediction.net home page
Task 13564183

Task 13564183

Name hadcm3n_yii5_1900_40_007527096_4
Workunit 7724571
Created 29 Oct 2011, 5:05:06 UTC
Sent 29 Oct 2011, 5:25:37 UTC
Report deadline 28 Jan 2012, 12:52:48 UTC
Received 23 Nov 2011, 17:10:28 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1158471
Run time 13 days 4 hours 7 min 12 sec
CPU time 12 days 17 hours 55 min 57 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.60 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
14:34:07 (16903): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:36:14 (17721): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:38:12 (17753): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:40:45 (17785): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:45:24 (17817): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
14:44:44 (1971): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:45:56 (2036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:49:20 (2064): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:52:12 (2092): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:56:05 (2120): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:59:33 (2148): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:04:00 (2176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:01:07 (5567): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:04:20 (6426): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:06:14 (6454): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:07:51 (6484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:09:29 (6513): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:10:51 (6543): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:12:34 (6571): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:13:51 (6602): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:16:20 (6630): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:17:38 (6661): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:19:10 (6693): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:20:47 (6722): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:22:25 (6751): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:23:52 (6781): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:25:30 (6809): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:26:51 (6840): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:28:34 (6870): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:30:07 (6900): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:31:34 (6929): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:33:11 (6958): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:34:49 (6987): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:36:11 (7016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:37:38 (7045): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:39:11 (7074): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:40:48 (7103): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:42:26 (7132): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:43:53 (7162): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:45:20 (7190): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:46:52 (7219): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:48:10 (7248): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:49:48 (7277): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:51:25 (7306): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED.                                                                                                                                                                                                                     tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:02:27 (7784): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
20:03:38 (7974): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
20:06:07 (8003): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
20:10:01 (8034): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08e23638 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6eb72)[0xf7472b72]
/lib32/libc.so.6(+0x6f812)[0xf7473812]
/lib32/libc.so.6(cfree+0x6d)[0xf74768cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf7673baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf7673c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf741d0f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:01 13763033                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:01 13763033                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
08dce000-08e33000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6f86000-f7401000 rw-s 00000000 08:01 14811345                           /var/lib/boinc-client/slots/14/136605
f7401000-f7404000 rw-p 00000000 00:00 0 
f7404000-f7577000 r-xp 00000000 08:01 20316217                           /lib32/libc-2.13.so
f7577000-f7578000 ---p 00173000 08:01 20316217                           /lib32/libc-2.13.so
f7578000-f757a000 r--p 00173000 08:01 20316217                           /lib32/libc-2.13.so
f757a000-f757b000 rw-p 00175000 08:01 20316217                           /lib32/libc-2.13.so
f757b000-f757e000 rw-p 00000000 00:00 0 
f757e000-f759a000 r-xp 00000000 08:01 7340059                            /usr/lib32/libgcc_s.so.1
f759a000-f759b000 r--p 0001b000 08:01 7340059                            /usr/lib32/libgcc_s.so.1
f759b000-f759c000 rw-p 0001c000 08:01 7340059                            /usr/lib32/libgcc_s.so.1
f759c000-f75c4000 r-xp 00000000 08:01 20316229                           /lib32/libm-2.13.so
f75c4000-f75c5000 r--p 00028000 08:01 20316229                           /lib32/libm-2.13.so
f75c5000-f75c6000 rw-p 00029000 08:01 20316229                           /lib32/libm-2.13.so
f75c6000-f76a4000 r-xp 00000000 08:01 7340151                            /usr/lib32/libstdc++.so.6.0.16
f76a4000-f76a5000 ---p 000de000 08:01 7340151                            /usr/lib32/libstdc++.so.6.0.16
f76a5000-f76a9000 r--p 000de000 08:01 7340151                            /usr/lib32/libstdc++.so.6.0.16
f76a9000-f76aa000 rw-p 000e2000 08:01 7340151                            /usr/lib32/libstdc++.so.6.0.16
f76aa000-f76b2000 rw-p 00000000 00:00 0 
f76b2000-f76b5000 r-xp 00000000 08:01 20316218                           /lib32/libdl-2.13.so
f76b5000-f76b6000 r--p 00002000 08:01 20316218                           /lib32/libdl-2.13.so
f76b6000-f76b7000 rw-p 00003000 08:01 20316218                           /lib32/libdl-2.13.so
f76b7000-f76ce000 r-xp 00000000 08:01 20316225                           /lib32/libpthread-2.13.so
f76ce000-f76cf000 r--p 00016000 08:01 20316225                           /lib32/libpthread-2.13.so
f76cf000-f76d0000 rw-p 00017000 08:01 20316225                           /lib32/libpthread-2.13.so
f76d0000-f76d2000 rw-p 00000000 00:00 0 
f76ed000-f76ee000 rw-p 00000000 00:00 0 
f76ee000-f76ef000 ---p 00000000 00:00 0 
f76ef000-f76f2000 rw-p 00000000 00:00 0 
f76f2000-f76f4000 rw-s 00000000 08:01 14811342                           /var/lib/boinc-client/slots/14/boinc_mmap_file
f76f4000-f76f6000 rw-p 00000000 00:00 0 
f76f6000-f76f7000 r-xp 00000000 00:00 0                                  [vdso]
f76f7000-f7715000 r-xp 00000000 08:01 35389469                           /lib/i386-linux-gnu/ld-2.13.so
f7715000-f7716000 r--p 0001d000 08:01 35389469                           /lib/i386-linux-gnu/ld-2.13.so
f7716000-f7717000 rw-p 0001e000 08:01 35389469                           /lib/i386-linux-gnu/ld-2.13.so
ffcf9000-ffd69000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf76f6400]
[0xf76f6430]
/lib32/libc.so.6(gsignal+0x4f)[0xf7431c4f]
/lib32/libc.so.6(abort+0x175)[0xf7435175]
/lib32/libc.so.6(+0x63e8c)[0xf7467e8c]
/lib32/libc.so.6(+0x6eb72)[0xf7472b72]
/lib32/libc.so.6(+0x6f812)[0xf7473812]
/lib32/libc.so.6(cfree+0x6d)[0xf74768cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf7673baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf7673c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf741d0f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 Nov 2011 17:14:23 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 518,400 1,101,351 2.1245
22 Nov 2011 03:17:29 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 492,480 1,051,379 2.1349
21 Nov 2011 10:39:01 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 466,560 1,010,670 2.1662
20 Nov 2011 15:34:22 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 440,640 954,351 2.1658
20 Nov 2011 00:05:51 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 414,720 901,649 2.1741
19 Nov 2011 07:53:24 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 388,800 845,465 2.1745
18 Nov 2011 08:14:03 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 362,880 789,417 2.1754
17 Nov 2011 15:45:13 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 336,960 733,472 2.1767
16 Nov 2011 23:40:58 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 311,040 677,273 2.1774
16 Nov 2011 07:49:28 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 285,120 621,489 2.1797
15 Nov 2011 17:38:12 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 259,200 564,922 2.1795
15 Nov 2011 17:38:12 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 233,280 509,014 2.1820
15 Nov 2011 17:38:15 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 207,360 453,212 2.1856
10 Nov 2011 06:30:48 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 181,440 397,318 2.1898
08 Nov 2011 07:01:35 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 155,520 341,174 2.1938
07 Nov 2011 12:43:15 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 129,600 284,680 2.1966
06 Nov 2011 08:09:02 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 103,680 228,628 2.2051
03 Nov 2011 21:50:51 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 77,760 171,894 2.2106
31 Oct 2011 19:14:21 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 51,840 114,712 2.2128
31 Oct 2011 18:50:59 1158471 13564183 hadcm3n_yii5_1900_40_007527096_4 25,920 57,253 2.2088


©2024 climateprediction.net