climateprediction.net home page
Task 16052387

Task 16052387

Name hadcm3n_705y_1980_40_008411471_1
Workunit 8562327
Created 1 Oct 2013, 16:40:48 UTC
Sent 1 Oct 2013, 16:46:57 UTC
Report deadline 1 Jan 2014, 0:14:08 UTC
Received 6 Dec 2013, 8:27:31 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1239443
Run time 21 days 20 hours 33 min 45 sec
CPU time 18 days 11 hours 46 min 15 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 2.67 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.45</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
13:07:27 (28174): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
17:40:02 (4521): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:42:05 (20683): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
13:36:39 (1431): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:03:35 (4429): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
15:06:58 (16523): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:35:57 (25602): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:16:13 (28692): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
15:02:27 (19561): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:47:18 (4067): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:15:21 (26587): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0944ead8 ***
======= Backtrace: =========
/lib/libc.so.6[0x464df1]
/lib/libc.so.6[0x467531]
/usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x2b1552]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x2b15ae]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/libc.so.6(__libc_start_main+0xe6)[0x40ace6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
001e2000-001ff000 r-xp 00000000 fd:00 260205                             /lib/libgcc_s-4.4.7-20120601.so.1
001ff000-00200000 rw-p 0001d000 fd:00 260205                             /lib/libgcc_s-4.4.7-20120601.so.1
00202000-002e3000 r-xp 00000000 fd:00 20113                              /usr/lib/libstdc++.so.6.0.13
002e3000-002e7000 r--p 000e0000 fd:00 20113                              /usr/lib/libstdc++.so.6.0.13
002e7000-002e9000 rw-p 000e4000 fd:00 20113                              /usr/lib/libstdc++.so.6.0.13
002e9000-002ef000 rw-p 00000000 00:00 0 
003d2000-003f0000 r-xp 00000000 fd:00 261963                             /lib/ld-2.12.so
003f0000-003f1000 r--p 0001d000 fd:00 261963                             /lib/ld-2.12.so
003f1000-003f2000 rw-p 0001e000 fd:00 261963                             /lib/ld-2.12.so
003f4000-00584000 r-xp 00000000 fd:00 262632                             /lib/libc-2.12.so
00584000-00585000 ---p 00190000 fd:00 262632                             /lib/libc-2.12.so
00585000-00587000 r--p 00190000 fd:00 262632                             /lib/libc-2.12.so
00587000-00588000 rw-p 00192000 fd:00 262632                             /lib/libc-2.12.so
00588000-0058b000 rw-p 00000000 00:00 0 
0058d000-00590000 r-xp 00000000 fd:00 262731                             /lib/libdl-2.12.so
00590000-00591000 r--p 00002000 fd:00 262731                             /lib/libdl-2.12.so
00591000-00592000 rw-p 00003000 fd:00 262731                             /lib/libdl-2.12.so
00594000-005bc000 r-xp 00000000 fd:00 262739                             /lib/libm-2.12.so
005bc000-005bd000 r--p 00027000 fd:00 262739                             /lib/libm-2.12.so
005bd000-005be000 rw-p 00028000 fd:00 262739                             /lib/libm-2.12.so
005c0000-005d7000 r-xp 00000000 fd:00 261546                             /lib/libpthread-2.12.so
005d7000-005d8000 r--p 00016000 fd:00 261546                             /lib/libpthread-2.12.so
005d8000-005d9000 rw-p 00017000 fd:00 261546                             /lib/libpthread-2.12.so
005d9000-005db000 rw-p 00000000 00:00 0 
0091b000-0091c000 r-xp 00000000 00:00 0                                  [vdso]
08048000-080e3000 r-xp 00000000 fd:02 11011483                           /home/boinc/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 fd:02 11011483                           /home/boinc/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
093ff000-09485000 rw-p 00000000 00:00 0                                  [heap]
f7184000-f7185000 rw-p 00000000 00:00 0 
f7185000-f7600000 rw-s 00000000 fd:02 12845165                           /home/boinc/boinc/slots/0/131765
f7600000-f7621000 rw-p 00000000 00:00 0 
f7621000-f7700000 ---p 00000000 00:00 0 
f773b000-f773e000 rw-p 00000000 00:00 0 
f7753000-f7754000 ---p 00000000 00:00 0 
f7754000-f7757000 rw-p 00000000 00:00 0 
f7757000-f7759000 rw-s 00000000 fd:02 12845148                           /home/boinc/boinc/slots/0/boinc_mmap_file
f7759000-f775b000 rw-p 00000000 00:00 0 
ffcf8000-ffd67000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (18 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0x91b400]
[0x91b430]
/lib/libc.so.6(gsignal+0x51)[0x41ead1]
/lib/libc.so.6(abort+0x17a)[0x4203aa]
/lib/libc.so.6[0x45e995]
/lib/libc.so.6[0x464df1]
/lib/libc.so.6[0x467531]
/usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x2b1552]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x2b15ae]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/libc.so.6(__libc_start_main+0xe6)[0x40ace6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
06 Dec 2013 08:27:36 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 1,036,800 1,597,573 1.5409
05 Dec 2013 21:07:27 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 1,010,880 1,557,695 1.5409
05 Dec 2013 08:37:25 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 984,960 1,517,703 1.5409
04 Dec 2013 20:38:31 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 959,040 1,477,781 1.5409
04 Dec 2013 08:30:00 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 933,120 1,437,440 1.5405
03 Dec 2013 21:06:31 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 907,200 1,397,372 1.5403
03 Dec 2013 09:13:29 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 881,280 1,357,338 1.5402
02 Dec 2013 21:31:15 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 855,360 1,317,332 1.5401
02 Dec 2013 06:58:28 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 829,440 1,277,241 1.5399
01 Dec 2013 15:48:23 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 803,520 1,236,960 1.5394
01 Dec 2013 00:51:56 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 777,600 1,196,818 1.5391
30 Nov 2013 09:50:26 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 751,680 1,156,338 1.5383
29 Nov 2013 18:46:03 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 725,760 1,115,724 1.5373
29 Nov 2013 03:43:08 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 699,840 1,075,160 1.5363
28 Nov 2013 12:33:20 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 673,920 1,034,357 1.5348
27 Nov 2013 19:14:11 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 648,000 993,888 1.5338
27 Nov 2013 03:04:19 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 622,080 953,834 1.5333
26 Nov 2013 12:40:03 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 596,160 913,584 1.5324
26 Nov 2013 01:18:24 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 570,240 873,777 1.5323
25 Nov 2013 14:12:50 1239443 16052387 hadcm3n_705y_1980_40_008411471_1 544,320 833,940 1.5321


©2024 climateprediction.net