climateprediction.net home page
Task 15871090

Task 15871090

Name hadcm3n_38s2_1980_40_008365571_3
Workunit 8516430
Created 28 Jun 2013, 13:52:34 UTC
Sent 28 Jun 2013, 14:18:37 UTC
Report deadline 27 Sep 2013, 21:45:48 UTC
Received 29 Jun 2013, 18:26:44 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1266912
Run time 20 hours 20 min 8 sec
CPU time 18 hours 44 min 7 sec
Validate state Invalid
Credit 311.04
Device peak FLOPS 2.97 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.43</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
16:50:22 (13540): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:48:59 (14009): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:04:47 (18536): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:04:48 (18536): No heartbeat from core client for 30 sec - exiting
20:04:49 (18536): No heartbeat from core client for 30 sec - exiting
20:04:50 (18536): No heartbeat from core client for 30 sec - exiting
20:04:51 (18536): No heartbeat from core client for 30 sec - exiting
20:04:52 (18536): No heartbeat from core client for 30 sec - exiting
20:04:53 (18536): No heartbeat from core client for 30 sec - exiting
20:04:54 (18536): No heartbeat from core client for 30 sec - exiting
20:04:55 (18536): No heartbeat from core client for 30 sec - exiting
20:04:56 (18536): No heartbeat from core client for 30 sec - exiting
20:04:57 (18536): No heartbeat from core client for 30 sec - exiting
20:04:58 (18536): No heartbeat from core client for 30 sec - exiting
20:04:59 (18536): No heartbeat from core client for 30 sec - exiting
20:05:00 (18536): No heartbeat from core client for 30 sec - exiting
20:05:01 (18536): No heartbeat from core client for 30 sec - exiting
20:05:02 (18536): No heartbeat from core client for 30 sec - exiting
20:39:18 (21129): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:48 (22291): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:49 (22291): No heartbeat from core client for 30 sec - exiting
21:24:09 (23867): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:24:10 (23867): No heartbeat from core client for 30 sec - exiting
21:24:11 (23867): No heartbeat from core client for 30 sec - exiting
21:24:12 (23867): No heartbeat from core client for 30 sec - exiting
21:24:13 (23867): No heartbeat from core client for 30 sec - exiting
21:35:08 (24052): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:35:09 (24052): No heartbeat from core client for 30 sec - exiting
21:35:10 (24052): No heartbeat from core client for 30 sec - exiting
21:35:11 (24052): No heartbeat from core client for 30 sec - exiting
21:35:12 (24052): No heartbeat from core client for 30 sec - exiting
21:35:13 (24052): No heartbeat from core client for 30 sec - exiting
21:35:14 (24052): No heartbeat from core client for 30 sec - exiting
21:35:15 (24052): No heartbeat from core client for 30 sec - exiting
21:35:16 (24052): No heartbeat from core client for 30 sec - exiting
21:35:17 (24052): No heartbeat from core client for 30 sec - exiting
21:35:18 (24052): No heartbeat from core client for 30 sec - exiting
21:35:19 (24052): No heartbeat from core client for 30 sec - exiting
21:35:20 (24052): No heartbeat from core client for 30 sec - exiting
21:35:21 (24052): No heartbeat from core client for 30 sec - exiting
21:35:22 (24052): No heartbeat from core client for 30 sec - exiting
22:44:06 (24451): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:00:34 (27094): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:00:35 (27094): No heartbeat from core client for 30 sec - exiting
01:00:36 (27094): No heartbeat from core client for 30 sec - exiting
01:00:37 (27094): No heartbeat from core client for 30 sec - exiting
01:00:38 (27094): No heartbeat from core client for 30 sec - exiting
01:00:39 (27094): No heartbeat from core client for 30 sec - exiting
01:00:40 (27094): No heartbeat from core client for 30 sec - exiting
01:48:56 (952): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:48:57 (952): No heartbeat from core client for 30 sec - exiting
01:48:58 (952): No heartbeat from core client for 30 sec - exiting
01:48:59 (952): No heartbeat from core client for 30 sec - exiting
01:49:00 (952): No heartbeat from core client for 30 sec - exiting
01:49:01 (952): No heartbeat from core client for 30 sec - exiting
01:49:02 (952): No heartbeat from core client for 30 sec - exiting
01:49:03 (952): No heartbeat from core client for 30 sec - exiting
01:49:04 (952): No heartbeat from core client for 30 sec - exiting
01:49:05 (952): No heartbeat from core client for 30 sec - exiting
01:49:06 (952): No heartbeat from core client for 30 sec - exiting
01:49:07 (952): No heartbeat from core client for 30 sec - exiting
01:49:08 (952): No heartbeat from core client for 30 sec - exiting
01:49:09 (952): No heartbeat from core client for 30 sec - exiting
01:49:10 (952): No heartbeat from core client for 30 sec - exiting
01:49:11 (952): No heartbeat from core client for 30 sec - exiting
01:49:12 (952): No heartbeat from core client for 30 sec - exiting
01:56:56 (2802): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:56:57 (2802): No heartbeat from core client for 30 sec - exiting
02:12:56 (3068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:12:57 (3068): No heartbeat from core client for 30 sec - exiting
02:17:47 (3742): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:17:48 (3742): No heartbeat from core client for 30 sec - exiting
02:17:49 (3742): No heartbeat from core client for 30 sec - exiting
02:17:50 (3742): No heartbeat from core client for 30 sec - exiting
02:17:51 (3742): No heartbeat from core client for 30 sec - exiting
02:17:52 (3742): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
05:05:15 (8136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:05:16 (8136): No heartbeat from core client for 30 sec - exiting
05:05:17 (8136): No heartbeat from core client for 30 sec - exiting
05:05:18 (8136): No heartbeat from core client for 30 sec - exiting
05:05:19 (8136): No heartbeat from core client for 30 sec - exiting
05:05:20 (8136): No heartbeat from core client for 30 sec - exiting
05:05:21 (8136): No heartbeat from core client for 30 sec - exiting
05:05:22 (8136): No heartbeat from core client for 30 sec - exiting
05:05:23 (8136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
07:06:27 (13743): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
09:12:01 (17346): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:09:56 (18167): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:09:57 (18167): No heartbeat from core client for 30 sec - exiting
10:09:58 (18167): No heartbeat from core client for 30 sec - exiting
10:15:35 (19950): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:15:36 (19950): No heartbeat from core client for 30 sec - exiting
10:15:37 (19950): No heartbeat from core client for 30 sec - exiting
10:15:38 (19950): No heartbeat from core client for 30 sec - exiting
10:15:39 (19950): No heartbeat from core client for 30 sec - exiting
10:15:40 (19950): No heartbeat from core client for 30 sec - exiting
10:15:41 (19950): No heartbeat from core client for 30 sec - exiting
10:15:42 (19950): No heartbeat from core client for 30 sec - exiting
10:15:43 (19950): No heartbeat from core client for 30 sec - exiting
10:15:44 (19950): No heartbeat from core client for 30 sec - exiting
10:15:45 (19950): No heartbeat from core client for 30 sec - exiting
10:15:46 (19950): No heartbeat from core client for 30 sec - exiting
10:15:47 (19950): No heartbeat from core client for 30 sec - exiting
10:42:16 (20081): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:42:17 (20081): No heartbeat from core client for 30 sec - exiting
10:42:18 (20081): No heartbeat from core client for 30 sec - exiting
10:42:19 (20081): No heartbeat from core client for 30 sec - exiting
11:11:51 (20801): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:11:52 (20801): No heartbeat from core client for 30 sec - exiting
11:11:53 (20801): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
12:34:12 (24200): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:39:09 (24971): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:52:15 (25178): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:24:00 (25674): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:45:46 (26939): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
15:53:10 (32308): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:53:11 (32308): No heartbeat from core client for 30 sec - exiting
15:53:12 (32308): No heartbeat from core client for 30 sec - exiting
15:53:13 (32308): No heartbeat from core client for 30 sec - exiting
15:53:14 (32308): No heartbeat from core client for 30 sec - exiting
15:53:15 (32308): No heartbeat from core client for 30 sec - exiting
15:53:16 (32308): No heartbeat from core client for 30 sec - exiting
15:53:17 (32308): No heartbeat from core client for 30 sec - exiting
15:53:18 (32308): No heartbeat from core client for 30 sec - exiting
15:53:19 (32308): No heartbeat from core client for 30 sec - exiting
15:53:20 (32308): No heartbeat from core client for 30 sec - exiting
15:53:21 (32308): No heartbeat from core client for 30 sec - exiting
15:53:22 (32308): No heartbeat from core client for 30 sec - exiting
15:53:23 (32308): No heartbeat from core client for 30 sec - exiting
15:53:24 (32308): No heartbeat from core client for 30 sec - exiting
15:53:25 (32308): No heartbeat from core client for 30 sec - exiting
15:53:26 (32308): No heartbeat from core client for 30 sec - exiting
15:53:27 (32308): No heartbeat from core client for 30 sec - exiting
15:53:28 (32308): No heartbeat from core client for 30 sec - exiting
15:53:29 (32308): No heartbeat from core client for 30 sec - exiting
15:53:30 (32308): No heartbeat from core client for 30 sec - exiting
16:34:42 (32465): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:34:43 (32465): No heartbeat from core client for 30 sec - exiting
16:34:44 (32465): No heartbeat from core client for 30 sec - exiting
16:58:54 (1338): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:58:55 (1338): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
19:08:17 (5281): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:08:18 (5281): No heartbeat from core client for 30 sec - exiting
19:08:19 (5281): No heartbeat from core client for 30 sec - exiting
19:08:20 (5281): No heartbeat from core client for 30 sec - exiting
19:08:21 (5281): No heartbeat from core client for 30 sec - exiting
19:22:39 (5849): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:22:40 (5849): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf774c400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf774c430]
/lib/libc.so.6(gsignal+0x4f)[0xf758631f]
/lib/libc.so.6(abort+0x143)[0xf7587c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75713d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8426, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7737400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7737430]
/lib/libc.so.6(gsignal+0x4f)[0xf757131f]
/lib/libc.so.6(abort+0x143)[0xf7572c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf755c3d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8426, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf771c400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf771c430]
/lib/libc.so.6(gsignal+0x4f)[0xf755631f]
/lib/libc.so.6(abort+0x143)[0xf7557c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75413d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8426, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7789400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7789430]
/lib/libc.so.6(gsignal+0x4f)[0xf75c331f]
/lib/libc.so.6(abort+0x143)[0xf75c4c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75ae3d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8426, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77c4400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf77c4430]
/lib/libc.so.6(gsignal+0x4f)[0xf75fe31f]
/lib/libc.so.6(abort+0x143)[0xf75ffc03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75e93d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8426, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7765400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7765430]
/lib/libc.so.6(gsignal+0x4f)[0xf759f31f]
/lib/libc.so.6(abort+0x143)[0xf75a0c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf758a3d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8426, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
02 Jul 2013 10:00:57 1266912 15871090 hadcm3n_38s2_1980_40_008365571_3 25,920 44,238 1.7067


©2024 climateprediction.net