climateprediction.net home page
Task 13627203

Task 13627203

Name hadcm3n_o1hp_1940_40_007543576_0
Workunit 7740808
Created 10 Nov 2011, 2:08:05 UTC
Sent 16 Nov 2011, 7:02:29 UTC
Report deadline 15 Feb 2012, 14:29:40 UTC
Received 20 Nov 2011, 22:41:16 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1066269
Run time 2 days 12 hours 0 min 12 sec
CPU time 1 days 8 hours 56 min 9 sec
Validate state Invalid
Credit 622.08
Device peak FLOPS 2.93 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.12.42</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
10:18:52 (21867): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:12:35 (22146): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:12:36 (22146): No heartbeat from core client for 30 sec - exiting
11:12:37 (22146): No heartbeat from core client for 30 sec - exiting
13:30:07 (22368): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:30:08 (22368): No heartbeat from core client for 30 sec - exiting
13:30:12 (22368): No heartbeat from core client for 30 sec - exiting
13:30:13 (22368): No heartbeat from core client for 30 sec - exiting
13:30:14 (22368): No heartbeat from core client for 30 sec - exiting
13:30:15 (22368): No heartbeat from core client for 30 sec - exiting
13:30:16 (22368): No heartbeat from core client for 30 sec - exiting
13:30:17 (22368): No heartbeat from core client for 30 sec - exiting
13:30:18 (22368): No heartbeat from core client for 30 sec - exiting
13:30:19 (22368): No heartbeat from core client for 30 sec - exiting
13:30:20 (22368): No heartbeat from core client for 30 sec - exiting
13:30:27 (22368): No heartbeat from core client for 30 sec - exiting
13:30:28 (22368): No heartbeat from core client for 30 sec - exiting
13:30:29 (22368): No heartbeat from core client for 30 sec - exiting
13:30:30 (22368): No heartbeat from core client for 30 sec - exiting
13:30:31 (22368): No heartbeat from core client for 30 sec - exiting
13:30:32 (22368): No heartbeat from core client for 30 sec - exiting
13:30:37 (22368): No heartbeat from core client for 30 sec - exiting
13:30:38 (22368): No heartbeat from core client for 30 sec - exiting
15:10:57 (22955): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:10:58 (22955): No heartbeat from core client for 30 sec - exiting
15:10:59 (22955): No heartbeat from core client for 30 sec - exiting
15:11:05 (22955): No heartbeat from core client for 30 sec - exiting
15:11:06 (22955): No heartbeat from core client for 30 sec - exiting
15:11:07 (22955): No heartbeat from core client for 30 sec - exiting
15:11:08 (22955): No heartbeat from core client for 30 sec - exiting
15:11:09 (22955): No heartbeat from core client for 30 sec - exiting
15:11:10 (22955): No heartbeat from core client for 30 sec - exiting
15:11:11 (22955): No heartbeat from core client for 30 sec - exiting
15:11:20 (22955): No heartbeat from core client for 30 sec - exiting
15:11:21 (22955): No heartbeat from core client for 30 sec - exiting
15:11:22 (22955): No heartbeat from core client for 30 sec - exiting
15:11:23 (22955): No heartbeat from core client for 30 sec - exiting
15:11:24 (22955): No heartbeat from core client for 30 sec - exiting
15:11:25 (22955): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
03:15:58 (23323): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:15:59 (23323): No heartbeat from core client for 30 sec - exiting
03:16:00 (23323): No heartbeat from core client for 30 sec - exiting
03:16:01 (23323): No heartbeat from core client for 30 sec - exiting
03:16:02 (23323): No heartbeat from core client for 30 sec - exiting
03:16:03 (23323): No heartbeat from core client for 30 sec - exiting
03:16:13 (23323): No heartbeat from core client for 30 sec - exiting
03:16:14 (23323): No heartbeat from core client for 30 sec - exiting
03:16:15 (23323): No heartbeat from core client for 30 sec - exiting
03:16:16 (23323): No heartbeat from core client for 30 sec - exiting
03:16:17 (23323): No heartbeat from core client for 30 sec - exiting
03:16:25 (23323): No heartbeat from core client for 30 sec - exiting
03:16:33 (23323): No heartbeat from core client for 30 sec - exiting
03:16:34 (23323): No heartbeat from core client for 30 sec - exiting
03:16:35 (23323): No heartbeat from core client for 30 sec - exiting
03:16:36 (23323): No heartbeat from core client for 30 sec - exiting
03:16:37 (23323): No heartbeat from core client for 30 sec - exiting
03:16:38 (23323): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
07:21:31 (982): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:21:32 (982): No heartbeat from core client for 30 sec - exiting
07:21:33 (982): No heartbeat from core client for 30 sec - exiting
07:21:34 (982): No heartbeat from core client for 30 sec - exiting
07:21:35 (982): No heartbeat from core client for 30 sec - exiting
07:21:36 (982): No heartbeat from core client for 30 sec - exiting
07:21:37 (982): No heartbeat from core client for 30 sec - exiting
07:21:38 (982): No heartbeat from core client for 30 sec - exiting
07:21:39 (982): No heartbeat from core client for 30 sec - exiting
07:21:40 (982): No heartbeat from core client for 30 sec - exiting
07:21:41 (982): No heartbeat from core client for 30 sec - exiting
07:21:42 (982): No heartbeat from core client for 30 sec - exiting
07:21:43 (982): No heartbeat from core client for 30 sec - exiting
07:21:44 (982): No heartbeat from core client for 30 sec - exiting
07:21:45 (982): No heartbeat from core client for 30 sec - exiting
07:21:46 (982): No heartbeat from core client for 30 sec - exiting
07:21:47 (982): No heartbeat from core client for 30 sec - exiting
07:21:48 (982): No heartbeat from core client for 30 sec - exiting
07:21:56 (982): No heartbeat from core client for 30 sec - exiting
07:22:01 (982): No heartbeat from core client for 30 sec - exiting
07:22:02 (982): No heartbeat from core client for 30 sec - exiting
07:22:03 (982): No heartbeat from core client for 30 sec - exiting
07:22:04 (982): No heartbeat from core client for 30 sec - exiting
07:22:05 (982): No heartbeat from core client for 30 sec - exiting
07:22:06 (982): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
15:24:52 (17033): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:24:53 (17033): No heartbeat from core client for 30 sec - exiting
15:24:54 (17033): No heartbeat from core client for 30 sec - exiting
15:24:55 (17033): No heartbeat from core client for 30 sec - exiting
15:24:56 (17033): No heartbeat from core client for 30 sec - exiting
15:24:57 (17033): No heartbeat from core client for 30 sec - exiting
04:33:59 (2890): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:34:00 (2890): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
07:30:17 (20090): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:30:18 (20090): No heartbeat from core client for 30 sec - exiting
07:30:19 (20090): No heartbeat from core client for 30 sec - exiting
07:30:20 (20090): No heartbeat from core client for 30 sec - exiting
07:30:21 (20090): No heartbeat from core client for 30 sec - exiting
07:30:22 (20090): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
07:42:15 (20135): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:42:16 (20135): No heartbeat from core client for 30 sec - exiting
07:42:17 (20135): No heartbeat from core client for 30 sec - exiting
07:42:18 (20135): No heartbeat from core client for 30 sec - exiting
07:42:19 (20135): No heartbeat from core client for 30 sec - exiting
07:42:20 (20135): No heartbeat from core client for 30 sec - exiting
07:42:21 (20135): No heartbeat from core client for 30 sec - exiting
07:42:22 (20135): No heartbeat from core client for 30 sec - exiting
07:42:23 (20135): No heartbeat from core client for 30 sec - exiting
07:42:31 (20135): No heartbeat from core client for 30 sec - exiting
07:42:32 (20135): No heartbeat from core client for 30 sec - exiting
07:42:33 (20135): No heartbeat from core client for 30 sec - exiting
07:42:34 (20135): No heartbeat from core client for 30 sec - exiting
07:42:35 (20135): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
08:00:35 (20213): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:00:36 (20213): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
03:11:12 (26685): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:11:17 (26685): No heartbeat from core client for 30 sec - exiting
03:11:18 (26685): No heartbeat from core client for 30 sec - exiting
03:11:19 (26685): No heartbeat from core client for 30 sec - exiting
03:11:20 (26685): No heartbeat from core client for 30 sec - exiting
03:11:21 (26685): No heartbeat from core client for 30 sec - exiting
03:11:24 (26685): No heartbeat from core client for 30 sec - exiting
03:11:25 (26685): No heartbeat from core client for 30 sec - exiting
03:11:26 (26685): No heartbeat from core client for 30 sec - exiting
03:11:27 (26685): No heartbeat from core client for 30 sec - exiting
03:11:28 (26685): No heartbeat from core client for 30 sec - exiting
03:11:29 (26685): No heartbeat from core client for 30 sec - exiting
03:11:30 (26685): No heartbeat from core client for 30 sec - exiting
03:11:31 (26685): No heartbeat from core client for 30 sec - exiting
03:11:39 (26685): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
03:17:41 (27425): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGABRT: abort called
Stack trace (8 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf773f400]
[0xf773f430]
/lib32/libc.so.6(gsignal+0x50)[0xf7565a60]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xff)[0xf754e2cf]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28922, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (8 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf778f400]
[0xf778f430]
/lib32/libc.so.6(gsignal+0x50)[0xf75b5a60]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xff)[0xf759e2cf]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28922, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (8 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77cf400]
[0xf77cf430]
/lib32/libc.so.6(gsignal+0x50)[0xf75f5a60]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xff)[0xf75de2cf]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28922, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (8 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77be400]
[0xf77be430]
/lib32/libc.so.6(gsignal+0x50)[0xf75e4a60]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xff)[0xf75cd2cf]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28922, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (8 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf771b400]
[0xf771b430]
/lib32/libc.so.6(gsignal+0x50)[0xf7541a60]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xff)[0xf752a2cf]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28922, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (8 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf778d400]
[0xf778d430]
/lib32/libc.so.6(gsignal+0x50)[0xf75b3a60]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xff)[0xf759c2cf]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28922, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
19 Nov 2011 01:41:51 1066269 13627203 hadcm3n_o1hp_1940_40_007543576_0 51,840 81,245 1.5672
17 Nov 2011 06:52:48 1066269 13627203 hadcm3n_o1hp_1940_40_007543576_0 25,920 40,385 1.5581


©2024 climateprediction.net