climateprediction.net home page
Task 15865780

Task 15865780

Name hadcm3n_u1b4_2020_40_008336522_1
Workunit 8487383
Created 26 Jun 2013, 4:57:49 UTC
Sent 29 Jun 2013, 18:26:44 UTC
Report deadline 29 Sep 2013, 1:53:55 UTC
Received 2 Jul 2013, 19:20:54 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1266912
Run time 1 days 23 hours 43 min 56 sec
CPU time 1 days 20 hours 39 min 20 sec
Validate state Invalid
Credit 933.12
Device peak FLOPS 2.97 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.12.43</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
22:36:17 (9789): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:41:24 (11044): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:32:01 (11121): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
02:27:19 (15315): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:27:20 (15315): No heartbeat from core client for 30 sec - exiting
02:27:21 (15315): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
05:47:15 (19427): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
11:43:06 (25544): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:32:31 (25834): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:48:49 (26717): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:48:50 (26717): No heartbeat from core client for 30 sec - exiting
12:48:51 (26717): No heartbeat from core client for 30 sec - exiting
12:48:52 (26717): No heartbeat from core client for 30 sec - exiting
12:48:53 (26717): No heartbeat from core client for 30 sec - exiting
12:48:54 (26717): No heartbeat from core client for 30 sec - exiting
12:48:55 (26717): No heartbeat from core client for 30 sec - exiting
12:48:56 (26717): No heartbeat from core client for 30 sec - exiting
12:48:57 (26717): No heartbeat from core client for 30 sec - exiting
13:27:14 (26982): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:40:58 (27650): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
19:09:11 (32538): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:09:17 (32538): No heartbeat from core client for 30 sec - exiting
19:09:18 (32538): No heartbeat from core client for 30 sec - exiting
19:45:20 (1183): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:43:24 (1906): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:57:46 (2989): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
23:47:16 (6146): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:03:36 (6256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:14:58 (7163): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:44:57 (7442): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:15:57 (8339): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:16:08 (8339): No heartbeat from core client for 30 sec - exiting
01:16:09 (8339): No heartbeat from core client for 30 sec - exiting
01:16:10 (8339): No heartbeat from core client for 30 sec - exiting
01:16:11 (8339): No heartbeat from core client for 30 sec - exiting
01:16:12 (8339): No heartbeat from core client for 30 sec - exiting
01:16:13 (8339): No heartbeat from core client for 30 sec - exiting
01:16:14 (8339): No heartbeat from core client for 30 sec - exiting
01:16:15 (8339): No heartbeat from core client for 30 sec - exiting
01:16:16 (8339): No heartbeat from core client for 30 sec - exiting
01:16:17 (8339): No heartbeat from core client for 30 sec - exiting
01:16:18 (8339): No heartbeat from core client for 30 sec - exiting
01:16:19 (8339): No heartbeat from core client for 30 sec - exiting
01:16:20 (8339): No heartbeat from core client for 30 sec - exiting
01:16:21 (8339): No heartbeat from core client for 30 sec - exiting
01:30:42 (8937): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:20:47 (9213): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:20:53 (9213): No heartbeat from core client for 30 sec - exiting
02:20:54 (9213): No heartbeat from core client for 30 sec - exiting
02:20:55 (9213): No heartbeat from core client for 30 sec - exiting
02:26:27 (10123): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:26:28 (10123): No heartbeat from core client for 30 sec - exiting
02:26:29 (10123): No heartbeat from core client for 30 sec - exiting
02:26:30 (10123): No heartbeat from core client for 30 sec - exiting
02:26:31 (10123): No heartbeat from core client for 30 sec - exiting
02:26:32 (10123): No heartbeat from core client for 30 sec - exiting
02:26:33 (10123): No heartbeat from core client for 30 sec - exiting
02:26:34 (10123): No heartbeat from core client for 30 sec - exiting
02:26:35 (10123): No heartbeat from core client for 30 sec - exiting
02:35:49 (10281): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
05:39:08 (12662): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:57:49 (13702): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:43:49 (14036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:22:49 (16430): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:12:11 (16622): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:12:12 (16622): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
11:25:42 (19759): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:54:52 (20339): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
13:42:54 (23864): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:48:23 (24031): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:08:27 (607): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:08:28 (607): No heartbeat from core client for 30 sec - exiting
18:22:37 (1817): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:03:01 (2373): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:32:48 (3863): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:38:40 (4883): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
02:28:13 (21641): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
03:02:47 (22145): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:19:32 (23510): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:35:46 (24165): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:43:54 (24736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:18:56 (27353): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:55:50 (28711): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:55:51 (28711): No heartbeat from core client for 30 sec - exiting
05:55:52 (28711): No heartbeat from core client for 30 sec - exiting
06:04:00 (29963): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:04:01 (29963): No heartbeat from core client for 30 sec - exiting
06:04:02 (29963): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
09:27:32 (3797): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:38:22 (5679): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:38:23 (5679): No heartbeat from core client for 30 sec - exiting
09:38:24 (5679): No heartbeat from core client for 30 sec - exiting
09:38:25 (5679): No heartbeat from core client for 30 sec - exiting
09:38:26 (5679): No heartbeat from core client for 30 sec - exiting
09:38:27 (5679): No heartbeat from core client for 30 sec - exiting
09:38:28 (5679): No heartbeat from core client for 30 sec - exiting
09:38:29 (5679): No heartbeat from core client for 30 sec - exiting
09:38:30 (5679): No heartbeat from core client for 30 sec - exiting
09:38:31 (5679): No heartbeat from core client for 30 sec - exiting
09:38:32 (5679): No heartbeat from core client for 30 sec - exiting
09:38:33 (5679): No heartbeat from core client for 30 sec - exiting
09:38:34 (5679): No heartbeat from core client for 30 sec - exiting
09:38:35 (5679): No heartbeat from core client for 30 sec - exiting
09:38:36 (5679): No heartbeat from core client for 30 sec - exiting
09:38:37 (5679): No heartbeat from core client for 30 sec - exiting
09:38:38 (5679): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
12:03:58 (11227): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
13:12:42 (14160): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:06:20 (30220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:19:35 (30562): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:19:36 (30562): No heartbeat from core client for 30 sec - exiting
21:19:37 (30562): No heartbeat from core client for 30 sec - exiting
21:19:38 (30562): No heartbeat from core client for 30 sec - exiting
21:19:39 (30562): No heartbeat from core client for 30 sec - exiting
21:19:40 (30562): No heartbeat from core client for 30 sec - exiting
21:19:41 (30562): No heartbeat from core client for 30 sec - exiting
21:19:42 (30562): No heartbeat from core client for 30 sec - exiting
21:19:43 (30562): No heartbeat from core client for 30 sec - exiting
21:19:44 (30562): No heartbeat from core client for 30 sec - exiting
21:19:45 (30562): No heartbeat from core client for 30 sec - exiting
21:19:46 (30562): No heartbeat from core client for 30 sec - exiting
21:19:47 (30562): No heartbeat from core client for 30 sec - exiting
21:19:48 (30562): No heartbeat from core client for 30 sec - exiting
21:19:49 (30562): No heartbeat from core client for 30 sec - exiting
21:19:50 (30562): No heartbeat from core client for 30 sec - exiting
21:19:51 (30562): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77d9400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf77d9430]
/lib/libc.so.6(gsignal+0x4f)[0xf761331f]
/lib/libc.so.6(abort+0x143)[0xf7614c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75fe3d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=909, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7714400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7714430]
/lib/libc.so.6(gsignal+0x4f)[0xf754e31f]
/lib/libc.so.6(abort+0x143)[0xf754fc03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75393d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=909, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf774f400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf774f430]
/lib/libc.so.6(gsignal+0x4f)[0xf758931f]
/lib/libc.so.6(abort+0x143)[0xf758ac03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75743d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=909, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7726400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7726430]
/lib/libc.so.6(gsignal+0x4f)[0xf756031f]
/lib/libc.so.6(abort+0x143)[0xf7561c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf754b3d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=909, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7765400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7765430]
/lib/libc.so.6(gsignal+0x4f)[0xf759f31f]
/lib/libc.so.6(abort+0x143)[0xf75a0c03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf758a3d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=909, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77d4400]
linux-gate.so.1(__kernel_vsyscall+0x10)[0xf77d4430]
/lib/libc.so.6(gsignal+0x4f)[0xf760e31f]
/lib/libc.so.6(abort+0x143)[0xf760fc03]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75f93d5]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=909, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
02 Jul 2013 12:08:05 1266912 15865780 hadcm3n_u1b4_2020_40_008336522_1 77,760 140,358 1.8050
02 Jul 2013 11:34:12 1266912 15865780 hadcm3n_u1b4_2020_40_008336522_1 51,840 93,483 1.8033
02 Jul 2013 10:47:22 1266912 15865780 hadcm3n_u1b4_2020_40_008336522_1 25,920 46,757 1.8039


©2024 climateprediction.net