climateprediction.net home page
Task 13956521

Task 13956521

Name hadcm3n_t3d8_1940_40_007693716_2
Workunit 7848824
Created 23 Jan 2012, 19:39:08 UTC
Sent 24 Jan 2012, 4:33:42 UTC
Report deadline 24 Apr 2012, 12:00:53 UTC
Received 12 Feb 2012, 21:20:52 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1180524
Run time 5 days 17 hours 54 min 36 sec
CPU time 5 days 3 hours 40 min 12 sec
Validate state Invalid
Credit 4,665.60
Device peak FLOPS 3.31 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>6.12.34</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
09:16:50 (1676): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
12:56:48 (4456): No heartbeat from core client for 30 sec - exiting
12:56:49 (4456): No heartbeat from core client for 30 sec - exiting
12:56:50 (4456): No heartbeat from core client for 30 sec - exiting
12:56:51 (4456): No heartbeat from core client for 30 sec - exiting
12:56:52 (4456): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:38:09 (4856): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:06:36 (5704): No heartbeat from core client for 30 sec - exiting
14:11:14 (5704): No heartbeat from core client for 30 sec - exiting
14:11:15 (5704): No heartbeat from core client for 30 sec - exiting
14:11:16 (5704): No heartbeat from core client for 30 sec - exiting
14:11:17 (5704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:56:56 (5472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:15:13 (2308): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:57:57 (4268): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:00:46 (5324): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:05:23 (5412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:16:46 (2936): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:07:33 (6016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:21:08 (4512): Can't acquire lockfile (32) - waiting 35s
19:35:34 (4512): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
19:40:15 (4304): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:28:02 (4136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:36:09 (6128): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:39:58 (6324): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:42:39 (2988): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:56:31 (6512): No heartbeat from core client for 30 sec - exiting
20:56:32 (6512): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:10:39 (1076): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:32:04 (5160): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:35:03 (6732): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:35:45 (1484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:09:05 (1552): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:15:22 (3460): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:37:12 (5696): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:51:28 (6832): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:14:10 (5576): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:17:26 (5244): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:34:43 (276): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:35:20 (4052): No heartbeat from core client for 30 sec - exiting
05:35:21 (4052): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:39:39 (4164): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:00:35 (3064): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:02:43 (6332): No heartbeat from core client for 30 sec - exiting
06:02:44 (6332): No heartbeat from core client for 30 sec - exiting
06:02:46 (6332): No heartbeat from core client for 30 sec - exiting
06:02:47 (6332): No heartbeat from core client for 30 sec - exiting
06:02:48 (6332): No heartbeat from core client for 30 sec - exiting
06:02:49 (6332): No heartbeat from core client for 30 sec - exiting
06:02:50 (6332): No heartbeat from core client for 30 sec - exiting
06:02:51 (6332): No heartbeat from core client for 30 sec - exiting
06:02:52 (6332): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:11:12 (6768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/t3d8ko.pje1c10
Error converting file to netcdf: dataout/t3d8ko.pie1c10
Error converting file to netcdf: dataout/t3d8ko.pfe1c10
Error converting file to netcdf: dataout/t3d8ka.phe1c10
Error converting file to netcdf: dataout/t3d8ka.pge1c10
Error converting file to netcdf: dataout/t3d8ka.pee1c10
Error converting file to netcdf: dataout/t3d8ka.pde1c10
06:14:57 (5340): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:20:52 (7376): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:22:57 (5560): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:25:08 (7500): No heartbeat from core client for 30 sec - exiting
06:25:42 (7500): No heartbeat from core client for 30 sec - exiting
06:25:43 (7500): No heartbeat from core client for 30 sec - exiting
06:25:45 (7500): No heartbeat from core client for 30 sec - exiting
06:25:46 (7500): No heartbeat from core client for 30 sec - exiting
06:25:47 (7500): No heartbeat from core client for 30 sec - exiting
06:25:48 (7500): No heartbeat from core client for 30 sec - exiting
06:25:49 (7500): No heartbeat from core client for 30 sec - exiting
06:25:50 (7500): No heartbeat from core client for 30 sec - exiting
06:25:51 (7500): No heartbeat from core client for 30 sec - exiting
06:25:52 (7500): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:25:53 (7500): No heartbeat from core client for 30 sec - exiting
06:26:56 (8184): No heartbeat from core client for 30 sec - exiting
06:26:57 (8184): No heartbeat from core client for 30 sec - exiting
06:26:58 (8184): No heartbeat from core client for 30 sec - exiting
06:26:59 (8184): No heartbeat from core client for 30 sec - exiting
06:27:01 (8184): No heartbeat from core client for 30 sec - exiting
06:27:02 (8184): No heartbeat from core client for 30 sec - exiting
06:27:03 (8184): No heartbeat from core client for 30 sec - exiting
06:27:04 (8184): No heartbeat from core client for 30 sec - exiting
06:27:05 (8184): No heartbeat from core client for 30 sec - exiting
06:27:06 (8184): No heartbeat from core client for 30 sec - exiting
06:27:07 (8184): No heartbeat from core client for 30 sec - exiting
06:27:08 (8184): No heartbeat from core client for 30 sec - exiting
06:27:09 (8184): No heartbeat from core client for 30 sec - exiting
06:27:10 (8184): No heartbeat from core client for 30 sec - exiting
06:27:11 (8184): No heartbeat from core client for 30 sec - exiting
06:27:13 (8184): No heartbeat from core client for 30 sec - exiting
06:27:14 (8184): No heartbeat from core client for 30 sec - exiting
06:27:15 (8184): No heartbeat from core client for 30 sec - exiting
06:27:16 (8184): No heartbeat from core client for 30 sec - exiting
06:27:17 (8184): No heartbeat from core client for 30 sec - exiting
06:27:18 (8184): No heartbeat from core client for 30 sec - exiting
06:27:19 (8184): No heartbeat from core client for 30 sec - exiting
06:27:20 (8184): No heartbeat from core client for 30 sec - exiting
06:27:21 (8184): No heartbeat from core client for 30 sec - exiting
06:27:22 (8184): No heartbeat from core client for 30 sec - exiting
06:27:23 (8184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
06:28:34 (4144): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
12 Feb 2012 19:12:38 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 388,800 476,976 1.2268
12 Feb 2012 04:08:13 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 362,880 445,214 1.2269
11 Feb 2012 15:37:20 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 336,960 413,928 1.2284
11 Feb 2012 03:18:05 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 311,040 381,714 1.2272
10 Feb 2012 16:06:11 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 285,120 349,678 1.2264
10 Feb 2012 01:51:04 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 259,200 317,761 1.2259
09 Feb 2012 15:43:28 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 233,280 284,836 1.2210
09 Feb 2012 06:07:57 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 207,360 251,059 1.2107
08 Feb 2012 20:42:12 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 181,440 217,692 1.1998
08 Feb 2012 12:06:49 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 155,520 187,044 1.2027
08 Feb 2012 03:36:59 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 129,600 156,405 1.2068
07 Feb 2012 18:59:47 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 103,680 125,635 1.2118
07 Feb 2012 10:33:03 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 77,760 95,279 1.2253
07 Feb 2012 02:09:22 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 51,840 63,821 1.2311
06 Feb 2012 14:17:45 1180524 13956521 hadcm3n_t3d8_1940_40_007693716_2 25,920 30,692 1.1841


©2024 climateprediction.net