Task 16024838

Name	hadcm3n_84a1_1980_40_008463501_0
Workunit	8614340
Created	19 Sep 2013, 14:31:21 UTC
Sent	21 Sep 2013, 23:23:10 UTC
Report deadline	22 Dec 2013, 6:50:21 UTC
Received	30 Oct 2013, 8:20:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1125333
Run time	5 days 22 hours 39 min 14 sec
CPU time	5 days 11 hours 48 min 15 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	2.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 15:18:20 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:18:23 (4912): No heartbeat from core client for 30 sec - exiting 15:18:24 (4912): No heartbeat from core client for 30 sec - exiting 15:18:25 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:10:31 (8200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:36 (8200): No heartbeat from core client for 30 sec - exiting 17:10:38 (8200): No heartbeat from core client for 30 sec - exiting 17:10:39 (8200): No heartbeat from core client for 30 sec - exiting 17:10:40 (8200): No heartbeat from core client for 30 sec - exiting 17:10:42 (8200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:32:13 (11588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:32:20 (11588): No heartbeat from core client for 30 sec - exiting 23:32:23 (11588): No heartbeat from core client for 30 sec - exiting 23:32:24 (11588): No heartbeat from core client for 30 sec - exiting 23:32:25 (11588): No heartbeat from core client for 30 sec - exiting 23:32:29 (11588): No heartbeat from core client for 30 sec - exiting 23:32:30 (11588): No heartbeat from core client for 30 sec - exiting 23:32:31 (11588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:13:16 (12940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:13:18 (12940): No heartbeat from core client for 30 sec - exiting 13:13:19 (12940): No heartbeat from core client for 30 sec - exiting 13:13:20 (12940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:58:59 (11740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:41 (11432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:44 (11432): No heartbeat from core client for 30 sec - exiting 22:00:45 (11432): No heartbeat from core client for 30 sec - exiting 22:00:46 (11432): No heartbeat from core client for 30 sec - exiting 22:02:43 (12708): No heartbeat from core client for 30 sec - exiting 22:02:44 (12708): No heartbeat from core client for 30 sec - exiting 22:02:45 (12708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:00 (10344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:01 (10344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:48:15 (11656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:28:31 (12172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:28:38 (12172): No heartbeat from core client for 30 sec - exiting 18:28:40 (12172): No heartbeat from core client for 30 sec - exiting 18:28:41 (12172): No heartbeat from core client for 30 sec - exiting 18:28:43 (12172): No heartbeat from core client for 30 sec - exiting 18:28:44 (12172): No heartbeat from core client for 30 sec - exiting 18:28:45 (12172): No heartbeat from core client for 30 sec - exiting 18:28:46 (12172): No heartbeat from core client for 30 sec - exiting 18:28:47 (12172): No heartbeat from core client for 30 sec - exiting 18:28:48 (12172): No heartbeat from core client for 30 sec - exiting 18:28:49 (12172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:47:20 (10412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:47:27 (10412): No heartbeat from core client for 30 sec - exiting 06:47:28 (10412): No heartbeat from core client for 30 sec - exiting 06:47:29 (10412): No heartbeat from core client for 30 sec - exiting 06:47:30 (10412): No heartbeat from core client for 30 sec - exiting 06:47:31 (10412): No heartbeat from core client for 30 sec - exiting 06:47:32 (10412): No heartbeat from core client for 30 sec - exiting 06:47:33 (10412): No heartbeat from core client for 30 sec - exiting 06:47:34 (10412): No heartbeat from core client for 30 sec - exiting 06:47:35 (10412): No heartbeat from core client for 30 sec - exiting 06:47:37 (10412): No heartbeat from core client for 30 sec - exiting 06:47:38 (10412): No heartbeat from core client for 30 sec - exiting 06:47:39 (10412): No heartbeat from core client for 30 sec - exiting 06:51:34 (6352): No heartbeat from core client for 30 sec - exiting 06:51:35 (6352): No heartbeat from core client for 30 sec - exiting 06:51:36 (6352): No heartbeat from core client for 30 sec - exiting 06:51:37 (6352): No heartbeat from core client for 30 sec - exiting 06:51:38 (6352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Oct 2013 16:13:14	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	207,360	462,454	2.2302
28 Oct 2013 13:24:25	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	181,440	404,875	2.2315
27 Oct 2013 13:05:25	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	155,520	347,675	2.2356
26 Oct 2013 11:38:18	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	129,600	290,426	2.2409
25 Oct 2013 07:27:42	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	103,680	232,002	2.2377
24 Oct 2013 05:07:16	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	77,760	173,908	2.2365
23 Oct 2013 04:25:17	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	51,840	115,620	2.2303
21 Oct 2013 16:20:18	1125333	16024838	hadcm3n_84a1_1980_40_008463501_0	25,920	57,812	2.2304