Task 19228170

Name	hadam3p_anz_a2at_201412_12_313_010278182_0
Workunit	10278182
Created	28 Jan 2016, 10:48:51 UTC
Sent	28 Jan 2016, 23:37:28 UTC
Report deadline	10 Jan 2017, 4:57:28 UTC
Received	3 Feb 2016, 17:35:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1388347
Run time	3 days 10 hours 38 min 1 sec
CPU time	3 days 7 hours 35 min 12 sec
Validate state	Invalid
Credit	2,993.82
Device peak FLOPS	3.19 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 18:59:56 (2224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:01:40 (9068): start_timer_thread(): CreateThread() failed, errno 0 19:01:43 (4504): start_timer_thread(): CreateThread() failed, errno 0 19:57:59 (1644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:59:51 (10028): start_timer_thread(): CreateThread() failed, errno 0 20:56:03 (11056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:57:51 (6400): start_timer_thread(): CreateThread() failed, errno 0 22:51:59 (3708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:53:43 (10000): start_timer_thread(): CreateThread() failed, errno 0 23:49:56 (924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:51:47 (10852): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 01:16:42 (8592): start_timer_thread(): CreateThread() failed, errno 0 01:16:44 (6856): start_timer_thread(): CreateThread() failed, errno 0 02:46:46 (10620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:30 (8796): start_timer_thread(): CreateThread() failed, errno 0 02:48:32 (6852): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 05:40:36 (11468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:42:26 (11236): start_timer_thread(): CreateThread() failed, errno 0 05:42:28 (9872): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 06:38:33 (10404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:14 (12172): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 07:36:32 (4268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:38:32 (11340): start_timer_thread(): CreateThread() failed, errno 0 07:38:34 (3588): start_timer_thread(): CreateThread() failed, errno 0 13:24:18 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:25:56 (9876): start_timer_thread(): CreateThread() failed, errno 0 13:25:59 (8660): start_timer_thread(): CreateThread() failed, errno 0 14:22:25 (8312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:23:52 (6608): start_timer_thread(): CreateThread() failed, errno 0 14:23:54 (956): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 03:38:13 (10004): start_timer_thread(): CreateThread() failed, errno 0 03:38:15 (1796): start_timer_thread(): CreateThread() failed, errno 0 04:52:04 (14924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:53:37 (9480): start_timer_thread(): CreateThread() failed, errno 0 05:50:05 (12488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:51:35 (9816): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 10:04:02 (14272): start_timer_thread(): CreateThread() failed, errno 0 10:04:03 (12348): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:07:40 (13348): start_timer_thread(): CreateThread() failed, errno 0 15:29:38 (12440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:28 (5244): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:11:31 (14412): start_timer_thread(): CreateThread() failed, errno 0 21:11:33 (15636): start_timer_thread(): CreateThread() failed, errno 0 23:13:53 (12480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:15:27 (7104): start_timer_thread(): CreateThread() failed, errno 0 23:15:29 (15516): start_timer_thread(): CreateThread() failed, errno 0 00:11:50 (14028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:13:21 (12364): start_timer_thread(): CreateThread() failed, errno 0 00:13:23 (7004): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:30:44 (6152): start_timer_thread(): CreateThread() failed, errno 0 02:10:22 (15472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:27 (16824): start_timer_thread(): CreateThread() failed, errno 0 03:08:22 (13532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:09:59 (15616): start_timer_thread(): CreateThread() failed, errno 0 05:04:20 (11840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:02:19 (16960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:03:58 (15536): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:16:41 (16640): start_timer_thread(): CreateThread() failed, errno 0 12:16:43 (16892): start_timer_thread(): CreateThread() failed, errno 0 12:48:00 (17632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:49:48 (17660): start_timer_thread(): CreateThread() failed, errno 0 15:42:03 (18348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:52 (17768): start_timer_thread(): CreateThread() failed, errno 0 15:43:54 (13828): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:38:06 (17628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:01 (15996): start_timer_thread(): CreateThread() failed, errno 0 18:36:12 (19232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:37:44 (19172): start_timer_thread(): CreateThread() failed, errno 0 19:34:19 (15748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:35:55 (18664): start_timer_thread(): CreateThread() failed, errno 0 19:35:56 (17668): start_timer_thread(): CreateThread() failed, errno 0 20:32:25 (2908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:59 (9896): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:34:15 (5280): start_timer_thread(): CreateThread() failed, errno 0 21:34:18 (21732): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:37:25 (15780): start_timer_thread(): CreateThread() failed, errno 0 23:37:28 (16772): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:47:00 (25536): start_timer_thread(): CreateThread() failed, errno 0 00:47:02 (25020): start_timer_thread(): CreateThread() failed, errno 0 01:36:35 (17600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:37:18 (23176): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 02:34:31 (24832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:36:15 (23660): start_timer_thread(): CreateThread() failed, errno 0 03:32:35 (21564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:34:19 (19820): start_timer_thread(): CreateThread() failed, errno 0 05:28:33 (21752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:30:31 (21008): start_timer_thread(): CreateThread() failed, errno 0 08:22:25 (23572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:24:22 (4216): start_timer_thread(): CreateThread() failed, errno 0 13:12:20 (25388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:14:17 (22604): start_timer_thread(): CreateThread() failed, errno 0 14:10:29 (24204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:12:05 (24728): start_timer_thread(): CreateThread() failed, errno 0 16:06:28 (17748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:08:15 (23512): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:04:31 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:00:37 (15812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:02:30 (23720): start_timer_thread(): CreateThread() failed, errno 0 21:54:37 (21228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:56:25 (2960): start_timer_thread(): CreateThread() failed, errno 0 22:52:46 (16376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:52:18 (10364): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 00:25:56 (21824): start_timer_thread(): CreateThread() failed, errno 0 00:25:58 (19160): start_timer_thread(): CreateThread() failed, errno 0 01:48:09 (21352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:50:31 (22624): start_timer_thread(): CreateThread() failed, errno 0 01:50:33 (23280): start_timer_thread(): CreateThread() failed, errno 0 02:46:17 (19012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:47:41 (15396): start_timer_thread(): CreateThread() failed, errno 0 02:47:42 (21680): start_timer_thread(): CreateThread() failed, errno 0 03:44:23 (21984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:42:28 (16972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:38:31 (22912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:32:32 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:34:20 (21088): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=21088, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=22844, selfPID=22844, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=22844, selfPID=21272, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_a2at_201412_12_313_010278182_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2at_201412_12_313_010278182_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2at_201412_12_313_010278182_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2at_201412_12_313_010278182_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2at_201412_12_313_010278182_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2at_201412_12_313_010278182_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Feb 2016 14:03:22	1388347	19228170	hadam3p_anz_a2at_201412_12_313_010278182_0	69,419	278,026	4.0050
02 Feb 2016 21:59:15	1388347	19228170	hadam3p_anz_a2at_201412_12_313_010278182_0	57,899	230,396	3.9793
02 Feb 2016 08:09:05	1388347	19228170	hadam3p_anz_a2at_201412_12_313_010278182_0	46,379	185,347	3.9964
31 Jan 2016 13:54:13	1388347	19228170	hadam3p_anz_a2at_201412_12_313_010278182_0	34,859	137,443	3.9428
30 Jan 2016 17:28:31	1388347	19228170	hadam3p_anz_a2at_201412_12_313_010278182_0	23,339	89,774	3.8465
29 Jan 2016 15:41:41	1388347	19228170	hadam3p_anz_a2at_201412_12_313_010278182_0	11,819	46,516	3.9357