Task 15769650

Name	hadcm3n_4bqm_1940_40_008307601_1
Workunit	8458736
Created	9 May 2013, 21:47:35 UTC
Sent	9 May 2013, 21:47:50 UTC
Report deadline	9 Aug 2013, 5:15:01 UTC
Received	26 May 2013, 19:44:49 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	459505
Run time	9 days 19 hours 8 min 50 sec
CPU time	9 days 10 hours 15 min 47 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.41 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 05:27:49 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:27:50 (5960): No heartbeat from core client for 30 sec - exiting 12:03:15 (3168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:52:59 (2736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:53:01 (2736): No heartbeat from core client for 30 sec - exiting 21:46:20 (4224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:46:21 (4224): No heartbeat from core client for 30 sec - exiting 21:46:22 (4224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:03:03 (4244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:33:02 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:33:58 (6084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:33:59 (6084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:06:49 (4624): No heartbeat from core client for 30 sec - exiting 20:06:50 (4624): No heartbeat from core client for 30 sec - exiting 20:06:51 (4624): No heartbeat from core client for 30 sec - exiting 20:06:52 (4624): No heartbeat from core client for 30 sec - exiting 20:06:53 (4624): No heartbeat from core client for 30 sec - exiting 20:06:54 (4624): No heartbeat from core client for 30 sec - exiting 20:06:55 (4624): No heartbeat from core client for 30 sec - exiting 20:06:57 (4624): No heartbeat from core client for 30 sec - exiting 20:06:58 (4624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4bqmko.pje5c10 Error converting file to netcdf: dataout/4bqmko.pie5c10 Error converting file to netcdf: dataout/4bqmko.pfe5c10 Error converting file to netcdf: dataout/4bqmka.phe5c10 Error converting file to netcdf: dataout/4bqmka.pge5c10 Error converting file to netcdf: dataout/4bqmka.pee5c10 Error converting file to netcdf: dataout/4bqmka.pde5c10 CPDN Monitor - Quit request from BOINC... 09:22:20 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:40:21 (3836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:16:38 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:20:56 (5272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:20:57 (5272): No heartbeat from core client for 30 sec - exiting 16:20:58 (5272): No heartbeat from core client for 30 sec - exiting 16:20:59 (5272): No heartbeat from core client for 30 sec - exiting 17:53:31 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:25:09 (5872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:25 (1484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:26 (1484): No heartbeat from core client for 30 sec - exiting 00:57:02 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:57:03 (5052): No heartbeat from core client for 30 sec - exiting 04:25:56 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:25:57 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:39:21 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:39:22 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:26:12 (2624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:26:13 (2624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:19:51 (4908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:15:05 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:15:07 (3968): No heartbeat from core client for 30 sec - exiting 10:15:08 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:37:27 (1556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:45:32 (2680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_4bqm_1940_40_008307601/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 May 2013 18:45:50	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	259,200	814,643	3.1429
24 May 2013 11:58:54	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	233,280	731,819	3.1371
18 May 2013 03:46:41	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	207,360	650,163	3.1354
17 May 2013 01:57:40	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	181,440	568,102	3.1311
16 May 2013 02:24:56	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	155,520	486,550	3.1285
15 May 2013 01:12:55	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	129,600	405,159	3.1262
13 May 2013 22:14:16	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	103,680	324,331	3.1282
12 May 2013 21:53:21	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	77,760	242,596	3.1198
11 May 2013 22:07:05	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	51,840	162,304	3.1309
10 May 2013 22:52:34	459505	15769650	hadcm3n_4bqm_1940_40_008307601_1	25,920	81,707	3.1523