Task 12016848

Name	hadcm3igeo_w1hs_2000_80_06761121_2
Workunit	6964437
Created	12 Nov 2010, 15:32:03 UTC
Sent	14 Nov 2010, 9:11:39 UTC
Report deadline	15 May 2012, 5:54:48 UTC
Received	5 Feb 2011, 15:35:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1111520
Run time	33 days 14 hours 35 min 51 sec
CPU time	27 days 1 hours 55 min 1 sec
Validate state	Invalid
Credit	9,797.76
Device peak FLOPS	1.86 GFLOPS
Application version	HadCM3L Coupled Model Experiment Optimised File I/O v6.04 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=8108, selfPID=8108, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error in converting file dataout/w1hsfo.pjk2c10 to netcdf format. Error in converting file dataout/w1hsfo.pik2c10 to netcdf format. Error in converting file dataout/w1hsfo.pfk2c10 to netcdf format. Error in converting file dataout/w1hsfa.phk2c10 to netcdf format. Error in converting file dataout/w1hsfa.pgk2c10 to netcdf format. Error in converting file dataout/w1hsfa.pek2c10 to netcdf format. Error in converting file dataout/w1hsfa.pdk2c10 to netcdf format. scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz error: cannot delete old hadcm3igeo_w1hs_2000_80_06761121/jobs/yafbg.PRESM_O error: cannot delete old hadcm3igeo_w1hs_2000_80_06761121/datain/ancil/basin.index error: cannot delete old hadcm3igeo_w1hs_2000_80_06761121/jobs/yafbg.namelists forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3664, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3664, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3664, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN Monitor - Quit request from BOINC... scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... scan: 0ajx_16_2000_atmos.gz scan: 0ajx_16_2000_ocean.gz scan: 1063_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1hs_2000_80_06761121.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: heblhdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Feb 2011 10:00:08	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	777,600	2,278,686	2.9304
02 Feb 2011 23:03:06	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	751,680	2,203,568	2.9315
01 Feb 2011 12:47:35	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	725,760	2,128,527	2.9328
30 Jan 2011 22:04:00	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	699,840	2,052,776	2.9332
28 Jan 2011 19:12:00	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	673,920	1,975,556	2.9314
27 Jan 2011 11:21:59	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	648,000	1,899,671	2.9316
27 Dec 2010 18:39:26	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	622,080	1,822,193	2.9292
25 Dec 2010 18:32:05	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	596,160	1,747,164	2.9307
23 Dec 2010 11:02:07	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	570,240	1,672,230	2.9325
20 Dec 2010 18:16:06	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	544,320	1,594,260	2.9289
19 Dec 2010 10:49:20	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	518,400	1,515,964	2.9243
17 Dec 2010 17:13:55	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	492,480	1,439,824	2.9236
15 Dec 2010 13:27:54	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	466,560	1,364,055	2.9236
14 Dec 2010 10:09:04	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	440,640	1,286,089	2.9187
12 Dec 2010 12:56:25	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	414,720	1,206,176	2.9084
11 Dec 2010 00:00:48	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	388,800	1,128,544	2.9026
07 Dec 2010 19:07:21	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	362,880	1,050,568	2.8951
06 Dec 2010 12:28:00	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	336,960	974,424	2.8918
05 Dec 2010 09:04:14	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	311,040	897,303	2.8848
03 Dec 2010 21:19:49	1111520	12016848	hadcm3igeo_w1hs_2000_80_06761121_2	285,120	821,196	2.8802