 |
|
| Name | hadcm3n_o7fy_1900_40_007204977_2 |
| Workunit | 7403257 |
| Created | 28 Jun 2011, 0:03:43 UTC |
| Sent | 28 Jun 2011, 0:03:52 UTC |
| Report deadline | 27 Sep 2011, 7:31:03 UTC |
| Received | 18 Jul 2011, 1:50:34 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 22 (0x00000016) Unknown error code |
| Computer ID | 1159155 |
| Run time | 12 days 17 hours 8 min 19 sec |
| CPU time | 12 days 2 hours 13 min 17 sec |
| Validate state | Invalid |
| Credit | 12,441.60 |
| Device peak FLOPS | 3.45 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
| Stderr | <core_client_version>6.12.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4344, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 04:17:00 (3404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:34:19 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:50:00 (5240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:50:54 (9096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o7fyko.pjd0c10 Error converting file to netcdf: dataout/o7fyko.pid0c10 Error converting file to netcdf: dataout/o7fyko.pfd0c10 Error converting file to netcdf: dataout/o7fyko.pcd0c10 Error converting file to netcdf: dataout/o7fyko.pbd0c10 Error converting file to netcdf: dataout/o7fyko.pad0c10 Error converting file to netcdf: dataout/o7fyka.phd0c10 Error converting file to netcdf: dataout/o7fyka.pgd0c10 Error converting file to netcdf: dataout/o7fyka.ped0c10 Error converting file to netcdf: dataout/o7fyka.pdd0c10 03:46:57 (5788): Can't acquire lockfile (32) - waiting 35s 03:47:17 (2172): No heartbeat from core client for 30 sec - exiting 03:47:18 (2172): No heartbeat from core client for 30 sec - exiting 03:47:19 (2172): No heartbeat from core client for 30 sec - exiting 03:47:20 (2172): No heartbeat from core client for 30 sec - exiting 03:47:21 (2172): No heartbeat from core client for 30 sec - exiting 03:47:22 (2172): No heartbeat from core client for 30 sec - exiting 03:47:23 (2172): No heartbeat from core client for 30 sec - exiting 03:47:24 (2172): No heartbeat from core client for 30 sec - exiting 03:47:25 (2172): No heartbeat from core client for 30 sec - exiting 03:47:26 (2172): No heartbeat from core client for 30 sec - exiting 03:47:27 (2172): No heartbeat from core client for 30 sec - exiting 03:47:28 (2172): No heartbeat from core client for 30 sec - exiting 03:47:29 (2172): No heartbeat from core client for 30 sec - exiting 03:47:30 (2172): No heartbeat from core client for 30 sec - exiting 03:47:31 (2172): No heartbeat from core client for 30 sec - exiting 03:47:32 (5788): Can't acquire lockfile (32) - exiting 03:47:32 (5788): Error: The process cannot access the file because it is being used by another process. (0x20) 03:47:32 (2172): No heartbeat from core client for 30 sec - exiting 03:47:33 (2172): No heartbeat from core client for 30 sec - exiting 03:47:34 (2172): No heartbeat from core client for 30 sec - exiting 03:47:35 (2172): No heartbeat from core client for 30 sec - exiting 03:47:36 (2172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:47:37 (2172): No heartbeat from core client for 30 sec - exiting cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o7fy_1900_40_007204977/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 31 Jul 2011 12:53:43 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 1,036,800 | 342,199 | 0.3301 |
| 31 Jul 2011 01:34:39 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 1,010,880 | 307,032 | 0.3037 |
| 30 Jul 2011 11:25:20 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 984,960 | 272,014 | 0.2762 |
| 29 Jul 2011 20:02:29 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 959,040 | 237,142 | 0.2473 |
| 29 Jul 2011 06:59:30 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 933,120 | 202,229 | 0.2167 |
| 28 Jul 2011 18:51:40 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 907,200 | 167,256 | 0.1844 |
| 28 Jul 2011 05:01:55 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 881,280 | 132,626 | 0.1505 |
| 27 Jul 2011 10:54:39 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 855,360 | 98,027 | 0.1146 |
| 26 Jul 2011 21:23:12 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 829,440 | 63,220 | 0.0762 |
| 26 Jul 2011 01:25:25 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 803,520 | 30,023 | 0.0374 |
| 25 Jul 2011 16:39:58 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 777,600 | 1,044,824 | 1.3437 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 751,680 | 1,011,300 | 1.3454 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 725,760 | 976,989 | 1.3462 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 699,840 | 941,592 | 1.3454 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 673,920 | 906,319 | 1.3448 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 648,000 | 871,376 | 1.3447 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 622,080 | 836,844 | 1.3452 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 596,160 | 802,454 | 1.3460 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 570,240 | 768,156 | 1.3471 |
| 25 Jul 2011 13:02:16 | 1159155 | 13014229 | hadcm3n_o7fy_1900_40_007204977_2 | 544,320 | 733,243 | 1.3471 |
©2024 climateprediction.net