 |
|
| Name | hadcm3n_yi5n_1900_40_007356389_2 |
| Workunit | 7553819 |
| Created | 17 Jul 2011, 15:45:51 UTC |
| Sent | 17 Jul 2011, 15:48:48 UTC |
| Report deadline | 16 Oct 2011, 23:15:59 UTC |
| Received | 28 Aug 2011, 21:26:56 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1081404 |
| Run time | 29 days 0 hours 14 min 44 sec |
| CPU time | 24 days 12 hours 12 min |
| Validate state | Invalid |
| Credit | 12,441.60 |
| Device peak FLOPS | 2.53 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
| Stderr | <core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:55:16 (4628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:14:29 (3420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:24:55 (4064): No heartbeat from core client for 30 sec - exiting 07:24:56 (4064): No heartbeat from core client for 30 sec - exiting 07:24:57 (4064): No heartbeat from core client for 30 sec - exiting 07:24:58 (4064): No heartbeat from core client for 30 sec - exiting 07:24:59 (4064): No heartbeat from core client for 30 sec - exiting 07:25:00 (4064): No heartbeat from core client for 30 sec - exiting 07:25:01 (4064): No heartbeat from core client for 30 sec - exiting 07:25:02 (4064): No heartbeat from core client for 30 sec - exiting 07:25:03 (4064): No heartbeat from core client for 30 sec - exiting 07:25:04 (4064): No heartbeat from core client for 30 sec - exiting 07:25:06 (4064): No heartbeat from core client for 30 sec - exiting 07:25:07 (4064): No heartbeat from core client for 30 sec - exiting 07:25:08 (4064): No heartbeat from core client for 30 sec - exiting 07:25:09 (4064): No heartbeat from core client for 30 sec - exiting 07:25:10 (4064): No heartbeat from core client for 30 sec - exiting 07:25:11 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:32:52 (3804): No heartbeat from core client for 30 sec - exiting 16:32:53 (3804): No heartbeat from core client for 30 sec - exiting 16:32:55 (3804): No heartbeat from core client for 30 sec - exiting 16:32:56 (3804): No heartbeat from core client for 30 sec - exiting 16:32:57 (3804): No heartbeat from core client for 30 sec - exiting 16:32:58 (3804): No heartbeat from core client for 30 sec - exiting 16:32:59 (3804): No heartbeat from core client for 30 sec - exiting 16:33:00 (3804): No heartbeat from core client for 30 sec - exiting 16:33:01 (3804): No heartbeat from core client for 30 sec - exiting 16:33:02 (3804): No heartbeat from core client for 30 sec - exiting 16:33:03 (3804): No heartbeat from core client for 30 sec - exiting 16:33:04 (3804): No heartbeat from core client for 30 sec - exiting 16:33:06 (3804): No heartbeat from core client for 30 sec - exiting 16:33:07 (3804): No heartbeat from core client for 30 sec - exiting 16:33:08 (3804): No heartbeat from core client for 30 sec - exiting 16:33:09 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:55:04 (572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:47:21 (3268): No heartbeat from core client for 30 sec - exiting 12:47:22 (3268): No heartbeat from core client for 30 sec - exiting 12:47:23 (3268): No heartbeat from core client for 30 sec - exiting 12:47:24 (3268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:09:50 (3456): No heartbeat from core client for 30 sec - exiting 18:09:51 (3456): No heartbeat from core client for 30 sec - exiting 18:09:52 (3456): No heartbeat from core client for 30 sec - exiting 18:09:53 (3456): No heartbeat from core client for 30 sec - exiting 18:09:55 (3456): No heartbeat from core client for 30 sec - exiting 18:09:56 (3456): No heartbeat from core client for 30 sec - exiting 18:09:57 (3456): No heartbeat from core client for 30 sec - exiting 18:09:58 (3456): No heartbeat from core client for 30 sec - exiting 18:09:59 (3456): No heartbeat from core client for 30 sec - exiting 18:10:00 (3456): No heartbeat from core client for 30 sec - exiting 18:10:01 (3456): No heartbeat from core client for 30 sec - exiting 18:10:02 (3456): No heartbeat from core client for 30 sec - exiting 18:10:03 (3456): No heartbeat from core client for 30 sec - exiting 18:10:04 (3456): No heartbeat from core client for 30 sec - exiting 18:10:05 (3456): No heartbeat from core client for 30 sec - exiting 18:10:07 (3456): No heartbeat from core client for 30 sec - exiting 18:10:08 (3456): No heartbeat from core client for 30 sec - exiting 18:10:09 (3456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:49:56 (3016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:33:46 (4476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:01:55 (2456): No heartbeat from core client for 30 sec - exiting 17:01:56 (2456): No heartbeat from core client for 30 sec - exiting 17:01:57 (2456): No heartbeat from core client for 30 sec - exiting 17:01:58 (2456): No heartbeat from core client for 30 sec - exiting 17:01:59 (2456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:34:24 (4788): No heartbeat from core client for 30 sec - exiting 20:34:25 (4788): No heartbeat from core client for 30 sec - exiting 20:34:26 (4788): No heartbeat from core client for 30 sec - exiting 20:34:27 (4788): No heartbeat from core client for 30 sec - exiting 20:34:28 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:08:40 (2588): No heartbeat from core client for 30 sec - exiting 07:08:41 (2588): No heartbeat from core client for 30 sec - exiting 07:08:42 (2588): No heartbeat from core client for 30 sec - exiting 07:08:43 (2588): No heartbeat from core client for 30 sec - exiting 07:08:44 (2588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:03:27 (4148): No heartbeat from core client for 30 sec - exiting 11:03:29 (4148): No heartbeat from core client for 30 sec - exiting 11:03:30 (4148): No heartbeat from core client for 30 sec - exiting 11:03:31 (4148): No heartbeat from core client for 30 sec - exiting 11:03:32 (4148): No heartbeat from core client for 30 sec - exiting 11:03:33 (4148): No heartbeat from core client for 30 sec - exiting 11:03:34 (4148): No heartbeat from core client for 30 sec - exiting 11:03:35 (4148): No heartbeat from core client for 30 sec - exiting 11:03:36 (4148): No heartbeat from core client for 30 sec - exiting 11:03:37 (4148): No heartbeat from core client for 30 sec - exiting 11:03:38 (4148): No heartbeat from core client for 30 sec - exiting 11:03:39 (4148): No heartbeat from core client for 30 sec - exiting 11:03:41 (4148): No heartbeat from core client for 30 sec - exiting 11:03:42 (4148): No heartbeat from core client for 30 sec - exiting 11:03:43 (4148): No heartbeat from core client for 30 sec - exiting 11:03:44 (4148): No heartbeat from core client for 30 sec - exiting 11:03:45 (4148): No heartbeat from core client for 30 sec - exiting 11:03:46 (4148): No heartbeat from core client for 30 sec - exiting 11:03:47 (4148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:58:18 (5040): No heartbeat from core client for 30 sec - exiting 09:58:19 (5040): No heartbeat from core client for 30 sec - exiting 09:58:20 (5040): No heartbeat from core client for 30 sec - exiting 09:58:22 (5040): No heartbeat from core client for 30 sec - exiting 09:58:23 (5040): No heartbeat from core client for 30 sec - exiting 09:58:24 (5040): No heartbeat from core client for 30 sec - exiting 09:58:25 (5040): No heartbeat from core client for 30 sec - exiting 09:58:26 (5040): No heartbeat from core client for 30 sec - exiting 09:58:27 (5040): No heartbeat from core client for 30 sec - exiting 09:58:28 (5040): No heartbeat from core client for 30 sec - exiting 09:58:29 (5040): No heartbeat from core client for 30 sec - exiting 09:58:30 (5040): No heartbeat from core client for 30 sec - exiting 09:58:31 (5040): No heartbeat from core client for 30 sec - exiting 09:58:32 (5040): No heartbeat from core client for 30 sec - exiting 09:58:34 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:51:27 (5000): No heartbeat from core client for 30 sec - exiting 15:51:28 (5000): No heartbeat from core client for 30 sec - exiting 15:51:29 (5000): No heartbeat from core client for 30 sec - exiting 15:51:30 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:27:24 (3148): No heartbeat from core client for 30 sec - exiting 22:27:25 (3148): No heartbeat from core client for 30 sec - exiting 22:27:26 (3148): No heartbeat from core client for 30 sec - exiting 22:27:27 (3148): No heartbeat from core client for 30 sec - exiting 22:27:28 (3148): No heartbeat from core client for 30 sec - exiting 22:27:29 (3148): No heartbeat from core client for 30 sec - exiting 22:27:31 (3148): No heartbeat from core client for 30 sec - exiting 22:27:32 (3148): No heartbeat from core client for 30 sec - exiting 22:27:33 (3148): No heartbeat from core client for 30 sec - exiting 22:27:34 (3148): No heartbeat from core client for 30 sec - exiting 22:27:35 (3148): No heartbeat from core client for 30 sec - exiting 22:27:36 (3148): No heartbeat from core client for 30 sec - exiting 22:27:37 (3148): No heartbeat from core client for 30 sec - exiting 22:27:38 (3148): No heartbeat from core client for 30 sec - exiting 22:27:39 (3148): No heartbeat from core client for 30 sec - exiting 22:27:40 (3148): No heartbeat from core client for 30 sec - exiting 22:27:42 (3148): No heartbeat from core client for 30 sec - exiting 22:27:43 (3148): No heartbeat from core client for 30 sec - exiting 22:27:44 (3148): No heartbeat from core client for 30 sec - exiting 22:27:45 (3148): No heartbeat from core client for 30 sec - exiting 22:27:46 (3148): No heartbeat from core client for 30 sec - exiting 22:27:47 (3148): No heartbeat from core client for 30 sec - exiting 22:27:48 (3148): No heartbeat from core client for 30 sec - exiting 22:27:49 (3148): No heartbeat from core client for 30 sec - exiting 22:27:50 (3148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:43:42 (2144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:18:09 (4956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:46:56 (4248): No heartbeat from core client for 30 sec - exiting 14:46:57 (4248): No heartbeat from core client for 30 sec - exiting 14:46:58 (4248): No heartbeat from core client for 30 sec - exiting 14:46:59 (4248): No heartbeat from core client for 30 sec - exiting 14:47:00 (4248): No heartbeat from core client for 30 sec - exiting 14:47:01 (4248): No heartbeat from core client for 30 sec - exiting 14:47:02 (4248): No heartbeat from core client for 30 sec - exiting 14:47:03 (4248): No heartbeat from core client for 30 sec - exiting 14:47:04 (4248): No heartbeat from core client for 30 sec - exiting 14:47:05 (4248): No heartbeat from core client for 30 sec - exiting 14:47:06 (4248): No heartbeat from core client for 30 sec - exiting 14:47:08 (4248): No heartbeat from core client for 30 sec - exiting 14:47:09 (4248): No heartbeat from core client for 30 sec - exiting 14:47:10 (4248): No heartbeat from core client for 30 sec - exiting 14:47:11 (4248): No heartbeat from core client for 30 sec - exiting 14:47:12 (4248): No heartbeat from core client for 30 sec - exiting 14:47:13 (4248): No heartbeat from core client for 30 sec - exiting 14:47:14 (4248): No heartbeat from core client for 30 sec - exiting 14:47:15 (4248): No heartbeat from core client for 30 sec - exiting 14:47:16 (4248): No heartbeat from core client for 30 sec - exiting 14:47:17 (4248): No heartbeat from core client for 30 sec - exiting 14:47:18 (4248): No heartbeat from core client for 30 sec - exiting 14:47:20 (4248): No heartbeat from core client for 30 sec - exiting 14:47:21 (4248): No heartbeat from core client for 30 sec - exiting 14:47:22 (4248): No heartbeat from core client for 30 sec - exiting 14:47:23 (4248): No heartbeat from core client for 30 sec - exiting 14:47:24 (4248): No heartbeat from core client for 30 sec - exiting 14:47:25 (4248): No heartbeat from core client for 30 sec - exiting 14:47:26 (4248): No heartbeat from core client for 30 sec - exiting 14:47:27 (4248): No heartbeat from core client for 30 sec - exiting 14:47:28 (4248): No heartbeat from core client for 30 sec - exiting 14:47:29 (4248): No heartbeat from core client for 30 sec - exiting 14:47:30 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yi5nko.pjd2c10 Error converting file to netcdf: dataout/yi5nko.pid2c10 Error converting file to netcdf: dataout/yi5nko.pfd2c10 Error converting file to netcdf: dataout/yi5nka.phd2c10 Error converting file to netcdf: dataout/yi5nka.pgd2c10 Error converting file to netcdf: dataout/yi5nka.ped2c10 Error converting file to netcdf: dataout/yi5nka.pdd2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:15:28 (3132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:09:12 (4680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:36:33 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x777E3F79 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file K:\BOINC\DATA/projects/climateprediction.net/hadcm3n_yi5n_1900_40_007356389/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 28 Aug 2011 20:16:57 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 1,036,800 | 2,117,502 | 2.0423 |
| 27 Aug 2011 20:19:34 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 1,010,880 | 2,063,314 | 2.0411 |
| 26 Aug 2011 19:38:57 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 984,960 | 2,011,180 | 2.0419 |
| 25 Aug 2011 20:25:36 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 959,040 | 1,958,609 | 2.0423 |
| 24 Aug 2011 23:18:40 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 933,120 | 1,905,887 | 2.0425 |
| 24 Aug 2011 01:36:42 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 907,200 | 1,852,959 | 2.0425 |
| 23 Aug 2011 05:45:23 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 881,280 | 1,800,024 | 2.0425 |
| 22 Aug 2011 06:37:13 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 855,360 | 1,746,933 | 2.0423 |
| 21 Aug 2011 06:40:07 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 829,440 | 1,693,680 | 2.0420 |
| 20 Aug 2011 05:14:03 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 803,520 | 1,639,394 | 2.0403 |
| 19 Aug 2011 02:13:49 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 777,600 | 1,586,544 | 2.0403 |
| 18 Aug 2011 08:17:21 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 751,680 | 1,534,263 | 2.0411 |
| 17 Aug 2011 08:17:29 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 725,760 | 1,479,184 | 2.0381 |
| 16 Aug 2011 08:29:13 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 699,840 | 1,420,596 | 2.0299 |
| 15 Aug 2011 08:31:48 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 673,920 | 1,369,465 | 2.0321 |
| 14 Aug 2011 09:29:37 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 648,000 | 1,316,748 | 2.0320 |
| 13 Aug 2011 08:13:03 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 622,080 | 1,264,653 | 2.0329 |
| 11 Aug 2011 19:06:05 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 596,160 | 1,211,929 | 2.0329 |
| 10 Aug 2011 17:01:08 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 570,240 | 1,160,357 | 2.0349 |
| 09 Aug 2011 15:25:41 | 1081404 | 13143738 | hadcm3n_yi5n_1900_40_007356389_2 | 544,320 | 1,105,994 | 2.0319 |
©2024 cpdn.org