 |
|
| Name | hadcm3n_u1qk_1980_40_007544501_2 |
| Workunit | 7741733 |
| Created | 15 Nov 2011, 21:53:07 UTC |
| Sent | 15 Nov 2011, 21:55:15 UTC |
| Report deadline | 15 Feb 2012, 5:22:26 UTC |
| Received | 10 Dec 2011, 13:21:29 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1156207 |
| Run time | 14 days 20 hours 38 min 1 sec |
| CPU time | 12 days 1 hours 40 min 19 sec |
| Validate state | Invalid |
| Credit | 6,220.80 |
| Device peak FLOPS | 2.43 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
| Stderr | <core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:31:49 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:31:51 (7464): No heartbeat from core client for 30 sec - exiting 14:31:52 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 18:01:06 (10188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:07 (10188): No heartbeat from core client for 30 sec - exiting 21:56:28 (7444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:58:01 (2812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:29 (7012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:30 (7012): No heartbeat from core client for 30 sec - exiting Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... 16:02:34 (9604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:02:35 (9604): No heartbeat from core client for 30 sec - exiting 16:02:36 (9604): No heartbeat from core client for 30 sec - exiting 18:05:24 (8404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:25 (8404): No heartbeat from core client for 30 sec - exiting 18:53:17 (7504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:18 (7504): No heartbeat from core client for 30 sec - exiting 19:45:55 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:56 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 21:47:44 (7224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:45 (7224): No heartbeat from core client for 30 sec - exiting 21:47:46 (7224): No heartbeat from core client for 30 sec - exiting 21:47:47 (7224): No heartbeat from core client for 30 sec - exiting 21:47:48 (7224): No heartbeat from core client for 30 sec - exiting 21:47:49 (7224): No heartbeat from core client for 30 sec - exiting 21:47:50 (7224): No heartbeat from core client for 30 sec - exiting 21:47:51 (7224): No heartbeat from core client for 30 sec - exiting 21:47:52 (7224): No heartbeat from core client for 30 sec - exiting 21:47:53 (7224): No heartbeat from core client for 30 sec - exiting 21:47:54 (7224): No heartbeat from core client for 30 sec - exiting Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 23:12:48 (10048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:49 (10048): No heartbeat from core client for 30 sec - exiting 23:12:50 (10048): No heartbeat from core client for 30 sec - exiting 23:31:52 (9180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:53 (9576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:55 (9576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:50:45 (5764): No heartbeat from core client for 30 sec - exiting 22:50:46 (5764): No heartbeat from core client for 30 sec - exiting 22:50:47 (5764): No heartbeat from core client for 30 sec - exiting 22:50:48 (5764): No heartbeat from core client for 30 sec - exiting 22:50:49 (5764): No heartbeat from core client for 30 sec - exiting 22:50:50 (5764): No heartbeat from core client for 30 sec - exiting 22:50:51 (5764): No heartbeat from core client for 30 sec - exiting 22:50:52 (5764): No heartbeat from core client for 30 sec - exiting 22:50:53 (5764): No heartbeat from core client for 30 sec - exiting 22:50:54 (5764): No heartbeat from core client for 30 sec - exiting 22:50:55 (5764): No heartbeat from core client for 30 sec - exiting 22:50:56 (5764): No heartbeat from core client for 30 sec - exiting 22:50:57 (5764): No heartbeat from core client for 30 sec - exiting 22:50:58 (5764): No heartbeat from core client for 30 sec - exiting 22:50:59 (5764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:51:32 (6680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:51:34 (6680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:28:06 (4992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:07 (4992): No heartbeat from core client for 30 sec - exiting 14:49:20 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:22 (5428): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... 19:51:39 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:51 (4320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:18 (8260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:19 (8260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 09:02:58 (7800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:24:48 (6728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:49 (6728): No heartbeat from core client for 30 sec - exiting 09:24:50 (6728): No heartbeat from core client for 30 sec - exiting 10:54:56 (8436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:53:18 (3876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:10 (9296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:12 (9296): No heartbeat from core client for 30 sec - exiting 02:54:14 (4320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:40:41 (8892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:52:41 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:52:42 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 01:11:05 (9796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:11:07 (9796): No heartbeat from core client for 30 sec - exiting 01:11:08 (9796): No heartbeat from core client for 30 sec - exiting 01:11:09 (9796): No heartbeat from core client for 30 sec - exiting 01:11:10 (9796): No heartbeat from core client for 30 sec - exiting 01:11:11 (9796): No heartbeat from core client for 30 sec - exiting 01:11:12 (9796): No heartbeat from core client for 30 sec - exiting 01:11:13 (9796): No heartbeat from core client for 30 sec - exiting 01:11:14 (9796): No heartbeat from core client for 30 sec - exiting 01:11:15 (9796): No heartbeat from core client for 30 sec - exiting 01:11:16 (9796): No heartbeat from core client for 30 sec - exiting 01:11:17 (9796): No heartbeat from core client for 30 sec - exiting 01:11:18 (9796): No heartbeat from core client for 30 sec - exiting 01:11:19 (9796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 10 Dec 2011 04:20:08 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 518,400 | 1,042,813 | 2.0116 |
| 09 Dec 2011 06:52:09 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 492,480 | 987,238 | 2.0046 |
| 08 Dec 2011 05:42:14 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 466,560 | 928,337 | 1.9897 |
| 07 Dec 2011 03:29:10 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 440,640 | 870,930 | 1.9765 |
| 06 Dec 2011 04:28:29 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 414,720 | 836,765 | 2.0177 |
| 05 Dec 2011 08:15:47 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 388,800 | 777,303 | 1.9992 |
| 04 Dec 2011 13:00:55 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 362,880 | 770,345 | 2.1229 |
| 03 Dec 2011 18:14:03 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 336,960 | 710,474 | 2.1085 |
| 02 Dec 2011 22:58:40 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 311,040 | 652,418 | 2.0975 |
| 02 Dec 2011 02:55:22 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 285,120 | 594,467 | 2.0850 |
| 01 Dec 2011 05:54:47 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 259,200 | 534,326 | 2.0614 |
| 30 Nov 2011 10:16:59 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 233,280 | 475,593 | 2.0387 |
| 28 Nov 2011 05:04:29 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 207,360 | 437,853 | 2.1116 |
| 27 Nov 2011 10:46:14 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 181,440 | 384,424 | 2.1187 |
| 26 Nov 2011 14:20:56 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 155,520 | 326,694 | 2.1007 |
| 25 Nov 2011 15:02:34 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 129,600 | 267,681 | 2.0654 |
| 24 Nov 2011 16:18:36 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 103,680 | 212,596 | 2.0505 |
| 23 Nov 2011 20:32:47 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 77,760 | 159,387 | 2.0497 |
| 23 Nov 2011 03:11:19 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 51,840 | 108,201 | 2.0872 |
| 17 Nov 2011 17:15:36 | 1156207 | 13636448 | hadcm3n_u1qk_1980_40_007544501_2 | 25,920 | 55,832 | 2.1540 |
©2024 cpdn.org