Name | hadcm3n_88mm_1980_40_008720601_0 |
Workunit | 8866579 |
Created | 23 Apr 2014, 12:25:09 UTC |
Sent | 5 May 2014, 19:40:25 UTC |
Report deadline | 5 Aug 2014, 3:07:36 UTC |
Received | 7 Sep 2014, 15:55:19 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 1326607 |
Run time | 15 days 15 hours 26 min 5 sec |
CPU time | 15 days 9 hours 5 min 17 sec |
Validate state | Invalid |
Credit | 12,441.60 |
Device peak FLOPS | 2.64 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
Stderr | <core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:15:27 (3572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Ocean Restart file copy failed on 88mmko.dai4bu0 19:58:26 (3984): No heartbeat from core client for 30 sec - exiting 19:58:27 (3984): No heartbeat from core client for 30 sec - exiting 19:58:28 (3984): No heartbeat from core client for 30 sec - exiting 19:58:29 (3984): No heartbeat from core client for 30 sec - exiting 19:58:31 (3984): No heartbeat from core client for 30 sec - exiting 19:58:32 (3984): No heartbeat from core client for 30 sec - exiting 19:58:33 (3984): No heartbeat from core client for 30 sec - exiting 19:58:34 (3984): No heartbeat from core client for 30 sec - exiting 19:58:35 (3984): No heartbeat from core client for 30 sec - exiting 19:58:36 (3984): No heartbeat from core client for 30 sec - exiting 19:58:37 (3984): No heartbeat from core client for 30 sec - exiting 19:58:38 (3984): No heartbeat from core client for 30 sec - exiting 19:58:39 (3984): No heartbeat from core client for 30 sec - exiting 19:58:40 (3984): No heartbeat from core client for 30 sec - exiting 19:58:42 (3984): No heartbeat from core client for 30 sec - exiting 19:58:43 (3984): No heartbeat from core client for 30 sec - exiting 19:58:44 (3984): No heartbeat from core client for 30 sec - exiting 19:58:45 (3984): No heartbeat from core client for 30 sec - exiting 19:58:46 (3984): No heartbeat from core client for 30 sec - exiting 19:58:47 (3984): No heartbeat from core client for 30 sec - exiting 19:58:48 (3984): No heartbeat from core client for 30 sec - exiting 19:58:49 (3984): No heartbeat from core client for 30 sec - exiting 19:58:50 (3984): No heartbeat from core client for 30 sec - exiting 19:58:51 (3984): No heartbeat from core client for 30 sec - exiting 19:58:52 (3984): No heartbeat from core client for 30 sec - exiting 19:58:54 (3984): No heartbeat from core client for 30 sec - exiting 19:58:55 (3984): No heartbeat from core client for 30 sec - exiting 19:58:56 (3984): No heartbeat from core client for 30 sec - exiting 19:58:57 (3984): No heartbeat from core client for 30 sec - exiting 19:58:58 (3984): No heartbeat from core client for 30 sec - exiting 19:58:59 (3984): No heartbeat from core client for 30 sec - exiting 19:59:00 (3984): No heartbeat from core client for 30 sec - exiting 19:59:01 (3984): No heartbeat from core client for 30 sec - exiting 19:59:02 (3984): No heartbeat from core client for 30 sec - exiting 19:59:03 (3984): No heartbeat from core client for 30 sec - exiting 19:59:04 (3984): No heartbeat from core client for 30 sec - exiting 19:59:06 (3984): No heartbeat from core client for 30 sec - exiting 19:59:07 (3984): No heartbeat from core client for 30 sec - exiting 19:59:08 (3984): No heartbeat from core client for 30 sec - exiting 19:59:09 (3984): No heartbeat from core client for 30 sec - exiting 19:59:10 (3984): No heartbeat from core client for 30 sec - exiting 19:59:11 (3984): No heartbeat from core client for 30 sec - exiting 19:59:12 (3984): No heartbeat from core client for 30 sec - exiting 19:59:13 (3984): No heartbeat from core client for 30 sec - exiting 19:59:14 (3984): No heartbeat from core client for 30 sec - exiting 19:59:15 (3984): No heartbeat from core client for 30 sec - exiting 19:59:16 (3984): No heartbeat from core client for 30 sec - exiting 19:59:18 (3984): No heartbeat from core client for 30 sec - exiting 19:59:19 (3984): No heartbeat from core client for 30 sec - exiting 19:59:20 (3984): No heartbeat from core client for 30 sec - exiting 19:59:21 (3984): No heartbeat from core client for 30 sec - exiting 19:59:22 (3984): No heartbeat from core client for 30 sec - exiting 19:59:23 (3984): No heartbeat from core client for 30 sec - exiting 19:59:24 (3984): No heartbeat from core client for 30 sec - exiting 19:59:25 (3984): No heartbeat from core client for 30 sec - exiting 19:59:26 (3984): No heartbeat from core client for 30 sec - exiting 19:59:27 (3984): No heartbeat from core client for 30 sec - exiting 19:59:28 (3984): No heartbeat from core client for 30 sec - exiting 19:59:30 (3984): No heartbeat from core client for 30 sec - exiting 19:59:31 (3984): No heartbeat from core client for 30 sec - exiting 19:59:32 (3984): No heartbeat from core client for 30 sec - exiting 19:59:33 (3984): No heartbeat from core client for 30 sec - exiting 19:59:34 (3984): No heartbeat from core client for 30 sec - exiting 19:59:35 (3984): No heartbeat from core client for 30 sec - exiting 19:59:36 (3984): No heartbeat from core client for 30 sec - exiting 19:59:37 (3984): No heartbeat from core client for 30 sec - exiting 19:59:38 (3984): No heartbeat from core client for 30 sec - exiting 19:59:39 (3984): No heartbeat from core client for 30 sec - exiting 19:59:40 (3984): No heartbeat from core client for 30 sec - exiting 19:59:42 (3984): No heartbeat from core client for 30 sec - exiting 19:59:43 (3984): No heartbeat from core client for 30 sec - exiting 19:59:44 (3984): No heartbeat from core client for 30 sec - exiting 19:59:45 (3984): No heartbeat from core client for 30 sec - exiting 19:59:46 (3984): No heartbeat from core client for 30 sec - exiting 19:59:47 (3984): No heartbeat from core client for 30 sec - exiting 19:59:48 (3984): No heartbeat from core client for 30 sec - exiting 19:59:49 (3984): No heartbeat from core client for 30 sec - exiting 19:59:50 (3984): No heartbeat from core client for 30 sec - exiting 19:59:51 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:28:45 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:33:43 (3452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:19 (2764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... C05:51:11 (4020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:54:42 (4008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:54:43 (4008): No heartbeat from core client for 30 sec - exiting 22:54:45 (4008): No heartbeat from core client for 30 sec - exiting 22:54:46 (4008): No heartbeat from core client for 30 sec - exiting 22:54:47 (4008): No heartbeat from core client for 30 sec - exiting 22:54:48 (4008): No heartbeat from core client for 30 sec - exiting 22:54:49 (4008): No heartbeat from core client for 30 sec - exiting 22:54:50 (4008): No heartbeat from core client for 30 sec - exiting 22:54:51 (4008): No heartbeat from core client for 30 sec - exiting 22:54:52 (4008): No heartbeat from core client for 30 sec - exiting 22:54:53 (4008): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:10:00 (4128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1768, iMonCtr=1 Model crash detected, will try to restart... 13:41:25 (3672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:50:18 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:56:36 (2260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77352AB1 read attempt to address 0x40BDBA0C Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77352AB1 read attempt to address 0x40BDBA0C Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_88mm_1980_40_008720601/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
07 Sep 2014 13:20:11 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 1,036,800 | 1,328,193 | 1.2811 |
04 Sep 2014 19:39:21 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 1,010,880 | 1,294,500 | 1.2806 |
30 Aug 2014 20:41:06 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 984,960 | 1,261,206 | 1.2805 |
30 Aug 2014 10:54:50 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 959,040 | 1,229,326 | 1.2818 |
30 Aug 2014 05:51:15 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 933,120 | 1,196,631 | 1.2824 |
23 Aug 2014 20:15:31 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 907,200 | 1,163,358 | 1.2824 |
19 Aug 2014 18:30:01 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 881,280 | 1,129,448 | 1.2816 |
07 Aug 2014 21:29:09 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 855,360 | 1,096,779 | 1.2822 |
07 Aug 2014 21:29:09 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 829,440 | 1,064,073 | 1.2829 |
07 Aug 2014 21:29:09 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 803,520 | 1,030,726 | 1.2828 |
07 Aug 2014 21:29:09 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 777,600 | 996,984 | 1.2821 |
07 Aug 2014 21:29:09 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 751,680 | 964,226 | 1.2828 |
01 Aug 2014 07:17:21 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 725,760 | 930,506 | 1.2821 |
28 Jul 2014 13:17:45 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 699,840 | 898,147 | 1.2834 |
28 Jul 2014 04:56:36 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 673,920 | 865,981 | 1.2850 |
27 Jul 2014 19:17:43 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 648,000 | 833,838 | 1.2868 |
24 Jul 2014 16:21:41 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 622,080 | 799,825 | 1.2857 |
20 Jul 2014 16:52:43 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 596,160 | 767,254 | 1.2870 |
05 Jul 2014 04:29:24 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 570,240 | 734,311 | 1.2877 |
03 Jul 2014 20:28:16 | 1326607 | 16585387 | hadcm3n_88mm_1980_40_008720601_0 | 544,320 | 701,959 | 1.2896 |
©2024 cpdn.org