climateprediction.net home page
Task 11890829

Task 11890829

Name hadsm3dhet2_u1be_006725041_3
Workunit 6928384
Created 17 Sep 2010, 8:07:31 UTC
Sent 23 Sep 2010, 16:00:50 UTC
Report deadline 5 Sep 2011, 21:20:50 UTC
Received 14 Mar 2011, 17:04:05 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID 1024759
Run time 34 days 0 hours 57 min 5 sec
CPU time 30 days 6 hours 12 min 37 sec
Validate state Invalid
Credit 6,748.55
Device peak FLOPS 2.04 GFLOPS
Application version UK Met Office HadSM3 Slab Model v6.07
windows_intelx86
Stderr
<core_client_version>6.6.38</core_client_version>
<![CDATA[
<message>
 - exit code -1073741819 (0xc0000005)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1
Model crash detected, will try to restart...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1
Model crash detected, will try to restart...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3672, iMonCtr=1
Model crash detected, will try to restart...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
MainError:	08:31:08 AM	No files match the supplied pattern.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3848, iMonCtr=1
Model crash detected, will try to restart...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3156, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CCPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3440, iMonCtr=1
Model crash detected, will try to restart...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
MainError:	07:20:24 AM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3476, iMonCtr=1
Model crash detected, will try to restart...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x77EA053B write attempt to address 0x7CEFFFF8

Engaging BOINC Windows Runtime Debugger...



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x77C6355B read attempt to address 0x7CF00000

Engaging BOINC Windows Runtime Debugger...



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x773E053B write attempt to address 0x7CEFFFF8

Engaging BOINC Windows Runtime Debugger...



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x77BA355B read attempt to address 0x7CF00000

Engaging BOINC Windows Runtime Debugger...



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7738355B read attempt to address 0x7CF00000

Engaging BOINC Windows Runtime Debugger...

MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
MainError:	02:52:27 PM	No files match the supplied pattern.
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3572, iMonCtr=1
Model crash detected, will try to restart...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3852, iMonCtr=1
Model crash detected, will try to restart...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1796, iMonCtr=1
Model crash detected, will try to restart...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7773B84B write attempt to address 0x7CEFFFF8

Engaging BOINC Windows Runtime Debugger...


</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
04 Jan 2011 22:35:33 1024759 11890829 hadsm3dhet2_u1be_006725041_3 216,040 1,357,879 1.8486
02 Jan 2011 22:15:39 1024759 11890829 hadsm3dhet2_u1be_006725041_3 205,238 1,338,571 1.8495
01 Jan 2011 13:01:29 1024759 11890829 hadsm3dhet2_u1be_006725041_3 194,436 1,319,538 1.8509
31 Dec 2010 16:46:29 1024759 11890829 hadsm3dhet2_u1be_006725041_3 183,634 1,300,209 1.8518
30 Dec 2010 19:58:07 1024759 11890829 hadsm3dhet2_u1be_006725041_3 172,832 1,280,514 1.8523
30 Dec 2010 13:04:42 1024759 11890829 hadsm3dhet2_u1be_006725041_3 162,030 1,261,385 1.8535
28 Dec 2010 15:13:47 1024759 11890829 hadsm3dhet2_u1be_006725041_3 151,228 1,241,849 1.8543
28 Dec 2010 15:13:47 1024759 11890829 hadsm3dhet2_u1be_006725041_3 140,426 1,222,225 1.8549
26 Dec 2010 18:44:04 1024759 11890829 hadsm3dhet2_u1be_006725041_3 129,624 1,202,602 1.8555
26 Dec 2010 10:59:48 1024759 11890829 hadsm3dhet2_u1be_006725041_3 118,822 1,182,654 1.8557
23 Dec 2010 17:15:22 1024759 11890829 hadsm3dhet2_u1be_006725041_3 108,020 1,162,373 1.8553
22 Dec 2010 16:31:03 1024759 11890829 hadsm3dhet2_u1be_006725041_3 97,218 1,141,874 1.8546
20 Dec 2010 15:10:25 1024759 11890829 hadsm3dhet2_u1be_006725041_3 86,416 1,121,673 1.8543
19 Dec 2010 12:50:38 1024759 11890829 hadsm3dhet2_u1be_006725041_3 75,614 1,101,507 1.8540
18 Dec 2010 19:27:29 1024759 11890829 hadsm3dhet2_u1be_006725041_3 64,812 1,081,301 1.8537
17 Dec 2010 22:22:54 1024759 11890829 hadsm3dhet2_u1be_006725041_3 54,010 1,060,935 1.8531
16 Dec 2010 19:13:36 1024759 11890829 hadsm3dhet2_u1be_006725041_3 43,208 1,040,534 1.8525
12 Dec 2010 19:49:50 1024759 11890829 hadsm3dhet2_u1be_006725041_3 32,406 1,021,224 1.8537
10 Dec 2010 20:01:20 1024759 11890829 hadsm3dhet2_u1be_006725041_3 21,604 1,001,455 1.8542
08 Dec 2010 19:50:23 1024759 11890829 hadsm3dhet2_u1be_006725041_3 10,802 981,951 1.8552


©2024 climateprediction.net