climateprediction.net home page
Task 11203459

Task 11203459

Name hadsm3dhet2_k3hl_006614171_4
Workunit 6817544
Created 15 Mar 2010, 12:25:44 UTC
Sent 3 May 2010, 19:19:38 UTC
Report deadline 16 Apr 2011, 0:39:38 UTC
Received 22 Apr 2011, 7:50:11 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1038975
Run time 14 days 17 hours 54 min 45 sec
CPU time 11 days 4 hours 19 min 15 sec
Validate state Invalid
Credit 6,649.31
Device peak FLOPS 0.00 GFLOPS
Application version ---
Stderr
<core_client_version>6.10.21</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
MainError:	10:43:09 PM	No files match the supplied pattern.
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_k3hl_006614171/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
libirc.dylib       00B8B5E3  Unknown               Unknown  Unknown
libirc.dylib       00B8A2D3  Unknown               Unknown  Unknown
libifcoremt.dylib  008C5EFA  Unknown               Unknown  Unknown
libifcoremt.dylib  0083C2E0  Unknown               Unknown  Unknown
libifcoremt.dylib  0087A12F  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001A54B0  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001A3CD8  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0001FD8A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  00280294  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  00297421  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E8A5  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=983, iMonCtr=1
Model crash detected, will try to restart...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: 
No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
MainError:	02:48:22 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_k3hl_006614171/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
libirc.dylib       00B8B5E3  Unknown               Unknown  Unknown
libirc.dylib       00B8A2D3  Unknown               Unknown  Unknown
libifcoremt.dylib  008C5EFA  Unknown               Unknown  Unknown
libifcoremt.dylib  0083C2E0  Unknown               Unknown  Unknown
libifcoremt.dylib  0087A12F  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001A54B0  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001A3CD8  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0001FD8A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  00280294  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  00297421  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E8A5  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=942, iMonCtr=1
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_k3hl_006614171/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
libirc.dylib       00B8B5E3  Unknown               Unknown  Unknown
libirc.dylib       00B8A2D3  Unknown               Unknown  Unknown
libifcoremt.dylib  008C5EFA  Unknown               Unknown  Unknown
libifcoremt.dylib  0083C2E0  Unknown               Unknown  Unknown
libifcoremt.dylib  0087A0A8  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027EC0B  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  00297421  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E8A5  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
20 Apr 2011 22:34:58 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 205,238 954,957 1.3195
20 Apr 2011 22:34:58 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 194,436 940,987 1.3199
20 Apr 2011 22:34:58 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 183,634 927,077 1.3204
12 Apr 2011 11:29:09 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 172,832 913,022 1.3207
10 Apr 2011 15:02:51 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 162,030 899,068 1.3211
07 Apr 2011 02:36:37 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 151,228 885,147 1.3217
04 Apr 2011 10:26:49 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 140,426 870,632 1.3213
01 Apr 2011 09:15:13 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 129,624 855,710 1.3203
29 Mar 2011 07:42:16 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 118,822 840,886 1.3194
27 Mar 2011 00:29:09 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 108,020 825,988 1.3184
23 Mar 2011 21:10:33 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 97,218 811,134 1.3174
21 Mar 2011 10:40:07 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 86,416 796,284 1.3164
20 Mar 2011 00:48:33 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 75,614 781,422 1.3153
16 Mar 2011 20:48:16 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 64,812 766,601 1.3142
13 Mar 2011 22:55:52 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 54,010 751,873 1.3133
10 Mar 2011 05:14:49 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 43,208 737,908 1.3137
08 Mar 2011 16:49:38 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 32,406 724,152 1.3145
08 Mar 2011 16:49:38 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 21,604 710,176 1.3149
25 Feb 2011 14:30:00 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 10,802 696,255 1.3154
16 Feb 2011 00:55:59 1038975 11203459 hadsm3dhet2_k3hl_006614171_4 259,248 682,347 1.3160


©2024 climateprediction.net