climateprediction.net home page
Task 11521440

Task 11521440

Name famous_ugf0_1599_200_006654119_4
Workunit 6857491
Created 10 Jun 2010, 14:06:00 UTC
Sent 22 Aug 2010, 7:33:51 UTC
Report deadline 21 Nov 2010, 15:01:02 UTC
Received 25 Aug 2010, 10:59:17 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 4 (0x00000004) Unknown error code
Computer ID 1095507
Run time 22 hours 19 min 39 sec
CPU time 13 hours 56 min 22 sec
Validate state Invalid
Credit 895.65
Device peak FLOPS 2.84 GFLOPS
Application version UK Met Office FAMOUS v6.11
i686-apple-darwin
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process got signal 4
</message>
<stderr_txt>
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Sun Aug 22 05:18:16 2010

  0  0x0048fdcf  PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221
  1  0x0049054a  boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474
  2  0x91a741fb  0x91a741fb
  3  0xffffffff  0xffffffff
  4  0x0138f6b3  0x138f6b3
  5  0x0138e44f  0x138e44f
  6  0x003f5826  um_submodel_init (in famous_um_6.11_i686-apple-darwin) + 406
  7  0x00450bd8  um_shell (in famous_um_6.11_i686-apple-darwin) + 3520
  8  0x004698c0  main (in famous_um_6.11_i686-apple-darwin) + 1112
  9  0x000026b6  _start (in famous_um_6.11_i686-apple-darwin) + 216
 10  0x000025dd  start (in famous_um_6.11_i686-apple-darwin) + 41

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffcccc edx: 0x91a0e0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0xbfffcccc
   ss: 0x00000023 efl: 0x00000206 eip: 0x91a0e0fa  cs: 0x0000000b
   ds: 0x00000023  es: 0x00000023  fs: 0x00000000  gs: 0x0000000f

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12ea000 -  0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libsvml.dylib
 0x135d000 -  0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libifcoremt.dylib
 0x143d000 -  0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libimf.dylib
 0x16aa000 -  0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libirc.dylib
0x91a0d000 - 0x91bb3fff /usr/lib/libSystem.B.dylib
0x981a7000 - 0x981b5fff /usr/lib/libz.1.dylib
0x9872b000 - 0x9872efff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=55093, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Sun Aug 22 05:18:18 2010

  0  0x0048fdcf  PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221
  1  0x0049054a  boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474
  2  0x91a741fb  0x91a741fb
  3  0xffffffff  0xffffffff
  4  0x0138f6b3  0x138f6b3
  5  0x0138e44f  0x138e44f
  6  0x003f5826  um_submodel_init (in famous_um_6.11_i686-apple-darwin) + 406
  7  0x00450bd8  um_shell (in famous_um_6.11_i686-apple-darwin) + 3520
  8  0x004698c0  main (in famous_um_6.11_i686-apple-darwin) + 1112
  9  0x000026b6  _start (in famous_um_6.11_i686-apple-darwin) + 216
 10  0x000025dd  start (in famous_um_6.11_i686-apple-darwin) + 41

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffcccc edx: 0x91a0e0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0xbfffcccc
   ss: 0x00000023 efl: 0x00000206 eip: 0x91a0e0fa  cs: 0x0000000b
   ds: 0x00000023  es: 0x00000023  fs: 0x00000000  gs: 0x0000000f

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12ea000 -  0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libsvml.dylib
 0x135d000 -  0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libifcoremt.dylib
 0x143d000 -  0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libimf.dylib
 0x16aa000 -  0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libirc.dylib
0x91a0d000 - 0x91bb3fff /usr/lib/libSystem.B.dylib
0x981a7000 - 0x981b5fff /usr/lib/libz.1.dylib
0x9872b000 - 0x9872efff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=55093, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Sun Aug 22 16:55:53 2010

  0  0x0048fdcf  PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221
  1  0x0049054a  boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474
  2  0x91a741fb  0x91a741fb
  3  0xffffffff  0xffffffff
  4  0x0138f6b3  0x138f6b3
  5  0x0138e44f  0x138e44f
  6  0x003f5826  um_submodel_init (in famous_um_6.11_i686-apple-darwin) + 406
  7  0x00450bd8  um_shell (in famous_um_6.11_i686-apple-darwin) + 3520
  8  0x004698c0  main (in famous_um_6.11_i686-apple-darwin) + 1112
  9  0x000026b6  _start (in famous_um_6.11_i686-apple-darwin) + 216
 10  0x000025dd  start (in famous_um_6.11_i686-apple-darwin) + 41

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffcccc edx: 0x91a0e0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0xbfffcccc
   ss: 0x00000023 efl: 0x00000206 eip: 0x91a0e0fa  cs: 0x0000000b
   ds: 0x00000023  es: 0x00000023  fs: 0x00000000  gs: 0x0000000f

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12ea000 -  0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libsvml.dylib
 0x135d000 -  0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libifcoremt.dylib
 0x143d000 -  0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libimf.dylib
 0x16aa000 -  0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libirc.dylib
0x91a0d000 - 0x91bb3fff /usr/lib/libSystem.B.dylib
0x981a7000 - 0x981b5fff /usr/lib/libz.1.dylib
0x9872b000 - 0x9872efff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=64861, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
 (67247): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (68540): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Wed Aug 25 04:23:26 2010

  0  0x0048fdcf  PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221
  1  0x0049054a  boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474
  2  0x935411fb  0x935411fb
  3  0xffffffff  0xffffffff
  4  0x0138f6b3  0x138f6b3
  5  0x0138e44f  0x138e44f
  6  0x003f5826  um_submodel_init (in famous_um_6.11_i686-apple-darwin) + 406
  7  0x00450bd8  um_shell (in famous_um_6.11_i686-apple-darwin) + 3520
  8  0x004698c0  main (in famous_um_6.11_i686-apple-darwin) + 1112
  9  0x000026b6  _start (in famous_um_6.11_i686-apple-darwin) + 216
 10  0x000025dd  start (in famous_um_6.11_i686-apple-darwin) + 41
 11  0x00000003  0x00000003 (in famous_um_6.11_i686-apple-darwin)

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd48c edx: 0x934db0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4c8 esp: 0xbfffd48c
   ss: 0x00000023 efl: 0x00000206 eip: 0x934db0fa  cs: 0x0000000b
   ds: 0x00000023  es: 0x00000023  fs: 0x00000000  gs: 0x0000000f

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12ea000 -  0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libsvml.dylib
 0x135d000 -  0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libifcoremt.dylib
 0x143d000 -  0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libimf.dylib
 0x16aa000 -  0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libirc.dylib
0x934da000 - 0x93680fff /usr/lib/libSystem.B.dylib
0x93b25000 - 0x93b33fff /usr/lib/libz.1.dylib
0x94ab4000 - 0x94ab7fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6527, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Wed Aug 25 04:53:27 2010

  0  0x0048fdcf  PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221
  1  0x0049054a  boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474
  2  0x935411fb  0x935411fb
  3  0xffffffff  0xffffffff
  4  0x0138f6b3  0x138f6b3
  5  0x0138e44f  0x138e44f
  6  0x002eeac6  readcntl (in famous_um_6.11_i686-apple-darwin) + 2598
  7  0x003f55ea  um_setup (in famous_um_6.11_i686-apple-darwin) + 458
  8  0x00452abd  um_shell (in famous_um_6.11_i686-apple-darwin) + 11429
  9  0x004698c0  main (in famous_um_6.11_i686-apple-darwin) + 1112
 10  0x000026b6  _start (in famous_um_6.11_i686-apple-darwin) + 216
 11  0x000025dd  start (in famous_um_6.11_i686-apple-darwin) + 41
 12  0x00000003  0x00000003 (in famous_um_6.11_i686-apple-darwin)

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3fc edx: 0x934db0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc438 esp: 0xbfffc3fc
   ss: 0x00000023 efl: 0x00000206 eip: 0x934db0fa  cs: 0x0000000b
   ds: 0x00000023  es: 0x00000023  fs: 0x00000000  gs: 0x0000000f

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12ea000 -  0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libsvml.dylib
 0x135d000 -  0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libifcoremt.dylib
 0x143d000 -  0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libimf.dylib
 0x16aa000 -  0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ugf0_1599_200_006654119/lib/libirc.dylib
0x934da000 - 0x93680fff /usr/lib/libSystem.B.dylib
0x93b25000 - 0x93b33fff /usr/lib/libz.1.dylib
0x94ab4000 - 0x94ab7fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6967, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Aug 2010 02:36:48 1095507 11521440 famous_ugf0_1599_200_006654119_4 271,466 50,034 0.1843
25 Aug 2010 02:36:48 1095507 11521440 famous_ugf0_1599_200_006654119_4 262,106 48,229 0.1840
25 Aug 2010 01:34:42 1095507 11521440 famous_ugf0_1599_200_006654119_4 252,746 46,572 0.1843
25 Aug 2010 01:34:42 1095507 11521440 famous_ugf0_1599_200_006654119_4 243,386 44,905 0.1845
24 Aug 2010 19:53:02 1095507 11521440 famous_ugf0_1599_200_006654119_4 234,026 43,208 0.1846
24 Aug 2010 17:48:00 1095507 11521440 famous_ugf0_1599_200_006654119_4 224,666 41,467 0.1846
24 Aug 2010 14:31:04 1095507 11521440 famous_ugf0_1599_200_006654119_4 215,306 39,670 0.1842
24 Aug 2010 11:49:48 1095507 11521440 famous_ugf0_1599_200_006654119_4 205,946 37,865 0.1839
24 Aug 2010 11:12:45 1095507 11521440 famous_ugf0_1599_200_006654119_4 196,586 36,144 0.1839
24 Aug 2010 07:06:05 1095507 11521440 famous_ugf0_1599_200_006654119_4 187,226 34,403 0.1838
24 Aug 2010 04:41:00 1095507 11521440 famous_ugf0_1599_200_006654119_4 177,866 32,573 0.1831
24 Aug 2010 02:06:30 1095507 11521440 famous_ugf0_1599_200_006654119_4 168,506 30,619 0.1817
24 Aug 2010 01:36:49 1095507 11521440 famous_ugf0_1599_200_006654119_4 159,146 28,774 0.1808
24 Aug 2010 00:39:38 1095507 11521440 famous_ugf0_1599_200_006654119_4 149,786 26,965 0.1800
23 Aug 2010 19:58:57 1095507 11521440 famous_ugf0_1599_200_006654119_4 140,426 25,245 0.1798
23 Aug 2010 19:16:13 1095507 11521440 famous_ugf0_1599_200_006654119_4 131,066 23,550 0.1797
23 Aug 2010 18:38:31 1095507 11521440 famous_ugf0_1599_200_006654119_4 121,706 21,824 0.1793
23 Aug 2010 14:55:12 1095507 11521440 famous_ugf0_1599_200_006654119_4 112,346 20,177 0.1796
23 Aug 2010 13:00:03 1095507 11521440 famous_ugf0_1599_200_006654119_4 102,986 18,701 0.1816
23 Aug 2010 12:28:29 1095507 11521440 famous_ugf0_1599_200_006654119_4 93,626 17,228 0.1840


©2024 climateprediction.net