climateprediction.net home page
Task 12024287

Task 12024287

Name famous_v4nf_799_200_006736106_2
Workunit 6939446
Created 15 Nov 2010, 8:13:59 UTC
Sent 19 Nov 2010, 11:16:15 UTC
Report deadline 18 Feb 2011, 18:43:26 UTC
Received 24 Nov 2010, 18:02:22 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 942621
Run time 3 days 18 hours 0 min 24 sec
CPU time 3 days 11 hours 59 min 54 sec
Validate state Workunit error - check skipped
Credit 6,176.41
Device peak FLOPS 3.17 GFLOPS
Application version UK Met Office FAMOUS v6.11
i686-apple-darwin
Stderr
<core_client_version>6.6.36</core_client_version>
<![CDATA[
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Fri Nov 19 18:16:59 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21907, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Fri Nov 19 20:42:03 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24272, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Fri Nov 19 20:42:05 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24272, iMonCtr=1
Model crash detected, will try to restart...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Fri Nov 19 22:39:56 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26096, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Sat Nov 20 20:35:27 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5703, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Sun Nov 21 00:49:42 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8566, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Sun Nov 21 15:03:52 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16283, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Sun Nov 21 17:39:48 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18562, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Sun Nov 21 20:03:10 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9878c46b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x987260fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x987260fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x947e9000 - 0x947ecfff /usr/lib/system/libmathCommon.A.dylib
0x96500000 - 0x9650efff /usr/lib/libz.1.dylib
0x98725000 - 0x988ccfff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21237, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Nov 22 11:09:08 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1564, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Nov 22 12:08:51 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2037, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Nov 22 12:08:54 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2037, iMonCtr=1
Model crash detected, will try to restart...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Nov 22 13:50:54 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2902, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Nov 22 14:13:57 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3100, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Nov 22 16:37:45 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4214, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Nov 22 19:28:06 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5528, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Nov 23 09:58:53 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11948, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Nov 23 14:30:53 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13935, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Nov 23 17:07:09 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15093, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Nov 23 22:32:37 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17697, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Wed Nov 24 10:59:31 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26943, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Wed Nov 24 10:59:33 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26943, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Wed Nov 24 12:32:43 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27834, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Wed Nov 24 16:37:24 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29990, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Wed Nov 24 16:58:50 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd51c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd558 esp: 0xbfffd51c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30261, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Wed Nov 24 17:05:12 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9369946b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc48c edx: 0x936330fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc4c8 esp: 0xbfffc48c
   ss: 0x0000001f efl: 0x00000206 eip: 0x936330fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4nf_799_200_006736106/lib/libirc.dylib
0x934b5000 - 0x934c3fff /usr/lib/libz.1.dylib
0x93632000 - 0x937d9fff /usr/lib/libSystem.B.dylib
0x95eee000 - 0x95ef1fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30338, iMonCtr=1
Model crash detected, will try to restart...
 (30338): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
24 Nov 2010 17:51:32 942621 12024287 famous_v4nf_799_200_006736106_2 1,872,026 302,391 0.1615
24 Nov 2010 17:20:43 942621 12024287 famous_v4nf_799_200_006736106_2 1,862,666 300,869 0.1615
24 Nov 2010 16:18:30 942621 12024287 famous_v4nf_799_200_006736106_2 1,853,306 299,352 0.1615
24 Nov 2010 15:47:12 942621 12024287 famous_v4nf_799_200_006736106_2 1,843,946 297,814 0.1615
24 Nov 2010 14:58:44 942621 12024287 famous_v4nf_799_200_006736106_2 1,834,586 296,290 0.1615
24 Nov 2010 14:29:30 942621 12024287 famous_v4nf_799_200_006736106_2 1,825,226 294,773 0.1615
24 Nov 2010 14:04:02 942621 12024287 famous_v4nf_799_200_006736106_2 1,815,866 293,260 0.1615
24 Nov 2010 13:35:12 942621 12024287 famous_v4nf_799_200_006736106_2 1,806,506 291,747 0.1615
24 Nov 2010 13:07:07 942621 12024287 famous_v4nf_799_200_006736106_2 1,797,146 290,235 0.1615
24 Nov 2010 11:52:38 942621 12024287 famous_v4nf_799_200_006736106_2 1,787,786 288,735 0.1615
24 Nov 2010 11:04:56 942621 12024287 famous_v4nf_799_200_006736106_2 1,778,426 287,229 0.1615
24 Nov 2010 10:00:11 942621 12024287 famous_v4nf_799_200_006736106_2 1,769,066 285,730 0.1615
24 Nov 2010 09:43:44 942621 12024287 famous_v4nf_799_200_006736106_2 1,759,706 284,209 0.1615
24 Nov 2010 09:14:49 942621 12024287 famous_v4nf_799_200_006736106_2 1,750,346 282,697 0.1615
24 Nov 2010 08:53:08 942621 12024287 famous_v4nf_799_200_006736106_2 1,740,986 281,185 0.1615
24 Nov 2010 08:51:11 942621 12024287 famous_v4nf_799_200_006736106_2 1,731,626 279,666 0.1615
24 Nov 2010 06:36:38 942621 12024287 famous_v4nf_799_200_006736106_2 1,722,266 278,180 0.1615
24 Nov 2010 06:15:13 942621 12024287 famous_v4nf_799_200_006736106_2 1,712,906 276,702 0.1615
24 Nov 2010 06:15:13 942621 12024287 famous_v4nf_799_200_006736106_2 1,703,546 275,224 0.1616
24 Nov 2010 06:15:13 942621 12024287 famous_v4nf_799_200_006736106_2 1,694,186 273,747 0.1616


©2024 climateprediction.net