climateprediction.net home page
Task 16604082

Task 16604082

Name hadcm3n_o80p_1900_40_008465516_3
Workunit 8616355
Created 29 Apr 2014, 16:39:00 UTC
Sent 29 Apr 2014, 17:03:20 UTC
Report deadline 30 Jul 2014, 0:30:31 UTC
Received 12 May 2014, 3:28:41 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1332713
Run time 11 days 6 hours 45 min
CPU time 4 days 19 hours 48 min 54 sec
Validate state Invalid
Credit 11,197.44
Device peak FLOPS 1.82 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>7.2.33</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
SIGSEGV: segmentation violation
Stack trace (13 frames):
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7721400]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf7520825]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29572, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7716400]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf7515825]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29572, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf76f3400]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf74f2825]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29572, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf772a400]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf7529825]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29572, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7740400]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf753f825]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29572, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77d6400]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/home/jonathans/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf5)[0xf75d5825]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29572, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
11 May 2014 23:29:01 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 933,120 603,785 0.6471
11 May 2014 16:31:13 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 907,200 578,640 0.6378
11 May 2014 08:32:51 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 881,280 553,473 0.6280
11 May 2014 01:29:45 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 855,360 528,342 0.6177
10 May 2014 18:27:52 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 829,440 503,210 0.6067
10 May 2014 10:30:27 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 803,520 478,115 0.5950
10 May 2014 03:27:13 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 777,600 452,964 0.5825
09 May 2014 20:30:16 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 751,680 427,828 0.5692
09 May 2014 13:32:06 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 725,760 418,388 0.5765
09 May 2014 05:30:12 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 699,840 558,410 0.7979
08 May 2014 22:28:26 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 673,920 533,741 0.7920
08 May 2014 14:28:41 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 648,000 509,034 0.7855
08 May 2014 07:27:24 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 622,080 484,304 0.7785
07 May 2014 23:24:42 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 596,160 459,583 0.7709
07 May 2014 16:27:25 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 570,240 434,837 0.7626
06 May 2014 05:25:39 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 544,320 409,314 0.7520
05 May 2014 22:24:05 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 518,400 383,293 0.7394
05 May 2014 14:25:40 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 492,480 357,126 0.7252
05 May 2014 07:23:31 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 466,560 331,114 0.7097
04 May 2014 23:26:55 1306043 16604082 hadcm3n_o80p_1900_40_008465516_3 440,640 305,109 0.6924


©2024 climateprediction.net