Task 17576582

Name	hadam3pm2_i0cn_1959_10_009271593_1
Workunit	9364509
Created	16 Dec 2014, 4:28:18 UTC
Sent	16 Dec 2014, 5:33:54 UTC
Report deadline	28 Nov 2015, 10:53:54 UTC
Received	6 Feb 2015, 23:00:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1142687
Run time	11 days 21 hours 55 min 49 sec
CPU time	10 days 17 hours 42 min 49 sec
Validate state	Invalid
Credit	2,080.96
Device peak FLOPS	0.96 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadam3pm2_7.03_i686-pc-linux-gnu: free(): invalid pointer: 0xf73ea008 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6ed51)[0xf74edd51] /lib32/libc.so.6(+0x705d8)[0xf74ef5d8] /lib32/libc.so.6(cfree+0x6d)[0xf74f274d] ../../projects/climateprediction.net/hadam3pm2_7.03_i686-pc-linux-gnu[0x80b0797] [0xf775e400] ../../projects/climateprediction.net/hadam3pm2_7.03_i686-pc-linux-gnu[0x80d5200] ../../projects/climateprediction.net/hadam3pm2_7.03_i686-pc-linux-gnu[0x80c01a4] /lib32/libpthread.so.0(+0x596e)[0xf772696e] /lib32/libc.so.6(clone+0x5e)[0xf755661e] ======= Memory map: ======== 08048000-08107000 r-xp 00000000 08:01 9699820 /var/lib/boinc-client/projects/climateprediction.net/hadam3pm2_7.03_i686-pc-linux-gnu 08107000-08109000 rw-p 000bf000 08:01 9699820 /var/lib/boinc-client/projects/climateprediction.net/hadam3pm2_7.03_i686-pc-linux-gnu 08109000-08186000 rw-p 00000000 00:00 0 088bd000-088de000 rw-p 00000000 00:00 0 [heap] f7100000-f7121000 rw-p 00000000 00:00 0 f7121000-f7200000 ---p 00000000 00:00 0 f7253000-f747d000 rw-s 00000000 08:01 9738566 /var/lib/boinc-client/slots/2/158090 f747d000-f747f000 rw-p 00000000 00:00 0 f747f000-f75db000 r-xp 00000000 08:01 1443911 /lib32/libc-2.11.1.so f75db000-f75dc000 ---p 0015c000 08:01 1443911 /lib32/libc-2.11.1.so f75dc000-f75de000 r--p 0015c000 08:01 1443911 /lib32/libc-2.11.1.so f75de000-f75df000 rw-p 0015e000 08:01 1443911 /lib32/libc-2.11.1.so f75df000-f75e2000 rw-p 00000000 00:00 0 f75e2000-f75ff000 r-xp 00000000 08:01 16783532 /usr/lib32/libgcc_s.so.1 f75ff000-f7600000 r--p 0001c000 08:01 16783532 /usr/lib32/libgcc_s.so.1 f7600000-f7601000 rw-p 0001d000 08:01 16783532 /usr/lib32/libgcc_s.so.1 f7601000-f7625000 r-xp 00000000 08:01 1443929 /lib32/libm-2.11.1.so f7625000-f7626000 r--p 00023000 08:01 1443929 /lib32/libm-2.11.1.so f7626000-f7627000 rw-p 00024000 08:01 1443929 /lib32/libm-2.11.1.so f7627000-f7710000 r-xp 00000000 08:01 16780942 /usr/lib32/libstdc++.so.6.0.13 f7710000-f7714000 r--p 000e9000 08:01 16780942 /usr/lib32/libstdc++.so.6.0.13 f7714000-f7715000 rw-p 000ed000 08:01 16780942 /usr/lib32/libstdc++.so.6.0.13 f7715000-f771c000 rw-p 00000000 00:00 0 f771c000-f771e000 r-xp 00000000 08:01 1443933 /lib32/libdl-2.11.1.so f771e000-f771f000 r--p 00001000 08:01 1443933 /lib32/libdl-2.11.1.so f771f000-f7720000 rw-p 00002000 08:01 1443933 /lib32/libdl-2.11.1.so f7720000-f7721000 rw-p 00000000 00:00 0 f7721000-f7736000 r-xp 00000000 08:01 1441890 /lib32/libpthread-2.11.1.so f7736000-f7737000 r--p 00014000 08:01 1441890 /lib32/libpthread-2.11.1.so f7737000-f7738000 rw-p 00015000 08:01 1441890 /lib32/libpthread-2.11.1.so f7738000-f773a000 rw-p 00000000 00:00 0 f7750000-f7752000 rw-p 00000000 00:00 0 f7752000-f7753000 ---p 00000000 00:00 0 f7753000-f775a000 rw-p 00000000 00:00 0 f775a000-f775c000 rw-s 00000000 08:01 9737882 /var/lib/boinc-client/slots/2/boinc_mmap_file f775c000-f775e000 rw-p 00000000 00:00 0 f775e000-f775f000 r-xp 00000000 00:00 0 [vdso] f775f000-f777b000 r-xp 00000000 08:01 1443930 /lib32/ld-2.11.1.so f777b000-f777c000 r--p 0001b000 08:01 1443930 /lib32/ld-2.11.1.so f777c000-f777d000 rw-p 0001c000 08:01 1443930 /lib32/ld-2.11.1.so ff860000-ff875000 rw-p 00000000 00:00 0 [stack] CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: go forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadam3pm2_i0cn_1959_10_009271593/dataout/stdout_um.txt Image PC Routine Line Source hadam3pm2_um_7.03 0845AB9D Unknown Unknown Unknown hadam3pm2_um_7.03 08459995 Unknown Unknown Unknown hadam3pm2_um_7.03 0842865F Unknown Unknown Unknown hadam3pm2_um_7.03 083E7E1D Unknown Unknown Unknown hadam3pm2_um_7.03 083E7767 Unknown Unknown Unknown hadam3pm2_um_7.03 0841A2F9 Unknown Unknown Unknown hadam3pm2_um_7.03 08417BC7 Unknown Unknown Unknown hadam3pm2_um_7.03 08346323 Unknown Unknown Unknown hadam3pm2_um_7.03 082D4171 Unknown Unknown Unknown hadam3pm2_um_7.03 0832046D Unknown Unknown Unknown hadam3pm2_um_7.03 0834C82D Unknown Unknown Unknown libc.so.6 F0D3EBF6 Unknown Unknown Unknown hadam3pm2_um_7.03 0804C821 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27783, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file06:51:23 (27783): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Jan 2015 23:37:15	1142687	17576582	hadam3pm2_i0cn_1959_10_009271593_1	72,068	631,434	8.7616
02 Jan 2015 06:15:55	1142687	17576582	hadam3pm2_i0cn_1959_10_009271593_1	37,508	327,989	8.7445
17 Dec 2014 05:46:27	1142687	17576582	hadam3pm2_i0cn_1959_10_009271593_1	2,948	25,416	8.6214