Task 15522415

Name	hadcm3n_o6cr_2140_40_008269816_1
Workunit	8424940
Created	3 Jan 2013, 16:18:46 UTC
Sent	3 Jan 2013, 16:18:57 UTC
Report deadline	4 Apr 2013, 23:46:08 UTC
Received	18 Feb 2013, 12:54:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1241296
Run time	24 days 2 hours 38 min 31 sec
CPU time	20 days 13 hours 53 min 40 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	1.21 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... MainError: 04:32:28 PM No files match the supplied pattern. MainError: 04:32:28 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... MainError: 08:17:51 PM No files match the supplied pattern. MainError: 08:17:51 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... MainError: 01:11:13 AM No files match the supplied pattern. MainError: 01:11:13 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... MainError: 03:20:04 PM No files match the supplied pattern. MainError: 03:20:04 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... MainError: 06:32:18 PM No files match the supplied pattern. MainError: 06:32:18 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... MainError: 09:35:00 PM No files match the supplied pattern. MainError: 09:35:00 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... MainError: 12:29:20 AM No files match the supplied pattern. MainError: 12:29:20 AM No files match the supplied pattern. MainError: 06:53:25 PM No files match the supplied pattern. MainError: 06:53:25 PM No files match the supplied pattern. MainError: 02:04:47 PM No files match the supplied pattern. MainError: 02:04:47 PM No files match the supplied pattern. MainError: 09:57:10 AM No files match the supplied pattern. MainError: 09:57:10 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... Error converting file to netcdf: dataout/o6crka.ph11c10 Error converting file to netcdf: dataout/o6crka.pg11c10 Error converting file to netcdf: dataout/o6crka.pe11c10 MainError: 06:08:57 AM No files match the supplied pattern. MainError: 06:08:57 AM No files match the supplied pattern. BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Feb 2013 06:12:46	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	777,600	2,055,344	2.6432
17 Feb 2013 10:02:11	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	751,680	1,983,185	2.6383
16 Feb 2013 14:05:23	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	725,760	1,912,146	2.6347
15 Feb 2013 18:57:26	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	699,840	1,843,627	2.6344
15 Feb 2013 00:30:24	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	673,920	1,777,919	2.6382
13 Feb 2013 21:37:36	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	648,000	1,713,160	2.6438
12 Feb 2013 18:35:58	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	622,080	1,648,748	2.6504
11 Feb 2013 15:20:44	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	596,160	1,583,842	2.6567
10 Feb 2013 01:15:10	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	570,240	1,517,138	2.6605
08 Feb 2013 20:21:33	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	544,320	1,450,386	2.6646
07 Feb 2013 16:36:54	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	518,400	1,383,463	2.6687
05 Feb 2013 20:39:09	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	492,480	1,317,957	2.6762
04 Feb 2013 14:20:34	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	466,560	1,252,763	2.6851
02 Feb 2013 19:08:19	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	440,640	1,186,517	2.6927
30 Jan 2013 21:53:29	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	414,720	1,121,898	2.7052
29 Jan 2013 15:35:35	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	388,800	1,057,374	2.7196
29 Jan 2013 12:45:08	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	362,880	991,016	2.7310
26 Jan 2013 00:03:26	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	336,960	922,157	2.7367
24 Jan 2013 17:48:09	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	311,040	856,940	2.7551
22 Jan 2013 23:36:27	1241296	15522415	hadcm3n_o6cr_2140_40_008269816_1	285,120	792,031	2.7779