Task 17909446

Name	hadcm3n_xa1w_1940_40_009149322_2
Workunit	9279658
Created	8 Feb 2015, 2:59:27 UTC
Sent	8 Feb 2015, 3:06:52 UTC
Report deadline	10 May 2015, 10:34:03 UTC
Received	17 Feb 2015, 17:13:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1298787
Run time	6 days 4 hours 24 min 24 sec
CPU time	5 days 9 hours 39 min 40 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 03:14:35 (65928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:19:35 (66900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 03:24:36 (67636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:24:37 (67636): No heartbeat from core client for 30 sec - exiting 03:24:38 (67636): No heartbeat from core client for 30 sec - exiting 03:29:36 (68516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:33:09 (69180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:34:38 (69826): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:34:39 (69826): No heartbeat from core client for 30 sec - exiting 03:39:38 (70095): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:39:39 (70095): No heartbeat from core client for 30 sec - exiting 03:39:40 (70095): No heartbeat from core client for 30 sec - exiting 03:44:38 (71057): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:39 (71057): No heartbeat from core client for 30 sec - exiting 03:44:40 (71057): No heartbeat from core client for 30 sec - exiting 03:44:41 (71057): No heartbeat from core client for 30 sec - exiting 03:44:42 (71057): No heartbeat from core client for 30 sec - exiting 03:49:38 (71860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:54:38 (72787): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:59:39 (73553): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:04:39 (74457): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:09:40 (75308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:41 (76140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:19:42 (76910): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:24:41 (77829): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:24:42 (77829): No heartbeat from core client for 30 sec - exiting 04:24:43 (77829): No heartbeat from core client for 30 sec - exiting 04:24:44 (77829): No heartbeat from core client for 30 sec - exiting 04:29:41 (78558): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:34:42 (79438): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:39:42 (80134): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:40:49 (81097): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:44:43 (81273): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:43 (82058): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:44 (82058): No heartbeat from core client for 30 sec - exiting 04:54:44 (82804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:54:45 (82804): No heartbeat from core client for 30 sec - exiting 04:59:45 (83729): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:04:45 (84372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:04:46 (84372): No heartbeat from core client for 30 sec - exiting 05:09:46 (85479): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:09:47 (85479): No heartbeat from core client for 30 sec - exiting 05:14:48 (85996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:19:47 (86952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:24:47 (87654): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:29:48 (88588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:20 (89288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:22 (89288): No heartbeat from core client for 30 sec - exiting 05:34:23 (89288): No heartbeat from core client for 30 sec - exiting 05:36:04 (90124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:40:24 (90298): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:24 (91037): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 05:50:25 (91785): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 05:55:25 (92553): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:26 (93272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:05:27 (94029): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:10:27 (94666): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 06:20:26 (95501): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:30:27 (97535): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:29 (99733): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:41:31 (1960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:49:52 (19099): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:08:35 (21258): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:08:36 (21258): No heartbeat from core client for 30 sec - exiting 22:08:37 (21258): No heartbeat from core client for 30 sec - exiting 22:08:38 (21258): No heartbeat from core client for 30 sec - exiting 22:08:39 (21258): No heartbeat from core client for 30 sec - exiting 22:08:40 (21258): No heartbeat from core client for 30 sec - exiting 22:08:41 (21258): No heartbeat from core client for 30 sec - exiting 22:08:42 (21258): No heartbeat from core client for 30 sec - exiting 22:08:43 (21258): No heartbeat from core client for 30 sec - exiting 22:08:44 (21258): No heartbeat from core client for 30 sec - exiting 22:08:45 (21258): No heartbeat from core client for 30 sec - exiting 22:08:47 (21258): No heartbeat from core client for 30 sec - exiting 22:08:48 (21258): No heartbeat from core client for 30 sec - exiting 22:08:49 (21258): No heartbeat from core client for 30 sec - exiting 22:08:50 (21258): No heartbeat from core client for 30 sec - exiting 22:08:51 (21258): No heartbeat from core client for 30 sec - exiting 22:08:52 (21258): No heartbeat from core client for 30 sec - exiting 22:08:53 (21258): No heartbeat from core client for 30 sec - exiting 22:08:54 (21258): No heartbeat from core client for 30 sec - exiting 22:08:55 (21258): No heartbeat from core client for 30 sec - exiting 22:08:56 (21258): No heartbeat from core client for 30 sec - exiting 22:08:57 (21258): No heartbeat from core client for 30 sec - exiting 22:08:58 (21258): No heartbeat from core client for 30 sec - exiting 22:08:59 (21258): No heartbeat from core client for 30 sec - exiting 22:09:00 (21258): No heartbeat from core client for 30 sec - exiting 22:09:01 (21258): No heartbeat from core client for 30 sec - exiting 22:09:02 (21258): No heartbeat from core client for 30 sec - exiting 22:09:04 (21258): No heartbeat from core client for 30 sec - exiting 22:09:05 (21258): No heartbeat from core client for 30 sec - exiting 22:09:06 (21258): No heartbeat from core client for 30 sec - exiting 22:09:07 (21258): No heartbeat from core client for 30 sec - exiting 22:09:08 (21258): No heartbeat from core client for 30 sec - exiting 22:09:09 (21258): No heartbeat from core client for 30 sec - exiting 22:09:10 (21258): No heartbeat from core client for 30 sec - exiting 22:09:11 (21258): No heartbeat from core client for 30 sec - exiting 22:09:12 (21258): No heartbeat from core client for 30 sec - exiting 22:09:13 (21258): No heartbeat from core client for 30 sec - exiting 22:09:14 (21258): No heartbeat from core client for 30 sec - exiting 22:09:15 (21258): No heartbeat from core client for 30 sec - exiting 22:09:16 (21258): No heartbeat from core client for 30 sec - exiting 22:09:17 (21258): No heartbeat from core client for 30 sec - exiting 22:09:18 (21258): No heartbeat from core client for 30 sec - exiting 22:09:19 (21258): No heartbeat from core client for 30 sec - exiting 22:09:20 (21258): No heartbeat from core client for 30 sec - exiting 22:09:21 (21258): No heartbeat from core client for 30 sec - exiting 22:09:23 (21258): No heartbeat from core client for 30 sec - exiting 22:09:24 (21258): No heartbeat from core client for 30 sec - exiting 22:09:25 (21258): No heartbeat from core client for 30 sec - exiting 22:09:26 (21258): No heartbeat from core client for 30 sec - exiting 22:09:27 (21258): No heartbeat from core client for 30 sec - exiting 22:09:28 (21258): No heartbeat from core client for 30 sec - exiting 22:09:29 (21258): No heartbeat from core client for 30 sec - exiting 22:09:30 (21258): No heartbeat from core client for 30 sec - exiting 22:09:32 (21258): No heartbeat from core client for 30 sec - exiting 22:09:33 (21258): No heartbeat from core client for 30 sec - exiting 22:09:34 (21258): No heartbeat from core client for 30 sec - exiting 22:09:35 (21258): No heartbeat from core client for 30 sec - exiting 22:09:36 (21258): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:01:31 (31331): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:04:58 (35611): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:43:52 (54092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:35:03 (59918): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:17:29 (84356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:05:26 (69353): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: severe (17): syntax error in NAMELIST input, unit 5, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_xa1w_1940_40_009149322/jobs/climate.cpdc, line 393, position 19 Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003CACB0 Unknown Unknown Unknown hadcm3n_um_6.07_i 00261D1E Unknown Unknown Unknown hadcm3n_um_6.07_i 0016269A Unknown Unknown Unknown hadcm3n_um_6.07_i 00164453 Unknown Unknown Unknown hadcm3n_um_6.07_i 0038CE7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0038A1ED Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=87556, iMonCtr=1 Model crash detected, will try to restart... 10:44:55 (87556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:49:55 (9807): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:59:55 (10304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:19:58 (12567): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:17 (16955): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:24:59 (17434): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:29:59 (17785): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:39:59 (18626): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:40:00 (18626): No heartbeat from core client for 30 sec - exiting 11:44:59 (20674): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:48:18 (21658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:49:59 (21716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:50:01 (21716): No heartbeat from core client for 30 sec - exiting 11:53:31 (22137): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:53:33 (22137): No heartbeat from core client for 30 sec - exiting 11:53:34 (22137): No heartbeat from core client for 30 sec - exiting 11:53:35 (22137): No heartbeat from core client for 30 sec - exiting 11:53:37 (22137): No heartbeat from core client for 30 sec - exiting 11:53:38 (22137): No heartbeat from core client for 30 sec - exiting 11:53:39 (22137): No heartbeat from core client for 30 sec - exiting 11:53:40 (22137): No heartbeat from core client for 30 sec - exiting 11:53:41 (22137): No heartbeat from core client for 30 sec - exiting 11:53:42 (22137): No heartbeat from core client for 30 sec - exiting 11:53:43 (22137): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation 12:00:12 (22645): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 14:12:24 (1651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:09 (1816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:10 (1816): No heartbeat from core client for 30 sec - exiting 14:17:11 (1816): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Suspended CPDN Monitor - Suspend request from BOINC... forrtl: severe (17): syntax error in NAMELIST input, unit 5, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_xa1w_1940_40_009149322/jobs/climate.cpdc, line 393, position 19 Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003CACB0 Unknown Unknown Unknown hadcm3n_um_6.07_i 00261D1E Unknown Unknown Unknown hadcm3n_um_6.07_i 0016269A Unknown Unknown Unknown hadcm3n_um_6.07_i 00164453 Unknown Unknown Unknown hadcm3n_um_6.07_i 0038CE7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0038A1ED Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1894, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 14:22:33 (1894): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:22:34 (1894): No heartbeat from core client for 30 sec - exiting 14:22:36 (1894): No heartbeat from core client for 30 sec - exiting 14:22:37 (1894): No heartbeat from core client for 30 sec - exiting 14:28:46 (2953): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:32:38 (3621): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:38:20 (4517): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:21 (4517): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 14:44:14 (5409): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:50:16 (6298): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:56:59 (6714): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:00 (7697): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:07:00 (8502): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:00 (9444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:01 (9444): No heartbeat from core client for 30 sec - exiting 15:12:03 (9444): No heartbeat from core client for 30 sec - exiting 15:12:04 (9444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:07:42 (2965): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/xa1wko.pje6c10 is not a valid UM file. Error converting file to netcdf: dataout/xa1wko.pje6c10 Error: Input file: dataout/xa1wko.pie6c10 is not a valid UM file. Error converting file to netcdf: dataout/xa1wko.pie6c10 Error: Input file: dataout/xa1wko.pfe6c10 is not a valid UM file. Error converting file to netcdf: dataout/xa1wko.pfe6c10 Error: Input file: dataout/xa1wka.phe6c10 is not a valid UM file. Error converting file to netcdf: dataout/xa1wka.phe6c10 Error: Input file: dataout/xa1wka.pge6c10 is not a valid UM file. Error converting file to netcdf: dataout/xa1wka.pge6c10 Error: Input file: dataout/xa1wka.pee6c10 is not a valid UM file. Error converting file to netcdf: dataout/xa1wka.pee6c10 Error: Input file: dataout/xa1wka.pde6c10 is not a valid UM file. Error converting file to netcdf: dataout/xa1wka.pde6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:09:44 (46659): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:24:46 (60719): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:29:47 (63660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:38:14 (64464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:50:42 (66229): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:56:07 (63464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:08 (63464): No heartbeat from core client for 30 sec - exiting 18:56:09 (63464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:05:22 (63967): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:08:11 (64000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:12 (64000): No heartbeat from core client for 30 sec - exiting 19:08:13 (64000): No heartbeat from core client for 30 sec - exiting 19:08:14 (64000): No heartbeat from core client for 30 sec - exiting 19:11:31 (64182): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:19:00 (65032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... forrtl: severe (17): syntax error in NAMELIST input, unit 5, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_xa1w_1940_40_009149322/jobs/climate.cpdc, line 393, position 19 Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003CACB0 Unknown Unknown Unknown hadcm3n_um_6.07_i 00261D1E Unknown Unknown Unknown hadcm3n_um_6.07_i 0016269A Unknown Unknown Unknown hadcm3n_um_6.07_i 00164453 Unknown Unknown Unknown hadcm3n_um_6.07_i 0038CE7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0038A1ED Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=65073, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 22:19:12 (1606): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:50:55 (5812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:50:58 (32309): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:50:52 (50046): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:24:19 (61204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:24:56 (61226): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:26:37 (61244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:32:20 (61273): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:37:21 (61353): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:37:22 (61353): No heartbeat from core client for 30 sec - exiting 18:42:23 (61426): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:24 (61493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:25 (61493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:22:27 (62078): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:43 (62189): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(75538,0xa039f1d4) malloc: * error for object 0x201f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(75538,0xa039f1d4) malloc: * error for object 0x3026c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(75538,0xa039f1d4) malloc: * error for object 0x201f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(75538,0xa039f1d4) malloc: * error for object 0x3026c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(75538,0xa039f1d4) malloc: * error for object 0x201f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(75538,0xa039f1d4) malloc: * error for object 0x848604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(1587,0xa039f1d4) malloc: * error for object 0x1024604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug 16:12:35 (1587): No heartbeat from core client for 30 sec - exiting 16:12:36 (1587): No heartbeat from core client for 30 sec - exiting 16:12:37 (1587): No heartbeat from core client for 30 sec - exiting hadcm3n_6.07_i686-apple-darwin(1587,0xa039f1d4) malloc: * error for object 0x402f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug 16:12:38 (1587): No heartbeat from core client for 30 sec - exiting 16:12:39 (1587): No heartbeat from core client for 30 sec - exiting hadcm3n_6.07_i686-apple-darwin(1587,0xa039f1d4) malloc: * error for object 0x1024604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug 16:12:40 (1587): No heartbeat from core client for 30 sec - exiting 16:12:41 (1587): No heartbeat from core client for 30 sec - exiting 16:12:42 (1587): No heartbeat from core client for 30 sec - exiting 16:12:43 (1587): No heartbeat from core client for 30 sec - exiting 16:12:44 (1587): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation 16:12:45 (1587): No heartbeat from core client for 30 sec - exiting Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.2 build 14C109 Tue Feb 17 16:12:46 2015 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. 16:12:46 (1587): No heartbeat from core client for 30 sec - exiting 16:12:47 (1587): No heartbeat from core client for 30 sec - exiting Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fc98 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/1/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x902ce000 - 0x902d6fff /usr/lib/system/libsystem_pthread.dylib 0x902d7000 - 0x902e0fff /usr/lib/system/libcopyfile.dylib 0x90319000 - 0x9031afff /usr/lib/system/libsystem_blocks.dylib 0x90f25000 - 0x90f2afff /usr/lib/system/libcompiler_rt.dylib 0x90f50000 - 0x90f52fff /usr/lib/system/libsystem_sandbox.dylib 0x91e16000 - 0x91e16fff /usr/lib/system/libunc.dylib 0x91f4a000 - 0x91f50fff /usr/lib/system/libsystem_trace.dylib 0x93f0b000 - 0x93f0dfff /usr/lib/system/libsystem_coreservices.dylib 0x9427e000 - 0x9427efff /usr/lib/system/libkeymgr.dylib 0x947a2000 - 0x947c9fff /usr/lib/system/libdispatch.dylib 0x94892000 - 0x948b1fff /usr/lib/system/libsystem_kernel.dylib 0x950da000 - 0x95117fff /usr/lib/system/libsystem_network.dylib 0x9511d000 - 0x95186fff /usr/lib/system/libcorecrypto.dylib 0x95561000 - 0x95566fff /usr/lib/system/libmacho.dylib 0x9721f000 - 0x97245fff /usr/lib/libc++abi.dylib 0x9725c000 - 0x9725cfff /usr/lib/system/liblaunch.dylib 0x9725d000 - 0x97269fff /usr/lib/system/libcommonCrypto.dylib 0x97670000 - 0x97705fff /usr/lib/system/libsystem_c.dylib 0x978fa000 - 0x978fefff /usr/lib/system/libcache.dylib 0x97ac0000 - 0x97acefff /usr/lib/libz.1.dylib 0x97dd5000 - 0x97dd8fff /usr/lib/system/libdyld.dylib 0x98469000 - 0x9846bfff /usr/lib/system/libsystem_configuration.dylib 0x984a6000 - 0x984bdfff /usr/lib/system/libsystem_asl.dylib 0x99632000 - 0x99638fff /usr/lib/system/libsystem_networkextension.dylib 0x99639000 - 0x9963bfff /usr/lib/system/libsystem_secinit.dylib 0x99e84000 - 0x99e9dfff /usr/lib/system/libsystem_malloc.dylib 0x9a753000 - 0x9a75afff /usr/lib/system/libunwind.dylib 0x9aa03000 - 0x9aa04fff /usr/lib/libSystem.B.dylib 0x9aa4b000 - 0x9aa4dfff /usr/lib/system/libquarantine.dylib 0x9aa4e000 - 0x9aa4ffff /usr/lib/system/libremovefile.dylib 0x9b414000 - 0x9b467fff /usr/lib/libstdc++.6.dylib 0x9b99a000 - 0x9b9a3fff /usr/lib/system/libsystem_notify.dylib 0x9b9d2000 - 0x9b9d8fff /usr/lib/system/libsystem_platform.dylib 0x9ba57000 - 0x9ba68fff /usr/lib/system/libsystem_coretls.dylib 0x9bee0000 - 0x9bee8fff /usr/lib/system/libsystem_dnssd.dylib 0x9bee9000 - 0x9bf12fff /usr/lib/system/libsystem_info.dylib 0x9c07e000 - 0x9c0b1fff /usr/lib/system/libsystem_m.dylib 0x9c36a000 - 0x9c390fff /usr/lib/system/libxpc.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Feb 2015 14:19:45	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	259,200	461,046	1.7787
16 Feb 2015 23:24:10	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	233,280	414,336	1.7761
16 Feb 2015 04:44:25	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	207,360	367,856	1.7740
15 Feb 2015 12:48:34	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	181,440	321,309	1.7709
14 Feb 2015 18:55:48	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	155,520	275,708	1.7728
13 Feb 2015 23:33:00	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	129,600	229,981	1.7745
13 Feb 2015 06:18:45	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	103,680	183,725	1.7720
12 Feb 2015 07:26:25	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	77,760	137,553	1.7689
11 Feb 2015 15:38:25	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	51,840	91,296	1.7611
09 Feb 2015 08:43:12	1298787	17909446	hadcm3n_xa1w_1940_40_009149322_2	25,920	45,225	1.7448