Name | oifs_43r3_ps_0729_2001050100_123_970_12187373_1 |
Workunit | 12187373 |
Created | 22 Jan 2023, 1:08:52 UTC |
Sent | 22 Jan 2023, 1:27:38 UTC |
Report deadline | 23 Mar 2023, 1:27:38 UTC |
Received | 23 Jan 2023, 1:55:41 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1538374 |
Run time | 2 hours 43 min 29 sec |
CPU time | 2 hours 35 min 18 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 1.00 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,409.32 MB |
Peak swap size | 4,978.07 MB |
Peak disk usage | 2,095.96 MB |
Stderr | <core_client_version>7.20.2</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:153:153] [20230121:183106:1674354666:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 18:32:00 STEP 0 H= 0:00 +CPU= 51.099 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000000 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000000 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000000 18:32:36 STEP 1 H= 1:00 +CPU= 35.521 18:33:07 STEP 2 H= 2:00 +CPU= 30.919 18:33:56 STEP 3 H= 3:00 +CPU= 48.026 18:34:27 STEP 4 H= 4:00 +CPU= 31.093 18:34:57 STEP 5 H= 5:00 +CPU= 29.874 18:35:46 STEP 6 H= 6:00 +CPU= 48.256 18:36:16 STEP 7 H= 7:00 +CPU= 29.903 18:36:48 STEP 8 H= 8:00 +CPU= 31.045 18:37:35 STEP 9 H= 9:00 +CPU= 47.214 18:38:06 STEP 10 H= 10:00 +CPU= 30.760 18:38:37 STEP 11 H= 11:00 +CPU= 30.125 18:39:30 STEP 12 H= 12:00 +CPU= 52.333 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000012 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000012 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000012 18:40:00 STEP 13 H= 13:00 +CPU= 29.852 18:40:31 STEP 14 H= 14:00 +CPU= 30.840 18:41:19 STEP 15 H= 15:00 +CPU= 46.965 18:41:50 STEP 16 H= 16:00 +CPU= 30.795 18:42:20 STEP 17 H= 17:00 +CPU= 29.881 18:43:09 STEP 18 H= 18:00 +CPU= 48.271 18:43:38 STEP 19 H= 19:00 +CPU= 29.604 18:44:09 STEP 20 H= 20:00 +CPU= 30.636 18:44:57 STEP 21 H= 21:00 +CPU= 46.912 18:45:28 STEP 22 H= 22:00 +CPU= 30.793 18:46:00 STEP 23 H= 23:00 +CPU= 31.255 18:46:55 STEP 24 H= 24:00 +CPU= 54.675 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000024 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000024 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000024 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000000 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000000 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000000 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000012 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000012 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000012 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_0.zip Uploading the intermediate file: upload_file_0.zip 18:47:25 STEP 25 H= 25:00 +CPU= 29.616 18:47:56 STEP 26 H= 26:00 +CPU= 30.559 18:48:43 STEP 27 H= 27:00 +CPU= 47.015 18:49:14 STEP 28 H= 28:00 +CPU= 30.861 18:49:45 STEP 29 H= 29:00 +CPU= 29.870 18:50:33 STEP 30 H= 30:00 +CPU= 48.315 18:51:03 STEP 31 H= 31:00 +CPU= 29.748 18:51:35 STEP 32 H= 32:00 +CPU= 31.072 18:52:23 STEP 33 H= 33:00 +CPU= 47.435 18:52:54 STEP 34 H= 34:00 +CPU= 30.899 18:53:24 STEP 35 H= 35:00 +CPU= 29.850 18:54:17 STEP 36 H= 36:00 +CPU= 52.136 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000036 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000036 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000036 18:54:48 STEP 37 H= 37:00 +CPU= 30.176 18:55:19 STEP 38 H= 38:00 +CPU= 30.667 18:56:06 STEP 39 H= 39:00 +CPU= 46.992 18:56:38 STEP 40 H= 40:00 +CPU= 30.995 18:57:07 STEP 41 H= 41:00 +CPU= 29.687 18:57:56 STEP 42 H= 42:00 +CPU= 48.566 18:58:26 STEP 43 H= 43:00 +CPU= 29.706 18:58:57 STEP 44 H= 44:00 +CPU= 30.579 18:59:45 STEP 45 H= 45:00 +CPU= 47.043 19:00:16 STEP 46 H= 46:00 +CPU= 31.390 19:00:48 STEP 47 H= 47:00 +CPU= 31.426 19:01:45 STEP 48 H= 48:00 +CPU= 56.050 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000048 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000048 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000048 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000024 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000024 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000024 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000036 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000036 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000036 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_1.zip Uploading the intermediate file: upload_file_1.zip 19:02:15 STEP 49 H= 49:00 +CPU= 29.727 19:02:45 STEP 50 H= 50:00 +CPU= 30.573 19:03:33 STEP 51 H= 51:00 +CPU= 46.871 19:04:05 STEP 52 H= 52:00 +CPU= 31.240 19:04:35 STEP 53 H= 53:00 +CPU= 30.289 19:05:24 STEP 54 H= 54:00 +CPU= 48.378 19:05:54 STEP 55 H= 55:00 +CPU= 29.729 19:06:26 STEP 56 H= 56:00 +CPU= 30.810 19:07:13 STEP 57 H= 57:00 +CPU= 47.148 19:07:44 STEP 58 H= 58:00 +CPU= 30.505 19:08:14 STEP 59 H= 59:00 +CPU= 30.003 19:09:07 STEP 60 H= 60:00 +CPU= 52.258 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000060 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000060 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000060 19:09:38 STEP 61 H= 61:00 +CPU= 30.637 19:10:10 STEP 62 H= 62:00 +CPU= 31.518 19:10:58 STEP 63 H= 63:00 +CPU= 47.438 19:11:29 STEP 64 H= 64:00 +CPU= 30.921 19:11:59 STEP 65 H= 65:00 +CPU= 29.744 19:12:48 STEP 66 H= 66:00 +CPU= 48.152 19:13:18 STEP 67 H= 67:00 +CPU= 29.709 19:13:49 STEP 68 H= 68:00 +CPU= 30.588 19:14:36 STEP 69 H= 69:00 +CPU= 46.474 19:15:07 STEP 70 H= 70:00 +CPU= 30.871 19:15:39 STEP 71 H= 71:00 +CPU= 30.952 19:16:34 STEP 72 H= 72:00 +CPU= 54.857 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000072 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000072 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000072 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000048 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000048 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000048 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000060 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000060 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000060 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_2.zip Uploading the intermediate file: upload_file_2.zip 19:17:04 STEP 73 H= 73:00 +CPU= 29.549 19:17:35 STEP 74 H= 74:00 +CPU= 30.781 19:18:22 STEP 75 H= 75:00 +CPU= 46.724 19:18:53 STEP 76 H= 76:00 +CPU= 30.708 19:19:24 STEP 77 H= 77:00 +CPU= 30.124 19:20:13 STEP 78 H= 78:00 +CPU= 48.456 19:20:43 STEP 79 H= 79:00 +CPU= 29.848 19:21:14 STEP 80 H= 80:00 +CPU= 30.582 19:22:01 STEP 81 H= 81:00 +CPU= 46.822 19:22:32 STEP 82 H= 82:00 +CPU= 31.011 19:23:03 STEP 83 H= 83:00 +CPU= 30.001 19:23:56 STEP 84 H= 84:00 +CPU= 52.687 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000084 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000084 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000084 19:24:27 STEP 85 H= 85:00 +CPU= 30.602 19:24:58 STEP 86 H= 86:00 +CPU= 30.724 19:25:45 STEP 87 H= 87:00 +CPU= 46.985 19:26:17 STEP 88 H= 88:00 +CPU= 31.137 19:26:47 STEP 89 H= 89:00 +CPU= 29.832 19:27:36 STEP 90 H= 90:00 +CPU= 48.497 19:28:06 STEP 91 H= 91:00 +CPU= 29.830 19:28:37 STEP 92 H= 92:00 +CPU= 30.772 19:29:25 STEP 93 H= 93:00 +CPU= 47.582 19:29:57 STEP 94 H= 94:00 +CPU= 30.961 19:30:28 STEP 95 H= 95:00 +CPU= 30.777 19:31:24 STEP 96 H= 96:00 +CPU= 55.867 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000096 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000096 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000096 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000072 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000072 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000072 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000084 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000084 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000084 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_3.zip Uploading the intermediate file: upload_file_3.zip 19:31:55 STEP 97 H= 97:00 +CPU= 30.069 19:32:25 STEP 98 H= 98:00 +CPU= 30.568 19:33:13 STEP 99 H= 99:00 +CPU= 46.814 19:33:44 STEP 100 H= 100:00 +CPU= 30.715 19:34:15 STEP 101 H= 101:00 +CPU= 30.349 19:35:03 STEP 102 H= 102:00 +CPU= 48.113 19:35:33 STEP 103 H= 103:00 +CPU= 29.786 19:36:05 STEP 104 H= 104:00 +CPU= 31.206 19:36:52 STEP 105 H= 105:00 +CPU= 46.932 19:37:24 STEP 106 H= 106:00 +CPU= 30.972 19:37:54 STEP 107 H= 107:00 +CPU= 30.073 19:38:48 STEP 108 H= 108:00 +CPU= 52.526 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000108 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000108 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000108 19:39:20 STEP 109 H= 109:00 +CPU= 30.259 19:39:52 STEP 110 H= 110:00 +CPU= 31.027 19:40:40 STEP 111 H= 111:00 +CPU= 47.288 19:41:11 STEP 112 H= 112:00 +CPU= 31.037 19:41:41 STEP 113 H= 113:00 +CPU= 30.056 19:42:30 STEP 114 H= 114:00 +CPU= 48.018 19:43:00 STEP 115 H= 115:00 +CPU= 29.621 19:43:31 STEP 116 H= 116:00 +CPU= 31.262 19:44:19 STEP 117 H= 117:00 +CPU= 46.661 19:44:50 STEP 118 H= 118:00 +CPU= 30.721 19:45:21 STEP 119 H= 119:00 +CPU= 31.100 19:46:17 STEP 120 H= 120:00 +CPU= 55.327 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000120 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000120 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000120 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000096 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000096 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000096 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000108 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000108 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000108 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_4.zip Uploading the intermediate file: upload_file_4.zip 19:46:48 STEP 121 H= 121:00 +CPU= 29.837 19:47:19 STEP 122 H= 122:00 +CPU= 31.261 19:48:07 STEP 123 H= 123:00 +CPU= 46.934 19:48:38 STEP 124 H= 124:00 +CPU= 30.925 19:49:08 STEP 125 H= 125:00 +CPU= 29.666 19:49:56 STEP 126 H= 126:00 +CPU= 47.769 19:50:26 STEP 127 H= 127:00 +CPU= 29.642 19:50:57 STEP 128 H= 128:00 +CPU= 30.653 19:51:44 STEP 129 H= 129:00 +CPU= 46.805 19:52:15 STEP 130 H= 130:00 +CPU= 30.646 19:52:45 STEP 131 H= 131:00 +CPU= 29.677 19:53:38 STEP 132 H= 132:00 +CPU= 51.890 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000132 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000132 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000132 19:54:08 STEP 133 H= 133:00 +CPU= 29.638 19:54:39 STEP 134 H= 134:00 +CPU= 30.658 19:55:26 STEP 135 H= 135:00 +CPU= 46.766 19:55:57 STEP 136 H= 136:00 +CPU= 30.555 19:56:27 STEP 137 H= 137:00 +CPU= 29.580 19:57:15 STEP 138 H= 138:00 +CPU= 47.753 19:57:45 STEP 139 H= 139:00 +CPU= 29.548 19:58:16 STEP 140 H= 140:00 +CPU= 30.537 19:59:03 STEP 141 H= 141:00 +CPU= 46.640 19:59:34 STEP 142 H= 142:00 +CPU= 30.530 20:00:05 STEP 143 H= 143:00 +CPU= 30.631 20:01:01 STEP 144 H= 144:00 +CPU= 55.004 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000144 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000144 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000144 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000120 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000120 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000120 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000132 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000132 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000132 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_5.zip Uploading the intermediate file: upload_file_5.zip 20:01:31 STEP 145 H= 145:00 +CPU= 29.577 20:02:01 STEP 146 H= 146:00 +CPU= 30.489 20:02:49 STEP 147 H= 147:00 +CPU= 46.671 20:03:20 STEP 148 H= 148:00 +CPU= 30.541 20:03:49 STEP 149 H= 149:00 +CPU= 29.637 20:04:38 STEP 150 H= 150:00 +CPU= 47.801 20:05:08 STEP 151 H= 151:00 +CPU= 29.618 20:05:39 STEP 152 H= 152:00 +CPU= 30.604 20:06:26 STEP 153 H= 153:00 +CPU= 46.754 20:06:57 STEP 154 H= 154:00 +CPU= 30.596 20:07:27 STEP 155 H= 155:00 +CPU= 29.650 20:08:19 STEP 156 H= 156:00 +CPU= 51.718 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000156 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000156 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000156 20:08:49 STEP 157 H= 157:00 +CPU= 29.748 20:09:20 STEP 158 H= 158:00 +CPU= 30.577 20:10:07 STEP 159 H= 159:00 +CPU= 46.730 20:10:38 STEP 160 H= 160:00 +CPU= 30.681 20:11:08 STEP 161 H= 161:00 +CPU= 29.570 20:11:56 STEP 162 H= 162:00 +CPU= 47.809 20:12:26 STEP 163 H= 163:00 +CPU= 29.736 20:12:57 STEP 164 H= 164:00 +CPU= 30.670 20:13:45 STEP 165 H= 165:00 +CPU= 46.679 20:14:15 STEP 166 H= 166:00 +CPU= 30.553 20:14:46 STEP 167 H= 167:00 +CPU= 30.543 20:15:42 STEP 168 H= 168:00 +CPU= 54.774 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000168 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000168 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000168 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000144 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000144 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000144 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000156 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000156 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000156 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_6.zip Uploading the intermediate file: upload_file_6.zip 20:16:12 STEP 169 H= 169:00 +CPU= 29.634 20:16:43 STEP 170 H= 170:00 +CPU= 30.612 20:17:30 STEP 171 H= 171:00 +CPU= 46.635 20:18:01 STEP 172 H= 172:00 +CPU= 30.658 20:18:31 STEP 173 H= 173:00 +CPU= 29.672 20:19:19 STEP 174 H= 174:00 +CPU= 47.738 20:19:49 STEP 175 H= 175:00 +CPU= 29.714 20:20:20 STEP 176 H= 176:00 +CPU= 30.668 20:21:07 STEP 177 H= 177:00 +CPU= 46.720 20:21:38 STEP 178 H= 178:00 +CPU= 30.686 20:22:08 STEP 179 H= 179:00 +CPU= 29.760 20:23:00 STEP 180 H= 180:00 +CPU= 51.890 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000180 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000180 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000180 20:23:30 STEP 181 H= 181:00 +CPU= 29.742 20:24:01 STEP 182 H= 182:00 +CPU= 30.729 20:24:48 STEP 183 H= 183:00 +CPU= 46.678 20:25:19 STEP 184 H= 184:00 +CPU= 30.652 20:25:49 STEP 185 H= 185:00 +CPU= 29.732 20:26:38 STEP 186 H= 186:00 +CPU= 47.738 20:27:08 STEP 187 H= 187:00 +CPU= 29.679 20:27:39 STEP 188 H= 188:00 +CPU= 30.586 20:28:26 STEP 189 H= 189:00 +CPU= 46.600 20:28:56 STEP 190 H= 190:00 +CPU= 30.573 20:29:28 STEP 191 H= 191:00 +CPU= 30.699 20:30:23 STEP 192 H= 192:00 +CPU= 54.869 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000192 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000192 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000192 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000168 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000168 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000168 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000180 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000180 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000180 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_7.zip Uploading the intermediate file: upload_file_7.zip 20:30:53 STEP 193 H= 193:00 +CPU= 29.683 20:31:24 STEP 194 H= 194:00 +CPU= 30.609 20:32:11 STEP 195 H= 195:00 +CPU= 46.697 20:32:42 STEP 196 H= 196:00 +CPU= 30.662 20:33:12 STEP 197 H= 197:00 +CPU= 29.703 20:34:00 STEP 198 H= 198:00 +CPU= 47.842 20:34:31 STEP 199 H= 199:00 +CPU= 29.791 20:35:02 STEP 200 H= 200:00 +CPU= 30.766 20:35:49 STEP 201 H= 201:00 +CPU= 46.799 20:36:20 STEP 202 H= 202:00 +CPU= 30.713 20:36:50 STEP 203 H= 203:00 +CPU= 29.764 20:37:42 STEP 204 H= 204:00 +CPU= 51.927 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000204 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000204 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000204 20:38:12 STEP 205 H= 205:00 +CPU= 29.754 20:38:43 STEP 206 H= 206:00 +CPU= 30.756 20:39:31 STEP 207 H= 207:00 +CPU= 46.762 20:40:02 STEP 208 H= 208:00 +CPU= 30.724 20:40:32 STEP 209 H= 209:00 +CPU= 29.752 20:41:20 STEP 210 H= 210:00 +CPU= 47.798 20:41:50 STEP 211 H= 211:00 +CPU= 29.661 20:42:21 STEP 212 H= 212:00 +CPU= 30.643 20:43:08 STEP 213 H= 213:00 +CPU= 46.639 20:43:39 STEP 214 H= 214:00 +CPU= 30.599 20:44:10 STEP 215 H= 215:00 +CPU= 30.705 Quit request received from BOINC client, ending the child process ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2001050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0729 (argv4) batchid: 970 (argv5) wuid: 12187373 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /home/alex/slots/9 Project directory is: /home/alex/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373 ..mkdir for temp folder for results failed Copying: /home/alex/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/alex/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/alex/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_0d09fed5f0fb08a37253c50993083ddc to: /home/alex/slots/9/oifs_43r3_ps_0729_2001050100_123_970_12187373.zip Unzipping the namelist zip file: /home/alex/slots/9/oifs_43r3_ps_0729_2001050100_123_970_12187373.zip ic_ancil_file: ic_ancil_12187373 ifsdata_file: ifsdata_12187373 climate_data_file: clim_data_12187373 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_36f2056d6ca5afd3c09d0f5fc40a2639 to: /home/alex/slots/9/ic_ancil_12187373.zip Unzipping the IC ancils zip file: /home/alex/slots/9/ic_ancil_12187373.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_377557b5bee2cba3039daeb30b97a612 to: /home/alex/slots/9/ifsdata/ifsdata_12187373.zip Unzipping the ifsdata_zip file: /home/alex/slots/9/ifsdata/ifsdata_12187373.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/alex/slots/9/159l_2/clim_data_12187373.zip Unzipping the climate data zip file: /home/alex/slots/9/159l_2/clim_data_12187373.zip Checking for progress XML file: /home/alex/slots/9/progress_file_12187373.xml Opened progress file ok : /home/alex/slots/9/progress_file_12187373.xml -- Model is restarting -- Adjusting last_iter, 216, to previous model restart step. Creating progress file: /home/alex/slots/9/progress_file_12187373.xml last_cpu_time: 7942 upload_file_number: 8 last_iter: 215 last_upload: 691200 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061 The child process has been launched with process id: 126 Executing the command: /home/alex/slots/9/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:126:126] [20230121:214627:1674366387:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> The child process terminated with status: 0 ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2001050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0729 (argv4) batchid: 970 (argv5) wuid: 12187373 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /home/alex/slots/9 Project directory is: /home/alex/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373 ..mkdir for temp folder for results failed Copying: /home/alex/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/alex/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/alex/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_0d09fed5f0fb08a37253c50993083ddc to: /home/alex/slots/9/oifs_43r3_ps_0729_2001050100_123_970_12187373.zip Unzipping the namelist zip file: /home/alex/slots/9/oifs_43r3_ps_0729_2001050100_123_970_12187373.zip ic_ancil_file: ic_ancil_12187373 ifsdata_file: ifsdata_12187373 climate_data_file: clim_data_12187373 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_36f2056d6ca5afd3c09d0f5fc40a2639 to: /home/alex/slots/9/ic_ancil_12187373.zip Unzipping the IC ancils zip file: /home/alex/slots/9/ic_ancil_12187373.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_377557b5bee2cba3039daeb30b97a612 to: /home/alex/slots/9/ifsdata/ifsdata_12187373.zip Unzipping the ifsdata_zip file: /home/alex/slots/9/ifsdata/ifsdata_12187373.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/alex/slots/9/159l_2/clim_data_12187373.zip Unzipping the climate data zip file: /home/alex/slots/9/159l_2/clim_data_12187373.zip Checking for progress XML file: /home/alex/slots/9/progress_file_12187373.xml Opened progress file ok : /home/alex/slots/9/progress_file_12187373.xml -- Model is restarting -- Adjusting last_iter, 216, to previous model restart step. Creating progress file: /home/alex/slots/9/progress_file_12187373.xml last_cpu_time: 7949 upload_file_number: 8 last_iter: 215 last_upload: 691200 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061 The child process has been launched with process id: 121 Executing the command: /home/alex/slots/9/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:121:121] [20230121:215322:1674366802:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 21:54:43 STEP 216 H= 216:00 +CPU= 52.267 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000216 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000216 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000216 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000192 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000192 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000192 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000204 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000204 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000204 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_8.zip Uploading the intermediate file: upload_file_8.zip 21:56:09 STEP 217 H= 217:00 +CPU= 31.181 21:56:45 STEP 218 H= 218:00 +CPU= 29.881 21:57:39 STEP 219 H= 219:00 +CPU= 45.341 21:58:12 STEP 220 H= 220:00 +CPU= 29.746 21:58:46 STEP 221 H= 221:00 +CPU= 29.027 21:59:38 STEP 222 H= 222:00 +CPU= 46.276 22:00:11 STEP 223 H= 223:00 +CPU= 28.970 22:00:45 STEP 224 H= 224:00 +CPU= 29.920 22:01:36 STEP 225 H= 225:00 +CPU= 45.254 22:02:10 STEP 226 H= 226:00 +CPU= 29.582 22:02:43 STEP 227 H= 227:00 +CPU= 29.212 22:03:46 STEP 228 H= 228:00 +CPU= 50.296 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000228 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000228 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000228 22:04:15 STEP 229 H= 229:00 +CPU= 28.236 22:04:45 STEP 230 H= 230:00 +CPU= 29.112 22:05:31 STEP 231 H= 231:00 +CPU= 44.470 22:06:01 STEP 232 H= 232:00 +CPU= 29.136 22:06:30 STEP 233 H= 233:00 +CPU= 28.136 22:07:16 STEP 234 H= 234:00 +CPU= 45.334 22:07:45 STEP 235 H= 235:00 +CPU= 28.318 22:08:15 STEP 236 H= 236:00 +CPU= 29.087 22:09:03 STEP 237 H= 237:00 +CPU= 44.604 22:09:37 STEP 238 H= 238:00 +CPU= 29.682 22:10:11 STEP 239 H= 239:00 +CPU= 30.072 (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2001050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0729 (argv4) batchid: 970 (argv5) wuid: 12187373 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /home/alex/slots/9 Project directory is: /home/alex/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373 ..mkdir for temp folder for results failed Copying: /home/alex/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/alex/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/alex/slots/9/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_0d09fed5f0fb08a37253c50993083ddc to: /home/alex/slots/9/oifs_43r3_ps_0729_2001050100_123_970_12187373.zip Unzipping the namelist zip file: /home/alex/slots/9/oifs_43r3_ps_0729_2001050100_123_970_12187373.zip ic_ancil_file: ic_ancil_12187373 ifsdata_file: ifsdata_12187373 climate_data_file: clim_data_12187373 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_36f2056d6ca5afd3c09d0f5fc40a2639 to: /home/alex/slots/9/ic_ancil_12187373.zip Unzipping the IC ancils zip file: /home/alex/slots/9/ic_ancil_12187373.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_377557b5bee2cba3039daeb30b97a612 to: /home/alex/slots/9/ifsdata/ifsdata_12187373.zip Unzipping the ifsdata_zip file: /home/alex/slots/9/ifsdata/ifsdata_12187373.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/alex/slots/9/159l_2/clim_data_12187373.zip Unzipping the climate data zip file: /home/alex/slots/9/159l_2/clim_data_12187373.zip Checking for progress XML file: /home/alex/slots/9/progress_file_12187373.xml Opened progress file ok : /home/alex/slots/9/progress_file_12187373.xml -- Model is restarting -- Adjusting last_iter, 240, to previous model restart step. Creating progress file: /home/alex/slots/9/progress_file_12187373.xml last_cpu_time: 8834 upload_file_number: 9 last_iter: 239 last_upload: 777600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061 The child process has been launched with process id: 134 Executing the command: /home/alex/slots/9/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:REDHEATER:1:1:134:134] [20230122:001857:1674375537:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 00:20:06 STEP 240 H= 240:00 +CPU= 46.504 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000240 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000240 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000240 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000216 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000216 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000216 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMGGhq0f+000228 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMSHhq0f+000228 Adding to the zip: /home/alex/projects/climateprediction.net/oifs_43r3_ps_12187373/ICMUAhq0f+000228 Zipping up the intermediate file: /home/alex/projects/climateprediction.net/oifs_43r3_ps_0729_2001050100_123_970_12187373_1_r1244593061_9.zip Uploading the intermediate file: upload_file_9.zip 00:20:33 STEP 241 H= 241:00 +CPU= 26.247 00:21:12 STEP 242 H= 242:00 +CPU= 29.439 00:21:55 STEP 243 H= 243:00 +CPU= 41.460 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:22:38 STEP 244 H= 244:00 +CPU= 26.724 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:23:17 STEP 245 H= 245:00 +CPU= 22.550 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:24:26 STEP 246 H= 246:00 +CPU= 39.297 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:25:03 STEP 247 H= 247:00 +CPU= 23.140 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:25:47 STEP 248 H= 248:00 +CPU= 24.813 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:26:51 STEP 249 H= 249:00 +CPU= 38.429 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:27:38 STEP 250 H= 250:00 +CPU= 26.782 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:28:22 STEP 251 H= 251:00 +CPU= 24.945 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:29:23 STEP 252 H= 252:00 +CPU= 42.546 Moving to projects directory: /home/alex/slots/9/ICMGGhq0f+000252 Moving to projects directory: /home/alex/slots/9/ICMSHhq0f+000252 Moving to projects directory: /home/alex/slots/9/ICMUAhq0f+000252 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:30:01 STEP 253 H= 253:00 +CPU= 23.475 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:30:38 STEP 254 H= 254:00 +CPU= 25.013 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process The child process terminated with status: 0 ..Failed, model did not complete successfully </stderr_txt> ]]> |
No trickles! |
---|
©2024 cpdn.org