climateprediction.net home page
Task 22258641

Task 22258641

Name oifs_43r3_ps_0391_1988050100_123_957_12174035_0
Workunit 12174035
Created 21 Dec 2022, 13:47:23 UTC
Sent 14 Jan 2023, 7:45:37 UTC
Report deadline 13 Feb 2023, 7:45:37 UTC
Received 16 Jan 2023, 3:20:02 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1534496
Run time 17 hours 32 min 35 sec
CPU time 17 hours 13 min 41 sec
Validate state Invalid
Credit 13,063.68
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,405.70 MB
Peak swap size 4,978.08 MB
Peak disk usage 5,309.05 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
oinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_85.zip
Uploading the intermediate file: upload_file_85.zip
  15:40:53 STEP 2065 H=2065:00 +CPU= 18.639
  15:41:16 STEP 2066 H=2066:00 +CPU= 23.365
  15:41:48 STEP 2067 H=2067:00 +CPU= 31.389
  15:42:09 STEP 2068 H=2068:00 +CPU= 21.140
  15:42:30 STEP 2069 H=2069:00 +CPU= 20.375
  15:43:00 STEP 2070 H=2070:00 +CPU= 30.762
  15:43:20 STEP 2071 H=2071:00 +CPU= 19.173
  15:43:39 STEP 2072 H=2072:00 +CPU= 18.774
  15:44:12 STEP 2073 H=2073:00 +CPU= 32.752
  15:44:34 STEP 2074 H=2074:00 +CPU= 22.155
  15:44:55 STEP 2075 H=2075:00 +CPU= 20.649
  15:45:30 STEP 2076 H=2076:00 +CPU= 34.840
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002076
  15:45:49 STEP 2077 H=2077:00 +CPU= 19.305
  15:46:11 STEP 2078 H=2078:00 +CPU= 21.434
  15:46:43 STEP 2079 H=2079:00 +CPU= 32.501
  15:47:07 STEP 2080 H=2080:00 +CPU= 23.655
  15:47:25 STEP 2081 H=2081:00 +CPU= 18.101
  15:48:01 STEP 2082 H=2082:00 +CPU= 35.523
  15:48:23 STEP 2083 H=2083:00 +CPU= 21.363
  15:48:46 STEP 2084 H=2084:00 +CPU= 22.668
  15:49:19 STEP 2085 H=2085:00 +CPU= 33.482
  15:49:41 STEP 2086 H=2086:00 +CPU= 21.221
  15:50:02 STEP 2087 H=2087:00 +CPU= 21.345
  15:50:43 STEP 2088 H=2088:00 +CPU= 38.217
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_86.zip
Uploading the intermediate file: upload_file_86.zip
  15:51:06 STEP 2089 H=2089:00 +CPU= 22.408
  15:51:25 STEP 2090 H=2090:00 +CPU= 19.388
  15:51:58 STEP 2091 H=2091:00 +CPU= 31.854
  15:52:22 STEP 2092 H=2092:00 +CPU= 24.631
  15:52:44 STEP 2093 H=2093:00 +CPU= 21.519
  15:53:19 STEP 2094 H=2094:00 +CPU= 34.708
  15:53:40 STEP 2095 H=2095:00 +CPU= 20.909
  15:54:00 STEP 2096 H=2096:00 +CPU= 20.031
  15:54:33 STEP 2097 H=2097:00 +CPU= 32.519
  15:54:56 STEP 2098 H=2098:00 +CPU= 23.041
  15:55:16 STEP 2099 H=2099:00 +CPU= 19.916
  15:55:52 STEP 2100 H=2100:00 +CPU= 36.099
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002100
  15:56:11 STEP 2101 H=2101:00 +CPU= 18.571
  15:56:31 STEP 2102 H=2102:00 +CPU= 19.362
  15:57:01 STEP 2103 H=2103:00 +CPU= 30.369
  15:57:22 STEP 2104 H=2104:00 +CPU= 21.057
  15:57:47 STEP 2105 H=2105:00 +CPU= 24.017
  15:58:20 STEP 2106 H=2106:00 +CPU= 33.810
  15:58:43 STEP 2107 H=2107:00 +CPU= 22.106
  15:59:04 STEP 2108 H=2108:00 +CPU= 21.713
  15:59:37 STEP 2109 H=2109:00 +CPU= 32.314
  16:00:00 STEP 2110 H=2110:00 +CPU= 22.876
  16:00:22 STEP 2111 H=2111:00 +CPU= 21.496
  16:01:01 STEP 2112 H=2112:00 +CPU= 38.213
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_87.zip
Uploading the intermediate file: upload_file_87.zip
  16:01:23 STEP 2113 H=2113:00 +CPU= 21.907
  16:01:45 STEP 2114 H=2114:00 +CPU= 21.565
  16:02:18 STEP 2115 H=2115:00 +CPU= 32.290
  16:02:38 STEP 2116 H=2116:00 +CPU= 20.521
  16:02:59 STEP 2117 H=2117:00 +CPU= 20.901
  16:03:33 STEP 2118 H=2118:00 +CPU= 33.862
  16:03:55 STEP 2119 H=2119:00 +CPU= 21.918
  16:04:17 STEP 2120 H=2120:00 +CPU= 21.648
  16:04:48 STEP 2121 H=2121:00 +CPU= 30.305
  16:05:11 STEP 2122 H=2122:00 +CPU= 22.843
  16:05:30 STEP 2123 H=2123:00 +CPU= 19.523
  16:06:05 STEP 2124 H=2124:00 +CPU= 34.254
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002124
  16:06:26 STEP 2125 H=2125:00 +CPU= 21.265
  16:06:51 STEP 2126 H=2126:00 +CPU= 24.309
  16:07:22 STEP 2127 H=2127:00 +CPU= 30.910
  16:07:45 STEP 2128 H=2128:00 +CPU= 22.977
  16:08:06 STEP 2129 H=2129:00 +CPU= 20.310
  16:08:38 STEP 2130 H=2130:00 +CPU= 31.826
  16:08:59 STEP 2131 H=2131:00 +CPU= 20.636
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:09 STEP 2132 H=2132:00 +CPU= 22.113
  16:15:40 STEP 2133 H=2133:00 +CPU= 31.157
  16:16:03 STEP 2134 H=2134:00 +CPU= 22.607
  16:16:24 STEP 2135 H=2135:00 +CPU= 21.222
  16:17:02 STEP 2136 H=2136:00 +CPU= 37.063
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_88.zip
Uploading the intermediate file: upload_file_88.zip
  16:17:24 STEP 2137 H=2137:00 +CPU= 21.715
  16:17:47 STEP 2138 H=2138:00 +CPU= 22.560
  16:18:19 STEP 2139 H=2139:00 +CPU= 32.031
  16:18:39 STEP 2140 H=2140:00 +CPU= 20.344
  16:19:03 STEP 2141 H=2141:00 +CPU= 22.881
  16:19:36 STEP 2142 H=2142:00 +CPU= 32.827
  16:19:57 STEP 2143 H=2143:00 +CPU= 20.936
  16:20:22 STEP 2144 H=2144:00 +CPU= 24.814
  16:20:55 STEP 2145 H=2145:00 +CPU= 33.494
  16:21:19 STEP 2146 H=2146:00 +CPU= 23.510
  16:21:41 STEP 2147 H=2147:00 +CPU= 22.334
  16:22:21 STEP 2148 H=2148:00 +CPU= 39.933
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002148
  16:22:44 STEP 2149 H=2149:00 +CPU= 22.466
  16:23:04 STEP 2150 H=2150:00 +CPU= 19.582
  16:23:35 STEP 2151 H=2151:00 +CPU= 31.313
  16:23:56 STEP 2152 H=2152:00 +CPU= 20.498
  16:24:17 STEP 2153 H=2153:00 +CPU= 21.144
  16:24:51 STEP 2154 H=2154:00 +CPU= 33.331
  16:25:14 STEP 2155 H=2155:00 +CPU= 23.430
  16:25:36 STEP 2156 H=2156:00 +CPU= 22.010
  16:26:06 STEP 2157 H=2157:00 +CPU= 30.197
  16:26:26 STEP 2158 H=2158:00 +CPU= 19.644
  16:26:45 STEP 2159 H=2159:00 +CPU= 18.905
  16:27:24 STEP 2160 H=2160:00 +CPU= 37.861
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_89.zip
Uploading the intermediate file: upload_file_89.zip
  16:27:44 STEP 2161 H=2161:00 +CPU= 20.093
  16:28:07 STEP 2162 H=2162:00 +CPU= 22.545
  16:28:38 STEP 2163 H=2163:00 +CPU= 31.379
  16:29:01 STEP 2164 H=2164:00 +CPU= 22.865
  16:29:23 STEP 2165 H=2165:00 +CPU= 21.524
  16:29:54 STEP 2166 H=2166:00 +CPU= 30.991
  16:30:14 STEP 2167 H=2167:00 +CPU= 19.241
  16:30:34 STEP 2168 H=2168:00 +CPU= 20.504
  16:31:04 STEP 2169 H=2169:00 +CPU= 29.964
  16:31:26 STEP 2170 H=2170:00 +CPU= 21.698
  16:31:45 STEP 2171 H=2171:00 +CPU= 18.964
  16:32:21 STEP 2172 H=2172:00 +CPU= 35.019
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002172
  16:32:43 STEP 2173 H=2173:00 +CPU= 22.109
  16:33:06 STEP 2174 H=2174:00 +CPU= 22.778
  16:33:37 STEP 2175 H=2175:00 +CPU= 30.674
  16:33:59 STEP 2176 H=2176:00 +CPU= 21.284
  16:34:20 STEP 2177 H=2177:00 +CPU= 21.330
  16:34:52 STEP 2178 H=2178:00 +CPU= 31.567
  16:35:15 STEP 2179 H=2179:00 +CPU= 22.472
  16:35:38 STEP 2180 H=2180:00 +CPU= 23.200
  16:36:11 STEP 2181 H=2181:00 +CPU= 32.443
  16:36:31 STEP 2182 H=2182:00 +CPU= 20.136
  16:36:52 STEP 2183 H=2183:00 +CPU= 20.491
  16:37:31 STEP 2184 H=2184:00 +CPU= 38.416
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_90.zip
Uploading the intermediate file: upload_file_90.zip
  16:37:53 STEP 2185 H=2185:00 +CPU= 22.148
  16:38:17 STEP 2186 H=2186:00 +CPU= 23.472
  16:38:50 STEP 2187 H=2187:00 +CPU= 32.331
  16:39:08 STEP 2188 H=2188:00 +CPU= 18.770
  16:39:28 STEP 2189 H=2189:00 +CPU= 19.611
  16:40:02 STEP 2190 H=2190:00 +CPU= 33.684
  16:40:22 STEP 2191 H=2191:00 +CPU= 20.384
  16:40:44 STEP 2192 H=2192:00 +CPU= 21.217
  16:41:17 STEP 2193 H=2193:00 +CPU= 32.726
  16:41:36 STEP 2194 H=2194:00 +CPU= 19.700
  16:41:59 STEP 2195 H=2195:00 +CPU= 22.859
  16:42:35 STEP 2196 H=2196:00 +CPU= 35.997
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002196
  16:42:55 STEP 2197 H=2197:00 +CPU= 19.265
  16:43:16 STEP 2198 H=2198:00 +CPU= 21.118
  16:43:48 STEP 2199 H=2199:00 +CPU= 32.079
  16:44:06 STEP 2200 H=2200:00 +CPU= 17.823
  16:44:28 STEP 2201 H=2201:00 +CPU= 22.050
  16:45:03 STEP 2202 H=2202:00 +CPU= 34.314
  16:45:23 STEP 2203 H=2203:00 +CPU= 20.144
  16:45:46 STEP 2204 H=2204:00 +CPU= 22.759
  16:46:20 STEP 2205 H=2205:00 +CPU= 33.588
  16:46:41 STEP 2206 H=2206:00 +CPU= 21.553
  16:47:05 STEP 2207 H=2207:00 +CPU= 23.449
  16:47:45 STEP 2208 H=2208:00 +CPU= 39.456
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_91.zip
Uploading the intermediate file: upload_file_91.zip
  16:48:05 STEP 2209 H=2209:00 +CPU= 19.261
  16:48:28 STEP 2210 H=2210:00 +CPU= 22.887
  16:49:02 STEP 2211 H=2211:00 +CPU= 34.219
  16:49:24 STEP 2212 H=2212:00 +CPU= 21.543
  16:49:45 STEP 2213 H=2213:00 +CPU= 21.087
  16:50:19 STEP 2214 H=2214:00 +CPU= 34.290
  16:50:38 STEP 2215 H=2215:00 +CPU= 18.241
  16:50:59 STEP 2216 H=2216:00 +CPU= 21.523
  16:51:33 STEP 2217 H=2217:00 +CPU= 33.259
  16:51:52 STEP 2218 H=2218:00 +CPU= 18.773
  16:52:10 STEP 2219 H=2219:00 +CPU= 18.745
  16:52:47 STEP 2220 H=2220:00 +CPU= 35.959
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002220
  16:53:05 STEP 2221 H=2221:00 +CPU= 18.109
  16:53:24 STEP 2222 H=2222:00 +CPU= 18.401
  16:53:52 STEP 2223 H=2223:00 +CPU= 27.643
  16:54:12 STEP 2224 H=2224:00 +CPU= 20.653
  16:54:30 STEP 2225 H=2225:00 +CPU= 17.837
  16:55:04 STEP 2226 H=2226:00 +CPU= 33.821
  16:55:25 STEP 2227 H=2227:00 +CPU= 20.448
  16:55:46 STEP 2228 H=2228:00 +CPU= 20.926
  16:56:18 STEP 2229 H=2229:00 +CPU= 32.351
  16:56:36 STEP 2230 H=2230:00 +CPU= 17.514
  16:56:55 STEP 2231 H=2231:00 +CPU= 19.142
  16:57:31 STEP 2232 H=2232:00 +CPU= 35.096
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_92.zip
Uploading the intermediate file: upload_file_92.zip
  16:57:46 STEP 2233 H=2233:00 +CPU= 14.960
  16:58:01 STEP 2234 H=2234:00 +CPU= 14.759
  16:58:26 STEP 2235 H=2235:00 +CPU= 23.771
  16:58:40 STEP 2236 H=2236:00 +CPU= 14.856
  16:58:56 STEP 2237 H=2237:00 +CPU= 15.041
  16:59:20 STEP 2238 H=2238:00 +CPU= 24.237
  16:59:35 STEP 2239 H=2239:00 +CPU= 14.692
  16:59:50 STEP 2240 H=2240:00 +CPU= 15.072
  17:00:13 STEP 2241 H=2241:00 +CPU= 23.123
  17:00:28 STEP 2242 H=2242:00 +CPU= 14.691
  17:00:43 STEP 2243 H=2243:00 +CPU= 14.757
  17:01:08 STEP 2244 H=2244:00 +CPU= 25.707
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002244
  17:01:23 STEP 2245 H=2245:00 +CPU= 14.333
  17:01:38 STEP 2246 H=2246:00 +CPU= 15.035
  17:02:01 STEP 2247 H=2247:00 +CPU= 22.794
  17:02:16 STEP 2248 H=2248:00 +CPU= 15.186
  17:02:31 STEP 2249 H=2249:00 +CPU= 14.627
  17:02:54 STEP 2250 H=2250:00 +CPU= 23.203
  17:03:09 STEP 2251 H=2251:00 +CPU= 15.050
  17:03:24 STEP 2252 H=2252:00 +CPU= 14.693
  17:03:48 STEP 2253 H=2253:00 +CPU= 23.324
  17:04:02 STEP 2254 H=2254:00 +CPU= 14.810
  17:04:17 STEP 2255 H=2255:00 +CPU= 14.561
  17:04:46 STEP 2256 H=2256:00 +CPU= 28.651
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_93.zip
Uploading the intermediate file: upload_file_93.zip
  17:05:01 STEP 2257 H=2257:00 +CPU= 14.801
Uploading trickle at timestep: 8121600
  17:05:16 STEP 2258 H=2258:00 +CPU= 15.084
  17:05:39 STEP 2259 H=2259:00 +CPU= 23.167
  17:05:53 STEP 2260 H=2260:00 +CPU= 14.103
  17:06:08 STEP 2261 H=2261:00 +CPU= 14.654
  17:06:33 STEP 2262 H=2262:00 +CPU= 24.035
  17:06:48 STEP 2263 H=2263:00 +CPU= 14.853
  17:07:03 STEP 2264 H=2264:00 +CPU= 15.001
  17:07:26 STEP 2265 H=2265:00 +CPU= 22.956
  17:07:41 STEP 2266 H=2266:00 +CPU= 14.944
  17:07:56 STEP 2267 H=2267:00 +CPU= 14.620
  17:08:23 STEP 2268 H=2268:00 +CPU= 26.159
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002268
  17:08:37 STEP 2269 H=2269:00 +CPU= 14.314
  17:08:52 STEP 2270 H=2270:00 +CPU= 15.324
  17:09:14 STEP 2271 H=2271:00 +CPU= 21.584
  17:09:30 STEP 2272 H=2272:00 +CPU= 15.212
  17:09:44 STEP 2273 H=2273:00 +CPU= 14.242
  17:10:08 STEP 2274 H=2274:00 +CPU= 23.316
  17:10:22 STEP 2275 H=2275:00 +CPU= 14.462
  17:10:37 STEP 2276 H=2276:00 +CPU= 14.741
  17:11:01 STEP 2277 H=2277:00 +CPU= 23.504
  17:11:17 STEP 2278 H=2278:00 +CPU= 15.235
  17:11:31 STEP 2279 H=2279:00 +CPU= 14.371
  17:12:01 STEP 2280 H=2280:00 +CPU= 28.256
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_94.zip
Uploading the intermediate file: upload_file_94.zip
  17:12:17 STEP 2281 H=2281:00 +CPU= 14.882
  17:12:32 STEP 2282 H=2282:00 +CPU= 15.296
  17:12:56 STEP 2283 H=2283:00 +CPU= 23.366
  17:13:11 STEP 2284 H=2284:00 +CPU= 14.828
  17:13:26 STEP 2285 H=2285:00 +CPU= 14.828
  17:13:50 STEP 2286 H=2286:00 +CPU= 23.677
  17:14:05 STEP 2287 H=2287:00 +CPU= 14.746
  17:14:21 STEP 2288 H=2288:00 +CPU= 15.152
  17:14:45 STEP 2289 H=2289:00 +CPU= 23.200
  17:15:00 STEP 2290 H=2290:00 +CPU= 15.185
  17:15:15 STEP 2291 H=2291:00 +CPU= 14.562
  17:15:41 STEP 2292 H=2292:00 +CPU= 25.592
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002292
  17:15:56 STEP 2293 H=2293:00 +CPU= 14.716
  17:16:11 STEP 2294 H=2294:00 +CPU= 15.194
  17:16:35 STEP 2295 H=2295:00 +CPU= 22.971
  17:16:50 STEP 2296 H=2296:00 +CPU= 14.990
  17:17:05 STEP 2297 H=2297:00 +CPU= 14.625
  17:17:31 STEP 2298 H=2298:00 +CPU= 24.577
  17:17:46 STEP 2299 H=2299:00 +CPU= 14.916
  17:18:02 STEP 2300 H=2300:00 +CPU= 14.954
  17:18:24 STEP 2301 H=2301:00 +CPU= 22.414
  17:18:40 STEP 2302 H=2302:00 +CPU= 15.262
  17:18:55 STEP 2303 H=2303:00 +CPU= 14.517
  17:19:21 STEP 2304 H=2304:00 +CPU= 26.348
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_95.zip
Uploading the intermediate file: upload_file_95.zip
  17:19:36 STEP 2305 H=2305:00 +CPU= 14.426
  17:19:51 STEP 2306 H=2306:00 +CPU= 15.341
  17:20:14 STEP 2307 H=2307:00 +CPU= 22.986
  17:20:30 STEP 2308 H=2308:00 +CPU= 15.286
  17:20:44 STEP 2309 H=2309:00 +CPU= 14.043
  17:21:07 STEP 2310 H=2310:00 +CPU= 21.897
  17:21:20 STEP 2311 H=2311:00 +CPU= 13.371
  17:21:34 STEP 2312 H=2312:00 +CPU= 14.029
  17:21:58 STEP 2313 H=2313:00 +CPU= 23.120
  17:22:14 STEP 2314 H=2314:00 +CPU= 15.379
  17:22:28 STEP 2315 H=2315:00 +CPU= 14.491
  17:22:54 STEP 2316 H=2316:00 +CPU= 25.626
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002316
  17:23:10 STEP 2317 H=2317:00 +CPU= 14.855
  17:23:24 STEP 2318 H=2318:00 +CPU= 14.263
  17:23:46 STEP 2319 H=2319:00 +CPU= 21.283
  17:24:00 STEP 2320 H=2320:00 +CPU= 13.822
  17:24:13 STEP 2321 H=2321:00 +CPU= 13.408
  17:24:37 STEP 2322 H=2322:00 +CPU= 23.144
  17:24:52 STEP 2323 H=2323:00 +CPU= 14.519
  17:25:08 STEP 2324 H=2324:00 +CPU= 15.751
  17:25:36 STEP 2325 H=2325:00 +CPU= 26.038
  17:25:59 STEP 2326 H=2326:00 +CPU= 21.867
  17:26:22 STEP 2327 H=2327:00 +CPU= 21.658
  17:27:02 STEP 2328 H=2328:00 +CPU= 37.155
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_96.zip
Uploading the intermediate file: upload_file_96.zip
  17:27:24 STEP 2329 H=2329:00 +CPU= 20.604
  17:27:47 STEP 2330 H=2330:00 +CPU= 21.572
  17:28:20 STEP 2331 H=2331:00 +CPU= 30.833
  17:28:43 STEP 2332 H=2332:00 +CPU= 21.687
  17:29:06 STEP 2333 H=2333:00 +CPU= 21.738
  17:29:40 STEP 2334 H=2334:00 +CPU= 32.022
  17:30:02 STEP 2335 H=2335:00 +CPU= 20.977
  17:30:25 STEP 2336 H=2336:00 +CPU= 22.176
  17:30:58 STEP 2337 H=2337:00 +CPU= 31.680
  17:31:22 STEP 2338 H=2338:00 +CPU= 21.770
  17:31:44 STEP 2339 H=2339:00 +CPU= 21.098
  17:32:21 STEP 2340 H=2340:00 +CPU= 34.697
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002340
  17:32:43 STEP 2341 H=2341:00 +CPU= 21.115
  17:33:06 STEP 2342 H=2342:00 +CPU= 21.320
  17:33:40 STEP 2343 H=2343:00 +CPU= 31.798
  17:34:03 STEP 2344 H=2344:00 +CPU= 21.167
  17:34:24 STEP 2345 H=2345:00 +CPU= 20.252
  17:34:59 STEP 2346 H=2346:00 +CPU= 33.108
  17:35:20 STEP 2347 H=2347:00 +CPU= 19.755
  17:35:43 STEP 2348 H=2348:00 +CPU= 21.284
  17:36:18 STEP 2349 H=2349:00 +CPU= 32.730
  17:36:39 STEP 2350 H=2350:00 +CPU= 19.770
  17:37:02 STEP 2351 H=2351:00 +CPU= 21.951
  17:37:42 STEP 2352 H=2352:00 +CPU= 37.061
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_97.zip
Uploading the intermediate file: upload_file_97.zip
  17:38:05 STEP 2353 H=2353:00 +CPU= 21.377
  17:38:28 STEP 2354 H=2354:00 +CPU= 21.003
  17:39:03 STEP 2355 H=2355:00 +CPU= 32.749
  17:39:26 STEP 2356 H=2356:00 +CPU= 21.753
  17:39:48 STEP 2357 H=2357:00 +CPU= 21.242
  17:40:23 STEP 2358 H=2358:00 +CPU= 32.209
  17:40:45 STEP 2359 H=2359:00 +CPU= 21.333
  17:41:08 STEP 2360 H=2360:00 +CPU= 21.486
  17:41:42 STEP 2361 H=2361:00 +CPU= 31.661
  17:42:05 STEP 2362 H=2362:00 +CPU= 21.617
  17:42:27 STEP 2363 H=2363:00 +CPU= 20.951
  17:43:07 STEP 2364 H=2364:00 +CPU= 37.532
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002364
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1988050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0391
(argv4) batchid: 957
(argv5) wuid: 12174035
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bcbb00ac5a594958db6a91b3b7dda105 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0391_1988050100_123_957_12174035.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0391_1988050100_123_957_12174035.zip
ic_ancil_file: ic_ancil_12174035
ifsdata_file: ifsdata_12174035
climate_data_file: clim_data_12174035
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_fa8a5b395c1db06138be565344decfbb to: /var/lib/boinc-client/slots/6/ic_ancil_12174035.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12174035.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_919525c1940ea89f3795c886f33a1243 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12174035.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12174035.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12174035.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12174035.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12174035.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12174035.xml
-- Model is restarting --
Adjusting last_iter, 2365, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12174035.xml
last_cpu_time: 56943
upload_file_number: 98
last_iter: 2351
last_upload: 8467200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619
The child process has been launched with process id: 90840
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Mint18:1:1:90840:90840] [20230115:174430:1673829870:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  17:45:11 STEP 2352 H=2352:00 +CPU= 34.973
  17:45:32 STEP 2353 H=2353:00 +CPU= 19.706
  17:45:52 STEP 2354 H=2354:00 +CPU= 19.613
  17:46:26 STEP 2355 H=2355:00 +CPU= 31.555
  17:46:47 STEP 2356 H=2356:00 +CPU= 21.015
  17:47:07 STEP 2357 H=2357:00 +CPU= 18.225
  17:47:41 STEP 2358 H=2358:00 +CPU= 30.902
  17:48:01 STEP 2359 H=2359:00 +CPU= 19.556
  17:48:21 STEP 2360 H=2360:00 +CPU= 18.741
  17:48:50 STEP 2361 H=2361:00 +CPU= 27.550
  17:49:07 STEP 2362 H=2362:00 +CPU= 16.192
  17:49:24 STEP 2363 H=2363:00 +CPU= 17.049
  17:49:53 STEP 2364 H=2364:00 +CPU= 28.002
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002364
  17:50:09 STEP 2365 H=2365:00 +CPU= 15.693
  17:50:26 STEP 2366 H=2366:00 +CPU= 15.732
  17:50:48 STEP 2367 H=2367:00 +CPU= 22.361
  17:51:05 STEP 2368 H=2368:00 +CPU= 15.539
  17:51:21 STEP 2369 H=2369:00 +CPU= 15.798
  17:51:53 STEP 2370 H=2370:00 +CPU= 30.303
  17:52:14 STEP 2371 H=2371:00 +CPU= 19.127
  17:52:33 STEP 2372 H=2372:00 +CPU= 18.155
  17:53:04 STEP 2373 H=2373:00 +CPU= 29.130
  17:53:23 STEP 2374 H=2374:00 +CPU= 18.950
  17:53:45 STEP 2375 H=2375:00 +CPU= 20.446
  17:54:25 STEP 2376 H=2376:00 +CPU= 36.353
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_98.zip
Uploading the intermediate file: upload_file_98.zip
  17:54:47 STEP 2377 H=2377:00 +CPU= 20.175
  17:55:09 STEP 2378 H=2378:00 +CPU= 21.030
  17:55:35 STEP 2379 H=2379:00 +CPU= 25.713
  17:55:52 STEP 2380 H=2380:00 +CPU= 16.167
  17:56:08 STEP 2381 H=2381:00 +CPU= 15.822
  17:56:33 STEP 2382 H=2382:00 +CPU= 25.171
  17:56:49 STEP 2383 H=2383:00 +CPU= 15.602
  17:57:06 STEP 2384 H=2384:00 +CPU= 16.214
  17:57:31 STEP 2385 H=2385:00 +CPU= 24.251
  17:57:48 STEP 2386 H=2386:00 +CPU= 16.166
  17:58:03 STEP 2387 H=2387:00 +CPU= 15.341
  17:58:32 STEP 2388 H=2388:00 +CPU= 27.995
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002388
  17:58:49 STEP 2389 H=2389:00 +CPU= 16.518
  17:59:09 STEP 2390 H=2390:00 +CPU= 18.839
  17:59:39 STEP 2391 H=2391:00 +CPU= 28.738
  17:59:58 STEP 2392 H=2392:00 +CPU= 18.015
  18:00:17 STEP 2393 H=2393:00 +CPU= 18.988
  18:00:48 STEP 2394 H=2394:00 +CPU= 29.222
  18:01:08 STEP 2395 H=2395:00 +CPU= 18.873
  18:01:31 STEP 2396 H=2396:00 +CPU= 21.807
  18:02:06 STEP 2397 H=2397:00 +CPU= 33.030
  18:02:30 STEP 2398 H=2398:00 +CPU= 22.103
  18:02:51 STEP 2399 H=2399:00 +CPU= 20.661
  18:03:34 STEP 2400 H=2400:00 +CPU= 39.527
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_99.zip
Uploading the intermediate file: upload_file_99.zip
  18:03:57 STEP 2401 H=2401:00 +CPU= 21.103
  18:04:19 STEP 2402 H=2402:00 +CPU= 20.867
  18:04:54 STEP 2403 H=2403:00 +CPU= 32.961
  18:05:17 STEP 2404 H=2404:00 +CPU= 21.306
  18:05:37 STEP 2405 H=2405:00 +CPU= 19.650
  18:06:13 STEP 2406 H=2406:00 +CPU= 33.861
  18:06:35 STEP 2407 H=2407:00 +CPU= 21.205
  18:06:57 STEP 2408 H=2408:00 +CPU= 21.041
  18:07:31 STEP 2409 H=2409:00 +CPU= 32.399
  18:07:55 STEP 2410 H=2410:00 +CPU= 22.113
  18:08:16 STEP 2411 H=2411:00 +CPU= 19.841
  18:08:55 STEP 2412 H=2412:00 +CPU= 36.231
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002412
  18:09:17 STEP 2413 H=2413:00 +CPU= 21.820
  18:09:40 STEP 2414 H=2414:00 +CPU= 21.121
  18:10:15 STEP 2415 H=2415:00 +CPU= 34.027
  18:10:38 STEP 2416 H=2416:00 +CPU= 21.456
  18:11:00 STEP 2417 H=2417:00 +CPU= 20.238
  18:11:35 STEP 2418 H=2418:00 +CPU= 33.548
  18:11:58 STEP 2419 H=2419:00 +CPU= 21.378
  18:12:21 STEP 2420 H=2420:00 +CPU= 22.083
  18:12:49 STEP 2421 H=2421:00 +CPU= 26.705
  18:13:06 STEP 2422 H=2422:00 +CPU= 16.569
  18:13:23 STEP 2423 H=2423:00 +CPU= 16.073
  18:13:53 STEP 2424 H=2424:00 +CPU= 29.474
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_100.zip
Uploading the intermediate file: upload_file_100.zip
  18:14:10 STEP 2425 H=2425:00 +CPU= 15.927
  18:14:28 STEP 2426 H=2426:00 +CPU= 16.474
  18:14:56 STEP 2427 H=2427:00 +CPU= 25.072
  18:15:13 STEP 2428 H=2428:00 +CPU= 16.297
  18:15:32 STEP 2429 H=2429:00 +CPU= 16.434
  18:16:00 STEP 2430 H=2430:00 +CPU= 26.074
  18:16:17 STEP 2431 H=2431:00 +CPU= 15.772
  18:16:35 STEP 2432 H=2432:00 +CPU= 16.676
  18:17:04 STEP 2433 H=2433:00 +CPU= 25.722
  18:17:23 STEP 2434 H=2434:00 +CPU= 17.394
  18:17:40 STEP 2435 H=2435:00 +CPU= 15.980
  18:18:07 STEP 2436 H=2436:00 +CPU= 26.091
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002436
  18:18:23 STEP 2437 H=2437:00 +CPU= 15.209
  18:18:39 STEP 2438 H=2438:00 +CPU= 15.696
  18:19:04 STEP 2439 H=2439:00 +CPU= 24.027
  18:19:21 STEP 2440 H=2440:00 +CPU= 15.921
  18:19:37 STEP 2441 H=2441:00 +CPU= 15.030
  18:20:01 STEP 2442 H=2442:00 +CPU= 23.973
  18:20:17 STEP 2443 H=2443:00 +CPU= 14.883
  18:20:33 STEP 2444 H=2444:00 +CPU= 15.564
  18:20:58 STEP 2445 H=2445:00 +CPU= 23.925
  18:21:15 STEP 2446 H=2446:00 +CPU= 15.997
  18:21:31 STEP 2447 H=2447:00 +CPU= 15.602
  18:22:00 STEP 2448 H=2448:00 +CPU= 28.258
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_101.zip
Uploading the intermediate file: upload_file_101.zip
  18:22:16 STEP 2449 H=2449:00 +CPU= 15.220
  18:22:33 STEP 2450 H=2450:00 +CPU= 16.139
  18:23:00 STEP 2451 H=2451:00 +CPU= 25.477
  18:23:19 STEP 2452 H=2452:00 +CPU= 18.256
  18:23:35 STEP 2453 H=2453:00 +CPU= 15.016
  18:24:00 STEP 2454 H=2454:00 +CPU= 24.697
  18:24:17 STEP 2455 H=2455:00 +CPU= 15.542
  18:24:33 STEP 2456 H=2456:00 +CPU= 15.748
  18:24:58 STEP 2457 H=2457:00 +CPU= 23.713
  18:25:14 STEP 2458 H=2458:00 +CPU= 16.342
  18:25:31 STEP 2459 H=2459:00 +CPU= 15.578
  18:25:59 STEP 2460 H=2460:00 +CPU= 26.987
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002460
  18:26:15 STEP 2461 H=2461:00 +CPU= 15.196
  18:26:31 STEP 2462 H=2462:00 +CPU= 15.362
  18:26:57 STEP 2463 H=2463:00 +CPU= 24.271
  18:27:14 STEP 2464 H=2464:00 +CPU= 16.019
  18:27:30 STEP 2465 H=2465:00 +CPU= 15.506
  18:27:56 STEP 2466 H=2466:00 +CPU= 24.631
  18:28:12 STEP 2467 H=2467:00 +CPU= 15.146
  18:28:29 STEP 2468 H=2468:00 +CPU= 16.201
  18:28:55 STEP 2469 H=2469:00 +CPU= 24.089
  18:29:12 STEP 2470 H=2470:00 +CPU= 15.761
  18:29:28 STEP 2471 H=2471:00 +CPU= 15.576
  18:29:59 STEP 2472 H=2472:00 +CPU= 28.790
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_102.zip
Uploading the intermediate file: upload_file_102.zip
  18:30:14 STEP 2473 H=2473:00 +CPU= 14.347
  18:30:29 STEP 2474 H=2474:00 +CPU= 14.938
  18:30:53 STEP 2475 H=2475:00 +CPU= 23.317
  18:31:09 STEP 2476 H=2476:00 +CPU= 15.372
  18:31:23 STEP 2477 H=2477:00 +CPU= 14.125
  18:31:46 STEP 2478 H=2478:00 +CPU= 22.274
  18:32:01 STEP 2479 H=2479:00 +CPU= 13.965
  18:32:16 STEP 2480 H=2480:00 +CPU= 14.572
  18:32:43 STEP 2481 H=2481:00 +CPU= 25.156
  18:33:00 STEP 2482 H=2482:00 +CPU= 16.411
  18:33:16 STEP 2483 H=2483:00 +CPU= 15.134
  18:33:44 STEP 2484 H=2484:00 +CPU= 26.401
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002484
  18:33:59 STEP 2485 H=2485:00 +CPU= 14.961
  18:34:15 STEP 2486 H=2486:00 +CPU= 15.654
  18:34:40 STEP 2487 H=2487:00 +CPU= 23.540
  18:34:56 STEP 2488 H=2488:00 +CPU= 15.501
  18:35:12 STEP 2489 H=2489:00 +CPU= 15.101
  18:35:37 STEP 2490 H=2490:00 +CPU= 24.165
  18:35:52 STEP 2491 H=2491:00 +CPU= 14.799
  18:36:08 STEP 2492 H=2492:00 +CPU= 15.304
  18:36:32 STEP 2493 H=2493:00 +CPU= 23.525
  18:36:49 STEP 2494 H=2494:00 +CPU= 15.588
  18:37:05 STEP 2495 H=2495:00 +CPU= 15.661
  18:37:34 STEP 2496 H=2496:00 +CPU= 28.272
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_103.zip
Uploading the intermediate file: upload_file_103.zip
  18:37:50 STEP 2497 H=2497:00 +CPU= 14.922
  18:38:06 STEP 2498 H=2498:00 +CPU= 15.453
  18:38:30 STEP 2499 H=2499:00 +CPU= 23.479
  18:38:45 STEP 2500 H=2500:00 +CPU= 15.064
  18:38:59 STEP 2501 H=2501:00 +CPU= 14.029
  18:39:23 STEP 2502 H=2502:00 +CPU= 23.480
  18:39:37 STEP 2503 H=2503:00 +CPU= 14.328
  18:39:52 STEP 2504 H=2504:00 +CPU= 14.451
  18:40:16 STEP 2505 H=2505:00 +CPU= 23.319
  18:40:32 STEP 2506 H=2506:00 +CPU= 15.774
  18:40:48 STEP 2507 H=2507:00 +CPU= 15.278
  18:41:16 STEP 2508 H=2508:00 +CPU= 26.578
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002508
  18:41:31 STEP 2509 H=2509:00 +CPU= 14.824
  18:41:47 STEP 2510 H=2510:00 +CPU= 15.446
  18:42:11 STEP 2511 H=2511:00 +CPU= 23.250
  18:42:27 STEP 2512 H=2512:00 +CPU= 15.547
  18:42:43 STEP 2513 H=2513:00 +CPU= 15.151
  18:43:08 STEP 2514 H=2514:00 +CPU= 23.910
  18:43:23 STEP 2515 H=2515:00 +CPU= 14.328
  18:43:38 STEP 2516 H=2516:00 +CPU= 14.874
  18:44:02 STEP 2517 H=2517:00 +CPU= 22.999
  18:44:18 STEP 2518 H=2518:00 +CPU= 15.213
  18:44:34 STEP 2519 H=2519:00 +CPU= 15.247
  18:45:03 STEP 2520 H=2520:00 +CPU= 27.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_104.zip
Uploading the intermediate file: upload_file_104.zip
  18:55:24 STEP 2521 H=2521:00 +CPU= 14.622
  18:55:39 STEP 2522 H=2522:00 +CPU= 15.144
  18:56:06 STEP 2523 H=2523:00 +CPU= 25.068
  18:56:22 STEP 2524 H=2524:00 +CPU= 15.962
  18:56:39 STEP 2525 H=2525:00 +CPU= 15.877
  18:57:06 STEP 2526 H=2526:00 +CPU= 25.295
  18:57:22 STEP 2527 H=2527:00 +CPU= 16.023
  18:57:40 STEP 2528 H=2528:00 +CPU= 16.181
  18:58:07 STEP 2529 H=2529:00 +CPU= 24.964
  18:58:24 STEP 2530 H=2530:00 +CPU= 16.543
  18:58:40 STEP 2531 H=2531:00 +CPU= 15.386
  18:59:08 STEP 2532 H=2532:00 +CPU= 26.651
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002532
  18:59:25 STEP 2533 H=2533:00 +CPU= 15.750
  18:59:42 STEP 2534 H=2534:00 +CPU= 16.756
  19:00:08 STEP 2535 H=2535:00 +CPU= 25.236
  19:00:27 STEP 2536 H=2536:00 +CPU= 16.719
  19:00:45 STEP 2537 H=2537:00 +CPU= 17.043
  19:01:14 STEP 2538 H=2538:00 +CPU= 26.399
  19:01:32 STEP 2539 H=2539:00 +CPU= 16.639
  19:01:49 STEP 2540 H=2540:00 +CPU= 16.284
  19:02:15 STEP 2541 H=2541:00 +CPU= 24.932
  19:02:33 STEP 2542 H=2542:00 +CPU= 16.685
  19:02:50 STEP 2543 H=2543:00 +CPU= 16.062
  19:03:23 STEP 2544 H=2544:00 +CPU= 30.196
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_105.zip
Uploading the intermediate file: upload_file_105.zip
  19:03:40 STEP 2545 H=2545:00 +CPU= 16.104
  19:03:57 STEP 2546 H=2546:00 +CPU= 16.233
  19:04:22 STEP 2547 H=2547:00 +CPU= 24.634
  19:04:38 STEP 2548 H=2548:00 +CPU= 15.775
  19:04:55 STEP 2549 H=2549:00 +CPU= 16.036
  19:05:21 STEP 2550 H=2550:00 +CPU= 25.080
  19:05:35 STEP 2551 H=2551:00 +CPU= 14.732
  19:05:50 STEP 2552 H=2552:00 +CPU= 14.456
  19:06:13 STEP 2553 H=2553:00 +CPU= 22.357
  19:06:28 STEP 2554 H=2554:00 +CPU= 14.975
  19:06:43 STEP 2555 H=2555:00 +CPU= 14.609
  19:07:08 STEP 2556 H=2556:00 +CPU= 24.929
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002556
  19:07:21 STEP 2557 H=2557:00 +CPU= 13.532
  19:07:37 STEP 2558 H=2558:00 +CPU= 15.019
  19:08:00 STEP 2559 H=2559:00 +CPU= 22.774
  19:08:14 STEP 2560 H=2560:00 +CPU= 14.316
  19:08:31 STEP 2561 H=2561:00 +CPU= 15.662
  19:08:55 STEP 2562 H=2562:00 +CPU= 23.073
  19:09:09 STEP 2563 H=2563:00 +CPU= 14.006
  19:09:23 STEP 2564 H=2564:00 +CPU= 14.072
  19:09:47 STEP 2565 H=2565:00 +CPU= 23.381
  19:10:03 STEP 2566 H=2566:00 +CPU= 15.069
  19:10:17 STEP 2567 H=2567:00 +CPU= 14.541
  19:10:45 STEP 2568 H=2568:00 +CPU= 27.783
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_106.zip
Uploading the intermediate file: upload_file_106.zip
  19:10:59 STEP 2569 H=2569:00 +CPU= 13.458
  19:11:14 STEP 2570 H=2570:00 +CPU= 14.414
  19:11:37 STEP 2571 H=2571:00 +CPU= 22.931
  19:11:51 STEP 2572 H=2572:00 +CPU= 14.587
  19:12:07 STEP 2573 H=2573:00 +CPU= 15.433
  19:12:30 STEP 2574 H=2574:00 +CPU= 23.176
  19:12:44 STEP 2575 H=2575:00 +CPU= 13.333
  19:12:58 STEP 2576 H=2576:00 +CPU= 14.423
  19:13:21 STEP 2577 H=2577:00 +CPU= 22.564
  19:13:36 STEP 2578 H=2578:00 +CPU= 14.826
  19:13:50 STEP 2579 H=2579:00 +CPU= 14.220
  19:14:15 STEP 2580 H=2580:00 +CPU= 24.712
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002580
  19:14:28 STEP 2581 H=2581:00 +CPU= 13.589
  19:14:43 STEP 2582 H=2582:00 +CPU= 14.564
  19:15:06 STEP 2583 H=2583:00 +CPU= 22.971
  19:15:23 STEP 2584 H=2584:00 +CPU= 16.783
  19:15:40 STEP 2585 H=2585:00 +CPU= 17.679
  19:16:15 STEP 2586 H=2586:00 +CPU= 34.054
  19:16:38 STEP 2587 H=2587:00 +CPU= 22.998
  19:16:57 STEP 2588 H=2588:00 +CPU= 18.788
  19:17:28 STEP 2589 H=2589:00 +CPU= 31.119
  19:17:51 STEP 2590 H=2590:00 +CPU= 23.102
  19:18:11 STEP 2591 H=2591:00 +CPU= 19.351
  19:18:47 STEP 2592 H=2592:00 +CPU= 35.469
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174035/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0391_1988050100_123_957_12174035_0_r2045634619_107.zip
Uploading the intermediate file: upload_file_107.zip
  19:19:05 STEP 2593 H=2593:00 +CPU= 18.167
Uploading trickle at timestep: 9331200
  19:19:24 STEP 2594 H=2594:00 +CPU= 19.205
  19:19:52 STEP 2595 H=2595:00 +CPU= 28.080
  19:20:12 STEP 2596 H=2596:00 +CPU= 19.845
  19:20:34 STEP 2597 H=2597:00 +CPU= 21.097
  19:21:04 STEP 2598 H=2598:00 +CPU= 29.700
  19:21:24 STEP 2599 H=2599:00 +CPU= 20.131
  19:21:46 STEP 2600 H=2600:00 +CPU= 21.221
  19:22:15 STEP 2601 H=2601:00 +CPU= 29.369
  19:22:36 STEP 2602 H=2602:00 +CPU= 20.854
double free or corruption (out)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:21:40 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 9,331,200 61,789 0.0066
09 May 2023 16:21:30 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 8,121,600 54,717 0.0067
09 May 2023 16:21:01 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 6,220,800 41,909 0.0067
09 May 2023 16:20:56 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 5,356,800 36,133 0.0067
09 May 2023 16:20:37 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 4,492,800 30,129 0.0067
09 May 2023 16:19:59 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 3,456,000 24,146 0.0070
09 May 2023 16:19:45 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 2,592,000 18,557 0.0072
09 May 2023 16:19:19 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 1,728,000 12,335 0.0071
09 May 2023 16:19:06 1534496 22258641 oifs_43r3_ps_0391_1988050100_123_957_12174035_0 864,000 6,401 0.0074


©2024 climateprediction.net