Name | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 |
Workunit | 12190197 |
Created | 23 Dec 2022, 8:15:53 UTC |
Sent | 25 Dec 2022, 9:49:14 UTC |
Report deadline | 24 Jan 2023, 9:49:14 UTC |
Received | 15 Feb 2023, 20:28:20 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1495246 |
Run time | 14 hours 53 min 25 sec |
CPU time | 14 hours 34 min 38 sec |
Validate state | Invalid |
Credit | 12,216.96 |
Device peak FLOPS | 4.30 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,799.48 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,674.62 MB |
Stderr | <core_client_version>7.16.16</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> tment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230128:125731:1674907051:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:58:17 STEP 2184 H=2184:00 +CPU= 27.521 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:59:29 STEP 2185 H=2185:00 +CPU= 18.142 12:59:44 STEP 2186 H=2186:00 +CPU= 14.985 13:00:10 STEP 2187 H=2187:00 +CPU= 24.290 13:00:27 STEP 2188 H=2188:00 +CPU= 16.579 13:00:45 STEP 2189 H=2189:00 +CPU= 17.010 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:01:22 STEP 2190 H=2190:00 +CPU= 24.887 13:01:37 STEP 2191 H=2191:00 +CPU= 15.030 13:01:53 STEP 2192 H=2192:00 +CPU= 16.017 13:02:19 STEP 2193 H=2193:00 +CPU= 24.752 13:02:36 STEP 2194 H=2194:00 +CPU= 16.183 13:02:52 STEP 2195 H=2195:00 +CPU= 15.718 13:03:19 STEP 2196 H=2196:00 +CPU= 26.785 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002196 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002196 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002196 13:03:35 STEP 2197 H=2197:00 +CPU= 15.571 13:03:51 STEP 2198 H=2198:00 +CPU= 15.997 13:04:15 STEP 2199 H=2199:00 +CPU= 23.889 13:04:31 STEP 2200 H=2200:00 +CPU= 15.696 13:04:47 STEP 2201 H=2201:00 +CPU= 15.431 13:05:12 STEP 2202 H=2202:00 +CPU= 24.457 13:05:27 STEP 2203 H=2203:00 +CPU= 15.376 13:05:43 STEP 2204 H=2204:00 +CPU= 15.940 13:06:08 STEP 2205 H=2205:00 +CPU= 23.862 13:06:24 STEP 2206 H=2206:00 +CPU= 15.898 13:06:40 STEP 2207 H=2207:00 +CPU= 15.947 13:07:09 STEP 2208 H=2208:00 +CPU= 28.102 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002208 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002208 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002208 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002184 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002184 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002184 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002196 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002196 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002196 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_91.zip Uploading the intermediate file: upload_file_91.zip 13:07:25 STEP 2209 H=2209:00 +CPU= 15.879 13:07:41 STEP 2210 H=2210:00 +CPU= 15.835 13:08:07 STEP 2211 H=2211:00 +CPU= 24.765 13:08:23 STEP 2212 H=2212:00 +CPU= 16.035 13:08:39 STEP 2213 H=2213:00 +CPU= 15.593 13:09:05 STEP 2214 H=2214:00 +CPU= 24.892 13:09:20 STEP 2215 H=2215:00 +CPU= 15.534 13:09:37 STEP 2216 H=2216:00 +CPU= 16.301 13:10:03 STEP 2217 H=2217:00 +CPU= 24.504 13:10:19 STEP 2218 H=2218:00 +CPU= 15.992 13:10:36 STEP 2219 H=2219:00 +CPU= 16.444 13:11:07 STEP 2220 H=2220:00 +CPU= 28.719 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002220 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002220 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002220 13:11:23 STEP 2221 H=2221:00 +CPU= 15.813 13:11:41 STEP 2222 H=2222:00 +CPU= 16.970 13:12:07 STEP 2223 H=2223:00 +CPU= 24.896 13:12:23 STEP 2224 H=2224:00 +CPU= 16.252 13:12:40 STEP 2225 H=2225:00 +CPU= 16.238 13:13:08 STEP 2226 H=2226:00 +CPU= 26.854 13:13:24 STEP 2227 H=2227:00 +CPU= 15.875 13:13:41 STEP 2228 H=2228:00 +CPU= 16.178 13:14:05 STEP 2229 H=2229:00 +CPU= 24.483 13:14:21 STEP 2230 H=2230:00 +CPU= 15.811 13:14:37 STEP 2231 H=2231:00 +CPU= 16.118 13:15:05 STEP 2232 H=2232:00 +CPU= 27.805 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002232 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002232 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002232 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002208 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002208 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002208 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002220 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002220 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002220 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_92.zip Uploading the intermediate file: upload_file_92.zip 13:15:22 STEP 2233 H=2233:00 +CPU= 15.899 13:15:38 STEP 2234 H=2234:00 +CPU= 16.074 13:16:02 STEP 2235 H=2235:00 +CPU= 23.999 13:16:18 STEP 2236 H=2236:00 +CPU= 15.826 13:16:33 STEP 2237 H=2237:00 +CPU= 15.423 13:16:58 STEP 2238 H=2238:00 +CPU= 24.444 13:17:14 STEP 2239 H=2239:00 +CPU= 15.498 13:17:29 STEP 2240 H=2240:00 +CPU= 15.662 13:17:53 STEP 2241 H=2241:00 +CPU= 23.820 13:18:10 STEP 2242 H=2242:00 +CPU= 16.265 13:18:25 STEP 2243 H=2243:00 +CPU= 15.475 13:18:52 STEP 2244 H=2244:00 +CPU= 26.554 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002244 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002244 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002244 13:19:08 STEP 2245 H=2245:00 +CPU= 15.457 13:19:24 STEP 2246 H=2246:00 +CPU= 16.044 13:19:48 STEP 2247 H=2247:00 +CPU= 23.631 13:20:03 STEP 2248 H=2248:00 +CPU= 15.725 13:20:19 STEP 2249 H=2249:00 +CPU= 15.412 13:20:44 STEP 2250 H=2250:00 +CPU= 24.737 13:20:59 STEP 2251 H=2251:00 +CPU= 15.432 13:21:15 STEP 2252 H=2252:00 +CPU= 15.955 13:21:39 STEP 2253 H=2253:00 +CPU= 23.883 13:21:55 STEP 2254 H=2254:00 +CPU= 15.634 13:22:11 STEP 2255 H=2255:00 +CPU= 15.806 13:22:39 STEP 2256 H=2256:00 +CPU= 27.674 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002256 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002256 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002256 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002232 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002232 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002232 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002244 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002244 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002244 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_93.zip Uploading the intermediate file: upload_file_93.zip 13:22:55 STEP 2257 H=2257:00 +CPU= 15.857 13:23:11 STEP 2258 H=2258:00 +CPU= 15.987 13:23:35 STEP 2259 H=2259:00 +CPU= 23.761 13:23:51 STEP 2260 H=2260:00 +CPU= 15.739 13:24:06 STEP 2261 H=2261:00 +CPU= 15.416 13:24:31 STEP 2262 H=2262:00 +CPU= 24.375 13:24:46 STEP 2263 H=2263:00 +CPU= 15.222 13:25:02 STEP 2264 H=2264:00 +CPU= 15.899 13:25:26 STEP 2265 H=2265:00 +CPU= 23.936 13:25:42 STEP 2266 H=2266:00 +CPU= 15.935 13:25:58 STEP 2267 H=2267:00 +CPU= 15.616 13:26:25 STEP 2268 H=2268:00 +CPU= 26.733 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002268 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002268 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002268 13:26:40 STEP 2269 H=2269:00 +CPU= 15.417 13:26:56 STEP 2270 H=2270:00 +CPU= 15.960 13:27:21 STEP 2271 H=2271:00 +CPU= 24.305 13:27:37 STEP 2272 H=2272:00 +CPU= 15.829 13:27:52 STEP 2273 H=2273:00 +CPU= 15.323 13:28:17 STEP 2274 H=2274:00 +CPU= 24.693 13:28:32 STEP 2275 H=2275:00 +CPU= 15.476 13:28:49 STEP 2276 H=2276:00 +CPU= 15.896 13:29:13 STEP 2277 H=2277:00 +CPU= 23.918 13:29:29 STEP 2278 H=2278:00 +CPU= 15.970 13:29:45 STEP 2279 H=2279:00 +CPU= 15.912 13:30:13 STEP 2280 H=2280:00 +CPU= 28.419 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002280 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002280 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002280 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002256 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002256 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002256 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002268 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002268 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002268 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_94.zip Uploading the intermediate file: upload_file_94.zip 13:30:29 STEP 2281 H=2281:00 +CPU= 15.670 13:30:46 STEP 2282 H=2282:00 +CPU= 16.210 13:31:10 STEP 2283 H=2283:00 +CPU= 24.012 13:31:26 STEP 2284 H=2284:00 +CPU= 16.004 13:31:42 STEP 2285 H=2285:00 +CPU= 15.306 13:32:06 STEP 2286 H=2286:00 +CPU= 24.751 13:32:22 STEP 2287 H=2287:00 +CPU= 15.311 13:32:38 STEP 2288 H=2288:00 +CPU= 15.984 13:33:02 STEP 2289 H=2289:00 +CPU= 24.147 13:33:18 STEP 2290 H=2290:00 +CPU= 15.808 13:33:34 STEP 2291 H=2291:00 +CPU= 15.323 13:34:00 STEP 2292 H=2292:00 +CPU= 26.161 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002292 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002292 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002292 13:34:16 STEP 2293 H=2293:00 +CPU= 15.400 13:34:32 STEP 2294 H=2294:00 +CPU= 15.891 13:34:56 STEP 2295 H=2295:00 +CPU= 23.918 13:35:12 STEP 2296 H=2296:00 +CPU= 16.060 13:35:28 STEP 2297 H=2297:00 +CPU= 15.806 13:35:52 STEP 2298 H=2298:00 +CPU= 24.331 13:36:08 STEP 2299 H=2299:00 +CPU= 15.619 13:36:24 STEP 2300 H=2300:00 +CPU= 16.045 13:36:48 STEP 2301 H=2301:00 +CPU= 23.910 13:37:04 STEP 2302 H=2302:00 +CPU= 15.827 13:37:20 STEP 2303 H=2303:00 +CPU= 15.751 13:37:48 STEP 2304 H=2304:00 +CPU= 27.857 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002304 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002304 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002304 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002280 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002280 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002280 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002292 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002292 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002292 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_95.zip Uploading the intermediate file: upload_file_95.zip 13:38:04 STEP 2305 H=2305:00 +CPU= 15.792 13:38:21 STEP 2306 H=2306:00 +CPU= 16.506 13:38:45 STEP 2307 H=2307:00 +CPU= 23.866 13:39:01 STEP 2308 H=2308:00 +CPU= 15.809 13:39:16 STEP 2309 H=2309:00 +CPU= 15.474 13:39:41 STEP 2310 H=2310:00 +CPU= 24.297 13:39:56 STEP 2311 H=2311:00 +CPU= 15.543 13:40:12 STEP 2312 H=2312:00 +CPU= 15.777 13:40:36 STEP 2313 H=2313:00 +CPU= 23.908 13:40:52 STEP 2314 H=2314:00 +CPU= 16.100 13:41:08 STEP 2315 H=2315:00 +CPU= 15.632 13:41:35 STEP 2316 H=2316:00 +CPU= 26.807 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002316 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002316 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002316 13:41:50 STEP 2317 H=2317:00 +CPU= 15.209 13:42:06 STEP 2318 H=2318:00 +CPU= 15.861 13:42:31 STEP 2319 H=2319:00 +CPU= 23.906 13:42:47 STEP 2320 H=2320:00 +CPU= 15.988 13:43:02 STEP 2321 H=2321:00 +CPU= 15.457 13:43:27 STEP 2322 H=2322:00 +CPU= 24.261 13:43:42 STEP 2323 H=2323:00 +CPU= 15.494 13:44:00 STEP 2324 H=2324:00 +CPU= 17.881 13:44:25 STEP 2325 H=2325:00 +CPU= 24.412 13:44:42 STEP 2326 H=2326:00 +CPU= 15.898 13:44:58 STEP 2327 H=2327:00 +CPU= 16.182 13:45:26 STEP 2328 H=2328:00 +CPU= 27.958 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002328 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002328 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002328 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002304 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002304 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002304 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002316 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002316 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002316 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_96.zip Uploading the intermediate file: upload_file_96.zip 13:45:42 STEP 2329 H=2329:00 +CPU= 15.583 13:45:58 STEP 2330 H=2330:00 +CPU= 16.102 13:46:22 STEP 2331 H=2331:00 +CPU= 23.974 13:46:38 STEP 2332 H=2332:00 +CPU= 16.108 13:46:54 STEP 2333 H=2333:00 +CPU= 15.315 13:47:18 STEP 2334 H=2334:00 +CPU= 24.339 13:47:34 STEP 2335 H=2335:00 +CPU= 15.504 13:47:50 STEP 2336 H=2336:00 +CPU= 16.080 13:48:14 STEP 2337 H=2337:00 +CPU= 24.102 13:48:30 STEP 2338 H=2338:00 +CPU= 15.866 13:48:46 STEP 2339 H=2339:00 +CPU= 15.254 13:49:12 STEP 2340 H=2340:00 +CPU= 26.506 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002340 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002340 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002340 13:49:28 STEP 2341 H=2341:00 +CPU= 15.277 13:49:44 STEP 2342 H=2342:00 +CPU= 15.939 13:50:08 STEP 2343 H=2343:00 +CPU= 23.642 13:50:24 STEP 2344 H=2344:00 +CPU= 15.569 13:50:39 STEP 2345 H=2345:00 +CPU= 15.454 13:51:04 STEP 2346 H=2346:00 +CPU= 24.638 13:51:19 STEP 2347 H=2347:00 +CPU= 15.321 13:51:35 STEP 2348 H=2348:00 +CPU= 15.805 13:51:59 STEP 2349 H=2349:00 +CPU= 23.830 13:52:15 STEP 2350 H=2350:00 +CPU= 15.762 13:52:31 STEP 2351 H=2351:00 +CPU= 15.838 13:52:59 STEP 2352 H=2352:00 +CPU= 28.265 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002352 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002352 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002352 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002328 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002328 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002328 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002340 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002340 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002340 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_97.zip Uploading the intermediate file: upload_file_97.zip 13:53:15 STEP 2353 H=2353:00 +CPU= 15.884 13:53:31 STEP 2354 H=2354:00 +CPU= 15.964 13:53:55 STEP 2355 H=2355:00 +CPU= 23.653 13:54:11 STEP 2356 H=2356:00 +CPU= 15.780 13:54:27 STEP 2357 H=2357:00 +CPU= 15.375 13:54:51 STEP 2358 H=2358:00 +CPU= 24.455 13:55:07 STEP 2359 H=2359:00 +CPU= 15.422 13:55:23 STEP 2360 H=2360:00 +CPU= 16.172 13:55:48 STEP 2361 H=2361:00 +CPU= 24.011 13:56:03 STEP 2362 H=2362:00 +CPU= 15.748 13:56:19 STEP 2363 H=2363:00 +CPU= 15.422 13:56:45 STEP 2364 H=2364:00 +CPU= 26.183 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002364 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002364 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002364 13:57:01 STEP 2365 H=2365:00 +CPU= 15.347 13:57:17 STEP 2366 H=2366:00 +CPU= 16.155 13:57:41 STEP 2367 H=2367:00 +CPU= 23.892 13:57:58 STEP 2368 H=2368:00 +CPU= 16.658 13:58:13 STEP 2369 H=2369:00 +CPU= 15.503 13:58:38 STEP 2370 H=2370:00 +CPU= 24.489 13:58:53 STEP 2371 H=2371:00 +CPU= 15.368 13:59:09 STEP 2372 H=2372:00 +CPU= 16.102 13:59:34 STEP 2373 H=2373:00 +CPU= 23.952 13:59:49 STEP 2374 H=2374:00 +CPU= 15.821 14:00:06 STEP 2375 H=2375:00 +CPU= 16.110 14:00:34 STEP 2376 H=2376:00 +CPU= 27.800 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002376 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002376 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002376 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002352 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002352 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002352 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002364 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002364 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002364 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_98.zip Uploading the intermediate file: upload_file_98.zip 14:00:49 STEP 2377 H=2377:00 +CPU= 15.495 14:01:06 STEP 2378 H=2378:00 +CPU= 15.898 14:01:30 STEP 2379 H=2379:00 +CPU= 23.691 14:01:46 STEP 2380 H=2380:00 +CPU= 15.958 14:02:01 STEP 2381 H=2381:00 +CPU= 15.477 14:02:26 STEP 2382 H=2382:00 +CPU= 24.305 14:02:41 STEP 2383 H=2383:00 +CPU= 15.119 14:02:57 STEP 2384 H=2384:00 +CPU= 16.191 14:03:21 STEP 2385 H=2385:00 +CPU= 24.021 14:03:37 STEP 2386 H=2386:00 +CPU= 15.888 14:03:53 STEP 2387 H=2387:00 +CPU= 15.523 14:04:20 STEP 2388 H=2388:00 +CPU= 26.620 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002388 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002388 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002388 14:04:35 STEP 2389 H=2389:00 +CPU= 15.374 14:04:51 STEP 2390 H=2390:00 +CPU= 15.693 14:05:15 STEP 2391 H=2391:00 +CPU= 23.739 14:05:31 STEP 2392 H=2392:00 +CPU= 15.961 14:05:46 STEP 2393 H=2393:00 +CPU= 15.315 14:06:11 STEP 2394 H=2394:00 +CPU= 24.737 14:06:27 STEP 2395 H=2395:00 +CPU= 15.329 14:06:43 STEP 2396 H=2396:00 +CPU= 15.996 14:07:07 STEP 2397 H=2397:00 +CPU= 23.647 14:07:23 STEP 2398 H=2398:00 +CPU= 15.800 14:07:39 STEP 2399 H=2399:00 +CPU= 16.138 14:08:08 STEP 2400 H=2400:00 +CPU= 28.760 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002400 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002400 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002400 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002376 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002376 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002376 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002388 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002388 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002388 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_99.zip Uploading the intermediate file: upload_file_99.zip 19:15:39 STEP 2401 H=2401:00 +CPU= 16.955 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:16:21 STEP 2402 H=2402:00 +CPU= 18.886 19:16:47 STEP 2403 H=2403:00 +CPU= 24.935 19:17:04 STEP 2404 H=2404:00 +CPU= 16.347 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:17:43 STEP 2405 H=2405:00 +CPU= 16.836 19:18:11 STEP 2406 H=2406:00 +CPU= 26.086 19:18:29 STEP 2407 H=2407:00 +CPU= 16.883 19:18:44 STEP 2408 H=2408:00 +CPU= 15.631 19:19:11 STEP 2409 H=2409:00 +CPU= 25.668 19:19:29 STEP 2410 H=2410:00 +CPU= 17.120 19:19:46 STEP 2411 H=2411:00 +CPU= 16.112 19:20:12 STEP 2412 H=2412:00 +CPU= 26.089 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002412 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002412 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002412 19:20:28 STEP 2413 H=2413:00 +CPU= 15.988 19:20:45 STEP 2414 H=2414:00 +CPU= 16.069 19:21:08 STEP 2415 H=2415:00 +CPU= 23.534 19:21:24 STEP 2416 H=2416:00 +CPU= 15.384 19:21:39 STEP 2417 H=2417:00 +CPU= 14.950 19:22:03 STEP 2418 H=2418:00 +CPU= 24.076 19:22:18 STEP 2419 H=2419:00 +CPU= 15.121 19:22:34 STEP 2420 H=2420:00 +CPU= 15.637 19:22:58 STEP 2421 H=2421:00 +CPU= 23.300 19:23:14 STEP 2422 H=2422:00 +CPU= 15.822 19:23:30 STEP 2423 H=2423:00 +CPU= 15.857 19:23:57 STEP 2424 H=2424:00 +CPU= 27.646 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002424 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002424 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002424 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002400 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002400 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002400 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002412 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002412 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002412 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_100.zip Uploading the intermediate file: upload_file_100.zip 19:24:13 STEP 2425 H=2425:00 +CPU= 15.461 Uploading trickle at timestep: 8726400 19:24:29 STEP 2426 H=2426:00 +CPU= 15.681 19:24:53 STEP 2427 H=2427:00 +CPU= 23.602 19:25:08 STEP 2428 H=2428:00 +CPU= 15.479 19:25:23 STEP 2429 H=2429:00 +CPU= 14.936 19:25:47 STEP 2430 H=2430:00 +CPU= 24.018 19:26:03 STEP 2431 H=2431:00 +CPU= 15.260 19:26:18 STEP 2432 H=2432:00 +CPU= 15.598 18:51:01 STEP 2433 H=2433:00 +CPU= 23.687 18:51:18 STEP 2434 H=2434:00 +CPU= 16.473 18:51:35 STEP 2435 H=2435:00 +CPU= 16.665 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:52:37 STEP 2436 H=2436:00 +CPU= 28.469 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002436 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002436 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002436 18:52:53 STEP 2437 H=2437:00 +CPU= 15.769 18:53:10 STEP 2438 H=2438:00 +CPU= 16.225 18:53:34 STEP 2439 H=2439:00 +CPU= 24.125 18:53:51 STEP 2440 H=2440:00 +CPU= 16.174 18:54:07 STEP 2441 H=2441:00 +CPU= 15.308 18:54:33 STEP 2442 H=2442:00 +CPU= 25.479 18:54:50 STEP 2443 H=2443:00 +CPU= 15.934 18:55:07 STEP 2444 H=2444:00 +CPU= 16.667 18:55:34 STEP 2445 H=2445:00 +CPU= 25.672 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:56:03 STEP 2446 H=2446:00 +CPU= 16.546 18:56:18 STEP 2447 H=2447:00 +CPU= 15.237 18:56:47 STEP 2448 H=2448:00 +CPU= 28.550 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002448 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002448 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002448 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002424 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002424 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002424 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002436 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002436 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002436 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_101.zip Uploading the intermediate file: upload_file_101.zip 18:57:03 STEP 2449 H=2449:00 +CPU= 15.946 18:57:20 STEP 2450 H=2450:00 +CPU= 15.931 18:57:44 STEP 2451 H=2451:00 +CPU= 24.136 18:58:01 STEP 2452 H=2452:00 +CPU= 15.895 18:58:16 STEP 2453 H=2453:00 +CPU= 15.437 18:58:41 STEP 2454 H=2454:00 +CPU= 24.519 18:58:57 STEP 2455 H=2455:00 +CPU= 15.476 18:59:14 STEP 2456 H=2456:00 +CPU= 16.918 18:59:39 STEP 2457 H=2457:00 +CPU= 24.137 18:59:56 STEP 2458 H=2458:00 +CPU= 16.327 19:00:13 STEP 2459 H=2459:00 +CPU= 16.571 19:00:42 STEP 2460 H=2460:00 +CPU= 27.793 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002460 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002460 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002460 19:00:59 STEP 2461 H=2461:00 +CPU= 16.535 19:01:18 STEP 2462 H=2462:00 +CPU= 17.717 19:01:45 STEP 2463 H=2463:00 +CPU= 26.004 19:02:02 STEP 2464 H=2464:00 +CPU= 16.414 19:02:22 STEP 2465 H=2465:00 +CPU= 18.299 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:02:58 STEP 2466 H=2466:00 +CPU= 24.195 19:03:14 STEP 2467 H=2467:00 +CPU= 16.049 19:03:30 STEP 2468 H=2468:00 +CPU= 15.700 19:03:56 STEP 2469 H=2469:00 +CPU= 24.957 19:04:12 STEP 2470 H=2470:00 +CPU= 15.892 19:04:28 STEP 2471 H=2471:00 +CPU= 16.124 19:04:57 STEP 2472 H=2472:00 +CPU= 28.541 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002472 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002472 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002472 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002448 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002448 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002448 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002460 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002460 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002460 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_102.zip Uploading the intermediate file: upload_file_102.zip 19:05:14 STEP 2473 H=2473:00 +CPU= 15.732 19:05:30 STEP 2474 H=2474:00 +CPU= 16.405 19:05:54 STEP 2475 H=2475:00 +CPU= 23.805 19:06:10 STEP 2476 H=2476:00 +CPU= 15.706 19:06:26 STEP 2477 H=2477:00 +CPU= 15.466 19:06:51 STEP 2478 H=2478:00 +CPU= 24.649 19:07:06 STEP 2479 H=2479:00 +CPU= 15.418 19:07:23 STEP 2480 H=2480:00 +CPU= 16.102 19:07:47 STEP 2481 H=2481:00 +CPU= 24.047 19:08:03 STEP 2482 H=2482:00 +CPU= 16.010 19:08:18 STEP 2483 H=2483:00 +CPU= 15.256 19:08:45 STEP 2484 H=2484:00 +CPU= 26.445 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002484 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002484 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002484 19:09:01 STEP 2485 H=2485:00 +CPU= 15.500 19:09:17 STEP 2486 H=2486:00 +CPU= 15.983 19:09:41 STEP 2487 H=2487:00 +CPU= 23.777 19:09:57 STEP 2488 H=2488:00 +CPU= 15.859 19:10:12 STEP 2489 H=2489:00 +CPU= 15.294 19:10:37 STEP 2490 H=2490:00 +CPU= 24.402 19:10:53 STEP 2491 H=2491:00 +CPU= 15.550 19:11:09 STEP 2492 H=2492:00 +CPU= 16.083 19:11:33 STEP 2493 H=2493:00 +CPU= 23.606 19:11:50 STEP 2494 H=2494:00 +CPU= 16.764 19:12:06 STEP 2495 H=2495:00 +CPU= 16.073 19:12:35 STEP 2496 H=2496:00 +CPU= 28.234 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002496 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002496 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002496 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002472 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002472 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002472 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002484 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002484 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002484 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_103.zip Uploading the intermediate file: upload_file_103.zip 19:12:51 STEP 2497 H=2497:00 +CPU= 16.295 19:13:08 STEP 2498 H=2498:00 +CPU= 16.201 19:13:32 STEP 2499 H=2499:00 +CPU= 24.158 19:13:48 STEP 2500 H=2500:00 +CPU= 15.871 19:14:04 STEP 2501 H=2501:00 +CPU= 15.534 19:14:28 STEP 2502 H=2502:00 +CPU= 24.392 19:14:44 STEP 2503 H=2503:00 +CPU= 15.253 19:15:00 STEP 2504 H=2504:00 +CPU= 15.774 19:15:24 STEP 2505 H=2505:00 +CPU= 23.963 19:15:40 STEP 2506 H=2506:00 +CPU= 15.740 19:15:55 STEP 2507 H=2507:00 +CPU= 15.315 19:16:22 STEP 2508 H=2508:00 +CPU= 26.376 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002508 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002508 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002508 19:16:37 STEP 2509 H=2509:00 +CPU= 15.306 19:16:53 STEP 2510 H=2510:00 +CPU= 15.913 19:17:17 STEP 2511 H=2511:00 +CPU= 23.789 19:17:33 STEP 2512 H=2512:00 +CPU= 15.817 19:17:49 STEP 2513 H=2513:00 +CPU= 15.497 19:18:14 STEP 2514 H=2514:00 +CPU= 24.767 19:18:30 STEP 2515 H=2515:00 +CPU= 15.623 19:18:47 STEP 2516 H=2516:00 +CPU= 16.496 19:19:11 STEP 2517 H=2517:00 +CPU= 24.086 19:19:27 STEP 2518 H=2518:00 +CPU= 15.838 19:19:43 STEP 2519 H=2519:00 +CPU= 15.785 19:20:11 STEP 2520 H=2520:00 +CPU= 28.079 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002520 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002520 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002520 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002496 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002496 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002496 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002508 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002508 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002508 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_104.zip Uploading the intermediate file: upload_file_104.zip 19:20:27 STEP 2521 H=2521:00 +CPU= 15.979 19:20:43 STEP 2522 H=2522:00 +CPU= 16.031 19:21:08 STEP 2523 H=2523:00 +CPU= 24.188 19:21:24 STEP 2524 H=2524:00 +CPU= 15.934 19:21:40 STEP 2525 H=2525:00 +CPU= 15.590 19:22:05 STEP 2526 H=2526:00 +CPU= 24.707 19:22:20 STEP 2527 H=2527:00 +CPU= 15.403 19:22:36 STEP 2528 H=2528:00 +CPU= 15.933 19:23:00 STEP 2529 H=2529:00 +CPU= 24.071 19:23:16 STEP 2530 H=2530:00 +CPU= 16.003 19:23:32 STEP 2531 H=2531:00 +CPU= 15.489 19:23:59 STEP 2532 H=2532:00 +CPU= 26.728 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002532 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002532 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002532 19:24:14 STEP 2533 H=2533:00 +CPU= 15.436 19:24:30 STEP 2534 H=2534:00 +CPU= 15.946 19:24:55 STEP 2535 H=2535:00 +CPU= 24.011 19:25:11 STEP 2536 H=2536:00 +CPU= 16.031 19:25:27 STEP 2537 H=2537:00 +CPU= 15.337 19:25:51 STEP 2538 H=2538:00 +CPU= 24.370 19:26:07 STEP 2539 H=2539:00 +CPU= 15.462 19:26:23 STEP 2540 H=2540:00 +CPU= 15.826 19:26:47 STEP 2541 H=2541:00 +CPU= 23.835 19:27:03 STEP 2542 H=2542:00 +CPU= 15.876 19:27:19 STEP 2543 H=2543:00 +CPU= 16.000 19:27:48 STEP 2544 H=2544:00 +CPU= 28.575 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002544 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002544 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002544 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002520 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002520 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002520 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002532 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002532 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002532 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_105.zip Uploading the intermediate file: upload_file_105.zip 19:28:04 STEP 2545 H=2545:00 +CPU= 15.955 19:28:20 STEP 2546 H=2546:00 +CPU= 15.954 19:28:44 STEP 2547 H=2547:00 +CPU= 23.684 19:29:00 STEP 2548 H=2548:00 +CPU= 16.017 19:29:15 STEP 2549 H=2549:00 +CPU= 15.415 19:29:40 STEP 2550 H=2550:00 +CPU= 24.273 19:29:56 STEP 2551 H=2551:00 +CPU= 15.435 19:30:12 STEP 2552 H=2552:00 +CPU= 16.054 19:30:37 STEP 2553 H=2553:00 +CPU= 24.433 19:30:55 STEP 2554 H=2554:00 +CPU= 17.339 19:31:12 STEP 2555 H=2555:00 +CPU= 16.546 19:31:40 STEP 2556 H=2556:00 +CPU= 27.471 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002556 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002556 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002556 19:31:57 STEP 2557 H=2557:00 +CPU= 16.327 19:32:15 STEP 2558 H=2558:00 +CPU= 17.104 19:32:40 STEP 2559 H=2559:00 +CPU= 24.860 19:32:57 STEP 2560 H=2560:00 +CPU= 16.481 19:33:14 STEP 2561 H=2561:00 +CPU= 16.293 19:33:41 STEP 2562 H=2562:00 +CPU= 25.637 19:33:56 STEP 2563 H=2563:00 +CPU= 15.515 19:34:13 STEP 2564 H=2564:00 +CPU= 16.029 19:34:37 STEP 2565 H=2565:00 +CPU= 24.305 19:34:56 STEP 2566 H=2566:00 +CPU= 17.408 19:35:14 STEP 2567 H=2567:00 +CPU= 17.437 19:35:45 STEP 2568 H=2568:00 +CPU= 29.125 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002568 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002568 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002568 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002544 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002544 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002544 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002556 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002556 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002556 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_106.zip Uploading the intermediate file: upload_file_106.zip 19:36:01 STEP 2569 H=2569:00 +CPU= 15.944 19:36:18 STEP 2570 H=2570:00 +CPU= 16.792 19:36:43 STEP 2571 H=2571:00 +CPU= 24.395 19:36:59 STEP 2572 H=2572:00 +CPU= 16.109 19:37:18 STEP 2573 H=2573:00 +CPU= 18.225 19:37:44 STEP 2574 H=2574:00 +CPU= 26.262 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.024] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4795MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.026] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.026] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.026] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.026] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.026] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4795 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193749:1675190269:283218.027] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDSC [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193750:1675190270:283219.027] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1578:1578] [20230131:193750:1675190270:283219.027] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DB46A7 Unknown Unknown Unknown oifs_43r3_model.e 000000000103282E cloudsc_ 2946 cloudsc.F90 oifs_43r3_model.e 0000000001014D57 cloud_layer_ 131 cloud_layer.F90 oifs_43r3_model.e 0000000000F1A790 callpar_ 1223 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2004050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0553 (argv4) batchid: 973 (argv5) wuid: 12190197 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /home/boinc-client/slots/0 Project directory is: /home/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197 ..mkdir for temp folder for results failed Copying: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8faea2a9b983146710a477035ac73d78 to: /home/boinc-client/slots/0/oifs_43r3_ps_0553_2004050100_123_973_12190197.zip Unzipping the namelist zip file: /home/boinc-client/slots/0/oifs_43r3_ps_0553_2004050100_123_973_12190197.zip ic_ancil_file: ic_ancil_12190197 ifsdata_file: ifsdata_12190197 climate_data_file: clim_data_12190197 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_164efbb6da1fef1587acbeea4ac4204b to: /home/boinc-client/slots/0/ic_ancil_12190197.zip Unzipping the IC ancils zip file: /home/boinc-client/slots/0/ic_ancil_12190197.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_d7534339accfcb785e9a4b3add0e3dbb to: /home/boinc-client/slots/0/ifsdata/ifsdata_12190197.zip Unzipping the ifsdata_zip file: /home/boinc-client/slots/0/ifsdata/ifsdata_12190197.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/boinc-client/slots/0/159l_2/clim_data_12190197.zip Unzipping the climate data zip file: /home/boinc-client/slots/0/159l_2/clim_data_12190197.zip Checking for progress XML file: /home/boinc-client/slots/0/progress_file_12190197.xml Opened progress file ok : /home/boinc-client/slots/0/progress_file_12190197.xml -- Model is restarting -- Adjusting last_iter, 2575, to previous model restart step. Creating progress file: /home/boinc-client/slots/0/progress_file_12190197.xml last_cpu_time: 51235 upload_file_number: 107 last_iter: 2567 last_upload: 9244800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151 The child process has been launched with process id: 1969 Executing the command: /home/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Hp-ProBook-430-G4:1:1:1969:1969] [20230215:175913:1676480353:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 17:59:43 STEP 2568 H=2568:00 +CPU= 25.040 17:59:58 STEP 2569 H=2569:00 +CPU= 14.501 18:00:14 STEP 2570 H=2570:00 +CPU= 15.592 18:00:38 STEP 2571 H=2571:00 +CPU= 23.915 18:00:54 STEP 2572 H=2572:00 +CPU= 15.950 18:01:16 STEP 2573 H=2573:00 +CPU= 18.700 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:02:34 STEP 2574 H=2574:00 +CPU= 24.485 18:02:52 STEP 2575 H=2575:00 +CPU= 16.562 18:03:11 STEP 2576 H=2576:00 +CPU= 17.312 18:03:39 STEP 2577 H=2577:00 +CPU= 26.380 18:03:58 STEP 2578 H=2578:00 +CPU= 17.229 18:04:15 STEP 2579 H=2579:00 +CPU= 16.137 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:05:45 STEP 2580 H=2580:00 +CPU= 28.465 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002580 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002580 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002580 18:06:03 STEP 2581 H=2581:00 +CPU= 16.617 18:06:22 STEP 2582 H=2582:00 +CPU= 17.526 18:06:51 STEP 2583 H=2583:00 +CPU= 26.176 18:07:09 STEP 2584 H=2584:00 +CPU= 17.012 18:07:27 STEP 2585 H=2585:00 +CPU= 16.710 18:07:55 STEP 2586 H=2586:00 +CPU= 26.423 18:08:11 STEP 2587 H=2587:00 +CPU= 15.526 18:08:27 STEP 2588 H=2588:00 +CPU= 15.762 18:08:51 STEP 2589 H=2589:00 +CPU= 23.672 18:09:07 STEP 2590 H=2590:00 +CPU= 15.730 21:11:46 STEP 2591 H=2591:00 +CPU= 15.817 21:12:14 STEP 2592 H=2592:00 +CPU= 27.843 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002592 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002592 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002592 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002568 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002568 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002568 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002580 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002580 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002580 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_107.zip Uploading the intermediate file: upload_file_107.zip 21:12:30 STEP 2593 H=2593:00 +CPU= 15.670 21:12:46 STEP 2594 H=2594:00 +CPU= 15.824 21:13:11 STEP 2595 H=2595:00 +CPU= 24.038 21:13:27 STEP 2596 H=2596:00 +CPU= 15.740 21:13:42 STEP 2597 H=2597:00 +CPU= 15.282 21:14:07 STEP 2598 H=2598:00 +CPU= 24.366 21:14:23 STEP 2599 H=2599:00 +CPU= 15.328 21:14:39 STEP 2600 H=2600:00 +CPU= 15.816 21:15:03 STEP 2601 H=2601:00 +CPU= 23.840 21:15:19 STEP 2602 H=2602:00 +CPU= 16.156 21:15:36 STEP 2603 H=2603:00 +CPU= 15.822 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 21:17:23 STEP 2604 H=2604:00 +CPU= 30.129 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002604 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002604 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002604 21:17:39 STEP 2605 H=2605:00 +CPU= 15.271 21:17:59 STEP 2606 H=2606:00 +CPU= 17.841 21:18:25 STEP 2607 H=2607:00 +CPU= 25.015 21:18:42 STEP 2608 H=2608:00 +CPU= 16.759 21:19:00 STEP 2609 H=2609:00 +CPU= 16.248 21:19:27 STEP 2610 H=2610:00 +CPU= 25.744 21:19:45 STEP 2611 H=2611:00 +CPU= 16.629 21:20:03 STEP 2612 H=2612:00 +CPU= 17.157 21:20:32 STEP 2613 H=2613:00 +CPU= 26.317 21:20:49 STEP 2614 H=2614:00 +CPU= 16.458 21:21:07 STEP 2615 H=2615:00 +CPU= 16.710 21:21:39 STEP 2616 H=2616:00 +CPU= 29.953 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002616 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002616 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002616 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002592 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002592 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002592 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMGGhq0f+002604 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMSHhq0f+002604 Adding to the zip: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12190197/ICMUAhq0f+002604 Zipping up the intermediate file: /home/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0553_2004050100_123_973_12190197_0_r1375500151_108.zip Uploading the intermediate file: upload_file_108.zip 21:21:56 STEP 2617 H=2617:00 +CPU= 16.405 21:22:13 STEP 2618 H=2618:00 +CPU= 16.508 21:22:40 STEP 2619 H=2619:00 +CPU= 25.183 21:22:59 STEP 2620 H=2620:00 +CPU= 17.179 21:23:16 STEP 2621 H=2621:00 +CPU= 16.663 21:23:44 STEP 2622 H=2622:00 +CPU= 25.845 21:24:03 STEP 2623 H=2623:00 +CPU= 17.178 Suspend request received from the BOINC client, suspending the child process Resuming the child process 21:24:29 STEP 2624 H=2624:00 +CPU= 14.925 21:24:53 STEP 2625 H=2625:00 +CPU= 23.952 21:25:09 STEP 2626 H=2626:00 +CPU= 15.737 21:25:25 STEP 2627 H=2627:00 +CPU= 15.350 21:25:52 STEP 2628 H=2628:00 +CPU= 26.289 Moving to projects directory: /home/boinc-client/slots/0/ICMGGhq0f+002628 Moving to projects directory: /home/boinc-client/slots/0/ICMSHhq0f+002628 Moving to projects directory: /home/boinc-client/slots/0/ICMUAhq0f+002628 21:26:07 STEP 2629 H=2629:00 +CPU= 15.392 21:26:23 STEP 2630 H=2630:00 +CPU= 15.753 Abort request received from BOINC client, ending the child process ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:45:04 | 1495246 | 22275783 | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 | 8,726,400 | 48,377 | 0.0055 |
09 May 2023 16:37:32 | 1495246 | 22275783 | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 | 6,912,000 | 38,129 | 0.0055 |
09 May 2023 16:37:28 | 1495246 | 22275783 | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 | 6,048,000 | 33,573 | 0.0056 |
09 May 2023 16:37:22 | 1495246 | 22275783 | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 | 5,184,000 | 28,996 | 0.0056 |
09 May 2023 16:37:17 | 1495246 | 22275783 | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 | 4,320,000 | 24,407 | 0.0056 |
09 May 2023 16:15:48 | 1495246 | 22275783 | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 | 2,332,800 | 13,713 | 0.0059 |
09 May 2023 16:15:00 | 1495246 | 22275783 | oifs_43r3_ps_0553_2004050100_123_973_12190197_0 | 1,209,600 | 6,605 | 0.0055 |
©2024 cpdn.org