climateprediction.net home page
Task 22264991

Task 22264991

Name oifs_43r3_ps_0407_1994050100_123_963_12180051_0
Workunit 12180051
Created 22 Dec 2022, 5:39:42 UTC
Sent 19 Jan 2023, 14:32:58 UTC
Report deadline 20 Mar 2023, 14:32:58 UTC
Received 31 Jan 2023, 11:39:22 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1534826
Run time 11 hours 41 min 6 sec
CPU time 2 hours 32 min 46 sec
Validate state Invalid
Credit 6,168.96
Device peak FLOPS 7.39 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,771.55 MB
Peak swap size 4,978.16 MB
Peak disk usage 7,179.90 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ved from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:39:08 STEP 2271 H=2271:00 +CPU= 21.103
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:39:27 STEP 2272 H=2272:00 +CPU= 13.759
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:39:45 STEP 2273 H=2273:00 +CPU= 13.450
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:40:12 STEP 2274 H=2274:00 +CPU= 21.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:40:28 STEP 2275 H=2275:00 +CPU= 13.875
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:40:45 STEP 2276 H=2276:00 +CPU= 12.989
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:41:11 STEP 2277 H=2277:00 +CPU= 20.804
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:41:27 STEP 2278 H=2278:00 +CPU= 13.788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:41:44 STEP 2279 H=2279:00 +CPU= 13.062
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:42:16 STEP 2280 H=2280:00 +CPU= 25.126
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002280
Moving to projects directory: /root/slots/10/ICMSHhq0f+002280
Moving to projects directory: /root/slots/10/ICMUAhq0f+002280
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002256
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002256
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002256
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002268
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002268
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002268
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_15.zip
Uploading the intermediate file: upload_file_15.zip
  16:42:31 STEP 2281 H=2281:00 +CPU= 13.055
  16:42:43 STEP 2282 H=2282:00 +CPU= 12.641
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:43:08 STEP 2283 H=2283:00 +CPU= 20.582
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:43:25 STEP 2284 H=2284:00 +CPU= 13.475
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:43:40 STEP 2285 H=2285:00 +CPU= 12.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:44:07 STEP 2286 H=2286:00 +CPU= 21.596
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:44:24 STEP 2287 H=2287:00 +CPU= 12.455
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:44:39 STEP 2288 H=2288:00 +CPU= 13.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:45:06 STEP 2289 H=2289:00 +CPU= 20.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:45:23 STEP 2290 H=2290:00 +CPU= 12.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:45:38 STEP 2291 H=2291:00 +CPU= 12.791
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:46:04 STEP 2292 H=2292:00 +CPU= 22.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002292
Moving to projects directory: /root/slots/10/ICMSHhq0f+002292
Moving to projects directory: /root/slots/10/ICMUAhq0f+002292
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:46:22 STEP 2293 H=2293:00 +CPU= 13.783
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:46:38 STEP 2294 H=2294:00 +CPU= 13.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:47:05 STEP 2295 H=2295:00 +CPU= 21.437
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:47:23 STEP 2296 H=2296:00 +CPU= 13.537
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:47:38 STEP 2297 H=2297:00 +CPU= 13.074
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:48:04 STEP 2298 H=2298:00 +CPU= 20.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:48:22 STEP 2299 H=2299:00 +CPU= 13.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:48:37 STEP 2300 H=2300:00 +CPU= 13.447
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:49:04 STEP 2301 H=2301:00 +CPU= 20.933
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:49:22 STEP 2302 H=2302:00 +CPU= 13.590
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:49:37 STEP 2303 H=2303:00 +CPU= 13.428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:50:08 STEP 2304 H=2304:00 +CPU= 24.534
Moving to projects directory: /root/slots/10/ICMGGhq0f+002304
Moving to projects directory: /root/slots/10/ICMSHhq0f+002304
Moving to projects directory: /root/slots/10/ICMUAhq0f+002304
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002280
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002280
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002280
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002292
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002292
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002292
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_16.zip
Uploading the intermediate file: upload_file_16.zip
  16:50:21 STEP 2305 H=2305:00 +CPU= 12.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:50:36 STEP 2306 H=2306:00 +CPU= 13.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:51:00 STEP 2307 H=2307:00 +CPU= 19.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:51:17 STEP 2308 H=2308:00 +CPU= 13.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:51:34 STEP 2309 H=2309:00 +CPU= 13.087
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:51:59 STEP 2310 H=2310:00 +CPU= 20.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:52:16 STEP 2311 H=2311:00 +CPU= 13.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:52:33 STEP 2312 H=2312:00 +CPU= 12.867
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:52:57 STEP 2313 H=2313:00 +CPU= 20.128
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:53:14 STEP 2314 H=2314:00 +CPU= 13.248
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:53:32 STEP 2315 H=2315:00 +CPU= 13.700
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:53:58 STEP 2316 H=2316:00 +CPU= 22.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002316
Moving to projects directory: /root/slots/10/ICMSHhq0f+002316
Moving to projects directory: /root/slots/10/ICMUAhq0f+002316
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:54:16 STEP 2317 H=2317:00 +CPU= 13.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:54:34 STEP 2318 H=2318:00 +CPU= 14.211
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:55:01 STEP 2319 H=2319:00 +CPU= 20.812
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:55:17 STEP 2320 H=2320:00 +CPU= 13.938
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:55:35 STEP 2321 H=2321:00 +CPU= 13.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:56:02 STEP 2322 H=2322:00 +CPU= 20.955
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:56:17 STEP 2323 H=2323:00 +CPU= 13.782
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:56:35 STEP 2324 H=2324:00 +CPU= 13.929
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:56:59 STEP 2325 H=2325:00 +CPU= 19.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:57:17 STEP 2326 H=2326:00 +CPU= 13.835
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:57:35 STEP 2327 H=2327:00 +CPU= 13.981
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:58:05 STEP 2328 H=2328:00 +CPU= 24.322
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002328
Moving to projects directory: /root/slots/10/ICMSHhq0f+002328
Moving to projects directory: /root/slots/10/ICMUAhq0f+002328
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002304
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002304
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002304
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002316
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002316
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002316
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_17.zip
Uploading the intermediate file: upload_file_17.zip
  16:58:20 STEP 2329 H=2329:00 +CPU= 13.065
  16:58:34 STEP 2330 H=2330:00 +CPU= 13.810
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:58:59 STEP 2331 H=2331:00 +CPU= 20.558
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:59:16 STEP 2332 H=2332:00 +CPU= 13.137
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:59:33 STEP 2333 H=2333:00 +CPU= 13.087
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:59:58 STEP 2334 H=2334:00 +CPU= 20.764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1994050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0407
(argv4) batchid: 963
(argv5) wuid: 12180051
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /root/slots/10
Project directory is: /root/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /root/projects/climateprediction.net/oifs_43r3_ps_12180051
..mkdir for temp folder for results failed
Copying: /root/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /root/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /root/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ad0aede5e5b2962576b49028ecffcb48 to: /root/slots/10/oifs_43r3_ps_0407_1994050100_123_963_12180051.zip
Unzipping the namelist zip file: /root/slots/10/oifs_43r3_ps_0407_1994050100_123_963_12180051.zip
ic_ancil_file: ic_ancil_12180051
ifsdata_file: ifsdata_12180051
climate_data_file: clim_data_12180051
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_d3cb5cd7d264232902dc61bc92d6e311 to: /root/slots/10/ic_ancil_12180051.zip
Unzipping the IC ancils zip file: /root/slots/10/ic_ancil_12180051.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1e120d6feeced73350c1860406800e62 to: /root/slots/10/ifsdata/ifsdata_12180051.zip
Unzipping the ifsdata_zip file: /root/slots/10/ifsdata/ifsdata_12180051.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /root/slots/10/159l_2/clim_data_12180051.zip
Unzipping the climate data zip file: /root/slots/10/159l_2/clim_data_12180051.zip
Checking for progress XML file: /root/slots/10/progress_file_12180051.xml
Opened progress file ok : /root/slots/10/progress_file_12180051.xml
-- Model is restarting --
Adjusting last_iter, 2335, to previous model restart step.
Creating progress file: /root/slots/10/progress_file_12180051.xml
last_cpu_time: 7133
upload_file_number: 18
last_iter: 2327
last_upload: 8380800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388
The child process has been launched with process id: 4658
Executing the command: /root/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:4658:4658] [20230131:115544:1675162544:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:56:15 STEP 2328 H=2328:00 +CPU= 22.957
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:56:32 STEP 2329 H=2329:00 +CPU= 13.242
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:56:49 STEP 2330 H=2330:00 +CPU= 13.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:57:14 STEP 2331 H=2331:00 +CPU= 20.065
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:57:31 STEP 2332 H=2332:00 +CPU= 13.752
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:57:49 STEP 2333 H=2333:00 +CPU= 13.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:58:14 STEP 2334 H=2334:00 +CPU= 20.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:58:31 STEP 2335 H=2335:00 +CPU= 13.853
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:58:49 STEP 2336 H=2336:00 +CPU= 13.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:59:14 STEP 2337 H=2337:00 +CPU= 20.714
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:59:31 STEP 2338 H=2338:00 +CPU= 13.605
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:59:46 STEP 2339 H=2339:00 +CPU= 12.958
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:16 STEP 2340 H=2340:00 +CPU= 23.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002340
Moving to projects directory: /root/slots/10/ICMSHhq0f+002340
Moving to projects directory: /root/slots/10/ICMUAhq0f+002340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:33 STEP 2341 H=2341:00 +CPU= 12.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:50 STEP 2342 H=2342:00 +CPU= 13.091
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:01:15 STEP 2343 H=2343:00 +CPU= 21.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:01:32 STEP 2344 H=2344:00 +CPU= 13.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:01:49 STEP 2345 H=2345:00 +CPU= 12.872
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:02:14 STEP 2346 H=2346:00 +CPU= 21.322
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:02:33 STEP 2347 H=2347:00 +CPU= 13.545
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:02:50 STEP 2348 H=2348:00 +CPU= 13.472
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:03:15 STEP 2349 H=2349:00 +CPU= 20.702
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:03:33 STEP 2350 H=2350:00 +CPU= 14.002
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:03:51 STEP 2351 H=2351:00 +CPU= 13.809
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:04:21 STEP 2352 H=2352:00 +CPU= 24.001
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002352
Moving to projects directory: /root/slots/10/ICMSHhq0f+002352
Moving to projects directory: /root/slots/10/ICMUAhq0f+002352
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002328
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002328
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002328
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002340
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002340
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002340
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_18.zip
Uploading the intermediate file: upload_file_18.zip
  12:04:36 STEP 2353 H=2353:00 +CPU= 13.373
  12:04:50 STEP 2354 H=2354:00 +CPU= 13.361
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:05:14 STEP 2355 H=2355:00 +CPU= 20.622
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:05:32 STEP 2356 H=2356:00 +CPU= 13.380
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:05:49 STEP 2357 H=2357:00 +CPU= 13.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:06:14 STEP 2358 H=2358:00 +CPU= 20.775
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:06:31 STEP 2359 H=2359:00 +CPU= 13.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:06:46 STEP 2360 H=2360:00 +CPU= 13.184
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:12 STEP 2361 H=2361:00 +CPU= 20.181
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:29 STEP 2362 H=2362:00 +CPU= 12.833
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:45 STEP 2363 H=2363:00 +CPU= 13.526
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:08:13 STEP 2364 H=2364:00 +CPU= 22.507
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002364
Moving to projects directory: /root/slots/10/ICMSHhq0f+002364
Moving to projects directory: /root/slots/10/ICMUAhq0f+002364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:08:30 STEP 2365 H=2365:00 +CPU= 12.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:08:46 STEP 2366 H=2366:00 +CPU= 13.716
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:09:12 STEP 2367 H=2367:00 +CPU= 20.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:09:30 STEP 2368 H=2368:00 +CPU= 13.664
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:09:48 STEP 2369 H=2369:00 +CPU= 14.155
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:10:13 STEP 2370 H=2370:00 +CPU= 20.830
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:10:30 STEP 2371 H=2371:00 +CPU= 13.500
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:10:48 STEP 2372 H=2372:00 +CPU= 13.916
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:11:13 STEP 2373 H=2373:00 +CPU= 20.967
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:11:30 STEP 2374 H=2374:00 +CPU= 13.324
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:11:48 STEP 2375 H=2375:00 +CPU= 13.924
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:12:19 STEP 2376 H=2376:00 +CPU= 24.728
Moving to projects directory: /root/slots/10/ICMGGhq0f+002376
Moving to projects directory: /root/slots/10/ICMSHhq0f+002376
Moving to projects directory: /root/slots/10/ICMUAhq0f+002376
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002352
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002352
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002352
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002364
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002364
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002364
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_19.zip
Uploading the intermediate file: upload_file_19.zip
  12:12:32 STEP 2377 H=2377:00 +CPU= 13.124
  12:12:46 STEP 2378 H=2378:00 +CPU= 13.281
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:13:11 STEP 2379 H=2379:00 +CPU= 18.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:13:26 STEP 2380 H=2380:00 +CPU= 13.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:13:44 STEP 2381 H=2381:00 +CPU= 13.577
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:14:09 STEP 2382 H=2382:00 +CPU= 19.424
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:14:25 STEP 2383 H=2383:00 +CPU= 13.317
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:14:42 STEP 2384 H=2384:00 +CPU= 13.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:15:06 STEP 2385 H=2385:00 +CPU= 19.824
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:15:24 STEP 2386 H=2386:00 +CPU= 13.721
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:15:41 STEP 2387 H=2387:00 +CPU= 13.164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:16:07 STEP 2388 H=2388:00 +CPU= 21.461
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002388
Moving to projects directory: /root/slots/10/ICMSHhq0f+002388
Moving to projects directory: /root/slots/10/ICMUAhq0f+002388
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:16:24 STEP 2389 H=2389:00 +CPU= 13.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:16:41 STEP 2390 H=2390:00 +CPU= 13.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:17:08 STEP 2391 H=2391:00 +CPU= 20.478
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:17:25 STEP 2392 H=2392:00 +CPU= 14.564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:17:42 STEP 2393 H=2393:00 +CPU= 13.793
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:18:09 STEP 2394 H=2394:00 +CPU= 20.922
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:18:25 STEP 2395 H=2395:00 +CPU= 13.366
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:18:43 STEP 2396 H=2396:00 +CPU= 13.890
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:19:09 STEP 2397 H=2397:00 +CPU= 20.250
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:19:25 STEP 2398 H=2398:00 +CPU= 14.140
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:19:43 STEP 2399 H=2399:00 +CPU= 14.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:20:14 STEP 2400 H=2400:00 +CPU= 24.524
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002400
Moving to projects directory: /root/slots/10/ICMSHhq0f+002400
Moving to projects directory: /root/slots/10/ICMUAhq0f+002400
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002376
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002376
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002376
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002388
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002388
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002388
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_20.zip
Uploading the intermediate file: upload_file_20.zip
  12:20:28 STEP 2401 H=2401:00 +CPU= 12.430
  12:20:42 STEP 2402 H=2402:00 +CPU= 13.786
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:21:08 STEP 2403 H=2403:00 +CPU= 20.379
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:21:24 STEP 2404 H=2404:00 +CPU= 13.806
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:21:41 STEP 2405 H=2405:00 +CPU= 13.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:22:06 STEP 2406 H=2406:00 +CPU= 20.370
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:22:23 STEP 2407 H=2407:00 +CPU= 12.729
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:22:40 STEP 2408 H=2408:00 +CPU= 13.020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:23:04 STEP 2409 H=2409:00 +CPU= 19.918
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:23:19 STEP 2410 H=2410:00 +CPU= 11.959
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:23:35 STEP 2411 H=2411:00 +CPU= 13.262
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:24:04 STEP 2412 H=2412:00 +CPU= 22.834
Moving to projects directory: /root/slots/10/ICMGGhq0f+002412
Moving to projects directory: /root/slots/10/ICMSHhq0f+002412
Moving to projects directory: /root/slots/10/ICMUAhq0f+002412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:24:21 STEP 2413 H=2413:00 +CPU= 13.350
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:24:39 STEP 2414 H=2414:00 +CPU= 13.885
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:25:04 STEP 2415 H=2415:00 +CPU= 20.820
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:25:22 STEP 2416 H=2416:00 +CPU= 14.319
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:25:40 STEP 2417 H=2417:00 +CPU= 13.999
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:26:05 STEP 2418 H=2418:00 +CPU= 20.443
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:26:22 STEP 2419 H=2419:00 +CPU= 13.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:26:39 STEP 2420 H=2420:00 +CPU= 13.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:27:04 STEP 2421 H=2421:00 +CPU= 20.334
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:27:22 STEP 2422 H=2422:00 +CPU= 14.248
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:27:40 STEP 2423 H=2423:00 +CPU= 13.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:28:09 STEP 2424 H=2424:00 +CPU= 23.508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002424
Moving to projects directory: /root/slots/10/ICMSHhq0f+002424
Moving to projects directory: /root/slots/10/ICMUAhq0f+002424
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002400
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002400
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002400
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002412
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002412
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002412
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_21.zip
Uploading the intermediate file: upload_file_21.zip
  12:28:25 STEP 2425 H=2425:00 +CPU= 13.979
  12:28:38 STEP 2426 H=2426:00 +CPU= 12.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:29:02 STEP 2427 H=2427:00 +CPU= 20.239
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:29:19 STEP 2428 H=2428:00 +CPU= 12.901
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:29:35 STEP 2429 H=2429:00 +CPU= 13.414
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:30:01 STEP 2430 H=2430:00 +CPU= 19.376
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:30:18 STEP 2431 H=2431:00 +CPU= 13.230
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:30:35 STEP 2432 H=2432:00 +CPU= 14.405
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:31:01 STEP 2433 H=2433:00 +CPU= 20.652
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:31:19 STEP 2434 H=2434:00 +CPU= 13.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:31:35 STEP 2435 H=2435:00 +CPU= 13.652
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:32:04 STEP 2436 H=2436:00 +CPU= 23.000
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/10/ICMGGhq0f+002436
Moving to projects directory: /root/slots/10/ICMSHhq0f+002436
Moving to projects directory: /root/slots/10/ICMUAhq0f+002436
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:32:22 STEP 2437 H=2437:00 +CPU= 13.885
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:32:40 STEP 2438 H=2438:00 +CPU= 14.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:33:04 STEP 2439 H=2439:00 +CPU= 20.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:33:22 STEP 2440 H=2440:00 +CPU= 13.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:33:39 STEP 2441 H=2441:00 +CPU= 13.708
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:34:04 STEP 2442 H=2442:00 +CPU= 20.585
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:34:21 STEP 2443 H=2443:00 +CPU= 13.255
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:34:39 STEP 2444 H=2444:00 +CPU= 13.791
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:35:04 STEP 2445 H=2445:00 +CPU= 20.647
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:35:21 STEP 2446 H=2446:00 +CPU= 13.315
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:35:39 STEP 2447 H=2447:00 +CPU= 13.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:36:09 STEP 2448 H=2448:00 +CPU= 24.194
Moving to projects directory: /root/slots/10/ICMGGhq0f+002448
Moving to projects directory: /root/slots/10/ICMSHhq0f+002448
Moving to projects directory: /root/slots/10/ICMUAhq0f+002448
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002424
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002424
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002424
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMGGhq0f+002436
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMSHhq0f+002436
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12180051/ICMUAhq0f+002436
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0407_1994050100_123_963_12180051_0_r310147388_22.zip
Uploading the intermediate file: upload_file_22.zip
  12:36:23 STEP 2449 H=2449:00 +CPU= 13.582
  12:36:35 STEP 2450 H=2450:00 +CPU= 12.297
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:37:02 STEP 2451 H=2451:00 +CPU= 20.318
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:37:20 STEP 2452 H=2452:00 +CPU= 14.175
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:37:35 STEP 2453 H=2453:00 +CPU= 13.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:42:14 1534826 22264991 oifs_43r3_ps_0407_1994050100_123_963_12180051_0 4,406,400 17,449 0.0040
09 May 2023 16:42:11 1534826 22264991 oifs_43r3_ps_0407_1994050100_123_963_12180051_0 3,542,400 14,344 0.0040
09 May 2023 16:42:08 1534826 22264991 oifs_43r3_ps_0407_1994050100_123_963_12180051_0 2,678,400 11,259 0.0042
09 May 2023 16:42:05 1534826 22264991 oifs_43r3_ps_0407_1994050100_123_963_12180051_0 1,814,400 8,034 0.0044
09 May 2023 16:28:53 1534826 22264991 oifs_43r3_ps_0407_1994050100_123_963_12180051_0 864,000 3,665 0.0042


©2024 cpdn.org