climateprediction.net home page
Task 22252255

Task 22252255

Name oifs_43r3_ps_0381_1982050100_123_951_12168025_0
Workunit 12168025
Created 20 Dec 2022, 21:59:51 UTC
Sent 11 Jan 2023, 13:23:08 UTC
Report deadline 10 Feb 2023, 13:23:08 UTC
Received 12 Jan 2023, 9:55:03 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1537299
Run time 18 hours 51 min 11 sec
CPU time 18 hours 38 min 43 sec
Validate state Invalid
Credit 4,112.64
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.96 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,457.44 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
  511 H= 511:00 +CPU= 48.068
  14:44:55 STEP  512 H= 512:00 +CPU= 34.572
  14:45:34 STEP  513 H= 513:00 +CPU= 38.913
  14:46:09 STEP  514 H= 514:00 +CPU= 34.780
  14:46:56 STEP  515 H= 515:00 +CPU= 46.709
  14:47:50 STEP  516 H= 516:00 +CPU= 54.047
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000516
  14:48:15 STEP  517 H= 517:00 +CPU= 24.879
  14:48:45 STEP  518 H= 518:00 +CPU= 30.136
  14:49:49 STEP  519 H= 519:00 +CPU= 63.257
  14:50:19 STEP  520 H= 520:00 +CPU= 30.153
  14:50:43 STEP  521 H= 521:00 +CPU= 24.172
  14:51:28 STEP  522 H= 522:00 +CPU= 44.748
  14:52:07 STEP  523 H= 523:00 +CPU= 38.338
  14:52:54 STEP  524 H= 524:00 +CPU= 46.811
  14:53:34 STEP  525 H= 525:00 +CPU= 40.312
  14:54:01 STEP  526 H= 526:00 +CPU= 25.959
  14:54:31 STEP  527 H= 527:00 +CPU= 30.448
  14:55:47 STEP  528 H= 528:00 +CPU= 75.394
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_21.zip
Uploading the intermediate file: upload_file_21.zip
  14:56:17 STEP  529 H= 529:00 +CPU= 29.672
  14:56:44 STEP  530 H= 530:00 +CPU= 27.200
  14:57:30 STEP  531 H= 531:00 +CPU= 45.461
  14:58:11 STEP  532 H= 532:00 +CPU= 40.634
  14:58:41 STEP  533 H= 533:00 +CPU= 30.052
  14:59:22 STEP  534 H= 534:00 +CPU= 40.823
  14:59:56 STEP  535 H= 535:00 +CPU= 33.667
  15:00:36 STEP  536 H= 536:00 +CPU= 40.074
  15:01:26 STEP  537 H= 537:00 +CPU= 50.238
  15:02:02 STEP  538 H= 538:00 +CPU= 35.092
  15:02:46 STEP  539 H= 539:00 +CPU= 44.137
  15:03:25 STEP  540 H= 540:00 +CPU= 39.268
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000540
  15:03:47 STEP  541 H= 541:00 +CPU= 21.569
  15:04:10 STEP  542 H= 542:00 +CPU= 22.786
  15:05:00 STEP  543 H= 543:00 +CPU= 50.201
  15:05:47 STEP  544 H= 544:00 +CPU= 46.072
  15:06:17 STEP  545 H= 545:00 +CPU= 29.961
  15:06:53 STEP  546 H= 546:00 +CPU= 36.260
  15:07:26 STEP  547 H= 547:00 +CPU= 32.307
  15:08:07 STEP  548 H= 548:00 +CPU= 41.397
  15:09:01 STEP  549 H= 549:00 +CPU= 53.697
  15:09:23 STEP  550 H= 550:00 +CPU= 21.823
  15:09:45 STEP  551 H= 551:00 +CPU= 21.541
  15:10:45 STEP  552 H= 552:00 +CPU= 60.184
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_22.zip
Uploading the intermediate file: upload_file_22.zip
  15:11:29 STEP  553 H= 553:00 +CPU= 43.020
  15:11:58 STEP  554 H= 554:00 +CPU= 29.684
  15:12:37 STEP  555 H= 555:00 +CPU= 38.371
  15:13:04 STEP  556 H= 556:00 +CPU= 27.380
  15:13:41 STEP  557 H= 557:00 +CPU= 36.905
  15:14:22 STEP  558 H= 558:00 +CPU= 40.758
  15:14:44 STEP  559 H= 559:00 +CPU= 21.706
  15:15:06 STEP  560 H= 560:00 +CPU= 22.004
  15:15:43 STEP  561 H= 561:00 +CPU= 36.640
  15:16:27 STEP  562 H= 562:00 +CPU= 43.376
  15:17:16 STEP  563 H= 563:00 +CPU= 48.795
  15:18:08 STEP  564 H= 564:00 +CPU= 51.439
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000564
  15:18:40 STEP  565 H= 565:00 +CPU= 32.474
  15:19:16 STEP  566 H= 566:00 +CPU= 35.873
  15:20:05 STEP  567 H= 567:00 +CPU= 48.124
  15:20:29 STEP  568 H= 568:00 +CPU= 24.206
  15:21:03 STEP  569 H= 569:00 +CPU= 33.836
  15:21:47 STEP  570 H= 570:00 +CPU= 44.206
  15:22:16 STEP  571 H= 571:00 +CPU= 28.427
  15:22:47 STEP  572 H= 572:00 +CPU= 31.016
  15:23:42 STEP  573 H= 573:00 +CPU= 54.216
  15:24:09 STEP  574 H= 574:00 +CPU= 27.275
  15:24:35 STEP  575 H= 575:00 +CPU= 25.820
  15:25:26 STEP  576 H= 576:00 +CPU= 51.356
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_23.zip
Uploading the intermediate file: upload_file_23.zip
  15:26:07 STEP  577 H= 577:00 +CPU= 40.187
  15:26:43 STEP  578 H= 578:00 +CPU= 35.684
  15:27:23 STEP  579 H= 579:00 +CPU= 40.546
  15:28:01 STEP  580 H= 580:00 +CPU= 36.937
  15:28:45 STEP  581 H= 581:00 +CPU= 43.942
  15:30:20 STEP  582 H= 582:00 +CPU= 94.193
  15:31:31 STEP  583 H= 583:00 +CPU= 70.310
  15:32:29 STEP  584 H= 584:00 +CPU= 57.643
  15:34:17 STEP  585 H= 585:00 +CPU=106.918
  15:35:35 STEP  586 H= 586:00 +CPU= 77.502
  15:36:51 STEP  587 H= 587:00 +CPU= 75.429
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1982050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0381
(argv4) batchid: 951
(argv5) wuid: 12168025
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/28
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/28/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/28/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8510bd62237bef0d9507589efed46804 to: /var/lib/boinc-client/slots/28/oifs_43r3_ps_0381_1982050100_123_951_12168025.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/28/oifs_43r3_ps_0381_1982050100_123_951_12168025.zip
ic_ancil_file: ic_ancil_12168025
ifsdata_file: ifsdata_12168025
climate_data_file: clim_data_12168025
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b1ea7927b96be2c464948f3d2c290718 to: /var/lib/boinc-client/slots/28/ic_ancil_12168025.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/28/ic_ancil_12168025.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_55a405c57c19a66b6ae3a8b3b074b0a0 to: /var/lib/boinc-client/slots/28/ifsdata/ifsdata_12168025.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/28/ifsdata/ifsdata_12168025.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/28/159l_2/clim_data_12168025.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/28/159l_2/clim_data_12168025.zip
Checking for progress XML file: /var/lib/boinc-client/slots/28/progress_file_12168025.xml
Opened progress file ok : /var/lib/boinc-client/slots/28/progress_file_12168025.xml
-- Model is restarting --
Adjusting last_iter, 588, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/28/progress_file_12168025.xml
last_cpu_time: 26635
upload_file_number: 24
last_iter: 575
last_upload: 2073600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294
The child process has been launched with process id: 50468
Executing the command: /var/lib/boinc-client/slots/28/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50468:50468] [20230111:153958:1673473198:0.002] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:42:09 STEP  576 H= 576:00 +CPU=117.378
  15:43:04 STEP  577 H= 577:00 +CPU= 54.726
  15:43:30 STEP  578 H= 578:00 +CPU= 25.410
  15:44:16 STEP  579 H= 579:00 +CPU= 46.226
  15:44:58 STEP  580 H= 580:00 +CPU= 41.881
  15:45:42 STEP  581 H= 581:00 +CPU= 44.057
  15:46:26 STEP  582 H= 582:00 +CPU= 43.880
  15:46:57 STEP  583 H= 583:00 +CPU= 30.606
  15:47:41 STEP  584 H= 584:00 +CPU= 43.320
  15:48:16 STEP  585 H= 585:00 +CPU= 35.219
  15:48:40 STEP  586 H= 586:00 +CPU= 24.034
  15:49:03 STEP  587 H= 587:00 +CPU= 22.951
  15:49:57 STEP  588 H= 588:00 +CPU= 53.913
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000588
  15:50:36 STEP  589 H= 589:00 +CPU= 38.086
  15:51:00 STEP  590 H= 590:00 +CPU= 23.831
  15:51:38 STEP  591 H= 591:00 +CPU= 37.980
  15:52:07 STEP  592 H= 592:00 +CPU= 29.020
  15:52:58 STEP  593 H= 593:00 +CPU= 51.136
  15:53:40 STEP  594 H= 594:00 +CPU= 41.143
  15:54:05 STEP  595 H= 595:00 +CPU= 25.417
  15:54:34 STEP  596 H= 596:00 +CPU= 28.739
  15:55:30 STEP  597 H= 597:00 +CPU= 55.343
  15:56:19 STEP  598 H= 598:00 +CPU= 48.548
  15:57:05 STEP  599 H= 599:00 +CPU= 46.151
  15:58:15 STEP  600 H= 600:00 +CPU= 69.807
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_24.zip
Uploading the intermediate file: upload_file_24.zip
  15:58:55 STEP  601 H= 601:00 +CPU= 39.367
  15:59:32 STEP  602 H= 602:00 +CPU= 36.764
  16:01:03 STEP  603 H= 603:00 +CPU= 91.104
  16:02:22 STEP  604 H= 604:00 +CPU= 77.743
  16:03:41 STEP  605 H= 605:00 +CPU= 78.005
  16:05:40 STEP  606 H= 606:00 +CPU=117.257
  16:06:56 STEP  607 H= 607:00 +CPU= 75.179
  16:08:15 STEP  608 H= 608:00 +CPU= 78.391
  16:10:07 STEP  609 H= 609:00 +CPU=111.123
  16:11:28 STEP  610 H= 610:00 +CPU= 79.373
  16:12:44 STEP  611 H= 611:00 +CPU= 75.608
  16:14:54 STEP  612 H= 612:00 +CPU=128.853
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000612
  16:16:18 STEP  613 H= 613:00 +CPU= 83.402
  16:17:37 STEP  614 H= 614:00 +CPU= 78.413
  16:19:37 STEP  615 H= 615:00 +CPU=117.890
  16:20:55 STEP  616 H= 616:00 +CPU= 77.420
  16:22:12 STEP  617 H= 617:00 +CPU= 76.195
  16:24:04 STEP  618 H= 618:00 +CPU=111.012
  16:25:25 STEP  619 H= 619:00 +CPU= 79.776
  16:26:49 STEP  620 H= 620:00 +CPU= 82.596
  16:28:46 STEP  621 H= 621:00 +CPU=116.156
  16:30:03 STEP  622 H= 622:00 +CPU= 76.372
  16:31:29 STEP  623 H= 623:00 +CPU= 84.850
  16:33:47 STEP  624 H= 624:00 +CPU=136.753
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_25.zip
Uploading the intermediate file: upload_file_25.zip
  16:35:05 STEP  625 H= 625:00 +CPU= 77.118
  16:36:24 STEP  626 H= 626:00 +CPU= 77.803
  16:38:16 STEP  627 H= 627:00 +CPU=110.590
  16:39:33 STEP  628 H= 628:00 +CPU= 75.931
  16:40:49 STEP  629 H= 629:00 +CPU= 75.869
  16:42:41 STEP  630 H= 630:00 +CPU=110.239
  16:44:06 STEP  631 H= 631:00 +CPU= 84.784
  16:45:26 STEP  632 H= 632:00 +CPU= 79.259
  16:47:25 STEP  633 H= 633:00 +CPU=116.974
  16:48:45 STEP  634 H= 634:00 +CPU= 79.600
  16:50:02 STEP  635 H= 635:00 +CPU= 76.511
  16:52:03 STEP  636 H= 636:00 +CPU=119.831
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000636
  16:53:25 STEP  637 H= 637:00 +CPU= 80.846
  16:54:52 STEP  638 H= 638:00 +CPU= 86.810
  16:56:45 STEP  639 H= 639:00 +CPU=111.316
  16:58:04 STEP  640 H= 640:00 +CPU= 78.215
  16:59:20 STEP  641 H= 641:00 +CPU= 74.627
  17:01:11 STEP  642 H= 642:00 +CPU=109.666
  17:02:27 STEP  643 H= 643:00 +CPU= 75.014
  17:03:46 STEP  644 H= 644:00 +CPU= 78.264
  17:05:40 STEP  645 H= 645:00 +CPU=113.297
  17:06:59 STEP  646 H= 646:00 +CPU= 77.853
  17:08:16 STEP  647 H= 647:00 +CPU= 76.309
  17:10:28 STEP  648 H= 648:00 +CPU=130.568
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_26.zip
Uploading the intermediate file: upload_file_26.zip
  17:11:44 STEP  649 H= 649:00 +CPU= 75.190
  17:12:59 STEP  650 H= 650:00 +CPU= 74.376
  17:14:48 STEP  651 H= 651:00 +CPU=107.888
  17:16:10 STEP  652 H= 652:00 +CPU= 80.218
  17:17:27 STEP  653 H= 653:00 +CPU= 76.942
  17:19:18 STEP  654 H= 654:00 +CPU=109.779
  17:20:34 STEP  655 H= 655:00 +CPU= 74.260
  17:21:42 STEP  656 H= 656:00 +CPU= 67.311
  17:23:21 STEP  657 H= 657:00 +CPU= 98.581
  17:24:34 STEP  658 H= 658:00 +CPU= 72.603
  17:25:47 STEP  659 H= 659:00 +CPU= 71.482
  17:27:39 STEP  660 H= 660:00 +CPU=111.026
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000660
  17:28:49 STEP  661 H= 661:00 +CPU= 69.968
  17:29:56 STEP  662 H= 662:00 +CPU= 65.980
  17:31:38 STEP  663 H= 663:00 +CPU=101.192
  17:32:46 STEP  664 H= 664:00 +CPU= 66.709
  17:33:53 STEP  665 H= 665:00 +CPU= 66.885
  17:35:39 STEP  666 H= 666:00 +CPU=104.836
  17:36:50 STEP  667 H= 667:00 +CPU= 69.830
  17:38:00 STEP  668 H= 668:00 +CPU= 69.778
  17:39:42 STEP  669 H= 669:00 +CPU=101.377
  17:40:53 STEP  670 H= 670:00 +CPU= 70.126
  17:42:06 STEP  671 H= 671:00 +CPU= 72.484
  17:44:12 STEP  672 H= 672:00 +CPU=124.275
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_27.zip
Uploading the intermediate file: upload_file_27.zip
  17:45:21 STEP  673 H= 673:00 +CPU= 68.708
  17:46:36 STEP  674 H= 674:00 +CPU= 73.862
  17:48:16 STEP  675 H= 675:00 +CPU= 99.681
  17:49:27 STEP  676 H= 676:00 +CPU= 70.117
  17:50:37 STEP  677 H= 677:00 +CPU= 69.317
  17:52:19 STEP  678 H= 678:00 +CPU=101.173
  17:53:26 STEP  679 H= 679:00 +CPU= 66.017
  17:54:36 STEP  680 H= 680:00 +CPU= 69.797
  17:56:19 STEP  681 H= 681:00 +CPU=101.688
  17:57:24 STEP  682 H= 682:00 +CPU= 64.642
  17:58:34 STEP  683 H= 683:00 +CPU= 68.817
  18:00:22 STEP  684 H= 684:00 +CPU=107.784
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000684
  18:01:30 STEP  685 H= 685:00 +CPU= 66.756
  18:02:44 STEP  686 H= 686:00 +CPU= 73.143
  18:04:26 STEP  687 H= 687:00 +CPU=100.704
  18:05:40 STEP  688 H= 688:00 +CPU= 73.199
  18:06:52 STEP  689 H= 689:00 +CPU= 71.200
  18:08:31 STEP  690 H= 690:00 +CPU= 98.666
  18:09:42 STEP  691 H= 691:00 +CPU= 70.048
  18:10:56 STEP  692 H= 692:00 +CPU= 72.194
  18:12:40 STEP  693 H= 693:00 +CPU=102.779
  18:13:50 STEP  694 H= 694:00 +CPU= 69.617
  18:15:03 STEP  695 H= 695:00 +CPU= 71.715
  18:17:04 STEP  696 H= 696:00 +CPU=120.537
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_28.zip
Uploading the intermediate file: upload_file_28.zip
  18:18:17 STEP  697 H= 697:00 +CPU= 71.343
  18:19:27 STEP  698 H= 698:00 +CPU= 69.930
  18:21:07 STEP  699 H= 699:00 +CPU= 98.983
  18:22:19 STEP  700 H= 700:00 +CPU= 70.758
  18:23:29 STEP  701 H= 701:00 +CPU= 70.016
  18:25:11 STEP  702 H= 702:00 +CPU=100.993
  18:26:26 STEP  703 H= 703:00 +CPU= 73.954
  18:27:34 STEP  704 H= 704:00 +CPU= 67.136
  18:29:11 STEP  705 H= 705:00 +CPU= 96.387
  18:30:23 STEP  706 H= 706:00 +CPU= 71.659
  18:31:41 STEP  707 H= 707:00 +CPU= 77.253
  18:33:30 STEP  708 H= 708:00 +CPU=107.746
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000708
  18:34:42 STEP  709 H= 709:00 +CPU= 71.193
  18:35:54 STEP  710 H= 710:00 +CPU= 71.185
  18:37:34 STEP  711 H= 711:00 +CPU= 99.387
  18:38:41 STEP  712 H= 712:00 +CPU= 66.009
  18:39:49 STEP  713 H= 713:00 +CPU= 67.836
  18:41:32 STEP  714 H= 714:00 +CPU=101.413
  18:42:42 STEP  715 H= 715:00 +CPU= 69.082
  18:43:58 STEP  716 H= 716:00 +CPU= 75.613
  18:45:40 STEP  717 H= 717:00 +CPU=100.802
  18:46:47 STEP  718 H= 718:00 +CPU= 66.811
  18:48:01 STEP  719 H= 719:00 +CPU= 73.174
  18:50:00 STEP  720 H= 720:00 +CPU=117.670
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_29.zip
Uploading the intermediate file: upload_file_29.zip
  18:51:07 STEP  721 H= 721:00 +CPU= 66.272
  18:52:19 STEP  722 H= 722:00 +CPU= 70.813
  18:54:00 STEP  723 H= 723:00 +CPU= 99.652
  18:55:11 STEP  724 H= 724:00 +CPU= 71.017
  18:56:23 STEP  725 H= 725:00 +CPU= 70.338
  18:58:00 STEP  726 H= 726:00 +CPU= 96.504
  18:59:12 STEP  727 H= 727:00 +CPU= 70.991
  19:00:20 STEP  728 H= 728:00 +CPU= 68.087
  19:01:58 STEP  729 H= 729:00 +CPU= 96.393
  19:03:08 STEP  730 H= 730:00 +CPU= 69.791
  19:04:19 STEP  731 H= 731:00 +CPU= 69.591
  19:06:07 STEP  732 H= 732:00 +CPU=106.974
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000732
  19:07:16 STEP  733 H= 733:00 +CPU= 68.221
  19:08:25 STEP  734 H= 734:00 +CPU= 68.454
  19:10:07 STEP  735 H= 735:00 +CPU=100.864
  19:11:20 STEP  736 H= 736:00 +CPU= 71.846
  19:12:33 STEP  737 H= 737:00 +CPU= 71.847
  19:14:17 STEP  738 H= 738:00 +CPU=102.871
  19:15:27 STEP  739 H= 739:00 +CPU= 69.261
  19:16:37 STEP  740 H= 740:00 +CPU= 69.347
  19:18:17 STEP  741 H= 741:00 +CPU= 99.584
  19:19:29 STEP  742 H= 742:00 +CPU= 70.450
  19:20:36 STEP  743 H= 743:00 +CPU= 67.155
  19:22:42 STEP  744 H= 744:00 +CPU=124.285
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_30.zip
Uploading the intermediate file: upload_file_30.zip
  19:23:52 STEP  745 H= 745:00 +CPU= 69.084
  19:25:02 STEP  746 H= 746:00 +CPU= 69.538
  19:26:42 STEP  747 H= 747:00 +CPU= 98.521
  19:27:53 STEP  748 H= 748:00 +CPU= 70.631
  19:29:03 STEP  749 H= 749:00 +CPU= 69.180
  19:30:41 STEP  750 H= 750:00 +CPU= 97.479
  19:31:51 STEP  751 H= 751:00 +CPU= 68.755
  19:33:04 STEP  752 H= 752:00 +CPU= 72.864
  19:34:43 STEP  753 H= 753:00 +CPU= 98.018
  19:35:54 STEP  754 H= 754:00 +CPU= 70.470
  19:37:04 STEP  755 H= 755:00 +CPU= 68.566
  19:38:56 STEP  756 H= 756:00 +CPU=110.659
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000756
  19:40:00 STEP  757 H= 757:00 +CPU= 64.238
  19:41:12 STEP  758 H= 758:00 +CPU= 71.188
  19:42:51 STEP  759 H= 759:00 +CPU= 97.868
  19:44:05 STEP  760 H= 760:00 +CPU= 73.223
  19:45:15 STEP  761 H= 761:00 +CPU= 69.660
  19:46:58 STEP  762 H= 762:00 +CPU=101.899
  19:48:07 STEP  763 H= 763:00 +CPU= 68.116
  19:49:20 STEP  764 H= 764:00 +CPU= 73.194
  19:51:05 STEP  765 H= 765:00 +CPU=103.408
  19:52:14 STEP  766 H= 766:00 +CPU= 68.757
  19:53:24 STEP  767 H= 767:00 +CPU= 68.742
  19:55:36 STEP  768 H= 768:00 +CPU=130.897
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_31.zip
Uploading the intermediate file: upload_file_31.zip
  19:56:47 STEP  769 H= 769:00 +CPU= 70.860
  19:58:01 STEP  770 H= 770:00 +CPU= 73.118
  19:59:46 STEP  771 H= 771:00 +CPU=103.375
  20:00:59 STEP  772 H= 772:00 +CPU= 73.033
  20:02:09 STEP  773 H= 773:00 +CPU= 69.091
  20:03:48 STEP  774 H= 774:00 +CPU= 98.318
  20:04:55 STEP  775 H= 775:00 +CPU= 66.204
  20:06:03 STEP  776 H= 776:00 +CPU= 66.919
  20:07:43 STEP  777 H= 777:00 +CPU= 99.483
  20:08:53 STEP  778 H= 778:00 +CPU= 69.286
  20:10:01 STEP  779 H= 779:00 +CPU= 66.713
  20:11:53 STEP  780 H= 780:00 +CPU=111.047
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000780
  20:13:04 STEP  781 H= 781:00 +CPU= 69.741
  20:14:13 STEP  782 H= 782:00 +CPU= 68.892
  20:15:56 STEP  783 H= 783:00 +CPU=101.844
  20:17:06 STEP  784 H= 784:00 +CPU= 69.238
  20:18:18 STEP  785 H= 785:00 +CPU= 70.761
  20:20:01 STEP  786 H= 786:00 +CPU=101.873
  20:21:08 STEP  787 H= 787:00 +CPU= 67.091
  20:22:19 STEP  788 H= 788:00 +CPU= 69.878
  20:23:57 STEP  789 H= 789:00 +CPU= 96.749
  20:25:08 STEP  790 H= 790:00 +CPU= 70.517
  20:26:22 STEP  791 H= 791:00 +CPU= 73.573
  20:28:23 STEP  792 H= 792:00 +CPU=119.367
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000780
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_32.zip
Uploading the intermediate file: upload_file_32.zip
  20:29:31 STEP  793 H= 793:00 +CPU= 67.365
  20:30:42 STEP  794 H= 794:00 +CPU= 70.261
  20:32:25 STEP  795 H= 795:00 +CPU=101.944
  20:33:33 STEP  796 H= 796:00 +CPU= 67.790
  20:34:46 STEP  797 H= 797:00 +CPU= 71.453
  20:36:28 STEP  798 H= 798:00 +CPU=101.567
  20:37:36 STEP  799 H= 799:00 +CPU= 66.966
  20:38:44 STEP  800 H= 800:00 +CPU= 67.483
  20:40:27 STEP  801 H= 801:00 +CPU=102.043
  20:41:40 STEP  802 H= 802:00 +CPU= 72.163
  20:42:52 STEP  803 H= 803:00 +CPU= 71.584
  20:44:42 STEP  804 H= 804:00 +CPU=108.535
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000804
  20:45:49 STEP  805 H= 805:00 +CPU= 65.851
  20:46:57 STEP  806 H= 806:00 +CPU= 67.455
  20:48:41 STEP  807 H= 807:00 +CPU=103.550
  20:49:53 STEP  808 H= 808:00 +CPU= 71.067
  20:51:04 STEP  809 H= 809:00 +CPU= 69.789
  20:52:45 STEP  810 H= 810:00 +CPU=100.269
  20:53:54 STEP  811 H= 811:00 +CPU= 67.785
  20:55:06 STEP  812 H= 812:00 +CPU= 71.373
  20:56:52 STEP  813 H= 813:00 +CPU=104.674
  20:57:59 STEP  814 H= 814:00 +CPU= 66.765
  20:59:14 STEP  815 H= 815:00 +CPU= 73.833
  21:01:22 STEP  816 H= 816:00 +CPU=126.537
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000804
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_33.zip
Uploading the intermediate file: upload_file_33.zip
Uploading trickle at timestep: 2937600
  21:02:32 STEP  817 H= 817:00 +CPU= 69.292
  21:03:49 STEP  818 H= 818:00 +CPU= 76.525
  21:05:32 STEP  819 H= 819:00 +CPU=101.419
  21:06:45 STEP  820 H= 820:00 +CPU= 72.580
  21:07:55 STEP  821 H= 821:00 +CPU= 69.411
  21:09:46 STEP  822 H= 822:00 +CPU=109.627
  21:10:57 STEP  823 H= 823:00 +CPU= 70.086
  21:12:09 STEP  824 H= 824:00 +CPU= 71.912
  21:14:01 STEP  825 H= 825:00 +CPU=110.387
  21:15:11 STEP  826 H= 826:00 +CPU= 69.257
  21:16:21 STEP  827 H= 827:00 +CPU= 69.043
  21:18:12 STEP  828 H= 828:00 +CPU=110.790
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000828
  21:19:24 STEP  829 H= 829:00 +CPU= 70.314
  21:20:37 STEP  830 H= 830:00 +CPU= 72.709
  21:22:22 STEP  831 H= 831:00 +CPU=103.523
  21:23:34 STEP  832 H= 832:00 +CPU= 71.973
  21:24:46 STEP  833 H= 833:00 +CPU= 71.146
  21:26:31 STEP  834 H= 834:00 +CPU=104.110
  21:27:38 STEP  835 H= 835:00 +CPU= 66.526
  21:28:50 STEP  836 H= 836:00 +CPU= 70.626
  21:30:41 STEP  837 H= 837:00 +CPU=110.498
  21:31:56 STEP  838 H= 838:00 +CPU= 73.683
  21:33:06 STEP  839 H= 839:00 +CPU= 69.514
  21:35:09 STEP  840 H= 840:00 +CPU=121.901
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000828
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_34.zip
Uploading the intermediate file: upload_file_34.zip
  21:36:16 STEP  841 H= 841:00 +CPU= 66.354
  21:37:27 STEP  842 H= 842:00 +CPU= 70.021
  21:39:16 STEP  843 H= 843:00 +CPU=107.423
  21:40:29 STEP  844 H= 844:00 +CPU= 72.903
  21:41:38 STEP  845 H= 845:00 +CPU= 68.236
  21:43:26 STEP  846 H= 846:00 +CPU=106.517
  21:44:39 STEP  847 H= 847:00 +CPU= 72.212
  21:45:53 STEP  848 H= 848:00 +CPU= 73.306
  21:47:37 STEP  849 H= 849:00 +CPU=103.223
  21:49:01 STEP  850 H= 850:00 +CPU= 82.413
  21:50:15 STEP  851 H= 851:00 +CPU= 73.423
  21:52:15 STEP  852 H= 852:00 +CPU=118.262
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000852
  21:53:38 STEP  853 H= 853:00 +CPU= 82.502
  21:54:53 STEP  854 H= 854:00 +CPU= 74.442
  21:56:42 STEP  855 H= 855:00 +CPU=107.185
  21:58:02 STEP  856 H= 856:00 +CPU= 78.912
  21:59:18 STEP  857 H= 857:00 +CPU= 75.622
  22:01:09 STEP  858 H= 858:00 +CPU=109.701
  22:02:30 STEP  859 H= 859:00 +CPU= 80.422
  22:03:48 STEP  860 H= 860:00 +CPU= 77.029
  22:05:39 STEP  861 H= 861:00 +CPU=109.948
  22:06:52 STEP  862 H= 862:00 +CPU= 72.263
  22:08:13 STEP  863 H= 863:00 +CPU= 80.528
  22:10:26 STEP  864 H= 864:00 +CPU=130.758
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_35.zip
Uploading the intermediate file: upload_file_35.zip
  22:11:46 STEP  865 H= 865:00 +CPU= 79.746
  22:12:59 STEP  866 H= 866:00 +CPU= 72.364
  22:15:01 STEP  867 H= 867:00 +CPU=120.614
  22:16:27 STEP  868 H= 868:00 +CPU= 85.412
  22:17:40 STEP  869 H= 869:00 +CPU= 72.102
  22:19:35 STEP  870 H= 870:00 +CPU=113.079
  22:20:54 STEP  871 H= 871:00 +CPU= 78.306
  22:22:11 STEP  872 H= 872:00 +CPU= 76.869
  22:24:08 STEP  873 H= 873:00 +CPU=115.401
  22:25:29 STEP  874 H= 874:00 +CPU= 80.278
  22:26:50 STEP  875 H= 875:00 +CPU= 79.961
  22:29:03 STEP  876 H= 876:00 +CPU=131.683
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000876
  22:30:19 STEP  877 H= 877:00 +CPU= 75.588
  22:31:38 STEP  878 H= 878:00 +CPU= 78.115
  22:33:33 STEP  879 H= 879:00 +CPU=113.344
  22:34:49 STEP  880 H= 880:00 +CPU= 75.103
  22:36:07 STEP  881 H= 881:00 +CPU= 77.836
  22:38:08 STEP  882 H= 882:00 +CPU=119.383
  22:39:28 STEP  883 H= 883:00 +CPU= 78.514
  22:40:52 STEP  884 H= 884:00 +CPU= 83.977
  22:42:53 STEP  885 H= 885:00 +CPU=119.191
  22:44:10 STEP  886 H= 886:00 +CPU= 76.555
  22:45:28 STEP  887 H= 887:00 +CPU= 77.118
  22:47:43 STEP  888 H= 888:00 +CPU=132.756
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000876
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_36.zip
Uploading the intermediate file: upload_file_36.zip
  22:49:01 STEP  889 H= 889:00 +CPU= 77.028
  22:50:19 STEP  890 H= 890:00 +CPU= 77.097
  22:52:11 STEP  891 H= 891:00 +CPU=110.708
  22:53:37 STEP  892 H= 892:00 +CPU= 84.656
  22:54:57 STEP  893 H= 893:00 +CPU= 78.917
  22:56:57 STEP  894 H= 894:00 +CPU=118.473
  22:58:15 STEP  895 H= 895:00 +CPU= 77.553
  22:59:35 STEP  896 H= 896:00 +CPU= 78.917
  23:01:34 STEP  897 H= 897:00 +CPU=117.474
  23:03:05 STEP  898 H= 898:00 +CPU= 90.600
  23:04:27 STEP  899 H= 899:00 +CPU= 80.940
  23:06:27 STEP  900 H= 900:00 +CPU=118.168
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000900
  23:07:48 STEP  901 H= 901:00 +CPU= 80.271
  23:09:07 STEP  902 H= 902:00 +CPU= 77.872
  23:11:09 STEP  903 H= 903:00 +CPU=121.276
  23:12:36 STEP  904 H= 904:00 +CPU= 86.250
  23:13:52 STEP  905 H= 905:00 +CPU= 74.424
  23:15:45 STEP  906 H= 906:00 +CPU=112.230
  23:17:05 STEP  907 H= 907:00 +CPU= 78.101
  23:18:25 STEP  908 H= 908:00 +CPU= 79.421
  23:20:20 STEP  909 H= 909:00 +CPU=113.812
  23:21:32 STEP  910 H= 910:00 +CPU= 71.284
  23:22:50 STEP  911 H= 911:00 +CPU= 77.090
  23:25:09 STEP  912 H= 912:00 +CPU=135.851
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000900
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_37.zip
Uploading the intermediate file: upload_file_37.zip
  23:26:24 STEP  913 H= 913:00 +CPU= 74.363
  23:27:39 STEP  914 H= 914:00 +CPU= 74.573
  23:29:41 STEP  915 H= 915:00 +CPU=120.138
  23:31:02 STEP  916 H= 916:00 +CPU= 80.652
  23:32:19 STEP  917 H= 917:00 +CPU= 75.615
  23:34:18 STEP  918 H= 918:00 +CPU=118.385
  23:35:38 STEP  919 H= 919:00 +CPU= 78.894
  23:36:53 STEP  920 H= 920:00 +CPU= 74.160
  23:38:49 STEP  921 H= 921:00 +CPU=114.814
  23:40:10 STEP  922 H= 922:00 +CPU= 80.518
  23:41:28 STEP  923 H= 923:00 +CPU= 76.356
  23:43:23 STEP  924 H= 924:00 +CPU=114.214
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000924
  23:44:45 STEP  925 H= 925:00 +CPU= 80.904
  23:46:03 STEP  926 H= 926:00 +CPU= 77.073
  23:47:54 STEP  927 H= 927:00 +CPU=110.375
  23:49:15 STEP  928 H= 928:00 +CPU= 80.233
  23:50:30 STEP  929 H= 929:00 +CPU= 73.866
  23:52:11 STEP  930 H= 930:00 +CPU=100.359
  23:53:24 STEP  931 H= 931:00 +CPU= 72.243
  23:54:53 STEP  932 H= 932:00 +CPU= 87.513
  23:56:37 STEP  933 H= 933:00 +CPU=102.709
  23:57:56 STEP  934 H= 934:00 +CPU= 78.560
  23:59:18 STEP  935 H= 935:00 +CPU= 81.055
  00:01:27 STEP  936 H= 936:00 +CPU=127.578
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000924
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_38.zip
Uploading the intermediate file: upload_file_38.zip
  00:02:48 STEP  937 H= 937:00 +CPU= 80.144
  00:04:09 STEP  938 H= 938:00 +CPU= 80.251
  00:06:01 STEP  939 H= 939:00 +CPU=110.186
  00:07:16 STEP  940 H= 940:00 +CPU= 75.057
  00:08:42 STEP  941 H= 941:00 +CPU= 84.154
  00:10:53 STEP  942 H= 942:00 +CPU=130.525
  00:12:27 STEP  943 H= 943:00 +CPU= 93.143
  00:13:54 STEP  944 H= 944:00 +CPU= 85.581
  00:15:41 STEP  945 H= 945:00 +CPU=106.264
  00:17:07 STEP  946 H= 946:00 +CPU= 84.780
  00:18:22 STEP  947 H= 947:00 +CPU= 74.340
  00:20:20 STEP  948 H= 948:00 +CPU=116.847
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000948
  00:21:40 STEP  949 H= 949:00 +CPU= 78.593
  00:22:58 STEP  950 H= 950:00 +CPU= 77.384
  00:24:47 STEP  951 H= 951:00 +CPU=107.587
  00:26:03 STEP  952 H= 952:00 +CPU= 75.335
  00:27:22 STEP  953 H= 953:00 +CPU= 78.463
  00:29:34 STEP  954 H= 954:00 +CPU=130.066
  00:30:51 STEP  955 H= 955:00 +CPU= 76.108
  00:32:13 STEP  956 H= 956:00 +CPU= 81.478
  00:34:03 STEP  957 H= 957:00 +CPU=108.509
  00:35:26 STEP  958 H= 958:00 +CPU= 81.808
  00:36:41 STEP  959 H= 959:00 +CPU= 74.653
  00:38:56 STEP  960 H= 960:00 +CPU=132.927
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_39.zip
Uploading the intermediate file: upload_file_39.zip
  00:40:09 STEP  961 H= 961:00 +CPU= 72.598
  00:41:42 STEP  962 H= 962:00 +CPU= 92.173
  00:43:28 STEP  963 H= 963:00 +CPU=104.993
  00:44:44 STEP  964 H= 964:00 +CPU= 75.166
  00:46:05 STEP  965 H= 965:00 +CPU= 80.252
  00:48:03 STEP  966 H= 966:00 +CPU=116.507
  00:49:23 STEP  967 H= 967:00 +CPU= 79.192
  00:50:36 STEP  968 H= 968:00 +CPU= 72.249
  00:52:23 STEP  969 H= 969:00 +CPU=105.863
  00:53:45 STEP  970 H= 970:00 +CPU= 81.791
  00:55:08 STEP  971 H= 971:00 +CPU= 81.643
  00:57:04 STEP  972 H= 972:00 +CPU=114.376
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000972
  00:58:19 STEP  973 H= 973:00 +CPU= 74.887
  00:59:37 STEP  974 H= 974:00 +CPU= 76.848
  01:01:24 STEP  975 H= 975:00 +CPU=106.321
  01:02:36 STEP  976 H= 976:00 +CPU= 71.357
  01:03:54 STEP  977 H= 977:00 +CPU= 76.980
  01:05:44 STEP  978 H= 978:00 +CPU=108.515
  01:06:56 STEP  979 H= 979:00 +CPU= 72.328
  01:08:14 STEP  980 H= 980:00 +CPU= 76.966
  01:10:10 STEP  981 H= 981:00 +CPU=114.395
  01:11:26 STEP  982 H= 982:00 +CPU= 75.578
  01:12:52 STEP  983 H= 983:00 +CPU= 84.828
  01:15:07 STEP  984 H= 984:00 +CPU=133.202
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_40.zip
Uploading the intermediate file: upload_file_40.zip
  01:16:26 STEP  985 H= 985:00 +CPU= 78.744
  01:17:46 STEP  986 H= 986:00 +CPU= 79.024
  01:19:40 STEP  987 H= 987:00 +CPU=112.526
  01:21:02 STEP  988 H= 988:00 +CPU= 81.415
  01:22:23 STEP  989 H= 989:00 +CPU= 80.136
  01:24:17 STEP  990 H= 990:00 +CPU=112.523
  01:25:35 STEP  991 H= 991:00 +CPU= 76.874
  01:26:53 STEP  992 H= 992:00 +CPU= 77.509
  01:29:00 STEP  993 H= 993:00 +CPU=126.002
  01:30:25 STEP  994 H= 994:00 +CPU= 84.525
  01:31:42 STEP  995 H= 995:00 +CPU= 76.066
  01:33:45 STEP  996 H= 996:00 +CPU=121.360
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+000996
  01:35:04 STEP  997 H= 997:00 +CPU= 77.804
  01:36:23 STEP  998 H= 998:00 +CPU= 78.088
  01:38:16 STEP  999 H= 999:00 +CPU=111.952
  01:39:41 STEP 1000 H=1000:00 +CPU= 84.645
  01:41:04 STEP 1001 H=1001:00 +CPU= 82.467
  01:42:55 STEP 1002 H=1002:00 +CPU=109.456
  01:44:13 STEP 1003 H=1003:00 +CPU= 76.998
  01:45:27 STEP 1004 H=1004:00 +CPU= 73.658
  01:47:18 STEP 1005 H=1005:00 +CPU=109.524
  01:48:31 STEP 1006 H=1006:00 +CPU= 73.104
  01:50:04 STEP 1007 H=1007:00 +CPU= 91.483
  01:52:24 STEP 1008 H=1008:00 +CPU=138.463
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+000996
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_41.zip
Uploading the intermediate file: upload_file_41.zip
  01:53:41 STEP 1009 H=1009:00 +CPU= 76.798
  01:55:02 STEP 1010 H=1010:00 +CPU= 80.026
  01:56:56 STEP 1011 H=1011:00 +CPU=112.792
  01:58:13 STEP 1012 H=1012:00 +CPU= 76.045
  01:59:29 STEP 1013 H=1013:00 +CPU= 75.122
  02:01:17 STEP 1014 H=1014:00 +CPU=106.780
  02:02:33 STEP 1015 H=1015:00 +CPU= 75.178
  02:03:48 STEP 1016 H=1016:00 +CPU= 74.934
  02:05:34 STEP 1017 H=1017:00 +CPU=104.642
  02:06:51 STEP 1018 H=1018:00 +CPU= 76.021
  02:08:09 STEP 1019 H=1019:00 +CPU= 77.322
  02:10:14 STEP 1020 H=1020:00 +CPU=124.330
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+001020
  02:11:34 STEP 1021 H=1021:00 +CPU= 79.342
  02:12:52 STEP 1022 H=1022:00 +CPU= 76.556
  02:14:45 STEP 1023 H=1023:00 +CPU=111.447
  02:16:00 STEP 1024 H=1024:00 +CPU= 74.698
  02:17:18 STEP 1025 H=1025:00 +CPU= 77.361
  02:19:08 STEP 1026 H=1026:00 +CPU=108.798
  02:20:21 STEP 1027 H=1027:00 +CPU= 72.359
  02:21:38 STEP 1028 H=1028:00 +CPU= 76.240
  02:23:32 STEP 1029 H=1029:00 +CPU=112.250
  02:24:48 STEP 1030 H=1030:00 +CPU= 75.171
  02:26:07 STEP 1031 H=1031:00 +CPU= 78.201
  02:28:20 STEP 1032 H=1032:00 +CPU=131.615
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168025/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0381_1982050100_123_951_12168025_0_r370780294_42.zip
Uploading the intermediate file: upload_file_42.zip
  02:29:35 STEP 1033 H=1033:00 +CPU= 74.248
  02:30:48 STEP 1034 H=1034:00 +CPU= 71.602
  02:32:31 STEP 1035 H=1035:00 +CPU=102.299
  02:33:44 STEP 1036 H=1036:00 +CPU= 72.424
  02:34:56 STEP 1037 H=1037:00 +CPU= 71.558
  02:36:44 STEP 1038 H=1038:00 +CPU=106.122
  02:37:57 STEP 1039 H=1039:00 +CPU= 72.847
  02:39:19 STEP 1040 H=1040:00 +CPU= 80.604
  02:41:06 STEP 1041 H=1041:00 +CPU=105.705
  02:42:22 STEP 1042 H=1042:00 +CPU= 75.887
  02:43:38 STEP 1043 H=1043:00 +CPU= 75.461
  02:45:53 STEP 1044 H=1044:00 +CPU=132.971
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/28/ICMUAhq0f+001044
  02:47:05 STEP 1045 H=1045:00 +CPU= 71.760
  02:48:20 STEP 1046 H=1046:00 +CPU= 74.567
  02:50:12 STEP 1047 H=1047:00 +CPU=111.005
  02:51:36 STEP 1048 H=1048:00 +CPU= 83.000
  02:52:51 STEP 1049 H=1049:00 +CPU= 74.260
  02:54:42 STEP 1050 H=1050:00 +CPU=110.076
  02:56:02 STEP 1051 H=1051:00 +CPU= 78.797
  02:57:17 STEP 1052 H=1052:00 +CPU= 74.134
  02:59:06 STEP 1053 H=1053:00 +CPU=108.454
  03:00:18 STEP 1054 H=1054:00 +CPU= 70.391
  03:01:40 STEP 1055 H=1055:00 +CPU= 81.922
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:16:16 1537299 22252255 oifs_43r3_ps_0381_1982050100_123_951_12168025_0 2,937,600 45,732 0.0156


©2024 cpdn.org