climateprediction.net home page
Task 22276607

Task 22276607

Name oifs_43r3_ps_0307_2005050100_123_974_12190951_0
Workunit 12190951
Created 23 Dec 2022, 10:14:01 UTC
Sent 6 Jan 2023, 4:46:47 UTC
Report deadline 5 Feb 2023, 4:46:47 UTC
Received 4 Feb 2023, 8:03:27 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1537623
Run time 14 hours 44 min 56 sec
CPU time 14 hours 34 min 55 sec
Validate state Valid
Credit 14,636.16
Device peak FLOPS 7.67 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.43 MB
Peak swap size 4,978.08 MB
Peak disk usage 5,260.96 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
.724
  21:59:26 STEP 2415 H=2415:00 +CPU= 22.084
  21:59:40 STEP 2416 H=2416:00 +CPU= 14.414
  21:59:54 STEP 2417 H=2417:00 +CPU= 13.908
  22:00:17 STEP 2418 H=2418:00 +CPU= 23.021
  22:00:32 STEP 2419 H=2419:00 +CPU= 14.359
  22:00:46 STEP 2420 H=2420:00 +CPU= 14.467
  22:01:08 STEP 2421 H=2421:00 +CPU= 21.968
  22:01:23 STEP 2422 H=2422:00 +CPU= 14.261
  22:01:37 STEP 2423 H=2423:00 +CPU= 14.374
  22:02:04 STEP 2424 H=2424:00 +CPU= 26.435
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002424
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002424
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002424
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002400
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002400
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002400
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002412
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002412
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002412
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_100.zip
Uploading the intermediate file: upload_file_100.zip
  22:02:18 STEP 2425 H=2425:00 +CPU= 13.881
  22:02:32 STEP 2426 H=2426:00 +CPU= 14.463
  22:02:55 STEP 2427 H=2427:00 +CPU= 22.048
  22:03:09 STEP 2428 H=2428:00 +CPU= 14.385
  22:03:23 STEP 2429 H=2429:00 +CPU= 14.049
  22:03:46 STEP 2430 H=2430:00 +CPU= 22.609
  22:04:00 STEP 2431 H=2431:00 +CPU= 13.923
  22:04:15 STEP 2432 H=2432:00 +CPU= 14.463
  22:04:37 STEP 2433 H=2433:00 +CPU= 21.996
  22:04:52 STEP 2434 H=2434:00 +CPU= 14.363
  22:05:06 STEP 2435 H=2435:00 +CPU= 14.109
  22:05:31 STEP 2436 H=2436:00 +CPU= 24.679
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002436
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002436
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002436
  22:05:45 STEP 2437 H=2437:00 +CPU= 14.042
  22:06:00 STEP 2438 H=2438:00 +CPU= 14.625
  22:06:22 STEP 2439 H=2439:00 +CPU= 22.000
  22:06:36 STEP 2440 H=2440:00 +CPU= 14.252
  22:06:50 STEP 2441 H=2441:00 +CPU= 13.805
  22:07:13 STEP 2442 H=2442:00 +CPU= 22.469
  22:07:27 STEP 2443 H=2443:00 +CPU= 13.796
  22:07:42 STEP 2444 H=2444:00 +CPU= 14.870
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2005050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0307
(argv4) batchid: 974
(argv5) wuid: 12190951
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /home/galfridian/slots/1
Project directory is: /home/galfridian/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951
..mkdir for temp folder for results failed
Copying: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/galfridian/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/galfridian/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_407c2d91c9548b9def06137b301ba480 to: /home/galfridian/slots/1/oifs_43r3_ps_0307_2005050100_123_974_12190951.zip
Unzipping the namelist zip file: /home/galfridian/slots/1/oifs_43r3_ps_0307_2005050100_123_974_12190951.zip
ic_ancil_file: ic_ancil_12190951
ifsdata_file: ifsdata_12190951
climate_data_file: clim_data_12190951
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0027c79ee60ce2eee0edd87dea46b016 to: /home/galfridian/slots/1/ic_ancil_12190951.zip
Unzipping the IC ancils zip file: /home/galfridian/slots/1/ic_ancil_12190951.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_27cd7d41071d5cdfdefd13520fb28d18 to: /home/galfridian/slots/1/ifsdata/ifsdata_12190951.zip
Unzipping the ifsdata_zip file: /home/galfridian/slots/1/ifsdata/ifsdata_12190951.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/galfridian/slots/1/159l_2/clim_data_12190951.zip
Unzipping the climate data zip file: /home/galfridian/slots/1/159l_2/clim_data_12190951.zip
Checking for progress XML file: /home/galfridian/slots/1/progress_file_12190951.xml
Opened progress file ok : /home/galfridian/slots/1/progress_file_12190951.xml
-- Model is restarting --
Adjusting last_iter, 2445, to previous model restart step.
Creating progress file: /home/galfridian/slots/1/progress_file_12190951.xml
last_cpu_time: 43490
upload_file_number: 101
last_iter: 2423
last_upload: 8726400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686
The child process has been launched with process id: 3252
Executing the command: /home/galfridian/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:galfridian-systemproductname:1:1:3252:3252] [20230204:060125:1675486885:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  06:01:57 STEP 2424 H=2424:00 +CPU= 25.099
  06:02:11 STEP 2425 H=2425:00 +CPU= 13.848
  06:02:26 STEP 2426 H=2426:00 +CPU= 14.496
  06:02:48 STEP 2427 H=2427:00 +CPU= 22.026
  06:03:02 STEP 2428 H=2428:00 +CPU= 14.417
  06:03:17 STEP 2429 H=2429:00 +CPU= 14.082
  06:03:39 STEP 2430 H=2430:00 +CPU= 22.485
  06:03:53 STEP 2431 H=2431:00 +CPU= 13.869
  06:04:08 STEP 2432 H=2432:00 +CPU= 14.298
  06:04:30 STEP 2433 H=2433:00 +CPU= 22.056
  06:04:44 STEP 2434 H=2434:00 +CPU= 14.378
  06:04:58 STEP 2435 H=2435:00 +CPU= 13.819
  06:05:23 STEP 2436 H=2436:00 +CPU= 24.653
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002436
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002436
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002436
  06:05:37 STEP 2437 H=2437:00 +CPU= 14.115
  06:05:52 STEP 2438 H=2438:00 +CPU= 14.638
  06:06:14 STEP 2439 H=2439:00 +CPU= 22.057
  06:06:29 STEP 2440 H=2440:00 +CPU= 14.528
  06:06:43 STEP 2441 H=2441:00 +CPU= 13.967
  06:07:06 STEP 2442 H=2442:00 +CPU= 22.484
  06:07:20 STEP 2443 H=2443:00 +CPU= 14.044
  06:07:35 STEP 2444 H=2444:00 +CPU= 14.268
  06:07:58 STEP 2445 H=2445:00 +CPU= 21.859
  06:08:12 STEP 2446 H=2446:00 +CPU= 14.284
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:27:34 STEP 2447 H=2447:00 +CPU= 14.142
  06:28:01 STEP 2448 H=2448:00 +CPU= 26.188
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002448
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002448
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002448
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002424
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002424
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002424
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002436
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002436
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002436
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_101.zip
Uploading the intermediate file: upload_file_101.zip
  06:28:15 STEP 2449 H=2449:00 +CPU= 13.666
  06:28:29 STEP 2450 H=2450:00 +CPU= 14.265
  06:28:51 STEP 2451 H=2451:00 +CPU= 21.753
  06:29:05 STEP 2452 H=2452:00 +CPU= 14.272
  06:29:19 STEP 2453 H=2453:00 +CPU= 13.935
  06:29:42 STEP 2454 H=2454:00 +CPU= 22.665
  06:29:56 STEP 2455 H=2455:00 +CPU= 13.874
  06:30:11 STEP 2456 H=2456:00 +CPU= 14.181
  06:30:33 STEP 2457 H=2457:00 +CPU= 21.819
  06:30:47 STEP 2458 H=2458:00 +CPU= 14.175
  06:31:01 STEP 2459 H=2459:00 +CPU= 13.759
  06:31:25 STEP 2460 H=2460:00 +CPU= 24.470
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002460
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002460
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002460
  06:31:39 STEP 2461 H=2461:00 +CPU= 13.784
  06:31:53 STEP 2462 H=2462:00 +CPU= 14.242
  06:32:15 STEP 2463 H=2463:00 +CPU= 21.828
  06:32:30 STEP 2464 H=2464:00 +CPU= 14.239
  06:32:44 STEP 2465 H=2465:00 +CPU= 13.879
  06:33:07 STEP 2466 H=2466:00 +CPU= 22.697
  06:33:21 STEP 2467 H=2467:00 +CPU= 13.914
  06:33:35 STEP 2468 H=2468:00 +CPU= 14.388
  06:33:57 STEP 2469 H=2469:00 +CPU= 21.810
  06:34:11 STEP 2470 H=2470:00 +CPU= 14.279
  06:34:26 STEP 2471 H=2471:00 +CPU= 14.279
  06:34:52 STEP 2472 H=2472:00 +CPU= 26.646
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002472
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002472
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002472
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002448
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002448
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002448
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002460
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002460
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002460
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_102.zip
Uploading the intermediate file: upload_file_102.zip
  06:35:07 STEP 2473 H=2473:00 +CPU= 13.874
  06:35:21 STEP 2474 H=2474:00 +CPU= 14.359
  06:35:43 STEP 2475 H=2475:00 +CPU= 22.084
  06:35:58 STEP 2476 H=2476:00 +CPU= 14.344
  06:36:12 STEP 2477 H=2477:00 +CPU= 13.940
  06:36:34 STEP 2478 H=2478:00 +CPU= 22.755
  06:36:49 STEP 2479 H=2479:00 +CPU= 14.004
  06:37:03 STEP 2480 H=2480:00 +CPU= 14.470
  06:37:25 STEP 2481 H=2481:00 +CPU= 21.852
  06:37:40 STEP 2482 H=2482:00 +CPU= 14.501
  06:37:54 STEP 2483 H=2483:00 +CPU= 14.008
  06:38:18 STEP 2484 H=2484:00 +CPU= 24.474
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002484
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002484
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002484
  06:38:32 STEP 2485 H=2485:00 +CPU= 13.767
  06:38:47 STEP 2486 H=2486:00 +CPU= 14.347
  06:39:09 STEP 2487 H=2487:00 +CPU= 22.199
  06:39:23 STEP 2488 H=2488:00 +CPU= 14.501
  06:39:37 STEP 2489 H=2489:00 +CPU= 14.037
  06:40:00 STEP 2490 H=2490:00 +CPU= 22.667
  06:40:14 STEP 2491 H=2491:00 +CPU= 13.868
  06:40:28 STEP 2492 H=2492:00 +CPU= 14.307
  06:40:50 STEP 2493 H=2493:00 +CPU= 21.943
  06:41:05 STEP 2494 H=2494:00 +CPU= 14.296
  06:41:19 STEP 2495 H=2495:00 +CPU= 14.390
  06:41:46 STEP 2496 H=2496:00 +CPU= 26.213
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002496
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002496
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002496
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002472
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002472
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002472
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002484
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002484
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002484
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_103.zip
Uploading the intermediate file: upload_file_103.zip
  06:42:00 STEP 2497 H=2497:00 +CPU= 13.906
  06:42:14 STEP 2498 H=2498:00 +CPU= 14.372
  06:42:37 STEP 2499 H=2499:00 +CPU= 22.225
  06:42:51 STEP 2500 H=2500:00 +CPU= 14.374
  06:43:05 STEP 2501 H=2501:00 +CPU= 13.930
  06:43:28 STEP 2502 H=2502:00 +CPU= 22.563
  06:43:42 STEP 2503 H=2503:00 +CPU= 13.830
  06:43:56 STEP 2504 H=2504:00 +CPU= 14.449
  06:44:18 STEP 2505 H=2505:00 +CPU= 21.915
  06:44:33 STEP 2506 H=2506:00 +CPU= 14.218
  06:44:46 STEP 2507 H=2507:00 +CPU= 13.907
  06:45:11 STEP 2508 H=2508:00 +CPU= 24.644
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002508
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002508
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002508
  06:45:25 STEP 2509 H=2509:00 +CPU= 14.199
  06:45:40 STEP 2510 H=2510:00 +CPU= 14.428
  06:46:02 STEP 2511 H=2511:00 +CPU= 21.748
  06:46:16 STEP 2512 H=2512:00 +CPU= 14.424
  06:46:30 STEP 2513 H=2513:00 +CPU= 13.799
  06:46:51 STEP 2514 H=2514:00 +CPU= 21.181
  06:47:04 STEP 2515 H=2515:00 +CPU= 12.269
  06:47:17 STEP 2516 H=2516:00 +CPU= 13.750
  06:47:40 STEP 2517 H=2517:00 +CPU= 22.161
  06:47:54 STEP 2518 H=2518:00 +CPU= 14.358
  06:48:09 STEP 2519 H=2519:00 +CPU= 14.559
  06:48:36 STEP 2520 H=2520:00 +CPU= 26.385
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002520
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002520
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002520
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002496
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002496
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002496
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002508
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002508
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002508
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_104.zip
Uploading the intermediate file: upload_file_104.zip
  06:48:50 STEP 2521 H=2521:00 +CPU= 13.958
  06:49:04 STEP 2522 H=2522:00 +CPU= 14.330
  06:49:27 STEP 2523 H=2523:00 +CPU= 21.991
  06:49:41 STEP 2524 H=2524:00 +CPU= 14.309
  06:49:55 STEP 2525 H=2525:00 +CPU= 13.802
  06:50:18 STEP 2526 H=2526:00 +CPU= 22.749
  06:50:32 STEP 2527 H=2527:00 +CPU= 13.937
  06:50:46 STEP 2528 H=2528:00 +CPU= 14.795
  06:51:09 STEP 2529 H=2529:00 +CPU= 21.994
  06:51:23 STEP 2530 H=2530:00 +CPU= 14.393
  06:51:37 STEP 2531 H=2531:00 +CPU= 13.969
  06:52:02 STEP 2532 H=2532:00 +CPU= 24.990
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002532
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002532
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002532
  06:52:17 STEP 2533 H=2533:00 +CPU= 14.069
  06:52:31 STEP 2534 H=2534:00 +CPU= 14.425
  06:52:53 STEP 2535 H=2535:00 +CPU= 22.002
  06:53:08 STEP 2536 H=2536:00 +CPU= 14.238
  06:53:22 STEP 2537 H=2537:00 +CPU= 13.768
  06:53:44 STEP 2538 H=2538:00 +CPU= 22.585
  06:53:58 STEP 2539 H=2539:00 +CPU= 13.893
  06:54:13 STEP 2540 H=2540:00 +CPU= 14.305
  06:54:35 STEP 2541 H=2541:00 +CPU= 21.988
  06:54:49 STEP 2542 H=2542:00 +CPU= 14.445
  06:55:04 STEP 2543 H=2543:00 +CPU= 14.263
  06:55:30 STEP 2544 H=2544:00 +CPU= 26.287
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002544
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002544
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002544
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002520
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002520
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002520
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002532
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002532
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002532
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_105.zip
Uploading the intermediate file: upload_file_105.zip
  06:55:44 STEP 2545 H=2545:00 +CPU= 13.728
  06:55:58 STEP 2546 H=2546:00 +CPU= 14.618
  06:56:21 STEP 2547 H=2547:00 +CPU= 22.155
  06:56:35 STEP 2548 H=2548:00 +CPU= 14.376
  06:56:49 STEP 2549 H=2549:00 +CPU= 14.142
  06:57:12 STEP 2550 H=2550:00 +CPU= 22.557
  06:57:26 STEP 2551 H=2551:00 +CPU= 14.106
  06:57:41 STEP 2552 H=2552:00 +CPU= 14.540
  06:58:03 STEP 2553 H=2553:00 +CPU= 22.090
  06:58:17 STEP 2554 H=2554:00 +CPU= 14.283
  06:58:31 STEP 2555 H=2555:00 +CPU= 13.879
  06:58:56 STEP 2556 H=2556:00 +CPU= 24.895
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002556
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002556
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002556
  06:59:10 STEP 2557 H=2557:00 +CPU= 13.851
  06:59:25 STEP 2558 H=2558:00 +CPU= 14.291
  06:59:47 STEP 2559 H=2559:00 +CPU= 21.745
  07:00:01 STEP 2560 H=2560:00 +CPU= 14.222
  07:00:15 STEP 2561 H=2561:00 +CPU= 14.014
  07:00:38 STEP 2562 H=2562:00 +CPU= 22.626
  07:00:52 STEP 2563 H=2563:00 +CPU= 13.794
  07:01:06 STEP 2564 H=2564:00 +CPU= 14.316
  07:01:28 STEP 2565 H=2565:00 +CPU= 22.107
  07:01:43 STEP 2566 H=2566:00 +CPU= 14.317
  07:01:57 STEP 2567 H=2567:00 +CPU= 14.204
  07:02:24 STEP 2568 H=2568:00 +CPU= 26.689
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002568
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002568
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002568
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002544
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002544
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002544
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002556
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002556
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002556
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_106.zip
Uploading the intermediate file: upload_file_106.zip
  07:02:38 STEP 2569 H=2569:00 +CPU= 13.838
  07:02:52 STEP 2570 H=2570:00 +CPU= 14.629
  07:03:14 STEP 2571 H=2571:00 +CPU= 21.931
  07:03:29 STEP 2572 H=2572:00 +CPU= 14.309
  07:03:43 STEP 2573 H=2573:00 +CPU= 14.010
  07:04:06 STEP 2574 H=2574:00 +CPU= 22.521
  07:04:20 STEP 2575 H=2575:00 +CPU= 13.898
  07:04:34 STEP 2576 H=2576:00 +CPU= 14.264
  07:04:56 STEP 2577 H=2577:00 +CPU= 21.850
  07:05:11 STEP 2578 H=2578:00 +CPU= 14.508
  07:05:25 STEP 2579 H=2579:00 +CPU= 13.959
  07:05:50 STEP 2580 H=2580:00 +CPU= 24.699
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002580
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002580
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002580
  07:06:05 STEP 2581 H=2581:00 +CPU= 13.917
  07:06:19 STEP 2582 H=2582:00 +CPU= 14.405
  07:06:42 STEP 2583 H=2583:00 +CPU= 22.046
  07:06:56 STEP 2584 H=2584:00 +CPU= 14.669
  07:07:11 STEP 2585 H=2585:00 +CPU= 14.446
  07:07:33 STEP 2586 H=2586:00 +CPU= 22.473
  07:07:47 STEP 2587 H=2587:00 +CPU= 13.927
  07:08:02 STEP 2588 H=2588:00 +CPU= 14.347
  07:08:24 STEP 2589 H=2589:00 +CPU= 21.906
  07:08:38 STEP 2590 H=2590:00 +CPU= 14.328
  07:08:53 STEP 2591 H=2591:00 +CPU= 14.306
  07:09:20 STEP 2592 H=2592:00 +CPU= 26.484
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002592
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002592
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002592
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002568
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002568
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002568
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002580
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002580
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002580
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_107.zip
Uploading the intermediate file: upload_file_107.zip
  07:09:34 STEP 2593 H=2593:00 +CPU= 13.868
  07:09:48 STEP 2594 H=2594:00 +CPU= 14.337
  07:10:10 STEP 2595 H=2595:00 +CPU= 22.134
  07:10:25 STEP 2596 H=2596:00 +CPU= 14.372
  07:10:39 STEP 2597 H=2597:00 +CPU= 14.038
  07:11:01 STEP 2598 H=2598:00 +CPU= 22.449
  07:11:15 STEP 2599 H=2599:00 +CPU= 13.916
  07:11:30 STEP 2600 H=2600:00 +CPU= 14.568
  07:11:52 STEP 2601 H=2601:00 +CPU= 22.010
  07:12:07 STEP 2602 H=2602:00 +CPU= 14.530
  07:12:21 STEP 2603 H=2603:00 +CPU= 13.935
  07:12:46 STEP 2604 H=2604:00 +CPU= 24.781
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002604
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002604
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002604
  07:13:00 STEP 2605 H=2605:00 +CPU= 14.071
  07:13:14 STEP 2606 H=2606:00 +CPU= 14.260
  07:13:37 STEP 2607 H=2607:00 +CPU= 21.962
  07:13:51 STEP 2608 H=2608:00 +CPU= 14.380
  07:14:05 STEP 2609 H=2609:00 +CPU= 13.861
  07:14:27 STEP 2610 H=2610:00 +CPU= 22.391
  07:14:41 STEP 2611 H=2611:00 +CPU= 13.835
  07:14:56 STEP 2612 H=2612:00 +CPU= 14.571
  07:15:18 STEP 2613 H=2613:00 +CPU= 21.767
  07:15:32 STEP 2614 H=2614:00 +CPU= 14.197
  07:15:46 STEP 2615 H=2615:00 +CPU= 14.252
  07:16:13 STEP 2616 H=2616:00 +CPU= 26.444
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002616
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002616
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002616
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002592
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002592
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002592
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002604
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002604
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002604
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_108.zip
Uploading the intermediate file: upload_file_108.zip
  07:16:26 STEP 2617 H=2617:00 +CPU= 13.464
  07:16:40 STEP 2618 H=2618:00 +CPU= 14.163
  07:17:02 STEP 2619 H=2619:00 +CPU= 21.492
  07:17:16 STEP 2620 H=2620:00 +CPU= 14.187
  07:17:30 STEP 2621 H=2621:00 +CPU= 13.563
  07:17:52 STEP 2622 H=2622:00 +CPU= 22.162
  07:18:05 STEP 2623 H=2623:00 +CPU= 13.579
  07:18:20 STEP 2624 H=2624:00 +CPU= 14.095
  07:18:42 STEP 2625 H=2625:00 +CPU= 21.835
  07:18:56 STEP 2626 H=2626:00 +CPU= 14.050
  07:19:10 STEP 2627 H=2627:00 +CPU= 13.890
  07:19:34 STEP 2628 H=2628:00 +CPU= 24.360
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002628
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002628
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002628
  07:19:48 STEP 2629 H=2629:00 +CPU= 13.868
  07:20:03 STEP 2630 H=2630:00 +CPU= 14.247
  07:20:25 STEP 2631 H=2631:00 +CPU= 21.836
  07:20:39 STEP 2632 H=2632:00 +CPU= 14.358
  07:20:53 STEP 2633 H=2633:00 +CPU= 13.882
  07:21:15 STEP 2634 H=2634:00 +CPU= 22.302
  07:21:29 STEP 2635 H=2635:00 +CPU= 13.897
  07:21:44 STEP 2636 H=2636:00 +CPU= 14.337
  07:22:06 STEP 2637 H=2637:00 +CPU= 21.892
  07:22:20 STEP 2638 H=2638:00 +CPU= 14.237
  07:22:35 STEP 2639 H=2639:00 +CPU= 14.229
  07:23:01 STEP 2640 H=2640:00 +CPU= 26.340
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002640
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002640
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002640
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002616
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002616
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002616
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002628
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002628
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002628
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_109.zip
Uploading the intermediate file: upload_file_109.zip
  07:23:15 STEP 2641 H=2641:00 +CPU= 13.823
  07:23:30 STEP 2642 H=2642:00 +CPU= 14.299
  07:23:52 STEP 2643 H=2643:00 +CPU= 21.880
  07:24:06 STEP 2644 H=2644:00 +CPU= 14.263
  07:24:20 STEP 2645 H=2645:00 +CPU= 13.849
  07:24:43 STEP 2646 H=2646:00 +CPU= 22.361
  07:24:57 STEP 2647 H=2647:00 +CPU= 13.779
  07:25:11 STEP 2648 H=2648:00 +CPU= 14.338
  07:25:33 STEP 2649 H=2649:00 +CPU= 22.000
  07:25:48 STEP 2650 H=2650:00 +CPU= 14.286
  07:26:01 STEP 2651 H=2651:00 +CPU= 13.892
  07:26:26 STEP 2652 H=2652:00 +CPU= 24.569
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002652
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002652
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002652
  07:26:40 STEP 2653 H=2653:00 +CPU= 13.784
  07:26:55 STEP 2654 H=2654:00 +CPU= 14.281
  07:27:17 STEP 2655 H=2655:00 +CPU= 21.810
  07:27:31 STEP 2656 H=2656:00 +CPU= 14.303
  07:27:45 STEP 2657 H=2657:00 +CPU= 13.688
  07:28:07 STEP 2658 H=2658:00 +CPU= 22.156
  07:28:21 STEP 2659 H=2659:00 +CPU= 13.805
  07:28:36 STEP 2660 H=2660:00 +CPU= 14.265
  07:28:57 STEP 2661 H=2661:00 +CPU= 21.553
  07:29:11 STEP 2662 H=2662:00 +CPU= 14.166
  07:29:26 STEP 2663 H=2663:00 +CPU= 14.135
  07:29:52 STEP 2664 H=2664:00 +CPU= 26.517
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002664
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002664
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002664
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002640
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002640
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002640
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002652
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002652
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002652
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_110.zip
Uploading the intermediate file: upload_file_110.zip
  07:30:06 STEP 2665 H=2665:00 +CPU= 13.743
Uploading trickle at timestep: 9590400
  07:30:20 STEP 2666 H=2666:00 +CPU= 14.201
  07:30:42 STEP 2667 H=2667:00 +CPU= 21.890
  07:30:57 STEP 2668 H=2668:00 +CPU= 14.234
  07:31:10 STEP 2669 H=2669:00 +CPU= 13.831
  07:31:33 STEP 2670 H=2670:00 +CPU= 22.345
  07:31:47 STEP 2671 H=2671:00 +CPU= 13.796
  07:32:01 STEP 2672 H=2672:00 +CPU= 14.241
  07:32:23 STEP 2673 H=2673:00 +CPU= 21.869
  07:32:38 STEP 2674 H=2674:00 +CPU= 14.267
  07:32:52 STEP 2675 H=2675:00 +CPU= 13.818
  07:33:16 STEP 2676 H=2676:00 +CPU= 24.458
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002676
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002676
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002676
  07:33:30 STEP 2677 H=2677:00 +CPU= 13.753
  07:33:44 STEP 2678 H=2678:00 +CPU= 14.386
  07:34:06 STEP 2679 H=2679:00 +CPU= 21.866
  07:34:21 STEP 2680 H=2680:00 +CPU= 14.355
  07:34:35 STEP 2681 H=2681:00 +CPU= 13.848
  07:34:58 STEP 2682 H=2682:00 +CPU= 22.685
  07:35:12 STEP 2683 H=2683:00 +CPU= 13.783
  07:35:26 STEP 2684 H=2684:00 +CPU= 14.289
  07:35:48 STEP 2685 H=2685:00 +CPU= 21.873
  07:36:02 STEP 2686 H=2686:00 +CPU= 14.196
  07:36:17 STEP 2687 H=2687:00 +CPU= 14.184
  07:36:43 STEP 2688 H=2688:00 +CPU= 26.402
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002688
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002688
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002688
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002664
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002664
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002664
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002676
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002676
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002676
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_111.zip
Uploading the intermediate file: upload_file_111.zip
  07:36:57 STEP 2689 H=2689:00 +CPU= 13.791
  07:37:12 STEP 2690 H=2690:00 +CPU= 14.146
  07:37:33 STEP 2691 H=2691:00 +CPU= 21.823
  07:37:48 STEP 2692 H=2692:00 +CPU= 14.333
  07:38:02 STEP 2693 H=2693:00 +CPU= 13.759
  07:38:24 STEP 2694 H=2694:00 +CPU= 22.277
  07:38:38 STEP 2695 H=2695:00 +CPU= 13.752
  07:38:52 STEP 2696 H=2696:00 +CPU= 14.328
  07:39:14 STEP 2697 H=2697:00 +CPU= 21.953
  07:39:29 STEP 2698 H=2698:00 +CPU= 14.358
  07:39:43 STEP 2699 H=2699:00 +CPU= 13.798
  07:40:07 STEP 2700 H=2700:00 +CPU= 24.465
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002700
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002700
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002700
  07:40:21 STEP 2701 H=2701:00 +CPU= 13.734
  07:40:35 STEP 2702 H=2702:00 +CPU= 14.235
  07:40:57 STEP 2703 H=2703:00 +CPU= 21.731
  07:41:11 STEP 2704 H=2704:00 +CPU= 14.206
  07:41:26 STEP 2705 H=2705:00 +CPU= 14.161
  07:41:48 STEP 2706 H=2706:00 +CPU= 22.249
  07:42:02 STEP 2707 H=2707:00 +CPU= 13.825
  07:42:16 STEP 2708 H=2708:00 +CPU= 14.449
  07:42:38 STEP 2709 H=2709:00 +CPU= 21.731
  07:42:53 STEP 2710 H=2710:00 +CPU= 14.172
  07:43:07 STEP 2711 H=2711:00 +CPU= 14.213
  07:43:33 STEP 2712 H=2712:00 +CPU= 26.288
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002712
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002712
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002712
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002688
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002688
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002688
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002700
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002700
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002700
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_112.zip
Uploading the intermediate file: upload_file_112.zip
  07:43:47 STEP 2713 H=2713:00 +CPU= 13.830
  07:44:01 STEP 2714 H=2714:00 +CPU= 14.210
  07:44:24 STEP 2715 H=2715:00 +CPU= 22.131
  07:44:38 STEP 2716 H=2716:00 +CPU= 14.243
  07:44:52 STEP 2717 H=2717:00 +CPU= 13.788
  07:45:14 STEP 2718 H=2718:00 +CPU= 22.538
  07:45:28 STEP 2719 H=2719:00 +CPU= 13.762
  07:45:43 STEP 2720 H=2720:00 +CPU= 14.293
  07:46:04 STEP 2721 H=2721:00 +CPU= 21.802
  07:46:19 STEP 2722 H=2722:00 +CPU= 14.313
  07:46:33 STEP 2723 H=2723:00 +CPU= 13.820
  07:46:58 STEP 2724 H=2724:00 +CPU= 24.558
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002724
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002724
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002724
  07:47:11 STEP 2725 H=2725:00 +CPU= 13.739
  07:47:26 STEP 2726 H=2726:00 +CPU= 14.201
  07:47:48 STEP 2727 H=2727:00 +CPU= 21.755
  07:48:02 STEP 2728 H=2728:00 +CPU= 14.267
  07:48:16 STEP 2729 H=2729:00 +CPU= 13.906
  07:48:38 STEP 2730 H=2730:00 +CPU= 22.338
  07:48:52 STEP 2731 H=2731:00 +CPU= 13.816
  07:49:07 STEP 2732 H=2732:00 +CPU= 14.247
  07:49:29 STEP 2733 H=2733:00 +CPU= 21.909
  07:49:43 STEP 2734 H=2734:00 +CPU= 14.207
  07:49:57 STEP 2735 H=2735:00 +CPU= 14.196
  07:50:24 STEP 2736 H=2736:00 +CPU= 26.397
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002736
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002736
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002736
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002712
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002712
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002712
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002724
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002724
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002724
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_113.zip
Uploading the intermediate file: upload_file_113.zip
  07:50:38 STEP 2737 H=2737:00 +CPU= 13.812
  07:50:52 STEP 2738 H=2738:00 +CPU= 14.224
  07:51:14 STEP 2739 H=2739:00 +CPU= 22.159
  07:51:29 STEP 2740 H=2740:00 +CPU= 14.286
  07:51:43 STEP 2741 H=2741:00 +CPU= 13.845
  07:52:05 STEP 2742 H=2742:00 +CPU= 22.424
  07:52:19 STEP 2743 H=2743:00 +CPU= 13.810
  07:52:33 STEP 2744 H=2744:00 +CPU= 14.296
  07:52:55 STEP 2745 H=2745:00 +CPU= 21.941
  07:53:10 STEP 2746 H=2746:00 +CPU= 14.355
  07:53:24 STEP 2747 H=2747:00 +CPU= 13.918
  07:53:49 STEP 2748 H=2748:00 +CPU= 24.675
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002748
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002748
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002748
  07:54:02 STEP 2749 H=2749:00 +CPU= 13.829
  07:54:17 STEP 2750 H=2750:00 +CPU= 14.351
  07:54:39 STEP 2751 H=2751:00 +CPU= 22.092
  07:54:53 STEP 2752 H=2752:00 +CPU= 14.213
  07:55:07 STEP 2753 H=2753:00 +CPU= 13.767
  07:55:30 STEP 2754 H=2754:00 +CPU= 22.419
  07:55:44 STEP 2755 H=2755:00 +CPU= 13.839
  07:55:58 STEP 2756 H=2756:00 +CPU= 14.270
  07:56:20 STEP 2757 H=2757:00 +CPU= 21.828
  07:56:35 STEP 2758 H=2758:00 +CPU= 14.463
  07:56:49 STEP 2759 H=2759:00 +CPU= 14.157
  07:57:16 STEP 2760 H=2760:00 +CPU= 26.452
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002760
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002760
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002760
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002736
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002736
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002736
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002748
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002748
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002748
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_114.zip
Uploading the intermediate file: upload_file_114.zip
  07:57:30 STEP 2761 H=2761:00 +CPU= 13.760
  07:57:44 STEP 2762 H=2762:00 +CPU= 14.158
  07:58:06 STEP 2763 H=2763:00 +CPU= 21.836
  07:58:20 STEP 2764 H=2764:00 +CPU= 14.151
  07:58:34 STEP 2765 H=2765:00 +CPU= 13.729
  07:58:56 STEP 2766 H=2766:00 +CPU= 22.249
  07:59:10 STEP 2767 H=2767:00 +CPU= 13.737
  07:59:24 STEP 2768 H=2768:00 +CPU= 14.211
  07:59:46 STEP 2769 H=2769:00 +CPU= 21.824
  08:00:01 STEP 2770 H=2770:00 +CPU= 14.338
  08:00:15 STEP 2771 H=2771:00 +CPU= 13.840
  08:00:40 STEP 2772 H=2772:00 +CPU= 24.401
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002772
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002772
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002772
  08:00:53 STEP 2773 H=2773:00 +CPU= 13.714
  08:01:08 STEP 2774 H=2774:00 +CPU= 14.286
  08:01:30 STEP 2775 H=2775:00 +CPU= 21.958
  08:01:44 STEP 2776 H=2776:00 +CPU= 14.344
  08:01:58 STEP 2777 H=2777:00 +CPU= 13.767
  08:02:21 STEP 2778 H=2778:00 +CPU= 22.358
  08:02:34 STEP 2779 H=2779:00 +CPU= 13.801
  08:02:49 STEP 2780 H=2780:00 +CPU= 14.199
  08:03:11 STEP 2781 H=2781:00 +CPU= 21.714
  08:03:25 STEP 2782 H=2782:00 +CPU= 14.152
  08:03:39 STEP 2783 H=2783:00 +CPU= 14.181
  08:04:06 STEP 2784 H=2784:00 +CPU= 26.448
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002784
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002784
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002784
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002760
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002760
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002760
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002772
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002772
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002772
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_115.zip
Uploading the intermediate file: upload_file_115.zip
  08:04:20 STEP 2785 H=2785:00 +CPU= 13.800
  08:04:34 STEP 2786 H=2786:00 +CPU= 14.311
  08:04:56 STEP 2787 H=2787:00 +CPU= 21.818
  08:05:10 STEP 2788 H=2788:00 +CPU= 14.382
  08:05:24 STEP 2789 H=2789:00 +CPU= 13.940
  08:05:47 STEP 2790 H=2790:00 +CPU= 22.458
  08:06:01 STEP 2791 H=2791:00 +CPU= 13.845
  08:06:15 STEP 2792 H=2792:00 +CPU= 14.306
  08:06:37 STEP 2793 H=2793:00 +CPU= 21.910
  08:06:52 STEP 2794 H=2794:00 +CPU= 14.386
  08:07:05 STEP 2795 H=2795:00 +CPU= 13.922
  08:07:30 STEP 2796 H=2796:00 +CPU= 24.480
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002796
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002796
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002796
  08:07:44 STEP 2797 H=2797:00 +CPU= 13.818
  08:07:58 STEP 2798 H=2798:00 +CPU= 14.330
  08:08:21 STEP 2799 H=2799:00 +CPU= 22.098
  08:08:35 STEP 2800 H=2800:00 +CPU= 14.240
  08:08:49 STEP 2801 H=2801:00 +CPU= 13.852
  08:09:11 STEP 2802 H=2802:00 +CPU= 22.263
  08:09:25 STEP 2803 H=2803:00 +CPU= 13.748
  08:09:39 STEP 2804 H=2804:00 +CPU= 14.209
  08:10:01 STEP 2805 H=2805:00 +CPU= 21.587
  08:10:15 STEP 2806 H=2806:00 +CPU= 14.168
  08:10:30 STEP 2807 H=2807:00 +CPU= 14.431
  08:10:56 STEP 2808 H=2808:00 +CPU= 26.248
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002808
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002808
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002808
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002784
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002784
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002784
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002796
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002796
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002796
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_116.zip
Uploading the intermediate file: upload_file_116.zip
  08:11:09 STEP 2809 H=2809:00 +CPU= 13.137
  08:11:22 STEP 2810 H=2810:00 +CPU= 13.082
  08:11:44 STEP 2811 H=2811:00 +CPU= 21.556
  08:11:58 STEP 2812 H=2812:00 +CPU= 13.815
  08:12:11 STEP 2813 H=2813:00 +CPU= 13.439
  08:12:33 STEP 2814 H=2814:00 +CPU= 22.078
  08:12:47 STEP 2815 H=2815:00 +CPU= 13.703
  08:13:01 STEP 2816 H=2816:00 +CPU= 14.121
  08:13:23 STEP 2817 H=2817:00 +CPU= 21.625
  08:13:37 STEP 2818 H=2818:00 +CPU= 14.144
  08:13:51 STEP 2819 H=2819:00 +CPU= 13.721
  08:14:15 STEP 2820 H=2820:00 +CPU= 24.208
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002820
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002820
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002820
  08:14:29 STEP 2821 H=2821:00 +CPU= 13.645
  08:14:43 STEP 2822 H=2822:00 +CPU= 14.229
  08:15:05 STEP 2823 H=2823:00 +CPU= 21.719
  08:15:19 STEP 2824 H=2824:00 +CPU= 14.185
  08:15:33 STEP 2825 H=2825:00 +CPU= 13.733
  08:15:55 STEP 2826 H=2826:00 +CPU= 22.188
  08:16:09 STEP 2827 H=2827:00 +CPU= 13.755
  08:16:23 STEP 2828 H=2828:00 +CPU= 14.211
  08:16:45 STEP 2829 H=2829:00 +CPU= 21.726
  08:16:59 STEP 2830 H=2830:00 +CPU= 14.169
  08:17:14 STEP 2831 H=2831:00 +CPU= 14.165
  08:17:40 STEP 2832 H=2832:00 +CPU= 26.211
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002832
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002832
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002832
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002808
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002808
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002808
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002820
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002820
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002820
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_117.zip
Uploading the intermediate file: upload_file_117.zip
  08:17:54 STEP 2833 H=2833:00 +CPU= 13.722
  08:18:08 STEP 2834 H=2834:00 +CPU= 14.192
  08:18:30 STEP 2835 H=2835:00 +CPU= 21.734
  08:18:44 STEP 2836 H=2836:00 +CPU= 14.263
  08:18:58 STEP 2837 H=2837:00 +CPU= 13.759
  08:19:20 STEP 2838 H=2838:00 +CPU= 22.273
  08:19:34 STEP 2839 H=2839:00 +CPU= 13.786
  08:19:48 STEP 2840 H=2840:00 +CPU= 14.201
  08:20:10 STEP 2841 H=2841:00 +CPU= 21.737
  08:20:24 STEP 2842 H=2842:00 +CPU= 14.218
  08:20:38 STEP 2843 H=2843:00 +CPU= 13.773
  08:21:03 STEP 2844 H=2844:00 +CPU= 24.377
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002844
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002844
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002844
  08:21:16 STEP 2845 H=2845:00 +CPU= 13.733
  08:21:31 STEP 2846 H=2846:00 +CPU= 14.227
  08:21:52 STEP 2847 H=2847:00 +CPU= 21.729
  08:22:07 STEP 2848 H=2848:00 +CPU= 14.317
  08:22:21 STEP 2849 H=2849:00 +CPU= 13.905
  08:22:43 STEP 2850 H=2850:00 +CPU= 22.241
  08:22:57 STEP 2851 H=2851:00 +CPU= 13.742
  08:23:11 STEP 2852 H=2852:00 +CPU= 14.209
  08:23:33 STEP 2853 H=2853:00 +CPU= 21.709
  08:23:47 STEP 2854 H=2854:00 +CPU= 14.213
  08:24:01 STEP 2855 H=2855:00 +CPU= 14.145
  08:24:28 STEP 2856 H=2856:00 +CPU= 26.310
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002856
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002856
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002856
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002832
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002832
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002832
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002844
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002844
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002844
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_118.zip
Uploading the intermediate file: upload_file_118.zip
  08:24:42 STEP 2857 H=2857:00 +CPU= 13.706
  08:24:56 STEP 2858 H=2858:00 +CPU= 14.413
  08:25:18 STEP 2859 H=2859:00 +CPU= 21.688
  08:25:32 STEP 2860 H=2860:00 +CPU= 14.145
  08:25:46 STEP 2861 H=2861:00 +CPU= 13.734
  08:26:08 STEP 2862 H=2862:00 +CPU= 22.262
  08:26:22 STEP 2863 H=2863:00 +CPU= 13.773
  08:26:36 STEP 2864 H=2864:00 +CPU= 14.196
  08:26:58 STEP 2865 H=2865:00 +CPU= 21.801
  08:27:13 STEP 2866 H=2866:00 +CPU= 14.241
  08:27:26 STEP 2867 H=2867:00 +CPU= 13.758
  08:27:51 STEP 2868 H=2868:00 +CPU= 24.462
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002868
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002868
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002868
  08:28:05 STEP 2869 H=2869:00 +CPU= 13.778
  08:28:19 STEP 2870 H=2870:00 +CPU= 14.199
  08:28:41 STEP 2871 H=2871:00 +CPU= 21.712
  08:28:55 STEP 2872 H=2872:00 +CPU= 14.207
  08:29:09 STEP 2873 H=2873:00 +CPU= 13.721
  08:29:31 STEP 2874 H=2874:00 +CPU= 22.182
  08:29:45 STEP 2875 H=2875:00 +CPU= 13.701
  08:29:59 STEP 2876 H=2876:00 +CPU= 14.186
  08:30:21 STEP 2877 H=2877:00 +CPU= 21.708
  08:30:35 STEP 2878 H=2878:00 +CPU= 14.146
  08:30:49 STEP 2879 H=2879:00 +CPU= 14.141
  08:31:16 STEP 2880 H=2880:00 +CPU= 26.179
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002880
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002880
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002880
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002856
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002856
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002856
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002868
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002868
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002868
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_119.zip
Uploading the intermediate file: upload_file_119.zip
  08:31:29 STEP 2881 H=2881:00 +CPU= 13.695
  08:31:44 STEP 2882 H=2882:00 +CPU= 14.170
  08:32:05 STEP 2883 H=2883:00 +CPU= 21.677
  08:32:20 STEP 2884 H=2884:00 +CPU= 14.122
  08:32:33 STEP 2885 H=2885:00 +CPU= 13.666
  08:32:56 STEP 2886 H=2886:00 +CPU= 22.232
  08:33:09 STEP 2887 H=2887:00 +CPU= 13.678
  08:33:24 STEP 2888 H=2888:00 +CPU= 14.200
  08:33:45 STEP 2889 H=2889:00 +CPU= 21.720
  08:34:00 STEP 2890 H=2890:00 +CPU= 14.189
  08:34:13 STEP 2891 H=2891:00 +CPU= 13.698
  08:34:38 STEP 2892 H=2892:00 +CPU= 24.381
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002892
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002892
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002892
  08:34:52 STEP 2893 H=2893:00 +CPU= 13.729
  08:35:06 STEP 2894 H=2894:00 +CPU= 14.212
  08:35:28 STEP 2895 H=2895:00 +CPU= 21.675
  08:35:42 STEP 2896 H=2896:00 +CPU= 14.152
  08:35:56 STEP 2897 H=2897:00 +CPU= 13.685
  08:36:18 STEP 2898 H=2898:00 +CPU= 22.283
  08:36:32 STEP 2899 H=2899:00 +CPU= 13.676
  08:36:46 STEP 2900 H=2900:00 +CPU= 14.188
  08:37:08 STEP 2901 H=2901:00 +CPU= 21.735
  08:37:22 STEP 2902 H=2902:00 +CPU= 14.147
  08:37:36 STEP 2903 H=2903:00 +CPU= 14.113
  08:38:02 STEP 2904 H=2904:00 +CPU= 26.152
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002904
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002904
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002904
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002880
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002880
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002880
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002892
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002892
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002892
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_120.zip
Uploading the intermediate file: upload_file_120.zip
  08:38:16 STEP 2905 H=2905:00 +CPU= 13.637
Uploading trickle at timestep: 10454400
  08:38:30 STEP 2906 H=2906:00 +CPU= 14.106
  08:38:52 STEP 2907 H=2907:00 +CPU= 21.663
  08:39:06 STEP 2908 H=2908:00 +CPU= 14.128
  08:39:20 STEP 2909 H=2909:00 +CPU= 13.695
  08:39:42 STEP 2910 H=2910:00 +CPU= 22.203
  08:39:56 STEP 2911 H=2911:00 +CPU= 13.725
  08:40:10 STEP 2912 H=2912:00 +CPU= 14.192
  08:40:32 STEP 2913 H=2913:00 +CPU= 21.767
  08:40:46 STEP 2914 H=2914:00 +CPU= 14.217
  08:41:00 STEP 2915 H=2915:00 +CPU= 13.732
  08:41:24 STEP 2916 H=2916:00 +CPU= 24.420
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002916
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002916
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002916
  08:41:38 STEP 2917 H=2917:00 +CPU= 13.753
  08:41:53 STEP 2918 H=2918:00 +CPU= 14.184
  08:42:14 STEP 2919 H=2919:00 +CPU= 21.716
  08:42:28 STEP 2920 H=2920:00 +CPU= 14.161
  08:42:42 STEP 2921 H=2921:00 +CPU= 13.726
  08:43:05 STEP 2922 H=2922:00 +CPU= 22.224
  08:43:18 STEP 2923 H=2923:00 +CPU= 13.703
  08:43:32 STEP 2924 H=2924:00 +CPU= 14.152
  08:43:54 STEP 2925 H=2925:00 +CPU= 21.646
  08:44:08 STEP 2926 H=2926:00 +CPU= 14.148
  08:44:23 STEP 2927 H=2927:00 +CPU= 14.123
  08:44:49 STEP 2928 H=2928:00 +CPU= 26.289
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002928
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002928
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002928
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002904
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002904
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002904
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002916
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002916
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002916
Zipping up the intermediate file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_121.zip
Uploading the intermediate file: upload_file_121.zip
  08:45:03 STEP 2929 H=2929:00 +CPU= 13.703
  08:45:17 STEP 2930 H=2930:00 +CPU= 14.141
  08:45:39 STEP 2931 H=2931:00 +CPU= 21.696
  08:45:53 STEP 2932 H=2932:00 +CPU= 14.189
  08:46:07 STEP 2933 H=2933:00 +CPU= 13.762
  08:46:29 STEP 2934 H=2934:00 +CPU= 22.225
  08:46:43 STEP 2935 H=2935:00 +CPU= 13.725
  08:46:57 STEP 2936 H=2936:00 +CPU= 14.243
  08:47:19 STEP 2937 H=2937:00 +CPU= 21.729
  08:47:33 STEP 2938 H=2938:00 +CPU= 14.229
  08:47:47 STEP 2939 H=2939:00 +CPU= 13.741
  08:48:11 STEP 2940 H=2940:00 +CPU= 24.350
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002940
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002940
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002940
  08:48:25 STEP 2941 H=2941:00 +CPU= 13.722
  08:48:39 STEP 2942 H=2942:00 +CPU= 14.194
  08:49:01 STEP 2943 H=2943:00 +CPU= 21.729
  08:49:15 STEP 2944 H=2944:00 +CPU= 14.139
  08:49:29 STEP 2945 H=2945:00 +CPU= 13.683
  08:49:51 STEP 2946 H=2946:00 +CPU= 22.192
  08:50:05 STEP 2947 H=2947:00 +CPU= 13.720
  08:50:19 STEP 2948 H=2948:00 +CPU= 14.121
  08:50:41 STEP 2949 H=2949:00 +CPU= 21.662
  08:50:56 STEP 2950 H=2950:00 +CPU= 14.517
  08:51:10 STEP 2951 H=2951:00 +CPU= 14.284
  08:51:35 STEP 2952 H=2952:00 +CPU= 25.497
The child process terminated with status: 0
Moving to projects directory: /home/galfridian/slots/1/ICMGGhq0f+002952
Moving to projects directory: /home/galfridian/slots/1/ICMSHhq0f+002952
Moving to projects directory: /home/galfridian/slots/1/ICMUAhq0f+002952
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002952
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002928
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002928
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002940
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002928
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMUAhq0f+002940
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMGGhq0f+002952
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002952
Adding to the zip: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_12190951/ICMSHhq0f+002940
Zipping up the final file: /home/galfridian/projects/climateprediction.net/oifs_43r3_ps_0307_2005050100_123_974_12190951_0_r1957012686_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
08:54:58 (3023): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:47:36 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 10,623,600 52,495 0.0049
09 May 2023 16:47:36 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 10,454,400 51,685 0.0049
09 May 2023 16:47:35 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 9,590,400 47,615 0.0050
09 May 2023 16:47:32 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 8,380,800 41,502 0.0050
09 May 2023 16:47:32 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 8,380,800 41,502 0.0050
09 May 2023 16:47:24 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 7,516,800 37,385 0.0050
09 May 2023 16:47:24 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 7,516,800 37,385 0.0050
09 May 2023 16:47:23 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 6,652,800 33,282 0.0050
09 May 2023 16:47:12 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 5,356,800 26,484 0.0049
09 May 2023 16:46:40 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 3,456,000 17,111 0.0050
09 May 2023 16:46:38 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 2,592,000 12,864 0.0050
09 May 2023 16:46:37 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 1,728,000 8,610 0.0050
09 May 2023 16:46:37 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 1,728,000 8,610 0.0050
09 May 2023 16:46:35 1537623 22276607 oifs_43r3_ps_0307_2005050100_123_974_12190951_0 864,000 4,319 0.0050


©2024 climateprediction.net