climateprediction.net home page
Task 22246764

Task 22246764

Name oifs_43r3_ps_1676_2021050100_123_946_12164765_0
Workunit 12164765
Created 29 Nov 2022, 12:01:29 UTC
Sent 29 Nov 2022, 12:30:33 UTC
Report deadline 29 Dec 2022, 12:30:33 UTC
Received 18 Mar 2023, 7:46:32 UTC
Server state Over
Outcome Computation error
Client state Aborted by user
Exit status 203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID 1524992
Run time 14 hours 30 min 27 sec
CPU time 38 min 19 sec
Validate state Invalid
Credit 11,370.24
Device peak FLOPS 6.24 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.01
x86_64-pc-linux-gnu
Peak working set size 4,771.17 MB
Peak swap size 4,978.10 MB
Peak disk usage 9,494.57 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<message>
aborted by user</message>
<stderr_txt>
tory: /var/lib/boinc/slots/35/ICMGGhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001860
  19:17:10 STEP 1861 H=1861:00 +CPU= 18.270
  19:17:29 STEP 1862 H=1862:00 +CPU= 18.721
  19:17:58 STEP 1863 H=1863:00 +CPU= 28.925
  19:18:17 STEP 1864 H=1864:00 +CPU= 18.879
  19:18:35 STEP 1865 H=1865:00 +CPU= 17.329
  19:19:04 STEP 1866 H=1866:00 +CPU= 28.973
  19:19:22 STEP 1867 H=1867:00 +CPU= 18.177
  19:19:41 STEP 1868 H=1868:00 +CPU= 18.368
  19:20:09 STEP 1869 H=1869:00 +CPU= 27.315
  19:20:28 STEP 1870 H=1870:00 +CPU= 18.799
  19:20:48 STEP 1871 H=1871:00 +CPU= 19.592
  19:21:21 STEP 1872 H=1872:00 +CPU= 32.184
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001872
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001872
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001860
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_13.zip
  19:21:38 STEP 1873 H=1873:00 +CPU= 17.621
Uploading the intermediate file: upload_file_13.zip
  19:21:56 STEP 1874 H=1874:00 +CPU= 17.735
Uploading trickle at timestep: 6739200
  19:22:25 STEP 1875 H=1875:00 +CPU= 28.554
  19:22:42 STEP 1876 H=1876:00 +CPU= 17.189
  19:22:59 STEP 1877 H=1877:00 +CPU= 16.995
  19:23:27 STEP 1878 H=1878:00 +CPU= 27.944
  19:23:45 STEP 1879 H=1879:00 +CPU= 17.836
  19:24:04 STEP 1880 H=1880:00 +CPU= 18.686
  19:24:31 STEP 1881 H=1881:00 +CPU= 26.804
  19:24:49 STEP 1882 H=1882:00 +CPU= 17.720
  19:25:07 STEP 1883 H=1883:00 +CPU= 18.335
  19:25:38 STEP 1884 H=1884:00 +CPU= 30.600
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1676
(argv4) batchid: 946
(argv5) wuid: 12164765
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/35
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/35/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/35/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5645b38caea7513aab803a620ff2367c to: /var/lib/boinc/slots/35/oifs_43r3_ps_1676_2021050100_123_946_12164765.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/35/oifs_43r3_ps_1676_2021050100_123_946_12164765.zip
ic_ancil_file: ic_ancil_12164765
ifsdata_file: ifsdata_12164765
climate_data_file: clim_data_12164765
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_d295b7e300a744537ce4c2bd808a504b to: /var/lib/boinc/slots/35/ic_ancil_12164765.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/35/ic_ancil_12164765.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_626a12131031864ba2b27332315b5bf4 to: /var/lib/boinc/slots/35/ifsdata/ifsdata_12164765.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/35/ifsdata/ifsdata_12164765.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/35/159l_2/clim_data_12164765.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/35/159l_2/clim_data_12164765.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719
The child process has been launched with process id: 272032
Executing the command: /var/lib/boinc/slots/35/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272032:272032] [20221203:192729:1670120849:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:28:19 STEP 1872 H=1872:00 +CPU= 30.380
  19:28:36 STEP 1873 H=1873:00 +CPU= 16.390
  19:28:53 STEP 1874 H=1874:00 +CPU= 16.900
  19:29:17 STEP 1875 H=1875:00 +CPU= 24.024
  19:29:34 STEP 1876 H=1876:00 +CPU= 16.556
  19:29:51 STEP 1877 H=1877:00 +CPU= 16.889
  19:30:19 STEP 1878 H=1878:00 +CPU= 28.794
  19:30:37 STEP 1879 H=1879:00 +CPU= 17.251
  19:30:55 STEP 1880 H=1880:00 +CPU= 17.918
  19:31:23 STEP 1881 H=1881:00 +CPU= 27.649
  19:31:40 STEP 1882 H=1882:00 +CPU= 17.199
  19:31:57 STEP 1883 H=1883:00 +CPU= 17.415
  19:32:30 STEP 1884 H=1884:00 +CPU= 31.870
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001884
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001884
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001884
  19:32:48 STEP 1885 H=1885:00 +CPU= 17.720
  19:33:07 STEP 1886 H=1886:00 +CPU= 18.299
  19:33:35 STEP 1887 H=1887:00 +CPU= 28.067
  19:33:53 STEP 1888 H=1888:00 +CPU= 17.873
  19:34:11 STEP 1889 H=1889:00 +CPU= 17.343
  19:34:39 STEP 1890 H=1890:00 +CPU= 28.420
  19:34:57 STEP 1891 H=1891:00 +CPU= 17.742
  19:35:15 STEP 1892 H=1892:00 +CPU= 17.708
  19:35:42 STEP 1893 H=1893:00 +CPU= 27.355
  19:36:00 STEP 1894 H=1894:00 +CPU= 17.847
  19:36:25 STEP 1895 H=1895:00 +CPU= 17.440
  19:37:04 STEP 1896 H=1896:00 +CPU= 31.932
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001896
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001896
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001884
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_1.zip
  19:37:21 STEP 1897 H=1897:00 +CPU= 16.805
  19:37:40 STEP 1898 H=1898:00 +CPU= 18.517
Uploading the intermediate file: upload_file_1.zip
  19:38:07 STEP 1899 H=1899:00 +CPU= 26.964
  19:38:25 STEP 1900 H=1900:00 +CPU= 17.355
Uploading trickle at timestep: 6825600
  19:38:42 STEP 1901 H=1901:00 +CPU= 16.800
  19:39:09 STEP 1902 H=1902:00 +CPU= 26.911
  19:39:26 STEP 1903 H=1903:00 +CPU= 17.509
  19:39:45 STEP 1904 H=1904:00 +CPU= 18.484
  19:40:13 STEP 1905 H=1905:00 +CPU= 28.042
  19:40:31 STEP 1906 H=1906:00 +CPU= 17.690
  19:40:48 STEP 1907 H=1907:00 +CPU= 16.977
  19:41:19 STEP 1908 H=1908:00 +CPU= 30.702
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001908
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001908
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001908
  19:41:37 STEP 1909 H=1909:00 +CPU= 18.193
  19:41:56 STEP 1910 H=1910:00 +CPU= 18.316
  19:42:23 STEP 1911 H=1911:00 +CPU= 26.980
  19:42:41 STEP 1912 H=1912:00 +CPU= 17.796
  19:42:59 STEP 1913 H=1913:00 +CPU= 17.293
  19:43:28 STEP 1914 H=1914:00 +CPU= 29.020
  19:43:45 STEP 1915 H=1915:00 +CPU= 17.359
  19:44:03 STEP 1916 H=1916:00 +CPU= 17.522
  19:44:30 STEP 1917 H=1917:00 +CPU= 27.497
  19:44:49 STEP 1918 H=1918:00 +CPU= 18.096
  19:45:07 STEP 1919 H=1919:00 +CPU= 18.514
  19:45:42 STEP 1920 H=1920:00 +CPU= 31.128
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001920
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001920
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001908
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_2.zip
Uploading the intermediate file: upload_file_2.zip
  19:45:58 STEP 1921 H=1921:00 +CPU= 15.444
Uploading trickle at timestep: 6912000
  19:46:15 STEP 1922 H=1922:00 +CPU= 16.923
  19:46:39 STEP 1923 H=1923:00 +CPU= 24.882
  19:46:56 STEP 1924 H=1924:00 +CPU= 16.474
  19:47:13 STEP 1925 H=1925:00 +CPU= 16.839
  19:47:39 STEP 1926 H=1926:00 +CPU= 25.570
  19:47:55 STEP 1927 H=1927:00 +CPU= 15.452
  19:48:12 STEP 1928 H=1928:00 +CPU= 17.490
  19:48:39 STEP 1929 H=1929:00 +CPU= 26.436
  19:48:58 STEP 1930 H=1930:00 +CPU= 19.032
  19:49:14 STEP 1931 H=1931:00 +CPU= 15.788
  19:49:40 STEP 1932 H=1932:00 +CPU= 26.134
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001932
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001932
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001932
  19:49:57 STEP 1933 H=1933:00 +CPU= 17.247
  19:50:13 STEP 1934 H=1934:00 +CPU= 15.603
  19:50:41 STEP 1935 H=1935:00 +CPU= 27.777
  19:50:57 STEP 1936 H=1936:00 +CPU= 16.384
  19:51:16 STEP 1937 H=1937:00 +CPU= 18.386
  19:51:43 STEP 1938 H=1938:00 +CPU= 27.108
  19:52:00 STEP 1939 H=1939:00 +CPU= 16.933
  19:52:18 STEP 1940 H=1940:00 +CPU= 17.542
  19:52:46 STEP 1941 H=1941:00 +CPU= 27.898
  19:53:04 STEP 1942 H=1942:00 +CPU= 17.564
  19:53:21 STEP 1943 H=1943:00 +CPU= 17.688
  19:53:56 STEP 1944 H=1944:00 +CPU= 32.893
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001944
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001944
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001932
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_3.zip
Uploading the intermediate file: upload_file_3.zip
  19:54:13 STEP 1945 H=1945:00 +CPU= 16.516
  19:54:27 STEP 1946 H=1946:00 +CPU= 14.624
Uploading trickle at timestep: 6998400
  19:54:52 STEP 1947 H=1947:00 +CPU= 24.601
  19:55:10 STEP 1948 H=1948:00 +CPU= 17.512
  19:55:28 STEP 1949 H=1949:00 +CPU= 17.698
  19:55:57 STEP 1950 H=1950:00 +CPU= 28.506
  19:56:12 STEP 1951 H=1951:00 +CPU= 15.745
  19:56:30 STEP 1952 H=1952:00 +CPU= 17.271
  19:56:57 STEP 1953 H=1953:00 +CPU= 26.828
  19:57:14 STEP 1954 H=1954:00 +CPU= 17.484
  19:57:32 STEP 1955 H=1955:00 +CPU= 17.653
  19:58:03 STEP 1956 H=1956:00 +CPU= 30.540
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001956
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001956
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001956
  19:58:20 STEP 1957 H=1957:00 +CPU= 17.500
  19:58:37 STEP 1958 H=1958:00 +CPU= 16.868
  19:59:05 STEP 1959 H=1959:00 +CPU= 27.509
  19:59:22 STEP 1960 H=1960:00 +CPU= 16.798
  19:59:38 STEP 1961 H=1961:00 +CPU= 16.314
  20:00:06 STEP 1962 H=1962:00 +CPU= 27.618
  20:00:24 STEP 1963 H=1963:00 +CPU= 17.621
  20:00:41 STEP 1964 H=1964:00 +CPU= 17.097
  20:01:07 STEP 1965 H=1965:00 +CPU= 26.425
  20:01:25 STEP 1966 H=1966:00 +CPU= 17.520
  20:01:42 STEP 1967 H=1967:00 +CPU= 17.223
  20:02:15 STEP 1968 H=1968:00 +CPU= 31.353
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001968
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001968
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001956
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_4.zip
Uploading the intermediate file: upload_file_4.zip
  20:02:31 STEP 1969 H=1969:00 +CPU= 16.531
Uploading trickle at timestep: 7084800
  20:02:49 STEP 1970 H=1970:00 +CPU= 18.026
  20:03:16 STEP 1971 H=1971:00 +CPU= 26.808
  20:03:33 STEP 1972 H=1972:00 +CPU= 16.703
  20:03:51 STEP 1973 H=1973:00 +CPU= 17.277
  20:04:17 STEP 1974 H=1974:00 +CPU= 26.036
  20:04:33 STEP 1975 H=1975:00 +CPU= 16.467
  20:04:52 STEP 1976 H=1976:00 +CPU= 18.519
  20:05:18 STEP 1977 H=1977:00 +CPU= 25.765
  20:05:35 STEP 1978 H=1978:00 +CPU= 16.877
  20:05:52 STEP 1979 H=1979:00 +CPU= 17.379
  20:06:22 STEP 1980 H=1980:00 +CPU= 29.705
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001980
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001980
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001980
  20:06:39 STEP 1981 H=1981:00 +CPU= 16.605
  20:06:57 STEP 1982 H=1982:00 +CPU= 17.574
  20:07:24 STEP 1983 H=1983:00 +CPU= 27.422
  20:07:41 STEP 1984 H=1984:00 +CPU= 16.809
  20:07:58 STEP 1985 H=1985:00 +CPU= 17.086
  20:08:27 STEP 1986 H=1986:00 +CPU= 28.391
  20:08:44 STEP 1987 H=1987:00 +CPU= 16.897
  20:09:03 STEP 1988 H=1988:00 +CPU= 19.216
  20:09:30 STEP 1989 H=1989:00 +CPU= 27.193
  20:09:48 STEP 1990 H=1990:00 +CPU= 17.306
  20:10:05 STEP 1991 H=1991:00 +CPU= 17.034
  20:10:38 STEP 1992 H=1992:00 +CPU= 30.923
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+001992
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+001992
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001980
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_5.zip
  20:10:55 STEP 1993 H=1993:00 +CPU= 17.051
Uploading the intermediate file: upload_file_5.zip
  20:11:12 STEP 1994 H=1994:00 +CPU= 16.611
Uploading trickle at timestep: 7171200
  20:11:40 STEP 1995 H=1995:00 +CPU= 27.762
  20:11:58 STEP 1996 H=1996:00 +CPU= 18.371
  20:12:15 STEP 1997 H=1997:00 +CPU= 16.942
  20:12:41 STEP 1998 H=1998:00 +CPU= 26.248
  20:12:57 STEP 1999 H=1999:00 +CPU= 15.524
  20:13:15 STEP 2000 H=2000:00 +CPU= 17.523
  20:13:39 STEP 2001 H=2001:00 +CPU= 24.619
  20:13:54 STEP 2002 H=2002:00 +CPU= 14.141
  20:14:10 STEP 2003 H=2003:00 +CPU= 16.828
  20:14:41 STEP 2004 H=2004:00 +CPU= 30.027
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002004
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002004
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002004
  20:14:59 STEP 2005 H=2005:00 +CPU= 17.822
  20:15:17 STEP 2006 H=2006:00 +CPU= 18.368
  20:15:45 STEP 2007 H=2007:00 +CPU= 27.769
  20:16:04 STEP 2008 H=2008:00 +CPU= 19.004
  20:16:22 STEP 2009 H=2009:00 +CPU= 17.639
  20:16:50 STEP 2010 H=2010:00 +CPU= 28.030
  20:17:07 STEP 2011 H=2011:00 +CPU= 17.596
  20:17:25 STEP 2012 H=2012:00 +CPU= 17.668
  20:17:53 STEP 2013 H=2013:00 +CPU= 27.640
  20:18:10 STEP 2014 H=2014:00 +CPU= 16.582
  20:18:27 STEP 2015 H=2015:00 +CPU= 17.020
  20:18:59 STEP 2016 H=2016:00 +CPU= 30.830
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002016
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002016
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002004
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_6.zip
  20:19:16 STEP 2017 H=2017:00 +CPU= 16.755
  20:19:34 STEP 2018 H=2018:00 +CPU= 17.120
Uploading the intermediate file: upload_file_6.zip
  20:20:02 STEP 2019 H=2019:00 +CPU= 28.252
Uploading trickle at timestep: 7257600
  20:20:20 STEP 2020 H=2020:00 +CPU= 18.284
  20:20:37 STEP 2021 H=2021:00 +CPU= 16.889
  20:21:04 STEP 2022 H=2022:00 +CPU= 26.554
  20:21:20 STEP 2023 H=2023:00 +CPU= 15.840
  20:21:38 STEP 2024 H=2024:00 +CPU= 18.354
  20:22:07 STEP 2025 H=2025:00 +CPU= 28.972
  20:22:24 STEP 2026 H=2026:00 +CPU= 16.298
  20:22:40 STEP 2027 H=2027:00 +CPU= 16.199
  20:23:10 STEP 2028 H=2028:00 +CPU= 29.345
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002028
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002028
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002028
  20:23:27 STEP 2029 H=2029:00 +CPU= 17.062
  20:23:45 STEP 2030 H=2030:00 +CPU= 17.455
  20:24:12 STEP 2031 H=2031:00 +CPU= 27.421
  20:24:29 STEP 2032 H=2032:00 +CPU= 16.061
  20:24:45 STEP 2033 H=2033:00 +CPU= 16.034
  20:25:14 STEP 2034 H=2034:00 +CPU= 28.871
  20:25:31 STEP 2035 H=2035:00 +CPU= 17.159
  20:25:50 STEP 2036 H=2036:00 +CPU= 18.381
  20:26:18 STEP 2037 H=2037:00 +CPU= 27.357
  20:26:37 STEP 2038 H=2038:00 +CPU= 19.324
  20:26:56 STEP 2039 H=2039:00 +CPU= 18.765
  20:27:33 STEP 2040 H=2040:00 +CPU= 32.730
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002040
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002040
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002028
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_7.zip
  20:27:50 STEP 2041 H=2041:00 +CPU= 17.152
Uploading the intermediate file: upload_file_7.zip
  20:28:09 STEP 2042 H=2042:00 +CPU= 18.864
Uploading trickle at timestep: 7344000
  20:28:32 STEP 2043 H=2043:00 +CPU= 22.985
  20:28:50 STEP 2044 H=2044:00 +CPU= 17.864
  20:29:07 STEP 2045 H=2045:00 +CPU= 16.938
  20:29:36 STEP 2046 H=2046:00 +CPU= 28.744
  20:29:53 STEP 2047 H=2047:00 +CPU= 17.262
  20:30:11 STEP 2048 H=2048:00 +CPU= 17.335
  20:30:39 STEP 2049 H=2049:00 +CPU= 27.824
  20:30:57 STEP 2050 H=2050:00 +CPU= 17.989
  20:31:14 STEP 2051 H=2051:00 +CPU= 16.979
  20:31:45 STEP 2052 H=2052:00 +CPU= 29.991
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002052
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002052
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002052
  20:31:59 STEP 2053 H=2053:00 +CPU= 13.814
  20:32:17 STEP 2054 H=2054:00 +CPU= 18.117
  20:32:41 STEP 2055 H=2055:00 +CPU= 24.118
  20:33:01 STEP 2056 H=2056:00 +CPU= 19.290
  20:33:18 STEP 2057 H=2057:00 +CPU= 17.545
  20:33:47 STEP 2058 H=2058:00 +CPU= 28.138
  20:34:05 STEP 2059 H=2059:00 +CPU= 17.992
  20:34:23 STEP 2060 H=2060:00 +CPU= 18.235
  20:34:51 STEP 2061 H=2061:00 +CPU= 27.302
  20:35:09 STEP 2062 H=2062:00 +CPU= 18.022
  20:35:28 STEP 2063 H=2063:00 +CPU= 18.305
  20:36:04 STEP 2064 H=2064:00 +CPU= 32.478
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002064
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002064
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002052
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_8.zip
Uploading the intermediate file: upload_file_8.zip
  20:36:22 STEP 2065 H=2065:00 +CPU= 17.876
  20:36:40 STEP 2066 H=2066:00 +CPU= 17.520
Uploading trickle at timestep: 7430400
  20:37:07 STEP 2067 H=2067:00 +CPU= 27.013
  20:37:25 STEP 2068 H=2068:00 +CPU= 18.272
  20:37:43 STEP 2069 H=2069:00 +CPU= 17.546
  20:38:10 STEP 2070 H=2070:00 +CPU= 26.811
  20:38:27 STEP 2071 H=2071:00 +CPU= 17.452
  20:38:44 STEP 2072 H=2072:00 +CPU= 17.021
  20:39:10 STEP 2073 H=2073:00 +CPU= 25.780
  20:39:26 STEP 2074 H=2074:00 +CPU= 15.472
  20:39:43 STEP 2075 H=2075:00 +CPU= 16.753
  20:40:13 STEP 2076 H=2076:00 +CPU= 29.818
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002076
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002076
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002076
  20:40:32 STEP 2077 H=2077:00 +CPU= 18.555
  20:40:50 STEP 2078 H=2078:00 +CPU= 18.266
  20:41:17 STEP 2079 H=2079:00 +CPU= 26.833
  20:41:36 STEP 2080 H=2080:00 +CPU= 18.845
  20:41:55 STEP 2081 H=2081:00 +CPU= 18.944
  20:42:22 STEP 2082 H=2082:00 +CPU= 26.554
  20:42:39 STEP 2083 H=2083:00 +CPU= 16.600
  20:42:57 STEP 2084 H=2084:00 +CPU= 17.903
  20:43:25 STEP 2085 H=2085:00 +CPU= 28.148
  20:43:43 STEP 2086 H=2086:00 +CPU= 17.584
  20:44:02 STEP 2087 H=2087:00 +CPU= 18.405
  20:44:48 STEP 2088 H=2088:00 +CPU= 33.508
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002088
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002088
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002076
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_9.zip
  20:45:04 STEP 2089 H=2089:00 +CPU= 15.711
Uploading the intermediate file: upload_file_9.zip
  20:45:21 STEP 2090 H=2090:00 +CPU= 17.025
Uploading trickle at timestep: 7516800
  20:45:48 STEP 2091 H=2091:00 +CPU= 26.644
  20:46:06 STEP 2092 H=2092:00 +CPU= 18.006
  20:46:23 STEP 2093 H=2093:00 +CPU= 16.236
  20:46:49 STEP 2094 H=2094:00 +CPU= 26.345
  20:47:07 STEP 2095 H=2095:00 +CPU= 17.680
  20:47:25 STEP 2096 H=2096:00 +CPU= 18.192
  20:47:52 STEP 2097 H=2097:00 +CPU= 26.459
  20:48:10 STEP 2098 H=2098:00 +CPU= 17.424
  20:48:27 STEP 2099 H=2099:00 +CPU= 17.495
  20:48:58 STEP 2100 H=2100:00 +CPU= 30.293
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002100
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002100
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002100
  20:49:15 STEP 2101 H=2101:00 +CPU= 16.736
  20:49:30 STEP 2102 H=2102:00 +CPU= 14.924
  20:49:57 STEP 2103 H=2103:00 +CPU= 27.163
  20:50:16 STEP 2104 H=2104:00 +CPU= 18.061
  20:50:34 STEP 2105 H=2105:00 +CPU= 18.192
  20:51:03 STEP 2106 H=2106:00 +CPU= 28.471
  20:51:22 STEP 2107 H=2107:00 +CPU= 18.763
  20:51:40 STEP 2108 H=2108:00 +CPU= 17.420
  20:52:04 STEP 2109 H=2109:00 +CPU= 23.833
  20:52:23 STEP 2110 H=2110:00 +CPU= 18.580
  20:52:42 STEP 2111 H=2111:00 +CPU= 18.634
  20:53:24 STEP 2112 H=2112:00 +CPU= 32.469
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002112
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002112
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002100
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_10.zip
Uploading the intermediate file: upload_file_10.zip
  20:53:41 STEP 2113 H=2113:00 +CPU= 17.134
Uploading trickle at timestep: 7603200
  20:54:00 STEP 2114 H=2114:00 +CPU= 19.144
  20:54:28 STEP 2115 H=2115:00 +CPU= 27.506
  20:54:45 STEP 2116 H=2116:00 +CPU= 17.524
  20:55:03 STEP 2117 H=2117:00 +CPU= 17.162
  20:55:30 STEP 2118 H=2118:00 +CPU= 27.535
  20:55:48 STEP 2119 H=2119:00 +CPU= 18.065
  20:56:06 STEP 2120 H=2120:00 +CPU= 17.744
  20:56:35 STEP 2121 H=2121:00 +CPU= 28.236
  20:56:52 STEP 2122 H=2122:00 +CPU= 17.645
  20:57:09 STEP 2123 H=2123:00 +CPU= 16.970
  20:57:40 STEP 2124 H=2124:00 +CPU= 30.074
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002124
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002124
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002124
  20:57:54 STEP 2125 H=2125:00 +CPU= 14.274
  20:58:12 STEP 2126 H=2126:00 +CPU= 17.688
  20:58:40 STEP 2127 H=2127:00 +CPU= 28.144
  20:58:59 STEP 2128 H=2128:00 +CPU= 19.144
  20:59:17 STEP 2129 H=2129:00 +CPU= 16.975
  20:59:44 STEP 2130 H=2130:00 +CPU= 27.355
  21:00:02 STEP 2131 H=2131:00 +CPU= 18.001
  21:00:20 STEP 2132 H=2132:00 +CPU= 17.408
  21:00:48 STEP 2133 H=2133:00 +CPU= 27.930
  21:01:07 STEP 2134 H=2134:00 +CPU= 18.607
  21:01:24 STEP 2135 H=2135:00 +CPU= 17.040
  21:01:58 STEP 2136 H=2136:00 +CPU= 32.738
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002136
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002136
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002124
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_11.zip
Uploading the intermediate file: upload_file_11.zip
  21:02:16 STEP 2137 H=2137:00 +CPU= 17.917
  21:02:36 STEP 2138 H=2138:00 +CPU= 19.551
Uploading trickle at timestep: 7689600
  21:03:04 STEP 2139 H=2139:00 +CPU= 27.438
  21:03:22 STEP 2140 H=2140:00 +CPU= 18.816
  21:03:40 STEP 2141 H=2141:00 +CPU= 17.345
  21:04:09 STEP 2142 H=2142:00 +CPU= 28.503
  21:04:27 STEP 2143 H=2143:00 +CPU= 18.206
  21:04:43 STEP 2144 H=2144:00 +CPU= 15.772
  21:05:08 STEP 2145 H=2145:00 +CPU= 25.134
  21:05:27 STEP 2146 H=2146:00 +CPU= 18.400
  21:05:44 STEP 2147 H=2147:00 +CPU= 16.988
  21:06:15 STEP 2148 H=2148:00 +CPU= 31.185
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002148
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002148
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002148
  21:06:33 STEP 2149 H=2149:00 +CPU= 17.454
  21:06:48 STEP 2150 H=2150:00 +CPU= 15.258
  21:07:14 STEP 2151 H=2151:00 +CPU= 25.961
  21:07:33 STEP 2152 H=2152:00 +CPU= 18.631
  21:07:50 STEP 2153 H=2153:00 +CPU= 17.382
  21:08:19 STEP 2154 H=2154:00 +CPU= 28.917
  21:08:38 STEP 2155 H=2155:00 +CPU= 18.027
  21:08:54 STEP 2156 H=2156:00 +CPU= 16.283
  21:09:21 STEP 2157 H=2157:00 +CPU= 26.651
  21:09:38 STEP 2158 H=2158:00 +CPU= 17.429
  21:09:57 STEP 2159 H=2159:00 +CPU= 18.632
  21:10:52 STEP 2160 H=2160:00 +CPU= 33.682
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002160
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002160
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002148
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_12.zip
  21:11:10 STEP 2161 H=2161:00 +CPU= 17.683
  21:11:28 STEP 2162 H=2162:00 +CPU= 18.389
Uploading the intermediate file: upload_file_12.zip
  21:11:58 STEP 2163 H=2163:00 +CPU= 29.025
Uploading trickle at timestep: 7776000
  21:12:16 STEP 2164 H=2164:00 +CPU= 18.233
  21:12:34 STEP 2165 H=2165:00 +CPU= 18.127
  21:13:03 STEP 2166 H=2166:00 +CPU= 28.500
  21:13:20 STEP 2167 H=2167:00 +CPU= 17.219
  21:13:39 STEP 2168 H=2168:00 +CPU= 18.760
  21:14:07 STEP 2169 H=2169:00 +CPU= 27.478
  21:14:25 STEP 2170 H=2170:00 +CPU= 18.025
  21:14:42 STEP 2171 H=2171:00 +CPU= 17.003
  21:15:14 STEP 2172 H=2172:00 +CPU= 32.023
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002172
  21:15:33 STEP 2173 H=2173:00 +CPU= 18.102
  21:15:51 STEP 2174 H=2174:00 +CPU= 18.325
  21:16:19 STEP 2175 H=2175:00 +CPU= 27.994
  21:16:38 STEP 2176 H=2176:00 +CPU= 18.384
  21:16:57 STEP 2177 H=2177:00 +CPU= 18.852
  21:17:27 STEP 2178 H=2178:00 +CPU= 29.676
  21:17:45 STEP 2179 H=2179:00 +CPU= 18.380
  21:18:04 STEP 2180 H=2180:00 +CPU= 18.154
  21:18:32 STEP 2181 H=2181:00 +CPU= 27.802
  21:18:50 STEP 2182 H=2182:00 +CPU= 18.472
  21:19:08 STEP 2183 H=2183:00 +CPU= 17.873
  21:19:41 STEP 2184 H=2184:00 +CPU= 31.435
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002184
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002184
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002172
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_13.zip
Uploading the intermediate file: upload_file_13.zip
  21:19:59 STEP 2185 H=2185:00 +CPU= 17.955
  21:20:18 STEP 2186 H=2186:00 +CPU= 18.761
Uploading trickle at timestep: 7862400
  21:20:47 STEP 2187 H=2187:00 +CPU= 28.333
  21:21:06 STEP 2188 H=2188:00 +CPU= 18.696
  21:21:24 STEP 2189 H=2189:00 +CPU= 17.292
  21:21:52 STEP 2190 H=2190:00 +CPU= 27.870
  21:22:09 STEP 2191 H=2191:00 +CPU= 16.786
  21:22:26 STEP 2192 H=2192:00 +CPU= 17.559
  21:22:54 STEP 2193 H=2193:00 +CPU= 27.277
  21:23:11 STEP 2194 H=2194:00 +CPU= 17.552
  21:23:30 STEP 2195 H=2195:00 +CPU= 18.200
  21:24:02 STEP 2196 H=2196:00 +CPU= 32.552
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002196
  21:24:20 STEP 2197 H=2197:00 +CPU= 17.584
  21:24:38 STEP 2198 H=2198:00 +CPU= 17.909
  21:25:07 STEP 2199 H=2199:00 +CPU= 28.765
  21:25:25 STEP 2200 H=2200:00 +CPU= 17.590
  21:25:42 STEP 2201 H=2201:00 +CPU= 16.618
  21:26:09 STEP 2202 H=2202:00 +CPU= 26.776
  21:26:26 STEP 2203 H=2203:00 +CPU= 16.886
  21:26:45 STEP 2204 H=2204:00 +CPU= 18.343
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1676
(argv4) batchid: 946
(argv5) wuid: 12164765
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/35
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/35/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/35/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5645b38caea7513aab803a620ff2367c to: /var/lib/boinc/slots/35/oifs_43r3_ps_1676_2021050100_123_946_12164765.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/35/oifs_43r3_ps_1676_2021050100_123_946_12164765.zip
ic_ancil_file: ic_ancil_12164765
ifsdata_file: ifsdata_12164765
climate_data_file: clim_data_12164765
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_d295b7e300a744537ce4c2bd808a504b to: /var/lib/boinc/slots/35/ic_ancil_12164765.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/35/ic_ancil_12164765.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_626a12131031864ba2b27332315b5bf4 to: /var/lib/boinc/slots/35/ifsdata/ifsdata_12164765.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/35/ifsdata/ifsdata_12164765.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/35/159l_2/clim_data_12164765.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/35/159l_2/clim_data_12164765.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719
The child process has been launched with process id: 275295
Executing the command: /var/lib/boinc/slots/35/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275295:275295] [20221203:212902:1670128142:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:29:38 STEP 2184 H=2184:00 +CPU= 32.201
  21:29:55 STEP 2185 H=2185:00 +CPU= 16.390
  21:30:09 STEP 2186 H=2186:00 +CPU= 14.245
  21:30:36 STEP 2187 H=2187:00 +CPU= 26.115
  21:30:55 STEP 2188 H=2188:00 +CPU= 17.899
  21:31:13 STEP 2189 H=2189:00 +CPU= 17.237
  21:31:41 STEP 2190 H=2190:00 +CPU= 27.909
  21:31:59 STEP 2191 H=2191:00 +CPU= 17.253
  21:32:17 STEP 2192 H=2192:00 +CPU= 17.625
  21:32:44 STEP 2193 H=2193:00 +CPU= 26.681
  21:33:02 STEP 2194 H=2194:00 +CPU= 17.185
  21:33:19 STEP 2195 H=2195:00 +CPU= 16.924
  21:33:49 STEP 2196 H=2196:00 +CPU= 29.582
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002196
  21:34:06 STEP 2197 H=2197:00 +CPU= 16.840
  21:34:24 STEP 2198 H=2198:00 +CPU= 17.647
  21:34:51 STEP 2199 H=2199:00 +CPU= 27.087
  21:35:09 STEP 2200 H=2200:00 +CPU= 17.800
  21:35:26 STEP 2201 H=2201:00 +CPU= 16.628
  21:35:55 STEP 2202 H=2202:00 +CPU= 27.497
  21:36:12 STEP 2203 H=2203:00 +CPU= 16.971
  21:36:31 STEP 2204 H=2204:00 +CPU= 17.998
  21:37:00 STEP 2205 H=2205:00 +CPU= 27.778
  21:37:18 STEP 2206 H=2206:00 +CPU= 17.165
  21:37:37 STEP 2207 H=2207:00 +CPU= 18.096
  21:38:19 STEP 2208 H=2208:00 +CPU= 28.689
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002208
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002208
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002196
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_1.zip
  21:38:35 STEP 2209 H=2209:00 +CPU= 14.946
Uploading the intermediate file: upload_file_1.zip
  21:38:50 STEP 2210 H=2210:00 +CPU= 14.454
Uploading trickle at timestep: 7948800
  21:39:13 STEP 2211 H=2211:00 +CPU= 23.320
  21:39:29 STEP 2212 H=2212:00 +CPU= 15.537
  21:39:45 STEP 2213 H=2213:00 +CPU= 15.816
  21:40:13 STEP 2214 H=2214:00 +CPU= 27.332
  21:40:29 STEP 2215 H=2215:00 +CPU= 15.290
  21:40:46 STEP 2216 H=2216:00 +CPU= 16.978
  21:41:13 STEP 2217 H=2217:00 +CPU= 26.629
  21:41:30 STEP 2218 H=2218:00 +CPU= 17.342
  21:41:48 STEP 2219 H=2219:00 +CPU= 16.942
  21:42:18 STEP 2220 H=2220:00 +CPU= 29.090
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002220
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002220
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002220
  21:42:36 STEP 2221 H=2221:00 +CPU= 17.590
  21:42:53 STEP 2222 H=2222:00 +CPU= 16.982
  21:43:19 STEP 2223 H=2223:00 +CPU= 25.750
  21:43:36 STEP 2224 H=2224:00 +CPU= 17.260
  21:43:53 STEP 2225 H=2225:00 +CPU= 16.177
  21:44:20 STEP 2226 H=2226:00 +CPU= 26.642
  21:44:37 STEP 2227 H=2227:00 +CPU= 17.258
  21:44:54 STEP 2228 H=2228:00 +CPU= 17.155
  21:45:20 STEP 2229 H=2229:00 +CPU= 25.809
  21:45:37 STEP 2230 H=2230:00 +CPU= 16.518
  21:45:55 STEP 2231 H=2231:00 +CPU= 16.951
  21:47:09 STEP 2232 H=2232:00 +CPU= 30.893
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002232
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002232
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002220
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_2.zip
Uploading the intermediate file: upload_file_2.zip
  21:47:27 STEP 2233 H=2233:00 +CPU= 16.916
Uploading trickle at timestep: 8035200
  21:47:43 STEP 2234 H=2234:00 +CPU= 16.603
  21:48:09 STEP 2235 H=2235:00 +CPU= 25.363
  21:48:26 STEP 2236 H=2236:00 +CPU= 17.252
  21:48:42 STEP 2237 H=2237:00 +CPU= 15.595
  21:49:09 STEP 2238 H=2238:00 +CPU= 27.251
  21:49:27 STEP 2239 H=2239:00 +CPU= 17.264
  21:49:45 STEP 2240 H=2240:00 +CPU= 17.812
  21:50:11 STEP 2241 H=2241:00 +CPU= 26.201
  21:50:30 STEP 2242 H=2242:00 +CPU= 18.867
  21:50:49 STEP 2243 H=2243:00 +CPU= 17.831
  21:51:22 STEP 2244 H=2244:00 +CPU= 30.369
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002244
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002244
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002244
  21:51:40 STEP 2245 H=2245:00 +CPU= 17.384
  21:51:57 STEP 2246 H=2246:00 +CPU= 17.023
  21:52:23 STEP 2247 H=2247:00 +CPU= 25.913
  21:52:40 STEP 2248 H=2248:00 +CPU= 17.591
  21:52:57 STEP 2249 H=2249:00 +CPU= 16.005
  21:53:22 STEP 2250 H=2250:00 +CPU= 24.888
  21:53:37 STEP 2251 H=2251:00 +CPU= 15.801
  21:53:53 STEP 2252 H=2252:00 +CPU= 15.259
  21:54:20 STEP 2253 H=2253:00 +CPU= 27.258
  21:54:39 STEP 2254 H=2254:00 +CPU= 18.391
  21:54:57 STEP 2255 H=2255:00 +CPU= 18.326
  21:55:32 STEP 2256 H=2256:00 +CPU= 32.205
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002256
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002256
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002244
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_3.zip
Uploading the intermediate file: upload_file_3.zip
  21:55:49 STEP 2257 H=2257:00 +CPU= 17.158
  21:56:03 STEP 2258 H=2258:00 +CPU= 14.053
Uploading trickle at timestep: 8121600
  21:56:27 STEP 2259 H=2259:00 +CPU= 23.534
  21:56:45 STEP 2260 H=2260:00 +CPU= 17.718
  21:57:02 STEP 2261 H=2261:00 +CPU= 17.132
  21:57:30 STEP 2262 H=2262:00 +CPU= 27.569
  21:57:47 STEP 2263 H=2263:00 +CPU= 17.032
  21:58:06 STEP 2264 H=2264:00 +CPU= 18.074
  21:58:34 STEP 2265 H=2265:00 +CPU= 27.683
  21:58:52 STEP 2266 H=2266:00 +CPU= 17.954
  21:59:10 STEP 2267 H=2267:00 +CPU= 17.240
  21:59:38 STEP 2268 H=2268:00 +CPU= 27.606
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002268
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002268
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002268
  21:59:55 STEP 2269 H=2269:00 +CPU= 17.686
  22:00:11 STEP 2270 H=2270:00 +CPU= 15.584
  22:00:39 STEP 2271 H=2271:00 +CPU= 27.860
  22:00:58 STEP 2272 H=2272:00 +CPU= 18.521
  22:01:15 STEP 2273 H=2273:00 +CPU= 16.874
  22:01:44 STEP 2274 H=2274:00 +CPU= 27.818
  22:02:00 STEP 2275 H=2275:00 +CPU= 16.404
  22:02:18 STEP 2276 H=2276:00 +CPU= 17.619
  22:02:43 STEP 2277 H=2277:00 +CPU= 24.990
  22:03:00 STEP 2278 H=2278:00 +CPU= 16.869
  22:03:18 STEP 2279 H=2279:00 +CPU= 17.492
  22:03:56 STEP 2280 H=2280:00 +CPU= 30.660
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002280
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002280
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002280
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMGGhpi1+002268
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMSHhpi1+002268
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765/ICMUAhpi1+002268
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1676_2021050100_123_946_12164765_0_r233656719_4.zip
  22:04:13 STEP 2281 H=2281:00 +CPU= 16.656
Uploading the intermediate file: upload_file_4.zip
  22:04:30 STEP 2282 H=2282:00 +CPU= 16.601
Uploading trickle at timestep: 8208000
  22:04:57 STEP 2283 H=2283:00 +CPU= 26.798
  22:05:15 STEP 2284 H=2284:00 +CPU= 17.929
  22:05:33 STEP 2285 H=2285:00 +CPU= 17.887
  22:06:01 STEP 2286 H=2286:00 +CPU= 27.994
  22:06:19 STEP 2287 H=2287:00 +CPU= 17.387
  22:06:37 STEP 2288 H=2288:00 +CPU= 18.124
  22:07:06 STEP 2289 H=2289:00 +CPU= 27.954
  22:07:25 STEP 2290 H=2290:00 +CPU= 18.494
  22:07:42 STEP 2291 H=2291:00 +CPU= 17.419
  22:08:13 STEP 2292 H=2292:00 +CPU= 30.189
Moving to projects directory: /var/lib/boinc/slots/35/ICMGGhpi1+002292
Moving to projects directory: /var/lib/boinc/slots/35/ICMSHhpi1+002292
Moving to projects directory: /var/lib/boinc/slots/35/ICMUAhpi1+002292
  22:08:31 STEP 2293 H=2293:00 +CPU= 18.030
  22:08:49 STEP 2294 H=2294:00 +CPU= 17.844
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1676
(argv4) batchid: 946
(argv5) wuid: 12164765
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/35
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164765
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/35/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/35/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5645b38caea7513aab803a620ff2367c to: /var/lib/boinc/slots/35/oifs_43r3_ps_1676_2021050100_123_946_12164765.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/35/oifs_43r3_ps_1676_2021050100_123_946_12164765.zip
ic_ancil_file: ic_ancil_12164765
ifsdata_file: ifsdata_12164765
climate_data_file: clim_data_12164765
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_d295b7e300a744537ce4c2bd808a504b to: /var/lib/boinc/slots/35/ic_ancil_12164765.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/35/ic_ancil_12164765.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_626a12131031864ba2b27332315b5bf4 to: /var/lib/boinc/slots/35/ifsdata/ifsdata_12164765.zip

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:32:53 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 8,208,000 1,997 0.0002
09 May 2023 15:32:54 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 8,121,600 1,509 0.0002
09 May 2023 15:32:53 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 8,035,200 1,010 0.0001
09 May 2023 15:32:54 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,948,800 528 0.0001
09 May 2023 15:32:46 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,862,400 6,593 0.0008
09 May 2023 15:32:46 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,776,000 6,066 0.0008
09 May 2023 15:32:46 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,689,600 5,559 0.0007
09 May 2023 15:32:46 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,603,200 5,047 0.0007
09 May 2023 15:32:46 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,516,800 4,554 0.0006
09 May 2023 15:32:46 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,430,400 4,043 0.0005
09 May 2023 15:32:46 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,344,000 3,541 0.0005
09 May 2023 15:32:37 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,257,600 3,038 0.0004
09 May 2023 15:32:37 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,171,200 2,537 0.0004
09 May 2023 15:32:36 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 7,084,800 2,041 0.0003
09 May 2023 15:32:37 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 6,998,400 1,540 0.0002
09 May 2023 15:32:37 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 6,912,000 1,048 0.0002
09 May 2023 15:32:37 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 6,825,600 542 0.0001
09 May 2023 15:32:29 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 6,739,200 6,666 0.0010
09 May 2023 15:32:29 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 6,652,800 6,133 0.0009
09 May 2023 15:32:29 1524992 22246764 oifs_43r3_ps_1676_2021050100_123_946_12164765_0 6,566,400 5,614 0.0009


©2024 cpdn.org