climateprediction.net home page
Task 22302939

Task 22302939

Name oifs_43r3_ps_0463_2010050100_123_979_12196107_2
Workunit 12196107
Created 24 Jan 2023, 14:01:09 UTC
Sent 24 Jan 2023, 14:34:21 UTC
Report deadline 25 Mar 2023, 14:34:21 UTC
Received 26 Jan 2023, 16:11:07 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1465949
Run time 13 hours 36 min 41 sec
CPU time 13 hours 13 min 5 sec
Validate state Invalid
Credit 9,072.00
Device peak FLOPS 9.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.51 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,682.17 MB
Stderr 
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
fs_43r3_ps_12196107/ICMGGhq0f+001800
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001800
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001800
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001812
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001812
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001812
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_75.zip
Uploading the intermediate file: upload_file_75.zip
  11:15:48 STEP 1825 H=1825:00 +CPU= 18.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:16:39 STEP 1826 H=1826:00 +CPU= 26.504
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:17:46 STEP 1827 H=1827:00 +CPU= 44.070
  11:18:21 STEP 1828 H=1828:00 +CPU= 33.924
  11:18:55 STEP 1829 H=1829:00 +CPU= 32.203
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:20:10 STEP 1830 H=1830:00 +CPU= 51.343
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:21:20 STEP 1831 H=1831:00 +CPU= 36.012
  11:21:56 STEP 1832 H=1832:00 +CPU= 34.785
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:23:52 STEP 1833 H=1833:00 +CPU= 57.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:24:35 STEP 1834 H=1834:00 +CPU= 31.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:25:43 STEP 1835 H=1835:00 +CPU= 33.360
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:27:21 STEP 1836 H=1836:00 +CPU= 62.720
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001836
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001836
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:28:22 STEP 1837 H=1837:00 +CPU= 37.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:29:22 STEP 1838 H=1838:00 +CPU= 36.892
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:30:32 STEP 1839 H=1839:00 +CPU= 56.809
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:31:31 STEP 1840 H=1840:00 +CPU= 35.357
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:32:36 STEP 1841 H=1841:00 +CPU= 30.723
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:33:47 STEP 1842 H=1842:00 +CPU= 47.842
  11:34:18 STEP 1843 H=1843:00 +CPU= 29.840
  11:34:51 STEP 1844 H=1844:00 +CPU= 32.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:35:55 STEP 1845 H=1845:00 +CPU= 50.779
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:36:39 STEP 1846 H=1846:00 +CPU= 32.715
  11:37:15 STEP 1847 H=1847:00 +CPU= 34.508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:38:59 STEP 1848 H=1848:00 +CPU= 58.230
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001848
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001848
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001848
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001824
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001824
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001824
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001836
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001836
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001836
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_76.zip
Uploading the intermediate file: upload_file_76.zip
  11:39:33 STEP 1849 H=1849:00 +CPU= 32.336
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:40:42 STEP 1850 H=1850:00 +CPU= 33.501
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:24 STEP 1851 H=1851:00 +CPU= 56.032
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:33 STEP 1852 H=1852:00 +CPU= 34.229
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:44:21 STEP 1853 H=1853:00 +CPU= 36.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:46:16 STEP 1854 H=1854:00 +CPU= 57.403
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:47:17 STEP 1855 H=1855:00 +CPU= 37.101
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:48:02 STEP 1856 H=1856:00 +CPU= 33.557
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:49:24 STEP 1857 H=1857:00 +CPU= 57.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:50:15 STEP 1858 H=1858:00 +CPU= 27.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:51:40 STEP 1859 H=1859:00 +CPU= 38.810
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2010050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0463
(argv4) batchid: 979
(argv5) wuid: 12196107
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /opt/fcs/BOINC/slots/2
Project directory is: /opt/fcs/BOINC/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107
..mkdir for temp folder for results failed
Copying: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6749c83de7e15c85092f8af7cbce001d to: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_0463_2010050100_123_979_12196107.zip
Unzipping the namelist zip file: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_0463_2010050100_123_979_12196107.zip
ic_ancil_file: ic_ancil_12196107
ifsdata_file: ifsdata_12196107
climate_data_file: clim_data_12196107
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_05300276878acbec4b045b4c392406a2 to: /opt/fcs/BOINC/slots/2/ic_ancil_12196107.zip
Unzipping the IC ancils zip file: /opt/fcs/BOINC/slots/2/ic_ancil_12196107.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3262b04ba6c93dc2a1cd8a754e4abfb3 to: /opt/fcs/BOINC/slots/2/ifsdata/ifsdata_12196107.zip
Unzipping the ifsdata_zip file: /opt/fcs/BOINC/slots/2/ifsdata/ifsdata_12196107.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /opt/fcs/BOINC/slots/2/159l_2/clim_data_12196107.zip
Unzipping the climate data zip file: /opt/fcs/BOINC/slots/2/159l_2/clim_data_12196107.zip
Checking for progress XML file: /opt/fcs/BOINC/slots/2/progress_file_12196107.xml
Opened progress file ok : /opt/fcs/BOINC/slots/2/progress_file_12196107.xml
-- Model is restarting --
Adjusting last_iter, 1860, to previous model restart step.
Creating progress file: /opt/fcs/BOINC/slots/2/progress_file_12196107.xml
last_cpu_time: 40976
upload_file_number: 77
last_iter: 1847
last_upload: 6652800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347
The child process has been launched with process id: 803932
Executing the command: /opt/fcs/BOINC/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:803932:803932] [20230126:123923:1674736763:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:41:07 STEP 1848 H=1848:00 +CPU= 62.167
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:42:15 STEP 1849 H=1849:00 +CPU= 33.621
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:43:26 STEP 1850 H=1850:00 +CPU= 36.912
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:44:37 STEP 1851 H=1851:00 +CPU= 57.642
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:45:36 STEP 1852 H=1852:00 +CPU= 34.914
  12:46:05 STEP 1853 H=1853:00 +CPU= 28.490
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:47:09 STEP 1854 H=1854:00 +CPU= 51.495
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:47:50 STEP 1855 H=1855:00 +CPU= 28.531
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:48:51 STEP 1856 H=1856:00 +CPU= 37.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:50:08 STEP 1857 H=1857:00 +CPU= 53.540
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:51:32 STEP 1858 H=1858:00 +CPU= 38.380
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:52:28 STEP 1859 H=1859:00 +CPU= 31.877
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:54:04 STEP 1860 H=1860:00 +CPU= 60.237
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001860
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001860
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001860
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:54:45 STEP 1861 H=1861:00 +CPU= 28.837
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:55:31 STEP 1862 H=1862:00 +CPU= 34.497
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:57:12 STEP 1863 H=1863:00 +CPU= 54.301
  12:57:43 STEP 1864 H=1864:00 +CPU= 30.870
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:58:27 STEP 1865 H=1865:00 +CPU= 31.471
  12:59:23 STEP 1866 H=1866:00 +CPU= 54.614
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:00:39 STEP 1867 H=1867:00 +CPU= 30.477
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:01:25 STEP 1868 H=1868:00 +CPU= 33.495
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:02:34 STEP 1869 H=1869:00 +CPU= 55.774
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:03:24 STEP 1870 H=1870:00 +CPU= 27.335
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:04:25 STEP 1871 H=1871:00 +CPU= 37.331
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:06:12 STEP 1872 H=1872:00 +CPU= 59.876
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001872
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001872
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001872
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001848
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001848
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001848
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001860
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001860
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001860
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_77.zip
Uploading the intermediate file: upload_file_77.zip
  13:06:26 STEP 1873 H=1873:00 +CPU= 14.021
  13:06:49 STEP 1874 H=1874:00 +CPU= 22.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:08:15 STEP 1875 H=1875:00 +CPU= 51.688
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:09:35 STEP 1876 H=1876:00 +CPU= 33.825
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:10:46 STEP 1877 H=1877:00 +CPU= 36.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:12:05 STEP 1878 H=1878:00 +CPU= 44.899
  13:12:35 STEP 1879 H=1879:00 +CPU= 28.953
  13:13:07 STEP 1880 H=1880:00 +CPU= 30.539
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:14:11 STEP 1881 H=1881:00 +CPU= 51.093
  13:14:41 STEP 1882 H=1882:00 +CPU= 29.670
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:15:27 STEP 1883 H=1883:00 +CPU= 33.915
  13:16:20 STEP 1884 H=1884:00 +CPU= 51.432
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001884
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001884
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:17:07 STEP 1885 H=1885:00 +CPU= 34.032
  13:17:39 STEP 1886 H=1886:00 +CPU= 30.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:18:45 STEP 1887 H=1887:00 +CPU= 52.723
  13:19:17 STEP 1888 H=1888:00 +CPU= 30.793
  13:19:48 STEP 1889 H=1889:00 +CPU= 29.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:20:54 STEP 1890 H=1890:00 +CPU= 53.428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:21:33 STEP 1891 H=1891:00 +CPU= 27.583
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:22:33 STEP 1892 H=1892:00 +CPU= 35.493
  13:23:20 STEP 1893 H=1893:00 +CPU= 45.635
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:24:16 STEP 1894 H=1894:00 +CPU= 32.096
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:24:55 STEP 1895 H=1895:00 +CPU= 28.096
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:26:12 STEP 1896 H=1896:00 +CPU= 63.108
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001896
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001896
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001896
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001872
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001872
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001872
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001884
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001884
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001884
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_78.zip
Uploading the intermediate file: upload_file_78.zip
  13:26:38 STEP 1897 H=1897:00 +CPU= 25.942
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:27:37 STEP 1898 H=1898:00 +CPU= 35.259
  13:28:26 STEP 1899 H=1899:00 +CPU= 47.274
  13:29:01 STEP 1900 H=1900:00 +CPU= 33.975
  13:29:35 STEP 1901 H=1901:00 +CPU= 33.244
  13:30:25 STEP 1902 H=1902:00 +CPU= 47.993
  13:31:00 STEP 1903 H=1903:00 +CPU= 33.788
  13:31:31 STEP 1904 H=1904:00 +CPU= 29.695
  13:32:25 STEP 1905 H=1905:00 +CPU= 52.744
  13:32:56 STEP 1906 H=1906:00 +CPU= 29.875
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:33:38 STEP 1907 H=1907:00 +CPU= 29.883
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:34:45 STEP 1908 H=1908:00 +CPU= 54.198
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001908
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001908
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001908
  13:35:27 STEP 1909 H=1909:00 +CPU= 30.388
  13:36:00 STEP 1910 H=1910:00 +CPU= 32.127
  13:36:52 STEP 1911 H=1911:00 +CPU= 50.338
  13:37:23 STEP 1912 H=1912:00 +CPU= 30.390
  13:37:54 STEP 1913 H=1913:00 +CPU= 29.415
  13:38:47 STEP 1914 H=1914:00 +CPU= 51.986
  13:39:17 STEP 1915 H=1915:00 +CPU= 29.049
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:40:15 STEP 1916 H=1916:00 +CPU= 33.953
  13:41:00 STEP 1917 H=1917:00 +CPU= 44.396
  13:41:38 STEP 1918 H=1918:00 +CPU= 36.378
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:42:28 STEP 1919 H=1919:00 +CPU= 26.738
  13:43:34 STEP 1920 H=1920:00 +CPU= 64.107
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001920
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001920
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001920
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001896
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001896
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001896
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001908
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001908
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001908
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_79.zip
Uploading the intermediate file: upload_file_79.zip
  13:44:03 STEP 1921 H=1921:00 +CPU= 27.914
  13:44:37 STEP 1922 H=1922:00 +CPU= 33.046
  13:45:29 STEP 1923 H=1923:00 +CPU= 50.402
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:46:14 STEP 1924 H=1924:00 +CPU= 32.852
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:46:58 STEP 1925 H=1925:00 +CPU= 32.122
  13:47:46 STEP 1926 H=1926:00 +CPU= 46.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:48:42 STEP 1927 H=1927:00 +CPU= 32.959
  13:49:10 STEP 1928 H=1928:00 +CPU= 26.907
  13:50:06 STEP 1929 H=1929:00 +CPU= 54.137
  13:50:35 STEP 1930 H=1930:00 +CPU= 28.032
  13:51:07 STEP 1931 H=1931:00 +CPU= 30.751
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:52:35 STEP 1932 H=1932:00 +CPU= 53.429
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001932
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001932
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001932
  13:53:06 STEP 1933 H=1933:00 +CPU= 30.670
  13:53:41 STEP 1934 H=1934:00 +CPU= 33.460
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:54:41 STEP 1935 H=1935:00 +CPU= 47.812
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:55:26 STEP 1936 H=1936:00 +CPU= 32.613
  13:55:57 STEP 1937 H=1937:00 +CPU= 30.428
  13:56:49 STEP 1938 H=1938:00 +CPU= 50.453
  13:57:22 STEP 1939 H=1939:00 +CPU= 31.204
  13:57:54 STEP 1940 H=1940:00 +CPU= 31.578
  13:58:46 STEP 1941 H=1941:00 +CPU= 50.629
  13:59:19 STEP 1942 H=1942:00 +CPU= 31.515
  13:59:53 STEP 1943 H=1943:00 +CPU= 32.917
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:01:00 STEP 1944 H=1944:00 +CPU= 54.401
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001944
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001944
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001944
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001920
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001920
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001920
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001932
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001932
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001932
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_80.zip
Uploading the intermediate file: upload_file_80.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:01:44 STEP 1945 H=1945:00 +CPU= 31.683
  14:02:17 STEP 1946 H=1946:00 +CPU= 31.487
  14:03:07 STEP 1947 H=1947:00 +CPU= 48.899
  14:03:40 STEP 1948 H=1948:00 +CPU= 31.533
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:22 STEP 1949 H=1949:00 +CPU= 30.071
  14:05:15 STEP 1950 H=1950:00 +CPU= 51.318
  14:05:47 STEP 1951 H=1951:00 +CPU= 31.567
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:06:31 STEP 1952 H=1952:00 +CPU= 31.501
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:07:33 STEP 1953 H=1953:00 +CPU= 49.503
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:08:13 STEP 1954 H=1954:00 +CPU= 28.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:08:56 STEP 1955 H=1955:00 +CPU= 30.266
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:10:14 STEP 1956 H=1956:00 +CPU= 53.448
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001956
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001956
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001956
  14:10:45 STEP 1957 H=1957:00 +CPU= 30.279
  14:11:18 STEP 1958 H=1958:00 +CPU= 31.932
  14:12:09 STEP 1959 H=1959:00 +CPU= 49.285
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:12:50 STEP 1960 H=1960:00 +CPU= 29.676
  14:13:24 STEP 1961 H=1961:00 +CPU= 33.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:14:49 STEP 1962 H=1962:00 +CPU= 59.101
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:15:37 STEP 1963 H=1963:00 +CPU= 35.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:16:30 STEP 1964 H=1964:00 +CPU= 29.975
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:17:38 STEP 1965 H=1965:00 +CPU= 55.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:18:19 STEP 1966 H=1966:00 +CPU= 28.938
  14:18:56 STEP 1967 H=1967:00 +CPU= 35.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:20:33 STEP 1968 H=1968:00 +CPU= 61.860
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001968
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001968
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001968
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001944
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001944
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001944
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001956
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001956
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001956
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_81.zip
Uploading the intermediate file: upload_file_81.zip
  14:21:07 STEP 1969 H=1969:00 +CPU= 32.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:21:50 STEP 1970 H=1970:00 +CPU= 31.378
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:23:24 STEP 1971 H=1971:00 +CPU= 58.122
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:24:11 STEP 1972 H=1972:00 +CPU= 34.552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:25:05 STEP 1973 H=1973:00 +CPU= 31.012
  14:25:58 STEP 1974 H=1974:00 +CPU= 51.825
  14:26:32 STEP 1975 H=1975:00 +CPU= 32.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:27:31 STEP 1976 H=1976:00 +CPU= 34.405
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:46 STEP 1977 H=1977:00 +CPU= 51.300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:29:53 STEP 1978 H=1978:00 +CPU= 32.146
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:30:58 STEP 1979 H=1979:00 +CPU= 31.695
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:32:14 STEP 1980 H=1980:00 +CPU= 62.256
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001980
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001980
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001980
  14:32:44 STEP 1981 H=1981:00 +CPU= 29.384
  14:33:19 STEP 1982 H=1982:00 +CPU= 33.354
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:34:34 STEP 1983 H=1983:00 +CPU= 51.359
  14:35:07 STEP 1984 H=1984:00 +CPU= 32.597
  14:35:43 STEP 1985 H=1985:00 +CPU= 34.172
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:36:57 STEP 1986 H=1986:00 +CPU= 50.251
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:38:00 STEP 1987 H=1987:00 +CPU= 38.870
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:38:45 STEP 1988 H=1988:00 +CPU= 33.177
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:40:43 STEP 1989 H=1989:00 +CPU= 60.184
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:41:39 STEP 1990 H=1990:00 +CPU= 33.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:42:40 STEP 1991 H=1991:00 +CPU= 37.005
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2010050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0463
(argv4) batchid: 979
(argv5) wuid: 12196107
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /opt/fcs/BOINC/slots/2
Project directory is: /opt/fcs/BOINC/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107
..mkdir for temp folder for results failed
Copying: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6749c83de7e15c85092f8af7cbce001d to: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_0463_2010050100_123_979_12196107.zip
Unzipping the namelist zip file: /opt/fcs/BOINC/slots/2/oifs_43r3_ps_0463_2010050100_123_979_12196107.zip
ic_ancil_file: ic_ancil_12196107
ifsdata_file: ifsdata_12196107
climate_data_file: clim_data_12196107
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_05300276878acbec4b045b4c392406a2 to: /opt/fcs/BOINC/slots/2/ic_ancil_12196107.zip
Unzipping the IC ancils zip file: /opt/fcs/BOINC/slots/2/ic_ancil_12196107.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3262b04ba6c93dc2a1cd8a754e4abfb3 to: /opt/fcs/BOINC/slots/2/ifsdata/ifsdata_12196107.zip
Unzipping the ifsdata_zip file: /opt/fcs/BOINC/slots/2/ifsdata/ifsdata_12196107.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /opt/fcs/BOINC/slots/2/159l_2/clim_data_12196107.zip
Unzipping the climate data zip file: /opt/fcs/BOINC/slots/2/159l_2/clim_data_12196107.zip
Checking for progress XML file: /opt/fcs/BOINC/slots/2/progress_file_12196107.xml
Opened progress file ok : /opt/fcs/BOINC/slots/2/progress_file_12196107.xml
-- Model is restarting --
Adjusting last_iter, 1992, to previous model restart step.
Creating progress file: /opt/fcs/BOINC/slots/2/progress_file_12196107.xml
last_cpu_time: 46611
upload_file_number: 82
last_iter: 1991
last_upload: 7084800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347
The child process has been launched with process id: 846986
Executing the command: /opt/fcs/BOINC/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:newhawk:1:1:846986:846986] [20230126:154526:1674747926:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:47:22 STEP 1992 H=1992:00 +CPU= 59.414
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+001992
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+001992
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+001992
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001968
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001968
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001968
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMGGhq0f+001980
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMSHhq0f+001980
Adding to the zip: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_12196107/ICMUAhq0f+001980
Zipping up the intermediate file: /opt/fcs/BOINC/projects/climateprediction.net/oifs_43r3_ps_0463_2010050100_123_979_12196107_2_r1174966347_82.zip
Uploading the intermediate file: upload_file_82.zip
  15:47:53 STEP 1993 H=1993:00 +CPU= 18.582
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:48:39 STEP 1994 H=1994:00 +CPU= 22.690
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:50:14 STEP 1995 H=1995:00 +CPU= 49.110
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:51:02 STEP 1996 H=1996:00 +CPU= 35.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:52:11 STEP 1997 H=1997:00 +CPU= 34.853
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:53:35 STEP 1998 H=1998:00 +CPU= 48.977
  15:54:11 STEP 1999 H=1999:00 +CPU= 34.723
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:55:23 STEP 2000 H=2000:00 +CPU= 36.915
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:56:55 STEP 2001 H=2001:00 +CPU= 56.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:57:46 STEP 2002 H=2002:00 +CPU= 38.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:45 STEP 2003 H=2003:00 +CPU= 35.850
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:07 STEP 2004 H=2004:00 +CPU= 57.964
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMGGhq0f+002004
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMSHhq0f+002004
Moving to projects directory: /opt/fcs/BOINC/slots/2/ICMUAhq0f+002004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:48 STEP 2005 H=2005:00 +CPU= 28.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:01:37 STEP 2006 H=2006:00 +CPU= 35.823
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:02:44 STEP 2007 H=2007:00 +CPU= 54.731
  16:03:16 STEP 2008 H=2008:00 +CPU= 30.923
  16:03:51 STEP 2009 H=2009:00 +CPU= 33.811
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:05:21 STEP 2010 H=2010:00 +CPU= 54.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:06:05 STEP 2011 H=2011:00 +CPU= 31.830
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:06:50 STEP 2012 H=2012:00 +CPU= 32.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:07:56 STEP 2013 H=2013:00 +CPU= 53.023
  16:08:30 STEP 2014 H=2014:00 +CPU= 32.213
  16:09:03 STEP 2015 H=2015:00 +CPU= 32.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:41:58 1465949 22302939 oifs_43r3_ps_0463_2010050100_123_979_12196107_2 6,480,000 37,977 0.0059
09 May 2023 16:41:52 1465949 22302939 oifs_43r3_ps_0463_2010050100_123_979_12196107_2 5,616,000 32,512 0.0058
09 May 2023 16:41:46 1465949 22302939 oifs_43r3_ps_0463_2010050100_123_979_12196107_2 4,752,000 27,462 0.0058
09 May 2023 16:41:43 1465949 22302939 oifs_43r3_ps_0463_2010050100_123_979_12196107_2 3,888,000 23,063 0.0059


©2024 cpdn.org