Name | oifs_43r3_00sb_2019110100_123_993_12214457_1 |
Workunit | 12214457 |
Created | 26 Mar 2023, 6:57:09 UTC |
Sent | 26 Mar 2023, 6:57:45 UTC |
Report deadline | 25 May 2023, 6:57:45 UTC |
Received | 26 Mar 2023, 19:58:02 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 1534740 |
Run time | 10 hours 35 min |
CPU time | 53 sec |
Validate state | Valid |
Credit | 0.00 |
Device peak FLOPS | 3.20 GFLOPS |
Application version | OpenIFS 43r3 v1.21 x86_64-pc-linux-gnu |
Peak working set size | 4,414.09 MB |
Peak swap size | 4,974.17 MB |
Peak disk usage | 1,258.26 MB |
Stderr | <core_client_version>7.21.0</core_client_version> <![CDATA[ <stderr_txt> ook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135878:135878] [20230327:063349:1679859229:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 06:34:07 STEP 2952 H=2952:00 +CPU= 17.373 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 20.4857879343792 35718 HEX: BFCA0DD061C16CF0 C032466DFA833460 40347C5C991AF335 Air density 1.19864052584200 0.692933494371424 1.56888375802956 35718 HEX: 3FF32DA1B0226D2D 3FE62C82DD141AC0 3FF91A25DAFF6690 w* 0.559897810397319 0.000000000000000E+000 2.21749075026972 35718 HEX: 3FE1EAAED0184598 0000000000000000 4001BD6BCA5CBA45 Sea ice fraction -291.821233622771 -999.000000000000 1.00000000000000 35718 HEX: C0723D23C5DE02CD C08F380000000000 3FF0000000000000 SUB. GETWND : WIND SPEEDS LOWER THAN 1.00000000000000 M/S WERE RESET TO 1.00000000000000 M/S SUB. INITMDL: PREWIND DONE SUB. INITMDL: CIREDUCE DONE I_GET_UNIT: U=140 F=./BLS20191101000000_000122230000.1_1 F=UNFORMATTED A=SEQUENTIAL P=ASIS GETSPEC: RESTART FILENAME SET: T ./BLS20191101000000_000122230000.1_1 SPECTRUM FILE READ IN............... CDTPRO = 20200302230000 SUB. INITMDL: SPECTRA READ IN WAMNORM ON 20200302230000 AVERAGE MINIMUM MAXIMUM SEA POINTS !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302231500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302233000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302234500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200303000000 !!!!!!!!!!!! WVCOUPLE: WAM RESTARTED FOR DATE = 20200302230000 06:34:07 STEP 2952 H=2952:00 +CPU= 17.373 END CNT3 NSTEP = 2952 CNT0 000000000 16.686 16.686 16.758 0.996 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 35/ 804992476 WRITTEN 0/ 0 -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 3 0 35 786125 0 0 0.6 0.6 0.0 0.9 0.0 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 18820.5 18820.5 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 17446.6 17446.6 92.70 0.00 8 SCAN2M - GRID-POINT DYNAMICS 1 1029.4 1029.4 5.47 0.00 10 SCAN2M - PHYSICS 1 5714.4 5714.4 30.36 0.00 12 SPNORM - SPECTRAL NORM COMP. 1 9.4 9.4 0.05 0.00 13 SCAN2M - RADIATION CALC. 1 5797.0 5797.0 30.80 0.00 28 RESTART FILES - READING 1 1144.5 1144.5 6.08 0.00 37 CPGLAG - SL COMPUTATIONS 1 1472.3 1472.3 7.82 0.00 38 WAM - TOTAL COST OF WAVE MODEL 1 2306.2 2306.2 12.25 0.00 39 SU0YOMB 1 158.0 158.0 0.84 0.00 51 SCAN2M - SL COMM. PART 1 1 6.0 6.0 0.03 0.00 56 SPNORM - SPECTRAL NORM COMM. 1 0.0 0.0 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 6 40.2 40.2 1.28 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 6 13.2 13.2 0.42 0.00 140 SULEG - COMP. OF LEGENDRE POL. 2 9.7 9.7 0.10 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 6 4.6 4.6 0.15 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 6 8.8 8.8 0.28 0.00 400 GSTATS 1260 0.0 0.0 0.00 0.00 401 GSTATS HOOK 1137 0.0 0.0 0.01 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 18.8 CPU TIME 18.7 VECTOR TIME 18.7 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 609127.3 *** END CNT0 *** ------------------------------------------------ Adding to the zip: /home/michael/media/BOINC/slots/6/NODE.001_01 Adding to the zip: /home/michael/media/BOINC/slots/6/ifs.stat Zipping up the final file: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020_0.zip Uploading the final file: upload_file_0.zip Uploading trickle at timestep: 0 (argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu (argv1) start_date: 2019110100 (argv2) exptid: hnw4 (argv3) unique_member_id: 00sb (argv4) batchid: 993 (argv5) wuid: 12214457 (argv6) fclen: 123 (argv7) app_name: oifs_43r3 (argv8) nthreads: 1 Working directory is: /home/michael/media/BOINC/slots/6 Project directory is: /home/michael/media/BOINC/projects/climateprediction.net/ app name: oifs_43r3 version: 1.21 Location of temp folder: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_12214457 Copying: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_500c7fb1a5e11b8824947c7e40bdce7d to: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip Unzipping the namelist zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip ic_ancil_file: ic_ancil_12214457 ifsdata_file: ifsdata_12214457 climate_data_file: clim_data_12214457 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_dec701a483f2d997c6243a0d01080bd4 to: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip Unzipping the IC ancils zip file: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7a5856a2e9a60c83897a4fb22bc226b1 to: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip Unzipping the ifsdata_zip file: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Unzipping the climate data zip file: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Checking for progress XML file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml Opened progress file ok : /home/michael/media/BOINC/slots/6/progress_file_12214457.xml -- Model is restarting -- Adjusting last_iter, 2952, to previous model restart step. Creating progress file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml last_cpu_time: 9 upload_file_number: 0 last_iter: 2953 last_upload: 0 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020 The child process has been launched with process id: 135901 Executing the command: /home/michael/media/BOINC/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135901:135901] [20230327:063803:1679859483:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 06:38:21 STEP 2952 H=2952:00 +CPU= 17.397 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 20.4857879343792 35718 HEX: BFCA0DD061C16CF0 C032466DFA833460 40347C5C991AF335 Air density 1.19864052584200 0.692933494371424 1.56888375802956 35718 HEX: 3FF32DA1B0226D2D 3FE62C82DD141AC0 3FF91A25DAFF6690 w* 0.559897810397319 0.000000000000000E+000 2.21749075026972 35718 HEX: 3FE1EAAED0184598 0000000000000000 4001BD6BCA5CBA45 Sea ice fraction -291.821233622771 -999.000000000000 1.00000000000000 35718 HEX: C0723D23C5DE02CD C08F380000000000 3FF0000000000000 SUB. GETWND : WIND SPEEDS LOWER THAN 1.00000000000000 M/S WERE RESET TO 1.00000000000000 M/S SUB. INITMDL: PREWIND DONE SUB. INITMDL: CIREDUCE DONE I_GET_UNIT: U=140 F=./BLS20191101000000_000122230000.1_1 F=UNFORMATTED A=SEQUENTIAL P=ASIS GETSPEC: RESTART FILENAME SET: T ./BLS20191101000000_000122230000.1_1 SPECTRUM FILE READ IN............... CDTPRO = 20200302230000 SUB. INITMDL: SPECTRA READ IN WAMNORM ON 20200302230000 AVERAGE MINIMUM MAXIMUM SEA POINTS !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302231500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302233000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302234500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200303000000 !!!!!!!!!!!! WVCOUPLE: WAM RESTARTED FOR DATE = 20200302230000 06:38:21 STEP 2952 H=2952:00 +CPU= 17.397 END CNT3 NSTEP = 2952 CNT0 000000000 16.715 16.715 16.781 0.996 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 35/ 804992476 WRITTEN 0/ 0 -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 3 0 35 786125 0 0 0.6 0.6 0.0 0.9 0.0 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 18835.5 18835.5 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 17465.2 17465.2 92.72 0.00 8 SCAN2M - GRID-POINT DYNAMICS 1 1020.6 1020.6 5.42 0.00 10 SCAN2M - PHYSICS 1 5753.7 5753.7 30.55 0.00 12 SPNORM - SPECTRAL NORM COMP. 1 8.7 8.7 0.05 0.00 13 SCAN2M - RADIATION CALC. 1 5784.8 5784.8 30.71 0.00 28 RESTART FILES - READING 1 1117.3 1117.3 5.93 0.00 37 CPGLAG - SL COMPUTATIONS 1 1476.4 1476.4 7.84 0.00 38 WAM - TOTAL COST OF WAVE MODEL 1 2309.3 2309.3 12.26 0.00 39 SU0YOMB 1 181.0 181.0 0.96 0.00 51 SCAN2M - SL COMM. PART 1 1 5.5 5.5 0.03 0.00 56 SPNORM - SPECTRAL NORM COMM. 1 0.1 0.1 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 6 40.7 40.7 1.29 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 6 12.9 12.9 0.41 0.00 140 SULEG - COMP. OF LEGENDRE POL. 2 9.4 9.4 0.10 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 6 4.5 4.5 0.14 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 6 8.8 8.8 0.28 0.00 400 GSTATS 1260 0.0 0.0 0.00 0.00 401 GSTATS HOOK 1137 0.0 0.0 0.00 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 18.8 CPU TIME 18.7 VECTOR TIME 18.7 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 608478.6 *** END CNT0 *** ------------------------------------------------ Adding to the zip: /home/michael/media/BOINC/slots/6/NODE.001_01 Adding to the zip: /home/michael/media/BOINC/slots/6/ifs.stat Zipping up the final file: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020_0.zip Uploading the final file: upload_file_0.zip Uploading trickle at timestep: 10630800 (argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu (argv1) start_date: 2019110100 (argv2) exptid: hnw4 (argv3) unique_member_id: 00sb (argv4) batchid: 993 (argv5) wuid: 12214457 (argv6) fclen: 123 (argv7) app_name: oifs_43r3 (argv8) nthreads: 1 Working directory is: /home/michael/media/BOINC/slots/6 Project directory is: /home/michael/media/BOINC/projects/climateprediction.net/ app name: oifs_43r3 version: 1.21 Location of temp folder: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_12214457 Copying: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_500c7fb1a5e11b8824947c7e40bdce7d to: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip Unzipping the namelist zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip ic_ancil_file: ic_ancil_12214457 ifsdata_file: ifsdata_12214457 climate_data_file: clim_data_12214457 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_dec701a483f2d997c6243a0d01080bd4 to: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip Unzipping the IC ancils zip file: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7a5856a2e9a60c83897a4fb22bc226b1 to: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip Unzipping the ifsdata_zip file: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Unzipping the climate data zip file: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Checking for progress XML file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml Opened progress file ok : /home/michael/media/BOINC/slots/6/progress_file_12214457.xml -- Model is restarting -- Adjusting last_iter, 2952, to previous model restart step. Creating progress file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml last_cpu_time: 17 upload_file_number: 0 last_iter: 2953 last_upload: 0 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020 The child process has been launched with process id: 135925 Executing the command: /home/michael/media/BOINC/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135925:135925] [20230327:064058:1679859658:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 06:41:17 STEP 2952 H=2952:00 +CPU= 17.459 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 20.4857879343792 35718 HEX: BFCA0DD061C16CF0 C032466DFA833460 40347C5C991AF335 Air density 1.19864052584200 0.692933494371424 1.56888375802956 35718 HEX: 3FF32DA1B0226D2D 3FE62C82DD141AC0 3FF91A25DAFF6690 w* 0.559897810397319 0.000000000000000E+000 2.21749075026972 35718 HEX: 3FE1EAAED0184598 0000000000000000 4001BD6BCA5CBA45 Sea ice fraction -291.821233622771 -999.000000000000 1.00000000000000 35718 HEX: C0723D23C5DE02CD C08F380000000000 3FF0000000000000 SUB. GETWND : WIND SPEEDS LOWER THAN 1.00000000000000 M/S WERE RESET TO 1.00000000000000 M/S SUB. INITMDL: PREWIND DONE SUB. INITMDL: CIREDUCE DONE I_GET_UNIT: U=140 F=./BLS20191101000000_000122230000.1_1 F=UNFORMATTED A=SEQUENTIAL P=ASIS GETSPEC: RESTART FILENAME SET: T ./BLS20191101000000_000122230000.1_1 SPECTRUM FILE READ IN............... CDTPRO = 20200302230000 SUB. INITMDL: SPECTRA READ IN WAMNORM ON 20200302230000 AVERAGE MINIMUM MAXIMUM SEA POINTS !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302231500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302233000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302234500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200303000000 !!!!!!!!!!!! WVCOUPLE: WAM RESTARTED FOR DATE = 20200302230000 06:41:17 STEP 2952 H=2952:00 +CPU= 17.459 END CNT3 NSTEP = 2952 CNT0 000000000 16.782 16.782 16.834 0.997 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 35/ 804992476 WRITTEN 0/ 0 -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 3 0 35 786125 0 0 0.6 0.6 0.0 0.8 0.0 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 18824.9 18824.9 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 17514.1 17514.1 93.04 0.00 8 SCAN2M - GRID-POINT DYNAMICS 1 1041.6 1041.6 5.53 0.00 10 SCAN2M - PHYSICS 1 5768.6 5768.6 30.64 0.00 12 SPNORM - SPECTRAL NORM COMP. 1 8.3 8.3 0.04 0.00 13 SCAN2M - RADIATION CALC. 1 5808.9 5808.9 30.86 0.00 28 RESTART FILES - READING 1 1086.6 1086.6 5.77 0.00 37 CPGLAG - SL COMPUTATIONS 1 1534.5 1534.5 8.15 0.00 38 WAM - TOTAL COST OF WAVE MODEL 1 2251.5 2251.5 11.96 0.00 39 SU0YOMB 1 153.2 153.2 0.81 0.00 51 SCAN2M - SL COMM. PART 1 1 6.5 6.5 0.03 0.00 56 SPNORM - SPECTRAL NORM COMM. 1 0.1 0.1 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 6 39.4 39.4 1.26 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 6 12.9 12.9 0.41 0.00 140 SULEG - COMP. OF LEGENDRE POL. 2 9.4 9.4 0.10 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 6 4.7 4.7 0.15 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 6 8.3 8.3 0.26 0.00 400 GSTATS 1260 0.0 0.0 0.00 0.00 401 GSTATS HOOK 1137 0.0 0.0 0.00 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 18.8 CPU TIME 18.8 VECTOR TIME 18.8 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 606779.7 *** END CNT0 *** ------------------------------------------------ Adding to the zip: /home/michael/media/BOINC/slots/6/NODE.001_01 Adding to the zip: /home/michael/media/BOINC/slots/6/ifs.stat Zipping up the final file: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020_0.zip Uploading the final file: upload_file_0.zip Uploading trickle at timestep: 10630800 (argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu (argv1) start_date: 2019110100 (argv2) exptid: hnw4 (argv3) unique_member_id: 00sb (argv4) batchid: 993 (argv5) wuid: 12214457 (argv6) fclen: 123 (argv7) app_name: oifs_43r3 (argv8) nthreads: 1 Working directory is: /home/michael/media/BOINC/slots/6 Project directory is: /home/michael/media/BOINC/projects/climateprediction.net/ app name: oifs_43r3 version: 1.21 Location of temp folder: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_12214457 Copying: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_500c7fb1a5e11b8824947c7e40bdce7d to: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip Unzipping the namelist zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip ic_ancil_file: ic_ancil_12214457 ifsdata_file: ifsdata_12214457 climate_data_file: clim_data_12214457 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_dec701a483f2d997c6243a0d01080bd4 to: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip Unzipping the IC ancils zip file: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7a5856a2e9a60c83897a4fb22bc226b1 to: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip Unzipping the ifsdata_zip file: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Unzipping the climate data zip file: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Checking for progress XML file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml Opened progress file ok : /home/michael/media/BOINC/slots/6/progress_file_12214457.xml -- Model is restarting -- Adjusting last_iter, 2952, to previous model restart step. Creating progress file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml last_cpu_time: 26 upload_file_number: 0 last_iter: 2953 last_upload: 0 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020 The child process has been launched with process id: 135999 Executing the command: /home/michael/media/BOINC/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:135999:135999] [20230327:064659:1679860019:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 06:47:18 STEP 2952 H=2952:00 +CPU= 17.357 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 20.4857879343792 35718 HEX: BFCA0DD061C16CF0 C032466DFA833460 40347C5C991AF335 Air density 1.19864052584200 0.692933494371424 1.56888375802956 35718 HEX: 3FF32DA1B0226D2D 3FE62C82DD141AC0 3FF91A25DAFF6690 w* 0.559897810397319 0.000000000000000E+000 2.21749075026972 35718 HEX: 3FE1EAAED0184598 0000000000000000 4001BD6BCA5CBA45 Sea ice fraction -291.821233622771 -999.000000000000 1.00000000000000 35718 HEX: C0723D23C5DE02CD C08F380000000000 3FF0000000000000 SUB. GETWND : WIND SPEEDS LOWER THAN 1.00000000000000 M/S WERE RESET TO 1.00000000000000 M/S SUB. INITMDL: PREWIND DONE SUB. INITMDL: CIREDUCE DONE I_GET_UNIT: U=140 F=./BLS20191101000000_000122230000.1_1 F=UNFORMATTED A=SEQUENTIAL P=ASIS GETSPEC: RESTART FILENAME SET: T ./BLS20191101000000_000122230000.1_1 SPECTRUM FILE READ IN............... CDTPRO = 20200302230000 SUB. INITMDL: SPECTRA READ IN WAMNORM ON 20200302230000 AVERAGE MINIMUM MAXIMUM SEA POINTS !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302231500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302233000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302234500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200303000000 !!!!!!!!!!!! WVCOUPLE: WAM RESTARTED FOR DATE = 20200302230000 06:47:18 STEP 2952 H=2952:00 +CPU= 17.357 END CNT3 NSTEP = 2952 CNT0 000000000 16.684 16.684 16.760 0.995 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 35/ 804992476 WRITTEN 0/ 0 -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 3 0 35 786125 0 0 0.6 0.6 0.0 0.9 0.0 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 18775.1 18775.1 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 17438.5 17438.5 92.88 0.00 8 SCAN2M - GRID-POINT DYNAMICS 1 1022.8 1022.8 5.45 0.00 10 SCAN2M - PHYSICS 1 5756.5 5756.5 30.66 0.00 12 SPNORM - SPECTRAL NORM COMP. 1 9.0 9.0 0.05 0.00 13 SCAN2M - RADIATION CALC. 1 5758.3 5758.3 30.67 0.00 28 RESTART FILES - READING 1 1117.4 1117.4 5.95 0.00 37 CPGLAG - SL COMPUTATIONS 1 1461.9 1461.9 7.79 0.00 38 WAM - TOTAL COST OF WAVE MODEL 1 2326.9 2326.9 12.39 0.00 39 SU0YOMB 1 150.3 150.3 0.80 0.00 51 SCAN2M - SL COMM. PART 1 1 6.1 6.1 0.03 0.00 56 SPNORM - SPECTRAL NORM COMM. 1 0.0 0.0 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 6 39.7 39.7 1.27 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 6 13.2 13.2 0.42 0.00 140 SULEG - COMP. OF LEGENDRE POL. 2 9.4 9.4 0.10 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 6 4.7 4.7 0.15 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 6 8.7 8.7 0.28 0.00 400 GSTATS 1260 0.0 0.0 0.00 0.00 401 GSTATS HOOK 1137 0.0 0.0 0.00 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 18.8 CPU TIME 18.7 VECTOR TIME 18.7 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 609410.2 *** END CNT0 *** ------------------------------------------------ Adding to the zip: /home/michael/media/BOINC/slots/6/NODE.001_01 Adding to the zip: /home/michael/media/BOINC/slots/6/ifs.stat Zipping up the final file: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020_0.zip Uploading the final file: upload_file_0.zip (argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu (argv1) start_date: 2019110100 (argv2) exptid: hnw4 (argv3) unique_member_id: 00sb (argv4) batchid: 993 (argv5) wuid: 12214457 (argv6) fclen: 123 (argv7) app_name: oifs_43r3 (argv8) nthreads: 1 Working directory is: /home/michael/media/BOINC/slots/6 Project directory is: /home/michael/media/BOINC/projects/climateprediction.net/ app name: oifs_43r3 version: 1.21 Location of temp folder: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_12214457 ..mkdir for temp folder for results failed Copying: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_500c7fb1a5e11b8824947c7e40bdce7d to: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip Unzipping the namelist zip file: /home/michael/media/BOINC/slots/6/oifs_43r3_00sb_2019110100_123_993_12214457.zip ic_ancil_file: ic_ancil_12214457 ifsdata_file: ifsdata_12214457 climate_data_file: clim_data_12214457 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_dec701a483f2d997c6243a0d01080bd4 to: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip Unzipping the IC ancils zip file: /home/michael/media/BOINC/slots/6/ic_ancil_12214457.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7a5856a2e9a60c83897a4fb22bc226b1 to: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip Unzipping the ifsdata_zip file: /home/michael/media/BOINC/slots/6/ifsdata/ifsdata_12214457.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Unzipping the climate data zip file: /home/michael/media/BOINC/slots/6/159l_2/clim_data_12214457.zip Checking for progress XML file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml Opened progress file ok : /home/michael/media/BOINC/slots/6/progress_file_12214457.xml -- Model is restarting -- Adjusting last_iter, 2952, to previous model restart step. Creating progress file: /home/michael/media/BOINC/slots/6/progress_file_12214457.xml last_cpu_time: 35 upload_file_number: 0 last_iter: 2953 last_upload: 0 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020 The child process has been launched with process id: 136014 Executing the command: /home/michael/media/BOINC/slots/6/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil)) [EC_DRHOOK:Zen:1:1:136014:136014] [20230327:064839:1679860119:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 06:48:57 STEP 2952 H=2952:00 +CPU= 16.877 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 20.4857879343792 35718 HEX: BFCA0DD061C16CF0 C032466DFA833460 40347C5C991AF335 Air density 1.19864052584200 0.692933494371424 1.56888375802956 35718 HEX: 3FF32DA1B0226D2D 3FE62C82DD141AC0 3FF91A25DAFF6690 w* 0.559897810397319 0.000000000000000E+000 2.21749075026972 35718 HEX: 3FE1EAAED0184598 0000000000000000 4001BD6BCA5CBA45 Sea ice fraction -291.821233622771 -999.000000000000 1.00000000000000 35718 HEX: C0723D23C5DE02CD C08F380000000000 3FF0000000000000 SUB. GETWND : WIND SPEEDS LOWER THAN 1.00000000000000 M/S WERE RESET TO 1.00000000000000 M/S SUB. INITMDL: PREWIND DONE SUB. INITMDL: CIREDUCE DONE I_GET_UNIT: U=140 F=./BLS20191101000000_000122230000.1_1 F=UNFORMATTED A=SEQUENTIAL P=ASIS GETSPEC: RESTART FILENAME SET: T ./BLS20191101000000_000122230000.1_1 SPECTRUM FILE READ IN............... CDTPRO = 20200302230000 SUB. INITMDL: SPECTRA READ IN WAMNORM ON 20200302230000 AVERAGE MINIMUM MAXIMUM SEA POINTS !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302231500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302233000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302234500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200303000000 !!!!!!!!!!!! WVCOUPLE: WAM RESTARTED FOR DATE = 20200302230000 06:48:57 STEP 2952 H=2952:00 +CPU= 16.877 END CNT3 NSTEP = 2952 CNT0 000000000 16.206 16.206 16.237 0.998 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 35/ 804992476 WRITTEN 0/ 0 -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 3 0 35 786125 0 0 0.6 0.6 0.0 0.9 0.0 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 18257.4 18257.4 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 16912.8 16912.8 92.64 0.00 8 SCAN2M - GRID-POINT DYNAMICS 1 997.5 997.5 5.46 0.00 10 SCAN2M - PHYSICS 1 5608.9 5608.9 30.72 0.00 12 SPNORM - SPECTRAL NORM COMP. 1 8.4 8.4 0.05 0.00 13 SCAN2M - RADIATION CALC. 1 5600.2 5600.2 30.67 0.00 28 RESTART FILES - READING 1 1120.9 1120.9 6.14 0.00 37 CPGLAG - SL COMPUTATIONS 1 1408.3 1408.3 7.71 0.00 38 WAM - TOTAL COST OF WAVE MODEL 1 2217.4 2217.4 12.15 0.00 39 SU0YOMB 1 154.7 154.7 0.85 0.00 51 SCAN2M - SL COMM. PART 1 1 5.8 5.8 0.03 0.00 56 SPNORM - SPECTRAL NORM COMM. 1 0.1 0.1 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 6 36.7 36.7 1.21 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 6 12.2 12.2 0.40 0.00 140 SULEG - COMP. OF LEGENDRE POL. 2 9.1 9.1 0.10 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 6 4.4 4.4 0.15 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 6 8.4 8.4 0.28 0.00 400 GSTATS 1260 0.0 0.0 0.00 0.00 401 GSTATS HOOK 1137 0.0 0.0 0.00 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 18.3 CPU TIME 18.2 VECTOR TIME 18.2 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 628352.5 *** END CNT0 *** ------------------------------------------------ Adding to the zip: /home/michael/media/BOINC/slots/6/NODE.001_01 Adding to the zip: /home/michael/media/BOINC/slots/6/ifs.stat Zipping up the final file: /home/michael/media/BOINC/projects/climateprediction.net/oifs_43r3_00sb_2019110100_123_993_12214457_1_r1646478020_0.zip Uploading the final file: upload_file_0.zip Uploading trickle at timestep: 10630800 06:52:18 (136011): called boinc_finish(0) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 10,630,800 | 27 | 0.0000 |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 10,630,800 | 53 | 0.0000 |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 10,630,800 | 35 | 0.0000 |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 10,630,800 | 36,788 | 0.0035 |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 10,630,800 | 26 | 0.0000 |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 10,368,000 | 35,887 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 9,504,000 | 32,916 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 8,640,000 | 29,955 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 7,776,000 | 26,966 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 6,912,000 | 23,957 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 6,048,000 | 20,959 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 5,184,000 | 17,981 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 4,320,000 | 14,992 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 3,456,000 | 11,976 | 0.0035 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 2,592,000 | 8,923 | 0.0034 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 1,728,000 | 5,949 | 0.0034 |
09 May 2023 16:55:47 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 864,000 | 2,980 | 0.0034 |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 0 | 18 | 0.0000 |
09 May 2023 16:55:48 | 1534740 | 22317672 | oifs_43r3_00sb_2019110100_123_993_12214457_1 | 0 | 18 | 0.0000 |
©2024 cpdn.org