Name | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 |
Workunit | 12195670 |
Created | 23 Dec 2022, 22:35:23 UTC |
Sent | 10 Jan 2023, 19:44:12 UTC |
Report deadline | 9 Feb 2023, 19:44:12 UTC |
Received | 13 Jan 2023, 17:00:42 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 1519251 |
Run time | 19 hours 25 min 37 sec |
CPU time | 19 hours 19 min 35 sec |
Validate state | Valid |
Credit | 14,873.04 |
Device peak FLOPS | 4.94 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,415.15 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 3,707.94 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <stderr_txt> request received from the BOINC client, suspending the child process Resuming the child process 06:24:05 STEP 2775 H=2775:00 +CPU= 33.560 06:24:27 STEP 2776 H=2776:00 +CPU= 20.978 06:24:47 STEP 2777 H=2777:00 +CPU= 20.433 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:26:52 STEP 2778 H=2778:00 +CPU= 33.566 06:27:13 STEP 2779 H=2779:00 +CPU= 20.683 06:27:35 STEP 2780 H=2780:00 +CPU= 21.599 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:29:38 STEP 2781 H=2781:00 +CPU= 32.607 06:30:00 STEP 2782 H=2782:00 +CPU= 21.092 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:31:33 STEP 2783 H=2783:00 +CPU= 20.853 06:32:12 STEP 2784 H=2784:00 +CPU= 39.095 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002784 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002784 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002772 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002772 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002772 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_115.zip Uploading the intermediate file: upload_file_115.zip 06:32:31 STEP 2785 H=2785:00 +CPU= 19.208 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:34:09 STEP 2786 H=2786:00 +CPU= 21.503 06:34:42 STEP 2787 H=2787:00 +CPU= 33.183 06:35:04 STEP 2788 H=2788:00 +CPU= 21.522 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:37:16 STEP 2789 H=2789:00 +CPU= 20.683 06:37:50 STEP 2790 H=2790:00 +CPU= 33.633 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:39:13 STEP 2791 H=2791:00 +CPU= 20.684 06:39:33 STEP 2792 H=2792:00 +CPU= 20.512 06:40:06 STEP 2793 H=2793:00 +CPU= 32.265 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:41:47 STEP 2794 H=2794:00 +CPU= 20.458 06:42:08 STEP 2795 H=2795:00 +CPU= 20.176 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:43:56 STEP 2796 H=2796:00 +CPU= 36.634 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002796 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002796 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002796 06:44:16 STEP 2797 H=2797:00 +CPU= 20.600 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:45:48 STEP 2798 H=2798:00 +CPU= 20.492 06:46:22 STEP 2799 H=2799:00 +CPU= 33.802 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:48:35 STEP 2800 H=2800:00 +CPU= 21.258 06:48:57 STEP 2801 H=2801:00 +CPU= 21.970 06:49:31 STEP 2802 H=2802:00 +CPU= 33.848 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:51:23 STEP 2803 H=2803:00 +CPU= 20.331 06:51:45 STEP 2804 H=2804:00 +CPU= 21.075 06:52:18 STEP 2805 H=2805:00 +CPU= 32.783 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:53:50 STEP 2806 H=2806:00 +CPU= 21.210 06:54:11 STEP 2807 H=2807:00 +CPU= 20.532 06:54:49 STEP 2808 H=2808:00 +CPU= 37.973 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002808 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002808 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002796 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002796 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002796 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_116.zip Uploading the intermediate file: upload_file_116.zip 06:55:08 STEP 2809 H=2809:00 +CPU= 19.037 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:56:50 STEP 2810 H=2810:00 +CPU= 21.038 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:58:25 STEP 2811 H=2811:00 +CPU= 33.182 06:58:46 STEP 2812 H=2812:00 +CPU= 21.569 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:00:18 STEP 2813 H=2813:00 +CPU= 20.188 07:00:51 STEP 2814 H=2814:00 +CPU= 32.953 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:01:53 STEP 2815 H=2815:00 +CPU= 20.837 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:03:26 STEP 2816 H=2816:00 +CPU= 21.300 07:03:58 STEP 2817 H=2817:00 +CPU= 31.830 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:05:30 STEP 2818 H=2818:00 +CPU= 21.466 07:05:50 STEP 2819 H=2819:00 +CPU= 20.502 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:07:38 STEP 2820 H=2820:00 +CPU= 35.730 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002820 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002820 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002820 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:09:00 STEP 2821 H=2821:00 +CPU= 20.403 07:09:21 STEP 2822 H=2822:00 +CPU= 21.397 07:09:54 STEP 2823 H=2823:00 +CPU= 32.241 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:11:36 STEP 2824 H=2824:00 +CPU= 21.657 07:11:57 STEP 2825 H=2825:00 +CPU= 20.017 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:13:22 STEP 2826 H=2826:00 +CPU= 33.492 07:13:43 STEP 2827 H=2827:00 +CPU= 20.596 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:15:14 STEP 2828 H=2828:00 +CPU= 20.981 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:16:58 STEP 2829 H=2829:00 +CPU= 32.776 07:17:19 STEP 2830 H=2830:00 +CPU= 21.439 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:18:42 STEP 2831 H=2831:00 +CPU= 21.109 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:20:12 STEP 2832 H=2832:00 +CPU= 38.675 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002832 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002832 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002820 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002820 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002820 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_117.zip Uploading the intermediate file: upload_file_117.zip 07:20:32 STEP 2833 H=2833:00 +CPU= 20.462 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:21:39 STEP 2834 H=2834:00 +CPU= 20.394 07:22:11 STEP 2835 H=2835:00 +CPU= 32.273 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:23:43 STEP 2836 H=2836:00 +CPU= 20.488 07:24:04 STEP 2837 H=2837:00 +CPU= 21.499 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:25:38 STEP 2838 H=2838:00 +CPU= 33.223 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:26:58 STEP 2839 H=2839:00 +CPU= 20.681 07:27:19 STEP 2840 H=2840:00 +CPU= 20.987 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:29:04 STEP 2841 H=2841:00 +CPU= 32.305 07:29:26 STEP 2842 H=2842:00 +CPU= 21.752 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:30:47 STEP 2843 H=2843:00 +CPU= 20.514 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:32:25 STEP 2844 H=2844:00 +CPU= 36.504 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002844 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002844 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002844 07:32:46 STEP 2845 H=2845:00 +CPU= 20.650 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:34:18 STEP 2846 H=2846:00 +CPU= 20.911 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:35:42 STEP 2847 H=2847:00 +CPU= 33.330 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:37:15 STEP 2848 H=2848:00 +CPU= 21.089 07:37:35 STEP 2849 H=2849:00 +CPU= 20.361 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:40:10 STEP 2850 H=2850:00 +CPU= 33.163 07:40:31 STEP 2851 H=2851:00 +CPU= 20.693 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:42:14 STEP 2852 H=2852:00 +CPU= 21.374 07:42:46 STEP 2853 H=2853:00 +CPU= 32.165 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:44:49 STEP 2854 H=2854:00 +CPU= 21.489 07:45:09 STEP 2855 H=2855:00 +CPU= 20.872 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:47:00 STEP 2856 H=2856:00 +CPU= 38.673 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002856 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002856 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002844 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002844 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002844 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_118.zip Uploading the intermediate file: upload_file_118.zip 07:47:19 STEP 2857 H=2857:00 +CPU= 18.506 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:48:26 STEP 2858 H=2858:00 +CPU= 20.972 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:50:00 STEP 2859 H=2859:00 +CPU= 33.193 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:51:33 STEP 2860 H=2860:00 +CPU= 21.280 07:51:54 STEP 2861 H=2861:00 +CPU= 21.587 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:53:19 STEP 2862 H=2862:00 +CPU= 34.604 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:54:41 STEP 2863 H=2863:00 +CPU= 20.775 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:56:05 STEP 2864 H=2864:00 +CPU= 21.815 07:56:38 STEP 2865 H=2865:00 +CPU= 32.517 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:58:10 STEP 2866 H=2866:00 +CPU= 20.858 07:58:30 STEP 2867 H=2867:00 +CPU= 20.178 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:00:08 STEP 2868 H=2868:00 +CPU= 36.492 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002868 Quit request received from BOINC client, ending the child process ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2010050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0026 (argv4) batchid: 979 (argv5) wuid: 12195670 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_75402b9259fe5d2bdf564213787f7884 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0026_2010050100_123_979_12195670.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0026_2010050100_123_979_12195670.zip ic_ancil_file: ic_ancil_12195670 ifsdata_file: ifsdata_12195670 climate_data_file: clim_data_12195670 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_a9f0cb7682b4e38df14999ab41d0e971 to: /var/lib/boinc-client/slots/0/ic_ancil_12195670.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12195670.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3748859929ec4965f83945320a644dfd to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12195670.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12195670.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12195670.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12195670.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12195670.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12195670.xml -- Model is restarting -- Adjusting last_iter, 2869, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12195670.xml last_cpu_time: 66367 upload_file_number: 119 last_iter: 2855 last_upload: 10281600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490 The child process has been launched with process id: 1382834 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1382834:1382834] [20230112:080345:1673478225:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:05:29 STEP 2856 H=2856:00 +CPU= 38.258 08:05:50 STEP 2857 H=2857:00 +CPU= 20.527 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:07:12 STEP 2858 H=2858:00 +CPU= 21.562 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:08:47 STEP 2859 H=2859:00 +CPU= 32.411 08:09:08 STEP 2860 H=2860:00 +CPU= 21.093 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:10:50 STEP 2861 H=2861:00 +CPU= 20.926 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:12:34 STEP 2862 H=2862:00 +CPU= 33.676 08:12:56 STEP 2863 H=2863:00 +CPU= 21.607 08:13:16 STEP 2864 H=2864:00 +CPU= 20.763 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:15:21 STEP 2865 H=2865:00 +CPU= 32.688 08:15:42 STEP 2866 H=2866:00 +CPU= 20.994 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:17:43 STEP 2867 H=2867:00 +CPU= 20.009 08:18:19 STEP 2868 H=2868:00 +CPU= 34.964 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002868 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:19:49 STEP 2869 H=2869:00 +CPU= 19.993 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:20:52 STEP 2870 H=2870:00 +CPU= 22.054 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:22:37 STEP 2871 H=2871:00 +CPU= 32.559 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:24:07 STEP 2872 H=2872:00 +CPU= 20.809 08:24:29 STEP 2873 H=2873:00 +CPU= 21.125 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:26:13 STEP 2874 H=2874:00 +CPU= 33.055 08:26:34 STEP 2875 H=2875:00 +CPU= 20.908 08:26:55 STEP 2876 H=2876:00 +CPU= 20.563 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:28:58 STEP 2877 H=2877:00 +CPU= 32.240 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:30:30 STEP 2878 H=2878:00 +CPU= 20.992 08:30:51 STEP 2879 H=2879:00 +CPU= 21.173 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:32:51 STEP 2880 H=2880:00 +CPU= 37.988 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002880 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002880 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002868 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002868 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002868 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_119.zip Uploading the intermediate file: upload_file_119.zip 08:33:10 STEP 2881 H=2881:00 +CPU= 19.329 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:34:53 STEP 2882 H=2882:00 +CPU= 20.539 08:35:26 STEP 2883 H=2883:00 +CPU= 33.164 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:37:08 STEP 2884 H=2884:00 +CPU= 21.405 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:38:41 STEP 2885 H=2885:00 +CPU= 21.139 08:39:14 STEP 2886 H=2886:00 +CPU= 33.186 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:40:46 STEP 2887 H=2887:00 +CPU= 20.936 08:41:08 STEP 2888 H=2888:00 +CPU= 21.718 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:42:52 STEP 2889 H=2889:00 +CPU= 33.325 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:44:16 STEP 2890 H=2890:00 +CPU= 21.925 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:46:17 STEP 2891 H=2891:00 +CPU= 20.730 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:47:46 STEP 2892 H=2892:00 +CPU= 36.375 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002892 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002892 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002892 08:48:07 STEP 2893 H=2893:00 +CPU= 21.180 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:49:29 STEP 2894 H=2894:00 +CPU= 21.207 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:51:23 STEP 2895 H=2895:00 +CPU= 32.850 08:51:45 STEP 2896 H=2896:00 +CPU= 21.905 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:53:57 STEP 2897 H=2897:00 +CPU= 21.679 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:56:13 STEP 2898 H=2898:00 +CPU= 34.570 08:56:35 STEP 2899 H=2899:00 +CPU= 21.798 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:58:28 STEP 2900 H=2900:00 +CPU= 21.434 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:00:23 STEP 2901 H=2901:00 +CPU= 33.277 09:00:45 STEP 2902 H=2902:00 +CPU= 21.661 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:02:27 STEP 2903 H=2903:00 +CPU= 21.403 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:04:37 STEP 2904 H=2904:00 +CPU= 38.278 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002904 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002904 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002892 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002892 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002892 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_120.zip Uploading the intermediate file: upload_file_120.zip 09:04:58 STEP 2905 H=2905:00 +CPU= 21.091 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:06:34 STEP 2906 H=2906:00 +CPU= 20.517 09:07:07 STEP 2907 H=2907:00 +CPU= 32.705 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:09:11 STEP 2908 H=2908:00 +CPU= 22.670 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:11:14 STEP 2909 H=2909:00 +CPU= 21.742 09:11:48 STEP 2910 H=2910:00 +CPU= 33.939 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:13:31 STEP 2911 H=2911:00 +CPU= 21.860 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:28:56 STEP 2912 H=2912:00 +CPU= 21.759 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:31:01 STEP 2913 H=2913:00 +CPU= 33.373 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:33:15 STEP 2914 H=2914:00 +CPU= 22.032 09:33:36 STEP 2915 H=2915:00 +CPU= 21.046 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:35:53 STEP 2916 H=2916:00 +CPU= 35.991 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002916 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:37:46 STEP 2917 H=2917:00 +CPU= 20.926 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:39:19 STEP 2918 H=2918:00 +CPU= 21.339 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:41:33 STEP 2919 H=2919:00 +CPU= 33.035 09:41:55 STEP 2920 H=2920:00 +CPU= 21.928 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:43:57 STEP 2921 H=2921:00 +CPU= 21.246 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:46:45 STEP 2922 H=2922:00 +CPU= 34.834 09:47:06 STEP 2923 H=2923:00 +CPU= 21.460 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:49:29 STEP 2924 H=2924:00 +CPU= 21.642 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:51:54 STEP 2925 H=2925:00 +CPU= 33.497 09:52:17 STEP 2926 H=2926:00 +CPU= 22.296 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:54:49 STEP 2927 H=2927:00 +CPU= 21.083 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:56:40 STEP 2928 H=2928:00 +CPU= 39.232 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002928 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002928 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002916 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002916 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002916 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_121.zip Uploading the intermediate file: upload_file_121.zip 09:57:01 STEP 2929 H=2929:00 +CPU= 20.715 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:58:54 STEP 2930 H=2930:00 +CPU= 21.355 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:01:19 STEP 2931 H=2931:00 +CPU= 33.059 10:01:40 STEP 2932 H=2932:00 +CPU= 21.533 Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2010050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0026 (argv4) batchid: 979 (argv5) wuid: 12195670 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_75402b9259fe5d2bdf564213787f7884 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0026_2010050100_123_979_12195670.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0026_2010050100_123_979_12195670.zip ic_ancil_file: ic_ancil_12195670 ifsdata_file: ifsdata_12195670 climate_data_file: clim_data_12195670 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_a9f0cb7682b4e38df14999ab41d0e971 to: /var/lib/boinc-client/slots/0/ic_ancil_12195670.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12195670.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3748859929ec4965f83945320a644dfd to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12195670.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12195670.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12195670.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12195670.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12195670.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12195670.xml -- Model is restarting -- Adjusting last_iter, 2933, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12195670.xml last_cpu_time: 68349 upload_file_number: 122 last_iter: 2927 last_upload: 10540800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490 The child process has been launched with process id: 1446187 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1446187:1446187] [20230112:113827:1673491107:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:41:29 STEP 2928 H=2928:00 +CPU= 37.433 11:41:51 STEP 2929 H=2929:00 +CPU= 21.952 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:44:04 STEP 2930 H=2930:00 +CPU= 21.668 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:46:28 STEP 2931 H=2931:00 +CPU= 32.507 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:48:41 STEP 2932 H=2932:00 +CPU= 21.515 11:49:01 STEP 2933 H=2933:00 +CPU= 20.742 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:51:46 STEP 2934 H=2934:00 +CPU= 33.526 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:53:39 STEP 2935 H=2935:00 +CPU= 21.254 11:54:01 STEP 2936 H=2936:00 +CPU= 21.698 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:57:06 STEP 2937 H=2937:00 +CPU= 33.891 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:59:19 STEP 2938 H=2938:00 +CPU= 21.488 11:59:40 STEP 2939 H=2939:00 +CPU= 21.267 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:02:38 STEP 2940 H=2940:00 +CPU= 36.892 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002940 12:05:20 STEP 2941 H=2941:00 +CPU= 20.932 12:05:42 STEP 2942 H=2942:00 +CPU= 21.715 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:09:26 STEP 2943 H=2943:00 +CPU= 33.286 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:11:39 STEP 2944 H=2944:00 +CPU= 21.844 12:12:00 STEP 2945 H=2945:00 +CPU= 20.958 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:22:49 STEP 2946 H=2946:00 +CPU= 34.330 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:52:35 STEP 2947 H=2947:00 +CPU= 21.371 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:56:48 STEP 2948 H=2948:00 +CPU= 21.720 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:05:04 STEP 2949 H=2949:00 +CPU= 33.169 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2010050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0026 (argv4) batchid: 979 (argv5) wuid: 12195670 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_75402b9259fe5d2bdf564213787f7884 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0026_2010050100_123_979_12195670.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0026_2010050100_123_979_12195670.zip ic_ancil_file: ic_ancil_12195670 ifsdata_file: ifsdata_12195670 climate_data_file: clim_data_12195670 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_a9f0cb7682b4e38df14999ab41d0e971 to: /var/lib/boinc-client/slots/0/ic_ancil_12195670.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12195670.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3748859929ec4965f83945320a644dfd to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12195670.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12195670.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12195670.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12195670.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12195670.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12195670.xml -- Model is restarting -- Adjusting last_iter, 2949, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12195670.xml last_cpu_time: 68923 upload_file_number: 122 last_iter: 2927 last_upload: 10540800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490 The child process has been launched with process id: 1524551 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1524551:1524551] [20230112:150748:1673503668:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:11:02 STEP 2928 H=2928:00 +CPU= 37.479 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:13:45 STEP 2929 H=2929:00 +CPU= 21.302 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:16:58 STEP 2930 H=2930:00 +CPU= 21.474 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:20:03 STEP 2931 H=2931:00 +CPU= 33.115 15:20:24 STEP 2932 H=2932:00 +CPU= 20.922 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:23:06 STEP 2933 H=2933:00 +CPU= 21.108 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:28:12 STEP 2934 H=2934:00 +CPU= 33.486 15:28:33 STEP 2935 H=2935:00 +CPU= 20.868 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:31:06 STEP 2936 H=2936:00 +CPU= 21.213 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:34:10 STEP 2937 H=2937:00 +CPU= 32.236 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:37:13 STEP 2938 H=2938:00 +CPU= 21.960 15:37:34 STEP 2939 H=2939:00 +CPU= 20.713 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:43:12 STEP 2940 H=2940:00 +CPU= 36.766 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002940 15:43:32 STEP 2941 H=2941:00 +CPU= 20.242 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:46:45 STEP 2942 H=2942:00 +CPU= 21.604 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:49:39 STEP 2943 H=2943:00 +CPU= 32.856 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:52:32 STEP 2944 H=2944:00 +CPU= 21.826 15:52:53 STEP 2945 H=2945:00 +CPU= 20.551 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:57:18 STEP 2946 H=2946:00 +CPU= 33.463 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:00:30 STEP 2947 H=2947:00 +CPU= 20.534 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:35:58 STEP 2948 H=2948:00 +CPU= 21.409 16:36:31 STEP 2949 H=2949:00 +CPU= 33.127 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:44:24 STEP 2950 H=2950:00 +CPU= 21.753 16:44:45 STEP 2951 H=2951:00 +CPU= 20.966 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:56:38 STEP 2952 H=2952:00 +CPU= 38.146 The child process terminated with status: 0 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002952 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002952 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMGGhq0f+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMSHhq0f+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195670/ICMUAhq0f+002928 Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2010050100_123_979_12195670_0_r1362559490_122.zip Uploading the final file: upload_file_122.zip Uploading trickle at timestep: 10623600 17:00:02 (1524541): called boinc_finish(0) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:16:29 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 10,623,600 | 69,575 | 0.0065 |
09 May 2023 16:15:48 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 9,504,000 | 60,170 | 0.0063 |
09 May 2023 16:15:45 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 8,640,000 | 55,282 | 0.0064 |
09 May 2023 16:15:45 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 8,640,000 | 55,282 | 0.0064 |
09 May 2023 16:15:41 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 7,776,000 | 50,394 | 0.0065 |
09 May 2023 16:15:38 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 6,912,000 | 45,507 | 0.0066 |
09 May 2023 16:15:23 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 6,048,000 | 39,627 | 0.0066 |
09 May 2023 16:15:19 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 5,184,000 | 33,410 | 0.0064 |
09 May 2023 16:15:14 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 4,320,000 | 27,432 | 0.0064 |
09 May 2023 16:15:12 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 3,456,000 | 21,751 | 0.0063 |
09 May 2023 16:15:12 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 3,456,000 | 21,751 | 0.0063 |
09 May 2023 16:15:08 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 2,592,000 | 16,131 | 0.0062 |
09 May 2023 16:15:08 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 2,592,000 | 16,131 | 0.0062 |
09 May 2023 16:15:06 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 1,728,000 | 10,759 | 0.0062 |
09 May 2023 16:15:06 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 1,728,000 | 10,759 | 0.0062 |
09 May 2023 16:15:02 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 864,000 | 5,418 | 0.0063 |
09 May 2023 16:15:02 | 1519251 | 22281612 | oifs_43r3_ps_0026_2010050100_123_979_12195670_0 | 864,000 | 5,418 | 0.0063 |
©2024 climateprediction.net